Lens distortion correction hardware implementation system and method

Abstract

The invention discloses a lens distortion correction hardware implementation system and method, and belongs to the technical field of image signal processing. The system comprises a row cache module, a distribution module and a plurality of parallel interpolation processing sub-modules, including a coarse correction module, a fine correction module, a reinforcement correction module, a gap processing module and the like. Single-line caching is realized through a line caching module, and input images are distributed to different processing paths according to a raster scanning sequence and are corrected in combination with a mapping table and a mask value generated by software preprocessing. Frame-level image caching is not needed, processing delay is avoided, on-chip storage resources and external bandwidth overhead are remarkably reduced, efficient real-time lens distortion correction can be achieved under the large-resolution condition, and the method is suitable for image signal processor chip design of high-end imaging equipment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image signal processing, in particular to a hardware implementation system and method for lens distortion correction (LDC) in a large-resolution image sensor, and specifically to a real-time distortion correction architecture based on a line buffer, a preprocessing mapping table and a multi-stage parallel processing unit. BACKGROUND

[0002] With the development of smart phones, drones, high-definition monitoring cameras and industrial cameras, the resolution of image sensors has generally reached tens of millions of pixels or even hundreds of millions of pixels. High-resolution images often undergo geometric deformation during shooting due to the inherent characteristics of the lens, i.e., lens distortion, which mainly includes barrel distortion and pincushion distortion. In order to restore the true geometric relationship of the image, lens distortion correction (LDC) is required.

[0003] Traditional LDC algorithms usually use bilinear interpolation or higher-order interpolation methods to generate a mapping table according to a pre-calibrated distortion model, map the target image coordinates back to the original distorted image coordinates, and obtain the corrected pixel values through interpolation. However, in the large-resolution scenario, the traditional scheme faces the following challenges:

[0004] A real-time distortion correction scheme based on field programmable gate array (FPGA) proposed by Beijing University of Aeronautics and Astronautics (see document: Real-time correction of distortion image based on FPGA) improves the calculation speed by storing the image in odd and even rows and columns, but requires on-chip frame-level caching of the entire distorted image, resulting in a huge consumption of on-chip storage resources, which is difficult to implement at high resolution.

[0005] Another type of scheme (as described in Chinese patent application No. CN202411211266.0) uses DDR or SDRAM external memory for off-chip frame caching and combines a block processing method to read the image for correction. Although this type of scheme can alleviate the on-chip resource pressure, it has a serious bandwidth bottleneck, and the frequent reading and writing of distorted images leads to low efficiency and introduces a processing delay of one frame of image.

[0006] The above-mentioned existing technologies generally have the following defects: frame-level caching is required, causing a processing delay of at least one frame of image; on-chip caching schemes have large area overhead and high cost; off-chip caching schemes have high bandwidth demand, high power consumption and low system efficiency.

[0007] Therefore, there is an urgent need for a lens distortion correction hardware implementation scheme that can achieve real-time, efficient and low resource consumption under large resolution conditions. SUMMARY

[0008] The present application aims to provide a lens distortion correction hardware implementation system and method, which aims to solve the problems of large delay, high bandwidth and high resource consumption caused by frame-level caching in the prior art. By introducing an optimized hardware pipeline structure and software preprocessing mechanism, real-time distortion correction without frame-level delay in large resolution ISP chips is achieved, significantly reducing on-chip storage and external bandwidth overhead.

[0009] To achieve the above-mentioned purpose, the present application provides a lens distortion correction hardware implementation system, comprising:

[0010] a row buffer module configured to perform single-row buffering on distortion images input in raster scan order and output pixel blocks with a width of N and a height of 2;

[0011] a distribution module connected to the row buffer module, configured to combine adjacent pixel blocks into processing units with a height of N*2 and a height of 2, and distribute the processing units to a plurality of parallel interpolation processing sub-modules according to mask values generated by preprocessing;

[0012] The plurality of parallel interpolation processing sub-modules are respectively connected to the distribution module and comprise:

[0013] a coarse correction module configured to process the mapping condition that all reference pixels of a target correction pixel block fall within the same processing unit and directly output the corrected pixel block;

[0014] a fine correction module configured to process the condition that reference pixels of a target correction pixel block span multiple original pixel blocks, temporarily buffer part of the calculation results in an internal on-chip storage unit, complete the remaining calculation after subsequent related pixel blocks are input, output the complete correction pixel block, and release the on-chip storage space;

[0015] a reinforcement correction module configured to process complex mapping conditions that cannot be efficiently processed by the fine correction module, allowing a single input pixel block to participate in the calculation of multiple target pixel blocks;

[0016] a gap processing module configured to process all remaining target pixel blocks that are not allocated by the aforementioned modules in a single-pixel manner;

[0017] a boundary padding module configured to generate pixel values whose mapping coordinates exceed the boundaries of the original image and output a preset padding value;

[0018] In the entire distortion correction process, the system does not need to cache the entire frame of distortion images.

[0019] Furthermore, the precision calibration module is a multi-level design, with each level executing in parallel, used to reduce air bubbles and back pressure in the water flow process.

[0020] Furthermore, the on-chip storage unit address index inside the fine correction module is generated based on the low-order bits of the target correction pixel block coordinates.

[0021] Furthermore, the on-chip storage unit index information used by the reinforcement and correction module is embedded in the mapping table by the preprocessing and is input along with the interpolation coefficients.

[0022] Furthermore, the preprocessing includes: classifying the target correction pixel blocks into the coarse correction module, fine correction module, gap processing module, or boundary filling module according to their mapping characteristics, and sorting the mapping table entries according to the scanning order of the input image.

[0023] Furthermore, the distribution module allocates processing units with a width of N×2 and a height of 2 to the corresponding parallel processing sub-modules based on the mask value.

[0024] Furthermore, it also includes an arbitration and sorting module, which is connected to the output end of the multiple parallel interpolation processing submodules, and is used to reorganize and sort the correction results output by the multiple parallel interpolation processing submodules according to the position information of the target pixel, and write them into the off-chip storage unit.

[0025] Furthermore, the coarse correction module performs interpolation operations when each input original pixel block with a width of N×2 and a height of 2 is used only for the calculation of a target pixel block.

[0026] Furthermore, the system receives the input image through the AXI4-Stream interface and reads the preprocessed distortion mapping table from the off-chip storage unit through the AXI Master interface.

[0027] This invention also proposes a lens distortion correction method, comprising:

[0028] Receive N-pixel parallel distorted images input in raster scan order;

[0029] The distorted image is cached in a single row, and a pixel block with a width of N and a height of 2 is generated based on the current row and the previous row;

[0030] Two adjacent pixel blocks are joined horizontally to form a processing window with a width of 2N and a height of 2.

[0031] Based on the pre-generated mask value, the processing window is assigned to the corresponding interpolation processing path, wherein the mask value is obtained by preprocessing the distortion mapping table and is used to indicate the distribution characteristics of the reference pixels of the target correction pixel block in the original image.

[0032] Interpolation calculations are performed on the pixels in each processing path according to the distortion mapping table, specifically including:

[0033] If all reference pixels of the target correction pixel block are located within the same processing window, interpolation is performed directly based on that window.

[0034] If the reference pixels of the target correction pixel block span multiple processing windows, interpolation is performed in stages, with some intermediate results cached first, and the remaining calculations completed after the relevant pixel data is complete.

[0035] If the mapping relationship of the target correction pixel blocks is complex and cannot be efficiently processed by the aforementioned methods, a flexible interpolation strategy that supports multi-target output is adopted.

[0036] For the remaining target pixels not covered by any of the above methods, interpolation is performed independently on a per-pixel basis;

[0037] At the same time, preset fill values ​​are generated for pixels whose mapped coordinates exceed the boundaries of the original image;

[0038] Each interpolation calculation result is calculated based on its corresponding target pixel position and then written to external memory.

[0039] Compared with the prior art, the present invention has the following significant advantages:

[0040] 1. No frame-level latency: It only uses line-by-line caching and does not require storing the entire frame, realizing true pipelined real-time processing and avoiding the processing latency of at least one frame in traditional solutions.

[0041] 2. Significantly reduces storage resources and area overhead: By eliminating large-capacity on-chip storage units or frame buffers, chip area and manufacturing costs are significantly reduced.

[0042] 3. Saves external bandwidth: There is no need to write the entire frame of distorted image to external memory and then read it out. Only the pre-processed mapping table needs to be read, which greatly reduces the bandwidth requirements for accessing external memory.

[0043] 4. High processing efficiency: The mapping table is classified through preprocessing, and combined with multi-level parallel processing mechanisms such as coarse correction module, fine correction module, reinforcement correction module, gap processing module, and boundary filling module, the efficient processing path (coarse correction module) is maximized to improve the overall throughput.

[0044] 5. Suitable for high-resolution and high-distortion scenarios: This architecture is highly scalable and can effectively support 4K, 8K and even higher resolution image processing, meeting the real-time requirements of high-end imaging equipment.

[0045] The following text will further explain the lens distortion correction (LDC) system, its characteristics, technical features, advantages, and implementation methods in a clear and easy-to-understand manner, with the aid of accompanying figures. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of the top-level interface of the lens distortion correction hardware implementation system of the present invention;

[0047] Figure 2 This is a structural block diagram of the lens distortion correction hardware implementation system of the present invention;

[0048] Figure 3 This is a schematic diagram illustrating the working principle of the lens distortion correction hardware implementation system of the present invention;

[0049] Figure 4 This is a schematic diagram illustrating the calculation process of the coarse correction module of the present invention when processing the target pixel block;

[0050] Figure 5 This is a schematic diagram illustrating the calculation process of the fine correction module of the present invention when handling target pixels across blocks;

[0051] Figure 6 This is a schematic diagram illustrating the complex mapping situation processed by the reinforcement and correction module of the present invention;

[0052] Figure 7 This is a schematic diagram of the internal implementation and RAM cache structure of the fine correction module of the present invention;

[0053] Figure 8 This is a timing diagram of the sliding window processing method used in this invention. Detailed Implementation

[0054] To provide a clearer understanding of the technical features, objectives, and effects of the invention, specific embodiments of the invention are now described with reference to the accompanying drawings. In the drawings, the same reference numerals indicate components with the same or similar structures but the same function.

[0055] In this document, "illustrative" means "serving as an example, illustration, or description," and any illustrations or embodiments described herein as "illustrative" should not be construed as a more preferred or advantageous technical solution. For the sake of brevity, each figure only schematically shows the parts relevant to this exemplary embodiment, and they do not represent the actual structure or true proportions of the product.

[0056] Example 1: System Implementation

[0057] like Figure 1As shown, the Lens Distortion Correction (LDC) system module of this invention, as part of the Image Signal Processor (ISP) pipeline, receives raw image data from the image sensor or a pre-processor ISP module (such as a de-mosaic module). The input interface uses the AXI4-Stream protocol, supporting parallel transmission of N = 2, 4, 8, 16, or 32 pixels per cycle (using N = 8 as an example). The pixel format is 8-16 bit RGB or YUV, input line by line in raster scan order.

[0058] The LDC system module connects to external memory (e.g., DDR, SDRAM) via the AXIMaster interface to read the pre-processed distortion mapping table (containing the original floating-point coordinate positions and interpolation weights corresponding to the target pixel coordinates); and writes the corrected target image back to external memory. The entire process is completed without frame-level buffering, relying only on a small internal RAM and line buffer to achieve real-time pipelined processing.

[0059] like Figure 2 As shown, the lens distortion correction hardware system of the present invention includes the following interconnected modules, including:

[0060] The line buffer module 201 is configured to buffer the distorted image input in raster scan order in a single line and output a pixel block with a width of N and a height of 2.

[0061] Distribution module 202 (fork): Connects to the 101-line cache module, used to merge adjacent pixel blocks into a processing window with a width of 2N and a height of 2, and distribute the processing window to multiple parallel interpolation processing sub-modules according to the mask value generated by software preprocessing;

[0062] Coarse correction module 203: This module handles the mapping situation where all reference pixels of the target correction pixel block fall within the same processing window, and directly outputs the corrected pixel block.

[0063] The fine correction module 204 is used to handle cases where the reference pixels of the target correction pixel block span multiple original pixel blocks. It temporarily caches part of the calculation results through the internal on-chip storage unit, completes the remaining calculation after the input of subsequent related pixel blocks, outputs the complete correction pixel block, and releases the on-chip storage space.

[0064] The polishing module 205 is used to handle complex mapping situations that the polishing module cannot handle efficiently, allowing a single input pixel block to participate in the calculation of multiple target pixel blocks;

[0065] Gaps processing module 206: Used to process all remaining target pixel blocks not assigned by the aforementioned modules in a single-pixel manner;

[0066] Border filling module 207 (border): Used to generate pixel values ​​whose mapped coordinates exceed the original image boundary and output preset filling values ​​(such as black). This module does not receive image data from the line buffer or distribution module. Instead, when it detects that the mapping table entry contains an out-of-bounds flag, it directly generates the preset filling value and writes it to the external memory according to the corresponding target pixel coordinates.

[0067] Arbitration and reordering module 208: Connects to the output of the multiple parallel interpolation processing submodules and is used to coordinate write requests to external memory; however, each module can carry the target address and write directly to external memory, so this module is optional.

[0068] In this embodiment, the refine module has on-chip RAM whose address index is generated based on the low-order bits of the target pixel block coordinates, used to temporarily store some calculation results. The RAM index information used by the polish module is directly embedded into the mapping table by the preprocessing software and input along with the interpolation coefficients. The border module outputs black fill pixel values.

[0069] like Figure 3 As shown, the internal sub-module structure of the system further refines the data flow and control logic, demonstrating the connection relationships between the linebuffer module, the fork module, and each processing unit, as well as the paths for reading the mapping table and writing back the results. The output of each processing module is a pixel block or single pixel (gaps) with W=N, H=1, accompanied by target location information. It is directly written to external memory via address-based write requests, without the need for centralized reassembly.

[0070] Referring to the accompanying drawings, the lens distortion correction workflow in this embodiment is further explained as follows:

[0071] The line buffer module buffers the pixel blocks with W=N, H=1 input from the AXI4-Stream in parallel, and outputs them as pixel blocks with W=N, H=2 together with the pixels above them.

[0072] The fork module takes a pixel block with W=N and H=2 as input, and forms a pixel block with W=N*2 and H=2 with the pixel block to its left; it outputs the pixel block to each sub-module based on the mask value calculated by the preprocessing software.

[0073] The timing diagram of the sliding window processing method used in the above work is shown below. Figure 8 As shown.

[0074] The coarse module: When all the reference pixels of a certain correction image W=N,H=1 fall into the same distorted pixel block W=N*2,H=2, the coarse module processes them, and each distorted pixel block is assigned at most one correction image; most correction image blocks are obtained from this module.

[0075] The refine module is used for target pixel blocks that coarse cannot handle, i.e., when the reference pixels of the corrected image block W=N,H=1 cannot all fall within the same distorted pixel block W=N*2,H=2. It takes the original pixel block as input, interpolates some pixels in the corrected image block, and temporarily stores them in the internal RAM according to the rules. When the complete corrected image block W=N,H=1 is assembled, it is output. To ensure that the flow is bubble-free and there are RAM access rules and depth limitations, the first-level refine module cannot effectively fill in all pixels, so multiple levels are arranged.

[0076] Polish module: When the distortion is large, due to the various limitations of the refine module, no matter how many levels of refine module there are, it cannot guarantee that all the corrected image blocks will be allocated. In some cases, the efficiency of the refine module is too low. The polish module is specifically designed to handle this situation. The polish module is similar to a refine module without bubble limitations and RAM rules.

[0077] Gaps module: processes all remaining unassigned correction pixel blocks, single-pixel processing, low efficiency, and should normally handle as little computation as possible;

[0078] Border module: Used to generate pixel block values ​​mapped outside the image; no distorted pixel input is required, and the pixel result is a fixed value.

[0079] Taking 4-pixel parallel processing as an example, the coarse module processes... Figure 4 As shown in the figure, Figure 4 In the image, the black block represents the original distorted image block with W=8 and H=2 output from the pre-processing module, while the red block represents the target correction pixels. Since bilinear interpolation is used in the correction process, the target pixel block with W=4 and H=1 can be directly calculated from the original pixel block. The preprocessing software iterates through the mapping table and assigns this case to the coarse module for processing. Each input original pixel block is used only for the calculation of one target pixel block.

[0080] refine module processing Figure 5 As shown, the reference pixels for the target pixel block with W=4 and H=1 span multiple original pixel blocks. Preprocessing will assign this situation to the calculations of each refine module. Assuming... Figure 5The two original pixel blocks are at coordinates (X,Y) and (X,Y+1). When the (X,Y) block is piped in, the refine module calculates two target pixel points and temporarily stores them in the internal on-chip RAM. When the (X,Y+1) block is piped in, the remaining two points are calculated and combined into a pixel block with W=4 and H=1 for output, while the on-chip RAM is released. The RAM is indexed according to the low-order bits of the target pixel block position coordinates.

[0081] like Figure 6 As shown, the calculation rules of the polish module are consistent with those of the refine module, but without the limitation of pipeline backpressure. Multiple target pixel blocks can be calculated for each input raw pixel block. Simultaneously, a more flexible RAM cache indexing rule is used; the preprocessing software directly stores the RAM index in the preprocessing result, inputting it along with the interpolation coefficients. The purpose of the polish module is to reduce the remaining unassigned target correction pixel blocks as efficiently as possible. In most cases, only a small amount of calculation is performed; placing buffers before and after the polish module can reduce the impact of bubble backpressure.

[0082] The internal implementation of the refine module and the RAM cache structure are shown in the diagram below. Figure 7 As shown, to minimize pipeline back pressure, the refine module is designed as a bubble-free pipeline, meaning that each input raw pixel block is used only for the calculation of one target pixel block. Meanwhile, the on-chip RAM used to store incomplete pixel blocks may experience address conflicts or be full, so a multi-level refine submodule design is used.

[0083] The preprocessing software will allocate all remaining unallocated target pixel blocks to the gaps module to ensure the integrity of the target image. The gaps module performs single-pixel calculations internally, without on-chip buffering, and outputs directly after the calculation is completed, which is inefficient. In most cases, it will only allocate a very small amount of calculation.

[0084] Each module outputs a pixel block or single pixel (gaps) with W=N, H=1, along with position information, which is then written back to DDR after arbitration and sorting.

[0085] The distortion correction strategy involved in this invention comprises three core steps:

[0086] 1. The software completes camera calibration and generates a distortion mapping table;

[0087] 2. Using a pre-generated distortion mapping table, combined with the number of available hardware processing units and other parameters, software preprocessing and optimization are performed to prepare for efficient parallel computing.

[0088] 3. The distortion correction hardware module pipelines input distorted pixels and allocates the distorted pixel blocks to various sub-processing units to complete the correction based on the preprocessing results.

[0089] The first step above is the software preprocessing stage, which serves as a bridge connecting the camera calibration model and the hardware pipeline architecture. Its core objective is to transform the raw distortion map into a classification instruction stream that can be efficiently executed by the hardware, thereby maximizing the use of coarse correction paths and minimizing the overhead of complex paths. This preprocessing is completed offline before system runtime and includes the following steps:

[0090] Step P1: Generate the original distortion mapping table. Based on the camera calibration results, for each target pixel position in the corrected image, calculate its floating-point mapping coordinates in the original distorted image, thereby generating the original mapping table. The table entries are arranged according to the raster scan order of the target image.

[0091] Step P2: Determine the interpolation reference region. For each target pixel, based on its mapped coordinates, determine the coordinates of the four nearest integer pixels required for bilinear interpolation, and calculate the corresponding interpolation weights.

[0092] Step P3: Group by target pixel blocks. Divide the target image into continuous W=N, H=1 pixel blocks (consistent with the hardware output granularity). Collect the original coordinate set of all reference pixels for each target block.

[0093] Step P4: Classification Decision (Core Step). For each target block, based on the spatial distribution characteristics of its reference pixel set in the original image, the following classification logic is executed: If all reference pixels fall within a certain W=2N, H=2 original window, the block is marked as "coarse correction type" and assigned to coarse correction module 103; if the reference pixels span two horizontally adjacent W=2N, H=2 windows, but can be covered by two consecutive input windows, the block is marked as "fine correction type" and assigned to fine correction module 104; if the reference pixel distribution is complex (e.g., spanning non-adjacent windows, vertically spanning rows, or a single original block needs to serve multiple target blocks), and cannot be efficiently processed by the fine correction module (e.g., causing RAM address conflicts or pipeline backpressure), the block is marked as "reinforcement correction type" and assigned to reinforcement correction module 105, and a unique RAM cache index ID is generated by the software and embedded in the mapping table; if none of the above classifications apply (e.g., isolated pixels, boundary anomalies), the block is marked as "gap type" and assigned to gap processing module 106 for single-pixel processing.

[0094] Step P5: Generate mask and optimize mapping table. For each original input window (corresponding to hardware processing cycle t), summarize all target block types that need to be calculated by that window and generate an N-bit mask value mask[t]. Each bit indicates whether the window needs to be sent to the corresponding processing module (e.g., bit0=1 indicates sending to 103, bit1=1 indicates sending to 104, and so on). At the same time, rearrange the original mapping table into a data structure organized according to the original image raster scan order (i.e., hardware input order). Each item contains: target pixel coordinates, interpolation weight, classification label (coarse / refine / polish / gaps / border), if it is a polish type, add a RAM index ID, and if it is a border type, mark an out-of-bounds flag.

[0095] Step P6: Output preprocessing results. The final output consists of two files for hardware use: an optimized mapping table stored in external memory and read as needed via the AXI Master interface; and a mask sequence synchronized with the input image rows, driving the distribution module 102 for path allocation.

[0096] Through the above preprocessing, the hardware does not need to perform any mapping analysis or path decision-making during runtime; it only needs to "execute" according to the mask. More than 90% of the target blocks can typically be processed by the coarse correction module 103, significantly improving throughput; complex cases are handled by a dedicated module to ensure 100% coverage. This invention uses preprocessing software to classify the distortion mapping table by target pixel blocks according to their distortion status and sort them according to the scanning order of the input distorted image. Based on the preprocessing results, the hardware module allocates the pipelined input distorted pixel sliding window blocks to different hardware sub-modules for parallel processing and correction. Because the correction is performed according to the input order, frame-level buffer storage and bandwidth costs can be saved at high resolutions.

[0097] Example 2: Method Implementation

[0098] The present invention also provides a lens distortion correction method, comprising the following steps:

[0099] Step S1: Receive N-pixel parallel distorted images input in raster scan order;

[0100] Step S2: Cache the distorted image in a single row, and generate a pixel block with a width of N and a height of 2 based on the current row and the previous row;

[0101] Step S3: Join two adjacent pixel blocks horizontally to form a processing window with a width of 2N and a height of 2;

[0102] Step S4: Based on the pre-generated mask value, the processing window is assigned to the corresponding interpolation processing path, wherein the mask value is obtained by preprocessing the distortion mapping table and is used to indicate the distribution characteristics of the reference pixels of the target correction pixel block in the original image.

[0103] Step S5: Perform interpolation calculations on the pixels in each processing path according to the distortion mapping table, specifically including:

[0104] If all reference pixels of the target correction pixel block are located within the same processing window, interpolation is performed directly based on that window.

[0105] If the reference pixels of the target correction pixel block span multiple processing windows, interpolation is performed in stages. Some intermediate results are temporarily stored in on-chip storage units first, and the remaining calculations are completed after the relevant pixel data is complete.

[0106] If the mapping relationship of the target correction pixel block is complex and cannot be efficiently processed by the aforementioned method, a flexible interpolation strategy that supports single input and multiple outputs is adopted.

[0107] For the remaining target pixels not covered by any of the above methods, interpolation is performed independently on a per-pixel basis;

[0108] Step S6: Generate preset fill values ​​for pixels whose mapped coordinates exceed the boundaries of the original image;

[0109] Step S7: Write each interpolation calculation result directly into the external memory according to its corresponding target pixel position.

[0110] This method is suitable for real-time correction of 4K, 8K or higher resolution images, with a processing delay of no more than a few rows of pixels and no need for whole-frame image buffering.

[0111] It should be understood that although this specification describes various embodiments, not every embodiment contains only one independent technical solution. This way of describing the specification is only for clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

[0112] The detailed descriptions listed above are merely specific descriptions of feasible embodiments of the present invention, and are not intended to limit the scope of protection of the present invention. All equivalent embodiments or modifications made without departing from the spirit of the present invention should be included within the scope of protection of the present invention.

Claims

1. A lens distortion correction hardware system, characterized in that, include: The row buffer module is configured to buffer the distorted image input in raster scan order in a single row and output a pixel block with a width of N and a height of 2. The distribution module, connected to the row cache module, is used to merge adjacent pixel blocks into processing units with a height of N×2 and a height of 2, and to distribute the processing units to multiple parallel interpolation processing sub-modules according to the mask value generated by preprocessing. The plurality of parallel interpolation processing submodules are respectively connected to the distribution module, including: The coarse correction module is used to handle the mapping situation where all reference pixels of the target correction pixel block fall within the same processing unit, and directly outputs the corrected pixel block; The fine correction module is used to handle cases where the reference pixels of the target correction pixel block span multiple original pixel blocks. It temporarily caches part of the calculation results through the internal on-chip storage unit, completes the remaining calculation after the input of subsequent related pixel blocks, outputs the complete correction pixel block, and releases the on-chip storage space. The reinforcement and correction module is used to handle complex mapping situations that the fine correction module cannot handle efficiently, allowing a single input pixel block to participate in the calculation of multiple target pixel blocks; The gap processing module is used to process all remaining target pixel blocks that have not been allocated by the aforementioned modules in a single-pixel manner; The boundary fill module is used to generate pixel values ​​whose mapped coordinates exceed the boundaries of the original image and output preset fill values; The system does not require caching of the entire distorted image frame during the entire distortion correction process.

2. The system according to claim 1, characterized in that, The fine calibration module is a multi-level design, with each level executing in parallel, used to reduce air bubbles and back pressure in the water treatment process.

3. The system according to claim 1 or 2, characterized in that, The on-chip storage unit address index inside the fine correction module is generated based on the low-order bits of the target correction pixel block coordinates.

4. The system according to claim 1, characterized in that, The on-chip storage unit index information used by the reinforcement and correction module is embedded in the mapping table by the preprocessing and is input along with the interpolation coefficients.

5. The system according to claim 1, characterized in that, The preprocessing includes: classifying the target correction pixel blocks into the coarse correction module, fine correction module, gap processing module, or boundary filling module according to their mapping characteristics, and sorting the mapping table entries according to the scanning order of the input image.

6. The system according to claim 1, characterized in that, The distribution module allocates processing units with a width of N×2 and a height of 2 to the corresponding parallel processing sub-modules based on the mask value.

7. The system according to claim 1, characterized in that, It also includes an arbitration and sorting module, which is connected to the output of the multiple parallel interpolation processing submodules. It is used to reorganize and sort the correction results output by the multiple parallel interpolation processing submodules according to the position information of the target pixel, and write them into the off-chip storage unit. The fill pixel value output by the arbitration and sorting module is black, mirror value, or edge copy value.

8. The system according to claim 1, characterized in that, The coarse correction module performs interpolation operations when each input original pixel block with a width of N×2 and a height of 2 is used only for the calculation of a target pixel block.

9. The system according to claim 1, characterized in that, The system receives the input image through the AXI4-Stream interface and reads the preprocessed distortion mapping table from the off-chip storage unit through the AXI Master interface.

10. A lens distortion correction method, characterized in that, include: Receive N-pixel parallel distorted images input in raster scan order; The distorted image is cached in a single row, and a pixel block with a width of N and a height of 2 is generated based on the current row and the previous row; Two adjacent pixel blocks are joined horizontally to form a processing window with a width of 2N and a height of 2. Based on the pre-generated mask value, the processing window is assigned to the corresponding interpolation processing path, wherein the mask value is obtained by preprocessing the distortion mapping table and is used to indicate the distribution characteristics of the reference pixels of the target correction pixel block in the original image. Interpolation calculations are performed on the pixels in each processing path according to the distortion mapping table, specifically including: If all reference pixels of the target correction pixel block are located within the same processing window, interpolation is performed directly based on that window. If the reference pixels of the target correction pixel block span multiple processing windows, interpolation is performed in stages, with some intermediate results cached first, and the remaining calculations completed after the relevant pixel data is complete. If the mapping relationship of the target correction pixel block is complex and cannot be efficiently processed by the aforementioned method, a flexible interpolation strategy that supports multi-target output is adopted. For the remaining target pixels not covered by any of the above methods, interpolation is performed independently on a per-pixel basis; At the same time, preset fill values ​​are generated for pixels whose mapped coordinates exceed the boundaries of the original image; Each interpolation calculation result is calculated based on its corresponding target pixel position and then written to external memory.

Images