An FPGA-based optical display image interactive closed-loop calibration and real-time geometric correction method and system

Abstract

The application provides an FPGA-based optical display image interactive closed-loop calibration and real-time geometric correction method and system. The method comprises the following steps: acquiring calibration interactive input data containing target display coordinates and corresponding source image sampling coordinates; constructing a double-quadratic reverse mapping model on the processing system side, solving geometric correction coefficients online and performing fixed-point quantization processing; the quantized coefficients are written into shadow registers on the programmable logic side, and are switched to working registers to take effect when a display frame synchronization boundary is detected; on the PL side, the absolute sampling coordinates of the source image are obtained by performing coordinate reverse mapping on each target display pixel according to the effective coefficients, and the neighborhood pixels are read to perform bilinear interpolation output. The application highly integrates multi-entry interactive parameter adjustment, online matrix solution and hardware-level real-time pixel resampling in a single-chip closed-loop architecture, eliminates picture tearing and flickering in the parameter adjustment process while maintaining extremely low video delay, and significantly improves the efficiency of on-site calibration.

Description

Technical Field

[0001] This invention relates to the technical fields of optical display, image geometric correction, and real-time video processing, and specifically to an FPGA-based interactive closed-loop calibration and real-time geometric correction method and system for optical display images. Background Technology

[0002] Currently, most optical display systems are typically affected by glass curvature, projection angle, lens assembly, installation deviation, temperature drift, and mechanical assembly errors during actual display. This ultimately manifests as stretching, offset, geometric distortion, or local nonlinear distortion of the displayed content on the viewing plane, causing undesirable display deviations and reducing image accuracy. To address this issue, simply using fixed affine or homography transformations often fails to balance edge consistency and local accuracy across the entire image.

[0003] In existing systems, this contradiction mainly relies on manual, point-by-point fine-tuning, but this often involves numerous steps, slow interactive feedback, and difficulty in synchronizing with the real-time display. For interactive display projects that require rapid debugging, frequent resets, or on-site verification, the lack of a unified and comprehensive interactive entry point and a stable parameter update mechanism will affect calibration efficiency and consistency.

[0004] On the other hand, while traditional CPU software calibration schemes offer flexibility, they are susceptible to host load fluctuations in terms of high resolution, low latency, and frame-level synchronization, making them difficult to embed stably into display terminals over the long term. Therefore, there is an urgent need for a technical solution that unifies interactive calibration, parameter solving, and real-time correction to meet the requirements of high-efficiency distortion correction and high-precision optical display.

[0005] In existing technologies, geometric correction of optical display images is usually performed by mapping coordinates, resampling pixels, or reconstructing grids on the original image before the video image enters the display end, so that the image after projection, reflection, refraction, or imaging through a transparent medium is as close as possible to the target display form.

[0006] The first type of approach is a geometric transformation method based on a fixed matrix. This type of approach typically uses affine transformation, perspective transformation, or homography matrix to establish a mapping relationship between the input image coordinates and the target display coordinates based on a small number of calibration points, and then performs a uniform geometric transformation on the entire image.

[0007] The second type of approach is a pixel remapping method based on lookup tables or block grids. This type of approach typically divides the display area into several pixel grids or control grids, pre-configures the source image coordinates, offsets, or mapping parameters for the grid nodes, and generates the reconstructed video pixel grid based on the interpolation relationship between the grid nodes during the display process, thereby achieving distortion correction.

[0008] The third type of approach is an offline calibration method based on host computer image processing software. This type of approach typically involves acquiring calibration images on a PC or manually inputting calibration data, and then performing distortion model solving, parameter fitting, lookup table generation, or mesh file generation in the host computer software. Finally, the calculated matrix, coefficients, lookup table, or pixel mesh parameters are sent to the display device.

[0009] The fourth type of approach is the embedded image algorithm correction method. This type of approach typically runs distortion correction algorithms within the embedded processor, image processing unit, or programmable logic inside the display terminal, performing coordinate mapping and pixel resampling on the input video stream. It can be implemented by calculating the corrected image frame by frame, or by pre-writing the reconstructed video pixel grid, mapping table, or grid node parameters, and then having the embedded image processing link rearrange, interpolate, and output the video pixels according to these parameters.

[0010] While the aforementioned existing technologies can achieve geometric correction of optical display images to a certain extent, they still have the following shortcomings in applications such as HUDs, transparent displays, and projection overlay displays, which have high requirements for on-site calibration, real-time response, and display stability: First, geometric transformation methods based on fixed matrices are mainly applicable to regular deformation scenarios such as overall translation, scaling, rotation, trapezoidal distortion, or perspective distortion. When glass curvature, assembly deviation of combined mirrors, projection angle error, and local nonlinear deformation exist simultaneously in the optical path, a single affine matrix or homography matrix is ​​difficult to accurately describe the local distortion of the entire image, and residual errors are easily generated in edge regions or local areas.

[0011] Second, while pixel remapping methods based on lookup tables or block grids can describe more complex local deformations, they typically rely on pre-generated lookup tables, mapping tables, or pixel grid files. If the optical structure, installation location, viewing angle, or display carrier changes, the relevant parameters need to be regenerated or rewritten, resulting in a lengthy on-site adjustment process that makes it difficult to quickly complete closed-loop verification.

[0012] Third, offline calibration methods based on host computer image processing software typically separate calibration, solution, and display into different devices or different software processes. Parameters are calculated on the PC and then sent to the display device, easily forming a long "acquisition-calculation-transmission-display-verification" chain. During on-site debugging, it is necessary to repeatedly switch tools and steps, and the real-time feedback capability is insufficient.

[0013] Fourth, while embedded image algorithm correction methods can perform some image processing within the display terminal, they are essentially still unidirectional video algorithm links. These solutions focus more on pixel rearrangement and image resampling themselves, typically lacking a real-time interaction mechanism between distortion resolution and display feedback. Furthermore, in applications requiring manual on-site fine-tuning or multiple rounds of calibration, each adjustment necessitates reconstructing the pixel grid, rewriting the mapping table, or refreshing the entire frame parameters, easily leading to response latency, screen flickering, frame asynchrony, or discontinuous debugging processes, making it difficult to simultaneously meet the requirements of low-latency interaction and stable display output.

[0014] Fifth, existing technologies typically lack a unified interactive interface for on-site commissioning personnel and have poor scenario adaptability. Calibration point acquisition, parameter solving, parameter writing, calibration preview, and error verification are often scattered across different tools or modules and are only applicable to specific situations. This leads to the commissioning process relying on experience and makes it difficult to form a visualized, real-time, and closed-loop calibration process. Summary of the Invention

[0015] To address the aforementioned technical problems in the prior art, this application provides an FPGA-based interactive closed-loop calibration and real-time geometric correction method and system for optical display images.

[0016] According to a first aspect of the present invention, an interactive closed-loop calibration and real-time geometric correction method for optical display images based on FPGA is proposed, comprising: S1: Obtain calibration interactive input data, which includes multiple sets of calibration points composed of the target display coordinates and the corresponding source image sampling coordinates; S2: On the processing system side, based on multiple sets of calibration points, a biquadratic inverse mapping model is constructed, the biquadratic geometric correction coefficients are obtained, and the geometric correction coefficients are subjected to fixed-point quantization. S3: Write the quantized geometric correction coefficients into the shadow register located on the programmable logic (PL) side, and when the display frame synchronization boundary is detected, switch the geometric correction coefficients in the shadow register to the working register so that the geometric correction coefficients take effect from the next frame. S4: On the programmable logic side, based on the geometric correction coefficients in the working register, the target display pixel is reverse mapped to obtain the corresponding absolute sampling coordinates of the source image. Then, based on the absolute sampling coordinates of the source image, the source image pixels are read and interpolated to output the geometrically corrected display image.

[0017] In some specific embodiments, the step of constructing a biquadratic inverse mapping model and solving for the biquadratic geometric correction coefficients in S2 specifically includes: converting multiple sets of calibration points into centered coordinates with the display center as the origin, the centered coordinates of the target display point are denoted as (u,v), and the centered coordinates of the source image sampling points are denoted as (p,q); constructing the inverse mapping relationship from the target display coordinates to the source image sampling coordinates: p=A0+A1·u+A2·v+A3·u²+A4·v²+A5·u·v; q=B0+B1·u+B2·v+B3·u²+B4·v²+B5·u·v; where A0-A5 and B0-B5 are the 12 geometric correction coefficients to be solved.

[0018] In some specific embodiments, the geometric correction coefficients A0-A5 and B0-B5 are obtained by solving multiple sets of calibration points using the least squares method, specifically including: Collect N sets of calibration points, where N≥6; For the i-th calibration point, based on the centered coordinates (ui,vi) of its target display point, construct the corresponding basis function row vector Hi=[1,ui,vi,ui²,vi²,ui·vi], and combine the basis function row vectors of all N calibration points to form matrix H; The centered coordinates (pi, qi) of the source image sampling points corresponding to the N sets of calibration points are combined to form the horizontal centered coordinate vector P = [p1, p2, ..., pN] of the source image. T And the vertically centered coordinate vector Q=[q1,q2,...,qN] T ; Define the horizontal correction coefficient vector as A = [A0, A1, A2, A3, A4, A5]. T The vertical correction coefficient vector is B = [B0, B1, B2, B3, B4, B5]. T ; Solve using any one of the following methods: least squares, generalized inverse, QR decomposition, SVD, or equivalent numerical method, and obtain the solution using the formula A=(H). T ·H) -1 ·H T ·P and B=(H T ·H) -1 ·H T The geometric correction coefficients A0-A5 and B0-B5 are obtained by solving Q.

[0019] In some specific embodiments, the step of performing fixed-point quantization on the geometric correction coefficients in S2 specifically includes: converting the obtained floating-point geometric correction coefficients into a signed fixed-point number format with a preset bit width; the signed fixed-point number format is Q16.16 format, where the high 16 bits are used to represent the signed integer part and the low 16 bits are used to represent the fractional part.

[0020] In some specific embodiments, the steps of performing coordinate inverse mapping and interpolation in S4 specifically include: if the absolute sampling coordinates of the source image obtained by inverse mapping exceed the effective range of the source image, a boundary clamping method is used to limit the range of sampling coordinates to avoid interpolation addressing out of bounds; extract the integer part and the fractional part of the absolute sampling coordinates of the source image, convert the fractional part into a weight within a preset range, perform multiplication and addition operations based on the weight and four pixels in the neighborhood of the integer part, and then shift and normalize to obtain the output pixel.

[0021] In some specific embodiments, the output pixels are calculated as follows: Convert the horizontal fractional part α and the vertical fractional part β of the absolute sampling coordinates of the source image into 8-bit unsigned fractional weighted integers dx and dy, dx=round(α×255), dy=round(β×255); The four bilinear interpolation weights w00, w01, w10 and w11 corresponding to the target display pixel are calculated using dx and dy respectively. The calculation formulas are: w00=(255-dx)(255-dy), w01=dx(255-dy), w10=(255-dx)dy, w11=dx·dy; Extract four neighboring pixels P00, P01, P10, and P11 from the source image relative to the integer part (x0, y0), where P00 = I(x0, y0), P01 = I(x0+1, y0), P10 = I(x0, y0+1), and P11 = I(x0+1, y0+1), and I represents the color channel pixel value of the source image at the corresponding coordinates; The RGB color channels of the four source image pixels P00, P01, P10, and P11 are multiplied and added respectively, and then normalized according to the total weight scale to obtain the output pixel value Pout. The calculation formula is: Pout=(w00·P00+w01·P01+w10·P10+w11·P11) / 255².

[0022] In some specific embodiments, the calibration interactive input data is obtained through a multimodal unified solution architecture. The multimodal unified solution architecture is configured to extract drag data from the web page, gesture data from the touch screen, or visual detection results from an external camera into calibration point coordinates in a unified format, and send them into the same parameter solution link for matrix operations.

[0023] According to a second aspect of the present invention, an interactive closed-loop calibration and real-time geometric correction system for optical display images based on FPGA is proposed, comprising: The calibration interactive input unit is configured to acquire calibration interactive input data, which includes multiple sets of calibration points composed of the target display coordinates and the corresponding source image sampling coordinates. The system-side parameter solving and control unit is configured to construct a bi-quadratic inverse mapping model based on multiple sets of calibration points, solve for the bi-quadratic geometric correction coefficients, and perform fixed-point quantization on the geometric correction coefficients; The parameter register and synchronization update unit are configured with a shadow register and a working register. The shadow register is used to write the quantized geometric correction coefficients into the shadow register, and when the display frame synchronization boundary is detected, the geometric correction coefficients in the shadow register are switched to the working register. The programmable logic side real-time image geometric correction unit is configured to perform inverse coordinate mapping on the target display pixels according to the geometric correction coefficients in the working register, obtain the corresponding absolute sampling coordinates of the source image, read the source image pixels based on the absolute sampling coordinates of the source image, perform interpolation processing, and output the geometrically corrected display image.

[0024] In some specific embodiments, the parameter solving and control unit on the processing system side is connected to the parameter register and the synchronization update unit via the AXI-Lite on-chip bus; while the parameter register and the synchronization update unit are writing geometric correction coefficients into the shadow register, the current display frame continuously calls the original coefficients in the working register to maintain the image rendering.

[0025] In some specific embodiments, a display feedback unit is also included. The display feedback unit is connected to the real-time image geometry correction unit on the programmable logic side and is configured to output the corrected image and synchronously overlay calibration points, grid lines or error feedback information to form an online debugging closed loop with the calibration interactive input unit.

[0026] This invention proposes an interactive closed-loop calibration and real-time geometric correction method and system for optical display images based on FPGA. Through a hardware-software co-processing SoC closed-loop architecture, it highly integrates multi-entry calibration data interaction, online least-squares calculation of biquadratic polynomial parameters on the PS side, fixed-point quantization, and pixel-level coordinate inverse mapping, bilinear interpolation, and visualization feedback on the PL side into a single processing chip. This completely breaks down the technical barriers of severe separation between offline calibration, transmission, and low-level execution in traditional image geometric correction techniques. Furthermore, this invention employs a unique parameter shadow register and working register separation update mechanism to uniformly switch the effective coefficients at the frame boundaries where the vertical synchronization signal is accurately captured. It not only perfectly overcomes the long-link time overhead and bandwidth load of frequent rewriting and reconstruction of large-scale pixel lookup tables in traditional block grid correction schemes, but also completely eliminates the engineering pain points such as screen tearing, visual flickering and inconsistent local nonlinear deformation that inevitably accompany continuous online parameter tuning or dynamic fine-tuning preview from the hardware underlying mechanism. It achieves high-speed iterative convergence of correction effect and excellent long-term display consistency in complex optical distortion scenarios such as vehicle HUD, transparent display screen or multi-projection superposition. It has advantages such as high energy efficiency, flexible architecture, easy embedding in terminal and significantly shortened on-site assembly and verification cycle. Attached Figure Description

[0027] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments and, together with the description, serve to explain the principles of the invention. Other embodiments and many anticipated advantages of the embodiments will be readily recognized as they become better understood through reference to the following detailed description. Elements in the drawings are not necessarily to scale. The same reference numerals refer to corresponding similar parts.

[0028] Figure 1 This is a flowchart of an FPGA-based interactive closed-loop calibration and real-time geometric correction method for optical display images, according to an embodiment of the present invention. Figure 2 This is a framework diagram of an FPGA-based interactive closed-loop calibration and real-time geometric correction system for optical display images, according to an embodiment of the present invention. Figure 3 This is a schematic diagram of a specific embodiment of the FPGA-based interactive closed-loop calibration and real-time geometric correction system for optical display images. Figure 4 This is a flowchart of a specific embodiment of the interactive closed-loop calibration and real-time geometric correction method for optical display images based on FPGA of the present invention; Figure 5This is a timing diagram of parameter writing and frame synchronization switching according to a specific embodiment of the present invention; Figure 6 This is a schematic diagram of the connection between the parameter register and the bus interface circuit in a specific embodiment of the present invention; Figure 7 This is a schematic diagram of a web-based 9-point calibration interface according to a specific embodiment of the present invention; Figure 8 This is a schematic diagram of an automatic calibration pattern display interface according to a specific embodiment of the present invention; Figure 9 This is a physical diagram of the complete machine connection according to a specific embodiment of the present invention. Detailed Implementation

[0029] In the following detailed description, reference is made to the accompanying drawings, which form part of the detailed description and are illustrated by specific illustrative embodiments in which the invention may be practiced. In this regard, directional terms such as “top,” “bottom,” “left,” “right,” “up,” “down,” etc., are used with reference to the orientation of the described figures. Because components of the embodiments can be positioned in several different orientations, directional terms are used for illustrative purposes and are by no means limiting. It should be understood that other embodiments may be utilized or logical changes may be made without departing from the scope of the invention. Therefore, the following detailed description should not be taken in a limiting sense, and the scope of the invention is defined by the appended claims.

[0030] Figure 1 A flowchart illustrating an embodiment of the present invention provides an interactive closed-loop calibration and real-time geometric correction method for optical display images based on FPGA. Figure 1 As shown, the method includes: S1: Obtain calibration interactive input data, which includes multiple sets of calibration points consisting of the target display coordinates and the corresponding source image sampling coordinates.

[0031] S2: On the processing system side, based on multiple sets of calibration points, a biquadratic inverse mapping model is constructed, the biquadratic geometric correction coefficients are obtained, and the geometric correction coefficients are subjected to fixed-point quantization processing.

[0032] In a specific embodiment, the step of constructing a biquadratic inverse mapping model and solving for the biquadratic geometric correction coefficients in S2 specifically includes: converting multiple sets of calibration points into centered coordinates with the display center as the origin, the centered coordinates of the target display point are denoted as (u,v), and the centered coordinates of the source image sampling points are denoted as (p,q); constructing an inverse mapping relationship from the target display coordinates to the source image sampling coordinates: p=A0+A1·u+A2·v+A3·u²+A4·v²+A5·u·v; q=B0+B1·u+B2·v+B3·u²+B4·v²+B5·u·v; where A0-A5 and B0-B5 are the 12 geometric correction coefficients to be solved. By establishing an inverse mapping relationship using biquadratic polynomials, not only can the overall positional offset, proportional stretching, and trapezoidal distortion be compensated through low-order terms, but the local geometric compensation capability is also enhanced by quadratic terms and cross-coupling terms. This perfectly adapts to irregular and nonlinear bending distortions caused by glass curvature or lens assembly deviations. On this basis, a basis function matrix is ​​constructed using centered coordinates and online solution is performed using the least squares method. This ensures that when collecting multiple sets of calibration points, the system can detect outliers based on residuals or condition numbers and prompt for re-collection or removal before re-solving, thus obtaining correction parameters with mathematical stability and global optimal fitting accuracy.

[0033] In a specific embodiment, the geometric correction coefficients A0-A5 and B0-B5 are obtained by solving multiple sets of calibration points using the least squares method, specifically including: Collect N sets of calibration points, where N≥6; For the i-th calibration point, based on the centered coordinates (ui,vi) of its target display point, construct the corresponding basis function row vector Hi=[1,ui,vi,ui²,vi²,ui·vi], and combine the basis function row vectors of all N calibration points to form matrix H; The centered coordinates (pi, qi) of the source image sampling points corresponding to the N sets of calibration points are combined to form the horizontal centered coordinate vector P = [p1, p2, ..., pN] of the source image. T And the vertically centered coordinate vector Q=[q1,q2,...,qN] T ; Define the horizontal correction coefficient vector as A = [A0, A1, A2, A3, A4, A5]. T The vertical correction coefficient vector is B = [B0, B1, B2, B3, B4, B5]. T ; Solve using any one of the following methods: least squares, generalized inverse, QR decomposition, SVD, or equivalent numerical method, and obtain the solution using the formula A=(H). T ·H) -1 ·H T ·P and B=(HT ·H) -1 ·H T The geometric correction coefficients A0-A5 and B0-B5 are obtained by solving Q.

[0034] In a specific embodiment, the step of performing fixed-point quantization on the geometric correction coefficients in S2 specifically includes: converting the obtained floating-point geometric correction coefficients into a signed fixed-point number format with a preset bit width; the signed fixed-point number format is Q16.16, where the high 16 bits are used to represent the signed integer part and the low 16 bits are used to represent the fractional part. Since the obtained polynomial coefficients are initially in floating-point form, quantizing them into a signed fixed-point number format (Q16.16) with a high 16-bit integer and a low 16-bit fractional part can preserve the minute numerical changes of the second-order terms and cross terms with extremely high precision, greatly compressing the quantization error; at the same time, since floating-point operations are completely converted into fixed-point operations, when the programmable logic (PL) side calculates the high-order polynomial in each pixel cycle, it only needs to call ordinary fixed-point multipliers and adders, which greatly releases hardware logic resources and significantly improves the clock frequency.

[0035] S3: Write the quantized geometric correction coefficients into the shadow register located on the programmable logic side, and when the display frame synchronization boundary is detected, switch the geometric correction coefficients in the shadow register to the working register so that the geometric correction coefficients take effect from the next frame.

[0036] S4: On the programmable logic side, based on the geometric correction coefficients in the working register, the target display pixel is reverse mapped to obtain the corresponding absolute sampling coordinates of the source image. Then, based on the absolute sampling coordinates of the source image, the source image pixels are read and interpolated to output the geometrically corrected display image.

[0037] In a specific embodiment, the resampling interpolation processing performed on the programmable logic (PL) side can flexibly employ various spatial interpolation algorithms such as nearest neighbor interpolation, bilinear interpolation, or bicubic interpolation, depending on the system resource budget and image quality accuracy requirements. In this embodiment, the bilinear interpolation algorithm, which can be efficiently pipelined and implemented with low resources within the FPGA, is preferred. The steps of performing coordinate inverse mapping and interpolation processing in S4 specifically include: if the absolute sampling coordinates of the source image obtained by inverse mapping exceed the effective range of the source image, a boundary clamping method is used to limit the range of sampling coordinate values ​​to avoid interpolation addressing out of bounds; the integer part and the fractional part of the absolute sampling coordinates of the source image are extracted, the fractional part is converted into a weight within a preset range, and the weight and the four pixels in the neighborhood of the integer part are multiplied and added, then shifted and normalized to obtain the output pixel. By employing a reverse mapping from the target display coordinates to the source image, pixel holes, overlaps, or patching issues that may occur during forward distortion correction are fundamentally avoided. Furthermore, by performing boundary clamping (limiting to the effective range) on the absolute sampling coordinates, it is ensured that the underlying hardware pipeline will not experience addressing out-of-bounds errors, DDR / BRAM memory access anomalies, or system crashes when performing bilinear interpolation to address neighboring pixels. This significantly improves the hardware fault tolerance and operational stability of the device at extreme distortion edges.

[0038] In a specific embodiment, the output pixel is calculated as follows: Convert the horizontal fractional part α and the vertical fractional part β of the absolute sampling coordinates of the source image into 8-bit unsigned fractional weighted integers dx and dy, dx=round(α×255), dy=round(β×255); The four bilinear interpolation weights w00, w01, w10 and w11 corresponding to the target display pixel are calculated using dx and dy respectively. The calculation formulas are: w00=(255-dx)(255-dy), w01=dx(255-dy), w10=(255-dx)dy, w11=dx·dy; Extract four neighboring pixels P00, P01, P10, and P11 from the source image relative to the integer part (x0, y0), where P00 = I(x0, y0), P01 = I(x0+1, y0), P10 = I(x0, y0+1), and P11 = I(x0+1, y0+1), and I represents the color channel pixel value of the source image at the corresponding coordinates; The RGB color channels of the four source image pixels P00, P01, P10, and P11 are multiplied and added respectively, and then normalized according to the total weight scale to obtain the output pixel value Pout. The calculation formula is: Pout=(w00·P00+w01·P01+w10·P10+w11·P11) / 255².

[0039] In a specific embodiment, calibration interactive input data is acquired through a multimodal unified solution architecture. This architecture is configured to extract drag-and-drop data from the webpage, gesture data from the touchscreen, and visual detection results from an external camera into calibration point coordinates in a unified format, and then input them into the same parameter solution link for matrix operations. By abstracting and transforming various interactive input modalities, such as touchscreen gesture drag-and-drop, remote calibration from the webpage, and automatic structured light detection points from an external camera, into a unified format of standard point matrix correspondences input into the same least squares solution link, the system achieves unification and generalization of multi-source control entry points. This allows the system to flexibly accommodate different business application scenarios, such as manual fine-tuning, remote debugging, and fully automated factory production line calibration, greatly improving the flexibility and functional integration of the calibration system.

[0040] In a specific example, this invention employs a reverse mapping method from target display coordinates to source image sampling coordinates. Let the display resolution be W×H, and the display center coordinates be: Cx=W / 2; Cy=H / 2; in a specific embodiment, W=1024, H=600, therefore Cx=512, Cy=300. For any target pixel on the display, its target display coordinates are: Pd=(Xd,Yd), which are first converted to centered coordinates (u,v) with the display center as the origin: u=Xd–Cx, v=Yd–Cy. The system uses a biquadratic polynomial to establish the reverse mapping relationship between the target display coordinates and the source image sampling coordinates. Let the sampled coordinates of the centered source image be Ps_c=(p,q), where p represents the horizontal coordinate of the source image sample point relative to the image center, and q represents the vertical coordinate of the source image sample point relative to the image center. Then we have: p=A0+A1·u+A2·v+A3·u²+A4·v²+A5·u·v; q=B0+B1·u+B2·v+B3·u²+B4·v²+B5·u·v. The absolute sampling coordinates in the source image are: Ps=(Xs,Ys), which are calculated as: Xs=p+Cx, Ys=q+Cy; where A0-A5 and B0-B5 are the 12 geometric correction coefficients to be solved. Specifically, A0 and B0 are used to represent horizontal and vertical translation compensation; A1 and B2 are used to represent horizontal and vertical scaling compensation; A2 and B1 are used to represent shearing or rotational coupling compensation; A3, A4, B3, and B4 are used to represent second-order bending or local nonlinear compensation; and A5 and B5 are used to represent XY cross-coupling compensation.

[0041] During the calibration process, the system collects N sets of calibration points, where N≥6. Each set of calibration points includes the target display point and the corresponding source image sampling point: Pdi=(Xdi,Ydi); Psi=(Xsi,Ysi). The i-th calibration point is centered: ui=Xdi–Cx; vi=Ydi–Cy; pi=Xsi–Cx; qi=Ysi–Cy; the basis function row vector corresponding to the i-th calibration point is constructed: Hi=[1,ui,vi,ui²,vi²,ui·vi]; all N calibration points form a matrix: The horizontal correction coefficient vector is defined as: A = [A0, A1, A2, A3, A4, A5] T The vertical correction coefficient vector is defined as: B = [B0, B1, B2, B3, B4, B5] T The horizontally centered coordinate vector and the vertically centered coordinate vector of the source image are respectively: P=[p1,p2,...,pN] T Q = [q1, q2, ..., qN] T When N=6, the linear equation system can be solved directly: H·A=P, H·B=Q. When N>6, the least squares method is used. The coefficient vector can be obtained using normal equations, QR decomposition, singular value decomposition (SVD), or other equivalent numerical methods. In one embodiment, it can be expressed as: A=pinv(H)·P, B=pinv(H)·Q, where pinv(H) represents the generalized inverse of H. If the normal equation form is used, it can be written as: A=(H T ·H) -1 ·H T ·P, B=(H T ·H) -1 ·H T • Q. To improve solution stability, calibration points should cover the center, edges, and corners of the display area, and the calibration points should not be concentrated on the same straight line or in an excessively small local area. When the system detects that the matrix condition number is too large, the calibration points are abnormal, or the residual exceeds the threshold, it can prompt the user to re-acquire calibration points or remove abnormal points and solve the problem again.

[0042] After solving, the horizontal residual, vertical residual, and combined residual of the i-th calibration point can be calculated: exi = Hi·A – pi, eyi = Hi·B – qi, ei = sqrt(exi² + eyi²). The overall calibration error can be expressed as the root mean square error: ERMS = sqrt((1 / N)·Σ(ei²)). If ERMS is less than the preset threshold, the current parameters are considered to meet the correction requirements; if the error is large, the calibration points should be adjusted or the detection points should be reacquired.

[0043] After obtaining the 12 floating-point correction coefficients (A0-A5 and B0-B5) on the PS side, they are converted into a 32-bit signed fixed-point format. In a specific embodiment, the Q16.16 fixed-point format is used, where the high 16 bits represent the signed integer part and the low 16 bits represent the fractional part. The quantization relationship is: Q16.16_value = round(real_value × 65536). For example, 1.000000 corresponds to 0x00010000, 0.000160 corresponds to 0x0000000A, and -5.000000 corresponds to 0xFFFB0000. For display systems with higher resolution or higher precision requirements, equivalent fixed-point formats such as Q8.24, Q4.28, or coordinate normalization followed by quantization can also be used to reduce quantization errors of second-order terms and cross terms. On the PL side, within each target pixel period, the following calculations are performed sequentially based on the current target pixel coordinates: u = Xd – Cx, v = Yd – Cy, u², v², u·v. Then, the centered source image sampling coordinates are calculated using the following formulas: p = A0 + A1·u + A2·v + A3·u² + A4·v² + A5·u·v; q = B0 + B1·u + B2·v + B3·u² + B4·v² + B5·u·v. These are then converted to absolute source image sampling coordinates: Xs = p + Cx, Ys = q + Cy. Let the source image sampling coordinates obtained by the inverse mapping be: Xs = x0 + α, Ys = y0 + β, where: x0 = floor(Xs), y0 = floor(Ys), α = Xs - x0, β = Ys - y0. When the sampled coordinates of the source image obtained by reverse mapping exceed the valid range of the source image, the system can process them using methods such as boundary clamping, boundary duplication, black filling, or mirror filling. In a preferred embodiment, boundary clamping is used, that is, when Xs < 0, Xs = 0, and when Xs > W-1, Xs = W-1; when Ys < 0, Ys = 0, and when Ys > H-1, Ys = H-1. For bilinear interpolation scenarios that require access to x0+1 or y0+1, x0 can be further restricted to 0 to W-2, and y0 can be restricted to 0 to H-2, or adjacent boundary pixels can be copied at the edge to avoid accessing out-of-bounds addresses. Take four adjacent pixels from the source image: P00 = I(x0, y0), P01 = I(x0+1, y0), P10 = I(x0, y0+1), P11 = I(x0+1, y0+1). The output pixel is: Pout=(1-α)(1-β)P00+α(1-β)P01+(1-α)βP10+αβP11.

[0044] In hardware implementation, α and β can be converted into 8-bit decimal weights. For ease of multiplication and shift normalization, equivalent weights of 0 to 255 or 0 to 256 can be used. In a specific embodiment, let dx = round(α × 255); dy = round(β × 255); the corresponding weights are: w00 = (255 - dx)(255 - dy), w01 = dx(255 - dy), w10 = (255 - dx)dy, w11 = dx·dy. After multiplying and adding the three RGB color channels respectively, normalization is performed according to the total weight scale to obtain the output pixel: Pout = (w00·P00 + w01·P01 + w10·P10 + w11·P11) / 255².

[0045] In FPGA implementation, the above division can be accomplished by shifting, approximate normalization, or preset scaling factor; alternatively, a 256-weight system can be used, so that the normalization process is achieved by right shifting by 16 bits, thereby reducing hardware complexity.

[0046] A key inventive aspect of this invention lies in combining interactive calibration point input, matrix parameter solving, Q16 fixed-point quantization, AXI-Lite parameter writing, frame synchronization parameter switching, and PL pixel-level real-time correction into a single closed loop, rather than generating the pixel grid offline in the host computer or embedded software. With this structure, after the operator adjusts the calibration points on the touchscreen or web interface, the PS side can immediately reconstruct the matrix and solve for the 12 coefficients, then write the parameters to the PL side's shadow register, and the PL side uniformly switches and applies the changes at the next frame boundary. This method reduces screen flicker and tearing during parameter adjustment and improves the real-time feedback capability of on-site calibration.

[0047] Another important aspect of this invention is the use of an inverse biquadratic mapping from the target display coordinates to the source image coordinates. This allows the PL side to calculate only which position in the source image each output pixel should be sampled from, avoiding the holes, overlaps, or patching problems that may occur with forward mapping. Compared to a single affine matrix or homography matrix, the biquadratic model adds local compensation terms such as X², Y², and X·Y, making it suitable for scenarios in HUDs, transparent displays, or combined mirror displays where there are simultaneous overall offsets, trapezoidal changes, and local curvatures. Compared to a complete lookup table method, it reduces the need for large-scale pixel grid storage and frequent lookup table rewriting. For example, for overall offset distortion, if the displayed image is observed to be offset 8 pixels to the right and 5 pixels upward relative to the target position, the following inverse sampling model can be used: p=u+8, q=v–5, with corresponding coefficients A0=8, A1=1, B0=-5, B2=1, and the remaining coefficients being 0. For the target pixel Pd=(768,450), we have u=768-512=256, v=450-300=150. Substituting these into the above model, we get: p=256+8=264, q=150-5=145. Therefore, the absolute sampling coordinates of the source image are: Xs=p+Cx=264+512=776, Ys=q+Cy=145+300=445. That is, when the target pixel (768,450) is output, it is sampled from approximately (776,445) in the source image, thereby compensating for the overall positional offset. For example, for trapezoidal or approximate perspective distortion, if the upper edge of the displayed image is compressed and the lower edge is expanded, a u·v cross term can be introduced for compensation. Let the inverse sampling model be: p = u + 0.000160·u·v, q = v, then A5 = 0.000160, and its Q16 fixed-point value is round(0.000160 × 65536) = 10. For the target pixel Pd = (912, 100), we have u = 912 - 512 = 400, v = 100 - 300 = -200. Substituting these values ​​into the above model, we get: p = 400 + 0.000160 × 400 × (-200) = 387.2, q = -200. Therefore, the absolute sampling coordinates of the source image are: Xs = 387.2 + 512 = 899.2, Ys = -200 + 300 = 100. When the target pixel (912, 100) is output, sampling is performed from approximately (899.2, 100) in the source image to compensate for upper edge compression or trapezoidal distortion. For example, for edge curvature or local nonlinear distortion, if the optical path causes vertical curvature near the left and right edges, a u² term can be introduced for compensation. Let the inverse sampling model be: p = u, q = v + 0.000080·u², then B3 = 0.000080, and its Q16 fixed-point value is round(0.000080 × 65536) = 5.For the target pixel Pd=(912,300), we have u=912-512=400, v=300-300=0. Substituting these values ​​into the model above, we get: p=400, q=0+0.000080×400²=12.8. Therefore, the absolute sampling coordinates of the source image are: Xs=400+512=912, Ys=12.8+300=312.8. That is, when the target pixel (912,300) is output, it is sampled from approximately (912,312.8) in the source image, thereby compensating for the vertical offset caused by edge curvature.

[0048] In a scenario with comprehensive distortion, the system can display a 9-point or 6×5 calibration pattern and collect the corresponding relationships between each point. For example, if the target display coordinates of a calibration point are Pd=(768,450) and the corresponding source image sampling coordinates are Ps=(780,458), then we have: u=768-512=256, v=450-300=150, p=780-512=268, q=458-300=158. The basis function row vector corresponding to this calibration point is: Hi=[1,256,150,65536,22500,38400]. This row vector, together with other calibration points, forms matrix H and participates in the least squares solution of the horizontal coefficient vector A and the vertical coefficient vector B, respectively. The obtained A0-A5 and B0-B5 are written to the PL side parameter register after Q16 fixed-point quantization. The PL side uses the new parameters to perform real-time inverse mapping and bilinear interpolation at the frame boundary, and finally outputs the corrected display image.

[0049] Through the above technical solution, this invention achieves a complete closed loop from interactive calibration to real-time FPGA correction. Its technical advantages are: it can solve for real-time corrective parameters using a small number of calibration points or automatic detection points; it can express various optical distortion patterns using 12 coefficients; it can avoid frequent rewriting of large-scale pixel lookup tables; it can reduce tearing and flickering caused by parameter updates through a frame synchronization mechanism; it can ensure low-latency, deterministic pixel-level correction output through a PL-side pipeline; and it can form a unified interactive closed loop for field debugging through web pages, touch screens, serial / network connections, and optional camera detection.

[0050] Figure 2 This is a framework diagram of an FPGA-based interactive closed-loop calibration and real-time geometric correction system for optical display images, as an embodiment of the present invention. Figure 2As shown, the system includes a calibration interactive input unit 201, a parameter solving and control unit 202 on the processing system side, a parameter register and synchronization update unit 203, and a real-time image geometric correction unit 204 on the programmable logic side. The system includes a calibration interactive input unit 201 configured to acquire calibration interactive input data, which includes multiple sets of calibration points consisting of the target display coordinates and the corresponding source image sampling coordinates. A processing system-side parameter solving and control unit 202 is configured to construct a biquadratic inverse mapping model based on the multiple sets of calibration points, solve for the biquadratic geometric correction coefficients, and perform fixed-point quantization on the geometric correction coefficients. A parameter register and synchronization update unit 203 is configured with a shadow register and a working register, used to write the quantized geometric correction coefficients into the shadow register and, upon detecting a display frame synchronization boundary, uniformly switch the geometric correction coefficients in the shadow register to the working register. A programmable logic-side real-time image geometric correction unit 204 is configured to perform coordinate inverse mapping on the target display pixels according to the geometric correction coefficients in the working register, obtain the corresponding source image absolute sampling coordinates, and read the source image pixels based on the source image absolute sampling coordinates for linear interpolation processing, outputting the geometrically corrected display image. This system can be used to execute the specific steps of the aforementioned method.

[0051] In a specific embodiment, the parameter solving and control unit on the processing system side is connected to the parameter register and synchronization update unit via the AXI-Lite on-chip bus. During the writing of geometric correction coefficients to the shadow register, the parameter register and synchronization update unit continuously call the original coefficients in the working register to maintain the image rendering of the current display frame. By using the lightweight on-chip bus (AXI-Lite) for 32-bit read and write, the parameter writing operation on the PS side can be constrained within a dedicated shadow register space, while the video stream rendering engine on the PL side completely isolates and continuously calls the original correction parameters in the working register during the writing process. This structure cleverly achieves complete decoupling of software parameter distribution and hardware real-time pixel processing in terms of clock domain, timing, and data flow, eliminating from the underlying mechanism the possibility of partial image tearing or line misalignment caused by partial rewriting of parameters during the rendering of a single frame.

[0052] In specific embodiments, a display feedback unit is also included. This unit is connected to the real-time image geometry correction unit on the programmable logic side and is configured to output the corrected image and simultaneously overlay calibration points, grid lines, or error feedback information to form an online debugging closed loop with the calibration interactive input unit. By selectively and dynamically overlaying 9-point calibration patterns, calibration grid lines, preview point lock status, and real-time root mean square error (ERMS) feedback information on the final output screen, extremely intuitive real-time visual and quantitative verification basis is provided to on-site debugging personnel or closed-loop testing cameras. This establishes a millisecond-level intuitive mapping relationship between parameter adjustment input and effect verification, forming a high-speed convergence calibration closed loop of "adjustment-preview-judgment-re-iteration," significantly reducing equipment assembly and on-site verification time.

[0053] Figure 3 This is a schematic diagram of a specific embodiment of the present invention: an FPGA-based interactive closed-loop calibration and real-time geometric correction system for optical display images. Figure 3As shown, the system's overall architecture is mainly composed of a calibration interactive input unit, a PS-side parameter solving and control unit, a parameter register and synchronization update unit, a PL-side real-time image geometric correction unit, a pixel buffer and interpolation output unit, and a display feedback unit, which together construct a complete closed-loop structure of "interactive calibration—parameter solving—parameter writing—frame synchronization activation—real-time correction output—display feedback". At the system's input end, the HDMI video input, as the carrier of the raw video stream, is directly connected to the PL real-time display pipeline. At the same time, external camera image input, a touch panel supporting 9-point or gesture control, a web browser for remote calibration, and a parameter update terminal based on UART / network together constitute a multi-entry unified control calibration interactive input source. Manual control commands of different modalities, remote drag-and-drop data, or automatic camera detection results can all be uniformly entered into the same solution link and sent to the PS parameter domain for centralized processing. Within the intermediate processing system (PS) parameter domain, the processor first normalizes the received 9-point or 6×5 grid calibration points. Then, it uses the least squares method to calculate 12 biquadratic geometric correction coefficients (A0-A5 and B0-B5) in parallel online. Next, the obtained floating-point coefficients are converted to Q16 signed fixed-point format and packed into registers. Finally, the packed parameters are sent to the programmable logic (PL) side via AXI-Lite or an equivalent on-chip lightweight bus. These parameters are first written into the shadow register on the PL side. During the writing process, the current display frame continuously calls the old parameters in the working register to maintain image stability. Once the hardware pipeline detects a vertical synchronization signal or frame boundary, it switches the shadow parameters to the working register to make the new coefficients effective. This fundamentally eliminates tearing and flickering caused by mixing parameters within a single frame. In the real-time display pipeline on the right, the pixel-level hardware pipeline works at high speed and collaboratively within each target pixel cycle. Its micro-topology, on the one hand, forms a resampling data path through a dual quadratic inverse mapping module, a DDR / FIFO line buffer module, and a bilinear interpolation module. On the other hand, it uses a coordinate generation module and a pixel neighborhood reading module to calculate the absolute sampling coordinates of the current target pixel in the source image in real time and accurately retrieve the corresponding neighboring pixel values. Both converge at the output formatting module for multiplication, addition, and shift normalization, ultimately rendering a low-latency LCD / HUD display with correct perspective in real time at the output end. Finally, the display feedback unit at the output end dynamically overlays the latest preview point, calibration grid results, and frame synchronization register switching status onto the current output screen and returns them to the calibration interactive input end. This provides millisecond-level quantitative and intuitive data for on-site debugging personnel or camera verification, guiding the next round of calibration point fine-tuning and iteration. Ultimately, this achieves a fully embedded, highly integrated interactive geometric correction closed-loop device for optical display images.

[0054] Figure 4This is a flowchart of a specific embodiment of the FPGA-based interactive closed-loop calibration and real-time geometric correction method for optical display images, as shown below. Figure 4As shown, the process begins with step one, where a specific calibration pattern is actively output on the display. Then, in step two, the system efficiently acquires standard calibration data for a nine-point or six-by-five dot matrix through various entry points, including touch gestures, remote interaction via a web interface, serial or network parameters, or visual detection from an external camera. This transforms the actual optical distortion into discrete coordinate correspondences. Subsequently, the process enters the core computing domain of the processing system (PS) in step three. The processor centrally processes the acquired calibration point coordinates and constructs a basis function matrix. It then uses the least squares method to quickly solve for twelve biquadratic geometric correction coefficients (both horizontal and vertical), thus accurately fitting a complex nonlinear distortion compensation model mathematically. To adapt to the high-speed parallel processing of the underlying hardware pipeline, step four converts the obtained floating-point correction coefficients into signed Q16 fixed-point format and packages the register parameters. Finally, in step five, the PS side uses AXI-Lite or an equivalent on-chip lightweight bus to precisely write the quantized and packaged coefficients into the parameter registers on the programmable logic (PL) side. In step six, the hardware pipeline does not immediately apply the parameters to the currently rendered image. Instead, it keeps the working register running the old parameters and silently waits for the display frame synchronization signal. Only when the frame boundary of the vertical synchronization signal is detected is the new parameter in the shadow register switched to the working register to take effect, thus completely eliminating screen tearing and flickering. The hardware execution phase then begins. In step seven, the PL-side real-time correction pipeline performs high-speed inverse coordinate transformation and warping on each target display pixel based on the latest effective coefficients, calculating its absolute sampling coordinates in the source image. In step eight, it seamlessly calls neighboring pixels to complete high-precision fixed-point bilinear interpolation, and then in step nine, renders and outputs a geometrically corrected, high-quality perspective display image at the LCD or HUD display output. In step ten, the system provides intuitive display feedback by dynamically overlaying rich information such as preview points, calibration grids, or locked status on the final output screen. In step eleven, the system rigorously judges whether the current correction error has converged to within the preset threshold through on-site personnel's intuitive visual verification or fully automatic error residual calculation by external cameras. If the overall root mean square error is detected to be below the preset accuracy requirements, the system will quickly reverse the feedback result back to the calibration interactive input terminal, prompting the debugging personnel to continue online fine-tuning by dragging the calibration points or driving the camera to re-detect and extract coordinates. This allows the entire method to flow efficiently in the chain of "displaying calibration pattern - collecting grid data - calculating correction coefficients - packaging quantization and writing - frame synchronization boundary taking effect - PL real-time reverse mapping and interpolation - displaying screen preview feedback - closed-loop iterative convergence", ultimately realizing a fully embedded, highly integrated image interactive geometric correction closed-loop control process.

[0055] Figure 5This is a timing diagram of parameter writing and frame synchronization switching in a specific embodiment of the present invention, as shown below. Figure 5 As shown, this timing diagram extends horizontally along the time axis, dividing the continuously evolving video frame period into frames n, n+1, n+2, and n+3. Vertically, it parallels five core timing control and status axes from top to bottom: vertical synchronization / frame boundary signal, PS write to shadow register, working register parameter selection, display output parameters, and screen status description. On the topmost vertical synchronization / frame boundary signal axis, a high-level trigger pulse is periodically generated at the boundary of each frame, precisely defining the display frame boundary clock. Below this, on the PS write to shadow register axis, after the PS side completes the online calculation of the biquadratic geometric correction coefficients, the parameter distribution operation transcends the frame period limitation, starting in the middle to late part of frame n and continuously writing new Q16 fixed-point parameters A0-A5 and B0-B5 into the shadow register on the PL side. The entire writing process smoothly continues into the early to middle part of the next frame. Meanwhile, the third-layer working register parameter selection axis and the fourth-layer display output parameter axis perfectly embody the hardware isolation and buffering mechanism that separates the shadow register and the working register. Throughout the entire time interval during which the PS continuously writes new parameters to the shadow register, the control logic drives the working register to always latch the original old parameter Pold. This ensures that the currently output video frame uses only and exclusively the old parameter Pold for reverse mapping and resampling rendering within the corresponding frame period. The timing control logic of "no frame switching during writing" is clearly corresponding to the bottommost screen status axis, thus completely eliminating the mixing of old and new parameters caused by bus rewriting coefficients during the rendering of a single frame at the underlying hardware level. When the timing progresses to the first vertical synchronization pulse (frame boundary signal) after the parameters are issued, the on-chip synchronization control logic on the PL side instantly triggers atomic switching. This moves all the new parameters, which have been fully received and verified in the shadow register, to the working register at once. This causes the actual effective parameters in the working register to quickly switch to the new parameters Pnew. The display output parameter axis also immediately switches to the new parameters Pnew from the frame boundary onwards for high-speed warp rendering of the real-time video stream. This precise frame synchronization switching timing mechanism directly translates into a high-quality visual effect of "new parameters taking effect for the entire frame" in the latter half of the screen status axis. While maintaining extremely low video latency, it completely eliminates the engineering pain points that are easily caused by line misalignment and parameter asynchrony during continuous online parameter tuning or dynamic fine-tuning previews, such as screen tearing, visual flickering, and inconsistent local nonlinear deformations. This provides a solid micro-timing and circuit guarantee for achieving stable, continuous, and visually tear-free embedded closed-loop interactive parameter tuning.

[0056] Figure 6This is a schematic diagram of the connection between the parameter register and the bus interface circuit in a specific embodiment of the present invention, as shown below. Figure 6 As shown, in this architecture, the central control unit located on the processing system (PS) side establishes a direct data communication connection with the parameter register module on the PL side through a 32-bit wide AXI-Lite lightweight on-chip bus or an equivalent memory-mapped bus interface. The processor can precisely write the twelve biquadratic geometric correction coefficients (i.e., horizontal mapping coefficients A0 to A5 and vertical mapping coefficients B0 to B5) that have been solved online and completed fixed-point quantization processing into the corresponding register addresses according to the preset address mapping space. Simultaneously, this bus interface is also responsible for synchronously sending rich interactive control data, including control flags for scheduling the underlying hardware logic, touch coordinate data from the external panel, grid calibration coordinates for assisting visual feedback, and preview status information representing the current debugging process. As the core hub for the convergence of software and hardware data, the parameter register module has a bidirectional interactive capability in its structure: on the one hand, it provides a complete status read and write interface to the PS side, enabling the software control end to monitor the working status of the PL side pipeline in real time and verify the accuracy of parameter issuance; on the other hand, it directly drives the real-time image geometry correction unit on the PL side, stably and continuously outputting the currently effective correction parameters after internal frame synchronization switching mechanism to the underlying hardware rendering pipeline. This efficient connection and isolation method based on standardized bus and customized register group completely opens up the complete data link from the software side parameter solving and interactive status management to the hardware side pixel-level coordinate remapping at the physical circuit level, enabling high-level matrix operations and nanosecond-level low-level video stream interpolation processing to work seamlessly together within a unified FPGA or SoC system architecture.

[0057] Figure 7 This is a schematic diagram of a web-based 9-point calibration interface according to a specific embodiment of the present invention, as shown below. Figure 7As shown, the graphical user interface integrates a wealth of control and feedback elements. It includes modules for configuring and monitoring the serial port or network communication connection status of the underlying data link, as well as convenient buttons for automatic scan triggering, parameter sending, and topology reset. The core visual area of ​​the interface presents a calibration grid topology structure composed of nine freely adjustable calibration points, and displays real-time, digital feedback on the side of the interface showing the current calculated values ​​of various calibration parameters and the system's lock status. In actual field calibration and debugging, operators can either drag and drop the calibration points on the web interface for localized fine-tuning, trigger automatic detection with a single click to quickly obtain target coordinates, or execute a reset command to initialize the grid topology. The operator adjusts the spatial position of the calibration point at the front end and issues a transmission command. The latest calibration point coordinate data captured by the web page is quickly packaged and transmitted to the underlying processing system (PS). Subsequently, the control algorithm on the PS side responds immediately, reconstructs the matrix based on the updated coordinate array, and calculates the latest biquadratic geometric correction coefficients online. Finally, the quantized coefficients are sent to the parameter register on the programmable logic (PL) side via the on-chip bus, thereby driving the real-time display link to seamlessly update the screen at the frame boundary. This design, which tightly integrates intuitive drag-and-drop at the front end with high-speed reconstruction at the lower level, greatly reduces the debugging threshold of complex optical geometric correction and achieves a WYSIWYG, highly efficient iterative closed loop in the field.

[0058] Figure 8 This is a schematic diagram of an automatic calibration pattern display interface according to a specific embodiment of the present invention, such as... Figure 8 As shown, the interface uses a high-contrast, dark solid color as the background. The central area regularly generates and highlights a 6×5 standard dot matrix feature pattern composed of 6 horizontal columns and 5 vertical rows. A white rectangular bounding box is drawn around the dot matrix to physically define and select the effective optical projection area. In the actual automated calibration workflow, an external camera or imaging device captures the distorted image of this pattern in the actual optical display path and sends the image to the host computer or PS-side processing unit for calibration point identification, coordinate extraction, and distortion parameter fitting. Based on the correspondence between the identified calibration point coordinates and the target standard coordinates, the system solves for geometric correction parameters and sends them to the FPGA real-time correction link, thereby achieving automated distortion detection and closed-loop correction.

[0059] Figure 9 This is a physical diagram of the overall connection of a specific embodiment of the present invention, as shown below. Figure 9As shown, the entire hardware system includes an FPGA / SoC development board, an LCD touch screen, video input cables, display output cables, a power interface, and debugging cables. The FPGA development board serves as the core processing unit for image acquisition, parameter reception, real-time geometric correction, and display output. The LCD touch screen acts as a display feedback and human interaction input terminal, used to display the image to be corrected, calibration points, or the corrected image result. The video interface, touch interface, and related cables on the development board are used to complete video data input, touch coordinate acquisition, and correction result output. This physical diagram illustrates that the interactive calibration, parameter distribution, real-time correction on the PL side, and display feedback of this invention can be completed within the same system, thus forming a field-debuggable and verifiable closed-loop correction device.

[0060] The interactive closed-loop calibration and real-time geometric correction method and system for optical display images based on FPGA described in this invention have extremely high flexibility and scalability in their hardware architecture, input / output modes, and underlying algorithm models, and are not limited to the figures and the specific embodiments described above. The operating platform on the processing system (PS) side is not limited to a specific ZYNQ architecture. In other equivalent embodiments of this invention, it can be flexibly replaced with other FPGA or SoC hardware architecture platforms with embedded ARM control cores, external independent processors, or soft-core processors built inside the FPGA, depending on the overall cost, power consumption limitations, and computing performance requirements of the display system. As long as the hardware platform can complete the online solution of calibration parameters, multi-modal interactive state management, and on-chip real-time parameter distribution, it is considered an equivalent alternative implementation of this invention. Correspondingly, the calibration interactive input and control modes of the system can be further expanded to include touchscreen gesture interaction, remote operation via a web page, serial / network data communication, mobile terminal App control, or camera closed-loop detection, etc., depending on the field environment. These different input methods are abstracted and converted into standard format calibration point coordinates through standardized dot matrix coordinate extraction logic at the underlying level. Then, they are seamlessly sent into the same least squares parameter solution link for centralized matrix operation. This achieves deep integration of multi-source control entry points in the underlying parameter calculation mechanism and status display logic, thereby giving the system powerful multi-entry unified interaction and iterative closed-loop debugging capabilities.

[0061] In terms of the underlying geometric correction algorithm model and resampling execution chain, the system also demonstrates strong scene adaptability. For different display scales, video resolution levels, and optical correction accuracy requirements, the geometric correction mapping model in this embodiment can be smoothly replaced or upgraded from a biquadratic inverse mapping model to a higher-order polynomial fitting model, a block grid interpolation model, a three-dimensional mesh lookup table (LUT) model, or a hybrid correction model combining global homography matrix transformation and local residual error correction, according to actual needs. Furthermore, the fixed-point quantization bit width of the parameters can be adaptively customized without changing the core process of interactive solution and real-time hardware correction. Regarding output display carriers and application terminals, this invention not only perfectly adapts to automotive head-up display (HUD) systems but can also be widely applied to transparent displays, large-screen projection screens, automotive smart instrument displays, and any other display media with optical path distortion requiring optical correction. The core value of this invention lies in acquiring calibration points through the acquisition unit, utilizing the online fitting of mapping coefficients on the PS side and performing high-precision fixed-point quantization, and then using a hardware update structure that separates the parameter shadow register and the working register to achieve unified step-by-step switching at the frame synchronization boundary of the vertical synchronization signal. This ultimately drives the PL side to perform high-speed pixel-level coordinate inverse mapping and bilinear interpolation resampling output. This closed-loop parameter tuning design, which highly integrates a dynamic decoupling mechanism between software and hardware and shared parameter state logic across multiple inputs into a single chip, eliminates the overhead of large-capacity pixel lookup table reconstruction. From the underlying mechanism, it ensures that during continuous online parameter tuning or dynamic automatic calibration in the field, there will be no mid-process flickering, frame asynchrony, or image tearing. It offers excellent on-site debugging convenience and broad application prospects.

[0062] It is evident that those skilled in the art can make various modifications and alterations to the embodiments of the present invention without departing from the spirit and scope of the invention. In this way, the invention is also intended to cover such modifications and alterations if they fall within the scope of the claims and their equivalents. The word "comprising" does not exclude the presence of other elements or steps not listed in the claims. The simple fact that certain measures are described in mutually different dependent claims does not indicate that a combination of these measures cannot be used for profit. Any reference numerals in the claims should not be considered as limiting the scope.

Claims

1. A method for interactive closed-loop calibration and real-time geometric correction of optical display images based on FPGA, characterized in that, include: S1: Obtain calibration interactive input data, which includes multiple sets of calibration points composed of target display coordinates and corresponding source image sampling coordinates; S2: On the processing system side, based on the multiple sets of calibration points, a biquadratic inverse mapping model is constructed, the biquadratic geometric correction coefficients are obtained, and the geometric correction coefficients are subjected to fixed-point quantization processing; S3: Write the quantized geometric correction coefficients into the shadow register located on the programmable logic side, and when the display frame synchronization boundary is detected, switch the geometric correction coefficients in the shadow register to the working register so that the geometric correction coefficients take effect from the next frame; S4: On the programmable logic side, according to the geometric correction coefficient in the working register, the target display pixel is subjected to inverse coordinate mapping to obtain the corresponding source image absolute sampling coordinates, and the source image pixels are read based on the source image absolute sampling coordinates for interpolation processing to output the geometrically corrected display image.

2. The FPGA-based interactive closed-loop calibration and real-time geometric correction method for optical display images, as described in claim 1, is characterized in that... The step of constructing the biquadratic inverse mapping model and solving for the biquadratic geometric correction coefficients in S2 specifically includes: converting the multiple sets of calibration points into centered coordinates with the display center as the origin, the centered coordinates of the target display point are denoted as (u,v), and the centered coordinates of the source image sampling points are denoted as (p,q); constructing the inverse mapping relationship from the target display coordinates to the source image sampling coordinates: p=A0+A1·u+A2·v+A3·u²+A4·v²+A5·u·v; q=B0+B1·u+B2·v+B3·u²+B4·v²+B5·u·v; where A0-A5 and B0-B5 are the 12 geometric correction coefficients to be solved.

3. The FPGA-based interactive closed-loop calibration and real-time geometric correction method for optical display images, as described in claim 2, is characterized in that... The geometric correction coefficients A0-A5 and B0-B5 are obtained by solving multiple sets of calibration points using the least squares method, specifically including: Collect N sets of calibration points, where N≥6; For the i-th calibration point, based on the centered coordinates (ui,vi) of its target display point, construct the corresponding basis function row vector Hi=[1,ui,vi,ui²,vi²,ui·vi], and combine the basis function row vectors of all N calibration points to form matrix H; The centered coordinates (pi, qi) of the source image sampling points corresponding to the N sets of calibration points are combined to form the horizontal centered coordinate vector P = [p1, p2, ..., pN] of the source image. T And the vertically centered coordinate vector Q=[q1,q2,...,qN] T ; Define the horizontal correction coefficient vector as A = [A0, A1, A2, A3, A4, A5]. T The vertical correction coefficient vector is B = [B0, B1, B2, B3, B4, B5]. T ; Solve using any one of the following methods: least squares, generalized inverse, QR decomposition, SVD, or equivalent numerical method, and obtain the solution using the formula A=(H). T ·H) -1 ·H T ·P and B=(H T ·H) -1 ·H T The geometric correction coefficients A0-A5 and B0-B5 are obtained by solving Q.

4. A method for interactive closed-loop calibration and real-time geometric correction of optical display images based on FPGA according to any one of claims 1-3, characterized in that, The step of performing fixed-point quantization on the geometric correction coefficients in S2 specifically includes: converting the obtained floating-point geometric correction coefficients into a signed fixed-point number format with a preset bit width; the signed fixed-point number format is Q16.16 format, where the high 16 bits are used to represent the signed integer part and the low 16 bits are used to represent the fractional part.

5. The FPGA-based interactive closed-loop calibration and real-time geometric correction method for optical display images, as described in claim 1, is characterized in that... The steps of performing coordinate inverse mapping and interpolation in S4 specifically include: if the absolute sampling coordinates of the source image obtained by inverse mapping exceed the effective range of the source image, a boundary clamping method is used to limit the range of sampling coordinates and avoid interpolation addressing out of bounds; extract the integer part and the fractional part of the absolute sampling coordinates of the source image, convert the fractional part into a weight within a preset range, and perform multiplication and addition operations on the weight and four pixels in the neighborhood of the integer part, followed by shift normalization, to obtain the output pixel.

6. The FPGA-based interactive closed-loop calibration and real-time geometric correction method for optical display images according to claim 5, characterized in that, The specific calculation method for the output pixels is as follows: Convert the horizontal fractional part α and the vertical fractional part β of the absolute sampling coordinates of the source image into 8-bit unsigned fractional weighted integers dx and dy, dx=round(α×255), dy=round(β×255); The four bilinear interpolation weights w00, w01, w10 and w11 corresponding to the target display pixel are calculated using dx and dy respectively. The calculation formulas are: w00=(255-dx)(255-dy), w01=dx(255-dy), w10=(255-dx)dy, w11=dx·dy; Extract four neighboring pixels P00, P01, P10, and P11 from the source image that are adjacent to the integer part (x0, y0), where P00 = I(x0, y0), P01 = I(x0+1, y0), P10 = I(x0, y0+1), and P11 = I(x0+1, y0+1), and I represents the color channel pixel value of the source image at the corresponding coordinates; The RGB three color channels of the four source image pixels P00, P01, P10, and P11 are multiplied and added respectively, and normalized according to the total weight scale to obtain the output pixel value Pout. The calculation formula is: Pout=(w00·P00+w01·P01+w10·P10+w11·P11) / 255².

7. The interactive closed-loop calibration and real-time geometric correction method for optical display images based on FPGA according to claim 1, characterized in that, The calibration interactive input data is obtained through a multimodal unified solution architecture. The multimodal unified solution architecture is configured to extract drag data from the web page, gesture data from the touch screen, or visual detection results from an external camera into calibration point coordinates in a unified format, and send them into the same parameter solution link for matrix operations.

8. An interactive closed-loop calibration and real-time geometric correction system for optical display images based on FPGA, characterized in that, include: A calibration interactive input unit is configured to acquire calibration interactive input data, wherein the calibration interactive input data includes multiple sets of calibration points composed of target display coordinates and corresponding source image sampling coordinates; The system-side parameter solving and control unit is configured to construct a bi-quadratic inverse mapping model based on the multiple sets of calibration points, solve for the bi-quadratic geometric correction coefficients, and perform fixed-point quantization processing on the geometric correction coefficients; The parameter register and synchronization update unit are configured with a shadow register and a working register. The shadow register is used to write the quantized geometric correction coefficients into the shadow register, and when the display frame synchronization boundary is detected, the geometric correction coefficients in the shadow register are switched to the working register. The programmable logic side real-time image geometric correction unit is configured to perform inverse coordinate mapping on the target display pixels according to the geometric correction coefficients in the working register, obtain the corresponding absolute sampling coordinates of the source image, read the source image pixels based on the absolute sampling coordinates of the source image, perform interpolation processing, and output the geometrically corrected display image.

9. The FPGA-based interactive closed-loop calibration and real-time geometric correction system for optical display images according to claim 8, characterized in that, The parameter solving and control unit on the processing system side is connected to the parameter register and synchronization update unit via an AXI-Lite on-chip bus; while the parameter register and synchronization update unit writes the geometric correction coefficients into the shadow register, the current display frame continuously calls the original coefficients in the working register to maintain the image rendering.

10. The FPGA-based interactive closed-loop calibration and real-time geometric correction system for optical display images according to claim 8, characterized in that, It also includes a display feedback unit, which is connected to the real-time image geometry correction unit on the programmable logic side. The display feedback unit is configured to output the corrected image and simultaneously overlay calibration points, grid lines or error feedback information to form an online debugging closed loop with the calibration interactive input unit.

Images