Error handling in a geometric correction engine

Abstract

A method for error handling in a geometry correction engine (GCE) is provided, the method comprising receiving (800), by the GCE, configuration parameters, generating (802), by the GCE, an output block of an output frame based on a corresponding block of an input frame in accordance with the configuration parameters, detecting (804), by the GCE, a runtime error during the generating, and reporting (806), by the GCE, an event corresponding to the runtime error.

Description

Background Technology

[0001] Cameras with wide-angle lenses, such as fisheye cameras, are being used in a growing number of applications, including surveillance, robot vision, automotive rearview imaging systems, and automotive surround-view imaging systems, to provide a 180+° field of view. While such lenses offer a very large wide-angle view, the resulting image can be severely distorted when a hemispherical scene is projected onto a flat surface. Therefore, images captured by such cameras are corrected to approximate straight lines before being presented to the viewer. In real-time applications, a geometric correction engine (GCE) can be included in a video processing accelerator within a system-on-a-chip (SoC) to perform geometric image transformations on images from such wide-angle lenses to correct distortion. The GCE can also perform image perspective transformations and stereo correction. Summary of the Invention

[0002] Embodiments of this disclosure relate to error handling in a geometry correction engine (GCE). In one aspect, a method for error handling in a geometry correction engine (GCE) is provided, the method comprising receiving configuration parameters by the GCE, generating an output block of an output frame based on corresponding blocks of an input frame according to the configuration parameters by the GCE, detecting runtime errors by the GCE during generation, and reporting an event corresponding to the runtime error by the GCE.

[0003] In one aspect, a system-on-a-chip (SoC) is provided, the SoC including a geometry correction engine (GCE), memory, and at least one processor. The geometry correction engine is configured to receive configuration parameters from a user-provided control program, generate an output block of an output frame based on a corresponding input block of an input frame according to the configuration parameters, detect runtime errors while generating the output block, and report events corresponding to runtime errors to the user-provided control program. The memory is configured to store software instructions implementing the user-provided control program for the GCE, and at least one processor is coupled to the memory to execute the software instructions.

[0004] In one aspect, a geometric correction engine (GCE) is provided, comprising a transformation component and a reverse mapping component. The transformation component is configured to apply a perspective transformation to the angular coordinates of an output block of an output frame to generate first perspective distortion coordinates, calculate the angular coordinates of a mesh LUT block in a mesh lookup table (LUT) using the first perspective distortion coordinates, wherein the mesh LUT block corresponds to the output block, and detect and report a first runtime error when the size of the mesh LUT block is greater than the internal storage allocated to the mesh LUT block. The reverse mapping component is configured to reverse map the first perspective distortion coordinates to coordinates in an input frame using the mesh LUT block, calculate the bounding box of an input frame block corresponding to the output block based on the coordinates in the input frame, and detect and report a second runtime error when the size of the input frame block is greater than the internal storage of the GCE allocated to the input frame block. Attached Figure Description

[0005] Figure 1 illustrates the general operation of the geometry correction engine (GEC);

[0006] Figures 2A-2C This is an example of artifacts in the output image when a runtime error occurs in GCE;

[0007] Figure 3 This is a block diagram illustrating an example of a video processing accelerator (VPAC) incorporated into GCE;

[0008] Figure 4 This is a block diagram of an example GCE;

[0009] Figure 5 This is an example illustrating how the output frame is divided into 9 regions, each with a different block size.

[0010] Figure 6 It is a diagram. Figure 4 Examples of GCE operations;

[0011] Figure 7 This is an example of a boundary block used to acquire grid data;

[0012] Figure 8 This is a flowchart of the error handling method used in GCE; and

[0013] Figure 9 This is a high-level block diagram of an example multiprocessor system-on-a-chip (SoC). Detailed Implementation

[0014] This document describes specific embodiments of the present disclosure in detail with reference to the accompanying drawings. For consistency, the same elements in the various figures are indicated by the same reference numerals.

[0015] Figure 1 illustrates the general operation of a geometric correction engine (GEC) 104 according to an embodiment of the present disclosure. To correct a distorted image, the GEC 104 uses a technique called "reverse mapping," where pixel coordinates in the distorted input image are determined as a function of pixel coordinates in the undistorted output image. Correction involves reverse mapping each output pixel location to a pixel location in the input distorted image.

[0016] The GCE 104 operates on the input image block by block. The output frame buffer 102 is divided into blocks, and each output block is back-mapped by the GCE 104 to the input frame block in the input frame buffer 101 using a grid lookup table (LUT) 107, which provides a mapping from pixel locations in the output image to corresponding locations in the input image. For example, the GCE 104 performs perspective warping on the coordinates of the four corners of the output block 103 and uses the warped coordinates to locate the corresponding block 106 in the grid table 107. The GCE 104 then calculates the coordinates of the corners of the input block 105 used to generate the output block 103 by applying the back-mapping to the perspective warp angle pixel coordinates of the output block 103 using the grid table block 106. The input block 105 is retrieved from the input frame buffer 101, and the GCE 104 uses the input image block 105 to generate the output image block 103.

[0017] GCEs such as GCE 104 have several configuration parameters that users can determine offline through simulation. Due to limited internal memory, limited precision of integer operations in the GCE, and / or unsafe or aggressive configuration parameters provided by the user, runtime errors may occur in the GCE. When runtime errors occur, the GCE may produce artifacts in the output image. For example, as... Figure 2A As shown, the limited precision of integer arithmetic can affect the calculation of the coordinates of the desired block in the input image, causing some required pixels to be missing from the acquired input image block. In another example, such as Figure 2B As shown, aggressive parameter settings by the user result in vertical artifacts in the output image. In another example, such as... Figure 2C As shown, the size of the output block, set by user parameters, exceeds the size of the internal memory, resulting in horizontal artifacts in the output image.

[0018] Users prefer to use more aggressive GCE parameters to save memory bandwidth without worrying about runtime output image artifacts. Furthermore, users want to be able to change GCE parameters in real time and quickly determine whether parameter changes are safe; for example, allowing drivers to dynamically change the viewpoint in a vehicle surround-view system or providing real-time calibration of cameras in the system. Existing implementations of GCE do not provide elegant error handling and concealment of error effects, i.e., artifact hiding.

[0019] Embodiments of this disclosure provide for reporting runtime errors in the GCE due to issues such as aggressive parameter values, dynamic changes in parameter values ​​due to view changes (e.g., by the driver), and / or real-time camera calibration, as well as the automatic hiding of artifacts caused by some runtime errors. In some embodiments, the artifacts causing the errors are signaled to a user-specified control program via an interrupt, which allows the control program to modify GCE parameters in real time to avoid future errors. Furthermore, the GCE can hide visual artifacts in the output image caused by some errors by approximating the lost pixel data with the nearest available neighboring pixel data.

[0020] Figure 3 This is a block diagram illustrating an example of a video processing accelerator (VPAC) 300 incorporating a GCE 303 according to some embodiments. The VPAC 300, which can be included in a system-on-chip (SoC), includes a GCE 303, a scalar component 308, a noise filter (NF) component 310, two vision imaging subsystems (VISS) 309, a hardware thread scheduler 302, a configuration component 301, a shared memory interconnect 304, local shared memory 305, a direct memory access (DMA) port 306, and a master port 307.

[0021] As described herein, the GCE 303 reads the input image via the main port 307 and performs geometric corrections (e.g., geometric distortion correction), perspective transformations, and / or rotational video stabilization on the image to generate an output image. The output of the GCE 303 can be sent to external memory or other hardware components, such as the scalar component 308 or the noise filter component 310, for further preprocessing via local shared memory 305.

[0022] The scalar unit 308 reads data from the shared memory 304 and generates scaled outputs with various scaling ratios. The output of the scalar unit is stored in the local shared memory 305 and can be filtered by the noise filter unit 310 or written to external memory.

[0023] Noise filter unit 310 performs bilateral filtering to remove noise. The output of the noise filter unit can be sent from local shared memory 305 to external memory, or it can be further sized by scalar unit 308. Local shared memory 305 is used for exchanging data across hardware units (e.g., GCE 303, scalar unit 308, and noise filter unit 310) and to DMA port 306.

[0024] The visual imaging subsystem 309 performs raw data image processing, such as defect pixel correction, lens shading correction, global / local brightness and contrast enhancement, de-mosaicing, and color conversion.

[0025] The hardware thread scheduler 302 is used for communication between the various hardware components of the VPAC 300. The configuration manager 301 is used by external configuration software executing on the processor in the SoC to set parameter values ​​for the various hardware components. The VPAC 300 may also include typical hardware inputs such as clock, reset, network, data, and debug signals, as well as hardware outputs such as interrupt signals.

[0026] Figure 4 This is an example GCE 400 according to some embodiments (e.g. Figure 3 The block diagram of GCE 303 is shown in the references provided in this article. Figure 6 To explain in more detail, the GCE 400 is configured to report runtime errors that may occur at various points in the GCE process. Furthermore, for some runtime errors, the GCE 400 is configured to hide artifacts in the output frames caused by these errors.

[0027] The GCE 400 performs geometric correction operations on blocks of the input frame to generate corresponding blocks of the output frame. More specifically, the output frame can be generated using a single user-specified output block size for the entire output frame, or it can be divided into up to nine regions via user-specified configuration parameters, with each region having a region-specific output block size also specified by the user. Figure 5 This illustration shows an example of dividing an output frame into nine regions, each with a different block size. A user-provided control program, executing on the processor of the system-on-a-chip incorporated into the GCE 400, provides configuration parameter values ​​for specifying the number of regions, region sizes, and the output block size in each region. For example, additional information regarding dividing an output frame into multiple regions and output block sizes for GCE processing can be found in U.S. Patent Publication No. 2019 / 0096041, filed September 25, 2017, which is incorporated herein by reference. The GCE 400 operates on each region in a raster scan sequence to generate a geometrically corrected output block from the corresponding block in the input frame.

[0028] The GCE 400 includes a counter unit 402, a transform unit 404, a reverse mapping unit 406, a grid lookup table (LUT) 408, a multiplexing unit 415 configured to select whether to use the output of the transform unit 404 or the reverse mapping unit 406, a buffer 409, an interpolation unit 410, a frame buffer interface 411 coupled to external memory 401 that incorporates the input frame, and a shared memory interface 414 coupled to external shared memory 413 that stores the geometrically corrected output frame. A brief description of these components is provided herein. Additional details regarding the operation of examples of such components can be found in previously cited U.S. Patent Publication No. 2019 / 0096041.

[0029] The counter unit 402 is configured to calculate the coordinates 403 of the pixel position in the output block of the output frame of the area being processed. The coordinates are calculated based on the size of the output block in the user-specified area, the user-specified area size, and the position of the output block in the raster scan sequence of the area.

[0030] Transformation unit 404 and inverse mapping unit 406 are configured to generate coordinates 407 of the displacement pixels in the input frame, which correspond to the coordinates in the output block calculated by counter block 402. Transformation unit 404 is configured to apply affine and / or perspective transformations to the output block pixel coordinates to generate perspective-distorted output block coordinates 405 if enabled by a user-provided control program. If the application of transformations is not enabled, transformation unit block 404 outputs the unmodified coordinates of the output block generated by the counter unit.

[0031] The reverse mapping unit 406 is configured to generate coordinates 407 of the displacement pixels in the input frame corresponding to the output block coordinates received from the transformation unit 404. The corresponding coordinates in the input image can be calculated by combining the received output coordinates with offsets from a user-provided grid lookup table (LUT) 408 positioned based on the received output block coordinates. The grid LUT 408, also known as an offset table, is a regular grid that maps the positions of pixels in the output frame to their corresponding positions in the input frame and is provided by the user. The grid LUT 408 defines an offset vector (Δx, Δy) for the position in the output frame. The grid can be fully sampled or downsampled. A fully sampled grid defines an offset vector for each output pixel, thus accurately defining where the input data is taken to calculate the output pixel. If the grid is downsampled, bilinear interpolation is used to interpolate the missing offset vectors.

[0032] Buffer 409 is configured to receive input pixels retrieved from the frame buffer via frame buffer interface 411 and store them in external memory 401. If inverse mapping is enabled, the input pixels correspond to pixel coordinates generated by inverse mapping unit 406; if inverse mapping is not enabled, the input pixels correspond to pixel coordinates output by transformation unit 404.

[0033] Interpolation unit 410 is configured to interpolate the input pixel values ​​received from buffer unit 409 using bicubic or bilinear interpolation specified by a user-provided configuration program to generate the output pixel value, i.e., the output pixel value at the coordinates generated by counter unit 402. After interpolation, the geometrically corrected output block is stored in the output frame buffer in SL2 shared memory 413 via SL2 interface 414.

[0034] Figure 6This is an example illustrating the operation of GCE 400. For simplicity, this example assumes that perspective transformation and inverse mapping are enabled and the mesh LUT is downsampled. For the output block in the region of the output frame, counter unit 402 calculates the coordinates of the four corners of the output block, and transformation unit 404 applies perspective transformation 600 to these coordinates to generate perspective distortion coordinates for inverse mapping unit 406.

[0035] Transformation component 404 uses perspective warp coordinates to calculate the angular coordinates in the mesh LUT 408 of the mesh LUT block 618 corresponding to the output block 602. More specifically, transformation component 404 calculates the rectangular bounding box 740 for the mesh data (e.g., Figure 7 (as shown in the example) to obtain, for example, a mesh LUT block 618, such that the bounding box 740 contains mesh points corresponding to four perspective distortion angle coordinates, such as angle 711. Figure 7 The bounding box 730 of the output block 720, showing the perspective warp coordinates of the corners of the example output block 720 and the given output block 720, is displayed. The reverse mapping component 406 uses bilinear interpolation to upsample the downsampled grid points 701, 702, 703, 704 to pixel coordinate precision. For the grid point bilinear interpolation, the perspective warped bounding box 730 expands one column to the right and one row at the bottom, forming a rectangular bounding box 740.

[0036] If the size of the mesh LUT block 618 exceeds the internal storage allocated for the mesh LUT block in buffer 409 and / or any boundary of the block exceeds the hardware address size, the transform unit 404 causes a mesh block overflow event to be reported. The transform unit 404 will cause the event to be reported once for each mesh block where the error occurs, along with the top-left corner coordinates of the corresponding output block. This event may be referred to herein as mesh_iblk_memovf. This runtime error condition may result in visible corruption of one or more output blocks. For example, this error may occur due to an excessively large output block size, incorrect parameter values ​​for the affine or perspective transform, or an excessively small downsampling factor for the mesh LUT.

[0037] Refer again Figure 6The GCE 400 acquires 604 a mesh LUT block 618 corresponding to the mesh table boundary block calculated by the transform unit 404, and stores the mesh LUT block 618 in buffer 409. The inverse mapping unit 406 then uses the mesh LUT block 618 to inverse map 606 the perspective distortion angle of the output block calculated by the transform unit 404 to coordinates in the input frame, and uses the inverse-mapped angular coordinates to calculate 608 the bounding box of the input frame block corresponding to the output block. The inverse mapping unit 406 applies additional padding, such as one or more additional pixel rows and columns, around the inverse-mapped coordinates to calculate the final coordinates of the input bounding box. The amount and location of the padding depend on the type of interpolation to be performed by the interpolation unit 410. Additional user-defined padding can also be applied in all directions to calculate the final bounding box. The GCE 400 acquires 610 an input frame block 612 identified by the calculated input boundary block and stores the input frame block 612 in buffer 409.

[0038] If the size of the input frame block 612 exceeds the hardware address size of the internal storage allocated for the input frame block in buffer 409 and / or any block boundaries, the reverse mapping unit 406 causes a pixel block overflow event to be reported. The reverse mapping unit 406 will cause this event to be reported once for each input block where the error occurs, along with the top-left corner coordinate of the corresponding output block. This event may be referred to herein as pix_iblk_memovf. This runtime error condition may result in visible corruption of one or more output blocks. For example, this error may occur due to an excessively large output block size.

[0039] Once the input data required to generate the output block (i.e., mesh LUT block 618 and input frame block 612) is available, the GCE 400 performs block processing to generate the pixel values ​​of the output block from the corresponding geometrically corrected pixels of the input frame block 612. The counter unit 402 calculates the coordinates of each pixel position in the output block in raster scan order and provides the coordinates to the transform unit 404.

[0040] Transformer 404 applies perspective transformation 614 to these coordinates to generate perspective distortion coordinates for inverse mapping unit 406. If the value of an intermediate variable in the affine or perspective transformation operation exceeds a value that the GCE hardware can support, transformer 404 causes an internal precision error event to be reported the first time the error occurs in each frame. For example, this error may occur because the value of the user-specified transformation coefficient is incorrect. When such an error occurs, transformer 404 clips the intermediate variable value to a size supported by the hardware. This event may be referred to herein as int_szovf.

[0041] Furthermore, if the perspective warp coordinates are outside the boundaries of the mesh LUT, i.e., outside the frame size of the mesh data available in the mesh LUT, the transformation unit 404 clips the coordinates 616 to the boundaries of the mesh LUT. Additionally, the transformation unit 404 causes a frame out-of-bounds event to be reported the first time this error occurs in each frame. This event may be referred to herein as ifr_outofbound.

[0042] Furthermore, if the perspective warp coordinates are outside the boundary of the mesh LUT block 618, the transform component 404 clips the out-of-bounds coordinates 616 to the boundary of the mesh LUT block 618. Additionally, the transform component 404 causes a mesh block out-of-bounds event to be reported the first time this error occurs in each frame. This event may be referred to herein as mesh_iblk_outofbound.

[0043] The reverse mapping unit 406 uses a mesh LUT block 618 to reverse map 620 the perspective distortion coordinates of the output pixel position calculated by the transformation unit 404 to coordinates in the input frame block 612. Furthermore, if the input frame block coordinates are outside the boundaries of the input frame, the reverse mapping unit 406 clips the coordinates 622 to the boundaries of the input frame. Additionally, the reverse mapping unit 406 causes a frame out-of-bounds event to be reported the first time this error occurs in each frame. This event may be referred to herein as ifr_outofbound.

[0044] Furthermore, if the input block coordinates are outside the boundary of the input frame block 612, the p-mapping unit 406 clips the out-of-bounds coordinates 622 to the boundary of the input frame block 612. Clipping the out-of-bounds coordinates has the effect of hiding any artifacts in the output block that would occur in some way due to this error. Additionally, the inverse mapping unit 406 causes the input block out-of-bounds event, along with the corresponding top-left corner coordinates of the output block, to be reported the first time this error occurs in each frame. This event may be referred to herein as pix_iblk_outofbound.

[0045] The pixel value to be stored at the output block coordinates is then interpolated 624 by the interpolation unit 410 based on the coordinates provided by the reverse mapping unit. For example, if bicubic interpolation is selected, the output pixel value is interpolated based on 16 pixel values ​​in a 4x4 square around the position indicated by the input block coordinates. If bilinear interpolation is selected, the output pixel value is interpolated based on four pixel values ​​in a 2x2 square around the position indicated by the input block coordinates.

[0046] Figure 8 It is according to some embodiments for use in GCE (e.g., Figure 4The flowchart illustrates the error handling method in the GCE (400). Initially, the GCE receives configuration parameters (800) from a user-provided control program to perform geometric corrections to generate an output frame. Examples of such configuration parameters have been described previously. The GCE then generates an output block of the output frame based on the input block of the input frame according to the configuration parameters. The generation of the output block has been described previously. During the generation of the output block, the GCE can detect runtime errors (804), as previously described herein. When a runtime error is detected, the GCE reports an event (806) corresponding to the runtime error to the user-provided control program. Depending on the specific runtime error, the GCE can also perform corrective actions adapted to the error described previously herein, such as cropping coordinates to boundaries to hide potential artifacts in the output image that may be caused by the runtime error. In response to a specific runtime error, the user-provided control program can modify one or more configuration parameters, and the GCE will receive the modified configuration parameters (808) for generating another output frame.

[0047] Figure 9 This is a high-level block diagram of an example multiprocessor system-on-a-chip (SoC) 900 that can be configured to perform error handling as described in the GCE herein. Specifically, the example SoC 900 is an embodiment of the TDA4VM SoC available from Texas Instruments, Inc. A high-level description of the components of the SoC 900 is provided herein. A more detailed description of the example components can be found on pages 1-311 of the February 2019 issue of Texas Instruments, Inc.'s SPRSP36E, revised December 2019 edition. TM The content of this paper is incorporated herein by reference from “Automotive Processors for ADAS and Autonomous Vehicles Silicon Revision 1.0”.

[0048] The SoC 900 includes numerous subsystems across different domains, such as a dual-core 64-bit... -A72 microprocessor subsystem 904, based on two dual-core The R5F MCU's microcontroller unit (MCU) island 906 has four additional dual-core cores in the main domain. The SoC 900 comprises an R5FMCU 912, two C66x floating-point digital signal processors (DSPs) 908, a C71x floating-point vector DSP 910 including a deep learning matrix multiplication accelerator (MMA), and a 3D graphics processing unit (GPU) 913. The SoC 900 also includes a memory subsystem 914, which includes up to 8MB of on-chip static random access memory (SRAM), an internal DMA engine, a general-purpose memory controller (GPMC), and an external memory interface (EMIF) module (EMIF). Furthermore, the SoC 900 includes a capture subsystem 916 with two camera stream interfaces, a vision processing accelerator (VPAC) 902 including GCE, a depth and motion processing accelerator (DMPAC) 918, and a video acceleration module 920. The SoC 900 also includes a display subsystem 922, an Ethernet subsystem 924, a navigation subsystem 926, various security accelerators 928, system service support 930, and various other interfaces 932.

[0049] The software instructions for implementing the user-provided control program as described herein can be stored in the memory subsystem 914 (e.g., a computer-readable medium) and can be executed on one or more programmable processors (e.g., DSP910) of the SOC 900. Furthermore, the GCE in the VPAC 902 can be, for example, a... Figure 4 An example of the GCE 400.

[0050] Other embodiments

[0051] Although this disclosure has been described with respect to a limited number of embodiments, those skilled in the art who benefit from this disclosure will understand that other embodiments can be devised without departing from the scope of this disclosure.

[0052] For example, this document has described an embodiment in which the angular coordinates of the output blocks where errors occur are reported for some error events. In other embodiments, boundary blocks covering all output blocks in the region corresponding to the error event can be reported for some error events (e.g., mesh_iblk_memovf, pix_iblk_memovf, int_szovf, pix_iblk_outofbound, and ifr_outofbound), i.e., triggering a runtime error corresponding to the error event. In some such embodiments, options are provided such that the user program can choose to report boundary blocks for mesh_iblk_memovf, pix_iblk_memovf, and int_szovf, or all five events. In all such embodiments, after the GCE completes image processing, the user-provided control program can inspect such bounding boxes. The user-provided control program can choose to have the GCE re-execute on one or more regions identified by one or more bounding boxes with different parameters, in which case the GCE output of one or more regions will replace the output of the original GCE execution. The user-provided control program can also choose to modify the configuration parameters of the GCE to process the next image, rather than attempting to change the GCE output of the current image.

[0053] Therefore, it is intended that the appended claims will cover any such modifications to embodiments that fall within the true scope of this disclosure.

Claims

1. A method for error handling in a geometry correction engine (GCE), the method comprising: The configuration parameters are received by the GCE; The GCE generates the output block of the output frame based on the corresponding block of the input frame according to the configuration parameters; The GCE detects whether a runtime error occurred during the generation of the output block; Based on the runtime error, the GCE corrects the value that has exceeded the boundary, so that the corrected value is at the boundary; as well as The GCE reports an event corresponding to the runtime error.

2. The method of claim 1, wherein the runtime error can cause artifacts in the output frame.

3. The method of claim 1, wherein the report further includes reporting the top-left corner coordinates of the output block corresponding to the event.

4. The method of claim 1, wherein the report further includes reporting bounding boxes of all output blocks covering the region of the output frame that caused the runtime error.

5. The method of claim 4, further comprising receiving one or more modification parameters by the GCE, wherein the GCE generates a new output block for the output block in the bounding box based on the one or more modification parameters.

6. The method according to claim 1, wherein: The runtime error is the first runtime error; Generating the output block includes applying a perspective transformation to the angular coordinates of the output block to generate perspective distortion angular coordinates, and calculating the corresponding mesh lookup table block, i.e., the mesh LUT block, based on the perspective distortion angular coordinates; and The method further includes detecting whether a second runtime error has occurred, the second runtime error being associated with a grid LUT block size greater than the internal storage of the GCE allocated to the grid LUT block.

7. The method according to claim 6, wherein: Generating the output block includes using the mesh LUT block to reverse-map the perspective distortion angle coordinates to coordinates in the input frame, and calculating the bounding box of the input frame block corresponding to the output block based on the coordinates in the input frame; and The method further includes detecting whether a third runtime error has occurred, the third runtime error being associated with the input frame block being larger than the internal storage of the GCE allocated to the input frame block.

8. The method according to claim 1, wherein: Generating the output block includes applying a perspective transformation to the coordinates of the pixel positions in the output block to generate perspective distortion coordinates; The runtime error is associated with the value of the intermediate variable of the perspective transformation exceeding the size supported by the GCE; and Correcting the value involves setting the value of the intermediate variable to the size supported by the GCE.

9. The method according to claim 1, wherein Generating the output block includes applying a perspective transformation to the coordinates of the pixel positions in the output block to generate perspective distortion coordinates; The runtime error is associated with the perspective distortion coordinates being outside the output frame size of the mesh data available in the mesh lookup table, i.e., the mesh LUT, indicated by one of the configuration parameters; and Correcting the value involves setting the perspective distortion coordinates to the boundary of the mesh LUT.

10. The method according to claim 9, wherein: The runtime error is the first runtime error; The method further includes detecting whether a second runtime error has occurred, the second runtime error being associated with the perspective warp coordinates being outside the boundary of the mesh LUT block corresponding to the output block; and Based on the second runtime error, the perspective distortion coordinates are set to the boundary of the mesh LUT block.

11. The method of claim 10, wherein: Generating the output block includes using the mesh LUT block to reverse-map the perspective distortion coordinates to coordinates in the input frame block corresponding to the output block; and The method further includes detecting whether a third runtime error has occurred, the third runtime error being associated with the input frame block coordinates being outside the boundary of the input frame; and Based on the third runtime error, the input frame block coordinates are set to the boundary of the input frame.

12. The method of claim 11, wherein the method further comprises: Detect whether a fourth runtime error has occurred, the fourth runtime error being associated with the input frame block coordinates being outside the boundary of the input frame block corresponding to the output block; and Based on the fourth runtime error, the input frame block coordinates are set to the boundary of the input frame block.

13. A system-on-a-chip (SoC) comprising: The geometry correction engine, or GCE, is configured as follows: Receive configuration parameters from the user-provided control program; Based on the configuration parameters, an output block of the output frame is generated based on the corresponding input block of the input frame; Detect whether a runtime error associated with the generation of the output block has occurred; Based on the runtime error, the GCE corrects the value that has exceeded the boundary by setting the value to the boundary; as well as The control program provided to the user reports the event corresponding to the runtime error; A memory configured to store software instructions for implementing the user-provided control program for the GCE; as well as At least one processor coupled to the memory to execute the software instructions.

14. The SoC of claim 13, wherein the runtime error can cause artifacts in the output frame.

15. The SoC of claim 13, wherein the GCE is further configured to report the top-left corner coordinates of the output block corresponding to the event.

16. The SoC of claim 13, wherein the GCE is further configured to report the bounding boxes of all output blocks covering the region of the output frame that causes the runtime error.

17. The SoC of claim 16, wherein the GCE is further configured to receive one or more modification parameters, wherein the GCE generates a new output block for the output block in the bounding box based on the one or more modification parameters.

18. The SoC of claim 13, wherein: The runtime error is a first runtime error; and The GCE is also configured to: The output block is generated by applying perspective transformation to the angular coordinates of the output block to generate perspective distortion angular coordinates, and then calculating the corresponding mesh lookup table block (i.e., mesh LUT block) based on these perspective distortion angular coordinates. Detect whether a second runtime error has occurred, which is associated with the size of the grid LUT block being greater than the internal storage of the GCE allocated to the grid LUT block.

19. The SoC of claim 18, wherein the GCE is further configured to: The output block is generated by back-mapping the perspective distortion angle coordinates to coordinates in the input frame using the mesh LUT block and calculating the bounding box of the input frame block corresponding to the output block based on the coordinates in the input frame; and Detect whether a third runtime error has occurred, which is associated with the size of the input frame block being greater than the internal storage of the GCE allocated to the input frame block.

20. The SoC of claim 13, wherein: The GCE is also configured to generate the output block by applying a perspective transformation to the coordinates of the pixel positions in the output block to generate perspective distortion coordinates; The runtime error is associated with the value of the intermediate variable of the perspective transformation exceeding the size supported by the GCE; and Correcting the value involves setting the value of the intermediate variable to the size supported by the GCE.

21. The SoC of claim 13, wherein: The GCE is also configured to generate the output block by applying a perspective transformation to the coordinates of the pixel positions in the output block to generate perspective distortion coordinates; The runtime error is associated with the perspective distortion coordinates being outside the output frame size of the mesh data available in the mesh lookup table, i.e., the mesh LUT, indicated by one of the configuration parameters; and Correcting the value involves setting the perspective distortion coordinates to the boundary of the mesh LUT.

22. The SoC of claim 21, wherein: The runtime error is a first runtime error; and The GCE is also configured to: Detect whether a second runtime error has occurred, which is associated with the perspective distortion coordinates being outside the boundary of the mesh LUT block corresponding to the output block; and Based on the second runtime error, the perspective distortion coordinates are set to the boundary of the mesh LUT block.

23. The SoC of claim 22, wherein the GCE is further configured to: The output block is generated by back-mapping the perspective distortion coordinates to the coordinates in the input frame block corresponding to the output block using the mesh LUT block; Detect whether a third runtime error has occurred, the third runtime error being associated with the input frame block coordinates being outside the boundary of the input frame; and Based on the third runtime error, the input frame block coordinates are set to the boundary of the input frame.

24. The SoC of claim 23, wherein the GCE is further configured to: Detect whether a fourth runtime error has occurred, the fourth runtime error being associated with the input frame block coordinates being outside the boundary of the input frame block corresponding to the output block; and Based on the fourth runtime error, the input frame block coordinates are set to the boundary of the input frame block.

25. A geometry correction engine, or GCE, comprising: The conversion unit is configured as follows: Apply the perspective transformation to the angular coordinates of the output block of the output frame to generate the first perspective distortion coordinates; The first perspective distortion coordinates are used to calculate the angular coordinates of the grid LUT block in the grid lookup table, i.e., the grid LUT block, which corresponds to the output block; as well as When the size of the grid LUT block is greater than the internal storage allocated to the grid LUT block, a first runtime error is detected and reported. as well as The reverse mapping component is configured as follows: The first perspective distortion coordinates are reverse-mapped to the coordinates in the input frame using the mesh LUT block; Calculate the bounding box of the input frame block corresponding to the output block based on the coordinates in the input frame; as well as When the size of the input frame block is greater than the internal storage of the GCE allocated to the input frame block, a second runtime error is detected and reported.

26. The GCE according to claim 25, wherein: The transformation unit is further configured to: The perspective transformation is applied to the coordinates of the pixel positions in the output block to generate second perspective distortion coordinates; When the value of the intermediate variable of the perspective transformation exceeds the size supported by the GCE, a third runtime error is detected and reported, wherein the transformation component prunes the value of the intermediate variable to the size supported by the GCE; as well as A fourth runtime error is detected and reported when the second perspective distortion coordinates are outside the output frame size of the available mesh data in the mesh LUT, wherein the transformation component clips the second perspective distortion coordinates to the boundary of the mesh LUT; and The reverse mapping component is also configured to: The second perspective distortion coordinates are reverse-mapped to the coordinates in the input frame block using the mesh LUT block; A fifth runtime error is detected and reported when the input frame block coordinates are outside the boundary of the input frame, wherein the reverse mapping component clips the input frame block coordinates to the boundary of the input frame; as well as A sixth runtime error is detected and reported when the input frame block coordinates are outside the boundary of the input frame block, wherein the reverse mapping component clips the input frame block coordinates to the boundary of the input frame block.

27. The GCE of claim 26, further comprising an interpolation component configured to interpolate the value of the pixel position in the output block based on the input frame block coordinates.

28. The GCE of claim 26, wherein the reverse mapping component is further configured to detect and report a seventh runtime error when the second perspective distortion coordinates are outside the boundary of the mesh LUT block, wherein the transformation component clips the perspective distortion coordinates to the boundary of the mesh LUT block.

Images