Video image real-time rotation method and system based on FPGA

Abstract

The invention discloses a video image real-time rotation method and system based on an FPGA (Field Programmable Gate Array), belongs to an image processing technology, and is widely applied to the fields of medical image analysis, aircraft image racemization and the like. The main process comprises the following steps of: firstly, reading an input source image and carrying out forward rotation mapping on each source pixel in sequence; then, traversing and determining alternative target pixel coordinates contained in the forward mapping coordinate range of the source image; then, reverse rotation mapping is carried out on alternative target pixel coordinates, effective reverse mapping in a source image coordinate range is carried out, and color values of target pixels are calculated by using a bilinear interpolation method; finally, these color values are saved and a final output image is generated. In the whole process, forward mapping and reverse mapping are combined, and the high-quality real-time rotation effect is ensured.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to a real-time video image rotation method and system based on FPGA. Background Art

[0002] Image rotation is an image processing technology that is widely used in medical image analysis, aircraft image derotation and other fields.

[0003] Image rotation methods mainly include forward mapping and reverse mapping. The forward mapping method fills the pixels in the source image into the target image based on the mapping relationship. This method directly forward calculates the target pixel coordinates and color from the source image pixel coordinates and color, and has good real-time performance; however, forward mapping will cause some target pixels to have no mapping relationship with the source pixels, and color information cannot be obtained, resulting in holes (defects). Due to the poor image quality after rotation, the forward mapping method is rarely used in practical applications. The reverse mapping method reversely maps the target pixel coordinates to the source image, reads the color of the adjacent coordinate points in the source image that wrap the reverse mapping coordinates, and obtains the target pixel color through linear interpolation. This method can obtain a good rotated image, but it needs to cache a frame of the image before rotation, and query the source image pixel color information adjacent to the reverse mapping coordinates in the source image cache during the rotation calculation process. Affected by the query delay, the real-time performance is not good. Summary of the invention

[0004] In order to solve the technical problems existing in the background technology, the present invention provides a real-time rotation method and system for video images based on FPGA, which can realize real-time rotation and scaling transformation of video images with a large resolution of more than 1920x1080@60Hz.

[0005] In order to solve the technical problem, the technical solution of the present invention is:

[0006] A real-time rotation method for video images based on FPGA, the method comprising: performing forward rotation mapping on an input image, traversing candidate target pixels in a forward mapping area; then, performing reverse mapping on the candidate target pixels, and screening out valid reverse mappings located in a source pixel area; finally, for valid reverse mappings, using a linear interpolation algorithm to calculate target pixel values, and updating an output image, and achieving a high-quality real-time rotation effect by combining forward and reverse mappings.

[0007] Further, the forward rotation mapping of the input image specifically includes:

[0008] Parse the input video image frame to generate the source pixel coordinates (i, j) and RGB color (R s ,G s ,B s), converting the source pixel corner coordinates (i, j) into coordinates (x s ,y s ):

[0009] x s =iW / 2

[0010] y s =jH / 2

[0011] Where W and H are the image width and height respectively;

[0012] Calculate the source pixel (x s ,y s ) The center coordinate after positive rotation (X′ t [i,j],Y′ t [i,j]):

[0013] X′ t [i,j]=α(x s cosθ-y s sinθ)

[0014] Y′ t [i,j]=α(x s sinθ+y s cosθ)

[0015] Among them, θ is the rotation angle and α is the scaling ratio;

[0016] Use the source pixel angle coordinates (i, j) as the address, cache the forward rotation center coordinates and source RGB information (X′ t [i,j],Y′ t [i,j],R s ,G s ,B s ).

[0017] Further, the valid candidate target pixels in the forward mapping area are traversed, specifically including:

[0018] Sequentially read the forward rotation center coordinates (X′) stored in the four adjacent cells with addresses (i, j), (i+1, j), (i, j+1), and (i+1, j+1) in the cache. t [i,j],Y′ t [i,j])、(X′ t [i+1,j],Y′ t [i+1,j]),(X′ t [i,j+1],Y′ t [i,j+1])、(X′ t [i+1,j+1],Y′ t[i+1,j+1]), calculate the minimum value of row coordinates, the maximum value of row coordinates, the minimum value of column coordinates, and the maximum value of column coordinates:

[0019] X′ t,min = Ceil(Min(X′ t [i,j],X′ t [i+1,j],X′ t [i,j+1],X′ t [i+1,j+1]))

[0020] X′ t,max =Floor(Max(X′ t [i,j],X′ t [i+1,j],X′ t [i,j+1],X′ t [i+1,j+1]))

[0021] Y′ t,min = Ceil(Min(Y′ t [i,j],Y′ t [i+1,j],Y′ t [i,j+1],Y′ t [i+1,j+1]))

[0022] Y′ t,max =Floor(Max(Y′ t [i,j],Y′ t [i+1,j],Y′ t [i,j+1],Y′ t [i+1,j+1]))

[0023] Among them, the functions Min() and Max() respectively find the minimum and maximum values ​​of the given data, and Ceil() and Floor() round up and down the given data respectively;

[0024] Based on this, the center coordinates of the candidate target pixels contained therein are traversed:

[0025] T={(X t ,Y t )|X′ t,min ≤X t ≤X′ t,max ,Y′ t,min ≤Y t ≤Y′ t,max ,X t and Y t All are integers}.

[0026] Furthermore, the reverse mapping specifically includes:

[0027] Reversely rotate all candidate target pixel coordinates to obtain their corresponding reversely rotated coordinates:

[0028] X′ t =(X t cosθ+Y t sinθ) / α

[0029] Y′ it =(-X t sinθ+Y t cosθ) / α

[0030] If the reverse rotation coordinates are within the corresponding 4 source coordinates, that is:

[0031] x s ≤X′ it ≤x s +1

[0032] y s ≤Y′ it ≤y s +1

[0033] Then take it as a valid target and read the RGB pixel color information stored in the four adjacent cells with addresses (i, j), (i+1, j), (i, j+1), and (i+1, j+1) in the cache in step 1, recorded as (R i,j ,G i,j ,B i,j )、(R i+1,j ,G i+1,j ,B i+1,j )、(R i,j+1 ,G i,j+1 ,B i,j+1 )、(R i+1,j+1 ,G i+1,j+1 ,B i+1,j+1 ), and calculate the target RGB information using bilinear interpolation:

[0034] R t =(R i,j (1-d i )+R i+1,j d i )(1-d j )+(R i,j+1 (1-d i )+R i+1,j+1 d i )d j

[0035] G t =(G i,j (1-d i )+Gi+1,j d i )(1-d j )+(G i,j+1 (1-d i )+G i+1,j+1 d i )d j

[0036] B t =(B i,j (1-d i )+B i+1,j d i )(1-d j )+(B i,j+1 (1-d i )+B i+1,j+1 d i )d j

[0037] Among them, the interpolation weight d i and d j They are:

[0038] d i =X′ it -x s

[0039] d j =Y′ it -y s

[0040] The target pixel center coordinate (X t ,Y t ) is converted to angular coordinates (x t ,y t ):

[0041] x t =X t +W / 2

[0042] y t =Y t +H / 2

[0043] Cache valid target pixel coordinates and corresponding target pixel RGB information (x t ,y t ,R t ,G t ,B t ).

[0044] Furthermore, the method further comprises: reading the target pixel coordinates and the target pixel RGB information in the cache, and t ,y t ), the target pixel RGB information (R t,G t ,B t ) is stored in the DDR storage unit, the target pixel RGB information stored in the DDR storage unit is sequentially read to form a video image frame, and output to the video output interface.

[0045] A real-time video image rotation system based on FPGA, the system is applied to any of the above methods, the system comprises: a forward mapping module, a target pixel traversal module, a reverse mapping module, a scheduling module, a linear interpolation module, a target pixel collection module, a DDR writing module, a DDR reading module and a video frame generation module;

[0046] Forward mapping module: receives video input, performs forward mapping of source pixel coordinates, caches forward mapping coordinates and their corresponding source image color information, outputs the starting coordinates, forward mapping coordinates and color values ​​of four adjacent cells, and supports parallel processing to meet real-time requirements in high-resolution and high-frame-rate video image rotation applications such as 4K;

[0047] Target pixel traversal module: traverses the candidate target pixel coordinates contained in the forward mapping coordinate range output by the forward mapping module, outputs the candidate target pixel coordinates and their corresponding source pixel coordinates and color information, and supports parallel processing to meet real-time requirements;

[0048] Reverse mapping module: reversely rotates candidate target pixel coordinates, filters reversely rotated coordinates that are not within the valid source pixel coordinate range, outputs valid target pixel coordinates, interpolation weights, and corresponding source pixel color information, and supports parallel processing to meet real-time requirements;

[0049] Scheduling module: receives the output of the reverse mapping module and distributes it to the linear interpolation module;

[0050] Linear interpolation module: calculates the color value of the target pixel according to the target pixel coordinates, interpolation weights and source pixel color information, and outputs the coordinates and color information of the target pixel, supporting parallel processing to meet real-time requirements;

[0051] Target pixel collection module: receives the output of the linear interpolation module and stores the color value in RAM for subsequent processing;

[0052] Write DDR module: Use burst write operation to efficiently write the color value of the target pixel into the DDR memory, and improve the writing efficiency by dividing the blocks;

[0053] Read DDR module: Use burst read operation to improve reading efficiency, read the block content with color data, and write it into RAM;

[0054] Video frame generation module: obtains image color data from the DDR read module in sequence, generates and outputs video synchronization control signals and corresponding color data.

[0055] Compared with the prior art, the advantages of the present invention are:

[0056] The quadratic mapping image rotation method can realize real-time rotation of high-resolution video images. Similar to the forward mapping method, the quadratic mapping image rotation method calculates the rotated output video pixels from the input video pixels. The overall processing flow is a forward output processing that directly calculates the output from the input, which is convenient for pipeline operation and makes it easier to realize real-time rotation processing of high-resolution video images. At the same time, the quadratic mapping method can avoid the problem of a large number of holes in the forward mapping method.

[0057] The secondary mapping image rotation method can avoid the delay problem of reading the source image pixel value during the reverse mapping method. In the reverse mapping method, due to the access delay of the DDR memory, the read operation cannot obtain data in time, resulting in the interruption of the rotation calculation process, making it difficult to achieve real-time rotation processing of high-resolution video images. The secondary mapping method performs forward output processing during the rotation calculation process. It is not necessary to cache the video frame to the DDR memory before the rotation operation, and it also avoids the problem of processing interruption caused by waiting for the DDR read data results during the rotation process.

[0058] During the implementation process, the FPGA-based secondary mapping method video image rotation system structure can adjust parameters such as the number of target pixel traversal modules, reverse mapping modules, linear interpolation modules, and the block size in the target pixel collection module according to the input source video resolution and frame rate, thereby reducing system resource consumption while meeting the real-time rotation of the video image.

[0059] Both rotation and scaling transformations can be achieved simultaneously. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 , two Illustration of the sub-mapped image rotation method (with scaling);

[0061] Figure 2 , Video image rotation system based on secondary mapping method based on FPGA. DETAILED DESCRIPTION

[0062] The specific implementation mode of the present invention is described below in conjunction with embodiments:

[0063] It should be noted that the structures, proportions, sizes, etc. shown in this specification are only used to match the contents disclosed in the specification so that people familiar with this technology can understand and read them, and are not used to limit the conditions under which the present invention can be implemented. Any structural modification, change in proportional relationship or adjustment of size should still fall within the scope of the technical content disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention.

[0064] At the same time, the terms such as "upper", "lower", "left", "right", "middle" and "one" cited in this specification are only for the convenience of description and are not used to limit the scope of implementation of the present invention. Changes or adjustments to their relative relationships should be regarded as the scope of implementation of the present invention without substantially changing the technical content.

[0065] Embodiment 1:

[0066] The present invention first proposes a secondary mapping image rotation method. The secondary mapping image rotation method is as follows: Figure 1 As shown, first, the input source image is forward mapped, and the candidate target pixels in the forward mapping area are traversed; then, the candidate target pixels are reverse mapped, and the valid reverse mappings located in the source pixel area are screened out; finally, for the valid reverse mappings, the linear interpolation algorithm is used to calculate the target pixel value, and the output image is updated.

[0067] Under the quadratic mapping image rotation method, the calculation and derivation process is slightly different depending on the linear interpolation algorithm used. The following description takes the bilinear interpolation method as an example.

[0068] 1. Forward rotation mapping:

[0069] Parse the input video data frame to generate the source pixel coordinates (i, j) and RGB color (R s ,G s ,B s ). The source pixel corner coordinates (i, j) are converted into coordinates (x s ,y s ):

[0070] x s =iW / 2

[0071] y s =jH / 2

[0072] Where W and H are the image width and height respectively.

[0073] Calculate the center coordinates of the rotated target pixel (X′ t [i,j],Y′ t[i,j]):

[0074] X′ t [i,j]=α(x s cosθ-y s sinθ)

[0075] Y′ t [i,j]=α(x s sinθ+y s cosθ)

[0076] Among them, θ is the rotation angle and α is the scaling ratio.

[0077] Using the source pixel corner coordinates (i, j) as the address, cache the target pixel center coordinates and source RGB information (X′ t [i,j],Y′ t [i,j],R s ,G s ,B s ).

[0078] 2. Traverse the target coordinates:

[0079] Sequentially read the forward rotation center coordinates (X′) stored in the four adjacent cells with addresses (i, j), (i+1, j), (i, j+1), and (i+1, j+1) in the cache in step 1 t [i,j],Y′ t [i,j])、(X′ t [i+1,j],Y′ t [i+1,j]),(X′ t [i,j+1],Y′ t [i,j+1])、(X′ t [i+1,j+1],Y′ t [i+1,j+1]), calculate the minimum value of row coordinates, the maximum value of row coordinates, the minimum value of column coordinates, and the maximum value of column coordinates:

[0080] X′ t,min = Ceil(Min(X′ t [i,j],X′ t [i+1,j],X′ t [i,j+1],X′ t [i+1,j+1]))

[0081] X′ t,max =Floor(Max(X′ t [i,j],X′ t [i+1,j],X′ t [i,j+1],X′ t[i+1,j+1]))

[0082] Y′ t,min = Ceil(Min(Y′ t [i,j],Y′ t [i+1,j],Y′ t [i,j+1],Y′ t [i+1,J+1]))

[0083] Y′ t,max =Floor(Max(Y′ t [i,j],Y′ t [i+1,j],Y′ t [i,j+1],Y′ t [i+1,j+1]))

[0084] Among them, the functions Min() and Max() respectively calculate the minimum and maximum values ​​of the given data, and Ceil() and Floor() round the given data up and down respectively.

[0085] Based on this, the center coordinates of the candidate target pixels contained therein are traversed:

[0086] T={(X t ,Y t )|X′ t,min ≤X t ≤X′ t,max ,Y′ t,min ≤Y t ≤Y′ t,max ,X t and Y t All are integers}

[0087] 3. Reverse rotation mapping:

[0088] Reversely rotate all candidate target pixel coordinates obtained in step 2 to obtain their corresponding reversely rotated coordinates:

[0089] X′ it =(X t cosθ+Y t sinθ) / α

[0090] Y′ it =(-X t sinθ+Y t cosθ) / α

[0091] If the reverse rotation coordinates are within the corresponding 4 source coordinates, that is:

[0092] x s ≤X′ it ≤xs +1

[0093] y s ≤Y′ it ≤y s +1

[0094] Then take it as a valid target and read the RGB pixel color information stored in the four adjacent cells with addresses (i, j), (i+1, j), (i, j+1), and (i+1, j+1) in the cache in step 1, recorded as (R i,j ,G i,j ,B i,j )、(R i+1,j ,G i+1,j ,B i+1,j )、(R i,j+1 ,G i,j+1 ,B i,j+1 )、(R i+1,j+1 ,G i+1,j+1 ,B i+1,j+1 ), and calculate the target RGB information using bilinear interpolation:

[0095] R t =(R i,j (1-d i )+R i+1,j d i )(1-d j )+(R i,j+1 (1-d i )+R i+1,j+1 d i )d j

[0096] G t =(G i,j (1-d i )+G i+1,j d i )(1-d j )+(G i,j+1 (1-d i )+G i+1,j+1 d i )d j

[0097] B t =(B i,j (1-d i )+B i+1,j d i )(1-d j )+(B i,j+1 (1-d i )+B i+1,j+1 d i )d j

[0098] Among them, the interpolation weight d i and d j They are:

[0099] d i =X′ it -x s

[0100] d j =Y′ it -y s

[0101] The target pixel center coordinate (X t ,Y t ) is converted to angular coordinates (x t ,y t ):

[0102] x t =X t +W / 2

[0103] y t =Y t +H / 2

[0104] Cache valid target pixel coordinates and corresponding target pixel RGB information (x t ,y t ,R t ,G t ,B t ).

[0105] 4. Save target pixel information:

[0106] Read the target pixel coordinates and target pixel RGB information in the cache in step 3, and calculate the target pixel coordinates (x t ,y t ), the target pixel RGB information (R t ,G t ,B t ) is stored in the DDR memory unit.

[0107] 5. Generate output data frame:

[0108] The target RGB information stored in the DDR storage unit is sequentially read to form a video data frame, which is output to the video output interface.

[0109] Secondly, the present invention proposes a real-time video image rotation system based on FPGA using a secondary mapping method. Figure 2As shown, the FPGA-based secondary mapping method video image rotation system is mainly composed of a forward mapping module, a target pixel traversal module, a reverse mapping module, a scheduling module, a linear interpolation module, a target pixel collection module, a DDR writing module, a DDR reading module and a video frame generation module.

[0110] The forward mapping module receives the parsed video input signal, completes the forward mapping of the source image coordinates, caches the forward mapping coordinates and source pixel colors, and outputs the starting coordinates (i, j) of the adjacent 4 cells, the forward mapping center coordinates (X′ t [i,j],Y′ t [i,j])、(X′ t [i+1,j],Y′ t [i+1,j]),(X′ t [i,j+1],Y′ t [i,j+1])、(X′ t [i+1,j+1],Y[i+1,j+1]), and the corresponding source image pixel color information:

[0111] (R i,j ,G i,j ,B i,j );

[0112] (R i+1,j ,G i+1,j ,B i+1,j );

[0113] (R i,j+1 ,G i,j+1 ,B i,j+1 );

[0114] (R i+1,j+1 ,G i+1,j+1 ,B i+1,j+1 ).

[0115] The target pixel traversal module traverses and outputs the candidate target pixel center coordinates contained therein according to the forward mapped center coordinates of the four adjacent pixels in the source image, and outputs the corresponding source pixel coordinates and color information.

[0116] The reverse mapping module reversely rotates the candidate target pixel coordinates obtained by traversal to obtain the corresponding reverse mapping coordinates, filters out invalid coordinates that are not within the range of the four adjacent source pixel coordinates of the starting coordinates (i, j), and outputs the valid target pixel coordinates and interpolation weights d i and d j , and the corresponding source image pixel color information.

[0117] The system supports simultaneous rotation and scaling transformations. If image enlargement is required, the density of candidate target pixel coordinates obtained by traversal is greater than the density of source pixels in the effective mapping area of ​​the image. In the case where the density of candidate target pixel coordinates is greater than the density of source pixels, m target pixel traversal modules and m reverse mapping modules can be used for parallel processing to meet real-time processing requirements.

[0118] The scheduling module receives the target pixel coordinates and interpolation weights d output by all reverse mapping modules. i and d j As well as the source image pixel color information, and assign it to the linear interpolation module.

[0119] The linear interpolation module calculates the pixel coordinates and interpolation weights d according to the target pixel coordinates and interpolation weights d i and d j The linear interpolation algorithm is used to calculate the target pixel color value based on the source image pixel color information, and the target pixel coordinates and color are output. If the target pixel coordinate density is greater than the source pixel density, n linear interpolation modules can be used for parallel processing to meet real-time processing requirements.

[0120] The target pixel collection module receives the target pixel coordinates and color values ​​output by the linear interpolation module, and stores the color values ​​into the RAM according to the target pixel coordinates.

[0121] In order to improve the writing efficiency of the DDR memory, the writing DDR module adopts a burst writing operation to write the pixel color value output by the target pixel collection module into the DDR memory.

[0122] To cooperate with the DDR burst write operation, the target pixel collection module divides the pixel color cache into blocks with a side length of S pixels. When it is detected that a block is full, all pixel colors in the block are written to the corresponding address area of ​​DDR through a burst write operation. After receiving the last valid pixel of a frame of input image, the residual block content that has not been written to DDR is written to the DDR memory.

[0123] In order to improve the efficiency of DDR memory reading, the DDR reading module also uses burst read operation. The DDR read burst length can be set to an integer multiple of the write burst length. Through a burst read operation, several block contents are read and written into the RAM according to the address. The RAM in the DDR reading module needs to store at least 2S lines of image color data, and the DDR reading and sequential output of the video image are completed in a ping-pong operation.

[0124] The video frame generation module sequentially obtains the required image color data from the DDR reading module, generates and outputs control signals such as video line synchronization, field synchronization, data enable, and corresponding color data.

[0125] Embodiment 2:

[0126] As a replaceable implementation method, this embodiment 2 replaces the original bilinear interpolation module with a bicubic interpolation module under the framework of the quadratic mapping method, and adapts the relevant coordinate calculation and module architecture design to ensure the output of high-quality rotated images under large-angle rotation or high scaling requirements.

[0127] The bicubic interpolation process needs to obtain the color values ​​of 16 neighboring pixels in a square window with a length and width of 4 pixels in the source image. For ease of description and to keep it consistent with the bilinear interpolation process, the pixel in the second row and second column of the window is taken as the starting point, and its coordinates are set to (i, j). The pixel coordinates in the window are:

[0128]

[0129] 1. Forward rotation mapping:

[0130] Parse the input video data frame to generate the source pixel coordinates (i, j) and RGB color (R s ,G s ,B s ). The source pixel corner coordinates (i, j) are converted into coordinates (x s ,y s ):

[0131] x s =iW / 2

[0132] y s =jH / 2

[0133] Where W and H are the image width and height respectively.

[0134] Calculate the center coordinates of the rotated target pixel (X′ t [i,j],Y′ t [i,j]):

[0135] X′ t [i,j]=α(x s cosθ-y s sinθ)

[0136] Y′ t [i,j]=α(x s sinθ+y s cosθ)

[0137] Among them, θ is the rotation angle and α is the scaling ratio.

[0138] Using the source pixel corner coordinates (i, j) as the address, cache the target pixel center coordinates and source RGB information (X′ t [i,j],Y′ t [i,j],R s ,G s ,B s ).

[0139] 2. Traverse the target coordinates:

[0140] Sequentially read the forward rotation center coordinates (X′) stored in the four adjacent cells with addresses (i, j), (i+1, j), (i, j+1), and (i+1, j+1) in the cache in step 1 t [i,j],Y′ t [i,j])、(X′ t [i+1,j],Y′ t [i+1,j]),(X′ t [i,j+1],Y′ t [i,j+1])、(X′ t [i+1,j+1],Y′ t [i+1,j+1]), calculate the minimum value of row coordinates, the maximum value of row coordinates, the minimum value of column coordinates, and the maximum value of column coordinates:

[0141] X′ t,min = Ceil(Min(X′ t [i,j],X′ t [i+1,j],X′ t [i,j+1],X′ t [i+1,j+1]))

[0142] X′ t,max =Floor(Max(X′ t [i,j],X′ t [i+1,j],X′ t [i,j+1],X′ t [i+1,j+1]))

[0143] Y′ t,min = Ceil(Min(Y′ t [i,j],Y′ t [i+1,j],Y′ t [i,j+1],Y′ t [i+1,j+1]))

[0144] Y′ t,max =Floor(Max(Y′ t [i,j],Y′ t[i+1,j],Y′ t [i,j+1],Y′ t [i+1,j+1]))

[0145] Among them, the functions Min() and Max() respectively calculate the minimum and maximum values ​​of the given data, and Ceil() and Floor() round the given data up and down respectively.

[0146] Based on this, the center coordinates of the candidate target pixels contained therein are traversed:

[0147] T={(X t ,Y t )|X′ t,min ≤X t ≤X′ t,max ,Y′ t,min ≤Y t ≤Y′ t,max ,X t and Y t All are integers}

[0148] 3. Reverse rotation mapping:

[0149] Reversely rotate all candidate target pixel coordinates obtained in step 2 to obtain their corresponding reversely rotated coordinates:

[0150] X′ it =(X t cosθ+Y t sinθ) / α

[0151] Y′ it =(-X t sinθ+Y t cosθ) / α

[0152] If the reverse rotation coordinates are within the corresponding 4 source coordinates, that is:

[0153] x s ≤X′ it ≤x s +1

[0154] y s ≤Y′ it ≤y s +1

[0155] Then use it as a valid target and read the RGB color information of 16 neighboring pixels in the square window with the starting address (i, j) in the cache in step 1:

[0156]

[0157] Use bicubic interpolation to calculate the target RGB information:

[0158]

[0159] Among them, the interpolation weight d i and d j They are:

[0160] d i =X′ it -x s

[0161] d j =Y′ it -y s

[0162] W() is an interpolation weighted cubic polynomial function, and its options include:

[0163]

[0164] The target pixel center coordinate (X t ,Y t ) is converted to angular coordinates (x t ,y t ):

[0165] x t =X t +W / 2

[0166] y t =Y t +H / 2

[0167] Cache valid target pixel coordinates and corresponding target pixel RGB information (x t ,y t ,R t ,G t ,B t ).

[0168] The cubic interpolation system is similar to the framework of the original system, but the interpolation module needs to be replaced and the input data range needs to be adapted.

[0169] Forward mapping module: The function remains unchanged, completes the forward mapping of the source image coordinates, caches the forward mapping coordinates and source pixel colors, outputs the forward mapping coordinates of the source pixels with the starting coordinates (i, j), (i+1, j), (i, j+1), (i+1, j+1), and the color information of the 16 adjacent source pixels in the window of the starting coordinates (i, j).

[0170] Target pixel traversal module: The function remains unchanged, but it needs to additionally output the color information of 16 source pixels corresponding to each target pixel.

[0171] Reverse mapping module: The function remains unchanged, outputting valid target pixel coordinates, interpolation weights and corresponding 16 source pixel color information.

[0172] Bicubic interpolation module (core improvement): replaces the bilinear interpolation module and performs bicubic interpolation calculations based on the interpolation weights of the target pixels and the color values ​​of the 16 source pixels.

[0173] Target pixel collection module: The function remains unchanged, receiving the target pixel coordinates and color values ​​output by the linear interpolation module, and storing the color values ​​in RAM according to the target pixel coordinates.

[0174] Write DDR module and read DDR module: Maintain the existing design and continue to optimize storage using burst operations.

[0175] Video frame generation module: The function remains unchanged, reading the processed target pixel data from the memory and generating the output image frame.

[0176] There are two main changes in the bicubic rotation process compared to the bilinear interpolation method:

[0177] 1. From the forward mapping module to the linear interpolation module, the source pixel color information is increased from 4 adjacent pixels to 16 adjacent pixels, requiring more pipeline registers.

[0178] 2. In the linear interpolation module, the bicubic interpolation calculation is larger, requiring the use of more hardware multipliers and optimizing the pipeline design to ensure that the module output is not blocked. In the case of limited FPGA resources, it is necessary to balance the module parallelism and hardware utilization.

[0179] Compared with the bilinear interpolation method, the bicubic interpolation method has a smoother effect after image rotation, sharper edges, and better output image quality. However, it has higher computational complexity and requires more hardware resources.

[0180] Through the above scheme, cubic interpolation can significantly improve the image quality in the FPGA image rotation system, and at the same time combine parallel optimization to meet the real-time processing requirements.

[0181] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0182] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0183] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0184] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0185] The preferred embodiments of the present invention are described in detail above, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

[0186] Many other changes and modifications may be made without departing from the concept and scope of the present invention.It should be understood that the present invention is not limited to the specific embodiments, and the scope of the present invention is defined by the appended claims.

Claims

1. A real-time video image rotation method based on FPGA, characterized in that: The method comprises: performing forward rotation mapping on an input image and traversing candidate target pixels in a forward mapping area; then, performing reverse mapping on the candidate target pixels and screening out effective reverse mappings located in a source pixel area; finally, for effective reverse mappings, using a linear interpolation algorithm to calculate target pixel values ​​and updating an output image, thereby achieving a high-quality real-time rotation effect by combining forward and reverse mappings.

2. The method for real-time rotation of video images based on FPGA according to claim 1, characterized in that: The forward rotation mapping of the input image specifically includes: Parse the input video image frame to generate the source pixel coordinates (i, j) and RGB color (R s ,G s ,B s ), converting the source pixel corner coordinates (i, j) into coordinates (x s ,y s ): x s =i-W / 2 y s =j-H / 2 Where W and H are the image width and height respectively; Calculate the source pixel (x s ,y s ) The center coordinate after positive rotation (X′ t [i,j],Y′ t [i,j]): X′ t [i,j]=α(x s cosθ-y s sinθ) Y′ t [i,j]=α(x s sinθ+y s cosθ) Among them, θ is the rotation angle and α is the scaling ratio; Use the source pixel angle coordinates (i, j) as the address, cache the forward rotation center coordinates and source RGB information (X′ t [i,j],Y′ t [i,j],R s ,G s ,B s ).

3. The method for real-time rotation of video images based on FPGA according to claim 2, characterized in that: Traverse the valid candidate target pixels in the forward mapping area, including: Sequentially read the forward rotation center coordinates (X′) stored in the four adjacent cells with addresses (i, j), (i+1, j), (i, j+1), and (i+1, j+1) in the cache. t [i,j],Y′ t [i,j])、(X′ t [i+1,j],Y′ t [i+1,j]),(X′ t [i,j+1],Y′ t [i,j+1])、(X′ t [i+1,j+1],Y′ t [i+1,j+1]), calculate the minimum value of row coordinates, the maximum value of row coordinates, the minimum value of column coordinates, and the maximum value of column coordinates: X′ t,min =Ceil(Min(X′ t [i,j],X′ t [i+1,j],X′ t [i,j+1],X′ t [i+1,j+1])) X′ t,max =Floor(Max(X′ t [i,j],X′ t [i+1,j],X′ t [i,j+1],X′ t [i+1,j+1])) Y′ t,min =Ceil(Min(Y′ t [i,j],Y′ t [i+1,j],Y′ t [i,j+1],Y′ t [i+1,j+1])) Y′ t,max =Floor(Max(Y′ t [i,j],Y′ t [i+1,j],Y′ t [i,j+1],Y′ t [i+1,j+1])) Among them, the functions Min() and Max() respectively find the minimum and maximum values ​​of the given data, and Ceil() and Floor() round up and down the given data respectively; Based on this, the center coordinates of the candidate target pixels contained therein are traversed: T={(X t ,Y t )|X′ t,min ≤X t ≤X′ t,max ,Y′ t,min ≤Y t ≤Y′ t,max ,X t and Y t All are integers}.

4. The method for real-time rotation of video images based on FPGA according to claim 1, characterized in that: The reverse mapping specifically includes: Reversely rotate all candidate target pixel coordinates to obtain their corresponding reversely rotated coordinates: X′ it =(X t cosθ+Y t sinθ) / a Y′ it =(-X t sinθ+Y t cosθ) / α If the reverse rotation coordinates are within the corresponding 4 source coordinates, that is: x s ≤X′ it ≤x s +1 and s ≤Y′ it ≤y s +1 Then take it as a valid target and read the RGB pixel color information stored in the four adjacent cells with addresses (i, j), (i+1, j), (i, j+1), and (i+1, j+1) in the cache in step 1, recorded as (R i,j ,G i,j ,B i,j )、(R i+1,j ,G i+1,j ,B i+1,j )、(R i,j+1 ,G i,j+1 ,B i,j+1 )、(R i+1,j+1 ,G i+1,j+1 ,B i+1,j+1 ), and calculate the target RGB information using bilinear interpolation: R t =(R i,j (1-d i )+R i+1,j d i )(1-d j )+(R i,j+1 (1-d i )+R i+1,j+1 d i )d j G t =(G i,j (1-d i )+G i+1,j d i )(1-d j )+(G i,j+1 (1-d i )+G i+1,j+1 d i )d j B t =(B i,j (1-d i )+B i+1,j d i )(1-d j )+(B i,j+1 (1-d i )+B i+1,j+1 d i )d j Among them, the interpolation weight d i and d j They are: d i =X′ iy -x s d j =Y′ it -y s The target pixel center coordinate (X t ,Y t ) is converted to angular coordinates (x t ,y t ): x t =X t +W / 2 y t =Y t +H / 2 Cache valid target pixel coordinates and corresponding target pixel RGB information (x t ,y t ,R t ,G t ,B t ).

5. The method for real-time rotation of video images based on FPGA according to claim 1, characterized in that: The method further comprises: reading the target pixel coordinates and the target pixel RGB information in the cache, and t ,y t ), the target pixel RGB information (R t ,G t ,B t ) is stored in the DDR storage unit, the target pixel RGB information stored in the DDR storage unit is sequentially read to form a video image frame, and output to the video output interface.

6. A real-time video image rotation system based on FPGA, characterized in that: The system is applied to the method described in any one of claims 1 to 5, and the system comprises: a forward mapping module, a target pixel traversal module, a reverse mapping module, a scheduling module, a linear interpolation module, a target pixel collection module, a DDR writing module, a DDR reading module and a video frame generation module; Forward mapping module: receives video input, performs forward mapping of source pixel coordinates, caches forward mapping coordinates and their corresponding source image color information, outputs the starting coordinates, forward mapping coordinates and color values ​​of four adjacent cells, and supports parallel processing to meet real-time requirements in high-resolution and high-frame-rate video image rotation applications such as 4K; Target pixel traversal module: traverses the candidate target pixel coordinates contained in the forward mapping coordinate range output by the forward mapping module, outputs the candidate target pixel coordinates and their corresponding source pixel coordinates and color information, and supports parallel processing to meet real-time requirements; Reverse mapping module: reversely rotates candidate target pixel coordinates, filters reversely rotated coordinates that are not within the valid source pixel coordinate range, outputs valid target pixel coordinates, interpolation weights, and corresponding source pixel color information, and supports parallel processing to meet real-time requirements; Scheduling module: receives the output of the reverse mapping module and distributes it to the linear interpolation module; Linear interpolation module: calculates the color value of the target pixel according to the target pixel coordinates, interpolation weights and source pixel color information, and outputs the coordinates and color information of the target pixel, supporting parallel processing to meet real-time requirements; Target pixel collection module: receives the output of the linear interpolation module and stores the color value in RAM for subsequent processing; Write DDR module: Use burst write operation to efficiently write the color value of the target pixel into the DDR memory, and improve the writing efficiency by dividing the blocks; Read DDR module: Use burst read operation to improve reading efficiency, read the block content with color data, and write it into RAM; Video frame generation module: obtains image color data from the DDR read module in sequence, generates and outputs video synchronization control signals and corresponding color data.

Images