Data processing method based on FPGA affine inverse transformation

Abstract

The present application relates to a kind of data processing methods based on FPGA affine inverse transform, belong to image processing technical field, solve the problem of insufficient BRAM space when loading original image data in prior art.The data processing method includes: determining the matte subgraph according to the filling subgraph and affine transformation inverse matrix;Determine the first preloading subgraph according to the matte subgraph;Judge whether the size of the first preloading subgraph is greater than the size of BRAM space, obtain the second preloading subgraph;The second preloading subgraph is stored to the BRAM space;Each pixel point in the filling subgraph is traversed in turn, and the pixel value of each pixel point is determined according to the affine transformation inverse matrix and the second preloading subgraph, and the pixel value of each pixel point is filled.Original image data is saved to BRAM space normally, and data execution is guaranteed to be successful.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a data processing method based on FPGA inverse affine transform. Background Technology

[0002] With the development of technology, people's demand for high-definition video is becoming increasingly strong. Upscaling low-resolution video to high-definition video has become a major challenge. With the increasingly widespread application of FPGA (Field Programmable Gate Array), implementing high-definition video based on FPGA is gradually becoming mainstream.

[0003] Affine transformation algorithms can perform functions such as image rotation, translation, and scaling, and are widely used in many video and image processing fields.

[0004] However, in the existing technology, dividing the image into blocks reduces the requirement for Block Random Access Memory (BRAM) space. When faced with the need to load large amounts of original image data due to image downsizing, the BRAM space is insufficient to hold the required original image data, resulting in data execution failure. Summary of the Invention

[0005] Based on the above analysis, the present invention aims to provide a data processing method based on FPGA inverse affine transform to solve the problem of insufficient BRAM space when loading raw image data in the prior art.

[0006] This invention provides a data processing method based on FPGA inverse affine transform, the data processing method comprising:

[0007] The matted sub-blocks are determined based on the filled sub-blocks and the inverse affine transformation matrix. The filled sub-blocks are sub-blocks divided from the result image, and the matted sub-blocks are sub-blocks of the original image.

[0008] A first preloaded sub-block is determined based on the cutout sub-block; it is determined whether the size of the first preloaded sub-block is greater than the size of the BRAM space; if it is, the first preloaded sub-block is reduced in size according to the size of the first preloaded sub-block and the size of the BRAM space to obtain a second preloaded sub-block; if it is not, the first preloaded sub-block is used as the second preloaded sub-block.

[0009] Store the second preloaded sub-plot into the BRAM space;

[0010] Each pixel in the filling sub-block is traversed sequentially, and the pixel value of each pixel is determined according to the inverse affine transformation matrix and the second preloaded sub-block. The pixel value of each pixel is then filled.

[0011] A further improvement to the above method, the step of determining the cutout sub-block based on the filled sub-block and the inverse affine transformation matrix, includes:

[0012] Determine the mapping coordinates corresponding to the mapping sub-block; the mapping coordinates include the coordinates of the four mapping vertices corresponding to the mapping sub-block.

[0013] Based on the inverse affine transformation, the matting coordinates of the matted sub-plot are determined according to the coordinates of the four matting vertices and the inverse affine transformation matrix. The matting coordinates include the coordinates of the four matting vertices.

[0014] The cutout sub-block is determined based on the coordinates of the four cutout vertices.

[0015] Based on a further improvement of the above method, the step of determining the first preloaded sub-block according to the cutout sub-block includes:

[0016] Based on the coordinates of the four cutout vertices of the cutout sub-block, the maximum and minimum values ​​of the cutout sub-block on the two coordinate axes are determined;

[0017] Based on the maximum and minimum values ​​on the two coordinate axes, determine the coordinates of the four vertices of the first preloaded sub-plot:

[0018]

[0019]

[0020] Where A, B, C, and D represent the coordinates of the four vertices of the first preloaded sub-plot, Xmax represents the maximum value of the cutout sub-plot on the X-axis, Xmin represents the minimum value of the cutout sub-plot on the X-axis, Ymax represents the maximum value of the cutout sub-plot on the Y-axis, and Ymin represents the minimum value of the cutout sub-plot on the Y-axis.

[0021] Based on a further improvement to the above method, determining whether the size of the first preloaded sub-plot is greater than the size of the BRAM space includes:

[0022] Determine the length and width of the first preloaded sub-block and determine the length and width of the BRAM space;

[0023] Calculate the length ratio of the first preloaded sub-plot to the length of the BRAM space, and calculate the width ratio of the first preloaded sub-plot to the width of the BRAM space;

[0024] If either the length ratio or the width ratio is greater than 1, it is judged as yes; otherwise, it is judged as no.

[0025] A further improvement to the above method, wherein the step of reducing the first preloaded sub-block according to the size of the first preloaded sub-block and the BRAM space to obtain the second preloaded sub-block, includes:

[0026] Determine whether the length ratio is greater than the width ratio; if yes, determine the pre-shrunk ratio parameter based on the length ratio; otherwise, determine the pre-shrunk ratio parameter based on the width ratio.

[0027] The first preloaded sub-block is reduced according to the pre-scaling ratio parameter to obtain the second preloaded sub-block.

[0028] Based on a further improvement of the above method, the pre-shrinkage ratio parameter is N / M, where N and M are both integers in the range [1, 6], and M > N.

[0029] A further improvement to the above method, wherein the step of reducing the first preloaded sub-block according to the pre-scaling ratio parameter to obtain the second preloaded sub-block, includes:

[0030] N-1 pixels are inserted between two adjacent pixels in each row of the first preloaded sub-block. Then, N-1 pixels are inserted between two adjacent pixels in each column of the first preloaded sub-block, thereby expanding the first preloaded sub-block by N times to obtain an intermediate sub-block.

[0031] In the intermediate sub-block, a row is selected every M-1 rows as a row to be sampled. In each row to be sampled, a pixel is selected every M-1 points as a sampling point, thereby reducing the intermediate sub-block by a factor of M to obtain the second preloaded sub-block.

[0032] Based on a further improvement to the above method, when inserting pixels into the first preloaded sub-block, the expression for the pixel value of the interpolated pixel is:

[0033] Dix_j=DixJ-j[(DixJ-DixK) / N];

[0034] Where DixJ and DixK represent the pixel values ​​of two adjacent pixels J and K in the first preloaded sub-block, respectively, and Dix_j represents the pixel value of the j-th pixel among the N-1 pixels inserted sequentially from pixel J to pixel K, where 1≤j≤(N-1).

[0035] A further improvement to the above method, the step of determining the pixel value of each pixel based on the inverse affine transformation matrix and the second preloaded sub-block, includes:

[0036] By combining the inverse affine transformation matrix and the pre-scaling ratio parameter, the first coordinates of each pixel in the BRAM space are determined.

[0037] The pixel value of each pixel is determined based on the first coordinate.

[0038] Based on a further improvement of the above method, determining the pixel value of each pixel point according to the first coordinate includes:

[0039] Let P(x, y) represent the first coordinate;

[0040] When both x and y are integers, the pixel value of point P is used as the pixel value of each corresponding pixel.

[0041] When x and y are not integers, four second coordinates around point P are determined based on point P, and the pixel values ​​of the four second coordinates are determined by addressing the BRAM space; based on the bilinear interpolation algorithm, the pixel value corresponding to the first coordinate is calculated through the pixel values ​​of the four second coordinates, and the pixel value corresponding to the first coordinate is used as the pixel value of each corresponding pixel point;

[0042] When only one of x and y is an integer, two third coordinates around point P are determined based on point P, and the pixel values ​​of the two third coordinates are determined by addressing the BRAM space. Based on the bilinear interpolation algorithm, the pixel value corresponding to the first coordinate is calculated using the pixel values ​​of the two third coordinates, and the pixel value corresponding to the first coordinate is used as the pixel value of each corresponding pixel.

[0043] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0044] 1. By determining whether the size of the original image data to be loaded (the first preloaded sub-plot) is greater than the size of the BRAM space; if the determination is yes, the original image data is reduced in size according to the original image data and the size of the BRAM space, and the reduced original image data is saved to the BRAM space, thus realizing the normal saving of the original image data to the BRAM space and ensuring the successful execution of data.

[0045] 2. By using the mapping coordinates of the mapping sub-blocks, the original image data to be loaded is determined based on the inverse affine transformation, which improves the accuracy of calculating the original image data to be loaded.

[0046] 3. Based on the length ratio and width ratio of the original image data to the BRAM space required for loading, the pre-reduction ratio parameter was further determined, which ensured the reduction range and reduced the loss of the original image data when reducing the original image data.

[0047] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0048] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0049] Figure 1 A schematic flowchart of the data processing method based on FPGA inverse affine transformation provided in an embodiment of the present invention;

[0050] Figure 2 This is a schematic diagram of the structure of the resulting image divided into filling sub-blocks according to an embodiment of the present invention;

[0051] Figure 3 This is one of the structural schematic diagrams of the cutout sub-block corresponding to the filling sub-block provided in the embodiments of the present invention;

[0052] Figure 4 This is the second schematic diagram of the structure of the cutout sub-block corresponding to the filling sub-block provided in the embodiment of the present invention;

[0053] Figure 5 This is a schematic diagram illustrating the interpolation and enlargement of a first preloaded sub-block according to an embodiment of the present invention;

[0054] Figure 6 This is a schematic diagram illustrating the segmentation and scaling of intermediate sub-plots using interval sampling, as provided in an embodiment of the present invention.

[0055] Figure 7 One of the structural schematic diagrams of the first coordinate provided in the embodiment of the present invention;

[0056] Figure 8 This is a second schematic diagram of the structure of the first coordinate provided in an embodiment of the present invention. Detailed Implementation

[0057] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0058] A specific embodiment of the steps of this invention discloses a data processing method based on FPGA inverse affine transform, such as... Figure 1 As shown.

[0059] Step S1: Determine the matted sub-image patches based on the filled sub-image patches and the inverse affine transformation matrix. The filled sub-image patches are sub-image patches divided according to the result image, and the matted sub-image patches are sub-image patches of the original image.

[0060] Step S2: Determine the first preloaded sub-block based on the cutout sub-block; determine whether the size of the first preloaded sub-block is greater than the size of the BRAM space; if the determination is yes, reduce the size of the first preloaded sub-block based on the size of the first preloaded sub-block and the size of the BRAM space to obtain the second preloaded sub-block; if the determination is no, use the first preloaded sub-block as the second preloaded sub-block;

[0061] Step S3: Store the second preloaded sub-block into the BRAM space;

[0062] Step S4: Iterate through each pixel in the filling sub-block, determine the pixel value of each pixel according to the inverse affine transformation matrix and the second preloaded sub-block, and fill the pixel value of each pixel.

[0063] Specifically, when using an FPGA for video image processing, the original image pixel data needs to be stored from the externally configured DDR (Double Data Rate) memory to the on-chip BRAM space of the FPGA. Then, the original image pixel data is mapped to the result image according to the affine transformation matrix, and the result image is also saved to the DDR.

[0064] Specifically, in step S1, the resulting image is divided into multiple sub-blocks as filling sub-blocks. Based on the row and column of the filling sub-block, they can be further divided into odd-column blocks, even-column blocks, odd-row blocks, and even-row blocks. Once the size of each filling sub-block is designed, it is not changed. For example, as shown... Figure 2 As shown, the size of the resulting image is 1280*1024, with a length of 1280 pixels and a width of 1024 pixels; the size of the filling sub-block is 128*128, with a length of 128 pixels and a width of 128 pixels; the resulting image is divided into 80 filling sub-blocks, each with a length of 10 and a width of 8.

[0065] Understandably, a two-dimensional coordinate system is used to locate all pixels in both the resulting and original images.

[0066] The inverse matrix of an affine transformation in a two-dimensional coordinate system includes:

[0067] 1. The 3x3 control matrix for two-dimensional translation is:

[0068] In this matrix, tx and ty represent the pixel distances of translation, corresponding to the directions of the arrows on the X and Y coordinate axes. For the inverse transform, its forward translation is exactly opposite to the direction of the arrows on the coordinate axes, and therefore is represented by a negative sign.

[0069] 2. The 3x3 control matrix for two-dimensional rotation is:

[0070] In this matrix, θ represents the rotation angle, and the sign of the sine parameter of this matrix is ​​exactly the opposite of that in the forward transformation.

[0071] 3. The 3x3 control matrix for two-dimensional scaling is:

[0072] In this matrix, Sx and Sy are magnification factors. Since the magnification factor of the forward transformation is the reduction factor of the inverse transformation, the magnification factors here exist in reciprocal form.

[0073] Based on the inverse affine transformation matrix, the image coordinates of the resulting image and the original image are rearranged. It's worth noting that the pixel coordinates of the resulting image are mapped to the coordinates of the original image, and then the pixel values ​​from the original image coordinates are transferred to the pixel coordinates of the resulting image. The matrix operation expression is as follows:

[0074]

[0075] Where (u, v) are the coordinates of any pixel in the resulting image, and (x, y) are the corresponding coordinates in the original image obtained from the inverse affine transformation matrix. It is the inverse matrix of the affine transformation.

[0076] It is worth noting that image transformation can be achieved by changing the parameters in the control matrix. Of course, multiple deformation parameters can also be mixed to achieve the purpose of multiple deformations occurring simultaneously in a single affine transformation.

[0077] Specifically, in step S1, the cutout sub-block is calculated based on the filled sub-block and the inverse affine transformation matrix. The cutout sub-block is a sub-block in the original image, and the cutout sub-block is the sub-block in the original image corresponding to the filled sub-block.

[0078] Preferably, determining the cutout sub-block based on the filled sub-block and the inverse affine transformation matrix includes:

[0079] Determine the mapping coordinates corresponding to the mapping sub-block; the mapping coordinates include the coordinates of the four mapping vertices corresponding to the mapping sub-block.

[0080] Based on the inverse affine transformation, the matting coordinates of the matted sub-plot are determined according to the coordinates of the four matting vertices and the inverse affine transformation matrix. The matting coordinates include the coordinates of the four matting vertices.

[0081] The cutout sub-block is determined based on the coordinates of the four cutout vertices.

[0082] Specifically, such as Figure 3 As shown, the filling sub-block is any sub-block in the result image. Combining the two-dimensional coordinate system of the result image, the coordinates of the four corresponding filling vertices Q1, Q2, Q3 and Q4 can be determined for this filling sub-block.

[0083] Based on the inverse affine transformation, according to the coordinates of the four filling vertices Q1, Q2, Q3, and Q4, the corresponding coordinates of the four matting vertices in the original image are determined by calculating the inverse affine transformation matrix, namely Q1', Q2', Q3', and Q4'. Figure 3 As shown, the cutout sub-block is the sub-block in the original image that corresponds to the filled sub-block.

[0084] Specifically, in step S2, a first preloaded sub-block is determined based on the matted sub-blocks. It's worth noting that the DDR memory configured externally to the FPGA is read in burst lengths. Directly extracting the pixel data corresponding to the matted sub-blocks from the DDR would result in significant timing waste. Therefore, the first preloaded sub-block needs to be determined based on the matted sub-blocks of the original image. The first preloaded sub-block contains the pixel data that needs to be stored from the DDR into the BRAM space.

[0085] Preferably, determining the first preloaded sub-block based on the cutout sub-block includes:

[0086] Based on the coordinates of the four cutout vertices of the cutout sub-block, the maximum and minimum values ​​of the cutout sub-block on the two coordinate axes are determined;

[0087] Based on the maximum and minimum values ​​on the two coordinate axes, determine the coordinates of the four vertices of the first preloaded sub-plot:

[0088]

[0089]

[0090] Where A, B, C, and D represent the coordinates of the four vertices of the first preloaded sub-plot, Xmax represents the maximum value of the cutout sub-plot on the X-axis, Xmin represents the minimum value of the cutout sub-plot on the X-axis, Ymax represents the maximum value of the cutout sub-plot on the Y-axis, and Ymin represents the minimum value of the cutout sub-plot on the Y-axis.

[0091] Specifically, such as Figure 3 As shown, the coordinates of the four cutout vertices Q1', Q2', Q3', and Q4' have a maximum value of Xmax and a minimum value of Xmin on the X-axis; and a maximum value of Ymax and a minimum value of Ymin on the Y-axis. The coordinates of the four vertices of the first preloaded sub-tile are...

[0092] It is worth noting that when reading the pixel value from the DDR, the coordinates of each pixel are integers.

[0093] It is understandable that the rectangle enclosed by the first preloaded sub-tile includes the cutout region formed by the cutout sub-tile, and the coordinates of the four vertices of the first preloaded sub-tile are all integers.

[0094] Specifically, in step S2, it is determined whether the size of the first preloaded sub-block is greater than the size of the BRAM space; if it is determined to be yes, the first preloaded sub-block is reduced according to the size of the first preloaded sub-block and the BRAM space to obtain the second preloaded sub-block; if it is determined to be no, the first preloaded sub-block is used as the second preloaded sub-block.

[0095] Specifically, the size of the BRAM space is generally designed to be 1.5-2 times the size of the filled sub-block. For example, if the size of the filled sub-block is 128*128, the size of the BRAM space is designed to be 256*256. Once the size of the BRAM space is determined, it will not be changed.

[0096] It is worth noting that when dealing with a scaled-down original image, loading pixel data from the original image based on the fill sub-blocks of the resulting image may require loading a large region of the original image. The pre-defined BRAM space may not be sufficient to hold the required pixel data from the original image, leading to data execution failure. Therefore, this invention compares the size of the first pre-loaded sub-block with the size of the BRAM space before storing it in the BRAM.

[0097] like Figure 3As shown, when the size of the first preloaded sub-block is larger than the size of the BRAM space, the first preloaded sub-block is shrunk according to the size of the first preloaded sub-block and the size of the BRAM space to obtain the second preloaded sub-block, so that the BRAM space is larger than the second preloaded sub-block, so that the second preloaded sub-block can be placed normally in the BRAM space and the data can be executed successfully.

[0098] like Figure 4 As shown, when the size of the first preloaded sub-block is less than or equal to the size of the BRAM space, the first preloaded sub-block is directly used as the second preloaded sub-block.

[0099] Preferably, determining whether the size of the first preloaded sub-plot is greater than the size of the BRAM space includes:

[0100] Determine the length and width of the first preloaded sub-block and determine the length and width of the BRAM space;

[0101] Calculate the length ratio of the first preloaded sub-plot to the length of the BRAM space, and calculate the width ratio of the first preloaded sub-plot to the width of the BRAM space;

[0102] If either the length ratio or the width ratio is greater than 1, it is judged as yes; otherwise, it is judged as no.

[0103] Specifically, the size of the BRAM space is fixed, such as L*W; L is the length, usually 256 pixels, and W is the width, usually 256 pixels. The size of the first preloaded sub-tile is determined based on the coordinates of its four vertices A, B, C, and D. The length of the first preloaded sub-tile is... 1 pixel, width is 1 pixel.

[0104] Specifically, the length ratio of the first preloaded sub-plot to the length of the BRAM space is 1. The width ratio of the first preloaded sub-tile to the width of the BRAM space is:

[0105] Specifically, compare length ratios Compare the width ratio with the size of 1. The value of the length ratio and the width ratio is determined by the following condition: if either the length ratio or the width ratio is greater than 1, it is considered yes; otherwise, it is considered no.

[0106] Specifically, in step S2, if the determination is yes, the first preloaded sub-block is reduced in size according to the size of the first preloaded sub-block and the BRAM space to obtain the second preloaded sub-block.

[0107] Preferably, the step of reducing the size of the first preloaded sub-block according to the dimensions of the first preloaded sub-block and the BRAM space to obtain the second preloaded sub-block includes:

[0108] Determine whether the length ratio is greater than the width ratio; if yes, determine the pre-shrunk ratio parameter based on the length ratio; otherwise, determine the pre-shrunk ratio parameter based on the width ratio.

[0109] The first preloaded sub-block is reduced according to the pre-scaling ratio parameter to obtain the second preloaded sub-block.

[0110] Specifically, determine the length ratio Ratio of width The size. If the length ratio Greater than the width ratio Then it is determined to be yes, based on the length ratio. Determine the pre-shrinkage ratio parameter; if the length ratio Less than or equal to the width ratio The result is then determined as no, based on the width ratio. Determine the pre-shrinkage ratio parameter.

[0111] Preferably, the pre-shrinkage ratio parameter is N / M, where N and M are both integers in the range [1, 6], and M > N.

[0112] It is worth noting that when the length ratio Greater than the width ratio When the length ratio is 1, the reciprocal of the length ratio is taken, and then the reciprocal is simplified to a fraction. The fraction is then made as close as possible to the reciprocal without exceeding it. This fraction is the pre-scaling factor parameter N / M, where N and M are both integers in the range [1, 6], and M > N. Similarly, when the length ratio is 1... Less than or equal to the width ratio When the width ratio is taken as a factor, the reciprocal of the width ratio is simplified to a fraction and the fractional value is made as close as possible to the reciprocal value without being greater than it. The fractional value is the pre-scaling ratio parameter N / M, such that N and M are both integers in the range [1,6] and M>N.

[0113] For example, if the calculated length ratio is 1.6 and the width ratio is 1.2, the pre-scaling ratio parameter can be determined to be 3 / 5 based on the length ratio. Specifically, first take the reciprocal of the length ratio as 1 / 1.6 = 0.625. Then select integers N and M such that N / M is no greater than 0.625 and as close to 0.625 as possible. Therefore, N is selected as 3 and M as 5, which simplifies the reciprocal value to the fraction 3 / 5, resulting in a pre-scaling ratio parameter of 3 / 5.

[0114] Preferably, the step of reducing the first preloaded sub-block according to the pre-scaling ratio parameter to obtain the second preloaded sub-block includes:

[0115] N-1 pixels are inserted between two adjacent pixels in each row of the first preloaded sub-block. Then, N-1 pixels are inserted between two adjacent pixels in each column of the first preloaded sub-block, thereby expanding the first preloaded sub-block by N times to obtain an intermediate sub-block.

[0116] In the intermediate sub-block, a row is selected every M-1 rows as a row to be sampled. In each row to be sampled, a pixel is selected every M-1 points as a sampling point, thereby reducing the intermediate sub-block by a factor of M to obtain the second preloaded sub-block.

[0117] Specifically, such as Figure 5 As shown, for example: when N is 2, one pixel is inserted between two adjacent pixels in each row of the first preloaded sub-block. Then, one pixel is inserted between two adjacent pixels in each column of the first preloaded sub-block, thereby expanding the first preloaded sub-block by a factor of 2 to obtain an intermediate sub-block. When N is 3, two pixels are inserted between two adjacent pixels in each row of the first preloaded sub-block. Then, two pixels are inserted between two adjacent pixels in each column of the first preloaded sub-block, thereby expanding the first preloaded sub-block by a factor of 3 to obtain an intermediate sub-block. The original pixels represent the pixels in the first preloaded sub-block, and the interpolated pixels represent the inserted pixels.

[0118] Preferably, when inserting pixels into the first preloaded sub-block, the expression for the pixel value of the interpolated pixel is:

[0119] Dix_j=DixJ-j[(DixJ-DixK) / N];

[0120] Where DixJ and DixK represent the pixel values ​​of two adjacent pixels J and K in the first preloaded sub-block, respectively, and Dix_j represents the pixel value of the j-th pixel among the N-1 pixels inserted sequentially from pixel J to pixel K, where 1≤j≤(N-1).

[0121] Specifically, such as Figure 5 As shown, for example: when N is 2, in the first row of the first preloaded sub-block, one pixel is inserted between two adjacent original pixels. From left to right, the pixel values ​​of the two adjacent original pixels are 20 and 30 respectively. Then the pixel value of the inserted pixel is 20-1*[(20-30) / 2], that is, the pixel value of the inserted pixel is 25. When N is 3, in the first row, two pixels are inserted between two adjacent original pixels. From left to right, the pixel values ​​of the two adjacent original pixels are 20 and 30 respectively. Then the pixel values ​​of the inserted pixels are 20-1*[(20-30) / 3] and 20-2*[(20-30) / 3], that is, the pixel values ​​of the two inserted pixels are 23 and 26.

[0122] Specifically, such as Figure 5 As shown, for example: when N is 2, in the first column of the first preloaded sub-block, one pixel is inserted between two adjacent original pixels. From top to bottom, the pixel values ​​of the two adjacent original pixels are 30 and 40 respectively. Then the pixel value of the inserted pixel is 30-1*[(30-40) / 2], that is, the pixel value of the inserted pixel is 35. When N is 3, in the first column of the first preloaded sub-block, two pixels are inserted between two adjacent original pixels. From top to bottom, the pixel values ​​of the two adjacent original pixels are 30 and 40 respectively. Then the pixel values ​​of the inserted pixels are 30-1*[(30-40) / 3] and 30-2*[(30-40) / 3], that is, the pixel values ​​of the two inserted pixels are 33 and 36.

[0123] Specifically, such as Figure 6 As shown, for example: when M is 2, in the intermediate sub-block, a row is selected every other row as a row to be sampled. In each row to be sampled, a pixel is selected every other point as a sampling point, thereby reducing the intermediate sub-block by a factor of 2 to obtain the second preloaded sub-block; when M is 3, in the intermediate sub-block, a row is selected every other row as a row to be sampled. In each row to be sampled, a pixel is selected every other point as a sampling point, thereby reducing the intermediate sub-block by a factor of 3 to obtain the second preloaded sub-block. Pixels represent the pixels of the sampled points, and discarded pixels represent the pixels that were not sampled.

[0124] Compared with existing technologies, the data processing method based on FPGA inverse affine transformation provided in this invention solves the technical problem that affine reduction transformation cannot be performed due to the original image being too large and the BRAM space being too small during the affine transformation process. This is achieved by determining whether the size of the original image data to be loaded is larger than the size of the BRAM space. If so, the original image data is reduced in size based on the original image data and the BRAM space, and the reduced original image data is saved to the BRAM space. Furthermore, by using the filling coordinates of the filling sub-blocks, the original image data to be loaded is determined based on the inverse affine transformation, improving the accuracy of calculating the original image data to be loaded. Finally, by determining the length ratio and width ratio of the original image data to be loaded to the BRAM space, a pre-reduction ratio parameter is further determined, ensuring the reduction range and reducing the loss of original image data during the reduction process.

[0125] In step S3, the second preloaded sub-block is stored in the BRAM space.

[0126] In step S4, each pixel in the filling sub-block is traversed sequentially, and the pixel value of each pixel is determined according to the inverse affine transformation matrix and the second preloaded sub-block. The pixel value of each pixel is then filled.

[0127] Preferably, determining the pixel value of each pixel based on the inverse affine transformation matrix and the second preloaded sub-block includes:

[0128] By combining the inverse affine transformation matrix and the pre-scaling ratio parameter, the first coordinates of each pixel in the BRAM space are determined.

[0129] The pixel value of each pixel is determined based on the first coordinate.

[0130] Specifically, by combining the inverse affine transformation matrix and the pre-scaling parameter, the first coordinates corresponding to each pixel in the BRAM space are determined, including:

[0131] The coordinates of the corresponding pixel in the original image are obtained by using the coordinates of each pixel and the inverse affine transformation matrix. The first coordinate is obtained by subtracting the first coordinate of the first preloaded sub-plot from the coordinates of the pixel in the original image and then multiplying it by the pre-scaling parameter.

[0132] Specifically, for any pixel in the filling sub-block with coordinates (u, v), the corresponding coordinates of the original image are calculated as (u', v') using the inverse of the radiometric transformation matrix. It's worth noting that (u, v) and (u', v') are both coordinates within the same coordinate system. Understandably, the coordinate system in BRAM space differs from the aforementioned coordinate system. Therefore, the corresponding coordinates (u'v') need to be subtracted from the first coordinate of the first preloaded sub-block corresponding to the filling sub-block to convert the corresponding coordinate point (u', v') into a coordinate point (u”, v”) in the BRAM space coordinate system. Based on the pre-scaling factor parameter N / M, the first coordinates of the pixel point (u, v) in BRAM space are obtained as (u”*N / M, v”*N / M).

[0133] Preferably, determining the pixel value of each pixel point based on the first coordinate includes:

[0134] Let P(x, y) represent the first coordinate;

[0135] When both x and y are integers, the pixel value of point P is used as the pixel value of each corresponding pixel.

[0136] When x and y are not integers, four second coordinates around point P are determined based on point P, and the pixel values ​​of the four second coordinates are determined by addressing the BRAM space; based on the bilinear interpolation algorithm, the pixel value corresponding to the first coordinate is calculated through the pixel values ​​of the four second coordinates, and the pixel value corresponding to the first coordinate is used as the pixel value of each corresponding pixel point;

[0137] When only one of x and y is an integer, two third coordinates around point P are determined based on point P, and the pixel values ​​of the two third coordinates are determined by addressing the BRAM space. Based on the bilinear interpolation algorithm, the pixel value corresponding to the first coordinate is calculated using the pixel values ​​of the two third coordinates, and the pixel value corresponding to the first coordinate is used as the pixel value of each corresponding pixel.

[0138] Specifically, when both x and y are integers, it means that the first coordinate P(x, y) is a pixel. In this case, the pixel value corresponding to point P in the BRAM space is directly taken out as the pixel value of the pixel in the filling sub-block.

[0139] When only one of x and y is an integer, for example, such as Figure 7As shown, y is an integer and x is not an integer, indicating that the first coordinate P(x, y) lies between the integer coordinates of two adjacent pixels. Based on point P, two third coordinates P1 and P2 are determined. The pixel values ​​corresponding to the third coordinates P1 and P2 in the BRAM space are extracted as DixP1 and DixP2. Therefore, the pixel value corresponding to the first coordinate P is DixP = DixP1*k1 + Dix*k2, where... Use the pixel value corresponding to point P as the pixel value of the pixel in the filled sub-block.

[0140] When both x and y are not integers, such as Figure 8 As shown, the first coordinate P(x, y) lies between four adjacent pixels. Based on point P, four second coordinates P1, P2, P3, and P4 are determined. The pixel values ​​of these second coordinates P1, P2, P3, and P4 in the BRAM space are extracted as DixP1, DixP2, DixP3, and DixP4. Therefore, the pixel value corresponding to the first coordinate P is DixP = (DixP1*k1 + DixP2*k3 + DixP3*k1 + DixP4*k3 + DixP1*k4 + DixP3*k2 + DixP2*k4 + DixP4*k2) / 4, where... Use the pixel value corresponding to point P as the pixel value of the pixel in the filled sub-block.

[0141] After obtaining the pixel values ​​of each filled sub-block in sequence using the above method, all filled sub-blocks form the result image.

[0142] Compared with the prior art, the data processing method based on FPGA inverse affine transformation provided in this embodiment of the invention determines the first coordinate in BRAM space by any pixel point in the filling sub-block, determines the pixel value corresponding to the first coordinate by different cases of the first coordinate, and uses the pixel value of the first coordinate as the pixel value of the corresponding pixel point in the filling sub-block.

[0143] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0144] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A data processing method based on FPGA inverse affine transform, characterized in that, The data processing method includes: The matting sub-blocks are determined based on the filled sub-blocks and the inverse affine transformation matrix. The filled sub-blocks are sub-blocks divided according to the result image. Once the size of each filled sub-block is designed, it will not be changed. The matting sub-blocks are sub-blocks of the original image. The first preloaded sub-block is determined based on the cutout sub-block; determining whether the size of the first preloaded sub-block is greater than the size of the BRAM space includes: Determine the length and width of the first preloaded sub-block and determine the length and width of the BRAM space; Calculate the length ratio of the first preloaded sub-plot to the length of the BRAM space, and calculate the width ratio of the first preloaded sub-plot to the width of the BRAM space; If either the length ratio or the width ratio is greater than 1, it is judged as yes; otherwise, it is judged as no. If the determination is yes, the first preloaded sub-block is reduced in size according to the first preloaded sub-block and the BRAM space to obtain a second preloaded sub-block, including: Determine whether the length ratio is greater than the width ratio; if yes, determine the pre-shrunk ratio parameter based on the length ratio; otherwise, determine the pre-shrunk ratio parameter based on the width ratio. The first preloaded sub-block is reduced according to the pre-scaling ratio parameter to obtain the second preloaded sub-block; If the determination is negative, the first preloaded sub-block is used as the second preloaded sub-block; Store the second preloaded sub-plot into the BRAM space; Each pixel in the filling sub-block is traversed sequentially, and the pixel value of each pixel is determined based on the inverse affine transformation matrix and the second preloaded sub-block, including: By combining the inverse affine transformation matrix and the pre-scaling ratio parameter, the first coordinates of each pixel in the BRAM space are determined. The pixel value of each pixel is determined based on the first coordinate, and the pixel value of each pixel is filled.

2. The data processing method according to claim 1, characterized in that, The step of determining the cutout sub-block based on the filled sub-block and the inverse affine transformation matrix includes: Determine the mapping coordinates corresponding to the mapping sub-block; the mapping coordinates include the coordinates of the four mapping vertices corresponding to the mapping sub-block. Based on the inverse affine transformation, the matting coordinates of the matted sub-plot are determined according to the coordinates of the four matting vertices and the inverse affine transformation matrix. The matting coordinates include the coordinates of the four matting vertices. The cutout sub-block is determined based on the coordinates of the four cutout vertices.

3. The data processing method according to claim 2, characterized in that, The step of determining the first preloaded sub-block based on the cutout sub-block includes: Based on the coordinates of the four cutout vertices of the cutout sub-block, the maximum and minimum values ​​of the cutout sub-block on the two coordinate axes are determined; Based on the maximum and minimum values ​​on the two coordinate axes, determine the coordinates of the four vertices of the first preloaded sub-plot: A( Xmin , Ymin );B( Xmin , Ymax );C ( Xmax , Ymin );D( Xmax , Ymax ); Where A, B, C, and D represent the coordinates of the four vertices of the first preloaded sub-plot, Xmax represents the maximum value of the cutout sub-plot on the X-axis, Xmin represents the minimum value of the cutout sub-plot on the X-axis, Ymax represents the maximum value of the cutout sub-plot on the Y-axis, and Ymin represents the minimum value of the cutout sub-plot on the Y-axis.

4. The data processing method according to claim 1, characterized in that, The pre-scaling ratio parameter is N / M, where N and M are both integers in the range [1, 6], and M > N.

5. The data processing method according to claim 4, characterized in that, The step of reducing the first preloaded sub-block according to the pre-scaling ratio parameter to obtain the second preloaded sub-block includes: N-1 pixels are inserted between two adjacent pixels in each row of the first preloaded sub-block. Then, N-1 pixels are inserted between two adjacent pixels in each column of the first preloaded sub-block, thereby expanding the first preloaded sub-block by N times to obtain an intermediate sub-block. In the intermediate sub-block, a row is selected every M-1 rows as a row to be sampled. In each row to be sampled, a pixel is selected every M-1 points as a sampling point, thereby reducing the intermediate sub-block by a factor of M to obtain the second preloaded sub-block.

6. The data processing method according to claim 5, characterized in that, When inserting a pixel in the first preloaded sub-block, the expression for the pixel value of the interpolated pixel is: Dix_j=DixJ-j[(DixJ-DixK) / N]; Where DixJ and DixK represent the pixel values ​​of two adjacent pixels J and K in the first preloaded sub-block, respectively, and Dix_j represents the pixel value of the j-th pixel among the N-1 pixels inserted sequentially from pixel J to pixel K, where 1≤j≤(N-1).

7. The data processing method according to claim 1, characterized in that, Determining the pixel value of each pixel point based on the first coordinate includes: Let P(x, y) represent the first coordinate; When both x and y are integers, the pixel value of point P is used as the pixel value of each corresponding pixel. When x and y are not integers, four second coordinates around point P are determined based on point P, and the pixel values ​​of the four second coordinates are determined by addressing the BRAM space; based on the bilinear interpolation algorithm, the pixel value corresponding to the first coordinate is calculated through the pixel values ​​of the four second coordinates, and the pixel value corresponding to the first coordinate is used as the pixel value of each corresponding pixel point; When only one of x and y is an integer, two third coordinates around point P are determined based on point P, and the pixel values ​​of the two third coordinates are determined by addressing the BRAM space. Based on the bilinear interpolation algorithm, the pixel value corresponding to the first coordinate is calculated using the pixel values ​​of the two third coordinates, and the pixel value corresponding to the first coordinate is used as the pixel value of each corresponding pixel.

Images