Data Acceleration Processing System Based on FPGA Affine Inverse Transformation

Abstract

The present invention relates to a data acceleration processing system based on FPGA affine inverse transformation, which belongs to the field of image processing technology and solves the problem that the FPGA in the prior art takes too long to process data and cannot realize affine reduction transformation due to insufficient BRAM space. The data acceleration processing system includes: a first external memory for storing multiple mapping sub-blocks; a second external memory for storing original images; an acceleration control module for controlling a preloading control module and a mapping control module to implement a preloading step and an addressing and reading step for odd column blocks and even column blocks in a ping-pong operation; a preloading control module for executing the preloading step; a mapping control module for executing the addressing and reading step; and a BRAM space, including a first BRAM space and a second BRAM space, which are respectively used to store the second preloaded sub-blocks corresponding to the odd column blocks and the even column blocks. Rapid processing of data by FPGA and normal execution of affine reduction transformation are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a data acceleration processing system based on FPGA affine inverse transformation. Background Art

[0002] With the development of science and technology, people's demand for high-definition video is getting stronger and stronger. Scaling low-resolution video to high-definition video has become a major issue. With the increasing application of FPGA (Field Programmable Gate Array), realizing high-definition video based on FPGA has gradually become the mainstream.

[0003] The affine transformation algorithm can realize functions such as image rotation, translation, and scaling, and is widely used in many video image processing fields.

[0004] However, in the prior art, data can only be read and written independently for random access BRAM (Block Random Access Memory) space, which causes FPGA to take too long to process data. At the same time, when facing image reduction transformation, large original image data needs to be loaded, and the BRAM space is insufficient to place the required original image data, resulting in data execution failure. Summary of the invention

[0005] In view of the above analysis, an embodiment of the present invention aims to provide a data acceleration processing system based on FPGA affine inverse transformation, so as to solve the problem in the prior art that FPGA takes too long to process data and cannot realize affine reduction transformation due to insufficient BRAM space.

[0006] An embodiment of the present invention provides a data acceleration processing system based on FPGA affine inverse transformation, and the data acceleration processing system includes:

[0007] A first external memory is used to store a plurality of mapping sub-blocks, wherein the mapping sub-blocks are sub-blocks divided according to the result image, and the mapping sub-blocks are divided into odd column blocks and even column blocks;

[0008] A second external memory for storing original images;

[0009] An acceleration control module, used for controlling the preloading control module and the mapping control module to implement the preloading step and the addressing and reading step on the odd column blocks and the even column blocks in a ping-pong operation manner;

[0010] a preloading control module, configured to execute the preloading step, wherein the preloading step comprises: determining a corresponding first preloading sub-image block in the original image according to the odd column block / even column block and an inverse matrix of an affine transformation, obtaining the first preloading sub-image block from the second external memory, calculating a second preloading sub-image block based on the first preloading sub-image block, and storing the second preloading sub-image block corresponding to the odd column block in a first BRAM space, and storing the second preloading sub-image block corresponding to the even column block in a second BRAM space;

[0011] A mapping control module, configured to execute the addressing and reading step, wherein the addressing and reading step comprises: sequentially traversing each pixel point in the odd column block / even column block, determining a pixel value of each pixel point according to the inverse affine transformation matrix and the second preloaded sub-image block in the first BRAM space / second BRAM space corresponding to the odd column block / even column block, and sending the pixel value to the first external memory for storage;

[0012] The BRAM space includes the first BRAM space and the second BRAM space, which are respectively used to store the second preloaded sub-image blocks corresponding to the odd column blocks and the even column blocks.

[0013] Based on the further improvement of the above system, the data acceleration processing system further includes:

[0014] A coordinate generator is used to generate the coordinates of any pixel point in the original image and the result image, and is used to generate the coordinates of any pixel point in the second preloaded sub-block.

[0015] Based on the further improvement of the above system, the data acceleration processing system further includes:

[0016] A buffer register is used to cache the pixel values ​​in the first preloaded sub-image block sent by the second external memory, and transmit the corresponding pixel values ​​to the preload control module according to the control instruction of the preload control module, and the preload control module obtains the pixel value of each point of the second preloaded sub-image block according to the acquired pixel value of the first preloaded sub-image block.

[0017] Based on a further improvement of the above system, determining the corresponding first preloaded sub-image block in the original image according to the odd column block / even column block and the inverse affine transformation matrix includes:

[0018] A cutout sub-image block is determined according to the odd column block / even column block and an inverse affine transformation matrix, and a first preloaded sub-image block is determined according to the cutout sub-image block.

[0019] Based on a further improvement of the above system, the step of calculating and obtaining the second preloaded sub-picture block based on the first preloaded sub-picture block includes:

[0020] Determine whether the size of the first preloaded sub-image block is larger than the size of the first BRAM space / the second BRAM space; if so, determine a pre-reduction ratio parameter according to the size of the first preloaded sub-image block and the first BRAM space / the second BRAM space, and reduce the first preloaded sub-image block based on the pre-reduction ratio parameter to obtain a second preloaded sub-image block; if not, use the first preloaded sub-image block as the second preloaded sub-image block.

[0021] Based on a further improvement of the above system, determining the pixel value of each pixel point according to the inverse affine transformation matrix and the second preloaded sub-image block in the first BRAM space or the second BRAM space corresponding to the odd column block / even column block includes:

[0022] The coordinates of the corresponding pixel in the original image are obtained according to the coordinates of each pixel and the inverse matrix of the affine transformation, and the first coordinates of the first preloaded sub-block are subtracted from the coordinates of the pixel in the original image and then multiplied by the pre-reduction ratio parameter to obtain the first coordinates;

[0023] The pixel value of each corresponding pixel point is determined according to the first coordinate.

[0024] Based on the further improvement of the above system, the ping-pong operation is used to implement the preloading step and the addressing and reading step for the odd column block and the even column block as follows:

[0025] Perform the preloading step and the addressing and reading step on the odd column blocks / even column blocks based on a ping-pong operation for all the mapping sub-blocks in the result image in a sequence from left to right and from top to bottom;

[0026] When the preloading step is performed on the odd column block / even column block, the addressing and reading step is performed on the even column block / odd column block at the same time.

[0027] Based on the further improvement of the above system, the determining of the cutout sub-block according to the odd column block / even column block and the inverse matrix of the affine transformation includes:

[0028] Determine the mapping coordinates corresponding to the odd column block / even column block; the mapping coordinates include four mapping vertex coordinates corresponding to the odd column block / even column block;

[0029] Based on the inverse affine transformation, determining the cutout coordinates of the cutout sub-block according to the four mapping vertex coordinates and the inverse affine transformation matrix, wherein the cutout coordinates include four cutout vertex coordinates;

[0030] The cutout sub-block is determined according to the coordinates of the four cutout vertices.

[0031] Based on a further improvement of the above system, the determining whether the size of the first preloaded sub-tile is larger than the size of the first BRAM space / the second BRAM space includes:

[0032] Determine the length and width of the first preloaded sub-tile and determine the length and width of the first BRAM space / second BRAM space;

[0033] Calculating a length ratio of the first preloaded sub-image block to a length of the first BRAM space / second BRAM space and calculating a width ratio of the first preloaded sub-image block to a width of the first BRAM space / second BRAM space; the first BRAM space and the second BRAM space have the same size;

[0034] If any one of the length ratio and the width ratio is greater than 1, it is judged as yes, otherwise, it is judged as no.

[0035] Based on a further improvement of the above system, determining the pre-reduction ratio parameter according to the size of the first preloaded sub-tile and the first BRAM space / the second BRAM space includes:

[0036] Determine whether the length ratio is greater than the width ratio; if so, determine the pre-reduction ratio parameter according to the length ratio; otherwise, determine the pre-reduction ratio parameter according to the width ratio; the pre-reduction ratio parameter is N / M, wherein N and M are both integers of [1,6], and M>N.

[0037] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0038] 1. Determine whether the size of the original image data to be loaded (the first preloaded sub-block) is larger than the size of the BRAM space; if so, reduce the original image data according to the size of the original image data and the BRAM space, and save the reduced original image data to the BRAM space, thereby realizing the normal saving of the original image data to the BRAM space and ensuring that the reduction transformation in the affine transformation can be successfully executed.

[0039] 2. By filling the mapping coordinates of the sub-block, the original image data to be loaded is determined based on the inverse affine transformation, thereby improving the accuracy of calculating the original image data to be loaded.

[0040] 3. The pre-reduction ratio parameters are further determined according to the length ratio and width ratio of the original image data to be loaded and the BRAM space, so that when the original image data is reduced, the reduction range is guaranteed and the loss of the original image data is reduced.

[0041] 4. The acceleration control module controls the preloading control module and the mapping control module to use ping-pong operation to perform preloading steps and addressing and reading steps on odd column blocks and even column blocks, thereby realizing rapid data processing by FPGA and reducing processing delay.

[0042] In the present invention, the above-mentioned technical solutions can also be combined with each other to achieve more preferred combination solutions. Other features and advantages of the present invention will be described in the subsequent description, and some advantages can become obvious from the description, or can be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained through the contents particularly pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like components throughout the drawings.

[0044] Figure 1 A schematic diagram of the structure of a data acceleration processing system based on FPGA affine inverse transformation provided by an embodiment of the present invention;

[0045] Figure 2 A schematic diagram of a structure in which a result image provided by an embodiment of the present invention is divided into mapping sub-blocks;

[0046] Figure 3 One of the structural schematic diagrams of the cutout sub-block corresponding to the fill sub-block provided in the embodiment of the present invention;

[0047] Figure 4 A second structural schematic diagram of a cutout sub-block corresponding to a fill sub-block provided in an embodiment of the present invention;

[0048] Figure 5 A schematic diagram of interpolating and expanding a first preloaded sub-image block provided by an embodiment of the present invention;

[0049] Figure 6 A schematic diagram of performing alternate point sampling and reduction on a middle sub-image block provided by an embodiment of the present invention;

[0050] Figure 7 One of the structural schematic diagrams of the first coordinate provided by an embodiment of the present invention;

[0051] Figure 8 The second structural diagram of the first coordinate provided by the embodiment of the present invention. DETAILED DESCRIPTION

[0052] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0053] A specific embodiment of the present invention discloses a data acceleration processing system based on FPGA affine inverse transformation, such as Figure 1 As shown, the data acceleration processing system includes:

[0054] A first external memory is used to store a plurality of mapping sub-blocks, wherein the mapping sub-blocks are sub-blocks divided according to the result image, and the mapping sub-blocks are divided into odd column blocks and even column blocks;

[0055] A second external memory for storing original images;

[0056] An acceleration control module, used for controlling the preloading control module and the mapping control module to implement the preloading step and the addressing and reading step on the odd column blocks and the even column blocks in a ping-pong operation manner;

[0057] a preloading control module, configured to execute the preloading step, wherein the preloading step comprises: determining a corresponding first preloading sub-image block in the original image according to the odd column block / even column block and an inverse matrix of an affine transformation, obtaining the first preloading sub-image block from the second external memory, calculating a second preloading sub-image block based on the first preloading sub-image block, and storing the second preloading sub-image block corresponding to the odd column block in a first BRAM space, and storing the second preloading sub-image block corresponding to the even column block in a second BRAM space;

[0058] A mapping control module, configured to execute the addressing and reading step, wherein the addressing and reading step comprises: sequentially traversing each pixel point in the odd column block / even column block, determining a pixel value of each pixel point according to the inverse affine transformation matrix and the second preloaded sub-image block in the first BRAM space / second BRAM space corresponding to the odd column block / even column block, and sending the pixel value to the first external memory for storage;

[0059] The BRAM space includes the first BRAM space and the second BRAM space, which are respectively used to store the second preloaded sub-image blocks corresponding to the odd column blocks and the even column blocks.

[0060] Specifically, when using FPGA for video image processing, the original image pixel data needs to be stored from the DDR (Double Data Rate) memory configured outside the FPGA to the BRAM space within the FPGA chip, and then the original image pixel data is mapped to the result image according to the affine transformation matrix, and the result image is also saved in the DDR.

[0061] It is worth noting that the first external memory is used to store the pixel value data of the result image, and the second external memory is used to store the pixel value data of the original image. The first external memory and the second external memory can be the same DDR configured outside the FPGA, or different DDRs. Specifically, the second external memory is connected to the preloading control module through the AXI4 bus, and the first external memory is connected to the mapping control module through the AXI4 bus. AXI (Advanced eXtensible Interface) is a bus protocol. This protocol is the most important part of the AMBA (Advanced Microcontroller Bus Architecture) 3.0 protocol proposed by ARM. It is an on-chip bus for high performance, high bandwidth, and low latency.

[0062] Specifically, in the first external memory, the result image is divided into a plurality of sub-blocks, which are used as fill sub-blocks. According to the rows and columns where the fill sub-blocks are located, they can be divided into odd column blocks and even column blocks. After the size of each fill sub-block is designed, it will not be changed. For example, Figure 2 As shown, the size of the resulting image is 1280*1024, the length of the resulting image is 1280 pixels, and the width is 1024 pixels; the size of the filling sub-block is 128*128, the length of the filling sub-block is 128 pixels, and the width is 128 pixels; the resulting image is divided into 10 filling sub-blocks with a length and 8 filling sub-blocks with a width, and the resulting image is divided into 80 filling sub-blocks in total.

[0063] Specifically, the second external memory is used to store a plurality of cutout sub-blocks, and the plurality of cutout sub-blocks constitute an original image.

[0064] Preferably, if Figure 1 As shown, the data acceleration processing system also includes:

[0065] A coordinate generator is used to generate the coordinates of any pixel point in the original image and the result image, and is used to generate the coordinates of any pixel point in the second preloaded sub-block.

[0066] It can be understood that the coordinate generator uses a two-dimensional coordinate system to generate coordinates for all pixel points in the result image and the original image.

[0067] It can be understood that a two-dimensional coordinate system is used to locate all pixel points of the result image and the original image.

[0068] The inverse matrix of the affine transformation in a two-dimensional coordinate system includes:

[0069] 1. The 3*3 control matrix of two-dimensional translation is:

[0070] In this matrix, tx and ty are the pixel distances of the translation, which correspond to the directions of the arrows of the coordinate axes of X and Y. For the inverse transformation, its positive translation is just opposite to the direction of the arrows of the coordinate axes, so it is represented by a negative sign.

[0071] 2. The 3*3 control matrix of two-dimensional rotation is:

[0072] In this matrix, θ represents the rotation angle, and the sign of the sine parameter of the matrix in the inverse transform is exactly opposite to that in the forward transform.

[0073] 3. The 3*3 control matrix for two-dimensional scaling is:

[0074] In this matrix, Sx and Sy are magnification factors. Since the magnification factor of the forward transformation is the reduction factor of the inverse transformation, the magnification factor here exists in the inverse form.

[0075] According to the inverse matrix of affine transformation, the image coordinates of the result image and the original image are rearranged. It is worth noting that the pixel coordinates of the result image are mapped to the coordinates of the original image, and then the pixel values ​​on the coordinates of the original image are moved to the pixel coordinates of the result image. The matrix operation expression is:

[0076]

[0077] Among them, the coordinates (u, v) are the coordinates of any pixel point in the result image, and the coordinates (x, y) are the corresponding coordinates in the original image obtained according to the inverse matrix of the affine transformation. is the inverse affine transformation matrix.

[0078] It is worth noting that the image transformation is achieved by changing the parameters in the control matrix. Of course, multiple deformation parameters can also be mixed to achieve the purpose of performing multiple deformations simultaneously in one affine transformation process.

[0079] Specifically, in the acceleration control module, a preloading step and an addressing and reading step are performed on the odd column block and the even column block based on a ping-pong operation; the preloading step is to store the pixel value data of the pixel points of the sub-block of the original image corresponding to the mapping sub-block (odd column block / even column block) divided in the first external memory result image into the corresponding BRAM space; the addressing and reading step is to fill the pixel value data stored in the corresponding BRAM space into each pixel point in the mapping sub-block (odd column block / even column block) divided in the result image, so that each pixel point in the mapping sub-block is filled with the corresponding pixel value data.

[0080] Preferably, the ping-pong operation method is used to implement the preloading step and the addressing and reading step for the odd column blocks and the even column blocks as follows:

[0081] Perform the preloading step and the addressing and reading step on the odd column blocks / even column blocks based on a ping-pong operation for all the mapping sub-blocks in the result image in a sequence from left to right and from top to bottom;

[0082] When the preloading step is performed on the odd column block / even column block, the addressing and reading step is performed on the even column block / odd column block at the same time.

[0083] Specifically, Figure 2 As shown, the result image is divided into 10 mapping sub-blocks with a length of 8 mapping sub-blocks and all the mapping sub-blocks divided in the result image are executed in order from left to right and from top to bottom. The result image includes a total of 40 odd column blocks and 40 even column blocks. In the first row of mapping sub-blocks, the 1st, 3rd, 5th, 7th and 9th mapping sub-blocks are odd column blocks, and the 2nd, 4th, 6th, 8th and 10th mapping sub-blocks are even column blocks; similarly, in the second row of mapping sub-blocks, the 11th mapping sub-block is an odd column block, and the 12th mapping sub-block is an even column block, and so on. It is worth noting that the execution order of all the mapping sub-blocks in the result in the present invention can also be in other orders, and all the mapping sub-blocks only need to be executed in a certain order.

[0084] Exemplary:

[0085] In the first execution cycle, the acceleration control module controls the preloading control module to perform a preloading step on the first mapping sub-block;

[0086] In the second execution cycle, the acceleration control module controls the preloading control module to perform a preloading step on the second mapping sub-block, and at the same time, the acceleration control module controls the mapping control module to perform an addressing and reading step on the first mapping sub-block;

[0087] In the third execution cycle, the acceleration control module controls the preloading control module to perform a preloading step on the third mapping sub-block, and at the same time, the acceleration control module controls the mapping control module to perform an addressing and reading step on the second mapping sub-block;

[0088] …

[0089] In the 80th execution cycle, the acceleration control module controls the preloading control module to perform a preloading step on the 80th mapping sub-block, and at the same time, the acceleration control module controls the mapping control module to perform an addressing and reading step on the 79th mapping sub-block;

[0090] In the 81st execution cycle, the acceleration control module controls the mapping control module to perform an addressing and reading step on the 80th mapping sub-block.

[0091] It is worth noting that the acceleration control module of the embodiment of the present invention controls the preloading control module and the mapping control module to use a ping-pong operation to perform preloading steps and addressing reading steps on the mapping sub-blocks divided from the result image, thereby accelerating the FPGA's processing speed of data, realizing the FPGA's rapid processing of data, and reducing processing delays.

[0092] Specifically, the preloading step is performed by the preloading control module, and the preloading step includes: determining the corresponding first preloading sub-image block in the original image according to the odd column block / even column block and the inverse matrix of the affine transformation, and obtaining the first preloading sub-image block from the second external memory, calculating the second preloading sub-image block based on the first preloading sub-image block, and storing the second preloading sub-image block corresponding to the odd column block in the first BRAM space, and storing the second preloading sub-image block corresponding to the even column block in the second BRAM space. Exemplary, as Figure 2 As shown, the second preloaded sub-picture blocks corresponding to the 1st, 3rd, 5th, 7th ... 79th mapping sub-picture blocks are stored in the first BRAM space, and the second preloaded sub-picture blocks corresponding to the 2nd, 4th, 6th, 8th ... 80th mapping sub-picture blocks are stored in the second BRAM space.

[0093] It is worth noting that the size design of the first BRAM space and the second BRAM space is the same. The first BRAM space and the second BRAM space are two independent BRAM spaces, and both belong to the on-chip BRAM space of the FPGA.

[0094] Preferably, determining the corresponding first preloaded sub-image block in the original image according to the odd column block / even column block and the inverse affine transformation matrix includes:

[0095] A cutout sub-image block is determined according to the odd column block / even column block and an inverse affine transformation matrix, and a first preloaded sub-image block is determined according to the cutout sub-image block.

[0096] Specifically, a cutout sub-block is calculated according to the odd column block / even column block and the inverse matrix of the affine transformation, the cutout sub-block is a sub-block in the original image, and the cutout sub-block is a sub-block corresponding to the fill sub-block in the original image.

[0097] Preferably, determining the cutout sub-block according to the odd column block / even column block and an inverse affine transformation matrix comprises:

[0098] Determine the mapping coordinates corresponding to the odd column block / even column block; the mapping coordinates include four mapping vertex coordinates corresponding to the odd column block / even column block;

[0099] Based on the inverse affine transformation, determining the cutout coordinates of the cutout sub-block according to the four mapping vertex coordinates and the inverse affine transformation matrix, wherein the cutout coordinates include four cutout vertex coordinates;

[0100] The cutout sub-block is determined according to the coordinates of the four cutout vertices.

[0101] Specifically, Figure 3 As shown, the mapping sub-block is any sub-block in the result image. The mapping sub-block can be an odd-column block or an even-column block. Combined with the two-dimensional coordinate system of the result image, the corresponding four mapping vertex coordinates Q1, Q2, Q3 and Q4 can be determined for the mapping sub-block.

[0102] Based on the inverse affine transformation, according to the four filling vertex coordinates Q1, Q2, Q3 and Q4, the corresponding four cutout vertex coordinates Q1', Q2', Q3' and Q4' in the original image are determined by the inverse affine transformation matrix calculation, as shown in Figure 3 As shown, the cutout sub-block is the sub-block corresponding to the fill sub-block in the original image.

[0103] It is worth noting that the DDR configured externally by the FPGA is read in burst length. If the pixel data corresponding to the cutout sub-block is to be directly extracted from the DDR, a large amount of timing waste will be generated. Therefore, it is necessary to determine the first preloaded sub-block according to the cutout sub-block of the original image. The first preloaded sub-block is the pixel value data that needs to be stored from the second external memory DDR to the BRAM space.

[0104] Preferably, determining the first preloaded sub-picture block according to the cutout sub-picture block includes:

[0105] According to the coordinates of the four cutout vertices of the cutout sub-block, determining the maximum value and the minimum value of the cutout sub-block on two coordinate axes;

[0106] Determine the coordinates of four vertices of the first preloaded sub-tile according to the maximum and minimum values ​​on the two coordinate axes:

[0107]

[0108] Among them, A, B, C and D represent the four vertex coordinates of the first preloaded sub-block, Xmax represents the maximum value of the cutout sub-block on the X-axis, Xmin represents the minimum value of the cutout sub-block on the X-axis, Ymax represents the maximum value of the cutout sub-block on the Y-axis, and Ymin represents the minimum value of the cutout sub-block on the Y-axis.

[0109] Specifically, Figure 3As shown, the four vertex coordinates Q1', Q2', Q3' and Q4' on the X-axis have a maximum value of Xmax and a minimum value of Xmin; the maximum value on the Y-axis is Ymax and the minimum value is Ymin. The four vertex coordinates of the first preloaded sub-block are It is worth noting that when reading the pixel value of a pixel point from the DDR, the coordinates of each pixel point are integers.

[0110] It can be understood that the rectangle enclosed by the first preloaded sub-image block includes the cutout area formed by the cutout sub-image block, and the four vertex coordinates of the first preloaded sub-image block are all integers.

[0111] Preferably, the step of calculating the second preloaded sub-image block based on the first preloaded sub-image block includes:

[0112] Determine whether the size of the first preloaded sub-image block is larger than the size of the first BRAM space / the second BRAM space; if so, determine a pre-reduction ratio parameter according to the size of the first preloaded sub-image block and the first BRAM space / the second BRAM space, and reduce the first preloaded sub-image block based on the pre-reduction ratio parameter to obtain a second preloaded sub-image block; if not, use the first preloaded sub-image block as the second preloaded sub-image block.

[0113] Specifically, the size of the first BRAM space and the second BRAM space are designed to be the same, and the size of the BRAM space is generally designed to be 1.5-2 times the size of the mapping sub-block. For example, the size of the mapping sub-block is 128*128, and the size of the BRAM space is designed to be 256*256. Once the size design of the BRAM space is determined, it will not be changed.

[0114] It is worth noting that when the original image is reduced, when pixel data is loaded from the original image according to the filling sub-block of the result image, a larger area of ​​the original image may need to be loaded, and the designed BRAM space may not be sufficient to place the required pixel data of the original image, resulting in data execution failure. Therefore, before storing the first preloaded sub-block into the BRAM space, the present invention compares the size of the first preloaded sub-block with the size of the BRAM space.

[0115] like Figure 3As shown, when the size of the first preloaded sub-block is larger than the size of the corresponding first BRAM space / second BRAM space, the first preloaded sub-block is reduced according to the pre-reduction ratio parameter to obtain the second preloaded sub-block, so that the first BRAM space / second BARM space is larger than the second preloaded sub-block, and the second preloaded sub-block can be placed normally in the first BRAM space / second BRAM space, so that the data is executed successfully.

[0116] like Figure 4 As shown, when the size of the first preloaded sub-image block is smaller than or equal to the size of the first BRAM space / the second BRAM space, the first preloaded sub-image block is directly used as the second preloaded sub-image block.

[0117] Preferably, the determining whether the size of the first preloaded sub-tile is larger than the size of the first BRAM space / the second BRAM space includes:

[0118] Determine the length and width of the first preloaded sub-tile and determine the length and width of the first BRAM space / second BRAM space;

[0119] Calculating a length ratio of the first preloaded sub-image block to a length of the first BRAM space / second BRAM space and calculating a width ratio of the first preloaded sub-image block to a width of the first BRAM space / second BRAM space; the first BRAM space and the second BRAM space have the same size;

[0120] If any one of the length ratio and the width ratio is greater than 1, it is judged as yes, otherwise, it is judged as no.

[0121] Specifically, the design size of the first BRAM space and the second BRAM space is the same, and the size of the first BRAM space and the second BRAM space is fixed, such as L*W; L is the length, usually 256 pixels, and W is the width, usually 256 pixels. The size of the first preloaded sub-block is calculated based on the four vertex coordinates A, B, C and D of the first preloaded sub-block. The length of the first preloaded sub-block is pixels, and a width of pixels.

[0122] Specifically, the length ratio of the first preloaded sub-block to the length of the first BRAM space / the second BRAM space is The ratio of the width of the first preloaded sub-tile to the width of the first BRAM space / the width of the second BRAM space is

[0123] Specifically, compare the length ratios Compare the width ratio with 1 and 1. If either the length ratio or the width ratio is greater than 1, it is judged as yes, otherwise, it is judged as no.

[0124] Specifically, the pre-reduction ratio parameter is determined according to the size of the first preloaded sub-tile and the first BRAM space / the second BRAM space.

[0125] Preferably, determining the pre-reduction ratio parameter according to the size of the first preloaded sub-tile and the first BRAM space / the second BRAM space includes:

[0126] Determine whether the length ratio is greater than the width ratio; if so, determine the pre-reduction ratio parameter according to the length ratio; otherwise, determine the pre-reduction ratio parameter according to the width ratio; the pre-reduction ratio parameter is N / M, wherein N and M are both integers of [1,6], and M>N.

[0127] Specifically, determine the length ratio Ratio of width to If the length ratio Greater than width ratio According to the length ratio Determine the pre-reduction ratio parameters; if the length ratio Less than or equal to width ratio According to the width ratio Determine the pre-reduction ratio parameters.

[0128] Preferably, the pre-reduction ratio parameter is N / M, wherein N and M are both integers of [1,6], and M>N.

[0129] It is worth noting that when the length ratio Greater than width ratio When , the inverse of the length ratio is taken, and then the inverse value is simplified into a fractional form, and the score value is made as close to the inverse value as possible without being greater than the inverse value. The fractional value is the pre-reduction ratio parameter N / M, so that N and M are both integers [1,6], and M>N. Similarly, when the length ratio is Less than or equal to width ratio When , take the reciprocal of the width ratio, then simplify the reciprocal value into a fractional form, and make the score value as close to the reciprocal value as possible without being greater than the reciprocal value. The fractional value is the pre-reduction ratio parameter N / M, so that N and M are both integers in [1,6], and M>N.

[0130] For example, when the calculated length ratio is 1.6 and the width ratio is 1.2, the pre-reduction ratio parameter determined according to the length ratio can be 3 / 5. Specifically, the reciprocal of the length ratio is first taken as 1 / 1.6=0.625, and then integers N and M are selected so that N / M is not greater than 0.625 and as close to 0.625 as possible, so N is selected as 3 and M is selected as 5, that is, the reciprocal value is simplified to the fraction 3 / 5, and the pre-reduction ratio parameter is obtained as 3 / 5.

[0131] Preferably, the step of reducing the first preloaded sub-image block based on the pre-reduction ratio parameter to obtain the second preloaded sub-image block includes:

[0132] Inserting N-1 pixels between two adjacent pixels in each row of the first preloaded sub-image block, and then inserting N-1 pixels between two adjacent pixels in each column of the first preloaded sub-image block, thereby expanding the first preloaded sub-image block by N times to obtain an intermediate sub-image block;

[0133] In the middle sub-image block, one row is selected as a row to be sampled every M-1 rows, and in each row to be sampled, one pixel point is selected as a sampling point every M-1 points, thereby reducing the middle sub-image block by M times to obtain a second preloaded sub-image block.

[0134] Specifically, Figure 5 As shown, exemplary: when N is 2, 1 pixel is inserted between two adjacent pixels in each row of the first preloaded sub-block, and then 1 pixel is inserted between two adjacent pixels in each column of the first preloaded sub-block, thereby expanding the first preloaded sub-block by 2 times to obtain an intermediate sub-block; when N is 3, 2 pixels are inserted between two adjacent pixels in each row of the first preloaded sub-block, and then 2 pixels are inserted between two adjacent pixels in each column of the first preloaded sub-block, thereby expanding the first preloaded sub-block by 3 times to obtain an intermediate sub-block. The original pixel represents the pixel in the first preloaded sub-block, and the interpolated pixel represents the inserted pixel.

[0135] Preferably, when inserting a pixel point in the first preloaded sub-image block, the expression of the pixel value of the interpolated pixel is:

[0136] Dix_j=DixJ-j[(DixJ-DixK) / N];

[0137] Among them, DixJ and DixK respectively represent the pixel values ​​of two adjacent pixel points J and K in the first preloaded sub-block, Dix_j represents the pixel value of the jth pixel point among the N-1 pixel points inserted sequentially from pixel point J to pixel point K, 1≤j≤(N-1).

[0138] Specifically, Figure 5 As shown, exemplarily: when N is 2, in the first row of the first preloaded sub-block, one pixel is inserted between two adjacent original pixels, from left to right, the pixel values ​​of the two adjacent original pixels are 20 and 30 respectively, then the pixel value of the inserted pixel is 20-1*[(20-30) / 2], that is, the pixel value of the inserted pixel is 25; when N is 3, in the first row, two pixels are inserted between two adjacent original pixels, from left to right, the pixel values ​​of the two adjacent original pixels are 20 and 30 respectively, then the pixel values ​​of the inserted pixel are 20-1*[(20-30) / 3] and 20-2*[(20-30) / 3], that is, the pixel values ​​of the inserted two pixels are 23 and 26.

[0139] Specifically, Figure 5 As shown, exemplarily: when N is 2, in the first column of the first preloaded sub-block, one pixel is inserted between two adjacent original pixels, and from top to bottom, the pixel values ​​of the two adjacent original pixels are 30 and 40 respectively, then the pixel value of the inserted pixel is 30-1*[(30-40) / 2], that is, the pixel value of the inserted pixel is 35; when N is 3, in the first column of the first preloaded sub-block, two pixels are inserted between two adjacent original pixels, and from top to bottom, the pixel values ​​of the two adjacent original pixels are 30 and 40 respectively, then the pixel values ​​of the inserted pixel are 30-1*[(30-40) / 3] and 30-2*[(30-40) / 3], that is, the pixel values ​​of the two inserted pixels are 33 and 36.

[0140] Specifically, Figure 6 As shown, exemplary: when M is 2, in the middle sub-block, one row is selected as the row to be sampled every other row, and in each row to be sampled, one pixel is selected as the sampling point every other point, thereby reducing the middle sub-block by 2 times to obtain the second preloaded sub-block; when M is 3, in the middle sub-block, one row is selected as the row to be sampled every other row, and in each row to be sampled, one pixel is selected as the sampling point every other point, thereby reducing the middle sub-block by 3 times to obtain the second preloaded sub-block. The adopted pixel represents the pixel of the adopted point, and the discarded pixel represents the pixel point that is not sampled.

[0141] Compared with the prior art, the data acceleration processing system based on affine inverse transformation of FPGA provided by the embodiment of the present invention performs preloading steps and addressing and reading steps on the mapping sub-blocks divided by the result image through ping-pong operation, thereby accelerating the processing speed of FPGA for data, realizing fast processing of data by FPGA, and reducing processing delay; by judging whether the size of the original image data to be loaded is larger than the size of the BRAM space; if it is judged to be, the original image data is reduced according to the size of the original image data and the size of the BRAM space, and the reduced original image data is saved to the BRAM space, thereby solving the technical problem that the affine reduction transformation cannot be realized due to the original image being too large and the BRAM space being too small during the affine transformation process; the original image data to be loaded is determined based on the affine inverse transformation through the mapping coordinates of the mapping sub-block, thereby improving the accuracy of calculating the original image data to be loaded; the pre-reduction ratio parameter is further determined according to the length ratio and width ratio of the original image data to be loaded and the BRAM space, so that when the original image data is reduced, the reduction range is guaranteed and the loss of the original image data is reduced.

[0142] Specifically, the BRAM space includes a first BRAM space and a second BRAM space, which are respectively used to store the second preloaded sub-image blocks corresponding to the odd column blocks and the even column blocks.

[0143] Preferably, if Figure 1 As shown, the data acceleration processing system also includes:

[0144] A buffer register is used to cache the pixel values ​​in the first preloaded sub-image block sent by the second external memory, and transmit the corresponding pixel values ​​to the preload control module according to the control instruction of the preload control module, and the preload control module obtains the pixel value of each point of the second preloaded sub-image block according to the acquired pixel value of the first preloaded sub-image block.

[0145] That is, when the size of the first preloaded sub-block is larger than the size of the BRAM space, the first preloaded sub-block is interpolated and sampled at alternate points to obtain the second preloaded sub-block; when the size of the first preloaded sub-block is not larger than the size of the BRAM space, the pixels of the first preloaded sub-block are directly used as the pixels of the second preloaded sub-block, and the pixel values ​​of the pixels in the second preloaded sub-block are stored in the BRAM space.

[0146] In the mapping control module, the addressing and reading step includes: traversing each pixel point in the odd column block / even column block in turn, determining the pixel value of each pixel point according to the inverse affine transformation matrix and the second preloaded sub-block in the first BRAM space / second BRAM space corresponding to the odd column block / even column block, and sending the pixel value to the first external memory for storage.

[0147] Preferably, determining the pixel value of each pixel point according to the inverse affine transformation matrix and the second preloaded sub-image block in the first BRAM space or the second BRAM space corresponding to the odd column block / even column block includes:

[0148] The coordinates of the corresponding pixel in the original image are obtained according to the coordinates of each pixel and the inverse matrix of the affine transformation, and the first coordinates of the first preloaded sub-block are subtracted from the coordinates of the pixel in the original image and then multiplied by the pre-reduction ratio parameter to obtain the first coordinates;

[0149] The pixel value of each corresponding pixel point is determined according to the first coordinate.

[0150] Specifically, for any pixel point in the mapping sub-block with coordinates (u, v), the coordinates of the corresponding original image are obtained by calculating the inverse matrix of the radial transformation (u', v'). It is worth noting that (u, v) and (u', v') are both coordinate points in the same coordinate system. It can be understood that the coordinate system in the BRAM space is different from the above coordinate system, and the corresponding coordinate (u'v') needs to be subtracted from the first coordinate of the first preloaded sub-block corresponding to the mapping sub-block, so that the corresponding coordinate point (u', v') is converted into a coordinate point (u", v") in the BRAM space coordinate system; according to the pre-reduction ratio parameter N / M, the first coordinate corresponding to the pixel point (u, v) in the BRAM space is obtained as (u"*N / M, v"*N / M).

[0151] It can be understood that the coordinate systems of the first BRAM space and the second BRAM space are the same.

[0152] Specifically, determining the pixel value of each corresponding pixel point according to the first coordinate includes:

[0153] Let P(x, y) denote the first coordinate;

[0154] When x and y are both integers, the pixel value of point P is used as the pixel value of each corresponding pixel point;

[0155] When both x and y are not integers, four second coordinates around point P are determined according to point P, and pixel values ​​of the four second coordinates are determined by addressing the BRAM space; based on a bilinear interpolation algorithm, the pixel value corresponding to the first coordinate is calculated through the pixel values ​​of the four second coordinates, and the pixel value corresponding to the first coordinate is used as the pixel value of each pixel point;

[0156] When only one of x and y is an integer, two third coordinates around point P are determined based on point P, and the pixel values ​​of the two third coordinates are determined by addressing the BRAM space; based on a bilinear interpolation algorithm, the pixel value corresponding to the first coordinate is calculated through the pixel values ​​of the two third coordinates, and the pixel value corresponding to the first coordinate is used as the corresponding pixel value of each pixel point.

[0157] Specifically, when x and y are both integers, it means that the first coordinate P (x, y) is a pixel point. At this time, the pixel value corresponding to the point P in the BRAM space is directly taken out as the pixel value of the pixel point in the mapping sub-block.

[0158] When only one of x and y is an integer, for example, Figure 7 As shown, y is an integer and x is not an integer, indicating that the first coordinate P (x, y) is between the integer coordinates of two adjacent pixel points. At this time, two third coordinates P1 and P2 are determined according to point P, and the pixel values ​​corresponding to the third coordinates P1 and P2 in the BRAM space are taken out DixP1 and DixP2. Then the pixel value corresponding to the first coordinate point P is DixP = DixP1*k1+Dix*k2, where The pixel value corresponding to point P is used as the pixel value of the pixel point in the mapping sub-block.

[0159] When both x and y are not integers, such as Figure 8 As shown, it means that the first coordinate P (x, y) is between four adjacent pixel points. At this time, four second coordinates P1, P2, P3 and P4 are determined according to point P, and the pixel values ​​of the second coordinates P1, P2, P3 and P4 in the BRAM space are taken out as DixP1, DixP2, DixP3 and DixP4. Then the pixel value corresponding to the first coordinate point P is DixP = (DixP1*k1+DixP2*k3+DixP3*k1+DixP4*k3+DixP1*k4+DixP3*k2+DixP2*k4+DixP4*k2) / 4, where The pixel value corresponding to point P is used as the pixel value of the pixel in the mapping sub-block. To round down, To round up.

[0160] After the pixel values ​​in each mapping sub-block are obtained in turn according to the above method, all the mapping sub-blocks form a result image.

[0161] Compared with the prior art, the data acceleration processing system based on FPGA affine inverse transformation provided by the embodiment of the present invention determines the first coordinate corresponding to the BRAM space through any pixel point in the mapping sub-block, determines the pixel value corresponding to the first coordinate through different situations of the first coordinate, and uses the pixel value of the first coordinate as the pixel value of the corresponding pixel point in the mapping sub-block.

[0162] Those skilled in the art will appreciate that all or part of the processes of the above embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0163] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A data acceleration processing system based on FPGA affine inverse transformation, characterized in that: The data acceleration processing system comprises: A first external memory is used to store a plurality of mapping sub-blocks, wherein the mapping sub-blocks are sub-blocks divided according to the result image, and the mapping sub-blocks are divided into odd column blocks and even column blocks; A second external memory for storing original images; An acceleration control module, used for controlling the preloading control module and the mapping control module to implement the preloading step and the addressing and reading step on the odd column blocks and the even column blocks in a ping-pong operation manner; a preloading control module, configured to execute the preloading step, wherein the preloading step comprises: determining a corresponding first preloading sub-image block in the original image according to the odd column block / even column block and an inverse matrix of an affine transformation, obtaining the first preloading sub-image block from the second external memory, calculating a second preloading sub-image block based on the first preloading sub-image block, and storing the second preloading sub-image block corresponding to the odd column block in a first BRAM space, and storing the second preloading sub-image block corresponding to the even column block in a second BRAM space; A mapping control module, configured to execute the addressing and reading step, wherein the addressing and reading step comprises: sequentially traversing each pixel point in the odd column block / even column block, determining a pixel value of each pixel point according to the inverse affine transformation matrix and the second preloaded sub-image block in the first BRAM space / second BRAM space corresponding to the odd column block / even column block, and sending the pixel value to the first external memory for storage; The BRAM space includes the first BRAM space and the second BRAM space, which are respectively used to store the second preloaded sub-image blocks corresponding to the odd column blocks and the even column blocks.

2. The data acceleration processing system according to claim 1, characterized in that: The data acceleration processing system also includes: A coordinate generator is used to generate the coordinates of any pixel point in the original image and the result image, and is used to generate the coordinates of any pixel point in the second preloaded sub-block.

3. The data acceleration processing system according to claim 2, characterized in that: The data acceleration processing system also includes: A buffer register is used to cache the pixel values ​​in the first preloaded sub-image block sent by the second external memory, and transmit the corresponding pixel values ​​to the preload control module according to the control instruction of the preload control module, and the preload control module obtains the pixel value of each point of the second preloaded sub-image block according to the acquired pixel value of the first preloaded sub-image block.

4. The data acceleration processing system according to claim 3, characterized in that: The determining a corresponding first preloaded sub-image block in the original image according to the odd column block / even column block and an inverse affine transformation matrix includes: A cutout sub-image block is determined according to the odd column block / even column block and an inverse matrix of affine transformation, and a first preloaded sub-image block is determined according to the cutout sub-image block.

5. The data acceleration processing system according to claim 4, characterized in that: The step of calculating the second preloaded sub-picture block based on the first preloaded sub-picture block includes: Determine whether the size of the first preloaded sub-image block is larger than the size of the first BRAM space / the second BRAM space; if so, determine a pre-reduction ratio parameter according to the size of the first preloaded sub-image block and the first BRAM space / the second BRAM space, and reduce the first preloaded sub-image block based on the pre-reduction ratio parameter to obtain a second preloaded sub-image block; if not, use the first preloaded sub-image block as the second preloaded sub-image block.

6. The data acceleration processing system according to claim 3, characterized in that: The determining the pixel value of each pixel point according to the inverse affine transformation matrix and the second preloaded sub-image block in the first BRAM space or the second BRAM space corresponding to the odd column block / even column block includes: The coordinates of the corresponding pixel in the original image are obtained according to the coordinates of each pixel and the inverse matrix of the affine transformation, and the first coordinates of the first preloaded sub-block are subtracted from the coordinates of the pixel in the original image and then multiplied by the pre-reduction ratio parameter to obtain the first coordinates; The pixel value of each corresponding pixel point is determined according to the first coordinate.

7. The data acceleration processing system according to claim 3, characterized in that: The ping-pong operation method is used to implement the preloading step and the addressing and reading step for the odd column block and the even column block. Specifically, the ping-pong operation method is as follows: Perform the preloading step and the addressing and reading step on the odd column blocks / even column blocks based on a ping-pong operation for all the mapping sub-blocks in the result image in a sequence from left to right and from top to bottom; When the preloading step is performed on the odd column block / even column block, the addressing and reading step is performed on the even column block / odd column block at the same time.

8. The data acceleration processing system according to claim 4, characterized in that: The determining the cutout sub-block according to the odd column block / even column block and the inverse matrix of the affine transformation comprises: Determine the mapping coordinates corresponding to the odd column block / even column block; the mapping coordinates include four mapping vertex coordinates corresponding to the odd column block / even column block; Based on the inverse affine transformation, determining the cutout coordinates of the cutout sub-block according to the four mapping vertex coordinates and the inverse affine transformation matrix, wherein the cutout coordinates include four cutout vertex coordinates; The cutout sub-block is determined according to the coordinates of the four cutout vertices.

9. The data acceleration processing system according to claim 5, characterized in that: The determining whether the size of the first preloaded sub-tile is greater than the size of the first BRAM space / the second BRAM space includes: Determine the length and width of the first preloaded sub-tile and determine the length and width of the first BRAM space / second BRAM space; Calculating a length ratio of the first preloaded sub-image block to a length of the first BRAM space / second BRAM space and calculating a width ratio of the first preloaded sub-image block to a width of the first BRAM space / second BRAM space; the first BRAM space and the second BRAM space have the same size; If any one of the length ratio and the width ratio is greater than 1, it is judged as yes, otherwise, it is judged as no.

10. The data acceleration processing system according to claim 9, characterized in that: The determining of the pre-reduction ratio parameter according to the size of the first preloaded sub-tile and the first BRAM space / the second BRAM space includes: Determine whether the length ratio is greater than the width ratio; if so, determine the pre-reduction ratio parameter according to the length ratio; otherwise, determine the pre-reduction ratio parameter according to the width ratio; the pre-reduction ratio parameter is N / M, wherein N and M are both integers of [1,6], and M>N.

Images