A hardware implementation method for affine transformation

Abstract

The present invention provides a hardware implementation method of affine transformation, which includes the following steps: S1, reading an image to be identified, wherein a target image and a source image are stored in a DDR, and the source image is sequentially input from a bus; S2, after radial changes, the target image is output in a disordered order; S3, the target points are sorted according to the input order of the source points by using the cache, and the disordered order is converted into a sequence, wherein the step of obtaining a horizontal group of the target point closest to the source point in each vertical group is optimized by using the one-to-one correspondence and monotonicity between the target row and the source row; S4, bus output. The method provides convenient hardware design and implementation. It can support full-function and full-range affine transformations. Under the same hardware area, the size of the supported target image is larger.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a hardware implementation method of affine transformation. Background Art

[0002] In artificial intelligence technology, image processing technology is also very important. Especially in life, we often hear about face recognition, license plate recognition, mask recognition, etc., which are all in the category of image recognition. Image recognition is one of the most mature directions of artificial intelligence technology. Before entering the neural network for recognition, the image needs to be processed to reduce the cost of recognition and improve the accuracy of recognition. Affine operation is a commonly used pre-processing algorithm, such as Figure 1 As shown in Figure 2. Common coordinate changes of two-dimensional images are as follows: Figure 2 As shown. Affine transformations include Figure 3 All transformations shown, and any combination of these transformations in any order.

[0003] Affine transformation actually describes the correspondence between target pixels and source pixels. This correspondence is easy to implement with software, but it is a disaster for integrated circuit hardware implementation. However, recognition efficiency is the biggest problem of software implementation. In public places, there may be hundreds or thousands of faces, and software face recognition is obviously impossible. Therefore, it is a general trend to implement affine transformation with integrated circuit hardware.

[0004] When implementing affine transformation in integrated circuit hardware, both the source image and the target image are very large and need to be stored in DDR. The efficiency of DDR usage becomes a difficulty in hardware implementation of affine transformation. Figure 4 As shown in the figure, to solve the problem of DDR usage efficiency, there are two solutions for hardware implementation of affine transformation:

[0005] 1) Bus input, use buffer to change the order into random order, the source image is input in random order, and the target image is output to the bus in order.

[0006] 2) The source image is input sequentially from the bus, and the target image is output in random order. Then a buffer is used to convert the random order into sequential order and output from the bus.

[0007] When the second solution is adopted, the target points need to be sorted according to the input order of the source points. If the target image size is too large, it will be disastrous for hardware implementation.

[0008] In order to reduce the complexity of target point sorting, existing integrated circuit hardware implementation solutions do not have full functional implementation. The main limitations are as follows:

[0009] 1) Only supports small size target images.

[0010] 2) Only certain transformations are supported, for example, only scaling is supported but not rotation.

[0011] 3) Only supports a specific range of transformations. For example: only supports scaling at a specific ratio, only supports rotation at a specific angle.

[0012] In addition, the commonly used terms in the prior art are as follows:

[0013] 1) Artificial intelligence.

[0014] 2) Image recognition.

[0015] 3) Image data format: including BGRA, NV12 format, etc.

[0016] 4) Affine transformation.

[0017] 5) Linear interpolation algorithm.

[0018] 6) Integrated circuit hardware design (IC). Summary of the invention

[0019] In order to solve the above problems, the purpose of this method is to propose a hardware implementation method of affine transformation, provide convenient hardware design and implementation, support full-function and full-range affine transformation, and support larger target graph size under the same hardware area.

[0020] Specifically, the present invention provides a hardware implementation method of affine transformation, the method comprising the following steps:

[0021] S1, read the image to be recognized, the target image and the source image are stored in DDR, and the source image is input sequentially from the bus;

[0022] S2, after affine transformation, the target image is output in random order;

[0023] S3, then use the cache to sort the target points according to the input order of the source points, and turn the disorder into order. Among them, the step of obtaining a horizontal group by finding the target point closest to the source point in each vertical group is optimized by using the one-to-one correspondence and monotonicity between the target row and the source row;

[0024] S4, bus output.

[0025] The target point sorting in S3 further includes:

[0026] S3.1: Group the target graph vertically.

[0027]

[0028] S3.2: Find the target point closest to the source point in each vertical group to obtain a horizontal group.

[0029] d0 d1 d2 … d(w-1) ;

[0030] S3.3: Find the target point closest to the source point in the horizontal group, and write the coordinates of this target point into the first-in-first-out order.

[0031] d[j][i] ;

[0032] S3.4: Remove the target points obtained in S3.3 and repeat S3.1-S3.3 until all target points are sorted.

[0033] In the method, the image before affine transformation is called source image, and the image after transformation is called target image. The source image is used as the function value and the target image is used as the independent variable. Then, the affine transformation formula is as follows:

[0034] (float)sx=(float)m0*(int)dx+(float)m2*(int)dy+(float)m4;

[0035] (float)sy=(float)m1*(int)dy+(float)m3*(int)dx+(float)m5; in the formula:

[0036] 1) m0-m5 are affine transformation coefficients, which are floating point types.

[0037] 2) The independent variable (dx, dy) is the coordinate of the target point, which is an integer;

[0038] 3) The function value (sx, sy) is the source point coordinate, which is a floating point type, where the integer part is the location information and the decimal part is the weight.

[0039] In the sorting process of step S3, according to the vertical scaling rule, the one-to-one correspondence between the target row and the source row is as follows:

[0040] When abs(m1), i.e. the absolute value of m1, is greater than 1, if the vertical scaling is reduction, the target row and the source row are in one-to-one correspondence;

[0041] When abs(m1), i.e. the absolute value of m1, is equal to 1, if the vertical scaling is unchanged, the target row and the source row are in one-to-one correspondence;

[0042] When abs(m1), i.e., the absolute value of m1, is less than 1, if the vertical scaling is enlargement, the target row and the source row are not one-to-one corresponding;

[0043] The monotonicity relationship is as follows:

[0044] If m1 is positive, the relationship between dy and sy is monotonically decreasing;

[0045] If m1 is negative, the relationship between dy and sy is monotonically increasing.

[0046] The steps S3.2 and S3.3 are functionally the same, both of which seek the target point closest to the source point, but the amount of computation in S2 is w times the width of the target graph in S3. Therefore, the optimization using the one-to-one correspondence and monotonicity between the target row and the source row solves the one-to-one correspondence problem of vertical magnification. The source graph is first vertically magnified by 2 to the power of n, and then an affine transformation is performed.

[0047] The source Figure 2 Power multiple amplification is very easy to implement in hardware design. The original formula is:

[0048] sx=m0*dx+m2*dy+m4;

[0049] sy=m1*dy+m3*dx+m5;

[0050] After decomposition, the first formula is:

[0051] sx=dx;

[0052] sy=1 / k*dy;

[0053] After decomposition, the second formula:

[0054] sx=m0*dx+m2*dy+m4;

[0055] sy=m1*k*dy+m3*k*dx+m5*k;

[0056] Here k needs to be a power of 2, and Abs(m1*k), i.e. the absolute value of m1*k, needs to be greater than 1.

[0057] Therefore, the advantages of the present application are: the method of the present invention provides convenient hardware design and implementation. It can support full-function and full-range affine transformations. Under the same hardware area, the supported target graph size is larger. The monotonicity and one-to-one correspondence between the target row and the source row are utilized to simplify the sorting of target points. It is more friendly to hardware design. It makes the full-function and full-range hardware design of affine transformation possible. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.

[0059] Figure 1 It is a schematic diagram of an image recognition preprocessing stage including affine transformation in the prior art.

[0060] Figure 2 This is a schematic diagram of common coordinate changes in two-dimensional images.

[0061] Figure 3 This is a schematic diagram of an affine transformation including the transformation shown in the figure.

[0062] Figure 4 When the integrated circuit hardware implements affine transformation, both the source image and the target image need to be placed in DDR, corresponding to the schematic diagram of the use of DDR.

[0063] Figure 5 It is a schematic diagram of the one-to-one correspondence between target rows and source rows according to the vertical scaling rule.

[0064] Figure 6 It is a schematic flow diagram of the method of the present invention. DETAILED DESCRIPTION

[0065] In order to more clearly understand the technical content and advantages of the present invention, the present invention is now further described in detail in conjunction with the accompanying drawings.

[0066] like Figure 6 As shown, a hardware implementation method of an affine transformation of the present invention comprises the following steps:

[0067] S1, read the image to be recognized, the target image and the source image are stored in DDR, and the source image is input sequentially from the bus; Figure 4 As shown;

[0068] S2, after affine transformation, the target image is output in random order;

[0069] S3, sorting the target points according to the input order of the source points, and converting the disordered order into order, wherein the step of obtaining a horizontal group by finding the target point closest to the source point in each vertical group is optimized by using the one-to-one correspondence and monotonicity between the target row and the source row;

[0070] S4, bus output.

[0071] The target point sorting in S3 further includes:

[0072] S3.1: Group the target graph vertically.

[0073]

[0074] S3.2: Find the target point closest to the source point in each vertical group to obtain a horizontal group.

[0075] d0 d1 d2 … d(w-1) ;

[0076] S3.3: Find the target point closest to the source point in the horizontal group, and write the coordinates of this target point into the first-in-first-out order.

[0077] d[j][i] ;

[0078] S3.4: Remove the target points obtained in S3.3 and repeat S3.1-S3.3 until all target points are sorted.

[0079] In the method, the image before affine transformation is called source image, and the image after transformation is called target image. The source image is used as the function value and the target image is used as the independent variable. Then, the affine transformation formula is as follows:

[0080] (float)sx=(float)m0*(int)dx+(float)m2*(int)dy+(float)m4;

[0081] (float)sy=(float)m1*(int)dy+(float)m3*(int)dx+(float)m5;

[0082] In the formula:

[0083] 1) m0-m5 are affine transformation coefficients, which are floating point types.

[0084] 2) The independent variable (dx, dy) is the coordinate of the target point, which is an integer;

[0085] 3) The function value (sx, sy) is the source point coordinate, which is a floating point type, where the integer part is the location information and the decimal part is the weight.

[0086] In the above process, S3.2 and S3.3 are functionally the same, both of which are to find the target point closest to the source point. However, the amount of computation of S3.2 is w (the width of the target graph) times that of S3.3. Therefore, the optimization of S3.2 is the key to hardware implementation.

[0087] This application optimizes S3.2 by utilizing the one-to-one correspondence and monotonicity between target rows and source rows.

[0088] In the sorting process of step S3, according to the vertical scaling rule, the one-to-one correspondence between the target row and the source row is as follows:

[0089] abs(m1) Vertical scaling Do the target rows and source rows correspond one to one? >1 Zoom out yes ＝＝1 constant yes <1 enlarge no

[0090] A simple schematic diagram is Figure 5 shown.

[0091] The monotonicity relationship is as follows:

[0092]

[0093]

[0094] It can be seen from the above diagram that solving the one-to-one correspondence problem of longitudinal enlargement is the key to the problem and is also the focus of the present invention.

[0095] Therefore, the optimization using the one-to-one correspondence and monotonicity between the target row and the source row solves the one-to-one correspondence problem of vertical magnification. The source image is firstly vertically magnified by 2 to the power of n, and then an affine transformation is performed.

[0096] The source Figure 2 Power multiple amplification is very easy to implement in hardware design. The original formula is:

[0097] sx=m0*dx+m2*dy+m4;

[0098] sy=m1*dy+m3*dx+m5;

[0099] After decomposition, the first formula is:

[0100] sx=dx;

[0101] sy=1 / k*dy;

[0102] After decomposition, the second formula:

[0103] sx=m0*dx+m2*dy+m4;

[0104] sy=m1*k*dy+m3*k*dx+m5*k;

[0105] Here k needs to be a power of 2, and Abs(m1*k), i.e. the absolute value of m1*k, needs to be greater than 1.

[0106] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for hardware implementation of affine transformation, characterized in that: The method comprises the following steps: S1, read the image to be recognized, the target image and the source image are stored in DDR, and the source image is input sequentially from the bus; S2, after affine transformation, the target image is output in random order; in the method, the image before affine transformation is called source image, and the image after transformation is called target image. The source image is used as the function value and the target image is used as the independent variable. The affine transformation formula is as follows: (float)sx=(float)m0*(int)dx+(float)m2*(int)dy+(float)m4; (float)sy=(float)m1*(int)dy+(float)m3*(int)dx+(float)m5; In the formula: 1) m0-m5 are affine transformation coefficients, which are floating point types; 2) The independent variable (dx, dy) is the coordinate of the target point, which is an integer; 3) The function value (sx, sy) is the source point coordinates, which is a floating point type, where the integer part is the location information and the decimal part is the weight; S3, sorting the target points according to the input order of the source points, and converting the disorder into order. The sorting of the target points in S3 further includes: S3.1: Group the target graph vertically; S3.2: Find the target point closest to the source point in each vertical group to obtain a horizontal group; S3.3: Find the target point closest to the source point in the horizontal group, and write the coordinates of this target point into the first-in-first-out order; S3.4: Remove the target points obtained in S3.3 and repeat S3.1-S3.3 until all target points are sorted; The step of obtaining a horizontal group by finding the target point closest to the source point in each vertical group is optimized by using the one-to-one correspondence and monotonicity between the target row and the source row; in step S2, the source image is firstly vertically enlarged by 2 times the power of n, and then an affine transformation is performed; S4, bus output.

2. The method for hardware implementation of affine transformation according to claim 1, characterized in that: The target point sorting in S3 further includes: S3.1: Group the target graph vertically. S3.2: Find the target point closest to the source point in each vertical group to obtain a horizontal group. ； S3.3: Find the target point closest to the source point in the horizontal group, and write the coordinates of this target point into the first-in-first-out order. ； S3.4: Remove the target points obtained in S3.3 and repeat S3.1-S3.3 until all target points are sorted.

3. The method for hardware implementation of affine transformation according to claim 1, characterized in that: In the sorting process of step S3, according to the vertical scaling rule, the one-to-one correspondence between the target row and the source row is as follows: When abs(m1), i.e. the absolute value of m1, is greater than 1, the vertical scaling is shrinking, and the target row and the source row are in one-to-one correspondence; When abs(m1), i.e. the absolute value of m1, is equal to 1, the vertical scaling is unchanged, and the target row and the source row are in one-to-one correspondence; When abs(m1), i.e., the absolute value of m1, is less than 1, vertical scaling is enlargement, and the target row and the source row are not one-to-one corresponding; The monotonicity relationship is as follows: If m1 is positive, the relationship between dy and sy is monotonically decreasing; If m1 is negative, the relationship between dy and sy is monotonically increasing.

4. The method for hardware implementation of affine transformation according to claim 3, characterized in that: The optimization using the one-to-one correspondence and monotonicity between the target row and the source row solves the one-to-one correspondence problem of vertical enlargement.

5. The method for hardware implementation of affine transformation according to claim 1, characterized in that: The original formula for magnifying the source image by a power of 2 during hardware design is: sx=m0*dx+m2*dy+m4; sy=m1*dy+m3*dx+m5; After decomposition, the first formula is: sx=dx; sy=1 / k*dy; After decomposition, the second formula: sx=m0*dx+m2*dy+m4; sy=m1*k*dy+m3*k*dx+m5*k; Here k needs to be a power of 2, and Abs(m1*k), i.e. the absolute value of m1*k, needs to be greater than 1.

Images