Image scaling method and system based on affine transformation
By using an image scaling method based on affine transformation to calculate rotation, scaling, and translation matrices using image coordinates and constructing an affine transformation matrix, the inefficiency problem in existing technologies is solved, achieving efficient parallel processing and accurate image scaling.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-27
- Publication Date
- 2026-03-03
AI Technical Summary
Existing proportional image scaling schemes are inefficient, their processing logic is difficult to optimize in parallel, and their inverse operations are complex, which affects image detection results.
An image scaling method based on affine transformation is adopted. By obtaining image coordinates, rotation, scaling and translation matrices are calculated to construct an affine transformation matrix, thereby achieving proportional scaling of the image. Homogeneous coordinates are used to simplify matrix operations and support parallel processing.
It simplifies the parallel processing of image processing, improves processing efficiency, and can reverse-engineer the original image, thereby improving the accuracy and stability of image detection.
Smart Images
Figure CN114331819B_ABST
Abstract
Description
Technical Field
[0001] This disclosure belongs to the field of image processing technology, specifically relating to a method and system for proportional scaling of images based on affine transformation. Background Technology
[0002] The statements in this section are merely background information relating to this disclosure and do not necessarily constitute prior art.
[0003] Image resizing is a fundamental and important image transformation process. In deep learning applications, image resizing is mainly used in the data preprocessing stage. Since the output and network structure of a typical deep learning network are fixed, the image must be input into the feature extraction network at a specified scale. Therefore, the original image needs to be scaled during the data preprocessing stage to meet the network requirements.
[0004] According to the inventors, image resizing mainly takes two forms: fixed-scale scaling and proportional scaling. Currently, most image object detection methods use fixed-scale scaling, such as Faster R-CNN and SSD, which only require scaling the input image to a certain scale (e.g., 512, 416, etc.). A few image object detection methods (e.g., YOLO series) use proportional scaling to preprocess the input image. However, in practical applications, most images are rectangular (e.g., 1080p video frames) with a aspect ratio that is not 1. Fixed-scale scaling will distort the proportion of the target in the image, directly affecting the detection effect. Because after scaling, default pixel padding needs to be added to the top, bottom, left, and right sides of the shorter side to keep the final network input image square, proportional scaling will maintain the aspect ratio of the target in the original image, making the detection results more stable and accurate.
[0005] Currently, proportional scaling schemes are generally implemented using the OpenCV library. The process involves determining the image scaling ratio, scaling it (usually through interpolation or downsampling), generating a fixed-scale image base with default pixel values, such as a 512x512 image with a pixel value of 128, and then copying the effective portion of the proportionally scaled image to cover the center of the image base. However, existing proportional scaling methods have the following problems:
[0006] (1) It involves a large number of image pixel operations, with many steps and low efficiency;
[0007] (2) The processing logic is difficult to optimize by matrixing and parallelization;
[0008] (3) The inverse operation is complex, for example: mapping the target detection coordinates back to the original image. Summary of the Invention
[0009] To address the aforementioned issues, this disclosure proposes an image scaling method and system based on affine transformation. This method simplifies the difficulty of parallel processing during image processing and improves efficiency by scaling images proportionally based on affine transformation.
[0010] According to some embodiments, the first solution of this disclosure provides an image scaling method based on affine transformation, which adopts the following technical solution:
[0011] A method for proportionally scaling images based on affine transformation includes the following steps:
[0012] Obtain the coordinates of the image to be processed;
[0013] Based on the obtained image coordinates, calculate the image's rotation matrix, scaling matrix, and translation matrix respectively;
[0014] Based on the rotation matrix, the scaling matrix, and the translation matrix, the affine transformation matrix of the image is obtained;
[0015] The processed image coordinates are obtained through the affine transformation matrix, thus achieving proportional scaling of the image.
[0016] As a further technical limitation, in the process of obtaining the coordinates of the image to be processed, the image to be processed is placed in a coordinate system, and the coordinates of all pixels on the image to be processed are obtained.
[0017] As a further technical limitation, the rotation matrix is related to the coordinate point value and rotation angle of the acquired image coordinates; the scaling matrix is related to the coordinate point value and scaling ratio of the acquired image coordinates; and the translation matrix is related to the coordinate point value and translation distance of the acquired image coordinates.
[0018] Furthermore, the rotation angle, scaling ratio, and translation distance are related to the image to be processed and the image after proportional scaling.
[0019] As a further technical limitation, in the homogeneous coordinates, N+1-dimensional coordinates are used to represent N-dimensional coordinates in a two-dimensional Cartesian coordinate system, and the translation matrix is transformed into the product of two matrices through homogeneous coordinates.
[0020] As a further technical limitation, the rotation matrix, the scaling matrix, and the translation matrix are transformed by matrix multiplication to obtain the affine transformation matrix of the image.
[0021] As a further technical limitation, the inverse matrix of the affine transformation matrix is calculated, and the processed image is restored using the inverse matrix and the processed image coordinates to obtain the original image to be processed.
[0022] According to some embodiments, the second solution of this disclosure provides an image scaling system based on affine transformation, which adopts the following technical solution:
[0023] An image scaling system based on affine transformation includes:
[0024] The acquisition module is configured to acquire the coordinates of the image to be processed;
[0025] The calculation module is configured to calculate the rotation matrix, scaling matrix, and translation matrix of the image based on the acquired image coordinates; and to obtain the affine transformation matrix of the image based on the rotation matrix, the scaling matrix, and the translation matrix.
[0026] The processing module is configured to obtain the processed image coordinates through the affine transformation matrix, thereby achieving proportional scaling of the image.
[0027] According to some embodiments, a third aspect of this disclosure provides a computer-readable storage medium, employing the following technical solution:
[0028] A computer-readable storage medium having a program stored thereon that, when executed by a processor, implements the steps of the image scaling method based on affine transformation as described in the first aspect of this disclosure.
[0029] According to some embodiments, the fourth solution of this disclosure provides an electronic device that adopts the following technical solution:
[0030] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the image scaling method based on affine transformation as described in the first aspect of this disclosure.
[0031] Compared with the prior art, the beneficial effects of this disclosure are as follows:
[0032] This disclosure performs proportional scaling of images based on affine transformation. The affine transformation matrix is calculated based on the coordinates of the image to be processed, thus achieving proportional scaling. The image processing is reversible; by solving for the inverse of the affine transformation matrix and combining it with the processed image, the original image to be processed can be obtained. This simplifies the difficulty of parallel processing in image processing and improves efficiency. Attached Figure Description
[0033] The accompanying drawings, which form part of this disclosure, are used to provide a further understanding of this disclosure. The illustrative embodiments of this disclosure and their descriptions are used to explain this disclosure and do not constitute an undue limitation of this disclosure.
[0034] Figure 1This is a flowchart of the image scaling method based on affine transformation in Embodiment 1 of this disclosure;
[0035] Figure 2 This is a schematic diagram of coordinate transformation in Embodiment 1 of this disclosure;
[0036] Figure 3(a) is the image to be processed in Embodiment 2 of this disclosure;
[0037] Figure 3(b) is a scaled image of Embodiment 2 of this disclosure;
[0038] Figure 3(c) is a translated image in Embodiment 2 of this disclosure;
[0039] Figure 4 This is a structural block diagram of the image scaling system based on affine transformation in Embodiment 3 of this disclosure. Detailed Implementation
[0040] The present disclosure will be further described below with reference to the accompanying drawings and embodiments.
[0041] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of this disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.
[0042] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this disclosure. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.
[0043] The specific meanings of the terms used in this disclosure should not be construed as limitations on this disclosure.
[0044] Where there is no conflict, the embodiments and features described herein can be combined with each other.
[0045] Example 1
[0046] Embodiment 1 of this disclosure introduces a method for proportional scaling of images based on affine transformation.
[0047] like Figure 1 The image scaling method based on affine transformation shown includes the following steps:
[0048] Obtain the coordinates of the image to be processed;
[0049] Based on the obtained image coordinates, calculate the image's rotation matrix, scaling matrix, and translation matrix respectively;
[0050] Based on the rotation matrix, the scaling matrix, and the translation matrix, the affine transformation matrix of the image is obtained;
[0051] The processed image coordinates are obtained through the affine transformation matrix, thus achieving proportional scaling of the image.
[0052] In constructing the coordinate system, the top left corner of the image is placed at the origin. With the top left corner as the origin, the x-axis is positive to the right, and the y-axis is positive downwards.
[0053] like Figure 2 The coordinate transformation shown is illustrated, where P is the original point and P' is the point to be rotated. P(x,y) represents the coordinates of the original point, and P'(x',y') represents the coordinates to be rotated. x and y represent the coordinates of the original point on the x-axis and y-axis, respectively, and x' and y' represent the coordinates of the point to be rotated on the x-axis and y-axis, respectively. m represents the distance from the original point to the origin, α represents the angle between the line connecting the original point and the origin and the x-axis, and θ represents the angle (rotation angle) between the line connecting the original point and the origin and the line connecting the point to be rotated and the origin. Therefore, we have...
[0054] cos(θ+α)=cosθcosα-sinθsinα (1)
[0055] sin(θ+α)=sinθcosα+cosθsinα (2)
[0056]
[0057] x'=cos(θ+α)×m=cosθ×x-sinθ×y (4)
[0058] y'=sin(θ+α)×m=sinθ×x+cosθ×y (5)
[0059] The rotation matrix R can then be expressed as:
[0060]
[0061] To scale point P to obtain the point to be scaled. Where scale represents the scaling ratio, the scaling matrix S can be expressed as:
[0062]
[0063] From formulas (6) and (7), the rotation scaling matrix R×S can be expressed as:
[0064]
[0065] To translate point P to obtain the translation point P”'(x”',y”')=P”'(x+ox,y+oy), where ox and oy represent the translation distances along the x-axis and y-axis, respectively, the translation matrix T can be expressed as:
[0066]
[0067] Because translation involves certain addition and subtraction operations, it cannot be directly combined with the rotation and scaling matrix R×S. In the Cartesian coordinate system (i.e., the general term for rectangular coordinate system and oblique coordinate system), translation transformation cannot be represented by a matrix, as shown in formula (9). Translation transformation T cannot be written as the product of two matrices. Therefore, the definition of homogeneous coordinate w is introduced. Homogeneous coordinates use N+1-dimensional coordinates to represent N-dimensional coordinates. We can add an extra variable w to the end of a two-dimensional Cartesian coordinate to construct its homogeneous coordinates. For example, we can transform a point (X, Y) in the Cartesian coordinate system into (x, y, w), which is the mapping of the former in the homogeneous coordinate system. To convert homogeneous coordinates to Cartesian coordinates, we only need to divide (x, y) by w, i.e., X = x / w, Y = y / w. By converting to homogeneous coordinates, we can write the translation transformation as the product of two matrices, as shown in formula (10). Thus, we construct the radial transformation matrix with the transformation matrices of formulas (6) and (7). Therefore, by transforming the translation matrix T, we obtain the transformed translation matrix T', which is...
[0068]
[0069] Using formulas (6), (7), and (10), combined with matrix multiplication, we obtain the affine transformation matrix M, that is:
[0070]
[0071] Based on formula (11), the coordinates of the processed image can be obtained, thus realizing the proportional scaling of the image to be processed.
[0072] Therefore, the pixels in the image can be scaled proportionally by performing three matrix transformations to obtain the processed image.
[0073] The process in this embodiment is reversible; the inverse matrix M is obtained by solving formula (11). -1 By combining the processed image, the original image to be processed can be obtained.
[0074] Example 2
[0075] Based on the image scaling method based on affine transformation introduced in Embodiment 1, Embodiment 2 of this disclosure provides a case study analysis.
[0076] Figures 3(a), 3(b), and 3(c) show the image to be processed, the scaled image, and the translated image. For object detection, the image needs to be scaled proportionally and centered. A specific example is shown in Figure 3(a). First, the base scale (e.g., 512) is determined, and the origin of the image to be processed coincides with that of the base image (in image processing, the top corner is generally the origin). An affine transformation is performed on each pixel of the image to be processed (including one scaling and one translation; no rotation is performed in this example) to obtain the network input image. Since matrix operations are performed on each pixel, parallel GPU processing can be easily implemented using CUDA.
[0077] After the image is processed by the object detection algorithm, the target's coordinates are obtained. Since the input image is scaled and centered, the target's coordinates are also based on the scaled image. Therefore, we need to map the scaled coordinates back to the original image. Due to the properties of the affine transformation matrix, the inverse operation simply involves multiplying the corresponding coordinates by the inverse of the affine transformation matrix. For the post-processing of object detection, we only need to multiply the output coordinates by the inverse matrix to obtain the corresponding target coordinates in the original image. Similarly, this process is accelerated by GPU using CUDA.
[0078] Example 3
[0079] Embodiment 3 of this disclosure introduces an image scaling system based on affine transformation.
[0080] like Figure 4 The image scaling system based on affine transformation includes:
[0081] The acquisition module is configured to acquire the coordinates of the image to be processed;
[0082] The calculation module is configured to calculate the rotation matrix, scaling matrix, and translation matrix of the image based on the acquired image coordinates; and to obtain the affine transformation matrix of the image based on the rotation matrix, the scaling matrix, and the translation matrix.
[0083] The processing module is configured to obtain the processed image coordinates through the affine transformation matrix, thereby achieving proportional scaling of the image.
[0084] The detailed steps are the same as the image scaling method based on affine transformation provided in Example 1, and will not be repeated here.
[0085] Example 4
[0086] Embodiment 4 of this disclosure provides a computer-readable storage medium.
[0087] A computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of the image scaling method based on affine transformation as described in Embodiment 1 of this disclosure.
[0088] The detailed steps are the same as the image scaling method based on affine transformation provided in Example 1, and will not be repeated here.
[0089] Example 5
[0090] Embodiment 5 of this disclosure provides an electronic device.
[0091] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the image scaling method based on affine transformation as described in Embodiment 1 of this disclosure.
[0092] The detailed steps are the same as the image scaling method based on affine transformation provided in Example 1, and will not be repeated here.
[0093] While the specific embodiments of this disclosure have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of this disclosure. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of this disclosure are still within the scope of protection of this disclosure.
Claims
1. An image scaling method based on affine transformation, characterized in that, The method comprises the following steps: obtaining coordinates of an image to be processed; calculating a rotation matrix, a scaling matrix and a translation matrix of the image respectively according to the obtained image coordinates; obtaining an affine transformation matrix of the image according to the rotation matrix, the scaling matrix and the translation matrix; obtaining processed image coordinates by the affine transformation matrix to realize equal scaling processing of the image; the rotation matrix is related to coordinate point values of the obtained image coordinates and a rotation angle; the scaling matrix is related to coordinate point values of the obtained image coordinates and a scaling ratio; the translation matrix is related to coordinate point values of the obtained image coordinates and a translation distance; the rotation angle, the scaling ratio and the translation distance are related to the image to be processed and the equal scaling image; the rotation matrix, the scaling matrix and the translation matrix are changed according to matrix multiplication to obtain the affine transformation matrix of the image.
2. The method for image scaling based on affine transformation as claimed in claim 1, wherein, In the process of obtaining the coordinates of the image to be processed, the image to be processed is placed in a coordinate system to obtain the coordinates of all pixel points on the image to be processed.
3. The method for image scaling based on affine transformation as claimed in claim 1, wherein, In homogeneous coordinates, a translation matrix is converted into the product of two matrices using N +1-dimensional coordinate representation N dimensional coordinate, a translation matrix is converted into the product of two matrices by homogeneous coordinates.
4. The method for image scaling based on affine transformation as claimed in claim 1, wherein, An inverse matrix of the affine transformation matrix is calculated, and the original image to be processed is obtained by restoring the processed image through the inverse matrix and the processed image coordinates.
5. An image scaling system based on affine transformation, characterized by, The method comprises the following steps: obtaining coordinates of an image to be processed; calculating a rotation matrix, a scaling matrix and a translation matrix of the image respectively according to the obtained image coordinates; obtaining an affine transformation matrix of the image according to the rotation matrix, the scaling matrix and the translation matrix; obtaining processed image coordinates by the affine transformation matrix to realize equal scaling processing of the image; the rotation matrix is related to coordinate point values of the obtained image coordinates and a rotation angle; the scaling matrix is related to coordinate point values of the obtained image coordinates and a scaling ratio; the translation matrix is related to coordinate point values of the obtained image coordinates and a translation distance; the rotation angle, the scaling ratio and the translation distance are related to the image to be processed and the equal scaling image; the rotation matrix, the scaling matrix and the translation matrix are changed according to matrix multiplication to obtain the affine transformation matrix of the image.
6. A computer-readable storage medium having stored thereon a program, characterized in that, The program is executed by the processor to realize the steps in the equal scaling processing method of the image based on the affine transformation.
7. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized by The processor executes the program to realize the steps in the equal scaling processing method of the image based on the affine transformation.