A method for determining the aspect ratio of a batch of images

Abstract

The application discloses a method for determining the aspect ratio of an image batch, wherein the resolution of the image batch is denoted as (h, w), the aspect ratio of the image batch is denoted as P, P=h / w; the resolution of each image in the batch is obtained, the resolution of the i-th image is denoted as (Hi, Wi), and the aspect ratio of the i-th image is denoted as Ri, Ri=Hi / Wi; and the following condition is met: the application has the advantages that the waste of computing power resources can be reduced.

Description

Technical Field

[0001] This invention belongs to the field of computer image processing, and particularly relates to a method for determining the aspect ratio of an image batch. Background Technology

[0002] In deep learning, accelerators (consisting of hardware and accompanying software, such as GPU or TPU suites) are typically used to accelerate computation. When processing image-related tasks, multiple image datasets are usually grouped into a batch and fed into the accelerator for processing at once, which is more efficient than processing individual image data multiple times. In image-related applications, a batch of data may contain images with various resolutions and aspect ratios, as well as images with different sizes. Accelerators require images in the input batch to have a uniform size; therefore, the images in the batch need to be scaled and / or padded to the same size as needed. To fully utilize computing resources, the proportion of pixels in the batch should be as large as possible, and the proportion of padded areas should be minimized. Therefore, how to choose an appropriate scaled batch resolution is a problem worthy of investigation.

[0003] Currently, the most common sampling method in deep learning systems is the baseline method. First, a maximum size MaxSize is set, such as MaxSize = 640 pixels. For each image in the batch, the larger dimension of its width and height is scaled to this maximum size while maintaining the aspect ratio. The smaller dimension will necessarily be smaller than the maximum size. Finally, the maximum width and height of all scaled images are counted and used as the input dimension of the batch.

[0004] To improve data processing efficiency, scaled images should be as "compact" as possible, meaning the average proportion of image content should be as high as possible while the proportion of filler should be as small as possible. This requires determining the best possible aspect ratio for a batch of images. Existing baseline methods clearly do not meet this principle. For example, if most images in a batch are taller than wide, and only one image is wider than tall, the final aspect ratio will be 1:1, resulting in most images requiring more filler and wasting significant computational resources.

[0005] In summary, the existing method for calculating the aspect ratio of a batch of images has the drawback of wasting a lot of computing resources. Summary of the Invention

[0006] The purpose of this invention is to provide a method for determining the aspect ratio of an image batch. This invention has the advantage of reducing the waste of computing resources.

[0007] The technical solution of the present invention is: a method for determining the aspect ratio of an image batch, wherein the resolution of an image batch is denoted as (h, w), and the aspect ratio of an image batch is denoted as P, where P = h / w;

[0008] To obtain the resolution of each image in a batch, the resolution of the i-th image is denoted as (Hi, Wi), and the aspect ratio of the i-th image is Ri, where Ri = Hi / Wi;

[0009] The value of p is determined using the following formula:

[0010]

[0011] The aforementioned method for determining the aspect ratio of an image batch includes the following steps in solving Formula 1:

[0012] a. Sort the Ri values ​​of all images in a batch of images in ascending order of polarity to obtain the sequence R1≤R1...≤R N We know that R1≤p≤R N ;

[0013] b. Let y(p) be the average image ratio after scaling all images in a batch, and assume that the optimal value of p lies in R. k and R k+1 Between, i.e., R k ≤p≤R k+1 At this point, we have Formula 2.

[0014]

[0015] c. Taking the first derivative of Equation 2, we obtain Equation 3.

[0016]

[0017] d. Taking the second derivative, we get Formula 4.

[0018]

[0019] e. The second derivative is always positive, indicating that in the interval (R... k R k+1 The minimum value is found within the range ), and the candidate for the maximum value is at the endpoint, i.e., p = R. k Or p = R k+1 In other words, to find the optimal value of p, we only need to calculate p∈{R1,R2,...,R... N The maximum y(p) value in the given information is the optimal solution, corresponding to the value of p, which leads to Formula 5.

[0020]

[0021] f. Formula 1 can be derived from Formula 5 and Formula 2:

[0022]

[0023] Compared with existing technologies, this invention provides a formula for calculating the optimal p-value, i.e., the optimal aspect ratio. Under the p-value obtained according to the formula, the proportion of image pixels in a batch is maximized, while the proportion of the corresponding filled area is minimized, thus avoiding waste of computing resources. Therefore, this invention has the advantage of reducing the waste of computing resources. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the multi-peak function y(p) in Example 1.

[0025] Figure 2 This is a schematic diagram of the single peak of the y(p) function in Example 1.

[0026] Figure 3 This is a schematic diagram of scaling and filling each image after determining the p-value using the existing method in Experiment and Comparison 2.

[0027] Figure 4 This is a schematic diagram of scaling and filling each image after determining the p value according to the present invention in Experiment and Comparison 2. Detailed Implementation

[0028] The present invention will be further described below with reference to the accompanying drawings and embodiments, but this should not be construed as limiting the present invention.

[0029] Example 1 (Best Example). Analysis: Suppose a batch contains N images {I1, I2, ..., I...} N The height and width of each image are {(H1, W1), (H2, W2), ..., (H... N W N The goal is to obtain the optimal batch resolution (h, w) that maximizes the average ratio of the scaled image content to the total area (referred to as average utilization), thereby minimizing the waste of computing resources.

[0030] Let the aspect ratio of the batch be P, where P = h / w; let the aspect ratio of the i-th image be R. i R i =H i / W i For the i-th image, the scaling factor S i =min(h / H) i w / W i The scaled size is (H) i *S i W i *Si If Y is the proportion of the scaled image content, then the image content is... i for:

[0031] .

[0032] The average image proportion y(p) after scaling all images within a batch is:

[0033]

[0034] Therefore, the value of p corresponding to the objective of maximizing the average image proportion y(p) is:

[0035] Where R is the real number field of the theoretical range of P.

[0036] The specific solution process is as follows:

[0037] First, the image is processed according to R. i The values ​​are sorted in ascending order, i.e., R1 ≤ R1 ... ≤ R N Clearly, the optimal value of p should be in R. i Within the range of values, i.e., R1≤p≤R N This is because the fill ratio of all images beyond this range will increase.

[0038] Suppose the optimal p value falls within R k and R k+1 Between, i.e., R k ≤p≤R k+1 At this point, the objective function is expressed as formula two:

[0039]

[0040] Taking the first derivative of Formula 2, we get Formula 3:

[0041]

[0042] Taking the second derivative, we get Formula 4:

[0043]

[0044] like Figure 1 and Figure 2 The figure shows the curve of y(p), where all-y(p) is the curve at all points p, vertex-y(p) is the curve at each point of the image with the corresponding aspect ratio, and max-y(p) is the position of the maximum value.

[0045] The second derivative is always positive, indicating that in the interval (R) k R k+1 The minimum value is found within the range ), and the candidate for the maximum value is at the endpoint, i.e., p = R.k Or p = R k+1 In other words, to find the optimal value of p, we only need to calculate p∈{R1,R2,...,R... N The maximum y(p) value in the given set is the corresponding p value, which is the optimal solution. The solution complexity is O(N), as shown in Formula 5.

[0046]

[0047] Based on Formula 5 and Formula 2, we can derive Formula 1:

[0048]

[0049] The P-value of the aspect ratio of an image batch obtained under the above conditions can maximize the reduction of the fill amount of the batch images and reduce the waste of computing resources. This leads to a method for determining a better aspect ratio for an image batch. The method in Example 1 is abbreviated as ours.

[0050] Example 2. R1 to R N Arrange the data in ascending order. When N is odd, p takes the aspect ratio of the median value after sorting, i.e., R. (N+1) / 2 When N is an even number, p takes the average of the two middle values, i.e., {R} N / 2 +R N / 2+1} / 2. The method of Example 2 is referred to as middle.

[0051] Example 3. The extreme value of p is determined by searching for the minimum Ri and the maximum Ri with equal step sizes, and the number of segments is 2N. The method in Example 3 is abbreviated as linear.

[0052] Compare with 1.

[0053] The deep learning object detection model was trained using all 118,287 images from the commonly used COCO2017 training dataset. Following standard procedures, the images were randomly arranged in each training round. The p-value was determined based on the existing baseline, middle, linear, and ours, and y(p) was calculated. The results are shown in Table 1.

[0054]

[0055] Table 1

[0056] As can be seen from Table 1, the y(p) value of ours is the largest, which can maximize the reduction of the fill amount of batch images and reduce the waste of computing resources.

[0057] Comparison 2. There is a batch of three images: the first image has a height of 1 and a width of 0.6, the second image has a height of 1 and a width of 0.7, and the third image has a height of 0.6 and a width of 1. The units of the values ​​are the same.

[0058] After calculation using the existing method (i.e., baseline), the determined p-value is clearly 1, and the effect after scaling and filling is as follows. Figure 3 As shown, gray represents the image area and white represents the filled area, with an average utilization rate of only 0.63.

[0059] After determining the p-value according to the method of this invention, the scaling and filling effect is as follows: Figure 4 As shown, the average utilization rate is 0.85, and the y(p) value increases significantly.

Claims

1. A method of determining the aspect ratio of a batch of images, the method comprising: The resolution of an image batch is denoted as (h, w), and the aspect ratio of an image batch is denoted as P, P = h / w; ​ The resolution of each image in a batch is obtained, and the resolution of the ith image is denoted as (Hi, Wi), and the aspect ratio of the ith image is denoted as Ri, Ri = Hi / Wi; The p value is determined according to formula one, Equation One: The solving method of formula one includes the following steps, a、Sort all the images in one image batch by the value of Ri from small to large, and get the sequence R1≤R1...≤R N , and know that R1≤p≤R N ; b、The average image proportion of all images in a batch after scaling is denoted as y(p), and the optimal p value is located between R k and R k+1 , that is, R k ≤ p ≤ R k+1 , at this time there is formula two, Equation Two: c. First-order derivation is performed on formula two to obtain formula three, Equation Three: d. Second-order derivation is performed to obtain formula four, Equation Four: e. The second derivative is always positive, indicating that there is a minimum in the interval (R k， R k+1 ) and the candidate for the maximum is at the endpoints, i.e. p = R k or p = R k+1 , that is, to find the optimal p value, we only need to calculate the maximum y(p) value among p e {R1, R2,..., R N} and the corresponding p value is the optimal solution, which gives equation five, Equation Five: f. According to formula five and formula two, formula one can be obtained:

Images