Fractional image downscaler

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for downscaling an input array with an arbitrary downscaling ratio. In one aspect, a system comprises receiving an input array comprising a first set of values and a first dimension size characterizing a first dimension and a second dimension size characterizing a corresponding dimension of an output array, determining a second set of values for the output array from the first set of values based at least on a corresponding set of scaled weight parameters using an accumulator counter and a rollover counter, and generating the output array comprising the second set of values.

Description

[0001] FRACTIONAL IMAGE DOWNSCALER

[0002] BACKGROUND

[0003] This specification relates to applications of digital signal processing for image downsampling, e.g., reducing the resolution or size of an input image to generate a downsampled image.

[0004] One example of a technique for image downsampling is a box filter, which is one of the simplest and most efficient filters widely used in many applications to downscale images to smaller sizes. To downsample an image by a factor of M, where M is an integer, an MxM box filter can be applied to compute an output pixel value from the average of pixel values within the MxM window. As another approach, applying a number of low-pass filters to attenuate signals based on a cutoff can be used to sample the signal at different frequencies using a filter kernel. In particular, bilinear, bicubic, or Gaussian filtering can be applied with a fixed number of taps, e.g., as determined by the filter-kernel size. However, downsampling with a bilinear or bicubic filter can result in aliasing effects between the original image and the downsampled image, especially at large downscaling ratios. Likewise, downsampling with a box or Gaussian filter with a limited number of taps can result in aliasing effects. In particular, the number of taps refers to the number of coefficients used to process the input signal in order to produce the output signal within the context of digital signal processing.

[0005] As an example, aliasing effects can include differences and artifacts such as moire patterns or jagged edges that result from overlapping frequency components in the downsampled image. In particular, an overlapping frequency can be a result of sampling at a frequency less than a defined rate, e.g., the Nyquist rate as defined in the Nyquist-Shannon sampling theorem. The Nyquist rate specifies that the sampling must be performed at twice the highest frequency of a given function or signal to ensure sampling withouth aliasing.

[0006] SUMMARY

[0007] This specification describes a system that can downscale an input array, e.g.. an input image, with an arbitrary downscaling ratio. In particular, the system can use a set of scalers to determine a set of scaled weight parameters for each value in the input array that can be used to determine the values in a downscaled output array, e.g., a downscaled image, for an arbitrary scaling factor, e.g., integer or fractional, defined for a corresponding dimension of the input and output array. More specifically, the system can normalize the values of the array using the scaled weight parameters, even for fractional scaling between a dimension of the input image and a dimension of the downscaled output. In this specification, the downscaling ratio refers to the factor by which the dimensions of the input array, e.g., the width and height of the input image, are reduced during the downsampling process. The downscaling ratio is ty pically expressed as a fraction or percentage and represents the relationship between the original size of the input array and the final downscaled output.

[0008] According to a first aspect there is provided a method for receiving an input array comprising a first set of values and a first dimension size characterizing a first dimension and a corresponding dimension size characterizing a corresponding dimension of an output array, determining a second set of values for the output array from the first set of values based at least on a corresponding set of scaled weight parameters using an accumulator counter and a rollover counter, wherein determining the second set of values comprises: identifying a threshold value based on the first dimension size and an iteration value based on the corresponding dimension size, calculating the set of scaled weight parameters using the accumulator counter to accumulate a sum of the rollover counter and iteration values at each value by iterating through the first set of values until the accumulator counter satisfies a criterion with respect to the threshold value at a first terminal value, and generating the output array comprising the second set of values.

[0009] Particular embodiments of the subject matter described in this specification can be implemented so as to realize one or more of the following advantages.

[0010] The techniques described in this specification can be used to perform fractional downscaling of an input array with significantly reduced aliasing artifacts. In addition, the techniques described in this specification can allow for arbitrary downscaling ratios. In contrast, box sampling requires the downscaling ratio between the dimension of the input image and the downsampled image to be integer-valued; and performing low-pass filtering with multiple loyv-pass filters can achieve fractional scaling, but at much greater computational cost and design complexify.

[0011] The system can provide flexible tuning to arbitrary downscaling ratios for each dimension of downscaling, unlike current approaches, and can generate an output array with less aliasing artifacts and better denoising effects compared to the results of applying low -pass filters yvith a fixed number of taps, especially for large downscaling ratios. In particular, the system can determine a set of scaled yveight parameters that can enhance the sampling of the underlying signal of the array, e.g., such that the sampling approximates the Nyquist rate. More specifically, the system can allow for sampling that approximates applying a filter of size K, e.g., the system can sample at the Nyquist rate of K / 2 with the filter of size K, where K being larger than the fixed number of taps guarantees less aliasing.

[0012] Additionally, the techniques described allow for computationally efficient image downscaling at large downscaling ratios. As an example, large downscaling ratios are often required for image-signal processing to downscale an input image from a sensor size, e.g.. 12 megapixels (MP), to low resolution, e.g., 512 x 288 or 256 x 192 pixel images, for machine learning interference. Furthermore, the techniques described reduce the computational resources needed to downscale an image by using the same line buffer to store the intermediate values for the output array, e g., without a divider, when downscaling for multiple dimensions. Using the same line buffer can allow for a resource-efficient hardware implementation. In particular, rather than performing low-pass filtering and storing the signal at different frequencies in separate locations in memory', the system can normalize the values of the input array in successive arbitrary directions using the same memory' location. For example, the line buffer can be used to store the scaled values from scaling in a first dimension while scaling is being performed in a subsequent dimension. Furthermore, the line buffer size can be restricted to the maximum dimension size of the output array. For example, in the case of downsampling an image, the system can perform horizontal scaling before vertical scaling, thereby requiring the size of the line buffer to depend on the desired downscaled horizontal w idth.

[0013] The details of one or more embodiments of the subject matter of this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims.

[0014] BRIEF DESCRIPTION OF THE DRAWINGS

[0015] FIG. 1 is a block diagram of an example downscaling system operating on a user device.

[0016] FIG. 2 illustrates an example scaling in one dimension, e.g., a horizontal scaling for an input image.

[0017] FIG. 3 illustrates an example scaling technique in two dimensions for an input image, e.g., using the example scaling technique depicted in FIG. 2.

[0018] FIG. 4 demonstrates an example cropping method for cropping the input array before scaling. FIG. 5 provides a comparison of an example image downscaled using the downscaling system of FIG. 1 and an example image downscaled using cubic filtering. FIG. 6 is a flow diagram of an example process for generating a downscaled output array by determining and applying a set of scaled weight parameters.

[0019] FIG. 7 is a flow diagram of an example process for determining the set of scaled weight parameters.

[0020] Like reference numbers and designations in the various drawings indicate like elements.

[0021] DETAILED DESCRIPTION

[0022] FIG. 1 shows an example downscaling system 100. The downscaling system 100 is an example of a system in which the techniques described below7are implemented.

[0023] As an example, the downscaling system 100 can be executed on a user device 105, e.g., to perform image downscaling. For example, the user device 105 can be a cell-phone 105a, tablet 105b, laptop 105c, wearable device, e.g., smart-watch 105d, internet of things (loT) device, gaming counsel, medical imaging device, desktop, etc.

[0024] In particular, the user device 105 can employ the downscaling system 100 to receive an original image, e g., the input array 110, and one or more output dimension size(s) 120 for the desired downscaled output array, e.g., the downscaled image. In some cases, a user can specify a desired downscaling ratio for video or image streaming, e.g., by downscaling the video frame or image to facilitate transmission. In other cases, a user can specify a desired downscaling ratio for images stored on the user device 105 and elect to maintain original image fdes remotely, e.g., in on-device storage or on remote cloud-storage.

[0025] The downscaling system 100 can include a downscaling engine 130 that can process an input array 110 with one or more input dimension size(s) 114 and one or more corresponding output dimension size(s) 120 to generate an output array 160 with the specified one or more output dimension size(s) 120. In some implementations, at least one of the output dimension size(s) 120 can be less than the input dimension size(s) 114, such that the output array 160 requires less computational resources to maintain than the input array.

[0026] The input 114 and output 120 dimension size(s) parameterize the size of each dimension of the input 110 and output 160 array. More specifically, the system 100 can process an input array 110 of any dimension, e.g.. one dimension, two dimensions, five dimensions, etc., where each dimension has a respective dimension size as specified by the corresponding number of input dimension size(s) 110 and can generate an output array 160 of the same dimension as the input array 110 with different, e.g., smaller, size as specified by the corresponding number of output dimension size(s) 120. For example, the system 100 can process a two-dimensional input array, e.g., an image, 1 10 of size 124 x 256 and generate a downscaled two-dimensional output array 160 of size 64 x 144. As another example, the system 100 can process a fivedimensional input array 110 of size 124 x 256 x 312 x 546 x 124 to generate a five-dimensional output array 160 of size 64 x 144 x 124 x 312 x 64.

[0027] In some cases, the input array 110 can include a set of cropping values 112 that specify that the input array 110 be cropped with respect to the values 112 before being processed. As an example, the cropping values 112 can specify a subset of the input array 110 for processing by defining a first value and a last value along each dimension of the input array. In the case in which the system 100 receives cropping values 112, e.g., from the user device 105, the system 100 can calculate one or more updates to the input dimension size(s) 114 of the initial input array 110 and select the corresponding subset of the input array 110 for scaling.

[0028] The ratio between the one or more input dimension size(s) 114 and the corresponding output dimension size(s) 120 determines the downscaling ratio for the particular dimension. As an example, in the case in which a particular input dimension size is 20 and the corresponding output dimension size is 3, the corresponding downscaling ratio for that dimension is 20 / 3, which can be rounded to the hundredths place as 6.67. As another example, in the case in which an input dimension size is 20 and the corresponding output dimension size is 7, the corresponding downscaling ratio for that dimension is 20 / 7, which can be rounded to the hundredths place as 2.86.

[0029] The system 100 can process an input array 110 to generate an output array 160 with any arbitrary downscaling ratio, e.g.. either integer or fractional, using the downscaling engine 130. As an example, the downscaling engine 130 can include a set of scalers, e.g., each corresponding to the dimensions specified by the dimension of the input array 110. As another example, the system can employ a single scaler, e.g., the first scaler 140, multiple times for each dimension in the input.

[0030] The set of scalers can be implemented as a data processing apparatus, logic circuitry, or another type of hardw are module. The system can program the hardware components, e.g., customized circuitry7such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuitry), to perform the downscaling and to interface with other system components, e.g., for storage.

[0031] In the particular example depicted, the downscaling engine 130 includes a first scaler 140, e.g., for a first dimension, and can additionally include one or more additional scaler(s) 150, e.g., for each additional dimension specified by the input dimension. In the particular example in which the input array 110 is an image, e.g.. a two-dimensional array, the first scaler 140 can be a horizontal scaler and the additional scaler 150 can be a vertical scaler. The downscaling engine 130 can use scalers 140, 150 to generate the output array 160 with the output dimension size(s) 120 specified by the input. For example, the downscaling engine 130 can process the input array 110 along each dimension, e.g., by iterating through the set of values of the input array 110 along an index of the dimension, and calculating a corresponding set of scaled weight value parameters 142, 152 for the particular dimension. The engine 130 can then apply the corresponding set of scaled weight value parameters 142, 152 to the values in the input array 110 along each dimension to determine a second set of values of the output array 160.

[0032] In the case that the input array 110 is an image, the engine 130 can calculate the corresponding set of scaled weight value parameters for the horizontal direction using the horizontal scaler and can calculate the corresponding set of scaled weight value parameters for the vertical direction using the vertical scaler. In this case, the set of scaled weight value parameters for the vertical direction can depend on the set of scaled weight value parameters for the horizontal direction, e.g., the system can perform the horizontal scaling, store the intermediate scaled values in a line buffer 145, and use the horizontally scaled values for the vertical scaling, as will be described in more detail in FIG. 3

[0033] In the case that the input array 110 has channels, e.g., when more than one value is used to represent a single array element, the engine 130 can calculate the corresponding set of scaled weight value parameters for each channel, e.g., by scaling each channel by a set of scaled weight value parameters shared among the channels. As an example, in the case of a red-blue- green image (RGB) as the input array, the engine 130 can determine the corresponding set of horizontal and vertical scaled weight value parameters for each pixel of the image and apply the corresponding scale weight to each of the red, blue, and green channels of the output image. The engine 130 can then aggregate each respective scaled weight value, e.g., by computing a normalized weighted sum over the channels, to determine the corresponding values of the output array 160. As another example, the system can determine the corresponding set of horizontal and vertical scaled weight value parameters for the combined channels.

[0034] The engine 130 can then calculate a weighted average of the input set of values using the corresponding set of scaled weight value parameters, e.g.. by applying each scaled weight value parameter as the weight for each value in the input array in a rolling normalized sum. As an example, the engine 130 can use a single line buffer 145 to perform a stream-based calculation using the corresponding set of scaled weight value parameters 142, 152 to accumulate a quantity of the pixels multiplied by their corresponding scaled weight value parameters until a terminal value, e.g., in a stream-based weighted average. The system can then determine each value in the second array by normalizing the quantity of the pixels multiplied by their corresponding scaled weight value parameters, e.g., by multiplying the final accumulated quantity’ by the inverse of a normalization quantity, e.g., 1 / normalization quantity. In particular, the engine 130 can use the line buffer to perform the rolling normalized sum in one or more dimensions.

[0035] More specifically, the engine 130 can maintain a single line buffer 145 to temporarily store data pertaining to the intermediate values of the downscaled output array 160, e.g., the intermediate values from previously scaled dimensions. In the case of downscaling an image, e.g., a two-dimensional array, the line buffer 145 can store data pertaining to the outputs of the first scaler multiplied by the weights of the second scaler, e.g., intermediate pixel coordinates and color values during the scaling in the second dimension using the second scaler. The line buffer 145 can store data pertaining to the rasterization of data from the input array 110 to the output array 160, e.g., along each dimension, thereby reducing the computer resources needed to perform the downscaling relative to other methods that require separate storage for each dimension, e g., low-pass filtering methods. Once the line buffer 145 contains the completed information for an output value, e.g., once scaling in each dimension has been completed and the value in the line buffer is no longer an intermediate value, the system can output the data, e.g., to a frame buffer for rendering in the case of dow nscaling an image.

[0036] For example, the system 100 can calculate the corresponding set of scaled weight value parameters for the values by iterating through the set of values in the input array 110. e.g., along an index of a particular dimension, and computing a rolling normalized sum along the particular dimension and then performing the same process along the one or more dimensions. In particular, at each value of the input array, the system can assign an unnormalized weight and accumulate a sum of the pixel values multiplied with the unnormalized weight until a threshold criteria is met at a terminal value. The system can then normalize the values to determine a value, e.g., an intermediate value if scaling in multiple dimensions or a value of the output array 160, and then repeat the iteration from the terminal value until every' value of the input array 110 has been considered. More specifically, the system 100 can repeat the process by iterating through the set of values in the input array 110 including and after the terminal value to determine a next value, e g., a next intermediate value if scaling in multiple dimensions, of the output array until every' element of the array has been considered in the values of the output array.

[0037] In the case in which the system 100 is scaling in multiple dimensions, the intermediate values, e.g., the intermediate values from a prior scaled dimension, can be stored in the line buffer 145 and used for scaling in the different directions until the intermediate value has been scaled and normalized in each dimension for the output array 160. As an example, the normalized values of the output from the first scaler 140 and the accumulated sum of the the scaled weight values of the additional scaler 150 multiplied with the normalized values of the output from the first scaler 140 can be accumulated in the line buffer 145. For example, the sequence of input values contributing to a value in the output array 160 can be temporarily stored in the line buffer 145 as the corresponding set of scaled values are applied by the next scaler along a different dimension, e.g., in order for the output array to accumulate a weighted sum along the next dimension. An example process for generating values for the output array using a shared line buffer 145 for scaling in two dimensions will be covered in more detail with respect to FIG. 3.

[0038] FIG. 2 demonstrates an example scaling process in one dimension. In particular, FIG. 2 demonstrates the scaling process 200 for an example fractional downscaling ratio, e.g., for a 20:3 downscaling ratio. In the particular example depicted, the 20:3 downscaling is performed along the horizontal dimension of an image, e.g., in which the ratio of the width of the input image to the width of the output image is 20:3.

[0039] For example, the system can determine an iteration value 210 and a threshold value 220 based on the downscaling ratio. As an example, the iteration value 210 can be the size of the output array from the particular downscaling ratio for the dimension being scaled, e.g., the horizontal width of the desired output image, and the threshold value 220 can be the size of the input array from the particular downscaling ratio for the dimension being scaled, e.g., the horizontal width of the input image. In this case, the iteration value 210 is 3 and the threshold value 220 is 20.

[0040] The system can use an accumulator counter 215 and a rollover counter 230 to determine the set of scaled weight parameter values based on the iteration value 210 and the threshold value 220, e.g., by calculating a rolling normalized sum by applying the scaled weight parameter values to the respective input values in a stream-based accumulation calculation. For example, the system can iterate through each value in the first set of values of the first line, assign an unnormalized weight of the iteration value 210 to each value and accumulate a quantity of each respective pixel value multiplied by the iteration value 210, e g., in the line buffer 145. The system can accumulate the weighted sum while accumulating a sum of the rollover counter 230 and the iteration value 210 at each value in the input set of values until the accumulator counter 215 satisfies a threshold criterion based on the threshold value 220 at a terminal value, e.g., when the accumulator counter 215 exceeds the threshold value 220. The weight of the terminal value can then impact the rolling sum of the scaled weight parameters that is normalized at the terminal value as well as the values after the terminal value, as will be described in more detail below.

[0041] As depicted for a horizontal line of pixels, e.g., the first horizontal line of pixels in the input image, the system can iterate through the subset of the first set of values in the first line to assign an unnormalized weight of the iteration value 210 of 3 while accumulating the iteration value 210 of 3 in an accumulator counter 215 at each value, e.g.. pixel element, in the horizontal line until the threshold value 220 of 20 is exceeded. The system can accumulate a weighted sum of the quantity of the iteration value 210 multiplied by each pixel value in the line buffer until the terminal value.

[0042] In the particular example depicted, the pixel index is used to characterize the relative location of each pixel element in the horizontal line. For example, the system accumulates the iteration value of 3 in the accumulator counter 215 at the zeroth pixel index, the system accumulates the iteration value of 3 at the first pixel index such that the accumulator counter 215 is 6, the system accumulates the iteration value of 3 at the second pixel index such that the accumulator counter 215 is 9, and so on up until the threshold value 220 of 20 is reached. In this case, the accumulator counter 215 exceeds the threshold value 220 in between the fifth and sixth pixel index, e.g., the system can define the terminal value as the first index element at which the accumulator counter 215 meets or has exceeded the threshold. Each terminal value defines the subset of values of the first set of values that contribute to each value of the output array, e.g., the last input array element contributing to the weighted sum that is being accumulated in the line buffer for a value in the output array. In particular, each terminal value can contribute to more than one value in the output array , e. g. , by being a part of the nonnalized sum of more than one value in the output array.

[0043] In this case, the sixth pixel element is the first terminal value 250, e.g., first relative to iterating through the sequence of values in the first horizontal line of the input array. The first tenninal value 250 can then be used to determine the rollover counter, e.g., based on the difference between the threshold value 220 and the accumulator counter 215 at the first terminal value 250. In this case, at the terminal value of the sixth pixel element, the rollover counter 230 is 1, and, the first terminal value 250 contributes the remainder of the difference between the accumulator counter 215 and the threshold value 220 to the set of values before the first terminal value 250 and the rollover counter 230 to the next set of values. More specifically, the first terminal value 250 is assigned an unnonnalized first weight 255 of 2 for the zeroth pixel element 292 of the output array elements 290 and an unnormalized second weight 260 of 1, e.g., the first terminal value 250 contributes the quantity of the unnormalized first weight 255 multiplied by the tenninal value 250 to the rolling normalized sum for the zeroth pixel element 292 and the quantity of the unnormalized second weight 260 of 2 multiplied by the terminal value 250 to the rolling normalized sum for the first pixel element 294.

[0044] After accumulating the quantity of the scaled weight parameters multiplied by their corresponding pixel values, the rolling sum is normalized at the terminal value, e.g.. by multiplying the rolling weighted sum by the quantity of the inverse of the threshold value, e.g., 1 / 20. For example, the system can then complete the rolling window for the zeroth pixel element 292 of the output array and normalize the values up to and including the first terminal value 250 by applying the scaled weight parameters to each respective input value in order to perform the normalization, e.g., the system can normalize the values corresponding with the current pixel element as soon as the weight for the corresponding terminal value, e.g., 2, has been determined and is added to the weighted sum. In other cases, the system can multiple each value up to the first terminal value 250 by a ratio of the iteration value 210 to the threshold value 220, e.g., 3 / 20, and multiple the first terminal value 250 by a ratio of the first weight 255 assigned to the terminal value 250 to the threshold value 220, e.g., 2 / 20, in order to determine the value for pixel element 292 of the output array.

[0045] The system can continue the rolling normalized sum by reinitializing the accumulator counter 215 to the rollover counter 230 value, e.g., 1, and iterating through each value after the first terminal value 250 in the input array. As previously described, the system can assign a scaled weight parameter of the iteration value 210 to each value before the accumulator counter exceeds the threshold value 220. In this case, since the rollover counter was 1, the accumulator counter 215 accumulates the iteration value 210 such that the accumulator counter is 4, 7, 10, 13, etc. for each element after the first terminal value 250 and before the accumulator counter 215 exceeds the threshold value 220 at a second terminal value 270. In this case, the system exceeds the threshold value in between the twelfth and thirteenth pixel index.

[0046] As described previously, the system then uses the rollover counter 240 value, e.g., 2 based on the difference between the threshold value 220 and the accumulator counter 215 at the second terminal value 270, to determine the relative weight of the second tenninal value 270 for the first pixel element 294, e.g., the current output value, as well as to rollover a relative weight for the next output value, e.g., the second pixel element 296. In the particular example depicted, the second terminal value 270 contributes the remainder of the difference between the accumulator counter 215 and the threshold value 220, e.g., the first weight 280, to the first pixel element 294, and contributes the second weight 275 to the next pixel element in the output array elements 290, e.g., the second pixel element 296.

[0047] The system can then complete the rolling window for the first pixel element 294 of the output array and normalize the values from the first terminal value 250 and up to and including the second terminal value 270. In the particular example depicted, the system can accumulate a sum of the first terminal value 250 multiplied by the second weight 260 to the threshold value 220, e.g., 1, each intermediate value until the next threshold value 220 multiplied by the iteration value 210, e g., 3, and the second terminal value 270 by the first weight 280 to the threshold value 220, e.g., 1, and then normalize by multiplying by 1 / threshold value.

[0048] The system can continue the rolling normalized sum by reinitializing the accumulator counter 215 to the rollover counter 240 value, e.g., 2, and iterating through each value after the second terminal value 270 in the input array and assigning a value of the iteration value 210 to each value before the accumulator counter 215 exceeds the threshold value 220. In this case, since the rollover counter 240 was 2, the accumulator counter 215 accumulates the iteration value 210 such that the accumulator counter is 5, 8. 11, 14, etc. for each element after the second terminal value 270 and before the accumulator counter 215 next exceeds the threshold value 220 at the third terminal value 285. In particular, the second terminal value 270 contributes the unnormalized second weight 275, the following values up until the third terminal value 285 contribute the iteration value 210, and, in this case, the third terminal value 285 also contributes the iteration value 210 since the accumulator counter 215 is equal to the threshold value 220 at the third terminal value 285 to the second pixel element 296 of the output array.

[0049] In the subset of pixels shown, the last value of the input array along the given index dimension ends at the threshold value 220, e.g.. there is no rollover counter value at the third terminal value 285. In the case that there is a rollover counter value at the last value of the input array along the given index dimension, the system can assign a first weight to the terminal value, e.g., based on the difference between the accumulator counter and the threshold value at the terminal value. The system can then continue along each index dimension until all subsets of the first set of values along the particular dimension have been accumulated and normalized in the output array. Likewise, the system can continue iterating through each value of the subset of the input array along the given index dimension until all values in the subset have been normalized at least once in the output array.

[0050] FIG. 3 illustrates an example scaling in two dimensions, e.g., using a horizontal scaler to scale the width of an input image and a vertical scaler to scale the height of an input image. As an example, the downscaling system 100 of FIG. 1 can employ the horizontal scaler 300 as the first scaler 140 to horizontally scale the width, can store the intermediate values in the accumulation line buffer 320, and can employ the vertical scaler 350 as an additional scaler 150 to vertically scale the height.

[0051] In particular, FIG. 3 provides an overview of iterating along an index of a particular dimension, e.g., the horizontal width dimension and the vertical height dimension of input image 205. to determine the corresponding set of scaled weight value parameters using scaling processes 310 and 360.

[0052] In the particular example depicted, the horizontal scaler 300 can first perform a horizontal scaling process, e.g., the horizontal scaling process 310, by iterating through each value in the set of values in the input image 305 and calculating the corresponding set of scaled weight value parameters for each value, e.g., as described with respect to the scaling technique detailed in FIG. 2. For example, the horizontal scaler 300 can iteratively process each subset of values corresponding with an index of the horizontal dimension, e.g., the input line 0 330, input line 1 332, input line 2 (not depicted), input line 3 336 (right), etc., to generate values of the output array. The intermediate values from the first scaling, e.g., the horizontal scaling 300, can then be stored in the accumulation line buffer 210, e g., such that the intermediate horizontally scaled values can be used for the vertical scaling process 360. In particular, the system can scale along consecutive dimensions by chaining the resultant scaled values in prior dimensions, e.g., the vertical scaler 350 can use the intermediate values stored in the accumulation line buffer 320 to determine the values of the output array, e.g., the output image 390. In some cases, the system can additionally use an intermediate frame buffer, e.g., to store output values before rendering the output image 390.

[0053] As an example, the system can determine the length of the accumulation line buffer 320 based on the maximum dimension of the specified output dimension size(s), e.g., the output dimension size(s) 120 of FIG. 1. In some cases, the system can receive the maximum output dimension size from a user. In other cases, the system can determine the maximum output dimension size using the specified output dimension size(s). As an example, in the case that the maximum output array dimension is 256, the system can set the length of the accumulation line buffer to be 256. As described in FIG. 1, the line buffer 210 can be used to temporarily store the intermediate output array data for each dimension of scaling for the output array as the system performs downscaling in a next dimension, e g., before the final normalization is performed to generate the value of the output array. More specifically, for each index defining a subset of the first set of values, e.g., a horizontal line along the horizontal dimension in the input image 205, the scaler 200 can process each value in the subset of the input array along the horizontal line sequentially to determine the value of the output array in the first dimension. As an example, the value of the output array can be calculated and generated in a stream, e.g., as depicted in FIG. 2. The system can determine and store a sequence of values along each index of the horizontal dimension using process 310. e.g., until every subset of the first set of values corresponding with each horizontal index has been considered and the intermediate values are stored in the accumulation line buffer 320

[0054] For example, the system can store the intermediate horizontally scaled values for input lines 0 330 and 1 332 in the accumulation line buffer 320 using the process 200 of FIG. 2. The system can then perform the vertical scaling 360 using the vertical scaler 350 on the intermediate horizontally scaled values. As described in FIG. 2, the system can determine an iteration value and a threshold value based on the downscaling ratio for the particular dimension. As an example, the system can determine the vertical scaling using the scaler 150 for an example fractional downscaling ratio, e.g., for a 15:7 downscaling ratio, e.g., in which the height of the input image 205 is 15 pixels and the width of the output image is 7 pixels (or some multiple). In this case, the iteration value is 7 and the threshold value is 15.

[0055] In the case that the first row in the window is the first row in the output array, the system can assign an unnormalized weight of the iteration value. In other cases, the system can assign an unnormalized weight based on the rollover value, e.g., the unnormalized weight for the first value in the subset of values along the dimension or the most recent threshold value can be the remainder betw een the rollover counter and the iteration value and the unnonnalized weight for the terminal value can be the rollover counter value.

[0056] For each horizontal line processed in the input image, e.g., the completed rows 370 or the completed row-s 380, the vertical scaler 350 can be used to determine the scaled weight parameters for the vertical direction. The system can iterate through each value in the subset of values determined by an index along the vertical dimension and assign the same unnormalized weight, e.g., the iteration value weight, to all intermediate values, e.g., the horizontally scaled values in the line buffer, before the first threshold value in the vertical direction.

[0057] For example, the system can use the same line buffer, e.g., the accumulation line buffer 320, to store the intermediate vertical information, e.g., the quantity of the scaled weight parameters multiplied by the horizontally scaled values for each index, w ithout the need for a divider or separate memory location for each dimension. In particular, the system can store the accumulated weighted sum in the vertical dimension in the line buffer 320 until the terminal value is reached and normalization in the vertical dimension is performed, e.g., by multiplying by the quantity of 1 / 15 in the 15:7 downscaling ratio. In the particular example depicted, input line 1 332 is not a terminal value for the vertical dimension, so the corresponding unnormalized scaled weight parameters are stored in the accumulation line buffer.

[0058] When the terminal value is reached, e.g., at input line 3 336, the system can normalize the accumulated weighted sum in the line buffer to generate the corresponding output value. In particular, after the horizontal accumulation is complete for line 3 336 and the unnormalized scaled weights are multiplied by horizontal scaled value before storing to the accumulation line buffer, the system can use the intermediate values stored in the accumulation line buffer 320 for the completed rows 380 to perform a rolling normalization using the iteration value, threshold value, accumulator counter, and a rollover counter along the vertical direction by weighting the normalized scaled values of the horizontal calculation along the additional dimension using the accumulated vertical quantity in the accumulation line buffer 320.

[0059] In particular, the system can accumulate the weighted sum of the input array in arbitrary dimension using the same line buffer 320, e.g., by stonng intermediate scaled values from the prior scaled dimension in the accumulation line buffer, since the downscaling depends on the first prior dimension(s) scaled, e.g., the system can use the horizontally scaled outputs as the unsealed values for the vertical direction.

[0060] FIG. 4 demonstrates an example cropping method 400 for cropping the input array before scaling, e.g., by selecting a subset of the input array for scaling.

[0061] In some implementations, the input array 110 can be cropped 400 before it is scaled, e.g., using the cropping values 112 of FIG. 1 to specify the subset of the input array that the downscaling engine 130 processes. As an example, the system can receive cropping values that specify a starting and ending value in the first set of values for each dimension of the input array.

[0062] In the particular example depicted, the input array 110 is a two-dimensional array that is cropped using method 400 before being scaled with a horizontal scaler 140 and a vertical scaler 150 to generate the output array 160. In this case, the system can receive a crop top 410, crop left 420, crop right 430, and crop bottom 440 value, e g., from a user. In particular, the crop top value 410 and crop left value 420 specify the starting value in the subset of input values, e.g., by defining the index value of the first row and the index value of the first column, and the crop right 430 and crop bottom values 440 specify the ending value in the subset of input values, e.g., by defining the index value of the last row and the index value of the last column. The system can then process the cropped input array 450, e.g., by using the horizontal scaling method of FIG. 2 and the vertical scaling method of FIG. 3, to generate the output array 160.

[0063] FIG. 5 provides a comparison of an example image that has been downscaled using the downscaling system of FIG. 1 and an example image that has been downscaled using low-pass cubic filtering, e.g., by interpolating neighboring pixels using a kernel with a fixed number of taps. In this example, image 500 was downscaled by a factor of eight using the horizontal scaling method of FIG. 2 and the vertical scaling method of FIG. 3, and the image 550 was downscaled by a factor of eight using cubic filtering.

[0064] In particular, the difference between image 500 and image 550 demonstrates how the techniques of FIGS. 2 and 3 can generate a downscaled image with less aliasing artifacts and better denoising effects compared to the results of applying low-pass filters with a fixed number of taps. While cubic filtering is generally employed to reduce artifacts from aliasing, image 550 has more artifacts, e.g., the artifacts 552, 554, and 556, as a result of downscaling than the image 500. More specifically, perfonning a rolling nonnalized sum using an accumulator counter and rollover counter to determine a set of scaled weight parameters can enhance the sampling of the underlying signal of the array, e.g., such that it is greater than the Nyquist rate.

[0065] FIG. 6 is a How diagram of an example process for generating a downscaled output array using a set of scaled weight parameters. For convenience, the process 600 will be described as being performed by a system, e.g., a system including one or more of a data processing apparatus, logic circuitry, or another type of hardware module. For example, a downscaling system, e.g., the downscaling system 100 of FIG. 1, appropriately programmed in accordance with this specification, can perform the process 600.

[0066] The system can first receive an input array including a first set of values with one or more dimension sizes (step 610) and one or more corresponding dimension size(s) of an output array (step 620). Each dimension size can characterize the size of a dimension of the input or output array. For example, the input array can be an input image with two dimension sizes characterizing the width and height dimension, e.g., specified sizes of the width and height, and the one or more corresponding dimension size(s) can define the downscaled output image width and height sizes. In particular, the system can receive the input array and the output dimension size(s) from a user-device.

[0067] In some cases, the system can additionally select a subset of the input array for scaling, e.g., a subset of the input array. In particular, the system can obtain a starting and ending value for the first set of values, e.g., by receiving a set of cropping values that select a subset of the first set of values for scaling and determining the starting and ending value using the set of cropping values. As an example, the set of cropping values can include crop top, left, right, and bottom values that characterize a subset of an image for downscaling.

[0068] The system can then calculate a set of scaled weight parameters for each value in the first set of values using an accumulator and rollover counter (step 630). In particular, the system can calculate a set of scaled weight parameters for each dimension of the input array. As an example, the system can then calculate a normalized rolling sum by determining the corresponding set of scaled weight parameters that can be used to normalize each of the values in the first set of values. An example for calculating a normalized rolling sum using an accumulator and rollover counter will be covered in more detail in FIG. 7.

[0069] The system can then determine the second set of values for the output array based on the corresponding set of scaled weight parameters (step 640). As an example, the system can apply each corresponding scaled weight parameter to the respective value in the input array, e.g., by multiplying the respective corresponding scaled weight parameter and aggregating the scaled values in a weighted average, to generate a value of the output array.

[0070] In the case that the input array is being scaled in more than one dimension, the system can accumulate intermediate values along each dimension being scaled, e.g., the normalized values, in a line buffer. The values in the line buffer can then be used for scaling in another, e.g., a next dimension. For example, the system can calculate a set of intermediate horizontally nonnalized values using the horizontal scaled weight parameters and can accumulate a set of vertical scaled weight parameters that can be applied to the intermediate horizontally scaled values by scaling in a vertical direction.

[0071] The system can then generate an output array that includes the second set of values (step 660). In particular, the system can generate a downscaled output array that requires less computational resources to maintain than the input array. In some cases, the generated output array can be used for one or more downstream processes, e.g., a machine learning process such as training an image processing model.

[0072] FIG. 7 is a flow diagram of an example process for determining the set of scaled weight parameters. For convenience, the process 700 will be described as being performed by a system, e.g., a system including one or more of a data processing apparatus, logic circuitry, or another type of hardware module. For example, a downscaling system, e.g., the downscaling system 100 of FIG. 1, appropriately programmed in accordance with this specification, can perform the process 700. As an example, the system can perform the process 700 as part of step 630 in FIG. 6 to downscale each dimension of the input array specified for downscaling. In particular, the system can identify a threshold value based on a first dimension size of the input array (step 710), and identify an iteration value based on a corresponding dimension size of the output array (step 720). More specifically, the system can identify the particular dimension size of the input array and the output array corresponding to the downscaling of a particular dimension, thereby determining the downscaling ratio. The downscaling ratio can be any integer or fractional value.

[0073] The system can then use the iteration value and the threshold value to determine the corresponding scaled weight parameters for each value of the first set of values. In particular, the system can accumulate a sum of a rollover counter, e.g., a counter that is defined as 0 or the amount that the accumulator counter last exceeded the threshold value, and the accumulator counter at each value in the first set of values until the threshold value is met at a first terminal value (step 730). For example, the system can iterate through each value in the first set of values and assign an unnormalized w eight of the iteration value to each value while accumulating a sum of the rollover counter and the iteration value at each value in the input set of values until the accumulator counter satisfies a threshold criterion based on the threshold value.

[0074] The system can then determine the rollover counter (step 740). For example, the threshold criterion can be based on a difference between the accumulator counter and the threshold value at a first terminal value. The w eight of the terminal value can then impact the w eight of the values after the first terminal value.

[0075] The system can assign a corresponding scaled weight parameter to each value based on the accumulator counter and threshold value (step 750). In particular, each value before the first terminal value can be assigned the iteration value and the terminal value can be assigned the difference betw een the iteration value and the rollover value, and the threshold value. The first terminal value can be assigned an additional unnormalized weight of the rollover value, e.g., an additional weight that can be used as the unnormalized weight for the next value of the output array.

[0076] In particular, the system can repeat the process 700 until every value of the input array along a particular index of a dimension has been considered. The system can iterate through each value and designate one or more terminal values where the threshold criterion is met. As an example, the system can use the rollover counter to assign two weights to each terminal value, e.g., such that each terminal value contributes to multiple output values of the output array in the rolling normalization calculated in step 640 of process 600 in FIG. 6, e.g., for each dimension being scaled. This specification uses the term “configured” in connection with systems and computer program components. For a system of one or more computers to be configured to perform particular operations or actions means that the system has installed on it software, firmware, hardware, or a combination of them that in operation cause the system to perform the operations or actions. For one or more computer programs to be configured to perform particular operations or actions means that the one or more programs include instructions that, when executed by data processing apparatus, cause the apparatus to perform the operations or actions.

[0077] Embodiments of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry', in tangibly-embodied computer software or firmware, in computer hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non-transitory storage medium for execution by, or to control the operation of, data processing apparatus. The computer storage medium can be a machine- readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them. Alternatively or in addition, the program instructions can be encoded on an artificially-generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus.

[0078] The term “data processing apparatus” refers to data processing hardware and encompasses all kinds of apparatus, devices, and machines for processing data, including byway of example a programmable processor, a computer, or multiple processors or computers. The apparatus can also be, or further include, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). The apparatus can optionally include, in addition to hardware, code that creates an execution environment for computer programs, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0079] A computer program, which may also be referred to or described as a program, software, a software application, an app, a module, a software module, a script, or code, can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages; and it can be deployed in any form, including as a stand- alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A program may, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data, e.g., one or more scripts stored in a markup language document, in a single file dedicated to the program in question, or in multiple coordinated files, e.g., files that store one or more modules, subprograms, or portions of code. A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a data communication network.

[0080] In this specification the term “engine” is used broadly to refer to a software-based system, subsystem, or process that is programmed to perform one or more specific functions. Generally, an engine will be implemented as one or more software modules or components, installed on one or more computers in one or more locations. In some cases, one or more computers will be dedicated to a particular engine; in other cases, multiple engines can be installed and running on the same computer or computers.

[0081] The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by special purpose logic circuitry, e.g., an FPGA or an ASIC, or by a combination of special purpose logic circuitry and one or more programmed computers.

[0082] Computers suitable for the execution of a computer program can be based on general or special purpose microprocessors or both, or any other kind of central processing unit. Generally, a central processing unit will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a central processing unit for performing or executing instructions and one or more memory devices for storing instructions and data. The central processing unit and the memory can be supplemented by, or incorporated in, special purpose logic circuitry. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device, e.g., a universal serial bus (USB) flash drive, to name just a few.

[0083] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD- ROM and DVD-ROM disks.

[0084] To provide for interaction with a user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g.. a mouse or a trackball, by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input. In addition, a computer can interact with a user by sending documents to and receiving documents from a device that is used by the user; for example, by sending web pages to a web browser on a user’s device in response to requests received from the web browser. Also, a computer can interact with a user by sending text messages or other forms of message to a personal device, e.g., a smartphone that is running a messaging application, and receiving responsive messages from the user in return.

[0085] Data processing apparatus for implementing machine learning models can also include, for example, special-purpose hardware accelerator units for processing common and computeintensive parts of machine learning training or production, i.e., inference, workloads.

[0086] Machine learning models can be implemented and deployed using a machine learning framework, e.g., a TensorFlow framework, or a Jax framework.

[0087] Embodiments of the subject matter described in this specification can be implemented in a computing system that includes a back-end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front-end component, e.g., a client computer having a graphical user interface, a web browser, or an app through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication, e g., a communication network. Examples of communication networks include a local area network (LAN) and a wide area network (WAN), e.g., the Internet.

[0088] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In some embodiments, a server transmits data, e.g., an HTML page, to a user device, e.g., for purposes of displaying data to and receiving user input from a user interacting with the device, which acts as a client. Data generated at the user device, e.g., a result of the user interaction, can be received at the server from the device.

[0089] In addition to the embodiments described above, the following embodiments are also innovative:

[0090] Embodiment 1 is a method comprising: receiving an input array comprising a first set of values and a first dimension size characterizing a first dimension and a second dimension size characterizing a corresponding dimension of an output array; determining a second set of values for the output array from the first set of values based at least on a corresponding set of scaled weight parameters using an accumulator counter and a rollover counter, wherein determining the second set of values comprises: identifying a threshold value based on the first dimension size and an iteration value based on the corresponding dimension size; calculating the set of scaled weight parameters using the accumulator counter to accumulate a sum of the rollover counter and iteration values at each value by iterating through the first set of values until the accumulator counter satisfies a criterion with respect to the threshold value at a first terminal value; and generating the output array comprising the second set of values.

[0091] Embodiment 2 is the method of embodiment 1, wherein calculating the set of scaled weight parameters using the accumulator counter to accumulate a sum of the rollover counter and iteration values at each value further comprises: determining the rollover counter based on a difference between the accumulator counter and the threshold value at the first terminal value; and resetting the accumulator counter.

[0092] Embodiment 3 is the method of embodiment 2, further comprising: accumulating the sum of the rollover counter and the iteration value in the accumulator counter at each value in the first set after the first terminal value.

[0093] Embodiment 4 is the method of embodiment 3, further comprising: determining the rollover counter based on a difference between the accumulator counter and the threshold value at a second terminal value. Embodiment 5 is the method of any one of embodiments 1-4, further comprising: assigning the corresponding scaled weight parameter to each value before the first terminal value as the iteration value; or assigning a corresponding scaled weight parameter to the first terminal value as a difference between the iteration value and the rollover value.

[0094] Embodiment 6 is the method of embodiment 5, wherein determining the second set of values further comprises determining an initial value in the second set of values, wherein determining the initial value comprises: multiplying the respective corresponding scaled weight parameter with each value before the terminal value and the terminal value to generate scaled values; and aggregating the scaled values to determine the initial value.

[0095] Embodiment 7 is the method of embodiment 6, when dependent on the method of embodiment 4, further comprising determining a next value in the second set of values, wherein determining the next value in the second set of values comprises: assigning a second corresponding scaled weight parameter to the first terminal value as a remainder of the difference between the iteration value and the rollover value; multiplying the second corresponding scaled weight parameter to the first terminal value to generate a rollover scaled value; multiplying the respective corresponding scaled weight parameter with each value after the first terminal value and before the second terminal value to generate scaled values; and aggregating the rollover scaled value and the scaled values to determine the next value.

[0096] Embodiment 8 is the method of anyone of embodiments 1-7, wherein the input array further comprises a second dimension size characterizing a second dimension.

[0097] Embodiment 9 is the method of embodiment 8, wherein determining the second set of values for the output image further comprises, for each dimension: identifying a threshold value based on the dimension size and an iteration value based on the corresponding dimension size; calculating an intermediate set of values for each index of the dimension, wherein calculating comprises: using the index to select a subset of the first set of values; calculating the set of scaled weight parameters along the subset; and storing a quantity of intermediate values multiplied by the set of scaled weight parameters in a line buffer, wherein a length of the line buffer has been determined by a maximum corresponding dimension size of the output array.

[0098] Embodiment 10 is the method of embodiment 9, further comprising: at the threshold value of each dimension, determining a value of the second set of values by normalizing the intermediate values in the line buffer with respect to the threshold value.

[0099] Embodiment 1 1 is the method of embodiment 10, wherein the input array represents an input image comprising the first set of values, and wherein each value in the first set of values corresponds with a pixel in the input image.

[0100] Embodiment 12 is the method of embodiment 11, wherein generating the output array comprising the second set of values comprises generating an output image wherein each value in the second set of values corresponds with a pixel in the output image.

[0101] Embodiment 13 is the method of embodiment 12, wherein the input and output images comprise a number of channels, and further comprising: determining a second set of values for each channel of the output array from the first set of values for the corresponding channel of the input array based at least on a corresponding set of scaled weight parameters.

[0102] Embodiment 14 is the method of any one of embodiments 1-13, further comprising: obtaining a starting value and an ending value for the first set of values; and determining the second set of values from the starting value to the ending value.

[0103] Embodiment 15 is the method of embodiment 14, wherein obtaining the starting value and an ending value for the first set of values comprises: receiving a set of cropping values indicative of selecting a subset of the first set of values; determining the starting value and the ending value using the set of cropping values.

[0104] Embodiment 16 is the method of any one of embodiments 1-15, further comprising: decreasing a measure of storage needed to maintain the first array by generating and maintaining the second array.

[0105] Embodiment 17 is the method of any one of embodiments 1 1-16, wherein generating the output image further comprises generating a downscaled input image.

[0106] Embodiment 18 is the method of any one of embodiments 1-17, further comprising using the output array for a machine learning process. Embodiment 19 is the method of embodiment 18, wherein the machine learning process comprises training an image processing model.

[0107] Embodiment 20 is a system comprising data processing apparatus configured to perform the method of any one of claims 1 to 19.

[0108] Embodiment 21 is a computer storage medium encoded with instructions that are operable, when executed by data processing apparatus, to cause the data processing apparatus to perform the method of any one of claims 1 to 19.

[0109] While this specification contains many specific implementation details, these should not be constmed as limitations on the scope of any invention or on the scope of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially be claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

[0110] Similarly, while operations are depicted in the drawings and recited in the claims in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple softw are products.

[0111] Particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results. As one example, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous.

[0112] What is claimed is:

Claims

CLAIMS1. A method comprising: receiving an input array comprising a first set of values and a first dimension size characterizing a first dimension and a second dimension size characterizing a corresponding dimension of an output array; determining a second set of values for the output array from the first set of values based at least on a corresponding set of scaled weight parameters using an accumulator counter and a rollover counter, wherein determining the second set of values comprises: identifying a threshold value based on the first dimension size and an iteration value based on the corresponding dimension size; calculating the set of scaled weight parameters using the accumulator counter to accumulate a sum of the rollover counter and iteration values at each value by iterating through the first set of values until the accumulator counter satisfies a criterion with respect to the threshold value at a first terminal value; and generating the output array comprising the second set of values.

2. The method of claim 1, wherein calculating the set of scaled weight parameters using the accumulator counter to accumulate a sum of the rollover counter and iteration values at each value further comprises: determining the rollover counter based on a difference between the accumulator counter and the threshold value at the first terminal value; and resetting the accumulator counter.

3. The method of claim 2, further comprising: accumulating the sum of the rollover counter and the iteration value in the accumulator counter at each value in the first set after the first terminal value.

4. The method of claim 3, further comprising: determining the rollover counter based on a difference between the accumulator counter and the threshold value at a second terminal value.

5. The method of any one of claims 1-4, further comprising:assigning the corresponding scaled weight parameter to each value before the first terminal value as the iteration value; or assigning a corresponding scaled weight parameter to the first terminal value as a difference between the iteration value and the rollover value.

6. The method of claim 5, wherein determining the second set of values further comprises determining an initial value in the second set of values, wherein determining the initial value comprises: multiplying the respective corresponding scaled weight parameter with each value before the terminal value and the terminal value to generate scaled values; and aggregating the scaled values to determine the initial value.

7. The method of claim 6, when dependent on claim 4, further comprising determining a next value in the second set of values, wherein determining the next value in the second set of values comprises: assigning a second corresponding scaled weight parameter to the first terminal value as a remainder of the difference between the iteration value and the rollover value; multiplying the second corresponding scaled weight parameter to the first terminal value to generate a rollover scaled value; multiplying the respective corresponding scaled weight parameter with each value after the first terminal value and before the second terminal value to generate scaled values; and aggregating the rollover scaled value and the scaled values to determine the next value.

8. The method of any one of claims 1-7, wherein the input array further comprises a second dimension size characterizing a second dimension.

9. The method of claim 8, wherein determining the second set of values for the output image further comprises, for each dimension: identifying a threshold value based on the dimension size and an iteration value based on the corresponding dimension size; calculating an intermediate set of values for each index of the dimension, wherein calculating comprises: using the index to select a subset of the first set of values; calculating the set of scaled weight parameters along the subset; andstoring a quantity of intermediate values multiplied by the set of scaled weight parameters in a line buffer, wherein a length of the line buffer has been determined by a maximum corresponding dimension size of the output array.

10. The method of claim 9, further comprising: at the threshold value of each dimension, determining a value of the second set of values by normalizing the intermediate values in the line buffer with respect to the threshold value.

11. The method of claim 10, wherein the input array represents an input image comprising the first set of values, and wherein each value in the first set of values corresponds with a pixel in the input image.

12. The method of claim 11, wherein generating the output array comprising the second set of values comprises generating an output image wherein each value in the second set of values corresponds with a pixel in the output image.

13. The method of claim 12, wherein the input and output images comprise a number of channels, and further comprising: determining a second set of values for each channel of the output array from the first set of values for the corresponding channel of the input array based at least on a corresponding set of scaled weight parameters.

14. The method of any one of claims 1-13, further comprising: obtaining a starting value and an ending value for the first set of values; and determining the second set of values from the starting value to the ending value.

15. The method of claim 14, wherein obtaining the starting value and an ending value for the first set of values comprises: receiving a set of cropping values indicative of selecting a subset of the first set of values; determining the starting value and the ending value using the set of cropping values.

16. The method of any one of claims 1-15, further comprising:decreasing a measure of storage needed to maintain the first array by generating and maintaining the second array.

17. The method of any one of claims 11-16, wherein generating the output image further comprises generating a downscaled input image.

18. The method of any one of claims 1-17, further comprising using the output array for a machine learning process.

19. The method of claim 18, wherein the machine learning process comprises training an image processing model.

20. A system comprising data processing apparatus configured to perform the method of any one of claims 1 to 19.

21. A computer storage medium encoded with instructions that are operable, when executed by data processing apparatus, to cause the data processing apparatus to perform the method of any one of claims 1 to 19.

Images