Image Scaling Method, Apparatus, Electronic Device, and Computer-Readable Storage Medium

Abstract

An image scaling method, an image scaling device, an electronic device, and a computer-readable storage medium. The image scaling method includes: obtaining a scaling coefficient; based on the scaling coefficient, determining a target row number in the input image corresponding to each data row of the output image, and determining a target column number in the input image corresponding to each data column of the output image, so as to obtain a plurality of target row numbers corresponding to a plurality of data rows of the output image and a plurality of target column numbers corresponding to a plurality of data columns of the output image; based on the plurality of target row numbers and the plurality of target column numbers, determining a target coordinate point in the input image corresponding to each coordinate point of the output image; based on the target coordinate point corresponding to each coordinate point of the output image, determining the pixel value of each coordinate point of the output image, so as to obtain the pixel data of the output image. This method can improve the efficiency of image scaling processing.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to an image scaling method, apparatus, electronic device, and computer-readable storage medium. Background Art

[0002] In the field of artificial intelligence, in machine vision applications, when implementing the image scaling function, the resize operator is frequently used. The resize operator can adjust the size of the input image data to achieve image scaling. Summary of the Invention

[0003] At least one embodiment of the present disclosure provides an image scaling method, including: obtaining a scaling coefficient; based on the scaling coefficient, determining a target row number in the input image corresponding to each data row of the output image, and determining a target column number in the input image corresponding to each data column of the output image, so as to obtain a plurality of target row numbers respectively corresponding to a plurality of data rows of the output image and a plurality of target column numbers respectively corresponding to a plurality of data columns of the output image; based on the plurality of target row numbers and the plurality of target column numbers, determining a target coordinate point in the input image corresponding to each coordinate point of the output image; and based on the target coordinate point corresponding to each coordinate point of the output image, determining the pixel value of each coordinate point of the output image, so as to obtain the pixel data of the output image.

[0004] For example, in the image scaling method provided by at least one example of the above embodiment of the present disclosure, the calculation process of determining the plurality of target row numbers and the plurality of target column numbers based on the scaling coefficient is executed by a first processor; the calculation process of determining the pixel value of each coordinate point of the output image is executed by a second processor; the first processor supports the calculation of double-precision floating-point numbers, and the second processor supports the calculation of at least one type of data other than double-precision floating-point numbers.

[0005] For example, in the image scaling method provided by at least one example of the above embodiment of the present disclosure, based on the plurality of target row numbers and the plurality of target column numbers, determining a target coordinate point in the input image corresponding to each coordinate point of the output image includes: the first processor converts the plurality of target row numbers and the plurality of target column numbers into integers to obtain the converted plurality of target row numbers and the converted plurality of target column numbers; the first processor copies the converted plurality of target row numbers and the converted plurality of target column numbers to the second processor; and the second processor determines a target coordinate point in the input image corresponding to each coordinate point of the output image based on the converted plurality of target row numbers and the converted plurality of target column numbers.

[0006] For example, in the image scaling method provided by at least one example of the above embodiments of the present disclosure, the first processor is a central processing unit, and the second processor is a graphics processing unit.

[0007] For example, in the image scaling method provided by at least one example of the above embodiments of the present disclosure, the scaling factor includes a row direction scaling ratio and a column direction scaling ratio; based on the scaling factor, determining the target row number in the input image corresponding to each data row of the output image includes: based on the column direction scaling ratio, determining the target row number in the input image corresponding to each data row of the output image. Based on the scaling factor, determining the target column number in the input image corresponding to each data column of the output image includes: based on the row direction scaling ratio, determining the target column number in the input image corresponding to each data column of the output image.

[0008] For example, in the image scaling method provided by at least one example of the above embodiments of the present disclosure, the output image includes a first coordinate point, and the first coordinate point corresponds to a first target coordinate point in the input image. Based on the target coordinate points corresponding to each coordinate point of the output image, determining the pixel value of each coordinate point of the output image includes: determining the pixel values of four reference points corresponding to the first target coordinate point in the input image; based on the pixel values of the four reference points, determining the pixel value of the first coordinate point.

[0009] For example, in the image scaling method provided by at least one example of the above embodiments of the present disclosure, based on the multiple target row numbers and the multiple target column numbers, determining the target coordinate points in the input image corresponding to each coordinate point of the output image includes: for each coordinate point of the output image, using the target row number corresponding to the row where the coordinate point is located and the target column number corresponding to the column where the coordinate point is located to determine the target coordinate point.

[0010] At least one embodiment of the present disclosure provides an image scaling device, including an acquisition unit, a first determination unit, a second determination unit, and a third determination unit. The acquisition unit is configured to acquire a scaling factor; the first determination unit is configured to, based on the scaling factor, determine the target row numbers in the input image corresponding to each data row of the output image, and determine the target column numbers in the input image corresponding to each data column of the output image, so as to obtain multiple target row numbers corresponding to multiple data rows of the output image and multiple target column numbers corresponding to multiple data columns of the output image; the second determination unit is configured to, based on the multiple target row numbers and the multiple target column numbers, determine the target coordinate points in the input image corresponding to each coordinate point of the output image; the third determination unit is configured to, based on the target coordinate points corresponding to each coordinate point of the output image, determine the pixel value of each coordinate point of the output image, so as to obtain the pixel data of the output image.

[0011] At least one embodiment of the present disclosure provides an electronic device, including a processor; a memory storing one or more computer program modules; wherein, the one or more computer program modules are configured to be executed by the processor to implement the image scaling method provided by any embodiment of the present disclosure.

[0012] At least one embodiment of the present disclosure provides a computer-readable storage medium storing non-transitory computer-readable instructions, which can implement the image scaling method provided by any embodiment of the present disclosure when executed by a computer. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] To more clearly illustrate the technical solutions of the embodiments of the present disclosure, the accompanying drawings of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description only relate to some embodiments of the present disclosure and do not limit the present disclosure.

[0014] Figure 1 Shows a schematic diagram of an image scaling;

[0015] Figure 2 Shows a flowchart of an image scaling method provided by at least one embodiment of the present disclosure;

[0016] Figure 3 Shows a schematic diagram of a processing procedure provided by at least one embodiment of the present disclosure;

[0017] Figure 4 Shows a schematic diagram of calculating pixel values provided by at least one embodiment of the present disclosure;

[0018] Figure 5 Shows a schematic diagram of another processing procedure provided by at least one embodiment of the present disclosure;

[0019] Figure 6 Shows a schematic diagram of another input image provided by at least one embodiment of the present disclosure;

[0020] Figure 7 Shows a schematic block diagram of an image scaling device provided by at least one embodiment of the present disclosure;

[0021] Figure 8A Shows a schematic block diagram of an electronic device provided by at least one embodiment of the present disclosure;

[0022] Figure 8B Shows a schematic block diagram of another electronic device provided by at least one embodiment of the present disclosure; and

[0023] Figure 9Schematic diagram of a computer-readable storage medium provided by at least one embodiment of the present disclosure is shown. Detailed implementation manners

[0024] To make the objectives, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present disclosure. Obviously, the described embodiments are some but not all of the embodiments of the present disclosure. All other embodiments obtained by those of ordinary skill in the art based on the described embodiments of the present disclosure without creative efforts shall fall within the protection scope of the present disclosure.

[0025] Unless otherwise defined, the technical terms or scientific terms used in the present disclosure shall have the ordinary meanings understood by those of ordinary skill in the art to which the present disclosure pertains. The terms "first", "second" and similar terms used in the present disclosure do not denote any order, quantity or importance, but are only used to distinguish different components. Similarly, the terms such as "a", "an" or "the" do not denote a quantity limitation, but mean that there is at least one. The terms such as "include" or "comprise" mean that the elements or objects appearing before this term cover the elements or objects listed after this term and their equivalents, without excluding other elements or objects. The terms such as "connect" or "couple" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The terms such as "upper", "lower", "left" and "right" are only used to represent relative positional relationships, and when the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0026] When performing scaling calculations, the resize operator can be used to calculate the coordinates (out x , out y ) of each point in the output image (scaled image) corresponding to the coordinates (in x , in y ) in the input image (image before scaling) according to the scaling factor, and output the pixel value at the position (in x , in y ) to the point at the coordinates (out x , out y ).

[0027] Figure 1 Schematic diagram of an image scaling is shown.

[0028] As Figure 1As shown in the figure, the output image 120 is reduced by a factor of k (k is a value greater than 1) relative to the input image 110. For each point in the output image 120, the target coordinates in the corresponding input image 110 are calculated according to the scaling factor, and the target coordinates are rounded to determine the target point group. Then, the pixel values of the corresponding points in the output image 120 are determined according to the pixel values of the target point group in the input image, realizing the mapping from the input image to the output image. For example, for the point Q00 in the output image 120, the calculated target coordinates in the input image corresponding to the point Q00 are the point P11; for the point Q01 in the output image 120, the calculated target coordinates in the input image corresponding to the point Q01 are the point P14.

[0029] In the above manner, it is necessary to perform a calculation for each point in the output image until each point in the output image is traversed, resulting in a long time for the scaling process. Since the scaling operator is frequently used in machine vision applications, the long calculation time of the scaling operator will cause the entire processing process to be extended and the processing efficiency to be low. Therefore, how to improve the efficiency of image scaling calculation is a problem that needs to be solved.

[0030] At least one embodiment of the present disclosure provides an image scaling method, an image scaling device, an electronic device, and a computer-readable storage medium. The image scaling method includes: obtaining a scaling factor; based on the scaling factor, determining the target row numbers in the input image corresponding to each data row of the output image, and determining the target column numbers in the input image corresponding to each data column of the output image, so as to obtain a plurality of target row numbers corresponding to a plurality of data rows of the output image and a plurality of target column numbers corresponding to a plurality of data columns of the output image; based on the plurality of target row numbers and the plurality of target column numbers, determining the target coordinate points in the input image corresponding to each coordinate point of the output image; based on the target coordinate points corresponding to each coordinate point of the output image, determining the pixel values of each coordinate point of the output image, so as to obtain the pixel data of the output image.

[0031] In the image scaling process of this image scaling method, rows and columns are used as the calculation granularity to calculate the target row numbers of the input image corresponding to each row of the output image and the target column numbers of the input image corresponding to each column of the output image. Based on this, it can be quickly determined which coordinate points in the output image correspond to which coordinate points in the output image, reducing the number of calculations. Compared with the solution of performing a coordinate calculation for each coordinate point in the output image, the image scaling method of the embodiment of the present disclosure reduces the number of scaling calculations from H*W to H+W, improving the performance, where H is the number of rows of the output image and W is the number of columns of the output image.

[0032] Figure 2 The flowchart of an image scaling method provided by at least one embodiment of the present disclosure is shown.

[0033] AsFigure 2 As shown, the method may include steps S210 to S240.

[0034] Step S210: Obtain a scaling factor.

[0035] Step S220: Based on the scaling factor, determine the target row numbers in the input image corresponding to each data row of the output image, and determine the target column numbers in the input image corresponding to each data column of the output image, so as to obtain multiple target row numbers corresponding to multiple data rows of the output image and multiple target column numbers corresponding to multiple data columns of the output image.

[0036] Step S230: Based on the multiple target row numbers and multiple target column numbers, determine the target coordinate points in the input image corresponding to each coordinate point of the output image.

[0037] Step S240: Based on the target coordinate points corresponding to each coordinate point of the output image, determine the pixel values of each coordinate point of the output image, so as to obtain the pixel data of the output image.

[0038] For example, the input image can be represented by a matrix, and the output image can also be represented by a matrix.

[0039] For example, the scaling factor can be a scaling ratio, which can be determined based on the size of the output image and the size of the input image. The scaling factor can be pre-calculated and stored in a memory, and in step S110, the scaling factor is read from the memory.

[0040] For example, multiple data points in the input image corresponding to multiple data points in each row of the output image can be located in the same data row. Similarly, multiple data points in the input image corresponding to multiple data points in each column of the output image can be located in the same data column. Therefore, in step S220, for each data row of the output image, the target row number of the corresponding input image can be determined to indicate which data row in the input image each data row in the output image corresponds to, and for each data column of the output image, the target column number of the corresponding input image can be determined to indicate which data column in the input image each data column in the output image corresponds to. Since the scaling multiple may be an integer or a non-integer, the target row number and the target column number can each be an integer or a non-integer. In addition, in other embodiments, the target row number and the target column number can also be represented by other characters, as long as they can characterize the position of the target row in the input image. For the convenience of description, in the following embodiments, "data row" may also be abbreviated as "row", "data column" may also be abbreviated as "column", and "data point" may also be abbreviated as "point".

[0041] For example, the scaling factor includes the row - direction scaling ratio and the column - direction scaling ratio. For example, the row - direction scaling ratio can be based on the row - direction dimension width of the input image src (e.g., the number of columns) and the row - direction dimension width of the output image dst to be determined. For example, it can be the quotient of the two For example, the column - direction scaling ratio can be based on the column - direction dimension height of the input image src (e.g., the number of rows) and the row - direction dimension height of the output image dst to be determined. For example, it can be the quotient of the two

[0042] For example, step S220 can further include: determining the target row number in the input image corresponding to each data row of the output image based on the column - direction scaling ratio; and determining the target column number in the input image corresponding to each data column of the output image based on the row - direction scaling ratio.

[0043] For example, the following formula (1) can be used to calculate the target column number in the input image corresponding to each column in the output image. out x represents the column number of a certain column in the output image, in x represents the target column number in the input image corresponding to out x The coordinates in the input image can be represented by double - precision floating - point numbers. The target column number can be represented as U + u, where U represents the integer part and u represents the decimal part. The following formula (2) can be used to calculate the target row number in the input image corresponding to each row in the output image. out y represents the row number of a certain row in the output image, in y represents the target row number in the input image corresponding to out y The target row number can be represented as V + v, where V represents the integer part and v represents the decimal part.

[0044]

[0045]

[0046] Figure 3 shows a schematic diagram of a processing procedure provided by at least one embodiment of the present disclosure.

[0047] As Figure 3As shown, according to step S220, multiple data rows of the output image respectively correspond to multiple target row numbers [y0, y1, y2, …, ym] of the input image 310, where m is the total number of rows of the output image, and multiple data columns of the output image respectively correspond to multiple target column numbers [x0, x1, x2, …, xn] of the input image 310, where n is the total number of columns of the output image.

[0048] For example, in step S230, according to the multiple target row numbers [y0, y1, y2, …, ym] and the multiple target column numbers [x0, x1, x2, …, xn], the target coordinate points in the input image corresponding to each coordinate point in the output image are obtained.

[0049] For example, step S230 may further include: for each coordinate point in the output image, determining the corresponding target coordinate point based on the target row number corresponding to the row where the coordinate point is located and the target column number corresponding to the column where the coordinate point is located. For example, the output image includes a first coordinate point. The target row number of the input image corresponding to the row where the first coordinate point is located is y0, and the target column number of the input image corresponding to the column where the first coordinate point is located is x0. Then, the target coordinate point of the input image corresponding to the first coordinate point is (x0, y0). For another example, the output image includes a second coordinate point. The target row number of the input image corresponding to the row where the second coordinate point is located is y1, and the target column number of the input image corresponding to the column where the second coordinate point is located is x2. Then, the target coordinate point of the input image corresponding to the second coordinate point is (x2, y1). In this way, the target coordinates of the input image corresponding to each point in the output image can be obtained, and then the coordinate matrix 320 corresponding to the output image can be obtained.

[0050] For example, in step S240, in some embodiments, the pixel values of the target coordinate points in the input image can be obtained, and the pixel values of the target coordinate points in the input image are used as the pixel values of the corresponding coordinate points in the output image, so that the pixel values corresponding to each point in the output image can be obtained, forming a pixel value matrix 330. The pixel value matrix 330 can be used as the pixel data of the output image. For example, continuing with the above example, the pixel value V00 of the target coordinate point (x0, y0) in the input image can be used as the pixel value of the first coordinate point in the output image, and the pixel value of the target coordinate point (x2, y1) in the input image can be used as the pixel value of the second coordinate point in the output image. Based on this, the pixel values of each coordinate point in the output image 330 can be obtained, and then the pixel data of the output image is obtained.

[0051] For example, taking the first coordinate point in the output image as an example, this first coordinate point corresponds to the first target coordinate point in the input image. In step S240, the pixel values of four reference points corresponding to the first target coordinate point in the input image can be determined; based on the pixel values of these four reference points, the pixel value of the first coordinate point is determined.

[0052] For example, in the case of using the pixel value of the first target coordinate point as the pixel value of the first coordinate point, the coordinates of the first target coordinate point may not be integers, while only the pixel values of data points with integer coordinates (hereinafter also referred to as integer points) may be recorded in the input image. In the case where the coordinates of the first target coordinate point are non-integers, the pixel value of the first target coordinate point can be calculated by means of interpolation calculation.

[0053] Figure 4 FIG. shows a schematic diagram of calculating a pixel value provided by at least one embodiment of the present disclosure.

[0054] As Figure 4 shown, the horizontal coordinate (i.e., the X-axis coordinate) of the first target coordinate point P is U+u, and the vertical coordinate (i.e., the Y-axis coordinate) of the first target coordinate point P is V+v. The four intersections A, B, C, and D of the four lines of the vertical line X = U, X = U+1 and the horizontal lines Y = V and Y = V+1 are used as the four reference points of the first target coordinate point P. The pixel value of point P is obtained by performing three linear interpolations on the four points A, B, C, and D. That is, a linear interpolation is performed on A, C and B, D in the Y-axis direction respectively to obtain R1 and R2, and then a linear interpolation is performed on R1 and R2 in the X-axis direction respectively. The calculation formula is as follows formula (3):

[0055] f(P) = f(R1)*(1 - u)+f(R2)*u

[0056] = (f(C)*(1 - v)+f(A)*v)*(1 - u)+(f(D)*(1 - v)+f(B)*v)*u (3)

[0058] wherein, U = floor(U+u), V = floor(V+v), and floor is the floor function. Rounding operations are performed on U+u and V+v respectively to obtain U and V, and the four reference points A / B / C / D are determined according to the integers U and V. The decimal parts u and v are the offsets of point P relative to the four reference points and serve as the weights in formula (3). Based on the pixel values of the four reference points and the weights u and v, the pixel value of point P can be calculated.

[0059] For example, in some other embodiments, in step S240, the pixel value of an integer point adjacent to the first target coordinate point can be used as the pixel value of the first coordinate point. For example, as Figure 4As shown, the pixel value of the integer point C adjacent to the first target coordinate point P is used as the pixel value of the first coordinate point corresponding to the P point in the output image.

[0060] For example, image scaling can include image reduction and image enlargement. In the case of image reduction, each point in the output image corresponds to a point in the input image. In the case of image enlargement, multiple points in the output image may correspond to the same point in the input image, that is, multiple points in the output image are mapped to one point in the input image.

[0061] According to the image scaling method of the embodiments of the present disclosure, during the image scaling process, taking rows and columns as the calculation granularity, calculating the target row number of the input image corresponding to each row of the output image and the target column number of the input image corresponding to each column of the output image, based on which it is possible to quickly determine which coordinate point in the output image each coordinate point in the output image corresponds to, reducing the number of calculations. Compared with the solution of performing a coordinate calculation once for each coordinate point in the output image, the image scaling method of the embodiments of the present disclosure reduces the number of scaling calculations from H*W to H+W, improving the performance, where H is the number of rows of the output image and W is the number of columns of the output image.

[0062] For example, since the scaling factor may not be an integer multiple, the scaling coefficient may have multiple decimal places (for example, more than 6 decimal places). For the operation of calculating the target row number and target column number based on the scaling coefficient, if a processor capable of processing double-precision floating-point numbers (double) is used to perform this operation, higher precision can be achieved, and thus more accurate results can be obtained. However, some processors for performing image scaling processing (such as some graphics processors) cannot process double-precision floating-point numbers and can only process data with lower precision, such as single-precision floating-point numbers (float) or integers, resulting in a reduction in calculation precision. To solve this problem, the following embodiments are provided in the present disclosure.

[0063] For example, the image scaling method of the embodiments of the present disclosure can be jointly executed by a first processor and a second processor. The first processor supports double-precision floating-point number calculation, and the second processor supports the calculation of at least one type of data other than double-precision floating-point numbers. For example, the second processor can support the calculation of single-precision floating-point numbers (float) or integers. In addition, the first processor can also support the calculation of other types of data other than double-precision floating-point numbers, such as single-precision floating-point numbers or integers, etc. For example, the first processor can be a central processing unit, and the second processor can be a graphics processing unit. However, the present disclosure is not limited thereto, and the first processor and the second processor can also be other types of processors.

[0064] For example, a double-precision floating-point number (double) is a data type used by a computer. It uses 64 bits (8 bytes) to store a floating-point number and can be accurate to 12 decimal places. The first processor can process double-precision floating-point numbers, that is, it can perform operations on double-precision floating-point numbers. The second processor can process data of types such as single-precision floating-point numbers (float) and integers, but cannot process double-precision floating-point numbers. A single-precision floating-point number uses 32 bits (4 bytes) to store a floating-point number and can be accurate to 6 decimal places.

[0065] For example, the calculation process of determining multiple target row numbers and multiple target column numbers based on a scaling factor (such as step S220) can be executed by the first processor; the calculation process of determining the pixel value of each coordinate point of the output image (such as step S240) can be executed by the second processor.

[0066] For example, in the calculation process of the target row numbers and target column numbers in step S220, data with multiple decimal places (for example, the number of digits after the decimal point is greater than or equal to 12) may be involved. In this case, by using the first processor to execute step S220 above, accurate calculation can be achieved. The first processor can copy the calculation result to the second processor to use the second processor to perform subsequent processing. For example, the first processor can put the calculation result into the memory, and the second processor reads the calculation result from the memory.

[0067] For example, step S230 may include: the first processor converts multiple target row numbers and multiple target column numbers into integers to obtain the converted multiple target row numbers and the converted multiple target column numbers; the first processor copies the converted multiple target row numbers and the converted multiple target column numbers to the second processor; the second processor determines the target coordinate points in the input image corresponding to each coordinate point of the output image based on the converted multiple target row numbers and the converted multiple target column numbers.

[0068] Figure 5 The figure shows a schematic diagram of another processing process provided by at least one embodiment of the present disclosure.

[0069] Such as Figure 5As shown, after obtaining multiple target row numbers [y0, y1, y2, …, ym] and multiple target column numbers [x0, x1, x2, …, xn] according to step S220, if the target row numbers and target column numbers are double-precision floating-point numbers, before copying the data to the second processor, the double-precision floating-point numbers can be rounded down using a floor function (such as the floor function) to convert the double-precision floating-point numbers into integers. For example, the double-precision floating-point numbers U+u and V+v are respectively converted into U and V. The multiple converted target row numbers and the multiple converted target column numbers are all integers. The multiple converted target row numbers are represented as the first vector [y0’, y1’, y2’, …, ym’], and the multiple converted target column numbers are represented as the second vector [x0’, x1’, x2’, …, xn’]. Then, the converted data is copied to the second processor. The second processor traverses each point of the output image and takes the corresponding values in the first vector and the second vector according to the row number and column number of the current point to form coordinates, thereby obtaining the coordinate matrix 520 corresponding to the output image. The coordinates in this coordinate matrix 520 are all integers. Then, the pixel value corresponding to each point of the output image is further calculated to obtain the pixel value matrix 530 corresponding to the output image.

[0070] For example, after the second processor obtains the integer target row numbers and target column numbers transmitted by the first processor, the second processor can, based on the target row numbers and target column numbers, determine the first target coordinate point of the input image corresponding to the first coordinate point in the output image, and the coordinates of the determined first target coordinate point are the integers U and V. In step S240, as Figure 4 shown, four reference points A / B / C / D can be determined based on the integers U and V. For example, due to the different precisions of double-type data and float-type data, there are some errors between the integers (U and V) determined based on double-type data and the integers (U and V) determined based on float-type data, and this error may be amplified to the integer 1. For example, U+u determined based on double-type data is 2.0356…, and the rounded-down integer U is 2; U+u determined based on float-type data is 1.8492…, and the rounded-down integer U is 1. Therefore, compared with double-type data, the integers determined based on float-type data have a relatively large error, that is, the integers determined based on double-type data are more accurate. The first processor copies the integers determined based on double-type data to the second processor, and the four reference points A / B / C / D determined by the second processor based on these integers are also more accurate, thereby making the pixel value of the calculated first coordinate point more accurate.

[0071] For example, in the process of calculating the pixel value of the first coordinate point based on four reference points A / B / C / D, the fractional parts u and v are required. For this requirement, the second processor can perform the calculations of formulas (1) and (2) based on float-type data internally to obtain the fractional parts u and v, and perform the interpolation calculation of formula (3) using the pixel values of the four reference points A / B / C / D and the fractional parts u and v.

[0072] For example, in some other embodiments, after the first processor calculates multiple target row numbers and multiple target column numbers, it can convert double-precision floating-point numbers into single-precision floating-point numbers (float) to obtain the converted multiple target row numbers and the converted multiple target column numbers, and both the converted multiple target row numbers and the converted multiple target column numbers are single-precision floating-point numbers. Copy the converted single-precision floating-point numbers to the second processor, and the second processor can determine the first target coordinate point of the input image corresponding to the first coordinate point of the output image. The determined first target coordinate point is a single-precision floating-point number. Round down the single-precision floating-point number to obtain U and V, determine the four reference points A / B / C / D corresponding to the first target coordinate point based on the integers U and V, and calculate the pixel value of the first coordinate point in the output image using the fractional parts u and v of the single-precision floating-point number and the pixel values of the four reference points. For example, compared with the method of converting double-precision floating-point numbers into single-precision floating-point numbers and then converting the single-precision floating-point numbers into integers, the integers determined by directly converting double-precision floating-point numbers into integers are more accurate.

[0073] For example, in some other embodiments, after the first processor calculates double-precision data, it rounds down the double-precision data to obtain an integer, and copies the integer and the first several decimal places (e.g., the first 6 decimal places) of the double-precision floating-point number to the second processor. The second processor determines four reference points based on the integer, and performs the calculation of formula (3) based on the pixel values of the four reference points and the copied fractional part to obtain the pixel value of the first data point.

[0074] For example, in some other embodiments, if the second processor (such as a graphics processor) supports the calculation of double-precision floating-point numbers, the above steps S210 to S240 can all be executed by the graphics processor. In this embodiment, the target row number and the target column number obtained in step S220 are both double-type data. In step S230, the coordinates of the determined first target coordinate point are double-type data. Round down the first target coordinate point to obtain integers U and V, determine the four reference points A / B / C / D based on the integers U and V, and calculate the pixel value of the first coordinate point in the output image based on the pixel values of the four reference points and the fractional parts u and v of the double-type data.

[0075] Figure 6 Shows a schematic diagram of another input image provided by at least one embodiment of the present disclosure.

[0076] As Figure 6 shown, the input image may include multiple data channels, such as data channels 601, 602, and 603. Each data channel includes a data matrix, and each data matrix includes multiple rows and multiple columns. Therefore, the input image includes a channel dimension, a row dimension, and a column dimension. Similarly, the output image may also include a channel dimension, a row dimension, and a column dimension, and the number of channels of the output image may be the same as that of the input image. In some embodiments of the present disclosure, steps S210 to S240 may be performed for each channel.

[0077] For example, the image scaling method of the embodiments of the present disclosure is used for a scaling operator (resize operator) in neural network computing. That is to say, the image scaling method of the embodiments of the present disclosure may be implemented as a scaling operator. During the neural network computing process, the scaling operator may be called at each node where image scaling needs to be performed to perform image scaling processing using steps S210 to S240. Based on this method, the computing efficiency can be greatly improved and the processing performance can be enhanced.

[0078] For example, the Resize operator may have three modes: BILINEAR (bilinear allocation method) / CUBIC (cubic convolution interpolation method) / NEAREST (nearest neighbor allocation method), and the coordinate mapping methods of each mode are different. Taking BILINEAR as an example, if the method of performing coordinate scaling calculation point by point on the output image is adopted, there will be repeated calculations in the same row and the same column, and the coordinate calculation generates a large amount of register occupancy. The embodiments of the present disclosure improve the resize operator. Taking the BILINEAR mode as an example, the row number and column number are scaled and calculated, and the calculation results are accessed row by row and column by column to obtain the corresponding coordinates. The number of scaling calculations is reduced from H*W to H+W, improving the processing efficiency.

[0079] The image scaling method of at least one embodiment of the present disclosure proposes to move the resize coordinate scaling calculation to the central processing unit, which can obtain the calculation results of data types not supported by the graphics processing unit, improve the accuracy, and reduce the register usage of the graphics processing unit, thereby improving the performance.

[0080] Figure 7 FIG. shows a schematic block diagram of an image scaling device 700 provided by at least one embodiment of the present disclosure.

[0081] For example, as Figure 7As shown, the image scaling device 700 includes an acquisition unit 710, a first determination unit 720, a second determination unit 730, and a third determination unit 740. These components are interconnected through a bus system and / or other forms of connection mechanisms (not shown). For example, these modules can be implemented through hardware (such as circuits), software modules, or any combination of the two. The same applies to the following embodiments and will not be elaborated further. For example, these units can be implemented through a central processing unit (CPU), a graphics processing unit (GPU), a tensor processing unit (TPU), a field-programmable gate array (FPGA), or other forms of processing units with data processing capabilities and / or instruction execution capabilities, as well as corresponding computer instructions. It should be noted that Figure 7 The components and structures of the image scaling device 700 shown are exemplary rather than restrictive. According to requirements, the image scaling device 700 can also have other components and structures.

[0082] The acquisition unit 710 is configured to acquire a scaling factor. The acquisition unit 710 can, for example, execute Figure 2 the steps described in S210.

[0083] The first determination unit 720 is configured to determine, based on the scaling factor, the target row numbers in the input image corresponding to each data row of the output image, and determine the target column numbers in the input image corresponding to each data column of the output image, so as to obtain a plurality of target row numbers corresponding to the plurality of data rows of the output image and a plurality of target column numbers corresponding to the plurality of data columns of the output image. The first determination unit 720 can, for example, execute Figure 2 the steps described in S220.

[0084] The second determination unit 730 is configured to determine, based on the plurality of target row numbers and the plurality of target column numbers, the target coordinate points in the input image corresponding to each coordinate point of the output image. The second determination unit 730 can, for example, execute Figure 2 the steps described in S230.

[0085] The third determination unit 740 is configured to determine, based on the target coordinate points corresponding to each coordinate point of the output image, the pixel value of each coordinate point of the output image, so as to obtain the pixel data of the output image. The third determination unit 740 can, for example, execute Figure 2 the steps described in S240.

[0086] For example, the obtaining unit 710, the first determination unit 720, the second determination unit 730, and the third determination unit 740 may be hardware, software, firmware, or any feasible combination thereof. For example, the obtaining unit 710, the first determination unit 720, the second determination unit 730, and the third determination unit 740 may be dedicated or general-purpose circuits, chips, or devices, etc., or may be a combination of a processor and a memory. Regarding the specific implementation forms of the above-mentioned respective units, the embodiments of the present disclosure do not limit this.

[0087] For example, the obtaining unit 710, the first determination unit 720, the second determination unit 730, and the third determination unit 740 may include codes and programs stored in a memory; the processor may execute the codes and programs to implement some or all of the functions of the obtaining unit 710, the first determination unit 720, the second determination unit 730, and the third determination unit 740 as described above. For example, the obtaining unit 710, the first determination unit 720, the second determination unit 730, and the third determination unit 740 may be dedicated hardware devices for implementing some or all of the functions of the obtaining unit 710, the first determination unit 720, the second determination unit 730, and the third determination unit 740 as described above. For example, the obtaining unit 710, the first determination unit 720, the second determination unit 730, and the third determination unit 740 may be a circuit board or a combination of multiple circuit boards for implementing the functions as described above. In the embodiments of the present disclosure, the combination of the one circuit board or multiple circuit boards may include: (1) one or more processors; (2) one or more non-transitory memories connected to the processor; and (3) firmware stored in the memory that can be executed by the processor.

[0088] It should be noted that in the embodiments of the present disclosure, each unit of the image scaling device 700 corresponds to each step of the foregoing image scaling method. For the specific functions of the image scaling device 700, reference may be made to the relevant descriptions of the image scaling method, and details are not elaborated herein. Figure 7 The components and structures of the illustrated image scaling device 700 are merely exemplary and not restrictive. According to requirements, the image scaling device 700 may further include other components and structures. The image scaling device 700 may include more or fewer circuits or units, and the connection relationships between the respective circuits or units are not limited and may be determined according to actual needs. The specific constitution manners of the respective circuits or units are not limited and may be constituted by analog devices according to circuit principles, or may be constituted by digital chips, or in other applicable manners.

[0089] For example, in the image scaling device provided by at least one example of the foregoing embodiments of the present disclosure, the first determination unit may be used in the first processor, that is, the first processor includes the first determination unit; the third determination unit may be used in the second processor, that is, the second processor includes the third determination unit; the first processor supports the calculation of double-precision floating-point numbers, and the second processor supports the calculation of at least one type of data other than double-precision floating-point numbers.

[0090] For example, in the image scaling device provided by at least one example of the foregoing embodiments of the present disclosure, the first processor converts the plurality of target row numbers and the plurality of target column numbers into integers to obtain the converted plurality of target row numbers and the converted plurality of target column numbers; the first processor copies the converted plurality of target row numbers and the converted plurality of target column numbers to the second processor; the second processor determines the target coordinate points in the input image corresponding to each coordinate point of the output image based on the converted plurality of target row numbers and the converted plurality of target column numbers.

[0091] For example, in the image scaling device provided by at least one example of the foregoing embodiments of the present disclosure, the first processor is a central processing unit, and the second processor is a graphics processing unit.

[0092] For example, in the image scaling device provided by at least one example of the foregoing embodiments of the present disclosure, the scaling factor includes a row-direction scaling ratio and a column-direction scaling ratio; the first determination unit is further configured to: determine the target row numbers in the input image corresponding to each data row of the output image based on the column-direction scaling ratio; determine the target column numbers in the input image corresponding to each data column of the output image based on the row-direction scaling ratio.

[0093] For example, in the image scaling device provided by at least one example of the foregoing embodiments of the present disclosure, the output image includes a first coordinate point, and the first coordinate point corresponds to a first target coordinate point in the input image. The fourth processing unit is further configured to: determine the pixel values of four reference points corresponding to the first target coordinate point in the input image; determine the pixel value of the first coordinate point based on the pixel values of the four reference points.

[0094] For example, in the image scaling device provided by at least one example of the foregoing embodiments of the present disclosure, the second processing unit is further configured to: for each coordinate point of the output image, determine the target coordinate point based on the target row number corresponding to the row where the coordinate point is located and the target column number corresponding to the column where the coordinate point is located.

[0095] At least one embodiment of the present disclosure further provides an electronic device, which includes a processor and a memory. The memory stores one or more computer program modules. The one or more computer program modules are configured to be executed by the processor to implement the above image scaling method. During the image scaling process, the electronic device takes rows and columns as the calculation granularity, calculates the target row number of the input image corresponding to each row of the output image and the target column number of the input image corresponding to each column of the output image. Based on this, it can quickly determine which coordinate point in the input image corresponds to each coordinate point in the output image, reducing the number of calculations and improving the processing efficiency.

[0096] Figure 8A FIG. is a schematic block diagram of an electronic device provided in some embodiments of the present disclosure. As Figure 8A shown, the electronic device 800 includes a processor 810 and a memory 820. The memory 820 stores non-transitory computer-readable instructions (such as one or more computer program modules). The processor 810 is configured to run the non-transitory computer-readable instructions, and when the non-transitory computer-readable instructions are run by the processor 810, one or more steps in the above-described image scaling method are executed. The memory 820 and the processor 810 may be interconnected through a bus system and / or other forms of connection mechanisms (not shown). For the specific implementation and related explanation content of each step of the image scaling method, reference may be made to the embodiments of the above image scaling method, and repeated parts will not be elaborated here.

[0097] It should be noted that Figure 8A the components of the electronic device 800 shown are exemplary and non-limiting. According to actual application needs, the electronic device 800 may also have other components.

[0098] For example, the processor 810 and the memory 820 may communicate directly or indirectly with each other.

[0099] For example, the processor 810 and the memory 820 may communicate through a network. The network may include a wireless network, a wired network, and / or any combination of a wireless network and a wired network. The processor 810 and the memory 820 may also communicate with each other through a system bus, and the present disclosure does not limit this.

[0100] For example, the processor 810 and the memory 820 may be disposed on the server side (or cloud).

[0101] For example, the processor 810 may control other components in the electronic device 800 to perform desired functions. For example, the processor 810 may be a central processing unit (CPU), a graphics processing unit (GPU), or other forms of processing units with data processing capabilities and / or program execution capabilities. For example, the central processing unit (CPU) may be of the X86 or ARM architecture, etc. The processor 810 may be a general-purpose processor or a dedicated processor, and may control other components in the electronic device 800 to perform desired functions.

[0102] For example, the memory 820 may include any combination of one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory, etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, flash memory, etc. One or more computer program modules may be stored on the computer-readable storage media, and the processor 810 may run one or more computer program modules to implement various functions of the electronic device 800. Various application programs and various data, as well as various data used and / or generated by the application programs, etc., may also be stored in the computer-readable storage media.

[0103] It should be noted that in the embodiments of the present disclosure, the specific functions and technical effects of the electronic device 800 may refer to the description of the image scaling method in the foregoing text, and will not be elaborated herein.

[0104] Figure 8B A schematic block diagram of another electronic device provided for some embodiments of the present disclosure. The electronic device 900 is, for example, suitable for implementing the image scaling method provided by the embodiments of the present disclosure. The electronic device 900 may be a terminal device, etc. It should be noted that Figure 8B The illustrated electronic device 900 is only an example, and it will not impose any limitations on the functions and usage scope of the embodiments of the present disclosure.

[0105] Such as Figure 8BAs shown, the electronic device 900 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 910, which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 920 or a program loaded from a storage device 980 into a random access memory (RAM) 930. In the RAM 930, various programs and data required for the operation of the electronic device 900 are also stored. The processing device 910, the ROM 920, and the RAM 930 are connected to each other through a bus 940. An input / output (I / O) interface 950 is also connected to the bus 940.

[0106] Generally, the following devices may be connected to the I / O interface 950: an input device 960 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 970 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 980 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 990. The communication device 990 may allow the electronic device 900 to communicate with other electronic devices wirelessly or wireline to exchange data. Although Figure 8B the electronic device 900 with various devices is shown, it should be understood that it is not required to implement or have all the shown devices, and the electronic device 900 may alternatively implement or have more or fewer devices.

[0107] For example, according to an embodiment of the present disclosure, the above image scaling method may be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes program codes for performing the above image scaling method. In such an embodiment, the computer program may be downloaded and installed from a network through the communication device 990, or installed from the storage device 980, or installed from the ROM 920. When the computer program is executed by the processing device 910, the functions defined in the image scaling method provided by the embodiment of the present disclosure may be implemented.

[0108] At least one embodiment of the present disclosure also provides a computer-readable storage medium, which stores non-transitory computer-readable instructions, and when the non-transitory computer-readable instructions are executed by a computer, the above image scaling method may be implemented. By using this computer-readable storage medium, during the image scaling process, taking rows and columns as the calculation granularity, calculating the target row numbers of the input image corresponding to each row of the output image and the target column numbers of the input image corresponding to each column of the output image, based on which each coordinate point in the output image can be quickly determined corresponding to which coordinate point in the output image, reducing the number of calculations and improving the processing efficiency.

[0109] Figure 9Schematic diagram of a storage medium provided by some embodiments of the present disclosure. As Figure 9 shown, the storage medium 1000 stores non-transitory computer-readable instructions 1010. For example, when the non-transitory computer-readable instructions 1010 are executed by a computer, one or more steps in the image scaling method described above are executed.

[0110] For example, the storage medium 1000 can be applied to the above-mentioned electronic device 800. For example, the storage medium 1000 can be Figure 8A the memory 820 in the electronic device 800 shown. For example, the relevant description of the storage medium 1000 can refer to Figure 8A the corresponding description of the memory 820 in the electronic device 800 shown, which will not be elaborated here.

[0111] The above description is only a preferred embodiment of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept. For example, the technical solutions formed by mutually replacing the above features with the (but not limited to) technical features with similar functions disclosed in the present disclosure.

[0112] In addition, although the operations are depicted in a specific order, this should not be construed as requiring that the operations be performed in the specific order shown or in sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments may also be implemented combinatorially in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented separately or in any suitable sub-combination in multiple embodiments.

[0113] Although the subject matter has been described in language specific to structural features and / or method logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. On the contrary, the specific features and acts described above are merely example forms for implementing the claims.

[0114] Regarding the present disclosure, the following points need to be noted:

[0115] (1) The drawings of the embodiments of the present disclosure only relate to the structures involved in the embodiments of the present disclosure, and other structures can refer to the general design.

[0116] (2) Without conflict, the embodiments of the present disclosure and the features in the embodiments may be combined with each other to obtain new embodiments.

[0117] As described above, it is only the specific implementation manner of the present disclosure, but the protection scope of the present disclosure is not limited thereto. The protection scope of the present disclosure shall be subject to the protection scope of the claims.

Claims

1. An image scaling method, comprising: Obtaining a scaling coefficient for an input image and an output image, wherein the scaling coefficient includes a scaling ratio, and the output image includes M data rows and N data columns; Performing M row coordinate scaling operations based on the scaling coefficient to determine M target row numbers in the input image corresponding to the M data rows respectively; Performing N column coordinate scaling operations based on the scaling coefficient to determine N target column numbers in the input image corresponding to the N data columns respectively; Determining target coordinate points in the input image corresponding to each coordinate point of the output image based on the M target row numbers and the N target column numbers; and Determining pixel values of each coordinate point of the output image based on the target coordinate points corresponding to each coordinate point of the output image to obtain pixel data of the output image; Wherein, determining target coordinate points in the input image corresponding to each coordinate point of the output image based on the M target row numbers and the N target column numbers includes: For a first coordinate point in the output image, finding a first target row number in the M target row numbers corresponding to the data row where the first coordinate point is located; Finding a first target column number in the N target column numbers corresponding to the data column where the first coordinate point is located; and Combining the first target row number and the first target column number to obtain a target coordinate point corresponding to the first coordinate point; Wherein, the calculation process of determining the M target row numbers and the N target column numbers based on the scaling coefficient is executed by a first processor; the calculation process of determining pixel values of each coordinate point of the output image is executed by a second processor; wherein, the first processor supports calculations of double-precision floating-point numbers, and the second processor supports calculations of at least one type of data other than double-precision floating-point numbers.

2. The image scaling method according to claim 1, wherein, Determining target coordinate points in the input image corresponding to each coordinate point of the output image based on the M target row numbers and the N target column numbers includes: The first processor converts the M target row numbers and the N target column numbers into integers to obtain the converted M target row numbers and the converted N target column numbers; The first processor copies the converted M target row numbers and the converted N target column numbers to the second processor; The second processor determines target coordinate points in the input image corresponding to each coordinate point of the output image based on the converted M target row numbers and the converted N target column numbers.

3. The image scaling method according to claim 1 or 2, wherein, The first processor is a central processing unit, and the second processor is a graphics processing unit.

4. The image scaling method according to claim 1, wherein, The scaling coefficient includes a row direction scaling ratio and a column direction scaling ratio; Determining target row numbers in the input image corresponding to each data row of the output image based on the scaling coefficient includes: determining target row numbers in the input image corresponding to each data row of the output image based on the column direction scaling ratio; Based on the scaling factor, determining the target column numbers in the input image corresponding to each data column of the output image, including: based on the row-direction scaling ratio, determining the target column numbers in the input image corresponding to each data column of the output image.

5. The image scaling method according to claim 1, wherein The output image includes a first coordinate point, and the first coordinate point corresponds to a first target coordinate point in the input image; Based on the target coordinate points corresponding to each coordinate point of the output image, determining the pixel value of each coordinate point of the output image, including: Determining the pixel values of four reference points corresponding to the first target coordinate point in the input image; Based on the pixel values of the four reference points, determining the pixel value of the first coordinate point.

6. An image scaling device, comprising: An acquisition unit configured to acquire a scaling factor regarding an input image and an output image, where the scaling factor includes a scaling ratio, and the output image includes M data rows and N data columns; A first determination unit configured to perform M row coordinate scaling operations based on the scaling factor to determine M target row numbers in the input image corresponding to the M data rows respectively, and perform N column coordinate scaling operations based on the scaling factor to determine N target column numbers in the input image corresponding to the N data columns respectively; A second determination unit configured to determine the target coordinate points in the input image corresponding to each coordinate point of the output image based on the M target row numbers and the N target column numbers; and A third determination unit configured to determine the pixel value of each coordinate point of the output image based on the target coordinate points corresponding to each coordinate point of the output image to obtain the pixel data of the output image; wherein the second determination unit is further configured to: For a first coordinate point in the output image, find the first target row number corresponding to the data row where the first coordinate point is located from the M target row numbers; Find the first target column number corresponding to the data column where the first coordinate point is located from the N target column numbers; and Combine the first target row number and the first target column number to obtain the target coordinate point corresponding to the first coordinate point; wherein the calculation process of determining the M target row numbers and the N target column numbers based on the scaling factor is executed by a first processor; the calculation process of determining the pixel value of each coordinate point of the output image is executed by a second processor; wherein the first processor supports the calculation of double-precision floating-point numbers, and the second processor supports the calculation of at least one type of data other than double-precision floating-point numbers.

7. An electronic device, comprising: A processor; A memory storing one or more computer program modules; wherein the one or more computer program modules are configured to be executed by the processor to implement the image scaling method according to any one of claims 1-5.

8. A computer-readable storage medium storing non-temporary computer-readable instructions that can implement the image scaling method according to any one of claims 1-5 when executed by a computer.

Images