Image processing method and device and computer storage medium

Abstract

The invention provides an image processing method and device, electronic equipment and a computer readable storage medium, and relates to the technical field of image processing. The method comprises the following steps: according to the line number proportion and / or column number proportion of a target output image and an input image, determining a scaling mode and a proportion combination scheme; wherein the proportion combination scheme comprises an ordered combination of at least one basic increasing and reducing proportion; calculating a filter coefficient group according to a proportional combination scheme; wherein the filter coefficient group comprises a current filter coefficient corresponding to each pixel converted from the input image to the output image; and calculating an output pixel value corresponding to each pixel of the output image based on the filter coefficient group, and outputting image data formed by the output pixel values. Determining an increasing and decreasing mode and proportion combination scheme through a row number proportion and / or a column number proportion, so that a filtering coefficient group covers all pixels from input to output at one time, and redundant calculation is reduced; the orderly combined basic increasing and reducing proportion enables the filtering coefficient to be reusable, and the storage demand is reduced.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, to an image processing method, apparatus, electronic device, and computer storage medium. Background Technology

[0002] In the field of image display technology, the mismatch between the input image resolution and the inherent resolution of the display device is a common technical challenge. Traditional solutions mainly employ two methods, but both have significant drawbacks: the first is direct row addition and deletion operations, which, while simple to implement, introduces obvious visual artifacts, resulting in uneven image transitions and visible stair-step effects and banding at gradient regions and texture edges, severely impairing the visual experience; the second is proportional scaling techniques, which, while employing multi-tap filtering algorithms to ensure good image quality, require complex or larger-scale convolution operations, leading to a geometric increase in computational complexity. Furthermore, it requires caching multiple rows of high-resolution image data, placing extremely high demands on hardware storage bandwidth and computing resources, significantly increasing chip area, power consumption, and design complexity, making it impractical in resource-constrained scenarios such as mobile devices. Existing technologies either sacrifice visual quality for ease of implementation or maintain quality at the cost of high hardware costs, lacking an effective solution that strikes a balance between visual quality and implementation complexity. Summary of the Invention

[0003] In view of this, the purpose of this application is to provide an image processing method, apparatus, electronic device, and computer storage medium to improve the above-mentioned problems existing in the prior art.

[0004] In a first aspect, embodiments of this application provide an image processing method, the method comprising: determining a scaling mode and a scaling ratio combination scheme based on the row ratio and / or column ratio of a target output image to an input image; wherein the scaling ratio combination scheme includes an ordered combination of at least one basic scaling ratio; calculating a filter coefficient set based on the scaling ratio combination scheme; wherein the filter coefficient set includes the current filter coefficient corresponding to each pixel of the input image converted to the output image; and calculating the output pixel value corresponding to each pixel of the output image based on the filter coefficient set, and outputting image data composed of the output pixel values.

[0005] In the above implementation process, the scaling mode and scaling combination scheme are directly driven by the row number ratio and / or column number ratio, so that the filter coefficient group covers all pixels from input to output at one time, reducing redundant calculations; the ordered combination of basic scaling ratios allows the filter coefficients to be reused, reducing storage requirements; the method of corresponding to the current filter coefficient pixel by pixel ensures that the pixel value of the output image is generated at one time, avoiding intermediate interpolation steps and saving computing resources; the overall process is indexed by the scaling relationship, and the calculation and rendering are completed simultaneously, shortening the processing link and improving real-time performance.

[0006] Optionally, the transformation from the input image to the output image includes at least one of the following: Transform row by row data to Rows of data; Transform column data to Columns of data; among which, and , and All are positive integers, and and , and They are not equal.

[0007] In the above implementation process, the scaling of rows and columns is handled independently, so that the horizontal and vertical sampling rate adjustments do not affect each other, adapting to input and output scenarios with different aspect ratios; Transform to and Transform to The separate mapping allows row and column buffers to be organized in a one-dimensional manner, reducing the hardware line buffer capacity; the unequal integer ratios can directly correspond to the filter order, avoiding floating-point operations and simplifying circuit implementation; at the same time, it supports row-only transformation, column-only transformation, or row-column combination transformation, providing a unified data path for subsequent splicing, trimming, and rotation, reducing redundant data handling.

[0008] Optionally, determining the scaling mode and scaling combination scheme based on the ratio of the number of rows and / or columns of the target output image to the input image includes: obtaining the scaling combination scheme by decomposing the target transformation ratio into an ordered combination of basic scaling ratios; wherein the combination of the basic scaling ratios only involves simultaneous scaling up or simultaneous scaling down; the basic scaling ratios include at least one of the following: a first type of basic ratio: Row data transformation Row data, or Row data transformation Row data; Second type of basic ratio: will Row data transformation Row data, or Row data transformation Row data; Third type of basic ratio: will Transform column data into Column data, Transform column data into List data; Fourth category of basic proportions: Transform column data into Column data, Transform column data into Column data; among which, , , , All are positive integers. and All less than and And they are not equal. and All less than and And they are not equal.

[0009] In the above implementation process, by decomposing any target transformation ratio into an ordered combination of basic scaling ratios that only simultaneously increase or decrease, unidirectional progressive chains are formed in both row and column directions, avoiding backtracking or recalculation caused by mixed scaling; each type of basic ratio only involves the number of adjacent pixels. The filter window is fixed at the minimum neighborhood, the coefficient table size is compressed to its limit, and the storage only needs to retain the corresponding values. The weights are sufficient; , , , The values ​​are all smaller than the original and target sizes and are all different from each other, ensuring that the filters corresponding to different base ratios do not overlap and that there are zero address conflicts when looking up the table in hardware; the overall ratio decomposition and combination steps are completely integerized, eliminating floating-point units and reducing power consumption, while supporting independent splitting of rows and columns, which facilitates parallel execution of the pipeline.

[0010] Optionally, the output image is obtained by row transformation of the input image, and the ratio combination scheme is determined by... The first type of basic ratio and In the case of an ordered composition of the second type of basic proportions: then and satisfy:

[0011]

[0012] exist In this case, both the first type of base scale and the second type of base scale are magnification scales: the first type of base scale adopts the method of... Row data transformation Row data, the second type of basic ratio adopts the following Row data transformation Row data;

[0013] exist In this case, both the first type of base ratio and the second type of base ratio are reduction ratios: the first type of base ratio adopts a reduction ratio. Row data transformation Row data, the second type of basic ratio adopts the following Row data transformation Row data;

[0014] in, The number of identical processing units into which a row transformation is decomposed.

[0015] In the above implementation process, utilizing , and The integer relationship decomposes any row ratio into two types of ±1 basic ratio repetitions, making the scaling up or scaling down operations uniformly progressive in the same direction, avoiding the directional judgment logic caused by mixing different methods; the formula is given directly. , The corresponding number of times can be generated by looking up the value of the value in the table and the counter, eliminating the need for real-time division and saving computing units. The total transformation is decomposed into multiple identical sub-cycles, with each sub-cycle processing only one sub-cycle. or The line buffer depth is fixed, and the hardware reuses the same memory block periodically, reducing the area; during scaling, only the data is accumulated. When shrinking, only cumulative reduction is performed. Address decoding is simplified, and power consumption decreases linearly with the number of operations.

[0016] Optionally, the output image is obtained by transforming the input image columns, and the ratio combination scheme is determined by... The third type of basic ratio and In the case of the ordered composition of the fourth type of basic proportions: then and satisfy:

[0017]

[0018] exist In this case, both the third type of base ratio and the fourth type of base ratio are scaling ratios: the third type of base ratio adopts a scaling ratio of... Transform column data into The data, the fourth type of basic ratio will Transform column data into Column data;

[0019] exist In this case, both the third type of base ratio and the fourth type of base ratio are reduction ratios: the third type of base ratio adopts a reduction ratio. Transform column data into The data, the fourth type of basic ratio, is adopted by... Transform column data into Column data;

[0020] in, The number of identical processing units is used to decompose the column transformation.

[0021] In the above implementation process, through , and The integer relationship decomposes any column ratio into two types of ±1 basic ratio repetitions, ensuring that horizontal scaling remains unidirectional and avoids control logic redundancy caused by direction switching; the formula is given directly. , The counter can complete the proportional allocation by taking the value, without the need for real-time multiplication or division, thus reducing the area of ​​the column processing unit. The total transform is divided into identical sub-cycles based on the number of identical processing units decomposed into, with each cycle processing only one sub-cycle. or The column cache depth is fixed, and the memory bank is reused periodically, reducing wiring complexity; during scaling, only accumulation is performed. When shrinking, only cumulative reduction is performed. The decoding path is unified, and the power consumption decreases linearly with the number of operations; the row and column formulas are symmetrical, and the hardware modules can be shared, improving resource utilization.

[0022] Optionally, in the ratio combination scheme by The first type of basic ratio or the third type of basic ratio, and The second type of basic ratio or the fourth type of basic ratio are arranged in an orderly manner, and In the case of: each of the second type of base ratio or the fourth type of base ratio is connected to as many of the first type of base ratio or the third type of base ratio as possible.

[0023] In the above implementation process, the smaller number of second or fourth type basic proportions are used as nodes, and the larger number of first or third type basic proportions are continuously attached before and after them, forming an arrangement of long chains and short nodes paired with long chains, so that adjacent two The interval between operations is maximized; the filtering window remains constant within the same chain, coefficient addresses are continuous, intermediate state saving is reduced, and the line buffer only needs to maintain the pixels corresponding to the current chain length, further compressing the capacity; long chains on both sides of a short node can share the same set of weight tables, the number of table lookups is diluted as the chain length increases, and the storage bandwidth requirement decreases; the control logic is simplified to a two-level state machine of chain counting and node switching, the timing path is shortened, and the clock frequency is increased; there are no jumps in chain operations, the pipeline cycle is uniform, the power consumption distribution is smooth, and the peak current is reduced.

[0024] Optionally, calculating the filter coefficient set according to the proportional combination scheme includes: determining the input parameters for the base scaling ratio. When the basic scaling ratio is the magnification ratio, that is, the output... The row / column refers to its first row. For each output position, calculate a pair of normalized filter coefficients. and ,in:

[0025] in, From 1 to The integer, the and satisfy .

[0026] In the above implementation process, the input length is directly used. Using integers as the denominator Generate a pair of complementary weights , This reduces the normalization operation to a single subtraction and addition, eliminating the need for a divider; the weight sum is always 1, so the hardware only needs to store one side of the coefficients, while the other side is obtained in real time through two's complement or subtraction, halving the capacity of the coefficient table; except for the endpoints, When the order increases sequentially, Linear decrease Linear incrementing, with a linear relationship between address and weight value, allows the use of a counter instead of a lookup table, further saving storage; the amplification ratio is fixed. Transform to The filtering window always covers two adjacent points, simplifying the wiring; the overall calculation only involves integer addition, subtraction and shifting, with the shortest operation path and power consumption increasing linearly with the output pixels.

[0027] Optionally, calculating the filter coefficient set according to the proportional combination scheme further includes: determining the input parameters for the basic scaling ratio. When the base scaling ratio is the scaling ratio, that is, the output... The row / column refers to its first row. For each output position, calculate a pair of normalized filter coefficients. and ,in:

[0028] in, From 1 to The integer, the and satisfy .

[0029] In the above implementation process, the input length is... Using integers as the denominator Generate complementary weights and This reduces the normalization operation to a single subtraction and addition, eliminating the need for a subtractor; the weight sum is always 1, requiring only the storage of one side's coefficients in the hardware, with the other side obtained in real-time through subtraction, halving the coefficient table capacity; except for the endpoints, When the order increases sequentially, Linear decrease Linearly increasing, with a linear relationship between address and weight value, a counter can be used instead of table lookup, further saving storage; the reduction ratio is fixed. Transform to The filtering window always covers two adjacent points, simplifying the wiring; the overall calculation only involves integer addition, subtraction and shifting, with the shortest operation path and power consumption decreasing linearly with the output pixels.

[0030] Optionally, calculating the filter coefficient set according to the proportional combination scheme further includes: for the output image located at the first... line, number Column pixels: Get the first column's pixels The row-normalized filter coefficient pair corresponding to the row ( , ), and the column normalized filter coefficients corresponding to column C ( , );in, , The row normalized filter coefficients are multiplied by the row normalized filter coefficients to obtain two-dimensional filter weights; the two-dimensional filter weights are used to perform a weighted summation of the corresponding adjacent pixel values ​​in the input image to obtain the output pixel values.

[0031] Independent computation in the above implementation process , The two sets of one-dimensional weights are then multiplied to obtain two-dimensional weights, keeping the row and column coefficient tables separate. This reduces the storage requirement from two dimensions to two parts of one dimension, halving the capacity again. The dot product operation requires only four multiplications, and the two-dimensional convolution kernel is generated instantly, eliminating the need to pre-store the entire weight map. On-chip memory usage is decoupled from image size. The weighting of four adjacent points is completed in one operation, while the line buffer is maintained. With a fixed depth, the data path width is minimized; the characteristic of weight summation to 1 is preserved, and the output pixel value is directly normalized, eliminating the need for subsequent division; the row and column symmetric structure allows the same set of dot product units to be used for scaling up and down, improving hardware reuse rate.

[0032] Secondly, embodiments of this application provide an image processing apparatus, the apparatus comprising: a scheme determination module, a coefficient calculation module, and an integrated output module; the scheme determination module is used to determine a scaling mode and a ratio combination scheme based on the row ratio and / or column ratio of the target output image and the input image; wherein, the ratio combination scheme includes an ordered combination of at least one basic scaling ratio; the coefficient calculation module is used to calculate a filter coefficient set based on the ratio combination scheme; wherein, the filter coefficient set includes the current filter coefficient corresponding to each pixel of the input image converted to the output image; the integrated output module is used to calculate the output pixel value corresponding to each pixel of the output image based on the filter coefficient set, and output image data composed of the output pixel values.

[0033] In the above implementation process, the configuration information in the row and column directions flows unidirectionally between modules, reducing interface signals and wiring area; the scheme determination module outputs the mode and combination sequence by looking up a table in integer proportion, and the coefficient calculation module generates the corresponding weights according to the sequence beat with a fixed delay; the integration output module only receives the weights required for the current pixel, and the size of the multiply-accumulate array is consistent with the window size; there is no feedback path between modules, the timing convergence constraint is relaxed, and the overall frequency is improved; the overall architecture supports two data streams, row and column separation or row and column merging, reuses the same set of modules, resource consumption scales with the scene, and power consumption corresponds linearly to the number of pixels processed.

[0034] Thirdly, embodiments of this application also provide an electronic device, which includes a memory and a processor. The memory stores program instructions, and when the processor reads and runs the program instructions, it executes the steps in any of the above implementation methods.

[0035] Fourthly, embodiments of this application also provide a computer-readable storage medium storing computer program instructions, which, when read and executed by a processor, perform the steps in any of the above implementations. Attached Figure Description

[0036] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a schematic diagram of an image processing method provided in an embodiment of this application; Figure 2 This is a schematic diagram of an image processing apparatus provided in an embodiment of this application; Figure 3 This is a block diagram of an electronic device provided in an embodiment of this application.

[0038] Icons: 001 - Scheme Determination Module; 002 - Coefficient Calculation Module; 003 - Integrated Output Module; 100 - Electronic Equipment; 111 - Memory; 112 - Memory Controller; 113 - Processor; 114 - Peripheral Interface; 115 - Input / Output Unit; 116 - Display Unit. Detailed Implementation

[0039] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of the embodiments of this application.

[0040] With the rapid development of mobile display technology, resolution adaptation between input images and display screens has become a common technical challenge. Traditional adaptation solutions face a dilemma: on the one hand, while simple methods of directly adding or deleting rows are easy to implement, they produce obvious visual breaks in the image, especially creating unnatural transitions in smooth gradient areas and at detail edges, severely affecting display quality; on the other hand, while proportional scaling techniques based on complex filtering algorithms can generate smooth output effects, their multi-tap convolution operations and large row buffer requirements lead to high hardware costs, resulting in high computational complexity, high power consumption, and difficulty in meeting the practical requirements of resource-constrained scenarios such as mobile devices in terms of chip area and design complexity. This inherent contradiction between image quality and implementation cost in the existing technology system restricts the development of cost-effective adaptive display technology.

[0041] A method for scaling up or down data rows and columns by a certain ratio addresses the issue of mismatched row counts and resolutions in input image data for some devices. Directly adding or deleting rows can result in an uneven transition in the image data. To ensure a smooth transition between the added (or removed) data and surrounding data, data processing using certain rules is typically required. However, using proportional scaling up or down operations is complex and consumes significant power. For example, traditional image scaling (such as bilinear or bicubic interpolation) requires dynamically calculating interpolation coefficients for each output pixel when handling non-integer scales (especially irregular scales, such as scaling from 100 rows to 273 rows), resulting in a large computational load and requiring substantial row buffers and complex control logic in hardware implementation. Therefore, the purpose of this application is to provide an image processing method, apparatus, electronic device, and computer storage medium to address the aforementioned problems in the prior art.

[0042] Firstly, embodiments of this application provide an image processing method, please refer to [link / reference needed]. Figure 1 This is a schematic diagram of an image processing method provided in an embodiment of this application.

[0043] The image processing method includes: determining a scaling mode and a scaling combination scheme based on the ratio of the number of rows and / or the number of columns between the target output image and the input image; wherein the scaling combination scheme includes an ordered combination of at least one basic scaling ratio; calculating a set of filtering coefficients based on the scaling combination scheme; wherein the set of filtering coefficients includes the current filtering coefficient corresponding to each pixel of the input image converted to the output image; and calculating the output pixel value corresponding to each pixel of the output image based on the set of filtering coefficients, and outputting image data composed of the output pixel values.

[0044] In the above implementation process, based on the adjacent rows / columns of image data in the increased / decreased rows / columns, several rows of data in the corresponding region are processed to achieve a smooth transition. The scaling ratio to be processed is decomposed into combinations of different ratios by reducing the visual effect, simplifying calculations and reducing complexity and data cache size. The complex global scaling ratio is broken down into a series of ordered combinations of preset, simpler basic scaling ratios. Through ordered, staged processing, the number of image rows required for each processing step is reduced. A smooth transition is achieved through multi-level filtering, avoiding obvious flaws caused by simplified calculations.

[0045] In one optimized embodiment of this application, to further improve the visual quality of the scaled image, the system optionally introduces a gamma transformation preprocessing mechanism. Before filtering, the image data undergoes a gamma space transformation, making the filtering operation more consistent with human visual perception and effectively improving the visual transition effect during scaling. This is particularly effective in achieving a more natural visual effect in smooth gradient regions and areas rich in detail. After processing, an inverse gamma transformation is performed to restore the image to the standard color space, ensuring output compatibility. This optimization significantly improves the subjective visual quality of the scaled image without excessively increasing computational complexity, resulting in a more natural and smooth overall appearance. It is particularly suitable for applications with high image quality requirements.

[0046] In one embodiment of this application, the ratio of the number of rows and / or columns of the target output image to the number of input images can be determined based on hardware requirements or set values. During the determination process, only the row direction, only the column direction, or rows and columns can be processed separately or simultaneously, and it is necessary to determine whether to zoom in or out. The core lies in the ratio combination decomposition, pre-setting a basic set of scaling ratios, for example... These ratios are typically simple fractions, and their interpolation filters (e.g., the number of taps) are designed to be fixed and pre-optimized. The ratio of the number of rows and / or columns of the target output image to the input image is decomposed into an ordered sequence of these basic ratios. Each basic ratio has its own pre-designed filter kernel. For example, Amplification could be achieved using a 6-tap polyphase filter. The scaling might be achieved using another 4-tap polyphase filter. For the entire scaling scheme, it's necessary to calculate the equivalent filter coefficients for each output pixel from the original input image to the final output image. This process essentially combines the cascaded multi-stage filters into a single equivalent filter. By analyzing the data stream of the multi-stage scaling, we can deduce which pixels in the original input each final output pixel depends on, and the weighting coefficients. This yields a lookup table or generation rule for any position in the output image. Each pixel has its corresponding input pixel window (e.g., M pixels in each direction) and the weight coefficient of each pixel. For a fixed scaling ratio, this set of filter coefficients is global and calculated all at once. For each pixel in the output image, the corresponding current filter coefficient is retrieved from the set of filter coefficients based on its coordinates. Then, based on the input pixel position (neighborhood) specified by this set of coefficients, the corresponding pixel value is read from the input image, a weighted sum is performed, and the pixel value is output. The calculation result is used as the final value of the output pixel. After all output pixel values ​​have been calculated, the processed image data is formed and can be output to the display, storage, or the next processing module.

[0047] Optionally, the transformation from the input image to the output image includes at least one of the following: Transform row by row data to Rows of data; Transform column data to Columns of data; among which, and , and All are positive integers, and and , and They are not equal.

[0048] In the above implementation, transforming N1 rows of data to M1 rows of data defines the vertical transformation, which processes an entire row of pixels, treating all columns as a single data unit. This transforms the N1 row height of the input image to the M1 row height of the output image. Transforming N2 columns of data to M2 columns of data defines the horizontal transformation, which processes an entire column of pixels, treating all rows as a single data unit. This transforms the N2 column width of the input image to the M2 column width of the output image. The number of rows or columns is represented by two positive integers, N and M, and it is emphasized that they are not equal. , This excludes identity transformations such as non-scaling or copying, meaning that the transformation under discussion must necessarily cause a change in the size of the image in that dimension. If Or This refers to amplification or augmentation operations, which require interpolating new rows / columns from scratch. Or This is a shrinking or compressing operation, which requires merging and compressing existing rows / columns.

[0049] In one embodiment of this application, the image transformation may be a vertical transformation only, for example, changing only the image height (changing 1080 rows to 2160 rows) while keeping the width unchanged.

[0050] In another embodiment of this application, the image transformation may be a horizontal transformation only, for example, changing only the image width (changing 1920 columns to 1280 columns) while keeping the height unchanged.

[0051] Of course, in the embodiments of this application, the image transformation can also be a combination of vertical and horizontal transformations, for example, by... Image transformation The image. At this time, the vertical transformation from row 1080 to row 720 and the horizontal transformation from column 1920 to column 1280 will be performed sequentially or in parallel.

[0052] Optionally, based on the ratio of the number of rows and / or columns of the target output image to the input image, the scaling mode and scaling combination scheme are determined, including: obtaining the scaling combination scheme by decomposing the target transformation ratio into an ordered combination of basic scaling ratios; wherein, the combination of basic scaling ratios only involves simultaneous scaling up or simultaneous scaling down; the basic scaling ratios include at least one of the following: First type of basic ratio: Row data transformation Row data, or Row data transformation Row data; Second type of basic ratio: will Row data transformation Row data, or Row data transformation Row data; Third type of basic ratio: will Transform column data into Column data, Transform column data into List data; Fourth category of basic proportions: Transform column data into Column data, Transform column data into Column data; among which, , , , All are positive integers. and All less than and And they are not equal. and All less than and And they are not equal.

[0053] In the above implementation, the combination of basic scaling ratios can only involve simultaneous scaling up or scaling down, requiring all basic operations in the entire decomposition chain to be consistent in direction. If it's scaling up, then all the decomposed basic ratios must be scaling up operations, A transforms to A+1, B transforms to B+1. If it's scaling down, then all the decomposed basic ratios must be scaling down operations, A transforms to A-1, B transforms to B-1. This avoids repeated oscillations between scaling up and scaling down during the scaling process, which would lead to unnecessary computational redundancy and potential image quality loss. Furthermore, repeated resampling would increase noise and blur.

[0054] In one embodiment of this application, the basic ratios are divided into four categories, essentially describing the simplest increase and decrease operations in two dimensions (rows / columns). The first type of basic ratio transforms row A data into row A+1 data, which is the smallest granularity amplification in the row direction. For example, turning 3 rows into 4 rows, the ratio is 4:3≈1.333. Its reverse operation transforms row A into row A-1, which is the smallest granularity shrinking in the row direction. For example, turning 4 rows into 3 rows, the ratio is 3:4=0.75. The second type of basic ratio transforms row B data into row B+1 data. A and B are not equal, and both are less than the original and target number of rows (N1, M1). This means that the system has multiple different row transformation granularities to choose from. For example, A=2, 2 is transformed into 3 (ratio of 1.5), B=5, 5 is transformed into 6 (ratio of 1.2), providing different row length choices, making the decomposition more flexible and accurate.

[0055] Similarly, in another embodiment of this application, namely the third type of basic ratio and the fourth type of basic ratio, these are corresponding operations in the column direction. The principle is exactly the same as that in the row direction, only the operation dimensions are different. C and D are not equal, providing different granularity choices for the column direction.

[0056] In the above process, A and B being less than N1 and M1 is to ensure that the basic operations are performed within the local range of the original and target images, avoiding the use of the entire image size as the base unit. The fact that A and B are not equal, and C and D are not equal, forces the system to prepare at least two different granularities of base transformations, which is crucial for achieving accurate approximation. Of course, in practice, only one base transformation can be used, or it can be expanded further. However, using a single base ratio may reduce accuracy, while using multiple base ratios increases computational complexity and the amount of hardware required.

[0057] Optionally, the output image is obtained by row transformation of the input image, and the scaling scheme is determined by... The first category of basic proportions and In the case of an ordered composition of the second type of basic proportions: then and satisfy:

[0058]

[0059] exist In this case, both the first and second basic scales are enlarged scales: the first basic scale uses... Row data transformation Line data, the second type of basic ratio adopts the following Row data transformation Row data;

[0060] exist In this case, both the first and second base scales are reduction scales: the first base scale adopts a scale that is reduced from the original scale. Row data transformation Line data, the second type of basic ratio adopts the following Row data transformation Row data;

[0061] in, The number of identical processing units into which a row transformation is decomposed.

[0062] In the above implementation, P1 is a positive integer, meaning that the entire scaling process can be viewed as dividing the image into P1 independent processing units or data blocks in the row direction. Each processing unit independently applies a scaling chain consisting of K1 and L1 basic operations, and finally all P1 units work together to complete the transformation from N1 to M1.

[0063] To explain the above formula, firstly, the proportional relationship is constrained:

[0064]

[0065] Left side This represents a relative change, indicating the number of rows that need to be added or removed per unit of the original row count, when magnified. When shrinking . This represents the total number of basic operations per unit. The key point is... As a scaling factor, it links microscopic basic operations with macroscopic proportional changes.

[0066] The example describes how a 7-line input image is enlarged to a 10-line output image, i.e., the number of input lines... Output the number of rows The image is divided into P1 identical processing units, each of which is scaled using a series of basic operations that increment by one row. Each unit must have an integer number of rows of input. If the integer is an integer, then each unit must have an integer line output. It is an integer, therefore, Must be divisible by both and Since the factors of 7 are 1 and 7, and the factors of 10 are 1, 2, 5, and 10, their common factor (common divisor) is 1. It can only be equal to 1, meaning the entire image is treated as a single processing unit, without any finer-grained parallel processing.

[0067] Given that in each basic operation, A is transformed to Or B transform to Each row requires an additional row, increasing the number of rows from 7 to 10, for a total of 3 rows added. Furthermore, In other words:

[0068]

[0069] Three basic amplification operations need to be performed across the entire 7 rows of data. These three operations consist of two types: Sub-first type (parameter A) and Sub-second type (parameter B) and It is a non-negative integer, and .

[0070] Consider how to transform 7 lines of input data into 10 lines of input data using 3 basic operations. This leads to the first type of operation, where each operation processes... Line input, generate Line output, second type of operation, processed each time. Line input, generate For line output, the total number of input lines covered by all operations must equal the number of cell input lines, which is 7 lines in this example:

[0071] The total number of output lines generated by all operations must equal the number of output lines of the unit, which is 10 lines in this example:

[0072] and, ,So, and Possible combinations are: , , , .

[0073] exist In this case, Since A must be a positive integer, this combination is not feasible.

[0074] exist In the case of obtaining and If both solutions are identical, then the condition is satisfied. Furthermore, since A and B must be positive integers, and usually... Therefore, the possible solutions for A and B are: ; ; .for This means one magnification operation from 1 to 2 plus one magnification operation from 5 to 6; for This means two magnification operations from 2 to 3 plus one magnification operation from 3 to 4; for This means two amplification operations from 3 to 4 plus one amplification operation from 1 to 2.

[0075] exist In the case of obtaining and If both solutions are identical, then the condition is satisfied. Furthermore, since A and B must be positive integers, and usually... Therefore, the possible solutions for A and B are: , , .for This means one magnification operation of 5 to 6 plus one magnification operation of 1 to 2; for This means one magnification operation of 3 to 4 plus two magnification operations of 2 to 3; for This means one magnification operation from 1 to 2 plus two magnification operations from 3 to 4.

[0076] exist In this case, As above, this is not feasible for non-positive integers.

[0077] In another embodiment of this application, if N1 and M1 have more common divisors, P1 can be greater than 1, enabling parallel processing. For example, Transform to The common divisors between the two include 1, 2, 3, and 6. Therefore, the optional... Three units process in parallel, each unit is Line transformation to Okay, that is, each unit needs to be increased In other words:

[0078]

[0079] We get K1 + L1 = 2, so the three units can process simultaneously, increasing throughput.

[0080] However, theoretically, it could also be done in another way. In one embodiment of this application, the solution with the simplest hardware implementation or the best visual quality is selected from multiple feasible solutions, following the approach mentioned in the row transformation. Let's illustrate the arrangement in this embodiment with an example. It's clear that two basic amplifiers need to be implemented: the first type is an amplifier that transforms 2 rows to 3 rows, and the second type is an amplifier that transforms 3 rows to 4 rows. On 7 rows of input data, the first type of operation is applied twice and the second type of operation is applied once in a specific order. For example, if the initial 7 rows of data are... For the first operation, use the second type of amplifier to convert 3 rows to 4 rows, selecting 3 consecutive rows, such as... Enlarge to 4 lines The total number of rows is now The second operation uses the first type of amplifier to transform 2 rows into 3 rows, selecting two consecutive rows (either from the middle or elsewhere) to amplify into 3 rows, thus increasing the total number of rows. The third operation uses the first type of amplifier to transform 2 rows to 3 rows, selecting an appropriate 2 rows and amplifying them to 3 rows, resulting in a final number of rows. .

[0081] Optionally, the output image is obtained by column transformation of the input image, and the scaling scheme is determined by... The third category of basic proportions and In the case of an ordered composition of the fourth type of basic proportions: then and satisfy:

[0082]

[0083] exist In the case of both the third and fourth basic scales, the scale is an enlargement scale: the third basic scale adopts the method of... Transform column data into The data shows that the fourth basic proportion will be... Transform column data into Column data;

[0084] exist In the case of both the third and fourth basic scales, the scales are reduced: the third basic scale adopts a scale that is reduced from the previous scale. Transform column data into The data, the fourth type of basic ratio adopts the following approach: Transform column data into Column data;

[0085] in, The number of identical processing units is used to decompose the column transformation.

[0086] In the above implementation, P2 is a positive integer, meaning that the entire scaling process can be viewed as dividing the image into P2 independent processing units or data blocks in the row direction. Each processing unit independently applies a scaling chain consisting of K2 and L2 basic operations, and finally all P2 units jointly complete the transformation from N2 to M2. The mathematical model of the column transformation is completely symmetrical to the row transformation, dividing the image horizontally into P2 vertical strips, each strip being scaled independently in the column direction. By solving a system of integer equations, the type, number, and parameters of the basic operations within each strip are determined. This decomposition method transforms complex column transformations with arbitrary scales into a series of simple, parallelizable basic operations, significantly reducing the complexity of hardware implementation while ensuring transformation accuracy.

[0087] In the embodiments of this application, especially in actual chip design, the selection of P1 and P2 needs to strike a balance between parallelism, hardware resources, and power consumption. Smaller P1 and P2 save hardware but reduce throughput, while larger P1 and P2 improve performance but increase design complexity. Through intelligent scheduling algorithms and optimized filter design, this method can achieve efficient and high-quality image scaling in various display and image processing applications. However, in terms of hardware processing, row transformation requires caching multiple rows of data and filtering in the vertical direction; while column transformation usually does not require caching the entire column and can be processed in real time on the scan line. Its implementation may also require some pixel caching to achieve multi-tap filtering.

[0088] Based on this, in one embodiment of this application, P2 must be divisible by both N2 and M2. The larger P2 is, the higher the parallelism, but the smaller the width of each unit, the greater the bandwidth required for parallel processing of P2 units, and the boundary between units may require special processing to avoid seams.

[0089] In the dimension of column transformation, the input width Output width The greatest common divisor is 1, 2, 4, 5, 8, 10, 16, 20, 32, 40, 64, 80, 160, 320, 640, which can be selected. It is then divided into 64 units, each unit consisting of Column transformation to In fact, if we consider this flexibly, the 640 units could also be changed from 3 columns to 4 columns. However, in this case, the larger P2 is, the higher the parallelism, but the smaller the width of each unit. The parallel processing of P2 units requires a larger bandwidth. Under certain hardware conditions, the larger P2 is, the higher the throughput, but the latency may increase.

[0090] Optionally, in the proportional combination scheme by The first or third basic ratio, and The second or fourth basic proportions are arranged in an orderly manner, and In this case: connect as many first-class or third-class base ratios as possible to each second-class or fourth-class base ratio.

[0091] In the above implementation process, when the proportional combination scheme consists of K first / third type operations and L second / fourth type operations, and In this case, a grouped join strategy can be used. The K main operations are divided into L groups (or... The system uses a multi-level marketing (MLM) approach, where each main operation is followed by a secondary operation, resulting in a more uniform scaling distribution. Firstly, in terms of hardware, the system can predict in advance how many main operations will be processed before switching to a secondary operation, eliminating the need for real-time calculation of the next operation type for each pixel to form a stable operation rhythm, such as a "AAA-B-AAA-B-..." pattern. Because the secondary operations are evenly distributed across different areas of the image, local overprocessing is avoided, and each secondary operation acts as a transition bridge between preceding and following main operation groups for smooth processing. The regular operation distribution avoids moiré patterns or ringing effects caused by random arrangements. The main operation hardware can operate continuously for multiple cycles before switching, while the secondary operation hardware is activated at specific intervals. Reducing frequent switching and interlocking between the two hardware units also makes the entire workflow more efficient. This can be adjusted according to different... The ratio dynamically adjusts the group size, and the hardware registers can be configured with group parameters to adapt to different scaling ratios. Optimization provides a controllable setting method.

[0092] Furthermore, given that the number of primary operations K is greater than the number of secondary operations L, it is also necessary to explicitly define the number of data units contained in each basic operation. Again, taking row transformation as an example, the first type of basic operation (parameter A) transforms row A / column A into... Rows / columns, each operation processes A input data units; the second type of basic operation (parameter B) transforms B rows / columns into... Rows / columns, each operation processes B input data units.

[0093] For different A and B (which could be C and D in column transformations), in order to achieve an effective uniform distribution and meaningful grouping, it is generally necessary to ensure the following two cases: First, the number of main operation processing units is relatively small, that is, The main operation (Type A) processes a small number of data units each time, such as... Transforming 2 rows into 3 rows, a secondary operation (Type B) processes a larger number of data units each time, such as... The transformation from 4 rows to 5 rows allows type A operations to serve as the basic unit, providing fine-tuning, while type B operations serve as a supplement. While maintaining overall uniformity, type B operations are used to process larger blocks, making it easier to achieve uniform distribution, and type A operations in smaller units are easier to insert evenly.

[0094] Secondly, the number of main operation processing units is relatively large, that is... Its main operation involves processing large blocks of data, such as... Transforming 5 rows into 6 rows, secondary operations process small blocks of data, such as... Transform from 2 lines to 3 lines. When most scaling needs to be done quickly, fine-tuning can be done using fine-tuning operations. Under certain hardware architectures, large-scale operations may be more efficient.

[0095] In general, A and B cannot be too large, otherwise the change in a single operation will be too drastic; nor can they be too small, otherwise too many operations will be required, increasing complexity. Furthermore, each basic operation corresponds to a filter, and A and B determine the size and complexity of the filter; in column transformations, these are C and D, which will not be elaborated upon here.

[0096] In the actual application of this application, the specific values ​​of A and B mainly depend on the required scaling ratio, and there is no absolute size requirement for them. For example, if the goal is to shrink 2000 rows to 1980 rows, the most direct configuration is to use the operation of transforming 100 rows to 99 rows. The key is that by combining different scaling numbers (such as transforming A to A+1, or B to B+1) and their corresponding number of operations, it is possible to flexibly and accurately approximate various complex scaling ratios (such as non-integer ratios like reducing 2000 rows / columns to 37 rows / columns). In actual configuration, it is preferable to make the values ​​of A and B as close as possible, which helps maintain the uniformity and consistency of processing during scaling and reduces visual defects. When they cannot be completely equal, the system will default to configuring the row values ​​that are used more frequently first and those used less frequently later to optimize the processing flow and resource allocation. Therefore, the setting of A and B is ratio-driven, pursues uniformity, and follows the principle of optimal efficiency in the operation order.

[0097] In one embodiment of this application, a large number of primary operations (K operations) are divided into several groups, and a secondary operation (L operations in total) is connected after each group to form a regular operation sequence. Taking 7 type A operations and 2 type B operations as an example, the most direct grouping method is to divide the 7 into two groups, each with approximately 3 to 4 type A operations, and then connect each group with a type B operation. The same applies to the third type of basic ratio (type C operation) and the fourth type of basic ratio (type D operation) in column transformations, and will not be elaborated here.

[0098] In this situation, it's observed that dividing K by L often results in non-divisible division, leaving remainders. Simple grouping methods may cause these remainders to concentrate in a certain range, leading to uneven processing. Therefore, given K primary operations and L secondary operations, where... And K divided by L has a remainder (i.e., Our goal is to distribute these primary operations as evenly as possible among the secondary operations, ensuring balanced processing across the entire image region. In other words, the remainders are no longer simply piled up at the beginning or end, but rather a smart, uniform distribution strategy is employed.

[0099] Strategy 1 uses a fixed-interval insertion method, controlling the insertion position of the remainder by setting an interval parameter X. For example, setting... This indicates that a group with remainders is inserted after every two standard groups. Specifically, first, standard operation groups are formed, each containing a base number of primary operations and one secondary operation. At set intervals, the standard groups are upgraded to groups with remainders (adding a primary operation) at specific positions, and the remainder counter is updated until all remainders are allocated.

[0100] Strategy two is based on the perfect uniform distribution of the algorithm. For more demanding applications, a similar idea to the Bressenham line algorithm can be used to achieve a perfect uniform distribution, using error accumulation to determine when to insert the remainder. Calculate the target operands (floating-point numbers, e.g., ...) for each group. If the initial error is set to 0, then for each group, the basic operands take the integer part of the target value, and the cumulative error is increased by the decimal part of the target value. If the cumulative error exceeds 1, then an operation is added to the current group, and the error is reduced by 1.

[0101] First, calculate several basic parameters. The number of operands in each set is equal to the integer part of K divided by L, and the number of remainders is equal to the remainders of K divided by L. Take 7 A operations and 2 B operations as an example: The number of baseline operations per group is (Take the integer part), the number of remainders is .

[0102] In this case, for strategy one, with For example, set (After inserting remainders after each group), the standard group is AAA-B, the group with remainders is AAAA-B (adding one A), and the final sequence is AAAA-B-AAA-B. Thus, the standard group has 3 A operations, and the remainder group has 4 A operations, with a relatively even distribution.

[0103] For strategy two, still use For example, the target value for each group is The first group has 3 base numbers with an error of 0.5; the second group has 3 base numbers, but the accumulated error reaches 1.0, so 1 more is added, resulting in 4 actual numbers, and the error returns to zero. The final distribution shows 3 in the first group and 4 in the second group.

[0104] In image scaling, the arrangement of the operation sequence directly affects the final processing result. We choose 8 basic operations of type I (Type A) and 3 basic operations of type II (Type B) as an analysis example. This combination better demonstrates the diversity and complexity of the uniform distribution strategy than the common 7A2B combination. The remainder of 8 divided by 3 is 2, meaning there are two additional Type A operations that need to be assigned to the three groups, resulting in more diverse allocation schemes. Furthermore, the three groups correspond precisely to the beginning, middle, and end segments of the image, facilitating the analysis of regional differences. The same applies to the basic operations of type III (Type C) and type IV (Type D) in column transformations, which will not be elaborated upon here.

[0105] In this embodiment, taking row operations as an example, the total number of A operands, K, is 8, the total number of B operands, L, is 3, and the number of base A operands for each group is... The remainder A operand is Therefore, the following strategies are provided: Strategy 1 involves simply appending remainder groups, resulting in remainders concentrated at the end. Both remainders A are added to the last group, forming a 2-2-4 distribution (AA-B-AA-B-AAAA-B). This distribution causes significant compression at the end of the image. Because the final region undergoes four consecutive large magnifications, it can lead to over-stretching and blurring of details, jagged edges, ringing effects, and inconsistencies with the processing style of the preceding regions.

[0106] Therefore, consider strategy two: inserting the remainders at uniform intervals. This involves evenly inserting the two remainder A operations into the three groups using an interval insertion algorithm. The possible distributions are: 3-3-2 distribution (AAA-B-AAA-B-AA-B), 3-2-3 distribution (AAA-B-AA-B-AAA-B), and 2-3-3 distribution (AA-B-AAA-B-AAA-B). Let's analyze the 3-3-2 distribution (AAA-B-AAA-B-AA-B) in detail. Its operation sequence structure is: Group 1, AAA-B; Group 2, AAA-B; Group 3, AA-B. In the initial region, three A operations quickly reach near the target size, suitable for areas insensitive to distortion or requiring emphasis. In the middle region, three A operations maintain the continuity of the processing rhythm. In the final region, two A operations are used. The human eye is less sensitive to distortion in image edge areas, but sharpness needs to be maintained. This distribution creates a strong-strong-weak processing rhythm, similar to rhythmic changes in music. For most natural images, this rhythm is appropriate, with the central region of the image usually being more important and fully processed, while the edge regions are appropriately simplified to maintain sharpness.

[0107] This embodiment also provides Strategy Three, which considers intelligent optimization distribution. Based on the dynamic distribution of image content, more advanced systems can dynamically adjust the distribution according to the characteristics of image content. For example, portrait photos can use a 2-3-3 distribution, AA-B-AAA-B-AAA-B. For portrait photos, the face is usually located slightly above the center of the image, so the processing of the middle area is strengthened, while the processing of the starting area (which may be the background or body) can be weaker. Landscape photos can use a 3-2-3 distribution, AAA-B-AA-B-AAA-B. The sky (upper part) and foreground (lower part) need to be processed well, while the horizon area in the middle can be processed gradually. For text documents, a near-uniform processing method can be used, such as AA-B-AA-B-AA-B-AA, to ensure the consistency of text by processing the entire image uniformly.

[0108] Optionally, based on the proportional combination scheme, a set of filter coefficients is calculated, including: input parameters for determining the base scaling ratio. When the base scaling ratio is the same as the magnification ratio, that is, the output... The row / column refers to its first row. For each output position, calculate a pair of normalized filter coefficients. and ,in:

[0109] in, From 1 to integers, and satisfy .

[0110] In the above implementation process, by observing the formula, we can find a clear pattern in the change of the coefficients with k1. When k1 takes the minimum value of 1, And there are This indicates that the first output position is entirely determined by the first input data, with no contribution from subsequent input data.

[0111] When k1 reaches its maximum value hour, And there are This indicates that the last output position is entirely determined by the last input data.

[0112] When k1 takes the middle value, that is, Both W1 and W2 are between 0 and 1, and W2 decreases linearly as k1 increases, while W1 increases linearly as k1 increases. The two change at the same rate but in opposite directions.

[0113] It is understandable that when increasing from N rows or columns to... When dealing with rows or columns, the extra row or column is actually distributed across the original N rows or columns. For N transformed to... The magnification is such that each output row or column is a linear combination of two adjacent input rows or columns; more precisely, the output positions are evenly distributed between 1 and 2. Within the range, each output position corresponds to an interpolation point between input rows or columns. k1 is not a simple index, but rather the relative position of the k1th output position in the input coordinate system. The denominator N ensures the rationality of weight allocation; the larger N is, the higher the dilution with each additional row; the smaller N is, the more drastic the weight changes. Based on the concept of the smallest unit mentioned earlier, this can be understood as performing transformations on the rows and columns of the smallest unit.

[0114] In one embodiment of this application, the number of rows or columns is changed from 4 to 5, with the extra row being evenly inserted into the original row or column intervals. Specifically, the first row or column of the output corresponds exactly to the first row or column of the input. The second row or column of the output is formed by combining 1 / 4 of the first row or column of the input and 3 / 4 of the second row or column of the input. The third row or column of the output is formed by combining 1 / 2 of the second row or column of the input and 1 / 2 of the third row or column of the input. The fourth row or column of the output is formed by combining 3 / 4 of the third row or column of the input and 1 / 4 of the fourth row or column of the input. The fifth row or column of the output corresponds exactly to the fourth row or column of the input. This set of coefficients visually demonstrates how each newly added output position is generated by linear interpolation of its two adjacent input positions relative to the input grid during the amplification process, ensuring the smoothness and continuity of the transformation. The specific values ​​in the example represent how much weight each input position contributes to the output position to its right. In practice, as long as each output position is ensured to be mixed from the two nearest input data on its two sides inverse proportion to the distance, and the total contribution of each input data conforms to the scaling ratio, the result will be unique and correct.

[0115] Optionally, calculating the filter coefficient set according to the proportional combination scheme also includes: input parameters for determining the base scaling ratio. When the base scaling ratio is the same as the scaling ratio, that is, the output... The row / column refers to its first row. For each output position, calculate a pair of normalized filter coefficients. and ,in:

[0116] in, From 1 to integers, and satisfy .

[0117] In the above implementation process, similar to the zoom-in operation, when it is necessary to shrink the image from N rows or columns to... When dealing with rows or columns, we are dealing with a basic scaling-down operation that reduces a data unit. This operation is symmetrical to the scaling-up operation but in the opposite direction, and it forms the basic unit for constructing scaled-down units of any scale. Similar to the scaling-up operation, the coefficients W3 and W4 also satisfy the normalization condition, ensuring the energy conservation of the output pixel values ​​and avoiding unnatural changes in brightness or color caused by the scaling-down operation.

[0118] In one embodiment of this application, suppose we want to reduce 5 lines of image data to 4 lines. , The first row or column of the output is taken from... Enter the first row or column plus Input the second line, and output the second line or column from... Enter the second row or column plus Input the 3rd row or column, and output the 3rd row or column or column (whichever is taken from the previous row). Enter the 3rd row or column plus Input the 4th row or column, output the 4th row or column or column taken from... Enter the 4th row or column plus Enter the 5th row or column.

[0119] However, in image downsampling, it's crucial to understand that it's not simply the reverse of image upsampling; these two operations differ fundamentally at the signal processing level. Upsampling (incrementing) involves increasing information density from sparse to dense, primarily focusing on smoothly filling gaps without introducing new high-frequency components. Downsampling (reducing) involves decreasing information density from dense to sparse, primarily addressing information loss and aliasing, potentially creating new pseudo-high frequencies (aliasing). When an image is downsized, if original high-frequency information (such as fine textures and sharp edges) is not properly processed, it can fold back into low-frequency regions, producing false, non-existent patterns—this is the aliasing effect.

[0120] In digital signal processing, the standard procedure for downsampling (reduction) is to resample (decimate) the original signal after low-pass filtering (anti-aliasing). Low-pass filtering is a key step that requires looking at multiple adjacent samples (input rows) to determine each output value in order to smooth out high-frequency components that may cause aliasing.

[0121] In this case, based on the standard position mapping of sampling theory, the normalized position of the input position in the output coordinate system is:

[0122] Based on precise location mapping, the optimized filter coefficient calculation formula is as follows: set up x is the largest integer less than or equal to x. If the fractional part is the decimal part, the filter coefficients are calculated as follows:

[0123] Its satisfaction .

[0124] In this embodiment, the number of rows is reduced from 5 to 4 ( For example, the scaling factor .

[0125] Output bit 1, that is In this case, position parameters, Integer part The decimal part Filter coefficients as well as In other words, output line 1 is... Input line 1 and The input consists of line 2. The output position is 2, i.e., when k2=2. , , , as well as In other words, output line 2 consists of 60% of input line 2 and 40% of input line 3. Similarly, output line 3 consists of 40% of input line 3 and 60% of input line 4, and output line 4 consists of 20% of input line 4 and 80% of input line 5. This approach results in a smoother transition, weight distribution that matches the actual geometric position, more natural boundaries, more reasonable handling of the first and last output lines, and linear interpolation providing basic anti-aliasing capabilities.

[0126] Optionally, calculating the filter coefficient set according to the proportional combination scheme further includes: for the output image located at the th line, number Column pixels: Get the first column's pixels The row-normalized filter coefficient pair corresponding to the row ( , ), and the column normalized filter coefficients corresponding to column C ( , );in, , The row-normalized filter coefficients are multiplied by each other to obtain the two-dimensional filter weights. The two-dimensional filter weights are then used to perform a weighted summation of the corresponding adjacent pixel values ​​in the input image to obtain the output pixel values.

[0127] The above implementation describes the calculation and application method of filter coefficients in two-dimensional image scaling, applicable to scenarios where scaling is performed simultaneously in both row and column directions. This method calculates normalized filter coefficients in the row and column directions separately, then combines them to form two-dimensional filter weights, and finally obtains the output pixel value through weighted summation. This method of separating row and column processing significantly reduces computational complexity while maintaining image quality.

[0128] In one embodiment of this application, for any pixel located in row R and column C of the output image, it is necessary to obtain its filtering coefficients in the row and column directions, respectively. For the row direction coefficients, the corresponding row normalized filtering coefficient pair is queried or calculated based on the output row number R. , These two coefficients satisfy the normalization condition. The coefficients are obtained based on the aforementioned row scaling ratio combination scheme. For column direction coefficients, the corresponding column normalization filter coefficient pair is queried or calculated based on the output column number C. , (This also satisfies the normalization condition) The coefficients are obtained based on the aforementioned column scaling scheme. After obtaining the row and column coefficients, the final two-dimensional filter weight matrix is ​​generated through a dot product operation. Considering the independence of the row and column directions, the two-dimensional filter weights can be obtained through the tensor product of the row and column coefficients. For the bilinear interpolation case, this produces a... Weight matrix:

[0129] These four weights satisfy the normalization condition:

[0130] Assuming the result is obtained through row scaling, the row coefficient corresponding to the Rth row is: , And through column scaling calculation, the column coefficient corresponding to column C is output as follows: , Based on output position The row and column coefficients determine the four input pixel units involved in the calculation. The final weight of the four input pixels is determined by the combination of the row and column coefficients: the weight of the top-left pixel block is equal to the product of the left weight and the top weight; the weight of the top-right pixel block is equal to the product of the right weight and the top weight; the weight of the bottom-left pixel block is equal to the product of the left weight and the bottom weight; and the weight of the bottom-right pixel block is equal to the product of the right weight and the bottom weight. Based on the above weight calculation method, the final output pixel value is obtained by weighted summation of the four input pixel units according to their corresponding weights.

[0131] Specifically, output the pixel value in row R and column C of the image. The calculation is as follows:

[0132] in to These represent the pixel values ​​of the four input pixel units. This calculation process involves independent interpolation in the row and column directions followed by combination, which is mathematically equivalent to standard two-dimensional bilinear interpolation, but significantly reduces computational complexity by separating row and column processing.

[0133] In this implementation, the aforementioned calculation process can be efficiently executed through a hardware pipeline. The system first determines the corresponding row and column filtering coefficients based on the output position, then calculates the two-dimensional weight matrix while simultaneously reading the corresponding input pixel data from memory. Finally, a weighted summation is performed using a multiply-accumulate unit. This structured processing method not only ensures the quality of image scaling but also achieves an optimal balance between computational efficiency and hardware resource utilization. By decomposing arbitrarily complex scaling ratios into a series of combinations of basic ratios and pre-calculating or generating filtering coefficients in real-time for each basic ratio, this application provides a highly flexible and efficient image scaling solution suitable for various display resolutions and image processing needs.

[0134] Secondly, embodiments of this application provide an image processing apparatus; please refer to [link to relevant documentation]. Figure 2 , Figure 2 This is a schematic diagram of an image processing apparatus provided in an embodiment of this application.

[0135] The image processing device includes: a scheme determination module 001, a coefficient calculation module 002, and an integrated output module 003; the scheme determination module 001 is used to determine the scaling mode and ratio combination scheme according to the ratio of the number of rows and / or the ratio of the number of columns of the target output image to the input image; wherein, the ratio combination scheme includes an ordered combination of at least one basic scaling ratio; the coefficient calculation module 002 is used to calculate a set of filtering coefficients according to the ratio combination scheme; wherein, the set of filtering coefficients includes the current filtering coefficient corresponding to each pixel of the input image converted to the output image; the integrated output module 003 is used to calculate the output pixel value corresponding to each pixel of the output image based on the set of filtering coefficients, and output image data composed of the output pixel values.

[0136] In the above implementation process, the scheme determination module 001 serves as the starting point of the processing flow. It receives the resolution information of the input image and the target output resolution requirements, and determines the optimal scaling strategy through algorithm analysis. This module not only determines the zoom-in or zoom-out mode but also decomposes the complex scaling ratio into an ordered combination of a series of basic ratios, laying the foundation for subsequent processing. The coefficient calculation module 002 calculates the corresponding filter coefficients for each basic scaling ratio based on the ratio combination scheme provided by the scheme determination module 001. This module generates a complete set of filter coefficients, covering the filter coefficients required for each pixel during the conversion from the input image to the output image. These coefficients are normalized to ensure energy conservation during the scaling process. The integrated output module 003 serves as the endpoint of the processing flow. Using the set of filter coefficients provided by the coefficient calculation module 002, it performs weighted calculations on the input image pixels to generate the final value for each output pixel. This module integrates all processing results, outputting complete scaled image data while ensuring that the output quality meets the expected requirements.

[0137] Since the principle of the device in this embodiment for solving the problem is similar to that of the aforementioned image processing method, the implementation of the device in this embodiment can refer to the description in the embodiments of the above method, and the repeated parts will not be described again.

[0138] Thirdly, embodiments of this application also provide an electronic device, which includes a memory and a processor. The memory stores program instructions, and when the processor reads and runs the program instructions, it executes the steps in any of the above implementation methods.

[0139] Optionally, please refer to Figure 3 , Figure 3 This is a block diagram illustrating an electronic device according to an embodiment of this application. The electronic device 100 may include a memory 111, a memory controller 112, a processor 113, a peripheral interface 114, an input / output unit 115, and a display unit 116. Those skilled in the art will understand that... Figure 3 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device 100. For example, the electronic device 100 may also include components that are more... Figure 3 The more or fewer components shown, or having the same Figure 3 The different configurations shown.

[0140] The aforementioned memory 111, memory controller 112, processor 113, peripheral interface 114, input / output unit 115, and display unit 116 are electrically connected directly or indirectly to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The aforementioned processor 113 is used to execute executable modules stored in the memory.

[0141] The memory 111 can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory 111 stores programs, and the processor 113 executes these programs upon receiving execution instructions. The methods executed by the electronic device 100 as defined in any embodiment of this application can be applied to the processor 113, or implemented by the processor 113.

[0142] The aforementioned processor 113 may be an integrated circuit chip with signal processing capabilities. The processor 113 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a digital signal processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor.

[0143] The peripheral interface 114 described above couples various input / output devices to the processor 113 and the memory 111. In some embodiments, the peripheral interface 114, the processor 113, and the memory controller 112 can be implemented on a single chip. In other instances, they can be implemented on separate chips.

[0144] The input / output unit 115 described above is used to provide user input data. The input / output unit 115 may be, but is not limited to, a mouse and a keyboard.

[0145] The aforementioned display unit 116 provides an interactive interface (e.g., a user interface) between the electronic device 100 and the user, or displays image data for the user's reference. In this embodiment, the display unit can be a liquid crystal display (LCD) or a touch display. If it is a touch display, it can be a capacitive touchscreen or a resistive touchscreen that supports single-point and multi-point touch operations. Supporting single-point and multi-point touch operations means that the touch display can sense touch operations generated simultaneously from one or more locations on the touch display and pass the sensed touch operations to the processor for calculation and processing.

[0146] Fourthly, embodiments of this application also provide a computer-readable storage medium storing computer program instructions, which are read and executed by a processor to perform the steps in any of the above implementations.

[0147] In summary, this application provides an image processing method, apparatus, electronic device, and computer-readable storage medium, relating to the field of image processing technology. The method includes: determining a scaling mode and a scaling ratio combination scheme based on the row and / or column ratio of the target output image and the input image; wherein the scaling ratio combination scheme includes an ordered combination of at least one basic scaling ratio; calculating a set of filtering coefficients based on the scaling ratio combination scheme; wherein the set of filtering coefficients includes the current filtering coefficient corresponding to each pixel of the input image converted to the output image; and calculating the output pixel value corresponding to each pixel of the output image based on the set of filtering coefficients, and outputting image data composed of the output pixel values. Determining the scaling mode and scaling ratio combination scheme by using the row and / or column ratios allows the set of filtering coefficients to cover all pixels from input to output at once, reducing redundant calculations; the ordered combination of basic scaling ratios allows the filtering coefficients to be reused, reducing storage requirements.

[0148] In the several embodiments provided in this application, it should be understood that the disclosed device can also be implemented in other ways. The device embodiments described above are merely illustrative; for example, the block diagrams in the accompanying drawings illustrate the possible architecture, functions, and operations of the device according to various embodiments of this application. In this regard, each block in the block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram, and combinations of block diagrams, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0149] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0150] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0151] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0152] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

[0153] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. An image processing method, characterized in that, The method includes: Based on the ratio of the number of rows and / or columns of the target output image to the input image, the scaling mode and the ratio combination scheme are determined; wherein, the ratio combination scheme includes an ordered combination of at least one basic scaling ratio; According to the ratio combination scheme, a set of filtering coefficients is calculated; wherein, the set of filtering coefficients includes the current filtering coefficient corresponding to each pixel of the input image converted to the output image; Based on the filter coefficient set, the output pixel value corresponding to each pixel of the output image is calculated, and the image data composed of the output pixel values ​​is output.

2. The method according to claim 1, characterized in that, in, The transformation from the input image to the output image includes at least one of the following: Will Transform row by row data to Rows of data; Transform column data to Columns of data; in, and , and All are positive integers, and and , and They are not equal.

3. The method according to claim 2, characterized in that, The step of determining the scaling mode and scaling combination scheme based on the ratio of the number of rows and / or columns of the target output image to the input image includes: The ratio combination scheme is obtained by breaking down the target transformation ratio into an ordered combination of the basic scaling ratios; Among them, the combination of the basic scaling ratio can only involve simultaneous scaling up or simultaneous scaling down; The base scaling ratio includes at least one of the following: First type of basic proportion: [The following is a partial translation and can be left as is] Row data transformation Row data, or Row data transformation Row data; Second type of basic ratio: [The following is a partial translation and doesn't need a direct translation] Row data transformation Row data, or Row data transformation Row data; Third type of basic ratio: [The following is a partial translation and doesn't need a direct translation] Transform column data into Column data, Transform column data into Column data; Fourth category of basic proportions: Transform column data into Column data, Transform column data into Column data; in, , , , All are positive integers. and All less than and And they are not equal. and All less than and And they are not equal.

4. The method according to claim 3, characterized in that, in, The output image is obtained by transforming the input image, and the ratio combination scheme is determined by... The first type of basic ratio and In the case of an ordered composition of the second type of basic proportions: but and satisfy: exist In this case, both the first type of base scale and the second type of base scale are magnification scales: the first type of base scale adopts the method of... Row data transformation Row data, the second type of basic ratio adopts the following Row data transformation Row data; exist In this case, both the first type of base ratio and the second type of base ratio are reduction ratios: the first type of base ratio adopts a reduction ratio. Row data transformation Row data, the second type of basic ratio adopts the following Row data transformation Row data; in, The number of identical processing units into which a row transformation is decomposed.

5. The method according to claim 3, characterized in that, in, The output image is obtained by transforming the input image column, and the ratio combination scheme is determined by... The third type of basic ratio and In the case of the ordered composition of the fourth type of basic proportions: but and satisfy: exist In this case, both the third type of base ratio and the fourth type of base ratio are scaling ratios: the third type of base ratio adopts a scaling ratio of... Transform column data into The data, the fourth type of basic ratio will Transform column data into Column data; exist In this case, both the third type of base ratio and the fourth type of base ratio are reduction ratios: the third type of base ratio adopts a reduction ratio. Transform column data into The data, the fourth type of basic ratio, is adopted by... Transform column data into Column data; in, The number of identical processing units is used to decompose the column transformation.

6. The method according to claim 1, characterized in that, The step of calculating the filter coefficient set according to the ratio combination scheme includes: Input parameters for determining the base scaling ratio ; When the base scaling ratio is the magnification ratio, that is, the output... The row / column refers to its first row. For each output position, calculate a pair of normalized filter coefficients. and ,in: in, From 1 to The integer, the and satisfy .

7. The method according to claim 1, characterized in that, The step of calculating the filter coefficient set according to the ratio combination scheme further includes: Input parameters for determining the base scaling ratio ; When the base scaling ratio is the scaling ratio, that is, the output... The row / column refers to its first row. For each output position, calculate a pair of normalized filter coefficients. and ,in: in, From 1 to The integer, the and satisfy .

8. The method according to claim 1, characterized in that, The step of calculating the filter coefficient set according to the ratio combination scheme further includes: For the output image located at the th line, number Column pixels: Get the The row-normalized filter coefficient pair corresponding to the row ( , ), and the column normalized filter coefficients corresponding to column C ( , ); in, , ; The two-dimensional filter weights are obtained by multiplying the row-normalized filter coefficient pairs by each other. The two-dimensional filter weights are used to perform a weighted summation of the corresponding adjacent pixel values ​​in the input image to obtain the output pixel value.

9. An image processing apparatus, characterized in that, The device includes: a scheme determination module, a coefficient calculation module, and an integrated output module; The scheme determination module is used to determine the scaling mode and ratio combination scheme based on the ratio of the number of rows and / or the ratio of the number of columns of the target output image to the input image; wherein, the ratio combination scheme includes an ordered combination of at least one basic scaling ratio; The coefficient calculation module is used to calculate a set of filtering coefficients according to the ratio combination scheme; wherein, the set of filtering coefficients includes the current filtering coefficient corresponding to each pixel of the input image converted to the output image; The integrated output module is used to calculate the output pixel value corresponding to each pixel of the output image based on the filter coefficient group, and output image data composed of the output pixel values.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a processor, perform the steps of the method according to any one of claims 1 to 8.

Images