Image processing method and device and computer readable storage medium

Abstract

The invention provides an image processing method and device and a computer readable storage medium, and relates to the technical field of digital image processing. The method is used for processing original image data based on requirements of a target image, and comprises the following steps: determining an image scaling ratio according to the original image data; determining a blocking mode of the original image according to the image scaling ratio; calculating a processing coefficient for processing the original image according to the blocking mode; and performing image processing on the original image according to the processing coefficient to obtain a target image. By calculating the image scaling ratio and the corresponding processing coefficient, high-quality image scaling is realized, and the quality of details can be kept no matter the image is scaled in or scaled out. The image processing method can be widely applied to various scenes needing image scaling, such as video editing, image enhancement, monitoring systems and the like. The image is dynamically processed according to the scaling, the method is more flexible and adjustable than a traditional method, and meanwhile the requirement for real-time processing is met.

Description

Technical Field

[0001] The present application relates to the technical field of digital image processing, and in particular to an image processing method, device and computer-readable storage medium. Background Art

[0002] In order to obtain better display effects on input images on devices with various resolutions, or to meet different display requirements of customers, it is usually necessary to scale the image. That is, after filtering the original image, it is enlarged or reduced to a target image with a specified resolution. Traditional image scaling algorithms generally use fixed filters to calculate filter coefficients for one or several specific scaling ratios by linear interpolation or bilinear interpolation, thereby achieving image scaling. Although these methods can meet the scaling requirements to a certain extent, they often cannot well calculate the filter coefficients by circuit adaptive calculation for any scaling ratio, so as to perform real-time circuit processing.

[0003] As image resolution increases, traditional image processing techniques may have difficulty processing high-resolution images effectively, especially maintaining sufficient details when scaling down images. Traditional image scaling methods may cause image quality loss, such as blurring, jagged edges, artifacts, or color distortion when zooming in or out. Some older image processing algorithms may be inefficient when processing large-scale image data, resulting in slow processing speeds that cannot meet the needs of real-time or near-real-time processing. In addition, some complex algorithms, such as deep learning-based methods, although they can support arbitrary scaling ratios, are difficult to implement on embedded devices due to the large amount of computation, which limits their scope of application. Summary of the invention

[0004] In view of this, the purpose of the embodiment of the present application is to provide an image processing method to improve the problem of image quality degradation during the scaling process in the prior art. The method is used to process the original image data based on the requirements of the target image, and the method includes: determining the image scaling ratio according to the original image data; determining the block division method of the original image according to the image scaling ratio; calculating the processing coefficient for processing the original image according to the block division method; performing image processing on the original image according to the processing coefficient to obtain the target image.

[0005] In the above implementation process, firstly, the scaling ratios in the horizontal and vertical directions are calculated according to the requirements of the target image and the size of the original image. According to the scaling ratio of the row, the range of input pixels that the output pixel depends on in the vertical direction is determined. According to the scaling ratio of the column, the range of input pixels that the output pixel depends on in the horizontal direction is determined. Based on the scaling ratio, a suitable block strategy is selected to determine the size of each block. For each block, the corresponding processing coefficient is calculated, and the calculated processing coefficient is applied to each block to generate new pixel values ​​through specific operations. Finally, the processed blocks are reassembled into a complete image.

[0006] Optionally, determining an image scaling ratio according to the original image data includes: the image scaling ratio is determined according to the ratio of the size of the original image data to the target image; wherein the image scaling ratio is divided into a row scaling ratio and a column scaling ratio; the row scaling ratio is RIN:ROUT; the column scaling ratio is CIN:COUT; the row scaling ratio is configured to be based on scaling in the row direction of the original image, and the column scaling ratio is configured to be based on scaling in the column direction of the original image, wherein scaling includes enlargement and / or reduction, RIN is the number of rows of the original image, ROUT is the number of rows of the target image, CIN is the number of columns of the original image, and COUT is the number of columns of the target image.

[0007] In the above implementation process, the height of the original image is compared with the height of the target image to calculate the row scaling ratio; or the width of the original image is compared with the width of the target image to calculate the column scaling ratio. The row scaling ratio (RIN:ROUT) is used to configure the scaling of the original image in the vertical direction. If the ratio is greater than 1, it means that it needs to be enlarged; if the ratio is less than 1, it means that it needs to be reduced. The column scaling ratio (CIN:COUT) is used to configure the scaling of the original image in the horizontal direction. Similar to the row direction, the size of the ratio determines the enlargement or reduction. Flexible adjustments can be made according to the requirements of the original image data and the target image to achieve high-quality image scaling.

[0008] Optionally, according to the image scaling ratio, determining the blocking method of the original image includes: determining a minimum repeating unit according to the image scaling ratio; wherein the minimum repeating unit is a minimum size that is divisible by the row scaling ratio or the column scaling ratio; and according to the minimum repeating unit, dividing the input original image into multiple image blocks, with the size of each small block being equal to the size of the minimum repeating unit.

[0009] In the above implementation process, based on the row scaling ratio RIN:ROUT and the column scaling ratio CIN:COUT, the minimum size that can be divided by these two ratios is determined, and this size will be the minimum repeating unit of the original image block. In the case of different row and column scaling ratios, the common multiple of the row and column scaling ratios is selected to scale evenly in both directions. The original image is traversed and the determined minimum repeating unit is used as the size of each block to divide the original image into multiple small blocks. During the segmentation process, the entire original image needs to be covered without missing any area, and overlap is also avoided.

[0010] Optionally, the calculating of the processing coefficient for processing the original image according to the block division method includes: for each image block of the original image, determining the input pixel neighborhood size corresponding to each output pixel; and determining the processing coefficient of each output pixel according to the input pixel neighborhood size.

[0011] In the above implementation process, based on the scaling ratio, for each block in the original image, the size of the input pixel neighborhood corresponding to the scaling process is analyzed. The pixel neighborhood size refers to the set of pixels in the original image that will affect the calculation of each pixel value in the block. For each output pixel, the processing coefficient is calculated according to its corresponding input pixel neighborhood size, and these coefficients reflect the contribution of each input pixel in the neighborhood to the output pixel value.

[0012] Optionally, for each image block of the original image, determining the corresponding input pixel neighborhood size includes: when the image scaling ratio is determined to be the row magnification ratio, determining the pixel neighborhood size corresponding to the first row of pixel outputs to be the pixel size of the first row; for pixels other than the first row, limiting the corresponding pixel neighborhood size range to [ceiling((m-1)×RIN / ROUT), ceiling(m×RIN / ROUT)]; when the image scaling ratio is determined to be the row reduction ratio, determining the pixel neighborhood size range corresponding to the first row of pixel outputs to be [1, ceiling(RIN / ROUT)]; for pixels other than the first row, limiting the corresponding pixel neighborhood size range to [ceiling((m-1)×RIN / ROUT), ceiling(m×RIN / ROUT)]; wherein ceiling(x) represents rounding up x, and m is the index of the current row in the target image.

[0013] In the above implementation process, if the image scaling ratio is determined as the row magnification ratio, then for the first row of pixels in the output image, the corresponding input pixel neighborhood size is the pixel size of the first row of the input image. This means that the first row of pixels in the output image is directly mapped to the first row of pixels in the input image. For pixels in the output image other than the first row, the corresponding input pixel neighborhood size range is limited to [ceiling((m-1)×RIN / ROUT),ceiling(m×RIN / ROUT)]. Here, m is the index of the current row in the output image. If the image scaling ratio is determined as the row reduction ratio, then for the first row of pixels in the output image, the corresponding input pixel neighborhood size range is from the first row of pixels to the ceiling(RIN / ROUT)th row of pixels. Here, RIN is the number of rows in the original image, ROUT is the number of rows in the target image, and ceiling(x) means rounding up x. Similarly, for pixels in non-first rows of the output image, the corresponding input pixel neighborhood size range is also limited to [ceiling((m-1)×RIN / ROUT), ceiling(m×RIN / ROUT)]. After determining the input neighborhood size of each output pixel, the processing coefficient can be calculated. The processing coefficient will be used to determine how the input pixel affects the value of the output pixel. Using the calculated processing coefficient, the pixels in each block are processed to generate new pixel values.

[0014] Optionally, for each image block of the original image, determining the corresponding input pixel neighborhood size, including: when the image scaling ratio is determined to be the column magnification ratio, determining the pixel neighborhood size corresponding to the first column pixel output to be the pixel size of the first column; for pixels other than the first column, limiting the corresponding pixel neighborhood size range to [ceiling((n-1)×CIN / COUT), ceiling(n×CIN / COUT)]; when the image scaling ratio is determined to be the column reduction ratio, determining the pixel neighborhood size range corresponding to the first column pixel output to be [1, ceiling(CIN / COUT)]; for pixels other than the first column, limiting the corresponding pixel neighborhood size range to [ceiling((n-1)×CIN / COUT), ceiling(n×CIN / COUT)]; wherein n is the index of the current column in the target image.

[0015] In the above implementation process, if the image scaling ratio is determined to be a column magnification ratio, then for the first column of pixels in the output image, the corresponding input pixel neighborhood size is the pixel size of the first column of the input image. That is, the first column of pixels in the output image is directly mapped to the first column of pixels in the input image. For pixels in non-first column of the output image, the corresponding input pixel neighborhood size range is limited to [ceiling((n-1)×CIN / COUT), ceiling(n×CIN / COUT)], where n is the index of the current column in the output image. If the image scaling ratio is determined to be a column reduction ratio, then for the first column of pixels in the output image, the corresponding input pixel neighborhood size range is from the first column of pixels to the ceiling(CIN / COUT)th column of pixels. Here, CIN is the number of columns of the original image, COUT is the number of columns of the target image, and ceiling(x) means rounding up x.

[0016] Optionally, determining a processing coefficient for each output pixel according to the size of the input pixel neighborhood includes: calculating a relative position of the output pixel and each input pixel in the neighborhood when the size of the input pixel neighborhood is determined; and determining the processing coefficient according to the relative positions.

[0017] In the above implementation, if the output image is larger than the input image, it is necessary to determine which pixels in the input image each output pixel should cover. For example, if a row of the output image is twice as long as a row of the input image, then each output pixel will correspond to two pixels in the input image. If the output image is smaller than the input image, it is also necessary to determine which pixels in the input image each output pixel should cover. For example, if a row of the output image is half as long as a row of the input image, then every two pixels in the input image will correspond to one pixel in the output image. For each output pixel, we need to know its specific position in the neighborhood of the input pixel. Once we know the position of the output pixel in the input image, we can calculate the contribution of each input pixel to the output pixel value, which is the processing coefficient. That is, assign a weight to each pixel in the input image, indicating how much influence each pixel has on the final output pixel value.

[0018] Optionally, according to the processing coefficient, image processing is performed on the original image to obtain a target image, including: applying the processing coefficient to the original image, and for each output pixel, summing up all pixel values ​​in its neighborhood weighted according to the processing coefficient to obtain the value of the output pixel; until all pixels in the output image are calculated to obtain the target image.

[0019] In the above implementation process, for each pixel in the output image, its corresponding processing coefficient has been calculated, which indicates the contribution of each neighborhood pixel in the input image to the output pixel value. The processing coefficient is applied to the original image, and for each output pixel, all pixel values ​​in its neighborhood weighted by the processing coefficient are summed to obtain the value of the output pixel. All pixels in the output image are traversed so that the value of each output pixel is calculated by the above weighted summation method until all pixels in the output image are calculated to obtain the processed complete image.

[0020] An embodiment of the present application also provides an image processing device, which includes: an image input module, a scaling ratio determination module, an image scaling module and an image output module; the image input module is configured to receive original image data; the scaling ratio determination module is configured to determine the image scaling ratio of the original image; the image scaling module is configured to determine the blocking method of the original image according to the image scaling ratio; calculate the processing coefficient for processing the original image according to the blocking method; perform image processing on the original image according to the processing coefficient; and the image output module is configured to output a target image.

[0021] In the above implementation process, the image input module receives the original image data and passes it to the scaling ratio determination module. The scaling ratio determination module calculates the scaling ratio of the image and passes the result to the image scaling module. The image scaling module divides the original image into blocks according to the scaling ratio and calculates the processing coefficient for each block. Using the calculated processing coefficient, the image scaling module processes each image block to generate a new pixel value. The target image blocks are reassembled into a complete image and then output by the image output module.

[0022] The image input module is responsible for receiving the original image data provided by the user. The scaling ratio determination module determines the scaling ratio of the image, including the row scaling ratio and the column scaling ratio. The image scaling module is responsible for processing the original image according to the scaling ratio, and determining the block division method of the original image according to the scaling ratio. The size of each block is the minimum unit size determined according to the scaling ratio. Based on this, the processing coefficient is calculated for each image block, and the processing coefficient is applied to process each image block to generate a new pixel value. The image output module is used to output the target image.

[0023] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer program instructions, and when the computer program instructions are read and executed by a processor, the steps in any of the above implementation methods are executed.

[0024] In the above implementation process, the computer-readable storage medium can be any form of physical storage device, such as a hard disk, a solid-state hard disk, an optical disk, a USB flash drive, etc. The storage medium stores computer program instructions, which specifically implement the various steps in the above image processing method, including image input, scaling ratio determination, image block processing, processing coefficient calculation, image processing execution, and image output. When a processor (such as a CPU) reads the computer program instructions in the storage medium, these instructions are loaded into the memory. The program instructions loaded into the memory are executed by the processor and begin to process the image according to the preset algorithm steps. The program first receives the original image data through the image input module, which can be uploaded by the user through the interface or directly obtained from other data sources. Then, the program runs the scaling ratio determination module to calculate the scaling ratio of the image, including the ratio of rows and columns. According to the determined scaling ratio, the program executes the image scaling module, determines the block mode, and calculates the processing coefficient of each block. The program uses the calculated processing coefficient to process each block of the original image to generate a new pixel value. The processed blocks are reassembled into a complete image and output through the image output module, which can be displayed on the screen or saved as a file. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 A flowchart of an image processing method provided in an embodiment of the present application;

[0027] Figure 2 An overall flow chart of the image processing method provided in the embodiment of the present application;

[0028] Figure 3 A schematic diagram of a minimum repeating unit provided in an embodiment of the present application;

[0029] Figure 4 A schematic diagram of processing a minimum repeating unit using a processing coefficient provided in an embodiment of the present application;

[0030] Figure 5 The image zoom ratio provided in the embodiment of the present application is determined as a row zoom ratio;

[0031] Figure 6 The image scaling ratio provided in the embodiment of the present application is determined as a row scaling ratio;

[0032] Figure 7 The image zoom ratio provided in the embodiment of the present application is determined as a column zoom ratio;

[0033] Figure 8 The image scaling ratio provided in the embodiment of the present application is determined as a column reduction ratio;

[0034] Fig. 9 A schematic diagram of an image processing device provided in an embodiment of the present application;

[0035] Fig.10 A schematic diagram of a computer-readable medium provided for an embodiment of the present application.

[0036] Icons: 11-image input module; 20-scaling ratio determination module; 30-image scaling module; 40-image output module; 100-electronic device; 111-memory; 112-storage controller; 113-processor; 114-peripheral interface; 115-input and output unit; 116-display unit. DETAILED DESCRIPTION

[0037] The technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments of the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the embodiments of the present application.

[0038] See also Figure 1 , Figure 1 A flow chart of the image processing method provided in an embodiment of the present application.

[0039] An embodiment of the present application provides an image processing method for processing original image data based on the requirements of a target image, the method comprising: determining an image scaling ratio according to the original image data; determining a block division method of the original image according to the image scaling ratio; calculating a processing coefficient for processing the original image according to the block division method; and performing image processing on the original image according to the processing coefficient to obtain a target image.

[0040] In the above implementation process, the scaling ratios in the horizontal and vertical directions are calculated according to the requirements of the target image and the size of the original image. The scaling ratio of the columns (col_ratio_in, col_ratio_out) is calculated in the horizontal direction, that is, the ratio of the number of columns in the original image to the number of columns in the target image. The scaling ratio of the rows (row_ratio_in, row_ratio_out) is calculated in the vertical direction, that is, the ratio of the number of rows in the original image to the number of rows in the target image. Based on the scaling ratio, a suitable blocking strategy is selected and the size of each block is determined. According to the scaling ratios of rows and columns, the image is divided into blocks. For example, if the row scaling ratio is 3:4 and the column scaling ratio is 3:4, a 3x3 input data block may be selected to correspond to a 4x4 output data block. The size of each block is determined, for example, if the scaling ratio is 3:4, each block may contain 3 rows and 3 columns of pixels. For each block, the corresponding processing coefficient is calculated. The processing coefficient is calculated according to the position and scaling ratio of each pixel in the block. This involves determining the range of input pixels that each output pixel depends on and calculating the weights of these input pixels. The calculated processing coefficients are applied to each block, new pixel values ​​are generated through specific operations, and all processed blocks are recombined according to the structure of the original image to form the final scaled image.

[0041] See also Figure 2 , Figure 2 This is an overall flow chart of the image processing method provided in the embodiment of the present application.

[0042] Optionally, the image scaling ratio is determined according to the original image data, including: the image scaling ratio is determined according to the ratio of the size of the original image data to the target image; wherein the image scaling ratio is divided into a row scaling ratio and a column scaling ratio; the row scaling ratio is RIN:ROUT; the column scaling ratio is CIN:COUT; the row scaling ratio is configured as scaling based on the row direction of the original image, and the column scaling ratio is configured as scaling based on the column direction of the original image, wherein scaling includes enlargement and / or reduction, RIN is the number of rows of the original image, ROUT is the number of rows of the target image, CIN is the number of columns of the original image, and COUT is the number of columns of the target image.

[0043] In the above implementation process, the ratio of the size of the original image data to the target image is determined based on user requirements or automatic recognition. Among them, the magnification / reduction ratio in the row direction is denoted as RIN = row_ratio_in, ROUT = row_ratio_out, and the magnification / reduction ratio in the column direction is denoted as CIN = col_ratio_in, COUT = col_ratio_out. Reducing the ratio to the simplest form is more conducive to circuit implementation. The scaling ratios of rows and columns are configured separately to achieve independent scaling in the vertical and horizontal directions. If the size of the target image is larger than the original image, the algorithm needs to increase pixels through techniques such as interpolation to achieve image magnification. If the size of the target image is smaller than the original image, the algorithm needs to reduce pixels to achieve image reduction while trying to retain important information. The row scaling ratio (RIN:ROUT) calculates the ratio between the number of rows of the original image and the number of rows of the target image. This ratio represents the scaling degree in the vertical direction and can be magnification (ROUT > RIN) or reduction (ROUT < RIN). The column scaling ratio (CIN:COUT) calculates the ratio between the number of columns of the original image and the number of columns of the target image. This ratio represents the scaling degree in the horizontal direction and can be magnification (COUT > CIN) or reduction (COUT < CIN).

[0044] Optionally, according to the image scaling ratio, determine the way of dividing the original image into blocks, including: determining the minimum repeating unit according to the image scaling ratio; where the minimum repeating unit is the smallest size divisible by the row scaling ratio or the column scaling ratio; dividing the input original image into multiple image blocks according to the minimum repeating unit, and the size of each small block is equal to the size of the minimum repeating unit.

[0045] In the above implementation process, according to the image scaling ratio, determine the minimum repeating unit used when processing the image. The minimum repeating unit refers to the smallest size that can be divisible by the row scaling ratio (RIN:ROUT) or the column scaling ratio (CIN:COUT). This unit will be used as the basis for dividing the image into blocks, so as to maintain the consistency of the ratio during the scaling process. Divide the original image into multiple small blocks according to the determined minimum repeating unit, and the size of each small block is equal to the size of the minimum repeating unit. In this way, during the scaling process, each small block can be processed evenly. By reasonably dividing the blocks, the algorithm can effectively utilize hardware resources, reduce the computational complexity, and improve the speed and effect of image processing.

[0046] Optionally, according to the way of dividing the blocks, calculate the processing coefficients for processing the original image, including: for each image block of the original image, determine the corresponding size of the input pixel neighborhood; according to the size of the input pixel neighborhood, determine the processing coefficient for each output pixel.

[0047] In the above implementation process, for each image block of the original image, the size of the input pixel neighborhood on which each output pixel depends is determined. If the output pixel is located in the mth row and the nth column, then its corresponding input neighborhood can be calculated by the scaling ratio. For the row direction, the start and end rows of the neighborhood can be calculated by the formulas start_row = ceiling((m-1)×RIN / ROUT) and end_row = ceiling(m×RIN / ROUT), and the same is true for the column direction. According to the size of the input pixel neighborhood, for each output pixel, according to its position in the input neighborhood, the corresponding processing coefficient is calculated, where the row / column processing coefficient is recorded as row_filter / col_filter respectively to reflect the contribution of the input pixel to the output pixel value. The calculation of the processing coefficient can be based on a variety of methods, such as nearest neighbor interpolation, bilinear interpolation, or more advanced interpolation methods. For example, in bilinear interpolation, the weight of each input pixel may be a function of its distance to the output pixel. The calculated processing coefficient is applied to each image block to generate a new pixel value. For each output pixel in each image block, the value of the output pixel is determined by weighted averaging or other specified calculation method using its corresponding processing coefficient and the pixel values ​​in the input neighborhood.

[0048] Optionally, determining the processing coefficient of each output pixel according to the size of the input pixel neighborhood includes: calculating the relative position of the output pixel and each input pixel in the neighborhood when the size of the input pixel neighborhood is determined; and determining the processing coefficient according to the relative position.

[0049] In the above implementation process, after determining the size of the input pixel neighborhood, for each output pixel, its position relative to each input pixel in the neighborhood is calculated according to its coordinates in the original image and the size of the input neighborhood. The processing coefficient is usually determined based on the relative position relationship between the input pixel neighborhood and the output pixel. There are many ways to calculate it. The filter coefficient of each neighborhood pixel can be determined by calculating the proportion in the row / column direction or the area ratio. For example, in bilinear interpolation, the processing coefficient is calculated based on the distance between the output pixel and the input pixel. The closer the distance, the greater the contribution of the input pixel to the output pixel. The calculation of the processing coefficient can adopt different interpolation methods, such as nearest neighbor interpolation, bilinear interpolation, bicubic interpolation, etc. Each method has its own specific weight allocation method. Data processing can also be performed by converting to another space, such as converting to YUV space, or linear space, etc.

[0050] Optionally, image processing is performed on the original image according to the processing coefficient to obtain a target image, including: applying the processing coefficient to the original image, and for each output pixel, summing up all pixel values ​​in its neighborhood weighted by the processing coefficient to obtain the value of the output pixel; until all pixels in the output image are calculated to obtain the target image.

[0051] In the above implementation process, for each output pixel, the weighted sum of pixel values ​​is calculated by multiplying each input pixel value by its corresponding processing coefficient according to its processing coefficient and the input pixel values ​​in the neighborhood. Each pixel position in the output image is traversed and the weighted sum operation is repeated until the values ​​of all output pixels are calculated. When all pixel values ​​in the output image are calculated, the final processed image is obtained.

[0052] See also Figure 3 and Figure 4 , Figure 3 A schematic diagram of a minimum repeating unit provided in an embodiment of the present application; Figure 4 A schematic diagram of processing a minimum repeating unit using a processing coefficient provided in an embodiment of the present application.

[0053] In one embodiment of the present application, the row / column magnification algorithm is used as an example for explanation. Assuming that the input image resolution is 810×1800 and the output image resolution is 1080×2400, it can be simplified to obtain row_ratio_in=3, row_ratio_out=4, that is, 1800:2400=3:4 is satisfied. Similarly, col_ratio_in=3, col_ratio_out=4. Then we can determine that on a panel of the same size, a 3x3 input data matrix will obtain a 4x4 output data matrix after the scaling algorithm, that is, a minimum repeating unit for calculating the processing coefficient. The blue box represents the input data, and the red dotted box represents the output data. Taking the output pixel (purple box) in the second row and second column of the output data matrix as an example, determine its corresponding input pixel neighborhood and calculate its corresponding processing coefficient. According to the formula [ceiling((m-1)×RIN / ROUT),ceiling(m×RIN / ROUT)], the corresponding input row neighborhood is calculated as [1,2] and column neighborhood is [1,2] according to the current output pixel row and column information, and the output pixel of the second row and second column and the input pixel area related to it are drawn separately. Next, the processing coefficient corresponding to the current output pixel is determined. According to the row & column ratio relationship, the up_row_para & left_col_para occupied by the current output pixel on the D11 input pixel can be determined. Assuming that the output pixel side length is unit 1, the input pixel side length is 4 / 3, then up_row_para=1 / 3, left_col_para=1 / 3, and similarly, down_row_para=2 / 3, right_col_para=2 / 3. Thus, the row processing coefficient corresponding to the current input pixel is row_filter=[up_row_para,down_row_para]=[1 / 3;2 / 3], and the column processing coefficient is col_filter=[left_col_para,right_col_para]=[1 / 3,2 / 3]. Multiplying the row and column processing, we get the processing coefficient matrix filter=row_filter×col_filter=[1 / 9,2 / 9;2 / 9,4 / 9]; then the current output pixel value value=filter×[Din1,Din2;Din3,Din4]=1 / 9×Din1+2 / 9×Din2+2 / 9×Din3+4 / 9×Din4.

[0054] In one embodiment of the present application, at the edge of the image block, special processing may be required because the neighborhood may exceed the boundary of the image. In this case, edge extension, mirroring or other techniques can be used to handle these boundary conditions.

[0055] See also Figure 5 and Figure 6 , Figure 5 The image zoom ratio provided in the embodiment of the present application is determined as a row zoom ratio; Figure 6 The embodiment of the present application provides a case where the image scaling ratio is determined as a row scaling ratio.

[0056] Optionally, for each image block of the original image, the corresponding input pixel neighborhood size is determined, including: when the image scaling ratio is determined to be the row magnification ratio, determining the pixel neighborhood size corresponding to the first row of pixel outputs to be the pixel size of the first row; for pixels other than the first row, limiting the corresponding pixel neighborhood size range to [ceiling((m-1)×RIN / ROUT), ceiling(m×RIN / ROUT)]; when the image scaling ratio is determined to be the row reduction ratio, determining the pixel neighborhood size range corresponding to the first row of pixel outputs to be [1, ceiling(RIN / ROUT)]; for pixels other than the first row, limiting the corresponding pixel neighborhood size range to [ceiling((m-1)×RIN / ROUT), ceiling(m×RIN / ROUT)]; wherein ceiling(x) represents rounding up x, and m is the index of the current row in the target image.

[0057] In the above implementation process, in the case of row magnification ratio, the pixel neighborhood size corresponding to the first row pixel output is determined. In the case of row magnification ratio, the neighborhood size of the first row pixel directly corresponds to the pixel size of the first row. That is, each output pixel of the first row will be directly associated with the corresponding pixel of the first row of the input image. For each pixel of the non-first row, its corresponding input neighborhood range is calculated by the formula [ceiling((m-1)×RIN / ROUT),ceiling(m×RIN / ROUT)], where m is the row index of the current output pixel, RIN is the number of rows of the original image, ROUT is the number of rows of the target image, and ceiling(x) means rounding up x. In the case of row reduction ratio, the pixel neighborhood size range corresponding to the first row pixel output is determined. In the case of row reduction ratio, the neighborhood size range of the first row pixel is determined to be [1,ceiling(RIN / ROUT)], that is, each output pixel of the first row will be associated with a smaller pixel range of the input image. Limit the pixel neighborhood size range corresponding to the non-first row pixels. For each pixel other than the first row, the corresponding input neighborhood range is also calculated by the formula [ceiling((m-1)×RIN / ROUT),ceiling(m×RIN / ROUT)].

[0058] Please see, Figure 7 and Figure 8 , Figure 7 The image zoom ratio provided in the embodiment of the present application is determined as a column zoom ratio; Figure 8 The image scaling ratio provided in the embodiment of the present application is determined as a column reduction ratio.

[0059] Optionally, for each image block of the original image, the corresponding input pixel neighborhood size is determined, including: when the image scaling ratio is determined to be the column magnification ratio, determining the pixel neighborhood size corresponding to the first column pixel output to be the pixel size of the first column; for pixels other than the first column, limiting the corresponding pixel neighborhood size range to [ceiling((n-1)×CIN / COUT), ceiling(n×CIN / COUT)]; when the image scaling ratio is determined to be the column reduction ratio, determining the pixel neighborhood size range corresponding to the first column pixel output to be [1, ceiling(CIN / COUT)]; for pixels other than the first column, limiting the corresponding pixel neighborhood size range to [ceiling((n-1)×CIN / COUT), ceiling(n×CIN / COUT)]; wherein n is the index of the current column in the target image.

[0060] In the above implementation process, in the case of column magnification ratio, the pixel neighborhood size corresponding to the first column pixel output is determined. In the case of column magnification ratio, the neighborhood size of the first column pixel directly corresponds to the pixel size of the first column, that is, each output pixel of the first column will be directly associated with the corresponding pixel of the first column of the input image. For each pixel other than the first column, its corresponding input neighborhood range is calculated by the formula [ceiling((n-1)×CIN / COUT),ceiling(n×CIN / COUT)], where n is the column index of the current output pixel, CIN is the number of columns of the original image, COUT is the number of columns of the target image, and ceiling(x) means rounding up x. In the case of column reduction ratio, the pixel neighborhood size range corresponding to the first column pixel output is determined. In the case of column reduction ratio, the neighborhood size range of the first column pixel is determined as [1,ceiling(CIN / COUT)], that is, each output pixel of the first column will be associated with a smaller pixel range of the input image. For each pixel other than the first column, the corresponding input neighborhood range is also calculated by the formula [ceiling((n-1)×CIN / COUT),ceiling(n×CIN / COUT)].

[0061] See also Fig. 9 , Fig. 9 A schematic diagram of an image processing device provided in an embodiment of the present application.

[0062] The embodiment of the present application also provides an image processing device, which includes: an image input module 11, a scaling ratio determination module 20, an image scaling module 30 and an image output module 40; the image input module 11 is configured to receive original image data; the scaling ratio determination module 20 is configured to determine the image scaling ratio of the original image; the image scaling module 30 is configured to determine the blocking method of the original image according to the image scaling ratio; according to the blocking method, calculate the processing coefficient for processing the original image; according to the processing coefficient, perform image processing on the original image; the image output module 40 is configured to output the target image.

[0063] In the above implementation process, the image input module 11 is responsible for receiving the original image data provided by the user. The scaling ratio determination module 20 determines the scaling ratio of the image, including the row scaling ratio and the column scaling ratio. The image scaling module 30 is responsible for processing the original image according to the scaling ratio, and determining the block division method of the original image according to the scaling ratio, and the size of each block is the minimum unit size determined according to the scaling ratio. Based on this, the processing coefficient is calculated for each image block, and the processing coefficient is applied to process each image block to generate a new pixel value; the image output module 40 is used to output the target image.

[0064] Optionally, see Fig.10 , Fig.10 A block diagram of a computer-readable storage medium provided in an embodiment of the present application.

[0065] The embodiment of the present application further provides a computer-readable storage medium, in which computer program instructions are stored. When the computer program instructions are read and executed by a processor, the steps in any of the above implementation methods are executed.

[0066] In the above implementation process, the computer-readable storage medium can be any form of physical storage device, such as a hard disk, a solid-state hard disk, an optical disk, a USB flash drive, etc. The storage medium stores computer program instructions, which specifically implement the various steps in the above image processing method, including image input, scaling ratio determination, image block processing, processing coefficient calculation, image processing execution, and image output. When a processor (such as a CPU) reads the computer program instructions in the storage medium, these instructions are loaded into the memory. The program instructions loaded into the memory are executed by the processor and begin to process the image according to the preset algorithm steps. The program first receives the original image data through the image input module, which can be uploaded by the user through the interface or directly obtained from other data sources. Then, the program runs the scaling ratio determination module to calculate the scaling ratio of the image, including the ratio of rows and columns. According to the determined scaling ratio, the program executes the image scaling module, determines the block mode, and calculates the processing coefficient of each block. The program uses the calculated processing coefficient to process each block of the original image to generate a new pixel value. The processed blocks are reassembled into a complete image and output through the image output module, which can be displayed on the screen or saved as a file. The electronic device 100 may include a memory 111, a storage controller 112, a processor 113, a peripheral interface 114, an input and output unit 115, and a display unit 116. It can be understood by those skilled in the art that Fig.10 The structure shown is only for illustration and does not limit the structure of the electronic device 100. For example, the electronic device 100 may further include Figure 8 More or fewer components as shown, or with Figure 8 Different configurations are shown.

[0067] The above-mentioned memory 111, storage controller 112, processor 113, peripheral interface 114, input / output unit 115 and display unit 116 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The above-mentioned processor 113 is used to execute the executable module stored in the memory. Among them, the memory 111 can be, but not limited to, random access memory (Random Access Memory, referred to as RAM), read only memory (Read Only Memory, referred to as ROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), erasable read-only memory (ErasableProgrammable Read-Only Memory, referred to as EPROM), electrically erasable read-only memory (Electric ErasableProgrammable Read-Only Memory, referred to as EEPROM), etc. Among them, the memory 111 is used to store programs, and the processor 113 executes the program after receiving the execution instruction. The method executed by the electronic device 100 defined by the process disclosed in any embodiment of the present application can be applied to the processor 113 or implemented by the processor 113.

[0068] The processor 113 may be an integrated circuit chip with signal processing capability. The processor 113 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or any conventional processor.

[0069] The peripheral interface 114 couples various input / output devices to the processor 113 and the memory 111. In some embodiments, the peripheral interface 114, the processor 113 and the memory controller 112 can be implemented in a single chip. In other embodiments, they can be implemented by separate chips.

[0070] The input and output unit 115 is used to provide input data to the user. The input and output unit 115 can be, but is not limited to, a mouse and a keyboard.

[0071] The above-mentioned display unit 116 provides an interactive interface (such as a user operation interface) between the electronic device 100 and the user or is used to display image data for the user's reference. In the present embodiment, the display unit may be a liquid crystal display or a touch display. If it is a touch display, it may be a capacitive touch screen or a resistive touch screen that supports single-point and multi-point touch operations. Supporting single-point and multi-point touch operations means that the touch display can sense touch operations generated simultaneously from one or more positions on the touch display, and the sensed touch operations are handed over to the processor for calculation and processing. In an embodiment of the present application, the display unit 116 can display input and / or output images.

[0072] In summary, in several embodiments provided in the present application, it should be understood that the disclosed device can also be implemented in other ways. The device embodiments described above are merely schematic, for example, the block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices according to the multiple embodiments of the present application. In this regard, each box in the block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram, and the combination of the block diagrams, can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0073] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0074] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0075] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

Claims

1. An image processing method, characterized in that: The method is used to process raw image data based on the requirements of the target image, and the method comprises: Determining an image scaling ratio according to the original image data; Determining a block division method of the original image according to the image scaling ratio; Calculating a processing coefficient for processing the original image according to the block division method; The original image is processed according to the processing coefficient to obtain a target image.

2. The method according to claim 1, characterized in that Determining an image scaling ratio according to the original image data includes: The image scaling ratio is determined according to the ratio of the original image data size to the target image; The image scaling ratio is divided into a row scaling ratio and a column scaling ratio; the row scaling ratio is RIN:ROUT; the column scaling ratio is CIN:COUT; the row scaling ratio is configured based on the scaling in the row direction of the original image, and the column scaling ratio is configured based on the scaling in the column direction of the original image. Scaling includes zooming in and / or zooming out, RIN is the number of rows of the original image, ROUT is the number of rows of the target image, CIN is the number of columns of the original image, and COUT is the number of columns of the target image.

3. The method according to claim 2, characterized in that Determining a block mode of the original image according to the image scaling ratio includes: Determine a minimum repeating unit according to the image scaling ratio; wherein the minimum repeating unit is a minimum size that is divisible by the row scaling ratio or the column scaling ratio; According to the minimum repeating unit, the input original image is divided into a plurality of image blocks, and the size of each small block is equal to the size of the minimum repeating unit.

4. The method according to claim 3, characterized in that The step of calculating the processing coefficient for processing the original image according to the block division method includes: For each of the image blocks of the original image, determining a size of an input pixel neighborhood corresponding to each output pixel; A processing coefficient for each output pixel is determined according to the size of the input pixel neighborhood.

5. The method according to claim 4, characterized in that Determining the size of the corresponding input pixel neighborhood for each image block of the original image includes: When the image scaling ratio is determined to be a row magnification ratio, determining the pixel neighborhood size corresponding to the first row of pixel outputs to be the pixel size of the first row; For pixels other than the first row, the size range of the corresponding pixel neighborhood is limited to [ceiling((m-1)×RIN / ROUT),ceiling(m×RIN / ROUT)]; When the image scaling ratio is determined to be a row reduction ratio, determining that the pixel neighborhood size range corresponding to the first row of pixel outputs is [1, ceiling (RIN / ROUT)]; For pixels other than the first row, the size range of the corresponding pixel neighborhood is limited to [ceiling((m-1)×RIN / ROUT),ceiling(m×RIN / ROUT)]; Wherein, ceiling(x) means rounding x upwards, and m is the index of the current row in the target image.

6. The method according to claim 4, characterized in that For each image block of the original image, determining a corresponding input pixel neighborhood size comprises: When the image scaling ratio is determined to be a column magnification ratio, determining the pixel neighborhood size corresponding to the first column pixel output to be the pixel size of the first column; For pixels other than the first column, the size range of the corresponding pixel neighborhood is limited to [ceiling((n-1)×CIN / COUT),ceiling(n×CIN / COUT)]; When the image scaling ratio is determined to be a column reduction ratio, determining that the pixel neighborhood size range corresponding to the first column pixel output is [1, ceiling (CIN / COUT)]; For pixels other than the first column, the size range of the corresponding pixel neighborhood is limited to [ceiling((n-1)×CIN / COUT),ceiling(n×CIN / COUT)]; Where n is the index of the current column in the target image.

7. The method according to claim 4, characterized in that Determining a processing coefficient for each output pixel according to the size of the input pixel neighborhood includes: When the size of the input pixel neighborhood is determined, the relative position of the output pixel and each input pixel in the neighborhood is calculated; and the processing coefficient is determined according to the relative position.

8. The method according to claim 7, characterized in that According to the processing coefficient, the original image is processed to obtain a target image, including: The processing coefficient is applied to the original image, and for each output pixel, all pixel values ​​in its neighborhood weighted by the processing coefficient are summed to obtain the value of the output pixel; until all pixels in the image are calculated, the target image is obtained.

9. An image processing device, characterized in that: The device comprises: an image input module, a scaling ratio determination module, an image scaling module and an image output module; The image input module is configured to receive original image data; the scaling ratio determination module is configured to determine the image scaling ratio of the original image; The image scaling module is configured to determine a block division method of the original image according to the image scaling ratio; calculate a processing coefficient for processing the original image according to the block division method; and perform image processing on the original image according to the processing coefficient; The image output module is configured to output a target image.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the steps in the method according to any one of claims 1 to 8 are executed.

Images