Image processing method and apparatus, and computer-readable storage medium

Abstract

The present disclosure relates to the technical field of digital image processing. Provided are an image processing method and apparatus, and a computer-readable storage medium. The method is used for processing original image data on the basis of the requirements for a target image. The method comprises: on the basis of original image data, determining an image scaling ratio; on the basis of the image scaling ratio, determining a block partitioning mode for an original image; on the basis of the block partitioning mode, calculating a processing coefficient for processing the original image; and on the basis of the processing coefficient, performing image processing on the original image, in order to obtain a target image. Image scaling ratios and corresponding processing coefficients are calculated to realize high-quality image scaling, thereby maintaining detail quality for both upscaling and downscaling. The image processing method can be widely applied to various scenarios requiring image scaling, such as video editing, image enhancement and surveillance systems. In the method, an image is dynamically processed on the basis of a scaling ratio, thereby achieving greater flexibility and adjustability over a conventional method, and also meeting the requirement of real-time processing.

Description

Image processing methods, apparatus and computer-readable storage media

[0001] Cross-references to related applications

[0002] This disclosure claims priority to Chinese Patent Application No. 2025101022877, filed on January 22, 2025, entitled "Image Processing Method, Apparatus and Computer-Readable Storage Medium", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of digital image processing technology, and more specifically, to an image processing method, apparatus, and computer-readable storage medium. Background Technology

[0004] To achieve better display of input images on devices with various resolutions or to meet different customer display requirements, image scaling is typically necessary. This involves filtering the original image and then enlarging or reducing it to a target image of a specified resolution. Traditional image scaling algorithms generally use fixed filters, calculating filter coefficients for one or more specific scaling ratios using linear or bilinear interpolation to achieve image scaling. While these methods can meet scaling requirements to some extent, they often fail to provide adequate real-time circuitry for adaptively calculating filter coefficients at arbitrary scaling ratios.

[0005] As image resolution increases, traditional image processing techniques may struggle to effectively handle high-resolution images, especially when reducing image size to maintain sufficient detail. Traditional image scaling methods can lead to image quality loss during enlargement or reduction, such as blurring, jagged edges, artifacts, or color distortion. Some older image processing algorithms may be inefficient when processing large-scale image data, resulting in slow processing speeds and failing to meet the demands of real-time or near real-time processing. Furthermore, some complex algorithms, such as deep learning-based methods, while supporting arbitrary scaling ratios, are computationally intensive and difficult to implement on embedded devices, limiting their application scope. Summary of the Invention

[0006] In a first aspect, embodiments of this disclosure provide an image processing method to improve the image quality degradation problem during scaling in the prior art. The method is used to process original image data based on the requirements of a target image. The method includes: determining an image scaling ratio based on the original image data; determining a block division method for the original image based on the image scaling ratio; calculating processing coefficients for processing the original image based on the block division method; and performing image processing on the original image based on the processing coefficients to obtain a target image.

[0007] In the above implementation process, firstly, the scaling ratios in the horizontal and vertical directions are calculated based on the requirements of the target image and the size of the original image. Based on the row scaling ratio, the range of input pixels that the output pixels depend on in the vertical direction is determined. Based on the column scaling ratio, the range of input pixels that the output pixels depend on in the horizontal direction is determined. Based on the scaling ratios, an appropriate block segmentation strategy is selected, and the size of each block is determined. For each block, corresponding processing coefficients are calculated and applied to each block, generating new pixel values ​​through specific operations. Finally, the processed blocks are reassembled into a complete image.

[0008] In one optional implementation, determining the image scaling ratio based on the original image data includes: the image scaling ratio being determined based on the ratio of the size of the original image data to the target image; wherein the image scaling ratio is divided into row scaling ratio and column scaling ratio; the row scaling ratio is RIN:ROUT; the column scaling ratio is CIN:COUT; the row scaling ratio is configured to scale based on the row direction of the original image, and the column scaling ratio is configured to scale based on the column direction of the original image, wherein scaling includes enlarging and / or reducing, RIN is the number of rows in the original image, ROUT is the number of rows in the target image, CIN is the number of columns in the original image, and COUT is the number of columns in the target image.

[0009] In the above implementation process, the height of the original image is compared with the height of the target image to calculate the row scaling ratio; or the width of the original image is compared with the width of the target image to calculate the column scaling ratio. The row scaling ratio (RIN:ROUT) configures the scaling of the original image in the vertical direction. If the ratio is greater than 1, it indicates that the image needs to be enlarged; if the ratio is less than 1, it indicates that the image needs to be reduced. The column scaling ratio (CIN:COUT) configures the scaling of the original image in the horizontal direction. Similar to the row direction, the size of the ratio determines whether the image is enlarged or reduced. This allows for flexible adjustments based on the requirements of the original image data and the target image, achieving high-quality image scaling.

[0010] In one optional implementation, determining the segmentation method of the original image according to the image scaling ratio includes: determining the minimum repeating unit according to the image scaling ratio; wherein the minimum repeating unit is the smallest size divisible by the row scaling ratio or the column scaling ratio; and segmenting the input original image into multiple image blocks according to the minimum repeating unit, wherein the size of each block is equal to the size of the minimum repeating unit.

[0011] In the above implementation, based on the row scaling ratio RIN:ROUT and the column scaling ratio CIN:COUT, the smallest size divisible by these two ratios is determined. This size will be the smallest repeating unit for the original image blocks. When the row and column scaling ratios are different, a common multiple of the row and column scaling ratios is chosen to ensure uniform scaling in both directions. The original image is traversed, using the determined smallest repeating unit as the size of each block, dividing the original image into multiple smaller blocks. During segmentation, the entire original image must be covered without omitting any areas, while also avoiding overlap.

[0012] In one optional implementation, the step of calculating the processing coefficients for processing the original image according to the block division method includes: for each image block of the original image, determining the input pixel neighborhood size corresponding to each output pixel; and determining the processing coefficients for each output pixel according to the input pixel neighborhood size.

[0013] In the above implementation process, based on the scaling ratio, for each block in the original image, the size of its corresponding input pixel neighborhood is analyzed during the scaling process. The pixel neighborhood size refers to the set of pixels in the original image that will affect the calculation of each pixel value in that block. For each output pixel, processing coefficients are calculated based on its corresponding input pixel neighborhood size. These coefficients reflect the degree of contribution of each input pixel in the neighborhood to the output pixel value.

[0014] In one optional implementation, determining the corresponding input pixel neighborhood size for each image block of the original image includes: when the image scaling ratio is determined to be a row scaling ratio, determining the pixel neighborhood size corresponding to the first row pixel output as the pixel size of the first row; for pixels not in the first row, limiting the range of their corresponding pixel neighborhood size to [ceiling((m-1)×RIN / ROUT),ceiling(m×RIN / ROUT)]; when the image scaling ratio is determined to be a row scaling ratio, determining the range of the pixel neighborhood size corresponding to the first row pixel output to [1,ceiling(RIN / ROUT)]; for pixels not in the first row, limiting the range of their corresponding pixel neighborhood size to [ceiling((m-1)×RIN / ROUT),ceiling(m×RIN / ROUT)]; where ceiling(x) represents rounding up x, and m is the index of the current row in the target image.

[0015] In the above implementation, if the image scaling ratio is determined as the row scaling ratio, then for the first row of pixels in the output image, the corresponding input pixel neighborhood size is the same as the pixel size of the first row of pixels in the input image. This means that the first row of pixels in the output image is directly mapped to the first row of pixels in the input image. For pixels in the output image that are not in the first row, the corresponding input pixel neighborhood size is limited to [ceiling((m-1)×RIN / ROUT),ceiling(m×RIN / ROUT)]. Here, m is the index of the current row in the output image. If the image scaling ratio is determined as the row scaling ratio, then for the first row of pixels in the output image, the corresponding input pixel neighborhood size ranges from the first row of pixels to the ceiling(RIN / ROUT) row of pixels. Here, RIN is the row number of the original image, ROUT is the row number of the target image, and ceiling(x) represents rounding up x. Similarly, for pixels in the output image that are not in the first row, the size of their corresponding input pixel neighborhood is also limited to [ceiling((m-1)×RIN / ROUT),ceiling(m×RIN / ROUT)]. Having determined the input neighborhood size for each output pixel, processing coefficients can be calculated. These coefficients determine how the input pixels affect the output pixel values. Using the calculated coefficients, pixels in each block are processed to generate new pixel values.

[0016] In one optional implementation, for each image block of the original image, determining the corresponding input pixel neighborhood size includes: when the image scaling ratio is determined to be a column scaling ratio, determining the pixel neighborhood size corresponding to the output of the first column pixel to be the pixel size of the first column; for pixels not in the first column, limiting the range of their corresponding pixel neighborhood size to [ceiling((n-1)×CIN / COUT),ceiling(n×CIN / COUT)]; when the image scaling ratio is determined to be a column scaling ratio, determining the range of the pixel neighborhood size corresponding to the output of the first column pixel to be [1,ceiling(CIN / COUT)]; for pixels not in the first column, limiting the range of their corresponding pixel neighborhood size to [ceiling((n-1)×CIN / COUT),ceiling(n×CIN / COUT)]; where n is the index of the current column in the target image.

[0017] In the above implementation, if the image scaling ratio is determined as a column magnification ratio, then for the first column of pixels in the output image, the corresponding input pixel neighborhood size is the same as the first column of pixels in the input image. That is, the first column of pixels in the output image is directly mapped to the first column of pixels in the input image. For pixels in the output image that are not in the first column, the corresponding input pixel neighborhood size is limited to [ceiling((n-1)×CIN / COUT),ceiling(n×CIN / COUT)], where n is the index of the current column in the output image. If the image scaling ratio is determined as a column reduction ratio, then for the first column of pixels in the output image, the corresponding input pixel neighborhood size ranges from the first column of pixels to the ceiling(CIN / COUT)th column of pixels. Here, CIN is the number of columns in the original image, COUT is the number of columns in the target image, and ceiling(x) represents rounding up x.

[0018] In one optional implementation, determining the processing coefficient of each output pixel based on the size of the input pixel neighborhood includes: calculating the relative position of the output pixel to each input pixel in the neighborhood, given the size of the input pixel neighborhood; and determining the processing coefficient based on the relative position.

[0019] In the above implementation process, if the output image is larger than the input image, it's necessary to determine which pixels in the input image each output pixel should cover. For example, if a row in the output image is twice the length of a row in the input image, then each output pixel will correspond to two pixels in the input image. Similarly, if the output image is smaller than the input image, it's also necessary to determine which pixels in the input image each output pixel should cover. For example, if a row in the output image is half the length of a row in the input image, then every two pixels in the input image will correspond to one pixel in the output image. For each output pixel, we need to know its specific location in the neighborhood of the input pixels. Knowing the location of the output pixel in the input image allows us to calculate the contribution of each input pixel to the output pixel value, i.e., the processing coefficient. That is, assigning a weight to each pixel in the input image indicates how much influence each pixel has on the final output pixel value.

[0020] In one optional implementation, image processing is performed on the original image according to the processing coefficients to obtain a target image, including: applying the processing coefficients to the original image; for each output pixel, summing all pixel values ​​in its neighborhood that are weighted according to the processing coefficients to obtain the value of the output pixel; until all pixels in the output image have been calculated to obtain the target image.

[0021] In the above implementation process, for each pixel in the output image, its corresponding processing coefficient has been calculated, representing the contribution of each neighboring pixel in the input image to the output pixel value. The processing coefficients are applied to the original image, and for each output pixel, the weighted pixel values ​​of its neighbors are summed to obtain the value of that output pixel. This process is repeated for all pixels in the output image, ensuring that the value of each output pixel is calculated using the weighted summation method described above, until all pixels in the output image have been calculated, resulting in the processed complete image.

[0022] Secondly, embodiments of this disclosure also provide an image processing apparatus, the apparatus comprising: an image input module, a scaling ratio determination module, an image scaling module, and an image output module; the image input module is configured to receive original image data; the scaling ratio determination module is configured to determine the image scaling ratio of the original image; the image scaling module is configured to determine the block division method of the original image according to the image scaling ratio; calculate the processing coefficients for processing the original image according to the block division method; perform image processing on the original image according to the processing coefficients; and the image output module is configured to output a target image.

[0023] In the above implementation process, the image input module receives the raw image data and passes it to the scaling ratio determination module. The scaling ratio determination module calculates the scaling ratio of the image and passes the result to the image scaling module. The image scaling module divides the raw image into blocks according to the scaling ratio and calculates processing coefficients for each block. Using the calculated processing coefficients, the image scaling module processes each image block to generate new pixel values. The target image blocks are reassembled into a complete image, which is then output by the image output module.

[0024] The image input module receives the raw image data provided by the user. The scaling determination module determines the image scaling ratio, including row and column scaling ratios. The image scaling module processes the raw image according to the scaling ratio, determining the image's block size based on the scaling ratio, with each block being the smallest unit size determined by the scaling ratio. Based on this, processing coefficients are calculated for each image block, and these coefficients are applied to each block to generate new pixel values. The image output module outputs the target image.

[0025] Thirdly, embodiments of this disclosure also provide a computer-readable storage medium storing computer program instructions, which, when read and executed by a processor, perform the steps in any of the above implementations.

[0026] In the above implementation process, the computer-readable storage medium can be any form of physical storage device, such as a hard disk, solid-state drive, optical disk, USB flash drive, etc. This storage medium stores computer program instructions, which specifically implement each step of the image processing method described above, including image input, scaling determination, image block processing, processing coefficient calculation, image processing execution, and image output. When the processor (such as the CPU) reads the computer program instructions from the storage medium, these instructions are loaded into memory. The program instructions loaded into memory are executed by the processor, starting to process the image according to the preset algorithm steps. The program first receives raw image data through the image input module. This data can be uploaded by the user through an interface or obtained directly from other data sources. Next, the program runs the scaling determination module to calculate the scaling ratio of the image, including the row and column ratios. Based on the determined scaling ratio, the program executes the image scaling module to determine the block division method and calculate the processing coefficient for each block. The program uses the calculated processing coefficients to process each block of the original image, generating new pixel values. The processed blocks are reassembled into a complete image and output through the image output module, which can either display it on the screen or save it as a file. Attached Figure Description

[0027] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments of this disclosure will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this disclosure and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 is a flowchart of the image processing method provided in an embodiment of this disclosure;

[0029] Figure 2 is an overall flowchart of the image processing method provided in the embodiments of this disclosure;

[0030] Figure 3 is a schematic diagram of a minimal repeating unit provided in an embodiment of this disclosure;

[0031] Figure 4 is a schematic diagram of the processing of the smallest repeating unit using processing coefficients according to an embodiment of this disclosure;

[0032] Figure 5 shows the case where the image scaling ratio is determined as the line magnification ratio according to an embodiment of this disclosure;

[0033] Figure 6 shows the case where the image scaling ratio is determined to be the row scaling ratio according to the embodiment of this disclosure;

[0034] Figure 7 shows the case where the image scaling ratio is determined as the column magnification ratio according to the embodiments of this disclosure;

[0035] Figure 8 shows the case where the image scaling ratio is determined as the column scaling ratio according to the embodiments of this disclosure;

[0036] Figure 9 is a schematic diagram of the image processing apparatus provided in an embodiment of this disclosure;

[0037] Figure 10 is a schematic diagram of a computer-readable medium provided in an embodiment of this disclosure.

[0038] Icons: 11-Image input module; 20-Scale ratio determination module; 30-Image scaling module; 40-Image output module; 100-Electronic device; 111-Memory; 112-Memory controller; 113-Processor; 114-Peripheral interface; 115-Input / output unit; 116-Display unit. Detailed Implementation

[0039] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of the embodiments of this disclosure.

[0040] Please refer to Figure 1, which is a flowchart of an image processing method provided in an embodiment of this disclosure.

[0041] This disclosure provides an image processing method for processing original image data based on the requirements of a target image. The method includes: determining an image scaling ratio based on the original image data; determining a block division method for the original image based on the image scaling ratio; calculating processing coefficients for processing the original image based on the block division method; and performing image processing on the original image based on the processing coefficients to obtain a target image.

[0042] In the above implementation process, the scaling ratios in the horizontal and vertical directions are calculated based on the requirements of the target image and the size of the original image. Horizontally, the column scaling ratio (col_ratio_in, col_ratio_out) is calculated, which is the ratio of the number of columns in the original image to the number of columns in the target image. Vertically, the row scaling ratio (row_ratio_in, row_ratio_out) is calculated, which is the ratio of the number of rows in the original image to the number of rows in the target image. Based on the scaling ratios, an appropriate partitioning strategy is selected, and the size of each block is determined. The image partitioning method is determined according to the row and column scaling ratios. For example, if the row scaling ratio is 3:4 and the column scaling ratio is 3:4, a 3x3 input data block might correspond to a 4x4 output data block. The size of each block is determined; for example, if the scaling ratio is 3:4, each block might contain 3 rows and 3 columns of pixels. For each block, the corresponding processing coefficients are calculated. The processing coefficients are calculated based on the position and scaling ratio of each pixel within the block. This involves determining the range of input pixels that each output pixel depends on and calculating the weights of these input pixels. The calculated processing coefficients are applied to each block, and new pixel values ​​are generated through specific operations. All processed blocks are then recombined according to the structure of the original image to form the final scaled image.

[0043] Please refer to Figure 2, which is an overall flowchart of the image processing method provided in the embodiments of this disclosure.

[0044] Optionally, the image scaling ratio is determined based on the original image data, including: the image scaling ratio is determined according to the ratio of the original image data size to the target image; wherein, the image scaling ratio is divided into row scaling ratio and column scaling ratio; the row scaling ratio is RIN:ROUT; the column scaling ratio is CIN:COUT; the row scaling ratio is configured to be based on scaling in the row direction of the original image, and the column scaling ratio is configured to be based on scaling in the column direction of the original image, wherein scaling includes enlarging and / or reducing, RIN is the number of rows of the original image, ROUT is the number of rows of the target image, CIN is the number of columns of the original image, and COUT is the number of columns of the target image.

[0045] In the above implementation process, the ratio of the size of the original image data to the target image is determined based on user requirements or automatic recognition. Among them, the magnification / shrinkage ratio in the row direction is denoted as RIN = row_ratio_in, ROUT = row_ratio_out, and the magnification / shrinkage ratio in the column direction is denoted as CIN = col_ratio_in, COUT = col_ratio_out. Reducing the ratio to the simplest form is more beneficial for circuit implementation. The scaling ratios of rows and columns are configured separately to achieve independent scaling in the vertical and horizontal directions. If the size of the target image is larger than the original image, the algorithm needs to increase pixels through techniques such as interpolation to achieve image magnification. If the size of the target image is smaller than the original image, the algorithm needs to reduce pixels to achieve image shrinkage while trying to retain important information. The row scaling ratio (RIN:ROUT) calculates the ratio between the number of rows of the original image and the number of rows of the target image. This ratio represents the scaling degree in the vertical direction and can be magnification (ROUT > RIN) or shrinkage (ROUT < RIN). The column scaling ratio (CIN:COUT) calculates the ratio between the number of columns of the original image and the number of columns of the target image. This ratio represents the scaling degree in the horizontal direction and can be magnification (COUT > CIN) or shrinkage (COUT < CIN).

[0046] Optionally, according to the image scaling ratio, determine the way of dividing the original image into blocks, including: determining the smallest repeating unit according to the image scaling ratio; where the smallest repeating unit is the smallest size divisible by the row scaling ratio or the column scaling ratio; dividing the input original image into multiple image blocks according to the smallest repeating unit, and the size of each small block is equal to the size of the smallest repeating unit.

[0047] In the above implementation process, according to the image scaling ratio, determine the smallest repeating unit used when processing the image. The smallest repeating unit refers to the smallest size that can be divisible by the row scaling ratio (RIN:ROUT) or the column scaling ratio (CIN:COUT). This unit will be used as the basis for dividing the image into blocks, so as to maintain the consistency of the ratio during the scaling process. Divide the original image into multiple small blocks according to the determined smallest repeating unit, and the size of each small block is equal to the size of the smallest repeating unit. In this way, during the scaling process, each small block can be evenly processed. By reasonably dividing the blocks, the algorithm can effectively utilize hardware resources, reduce computational complexity, and improve the speed and effect of image processing.

[0048] Optionally, according to the way of dividing the blocks, calculate the processing coefficients for processing the original image, including: for each image block of the original image, determine the corresponding size of the input pixel neighborhood; according to the size of the input pixel neighborhood, determine the processing coefficient for each output pixel.

[0049] In the above implementation process, for each image patch of the original image, the size of the input pixel neighborhood that each output pixel depends on is determined. If the output pixel is located in the m-th row and n-th column, its corresponding input neighborhood can be calculated by scaling. For the row direction, the start and end rows of the neighborhood can be calculated using the formulas start_row = ceiling((m-1) × RIN / ROUT) and end_row = ceiling(m × RIN / ROUT), and the same applies to the column direction. Based on the input pixel neighborhood size, for each output pixel, the corresponding processing coefficient is calculated according to its position in the input neighborhood. The row / column processing coefficients are denoted as row_filter / col_filter, respectively, to reflect the contribution of the input pixel to the output pixel value. The calculation of the processing coefficients can be based on various methods, such as nearest neighbor interpolation, bilinear interpolation, or more advanced interpolation methods. For example, in bilinear interpolation, the weight of each input pixel may be a function of its distance to the output pixel. The calculated processing coefficients are applied to each image patch to generate new pixel values. For each output pixel in each image block, the value of the output pixel is determined by weighted averaging or other specified calculation methods using its corresponding processing coefficient and the pixel values ​​in the input neighborhood.

[0050] Optionally, the processing coefficients for each output pixel are determined based on the size of the input pixel neighborhood, including: calculating the relative position of the output pixel to each input pixel in the neighborhood, given the size of the input pixel neighborhood; and determining the processing coefficients based on the relative positions.

[0051] In the above implementation process, after determining the size of the input pixel neighborhood, for each output pixel, its position relative to each input pixel within the neighborhood is calculated based on its coordinates in the original image and the size of the input neighborhood. Processing coefficients are typically determined based on the relative positional relationship between the input pixel neighborhood and the output pixel. There are various calculation methods, such as calculating the proportion in the row / column direction or the area ratio to determine the filter coefficients for each neighboring pixel. For example, in bilinear interpolation, the processing coefficients are calculated based on the distance between the output and input pixels. The closer the distance, the greater the contribution of the input pixel to the output pixel. Different interpolation methods can be used to calculate the processing coefficients, such as nearest neighbor interpolation, bilinear interpolation, and bicubic interpolation. Each method has its specific weighting method. Data processing can also be performed by transforming the data to another space, such as YUV space or linear space.

[0052] Optionally, image processing is performed on the original image according to the processing coefficients to obtain the target image, including: applying the processing coefficients to the original image, and for each output pixel, summing all the pixel values ​​in its neighborhood that are weighted according to the processing coefficients to obtain the value of the output pixel; until all pixels in the output image have been calculated to obtain the target image.

[0053] In the above implementation process, for each output pixel, based on its processing coefficient and the input pixel values ​​in its neighborhood, a weighted sum of pixel values ​​is calculated by multiplying each input pixel value by its corresponding processing coefficient. This weighted summation operation is repeated for each pixel position in the output image until the values ​​of all output pixels have been calculated. Once all pixel values ​​in the output image have been calculated, the final processed image is obtained.

[0054] Please refer to Figures 3 and 4. Figure 3 is a schematic diagram of a minimum repeating unit provided in an embodiment of this disclosure; Figure 4 is a schematic diagram of processing the minimum repeating unit with processing coefficients provided in an embodiment of this disclosure.

[0055] In one embodiment of this disclosure, a row / column scaling algorithm is used as an example. Assuming the input image resolution is 810×1800 and the output image resolution is 1080×2400, after simplification, we get row_ratio_in = 3 and row_ratio_out = 4, satisfying 1800:2400 = 3:4. Similarly, we get col_ratio_in = 3 and col_ratio_out = 4. Therefore, we can determine that on a panel of the same size, a 3x3 input data matrix, after the scaling algorithm, will result in a 4x4 output data matrix, which is the smallest repeating unit for calculating the processing coefficients. The blue boxes represent input data, and the red dashed boxes represent output data. Taking the output pixel (purple box) in the second row and second column of the output data matrix as an example, we determine its corresponding input pixel neighborhood and calculate its corresponding processing coefficient. Based on the formulas [ceiling((m-1)×RIN / ROUT),ceiling(m×RIN / ROUT)], the corresponding input row neighborhood is calculated as [1,2] and the column neighborhood is calculated as [1,2] based on the current output pixel's row and column information. The output pixel in the second row and second column, along with its associated input pixel region, is drawn separately. Next, the processing coefficients corresponding to the current output pixel are determined. Based on the row and column ratio, the up_row_para and left_col_para occupied by the current output pixel on the D11 input pixel can be determined. Assuming the output pixel's side length is 1 unit, the input pixel's side length is 4 / 3, so up_row_para = 1 / 3, left_col_para = 1 / 3. Similarly, down_row_para = 2 / 3, right_col_para = 2 / 3. Therefore, the row processing coefficients corresponding to the current input pixel are row_filter = [up_row_para, down_row_para] = [1 / 3; 2 / 3], and the column processing coefficients are col_filter = [left_col_para, right_col_para] = [1 / 3, 2 / 3]. Multiplying the row and column processing coefficients yields the processing coefficient matrix filter = row_filter × col_filter = [1 / 9, 2 / 9; 2 / 9, 4 / 9]. Then, the current output pixel value value = filter × [Din1, Din2; Din3, Din4] = 1 / 9 × Din1 + 2 / 9 × Din2 + 2 / 9 × Din3 + 4 / 9 × Din4.

[0056] In one embodiment of this disclosure, special handling may be required at the edges of image patches because the neighborhood may extend beyond the image boundary. In such cases, edge extension, mirroring, or other techniques can be used to handle these boundary situations.

[0057] Please refer to Figures 5 and 6. Figure 5 shows the case where the image scaling ratio is determined as the line magnification ratio according to an embodiment of this disclosure; Figure 6 shows the case where the image scaling ratio is determined as the line reduction ratio according to an embodiment of this disclosure.

[0058] Optionally, for each image block of the original image, the corresponding input pixel neighborhood size is determined, including: when the image scaling ratio is determined to be the row scaling ratio, the pixel neighborhood size corresponding to the output of the first row of pixels is determined to be the pixel size of the first row; for pixels not in the first row, the range of their corresponding pixel neighborhood size is limited to [ceiling((m-1)×RIN / ROUT),ceiling(m×RIN / ROUT)]; when the image scaling ratio is determined to be the row scaling ratio, the range of the pixel neighborhood size corresponding to the output of the first row of pixels is determined to be [1,ceiling(RIN / ROUT)]; for pixels not in the first row, the range of their corresponding pixel neighborhood size is limited to [ceiling((m-1)×RIN / ROUT),ceiling(m×RIN / ROUT)]; where ceiling(x) represents rounding up x, and m is the index of the current row in the target image.

[0059] In the above implementation process, under the line magnification ratio, the size of the pixel neighborhood corresponding to the output of the first row of pixels is determined. Under the line magnification ratio, the neighborhood size of the first row of pixels directly corresponds to the pixel size of the first row. That is, each output pixel in the first row will be directly associated with the corresponding pixel in the first row of the input image. For each pixel not in the first row, its corresponding input neighborhood range is calculated using the formula [ceiling((m-1)×RIN / ROUT),ceiling(m×RIN / ROUT)], where m is the row index of the current output pixel, RIN is the number of rows in the original image, ROUT is the number of rows in the target image, and ceiling(x) represents rounding up x. Under the line reduction ratio, the size range of the pixel neighborhood corresponding to the output of the first row of pixels is determined. Under the line reduction ratio, the size range of the neighborhood of the first row of pixels is determined to be [1,ceiling(RIN / ROUT)], that is, each output pixel in the first row will be associated with a smaller pixel range in the input image. The size range of the pixel neighborhood corresponding to pixels not in the first row is limited. For each pixel not in the first row, its corresponding input neighborhood range is also calculated using the formula [ceiling((m-1)×RIN / ROUT),ceiling(m×RIN / ROUT)].

[0060] Please refer to Figures 7 and 8. Figure 7 shows the case where the image scaling ratio is determined as a column magnification ratio according to an embodiment of this disclosure; Figure 8 shows the case where the image scaling ratio is determined as a column reduction ratio according to an embodiment of this disclosure.

[0061] Optionally, for each image block of the original image, the corresponding input pixel neighborhood size is determined, including: when the image scaling ratio is determined to be the column scaling ratio, the pixel neighborhood size corresponding to the output of the first column pixel is determined to be the pixel size of the first column; for pixels not in the first column, the range of their corresponding pixel neighborhood size is limited to [ceiling((n-1)×CIN / COUT),ceiling(n×CIN / COUT)]; when the image scaling ratio is determined to be the column scaling ratio, the range of the pixel neighborhood size corresponding to the output of the first column pixel is determined to be [1,ceiling(CIN / COUT)]; for pixels not in the first column, the range of their corresponding pixel neighborhood size is limited to [ceiling((n-1)×CIN / COUT),ceiling(n×CIN / COUT)]; where n is the index of the current column in the target image.

[0062] In the above implementation process, under the column magnification ratio, the pixel neighborhood size corresponding to the output of the first column of pixels is determined. Under the column magnification ratio, the neighborhood size of the first column of pixels directly corresponds to the pixel size of the first column, that is, each output pixel of the first column will be directly associated with the corresponding pixel of the first column of the input image. For each pixel other than the first column, its corresponding input neighborhood range is calculated by the formula [ceiling((n-1)×CIN / COUT),ceiling(n×CIN / COUT)], where n is the column index of the current output pixel, CIN is the number of columns in the original image, COUT is the number of columns in the target image, and ceiling(x) represents rounding up x. Under the column reduction ratio, the pixel neighborhood size range corresponding to the output of the first column of pixels is determined. Under the column reduction ratio, the neighborhood size range of the first column of pixels is determined to be [1,ceiling(CIN / COUT)], that is, each output pixel of the first column will be associated with a smaller pixel range of the input image. For each pixel not in the first column, its corresponding input neighborhood range is also calculated using the formula [ceiling((n-1)×CIN / COUT),ceiling(n×CIN / COUT)].

[0063] Please refer to Figure 9, which is a schematic diagram of an image processing apparatus provided in an embodiment of this disclosure.

[0064] This disclosure also provides an image processing apparatus, comprising: an image input module 11, a scaling ratio determination module 20, an image scaling module 30, and an image output module 40; the image input module 11 is configured to receive raw image data; the scaling ratio determination module 20 is configured to determine the image scaling ratio of the raw image; the image scaling module 30 is configured to determine the block division method of the raw image according to the image scaling ratio; calculate the processing coefficients for processing the raw image according to the block division method; perform image processing on the raw image according to the processing coefficients; and the image output module 40 is configured to output a target image.

[0065] In the above implementation process, the image input module 11 is responsible for receiving the original image data provided by the user. The scaling ratio determination module 20 determines the scaling ratio of the image, including the row scaling ratio and the column scaling ratio. The image scaling module 30 is responsible for processing the original image according to the scaling ratio, determining the block method of the original image according to the scaling ratio, and the size of each block is the smallest unit size determined according to the scaling ratio. Based on this, processing coefficients are calculated for each image block, and the processing coefficients are applied to process each image block to generate new pixel values; the image output module 40 is used to output the target image.

[0066] Optionally, please refer to Figure 10, which is a block diagram of a computer-readable storage medium provided in an embodiment of this disclosure.

[0067] This disclosure also provides a computer-readable storage medium storing computer program instructions. When the computer program instructions are read and executed by a processor, the steps in any of the above implementations are performed.

[0068] In the above implementation process, the computer-readable storage medium can be any form of physical storage device, such as a hard disk, solid-state drive, optical disk, USB flash drive, etc. This storage medium stores computer program instructions, which specifically implement each step of the image processing method described above, including image input, scaling determination, image block processing, processing coefficient calculation, image processing execution, and image output. When the processor (such as the CPU) reads the computer program instructions from the storage medium, these instructions are loaded into memory. The program instructions loaded into memory are executed by the processor, starting to process the image according to the preset algorithm steps. The program first receives raw image data through the image input module. This data can be uploaded by the user through an interface or obtained directly from other data sources. Next, the program runs the scaling determination module to calculate the scaling ratio of the image, including the row and column ratios. Based on the determined scaling ratio, the program executes the image scaling module to determine the block division method and calculate the processing coefficient for each block. The program uses the calculated processing coefficients to process each block of the original image, generating new pixel values. The processed segments are reassembled into a complete image and output through the image output module, which can either display it on a screen or save it as a file. Electronic device 100 may include a memory 111, a memory controller 112, a processor 113, a peripheral interface 114, an input / output unit 115, and a display unit 116. Those skilled in the art will understand that the structure shown in FIG10 is merely illustrative and does not limit the structure of electronic device 100. For example, electronic device 100 may also include more or fewer components than shown in FIG8, or have a different configuration than that shown in FIG8.

[0069] The aforementioned memory 111, storage controller 112, processor 113, peripheral interface 114, input / output unit 115, and display unit 116 are electrically connected directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The aforementioned processor 113 is configured to execute executable modules stored in the memory. The memory 111 can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory 111 is configured to store a program. After receiving an execution instruction, the processor 113 executes the program. The method executed by the electronic device 100, which is defined by the process disclosed in any embodiment of this disclosure, can be applied to the processor 113 or implemented by the processor 113.

[0070] The processor 113 described above may be an integrated circuit chip with signal processing capabilities. The processor 113 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a digital signal processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this disclosure. The general-purpose processor may be a microprocessor or any conventional processor.

[0071] The peripheral interface 114 described above couples various input / output devices to the processor 113 and the memory 111. In some embodiments, the peripheral interface 114, the processor 113, and the memory controller 112 can be implemented on a single chip. In other instances, they can be implemented on separate chips.

[0072] The input / output unit 115 described above is configured to provide user input data. The input / output unit 115 may be, but is not limited to, a mouse and keyboard, etc.

[0073] The aforementioned display unit 116 provides an interactive interface (e.g., a user interface) between the electronic device 100 and the user, or displays image data for the user's reference. In this embodiment, the display unit can be a liquid crystal display (LCD) or a touch display. If it is a touch display, it can be a capacitive touchscreen or a resistive touchscreen that supports single-point and multi-point touch operations. Supporting single-point and multi-point touch operations means that the touch display can sense touch operations generated simultaneously from one or more locations on the touch display and pass the sensed touch operations to the processor for calculation and processing. In this embodiment, the display unit 116 can display input and / or output images.

[0074] In summary, it should be understood from the several embodiments provided in this disclosure that the disclosed device can also be implemented in other ways. The device embodiments described above are merely illustrative; for example, the block diagrams in the accompanying drawings illustrate the possible architecture, functions, and operations of the device according to various embodiments of this disclosure. In this regard, each block in the block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram, and combinations of block diagrams, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0075] In addition, the functional modules in the various embodiments of this disclosure can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0076] The above description is merely an embodiment of this disclosure and is not intended to limit the scope of protection of this disclosure. Various modifications and variations can be made to this disclosure by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure. It should be noted that similar reference numerals and letters in the following figures denote similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0077] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element. Industrial applicability

[0078] The above scheme determines the image scaling ratio based on the original image data; it then determines the image segmentation method based on the scaling ratio; finally, it calculates the processing coefficients based on the segmentation method; and finally, it processes the original image using these coefficients to obtain the target image. By calculating the image scaling ratio and the corresponding processing coefficients, high-quality image scaling is achieved, maintaining detail quality regardless of whether the image is enlarged or reduced. This image processing method can be widely applied to various scenarios requiring image scaling, such as video editing, image enhancement, and surveillance systems. By dynamically processing the image according to the scaling ratio, it is more flexible and adjustable than traditional methods, while also meeting the requirements of real-time processing.

Claims

1. An image processing method, characterized in that, The method is used to process raw image data based on the requirements of the target image, and the method includes: Based on the original image data, determine the image scaling ratio; The segmentation method of the original image is determined based on the image scaling ratio; Based on the block division method, calculate the processing coefficients for processing the original image; Based on the processing coefficients, the original image is processed to obtain the target image.

2. The method according to claim 1, characterized in that, Based on the original image data, determine the image scaling ratio, including: The image scaling ratio is determined based on the ratio of the original image data size to the target image. The image scaling ratio is divided into row scaling ratio and column scaling ratio; the row scaling ratio is RIN:ROUT; the column scaling ratio is CIN:COUT; the row scaling ratio is configured based on scaling along the row direction of the original image, and the column scaling ratio is configured based on scaling along the column direction of the original image. Scaling includes enlarging and / or reducing, RIN is the number of rows in the original image, ROUT is the number of rows in the target image, CIN is the number of columns in the original image, and COUT is the number of columns in the target image.

3. The method according to claim 2, characterized in that, Determining the block division method of the original image based on the image scaling ratio includes: The smallest repeating unit is determined based on the image scaling ratio; wherein, the smallest repeating unit is the smallest size that is divisible by the row scaling ratio or the column scaling ratio. Based on the minimum repeating unit, the input original image is divided into multiple image blocks, and the size of each block is equal to the size of the minimum repeating unit.

4. The method according to claim 3, characterized in that, The step of calculating the processing coefficients for the original image according to the block division method includes: For each image block of the original image, determine the size of the input pixel neighborhood corresponding to each output pixel; The processing coefficient for each output pixel is determined based on the size of the input pixel neighborhood.

5. The method according to claim 4, characterized in that, Determining the corresponding input pixel neighborhood size for each image block of the original image includes: When the image scaling ratio is determined to be the row scaling ratio, the size of the pixel neighborhood corresponding to the first row pixel output is determined to be the pixel size of the first row; For pixels not in the first row, the size range of their corresponding pixel neighborhood is limited to [ceiling((m-1)×RIN / ROUT),ceiling(m×RIN / ROUT)]; When the image scaling ratio is determined to be the row scaling ratio, the size range of the pixel neighborhood corresponding to the first row pixel output is determined to be [1,ceiling(RIN / ROUT)]; For pixels not in the first row, the size range of their corresponding pixel neighborhood is limited to [ceiling((m-1)×RIN / ROUT),ceiling(m×RIN / ROUT)]; Where ceiling(x) represents rounding up x, and m is the index of the current row in the target image.

6. The method according to claim 4, characterized in that, For each image block of the original image, determine the corresponding input pixel neighborhood size, including: When the image scaling ratio is determined to be the column scaling ratio, the pixel neighborhood size corresponding to the first column pixel output is determined to be the pixel size of the first column; For pixels not in the first column, the size range of their corresponding pixel neighborhood is limited to [ceiling((n-1)×CIN / COUT),ceiling(n×CIN / COUT)]; When the image scaling ratio is determined to be a column scaling ratio, the size range of the pixel neighborhood corresponding to the first column pixel output is determined to be [1,ceiling(CIN / COUT)]; For pixels not in the first column, the size range of their corresponding pixel neighborhood is limited to [ceiling((n-1)×CIN / COUT),ceiling(n×CIN / COUT)]; Where n is the index of the current column in the target image.

7. The method according to claim 4, characterized in that, Based on the input pixel neighborhood size, the processing coefficients for each output pixel are determined, including: Given a determined input pixel neighborhood size, the relative position of the output pixel to each input pixel within the neighborhood is calculated; the processing coefficient is then determined based on the relative position.

8. The method according to claim 7, characterized in that, Based on the processing coefficients, image processing is performed on the original image to obtain the target image, including: The processing coefficients are applied to the original image. For each output pixel, the weighted pixel values ​​in its neighborhood are summed according to the processing coefficients to obtain the value of the output pixel. This process continues until all pixels in the image have been calculated to obtain the target image.

9. An image processing apparatus, characterized in that, The device includes: an image input module, a scaling ratio determination module, an image scaling module, and an image output module; The image input module is configured to receive raw image data; the scaling ratio determination module is configured to determine the image scaling ratio of the raw image. The image scaling module is configured to determine the block division method of the original image according to the image scaling ratio; calculate the processing coefficients of the original image according to the block division method; and perform image processing on the original image according to the processing coefficients. The image output module is configured to output the target image.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a processor, perform the steps of the method according to any one of claims 1-8.

Images