Image processing method, system, electronic device and medium

Abstract

An image processing method, system, electronic device and medium are disclosed, wherein the image processing method comprises the following steps: dividing a to-be-processed image into a plurality of image blocks; performing a shaping FFT transformation on the image blocks to obtain image block frequency domain data; performing frequency domain processing on the image block frequency domain data to obtain image block frequency domain processed data; performing an IFFT transformation on the image block frequency domain processed data to obtain image block IFFT transformation results; and obtaining a result image according to the image block IFFT transformation results corresponding to each image block. The present application effectively improves the parallelism of the operation and improves the processing speed. Furthermore, through reasonable setting of the data bit width in the operation process, the operation accuracy is ensured while the resources are effectively saved.

Description

[0001] This invention is a divisional application of Chinese patent application filed on June 28, 2020, with application number 2020105982401 and title "Image Processing Method, System, Electronic Device and Medium". Technical Field

[0002] This invention belongs to the field of image processing technology, and particularly relates to an image processing method, system, electronic device, and medium. Background Technology

[0003] The most common fast implementation method of Fourier transform is the FFT (Fast Fourier Transform) algorithm invented by Cooley and Tukey. The basic idea of ​​FFT is to decompose the original N-point sequence into two or more shorter sequences and recombine them into the DFT (Discrete Fourier Transform) of the original sequence. This makes the number of operations of the indirect algorithm much less than the number of operations of the direct DFT, thereby improving the computation speed of DFT.

[0004] On the one hand, the decomposition operations of FFT / IFFT (Inverse Fast Fourier Transform) can be basically divided into two categories: one is to decompose the time series x(n) successively, called the Decimation in Time (DIT) algorithm; the other is to decompose the Fourier transform sequence X(k), called the Decimation in Frequency (DIF) algorithm. Based on the number M of short sequences decomposed by the FFT, FFT / IFFT algorithms can be divided into several types, generally called radix-M algorithms. For example, the radix-2 FFT algorithm can decompose an N-point DFT operation into N / 2*log2N 2-point DFT operations; the radix-4 FFT algorithm can decompose an N-point DFT operation into N / 4*log4N 4-point DFT operations.

[0005] Currently, there are many optimizations for FFT processors, primarily focusing on optimizing the hardware implementation structure to reduce hardware resource consumption, power consumption, and processing speed. Compared to hardware-based FFT processors, traditional FFT software runs slowly.

[0006] The SRFFT (Split Basis FFT) algorithm is a typical example of FFT optimization, which significantly reduces the number of multiplications and additions required for FFT computation. However, despite this, the serial processing time is still insufficient for practical applications when multiple FFT operations need to be performed.

[0007] Due to their time-consuming and computationally complex nature, image processing algorithms developed for product applications are mostly based on spatial pixel domain processing, rarely involving frequency domain processing. The reason is not the inherent complexity of frequency domain algorithms, but rather the excessive time consumption of the two Fourier transforms and inverse transforms involved in time-frequency spatial transformation. To address this issue, many algorithms utilize the characteristics of time-frequency space to bypass frequency domain processing; for example, they use time-domain convolution to replace frequency-domain dot product operations.

[0008] However, this approach tends to fail with adaptive frequency domain filters because the equivalent spatial filter order is too high or too complex to be applied, making it impossible to reasonably replace time domain processing with frequency domain processing. Another solution is to implement the frequency domain processing in hardware to improve speed. However, hardware implementation is constrained by hardware area, has a long development cycle, lacks flexibility, and is unsuitable for rapid development and updates.

[0009] Mobile applications, taking smartphones as an example, emphasize real-time performance and require minimal hardware resource consumption. These conditions largely limit the adoption of frequency domain processing techniques for images, mainly because software-based implementations of time-frequency spatial transformations are too time-consuming and cannot meet the time requirements. Summary of the Invention

[0010] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art in that image processing takes a long time, and to provide an image processing method, system, electronic device and medium.

[0011] The present invention solves the above-mentioned technical problems through the following technical solution:

[0012] This invention provides an image processing method, comprising the following steps:

[0013] The image to be processed is divided into several image blocks;

[0014] The image patch is subjected to a reshaping FFT transform to obtain the frequency domain data of the image patch;

[0015] Perform frequency domain processing on the image block frequency domain data to obtain the image block frequency domain processed data;

[0016] The image patch IFFT transform is performed on the frequency domain processed data to obtain the image patch IFFT transform result;

[0017] The resulting image is obtained from the IFFT transform results of each image block.

[0018] Preferably, the step of dividing the image to be processed into several image blocks includes:

[0019] The image to be processed is divided into several image units. Each image unit consists of adjacent pixels, and no two image units overlap. The image units include a first type of image unit and a second type of image unit. The first type of image unit has N rows and N columns of pixels, where N is an integer power of 2. The second type of image unit is the image unit other than the first type of image unit.

[0020] The second type of image unit is expanded into an expanded unit with N rows and N columns of pixels;

[0021] The first type of image unit and the extended unit are used as image blocks.

[0022] Preferably, the steps for performing a reshaping FFT transform on the image patch include:

[0023] Construct a first integer array and a second integer array. The first integer array includes N first elements, each of which is used to store the real part of the input data. The second integer array includes N second elements, each of which is used to store the imaginary part of the input data.

[0024] Increase the bit width of the numerical portion of the input data to a first target bit width to form expanded data;

[0025] Butterfly operations are performed based on extended data.

[0026] Preferably, N is less than or equal to 64.

[0027] Preferably, the first target bit width Wt = N_in_fft + N_up, where N_in_fft represents the bit width of the numerical part of the input data, N_up = 32 - 1 - N_in_fft - 2 * k, k = log2(N), and k represents the number of butterfly operation stages required for FFT.

[0028] Preferably, the steps for performing butterfly operations based on extended data include:

[0029] The weighting factors used in the butterfly operation are shaped.

[0030] Preferably, the steps for performing butterfly operations based on extended data include:

[0031] Perform butterfly operations first in the row direction, then perform butterfly operations in the column direction;

[0032] Alternatively, perform a butterfly operation first along the column direction, and then perform a butterfly operation along the row direction.

[0033] Preferably, during the butterfly operation in the row direction, the butterfly operations corresponding to each row are performed in parallel; during the butterfly operation in the column direction, the butterfly operations corresponding to each column are performed in parallel.

[0034] Preferably, the steps of performing a shaped FFT transform on the image patch to obtain the image patch frequency domain data include:

[0035] The image blocks are subjected to parallel shaping FFT transformations to obtain the image block frequency domain data corresponding to each image block;

[0036] The steps for performing IFFT transform on the image patch data after frequency domain processing to obtain the image patch IFFT transform result include:

[0037] After frequency domain processing of each image block, the data are subjected to IFFT transformation in parallel to obtain the IFFT transformation result of each image block.

[0038] Preferably, the steps for performing IFFT transform on the image patch frequency domain processed data include:

[0039] The image patch data after frequency domain processing is conjugated to obtain conjugated data.

[0040] Increase the bit width of the numerical portion of the conjugate data to a second target bit width to form extended conjugate data;

[0041] Perform butterfly operations on extended conjugate data.

[0042] Preferably, the second target bit width is N_in_ifft+N_iup, where N_iup=32-2-N_in_ifft, N_in_ifft=N_in+2*k, N_in represents the bit width of the numerical part of the original data of the image block, and k represents the number of butterfly operation stages required by IFFT.

[0043] Preferably, the steps for performing a butterfly operation on conjugate data include:

[0044] Perform butterfly operations first in the row direction, then perform butterfly operations in the column direction.

[0045] Preferably, the steps for performing butterfly operations first, in the row direction, include:

[0046] After each butterfly operation is completed, the result of the butterfly operation in that stage is shifted one bit to the right until all the butterfly operations in k stages are completed; or, the butterfly operation corresponding to each stage is completed in sequence until all the butterfly operations in k stages are completed, and then the result of the butterfly operation is shifted k bits to the right.

[0047] The steps for performing a butterfly operation in the column direction include:

[0048] After each butterfly operation is completed, the result of the butterfly operation in that stage is shifted right by one bit, until all butterfly operations in k stages are completed; or, the butterfly operation corresponding to each stage is completed sequentially, until all butterfly operations in k stages are completed, and then the result of the butterfly operation is shifted right by k bits.

[0049] Preferably, the steps for obtaining the result image based on the IFFT transform results of each image patch include:

[0050] The IFFT transformation results of each image patch are stitched together to form an intermediate image.

[0051] The intermediate image is smoothed and filtered to obtain the final image.

[0052] Preferably, the steps for performing butterfly operations based on extended data include:

[0053] First, butterfly operations are performed in parallel on each row in the row direction. Then, butterfly operations are performed in parallel on several target columns in the column direction to obtain local frequency domain data. Then, based on the local frequency domain data and the conjugate symmetry of the Fourier transform, the frequency domain data of the image block is obtained. The several target columns are the [1, N / 2+1] columns of the image block.

[0054] Alternatively, the steps for performing a butterfly operation based on extended data include:

[0055] First, butterfly operations are performed in parallel on each column in the column direction. Then, butterfly operations are performed in parallel on several target rows in the row direction to obtain local frequency domain data. Then, based on the local frequency domain data, the image block frequency domain data is obtained according to the conjugate symmetry of the Fourier transform. Several target rows are the [1, N / 2+1] rows of the image block.

[0056] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the image processing method of the present invention.

[0057] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the image processing method of the present invention.

[0058] The present invention also provides an image processing system, including a partitioning unit, a shaping FFT transform unit, a frequency domain processing unit, an IFFT transform unit, and a result image generation unit;

[0059] The partitioning unit is used to divide the image to be processed into several image blocks;

[0060] The shaping FFT transform unit is used to perform shaping FFT transform on image blocks to obtain the frequency domain data of the image blocks;

[0061] The frequency domain processing unit is used to perform frequency domain processing on the image block frequency domain data to obtain the image block frequency domain processed data;

[0062] The IFFT transform unit is used to perform IFFT transform on the image patch data after frequency domain processing to obtain the image patch IFFT transform result;

[0063] The result image generation unit is used to obtain the result image based on the IFFT transform results of the image blocks corresponding to each image block.

[0064] Preferably, the partitioning unit is also used to divide the image to be processed into several image units, each image unit consisting of adjacent pixels, and no two image units overlap; the image units include a first type of image unit and a second type of image unit, the first type of image unit having N rows and N columns of pixels, where N is an integer power of 2, and the second type of image unit being image units other than the first type of image unit;

[0065] The partitioning unit is also used to expand the second type of image unit into an expanded unit with N rows and N columns of pixels;

[0066] The partitioning unit is also used to take the first type of image unit and the extended unit as image blocks.

[0067] Preferably, the integer FFT transform unit is also used to construct a first integer array and a second integer array. The first integer array includes N first elements, each of which is used to store the real part of the input data. The second integer array includes N second elements, each of which is used to store the imaginary part of the input data.

[0068] The shaping FFT transform unit is also used to increase the bit width of the numerical part of the input data to a first target bit width to form extended data;

[0069] The shaping FFT transform unit is also used for butterfly operations based on extended data.

[0070] Preferably, N is less than or equal to 64.

[0071] Preferably, the first target bit width Wt = N_in_fft + N_up, where N_in_fft represents the bit width of the numerical part of the input data, N_up = 32 - 1 - N_in_fft - 2 * k, k = log2(N), and k represents the number of butterfly operation stages required for FFT.

[0072] Preferably, the shaping FFT transform unit is also used to shape the weighting factors used in the butterfly operation.

[0073] Preferably, the shaping FFT transform unit is also used to perform butterfly operations first in the row direction and then in the column direction;

[0074] Alternatively, the shaping FFT transform unit can also be used to perform butterfly operations first in the column direction and then in the row direction.

[0075] Preferably, during the butterfly operation in the row direction, the butterfly operations corresponding to each row are performed in parallel; during the butterfly operation in the column direction, the butterfly operations corresponding to each column are performed in parallel.

[0076] Preferably, the shaping FFT transform unit is also used to perform shaping FFT transform on each image block in parallel to obtain the image block frequency domain data corresponding to each image block;

[0077] The IFFT transform unit is also used to perform IFFT transforms on the frequency domain processed data of each image block in parallel to obtain the image block IFFT transform result corresponding to each image block.

[0078] Preferably, the IFFT transform unit is also used to perform conjugate processing on the frequency domain processed data of the image patch to obtain conjugate data;

[0079] The IFFT transform unit is also used to increase the bit width of the numerical part of the conjugate data to a second target bit width to form extended conjugate data;

[0080] The IFFT transform unit is also used to perform butterfly operations on extended conjugate data.

[0081] Preferably, the second target bit width is N_in_ifft+N_iup, where N_iup=32-2-N_in_ifft, N_in_ifft=N_in+2*k, N_in represents the bit width of the numerical part of the original data of the image block, and k represents the number of butterfly operation stages required by IFFT.

[0082] Preferably, the IFFT transform unit is also used to perform butterfly operations first in the row direction and then in the column direction.

[0083] Preferably, when performing butterfly operations in the row direction, the IFFT transform unit is also used to shift the result of the butterfly operation of that stage to the right by one bit after each stage of butterfly operation is completed, until all k stages of butterfly operations have been completed.

[0084] Alternatively, the IFFT transform unit is also used to sequentially complete the butterfly operation corresponding to each stage until the butterfly operation of k stages has been completed, and then the result obtained from the butterfly operation is shifted right by k bits.

[0085] When performing butterfly operations in the column direction, the IFFT transform unit is also used to shift the result of the butterfly operation of that stage to the right by one bit after each stage of butterfly operation is completed, until all k stages of butterfly operations have been completed.

[0086] Alternatively, the IFFT transform unit is also used to sequentially complete the butterfly operation corresponding to each stage until all k stages of butterfly operation have been completed. Then, the result obtained from the butterfly operation is shifted right by k bits.

[0087] Preferably, the resulting image generation unit is also used to stitch together the IFFT transform results of the image blocks corresponding to each image block into an intermediate image;

[0088] The result image generation unit is also used to perform smoothing filtering on the intermediate image to obtain the result image.

[0089] Preferably, the shaping FFT transform unit first performs butterfly operations in parallel on each row in the row direction, and then performs butterfly operations in parallel on several target columns in the column direction to obtain local frequency domain data. Then, based on the local frequency domain data, the image block frequency domain data is obtained according to the conjugate symmetry of the Fourier transform. The several target columns are the [1, N / 2+1] columns of the image block.

[0090] Alternatively, the shaping FFT transform unit first performs butterfly operations in parallel on each column in the column direction, and then performs butterfly operations in parallel on several target rows in the row direction to obtain local frequency domain data. Then, based on the local frequency domain data, the image block frequency domain data is obtained according to the conjugate symmetry of the Fourier transform. Several target rows are the [1, N / 2+1] rows of the image block.

[0091] The positive and progressive effects of this invention are as follows: it effectively improves the parallelism of the operation and increases the processing speed; and by reasonably setting the data bit width during the operation, it effectively saves resources while ensuring the accuracy of the operation. Attached Figure Description

[0092] Figure 1 This is a flowchart of the image processing method of Embodiment 1 of the present invention.

[0093] Figure 2 This is a flowchart of step S2 of the image processing method of Embodiment 1 of the present invention.

[0094] Figure 3 This is a schematic diagram of the structure of the electronic device according to Embodiment 9 of the present invention.

[0095] Figure 4 This is a schematic diagram of the image processing system of Embodiment 11 of the present invention. Detailed Implementation

[0096] The present invention will be further illustrated by way of embodiments below, but the present invention is not limited to the scope of the embodiments described herein.

[0097] Example 1

[0098] This embodiment provides an image processing method. (Refer to...) Figure 1 The image processing method includes the following steps:

[0099] Step S1: Divide the image to be processed into several image blocks.

[0100] As an optional implementation, in step S1, the image to be processed is first divided into several image units. During the division, considering the overall size of the image to be processed and the position of the image units in the image to be processed, two types of image units are formed, wherein the first type of image units has a preset basic size; and the second type of image units does not have a preset basic size.

[0101] In practice, each image unit is composed of adjacent pixels, and the image units do not overlap. The first type of image unit is a square, composed of N rows and N columns of pixels, where N is an integer power of 2.

[0102] To achieve better spectrum control, the preferred range for N is no greater than 64. In one alternative implementation, N is 64; in other alternative implementations, N is less than 64. The first type of image unit is considered as the image block to be processed.

[0103] The second type of image unit is an image unit whose edge or corner region of the image to be processed does not have a preset size during the segmentation process. It is a "residual region" formed after the image to be processed is segmented.

[0104] As an optional implementation, the second type of image unit is expanded into an image block with a preset basic size. In a specific implementation, a corresponding number of pixels are added around the second type of image unit to form an image block with a preset basic size, and the data corresponding to the added pixels is 0.

[0105] Step S2: Perform a reshaping FFT transformation on the image block.

[0106] In practice, the two-dimensional FFT transformation is converted into two one-dimensional FFT transformations, one for each row and one for each column.

[0107] Reference Figure 2 Step S2 includes the following steps:

[0108] Step S21: Construct a first integer array and a second integer array. The first integer array includes N first elements, each of which stores the real part of the input data. The second integer array includes N second elements, each of which stores the imaginary part of the input data. Each element of the first integer array stores the real part data In_r[i] (i∈[1,N]), and each element of the second integer array stores the imaginary part data In_i[i] (i∈[1,N]).

[0109] Taking the butterfly operation along the row direction as an example, the real part data In_r[i] is the real part of the data corresponding to the i-th pixel of the row in the image block, and the imaginary part data In_i[i] is the imaginary part of the data corresponding to the i-th pixel of the row in the image block. The imaginary part of the initial input data is set to 0.

[0110] Taking the butterfly operation along the column direction as an example, the real part data In_r[i] is the real part of the data corresponding to the i-th pixel in the column of the image block, and the imaginary part data In_i[i] is the imaginary part of the data corresponding to the i-th pixel in the column of the image block. The imaginary part of the initial input data is set to 0.

[0111] Step S22: Increase the bit width of the input data to the first target bit width to form extended data. Specifically, increase the bit width of the numerical portion of the input data to N_in_fft + N_up. By increasing the bit width of the numerical portion of the input data, the accuracy of the computation process can be effectively guaranteed. N_in_fft is the bit width of the numerical portion of the original input data. When the original data is represented as a signed number, the bit width of the original input data is N_in_fft + 1, where the highest bit is the sign bit of the original input data.

[0112] In one optional implementation, taking 32-bit integer operations (i.e., operations performed using 32-bit integer data) as an example, the extended bit width for precision improvement is N_up = 32 - 1 - N_in_fft - 2*k, where the subtracted 1 corresponds to a sign bit in the data; k = log2(N), representing the number of butterfly operation stages required for FFT, where subtracting 2*k considers the maximum carry that may result from all accumulated sums in both row and column directions. After increasing the bit width of the input data, the extended bit width of the input data is N_in_fft + N_up + 1, where the highest bit is the sign bit, and the bit width of the numerical part of the extended input data is N_in_fft + N_up. That is, in this embodiment, a dynamic extended bit width of N_up is set for the input data. By increasing the bit width of the input data and adaptively setting the extended bit width according to the bit width of the original input data, the precision of the operation can be effectively guaranteed.

[0113] Step S23: Perform butterfly operation based on extended data.

[0114] To achieve higher computational accuracy, in one optional implementation, the weight factors whose original values ​​fall within the range of [-1, 1] are first shaped to broaden their value range. As an optional implementation, the range of the amplified weight factors is [-32768, 32767]. Because the weight factors are amplified by a preset factor during the butterfly operation to achieve higher computational accuracy, the resulting product is then reduced by the corresponding preset factor after multiplication. Specifically, the original weight factors are first shifted left by a preset bit width W; that is, the amplified weight factors are twice the size of the original weight factors. w If the product is multiplied by W, then after obtaining the product, the product is shifted right by W bits.

[0115] From 0 to k-1, the addition, subtraction and multiplication of butterfly operations are completed in stages.

[0116] In one optional implementation, butterfly operations are first performed row-wise. During the row-wise butterfly operations, the butterfly operations corresponding to each row are performed in parallel. Then, butterfly operations are performed column-wise. During the column-wise butterfly operations, the butterfly operations corresponding to each column are performed in parallel. This effectively improves the parallelism of the operations and increases the processing speed.

[0117] In another alternative implementation, butterfly operations are first performed column-wise. During the column-wise butterfly operations, the butterfly operations for each column are performed in parallel. Then, butterfly operations are performed row-wise. During the row-wise butterfly operations, the butterfly operations for each row are performed in parallel.

[0118] The butterfly operation involves multiplying extended data by weighting factors and performing addition and subtraction operations on the data. The specific implementation of the butterfly operation is readily achievable by those skilled in the art and will not be elaborated upon here. Similarly, setting appropriate values ​​for the weighting factors is also readily achievable by those skilled in the art and will not be elaborated upon here.

[0119] After performing an FFT transform on the image patch, the frequency domain data of the image patch is obtained.

[0120] In one alternative implementation, the FFT transformations of each image block are performed synchronously and in parallel; that is, the shaping FFT transformations of each image block are performed in parallel to obtain the frequency domain data of each image block. Executing the FFT transformation operations for each image block simultaneously can further improve the parallelism of the computation.

[0121] Step S3: Perform frequency domain processing on the image block frequency domain data.

[0122] Frequency domain processing can include noise reduction, image enhancement, etc. The specific implementation methods of frequency domain processing are capable of being implemented by those skilled in the art and will not be elaborated here. Frequency domain processing of image patch frequency domain data yields the image patch frequency domain processed data.

[0123] Step S4: Perform IFFT transform on the image block data after frequency domain processing.

[0124] The operation process of IFFT transformation is similar to that of FFT transformation.

[0125] When performing an IFFT transform, the image patch data is first processed in the frequency domain and then conjugated. Then, based on the conjugated data, a butterfly operation is performed following the same operational method as the FFT transform to achieve the IFFT transform.

[0126] Because of the FFT, the bit width of the real and imaginary parts of the image patch frequency domain data is increased by 2*k bits compared to the bit width of the original image patch data (time domain data). During the IFFT process, the bit width of the numerical part of the data is N_in_ifft + N_iup. That is, the bit width of the numerical part of the image patch data after frequency domain processing is increased to N_in_ifft + N_iup, resulting in a bit width of N_in_ifft + N_iup + 1 (the highest bit is the sign bit). Here, N_in_ifft is the bit width of the numerical part of the image patch data after frequency domain processing, which is the bit width of the data excluding the sign bit. The expanded bit width N_iup = 32 - 2 - N_in_ifft; at this point, N_in_ifft = N_in + 2*k, and N_in = N_in_fft.

[0127] In this embodiment, a dynamically expanded bit width of N_iup is set for the input data of IFFT. By increasing the bit width of the input data and adaptively setting the expanded bit width according to the bit width of the original input data, the accuracy of the operation can be effectively guaranteed. The expanded bit width N_iup = 32 - 2 - N_in_ifft, where the subtraction of 2 is used to denote one bit as the sign bit and the other bit as a redundant bit reserved to avoid overflow during numerical calculation.

[0128] In one optional implementation, when performing the IFFT transform, butterfly operations are first performed row-wise. During the row-wise butterfly operations, the butterfly operations corresponding to each row are performed in parallel. Specifically, from 0 to k-1, after each stage of butterfly operations is completed, the result of the butterfly operation for that stage is shifted one position to the right, until all k stages of butterfly operations have been completed.

[0129] After performing the butterfly operation in the row direction, the butterfly operation is then performed in the column direction. During the column-wise butterfly operation, the butterfly operations for each column are performed in parallel. In practice, from 0 to k-1, after each stage of the butterfly operation is completed, the result of that stage's butterfly operation is shifted one position to the right, until all k stages of the butterfly operation are completed, thus obtaining the result of the IFFT transform.

[0130] In one optional implementation, the IFFT transformations of each image block are performed synchronously and in parallel. That is, after frequency domain processing, the data of each image block is subjected to IFFT transformation in parallel to obtain the IFFT transformation result for each image block. Executing the IFFT transformation operations for each image block simultaneously can further improve the parallelism of the computation.

[0131] Step S5: Output the result image. In practice, the IFFT transformation results corresponding to each image block are stitched together according to the corresponding positions of each image block in the original image to obtain the result image.

[0132] The image processing method in this embodiment effectively improves the parallelism of the operation and increases the processing speed; furthermore, by reasonably setting the data bit width during the operation, it effectively saves resources while ensuring the accuracy of the operation.

[0133] Example 2

[0134] This embodiment provides an image processing method. The image processing method in this embodiment is largely the same as that in Embodiment 1, except that in step S2, the FFT transformation of each image block is performed sequentially. That is, a shaping FFT transformation is performed on each image block sequentially to obtain the frequency domain data of each image block.

[0135] In the process of performing the shaping FFT transformation on each image block in sequence, the same computing resources are used for each execution of the shaping FFT transformation. Therefore, performing the FFT transformation on each image block in sequence is beneficial to saving computing resources.

[0136] Alternatively, as an optional implementation, in step S4, the IFFT transformation of each image block is performed serially. That is, the data after frequency domain processing of each image block is sequentially subjected to IFFT transformation to obtain the IFFT transformation result of each image block.

[0137] In the process of performing IFFT transform on the data after frequency domain processing of each image block, the same computing resources are used for each IFFT transform. Therefore, performing IFFT transform on each image block serially is beneficial to saving computing resources.

[0138] Example 3

[0139] This embodiment provides an image processing method. The image processing method of this embodiment is largely the same as that of Embodiment 1 or Embodiment 2, except that step S2, which performs a shaping FFT transformation on the image block, is different in this embodiment.

[0140] In this embodiment, in step S2, butterfly operations are first performed in parallel on each row in the row direction, and then butterfly operations are performed in parallel on several target columns in the column direction to obtain local frequency domain data. Then, based on the local frequency domain data, the image block frequency domain data is obtained according to the conjugate symmetry of the Fourier transform.

[0141] In one optional implementation, the target columns are [1, N / 2+1] columns of the image block. That is, the target columns are the 1st to (N / 2+1)th columns of the image block. Specifically, when performing butterfly operations in the column direction, not all the data obtained from butterfly operations in the row direction is used; only local data is used to perform butterfly operations, thereby obtaining local frequency domain data.

[0142] Based on the conjugate symmetry of the Fourier transform, i.e., F*(u,v)=F(-u,-v), after obtaining the local frequency domain data, the image patch frequency domain data can be quickly obtained by leveraging the conjugate symmetry of the Fourier transform. This effectively reduces the computational load and improves computational efficiency.

[0143] Example 4

[0144] This embodiment provides an image processing method. The image processing method of this embodiment is largely the same as that of Embodiment 1 or Embodiment 2, except that step S2, which performs a shaping FFT transformation on the image block, is different in this embodiment.

[0145] In this embodiment, in step S2, butterfly operations are first performed in parallel on each column in the column direction, and then butterfly operations are performed in parallel on several target rows in the row direction to obtain local frequency domain data. Then, based on the local frequency domain data, the image block frequency domain data is obtained according to the conjugate symmetry of the Fourier transform.

[0146] In one optional implementation, the target rows are rows [1, N / 2+1] of the image block. That is, the target rows are rows 1 to (N / 2+1) of the image block. Specifically, when performing butterfly operations in the row direction, not all the data obtained from butterfly operations in the column direction is used; only local data is used to perform butterfly operations, thereby obtaining local frequency domain data.

[0147] Based on the conjugate symmetry of the Fourier transform, i.e., F*(u,v)=F(-u,-v), after obtaining the local frequency domain data, the image patch frequency domain data can be quickly obtained by leveraging the conjugate symmetry of the Fourier transform. This effectively reduces the computational load and improves computational efficiency.

[0148] Example 5

[0149] This embodiment provides an image processing method. The image processing method of this embodiment is largely the same as that of any one of the image processing methods in embodiments 1 to 4, except that step S4, which performs IFFT transformation on the image block frequency domain processed data, is different.

[0150] In this embodiment, when performing the IFFT transform, butterfly operations are first performed row-wise. During the row-wise butterfly operations, the butterfly operations corresponding to each row are performed in parallel. Specifically, from 0 to k-1, after each stage of butterfly operation is completed, the result of the butterfly operation for that stage is shifted one position to the right, until all k stages of butterfly operations have been completed.

[0151] After completing the butterfly operation in the row direction, the butterfly operation is performed in the column direction. During the butterfly operation in the column direction, the butterfly operations corresponding to each column are performed in parallel. In specific implementation, the butterfly operation corresponding to each stage is completed sequentially from 0 to k-1, until the butterfly operations of k stages are completed. Then, the total result of the butterfly operation is shifted right by k bits to obtain the result of the IFFT transform.

[0152] The image processing method in this embodiment effectively improves the parallelism of the operation and increases the processing speed; furthermore, by reasonably setting the data bit width during the operation, it effectively saves resources while ensuring the accuracy of the operation.

[0153] Example 6

[0154] This embodiment provides an image processing method. The image processing method of this embodiment is largely the same as that of any one of the image processing methods in embodiments 1 to 4, except that step S4, which performs IFFT transformation on the image block frequency domain processed data, is different.

[0155] In this embodiment, when performing the IFFT transform, butterfly operations are first performed row-wise. During the row-wise butterfly operations, the butterfly operations corresponding to each row are performed in parallel. Specifically, from 0 to k-1, the butterfly operations corresponding to each stage are completed sequentially until all k stages of butterfly operations are completed. Then, the result obtained from the row-wise butterfly operations is shifted right by k bits.

[0156] After performing the butterfly operation in the row direction, the butterfly operation is then performed in the column direction. During the column-wise butterfly operation, the butterfly operations for each column are performed in parallel. In practice, from 0 to k-1, after each stage of the butterfly operation is completed, the result of that stage's butterfly operation is shifted one position to the right, until all k stages of the butterfly operation are completed, thus obtaining the result of the IFFT transform.

[0157] The image processing method in this embodiment effectively improves the parallelism of the operation and increases the processing speed; furthermore, by reasonably setting the data bit width during the operation, it effectively saves resources while ensuring the accuracy of the operation.

[0158] Example 7

[0159] This embodiment provides an image processing method. The image processing method of this embodiment is largely the same as that of any one of the image processing methods in embodiments 1 to 4, except that step S4, which performs IFFT transformation on the image block frequency domain processed data, is different.

[0160] In this embodiment, when performing the IFFT transform, butterfly operations are first performed row-wise. During the row-wise butterfly operations, the butterfly operations corresponding to each row are performed in parallel. Specifically, from 0 to k-1, the butterfly operations corresponding to one stage are completed sequentially until all k stages of butterfly operations are completed. Then, the result obtained from the row-wise butterfly operations is shifted right by k bits.

[0161] After completing the butterfly operation in the row direction, the butterfly operation is performed in the column direction. During the butterfly operation in the column direction, the butterfly operations corresponding to each column are performed in parallel. In specific implementation, the butterfly operation corresponding to one stage is completed sequentially from 0 to k-1, until the butterfly operations of k stages are completed. Then, the total result of the butterfly operation is shifted right by k bits to obtain the result of the IFFT transform.

[0162] The image processing method in this embodiment effectively improves the parallelism of the operation and increases the processing speed; furthermore, by reasonably setting the data bit width during the operation, it effectively saves resources while ensuring the accuracy of the operation.

[0163] Example 8

[0164] Based on the image processing method of any one of the embodiments 1 to 7, this embodiment provides an image processing method. The image processing method of this embodiment is basically the same as the image processing methods in the above embodiments, except for step 5.

[0165] In this embodiment, in step 5, the results of the IFFT transformation corresponding to each image block are stitched together according to the position of each image block in the original image, and the stitched image is used as the intermediate image.

[0166] Since the initial processing is based on image patches, and the image patches do not overlap, there may be discontinuities between the IFFT transform results of two adjacent image patches after the initial processing. That is, the IFFT transform results of two image patches may be discontinuous at the boundary.

[0167] In order to improve the continuity of pixel data at the boundary of adjacent image blocks with the surrounding related pixel data, in step S5, the intermediate image is subjected to smoothing filtering to obtain the result image.

[0168] In one alternative implementation, the smoothing filtering process is performed based on all pixels of the intermediate image.

[0169] In another alternative implementation, the smoothing filtering process is performed based on pixels corresponding to the boundaries of image patches in the intermediate image, as well as several rows / columns of pixels near the boundaries of image patches. This effectively reduces the computational load of the smoothing filtering process and improves computational efficiency.

[0170] Smoothing filtering can improve the continuity of related pixel data at the edges of image blocks.

[0171] Example 9

[0172] Figure 3 This is a schematic diagram of the structure of an electronic device provided in this embodiment. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the image processing method of any one of embodiments 1 to 8. Figure 3 The electronic device 30 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0173] The electronic device 30 may be in the form of a general-purpose computing device, such as a server device. The components of the electronic device 30 may include, but are not limited to: at least one processor 31, at least one memory 32, and a bus 33 connecting different system components (including memory 32 and processor 31).

[0174] Bus 33 includes a data bus, an address bus, and a control bus.

[0175] The memory 32 may include volatile memory, such as random access memory (RAM) 321 and / or cache memory 322, and may further include read-only memory (ROM) 323.

[0176] The memory 32 may also include a program / utility 325 having a set (at least one) of program modules 324, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0177] The processor 31 executes various functional applications and data processing by running computer programs stored in the memory 32, such as the image processing method of any one of the embodiments 1 to 8 of the present invention.

[0178] Electronic device 30 can also communicate with one or more external devices 34 (e.g., keyboard, pointing device, etc.). This communication can be performed via input / output (I / O) interface 35. Furthermore, the model-generated device 30 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public network, such as the Internet) via network adapter 36. As shown, network adapter 36 communicates with other modules of the model-generated device 30 via bus 33. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the model-generated device 30, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems.

[0179] It should be noted that although several units / modules or sub-units / modules of the electronic device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0180] Example 10

[0181] This embodiment provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the image processing method of any one of embodiments 1 to 8.

[0182] The readable storage medium may be more specifically adopted, including but not limited to: portable disk, hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical storage device, magnetic storage device, or any suitable combination thereof.

[0183] In a possible implementation, the present invention can also be implemented as a program product comprising program code, wherein when the program product is run on a terminal device, the program code is used to cause the terminal device to perform the steps of implementing the image processing method of any one of embodiments 1 to 8.

[0184] The program code for executing the present invention can be written in any combination of one or more programming languages. The program code can be executed entirely on the user device, partially on the user device, as a standalone software package, partially on the user device and partially on a remote device, or entirely on a remote device.

[0185] Example 11

[0186] This embodiment provides an image processing system. (Refer to...) Figure 4 The image processing system includes a partitioning unit 101, a shaping FFT transformation unit 102, a frequency domain processing unit 103, an IFFT transformation unit 104, and a result image generation unit 105.

[0187] The partitioning unit 101 is used to divide the image to be processed into several image blocks;

[0188] The shaping FFT transform unit 102 is used to perform shaping FFT transform on the image block to obtain the image block frequency domain data;

[0189] Frequency domain processing unit 103 is used to perform frequency domain processing on the image block frequency domain data to obtain the image block frequency domain processed data;

[0190] The IFFT transform unit 104 is used to perform IFFT transform on the image block after frequency domain processing to obtain the image block IFFT transform result;

[0191] The result image generation unit 105 is used to obtain the result image based on the IFFT transformation result of each image block.

[0192] As an optional implementation, the partitioning unit 101 first divides the image to be processed into several image units. During partitioning, considering the overall size of the image to be processed and the position of the image units within it, two types of image units are formed. The first type of image unit has a preset basic size; the second type of image unit does not have a preset basic size. Specifically, each image unit consists of adjacent pixels, and the image units do not overlap. The first type of image unit is a square, composed of N rows and N columns of pixels, where N is an integer power of 2. To achieve better spectrum control, the preferred range of N is no greater than 64. In one optional implementation, N is 64; in other optional implementations, N is less than 64. The first type of image unit serves as the image block to be processed.

[0193] The second type of image unit refers to the image unit whose edge or corner regions of the image to be processed do not have a preset size during the segmentation process; it is a "residual region" formed after the image to be processed is segmented. As an optional implementation, the second type of image unit is expanded into an image block with a preset basic size. In specific implementation, a corresponding number of pixels are added around the second type of image unit to form an image block with a preset basic size, and the data corresponding to the added pixels is 0.

[0194] Then, the shaping FFT transformation unit 102 performs a shaping FFT transformation on the image block.

[0195] In practice, the two-dimensional FFT transform is converted into two one-dimensional FFT transforms, one for rows and one for columns. The integer FFT transform unit 102 first constructs a first integer array and a second integer array. The first integer array includes N first elements, each of which is used to store the real part of the input data. The second integer array includes N second elements, each of which is used to store the imaginary part of the input data. Each element of the first integer array stores the real part data In_r[i] (i∈[1,N]), and each element of the second integer array stores the imaginary part data In_i[i] (i∈[1,N]).

[0196] Taking the butterfly operation along the row direction as an example, the real part data In_r[i] is the real part of the data corresponding to the i-th pixel of the row in the image block, and the imaginary part data In_i[i] is the imaginary part of the data corresponding to the i-th pixel of the row in the image block. The imaginary part of the initial input data is set to 0.

[0197] Taking the butterfly operation along the column direction as an example, the real part data In_r[i] is the real part of the data corresponding to the i-th pixel in the column of the image block, and the imaginary part data In_i[i] is the imaginary part of the data corresponding to the i-th pixel in the column of the image block. The imaginary part of the initial input data is set to 0.

[0198] Then, the shaping FFT transformation unit 102 increases the bit width of the input data to a first target bit width to form extended data. Specifically, the bit width of the numerical portion of the input data is increased to N_in_fft + N_up. By increasing the bit width of the numerical portion of the input data, the accuracy of the computation process can be effectively guaranteed. N_in_fft is the bit width of the numerical portion of the original input data.

[0199] When the original data is represented as a signed number, the bit width of the original input data is N_in_fft+1, where the most significant bit is the sign bit of the original input data. In an optional implementation, taking 32-bit integer operations (i.e., the operation process uses 32-bit integer data) as an example, the extended bit width for precision improvement is N_up = 32 - 1 - N_in_fft - 2*k, where the subtracted 1 corresponds to one sign bit in the data; k = log2(N), representing the number of butterfly operation stages required for FFT, where subtracting 2*k considers the maximum carry that may result from all the sums in both row and column directions. After increasing the bit width of the input data, the extended bit width of the input data is N_in_fft + N_up + 1, where the most significant bit is the sign bit, and the bit width of the numerical part of the extended input data is N_in_fft + N_up. In other words, in this embodiment, a dynamic extended bit width of N_up is set for the input data. By increasing the bit width of the input data and adaptively setting the extended bit width according to the bit width of the original input data, the accuracy of the operation can be effectively guaranteed.

[0200] Then, the shaping FFT transform unit 102 performs butterfly operations based on the extended data.

[0201] To achieve higher computational accuracy, in one optional implementation, the weighting factors whose original values ​​fall within the range of [-1, 1] are first shaped to broaden their value range. As an optional implementation, the range of the amplified weighting factors is [-32768, 32767]. Because the weighting factors are amplified by a preset factor during the butterfly operation to achieve higher computational accuracy, the resulting product is then reduced by the corresponding preset factor after the product is obtained.

[0202] In practice, the original weighting factor is first shifted left by a preset bit width W, meaning the amplified weighting factor is twice the size of the original weighting factor. w If the product is multiplied by W, then after obtaining the product, the product is shifted right by W bits.

[0203] From 0 to k-1, the addition, subtraction and multiplication of butterfly operations are completed in stages.

[0204] In one optional implementation, butterfly operations are first performed row-wise. During the row-wise butterfly operations, the butterfly operations corresponding to each row are performed in parallel. Then, butterfly operations are performed column-wise. During the column-wise butterfly operations, the butterfly operations corresponding to each column are performed in parallel. This effectively improves the parallelism of the operations and increases the processing speed.

[0205] In another alternative implementation, butterfly operations are first performed column-wise. During the column-wise butterfly operations, the butterfly operations for each column are performed in parallel. Then, butterfly operations are performed row-wise. During the row-wise butterfly operations, the butterfly operations for each row are performed in parallel.

[0206] The butterfly operation extends the multiplication of data with weighting factors and the addition and subtraction of data. The specific implementation of the butterfly operation is readily achievable by those skilled in the art and will not be elaborated upon here. Similarly, setting reasonable values ​​for the weighting factors is also readily achievable by those skilled in the art and will not be elaborated upon here.

[0207] After the shaping FFT transformation unit 102 performs FFT transformation on the image block, it obtains the frequency domain data of the image block.

[0208] In one optional implementation, the shaping FFT transform unit 102 performs FFT transforms on each image block synchronously and in parallel. That is, the shaping FFT transform unit 102 performs shaping FFT transforms on each image block in parallel to obtain the image block frequency domain data corresponding to each image block. Executing the FFT transform operations corresponding to each image block simultaneously can further improve the parallelism of the operation.

[0209] Next, the frequency domain processing unit 103 performs frequency domain processing on the image patch frequency domain data. Frequency domain processing may include noise reduction, image enhancement, etc. The specific implementation of frequency domain processing is feasible for those skilled in the art and will not be elaborated here. After frequency domain processing of the image patch frequency domain data, the processed image patch frequency domain data is obtained.

[0210] Then, the IFFT transform unit 104 performs IFFT transform on the image block frequency domain processed data.

[0211] The IFFT transformation process is similar to the FFT transformation. When performing the IFFT transformation, the image patch data is first conjugated after frequency domain processing. Then, based on the conjugated data, a butterfly operation is performed, referring to the operation method of the FFT transformation, to achieve the IFFT transformation.

[0212] Because of the FFT, the bit width of the real and imaginary parts of the image block frequency domain data is increased by 2*k bits compared to the bit width of the original image block data (time domain data).

[0213] During the IFFT process, the bit width of the numerical part of the data is N_in_ifft + N_iup. That is, the bit width of the numerical part of the data after frequency domain processing of the image block is increased to N_in_ifft + N_iup, so the bit width of the data after frequency domain processing of the image block is N_in_ifft + N_iup + 1 (the highest bit is the sign bit); where N_in_ifft is the bit width of the numerical part of the data after frequency domain processing of the image block, that is, the bit width of the data after frequency domain processing of the image block excluding the sign bit; the extended bit width N_iup = 32 - 2 - N_in_ifft; at this time, N_in_ifft = N_in + 2*k, N_in = N_in_fft.

[0214] In this embodiment, a dynamically expanded bit width of N_iup is set for the input data of IFFT. By increasing the bit width of the input data and adaptively setting the expanded bit width according to the bit width of the original input data, the accuracy of the operation can be effectively guaranteed. The expanded bit width N_iup = 32 - 2 - N_in_ifft, where the subtraction of 2 is used to denote one bit as the sign bit and the other bit as a redundant bit reserved to avoid overflow during numerical calculation.

[0215] In one optional implementation, when performing the IFFT transform, butterfly operations are first performed row-wise. During the row-wise butterfly operations, the butterfly operations corresponding to each row are performed in parallel. Specifically, from 0 to k-1, after each stage of butterfly operations is completed, the result of the butterfly operation for that stage is shifted one position to the right, until all k stages of butterfly operations have been completed.

[0216] After performing the butterfly operation in the row direction, the butterfly operation is then performed in the column direction. During the column-wise butterfly operation, the butterfly operations for each column are performed in parallel. In practice, from 0 to k-1, after each stage of the butterfly operation is completed, the result of that stage's butterfly operation is shifted one position to the right, until all k stages of the butterfly operation are completed, thus obtaining the result of the IFFT transform.

[0217] In one optional implementation, the IFFT transform unit 104 performs IFFT transforms on each image block synchronously and in parallel. That is, the IFFT transform unit 104 performs IFFT transforms on the frequency domain processed data of each image block in parallel to obtain the image block IFFT transform result corresponding to each image block. Executing the IFFT transform operations corresponding to each image block simultaneously can further improve the parallelism of the operation.

[0218] Finally, the result image generation unit 105 outputs the result image. In specific implementation, the IFFT transformation results corresponding to each image block are stitched together according to the corresponding positions of each image block in the original image to obtain the result image.

[0219] The image processing system in this embodiment effectively improves the parallelism of operations and increases processing speed; furthermore, by reasonably setting the data bit width during the operation, it effectively saves resources while ensuring the accuracy of the operation.

[0220] Example 12

[0221] This embodiment provides an image processing system. The image processing system in this embodiment is largely the same as the image processing system in Embodiment 11, except that the shaping FFT transform unit 102 performs the FFT transform on each image block sequentially. That is, the shaping FFT transform unit 102 performs the shaping FFT transform on each image block sequentially to obtain the image block frequency domain data corresponding to each image block.

[0222] In the process of performing the shaping FFT transformation on each image block in sequence, the same computing resources are used for each execution of the shaping FFT transformation. Therefore, performing the FFT transformation on each image block in sequence is beneficial to saving computing resources.

[0223] In addition, as an optional implementation, the IFFT transform unit 104 performs the IFFT transform on each image block sequentially. That is, the IFFT transform unit 104 performs the IFFT transform on the frequency domain processed data of each image block in sequence to obtain the image block IFFT transform result corresponding to each image block.

[0224] In the process of performing IFFT transform on the data after frequency domain processing of each image block, the same computing resources are used for each IFFT transform. Therefore, performing IFFT transform on each image block serially is beneficial to saving computing resources.

[0225] Example 13

[0226] This embodiment provides an image processing system. The image processing system of this embodiment is largely the same as that of Embodiment 11 or Embodiment 12, except that the process of performing a shaping FFT transformation on the image block is different in this embodiment.

[0227] In this embodiment, the shaping FFT transform unit 102 first performs butterfly operations in parallel on each row in the row direction, and then performs butterfly operations in parallel on several target columns in the column direction to obtain local frequency domain data. Then, based on the local frequency domain data, the image block frequency domain data is obtained according to the conjugate symmetry of the Fourier transform.

[0228] In one optional implementation, the target columns are [1, N / 2+1] columns of the image block. That is, the target columns are the 1st to (N / 2+1)th columns of the image block. Specifically, when performing butterfly operations in the column direction, not all the data obtained from butterfly operations in the row direction is used; only local data is used to perform butterfly operations, thereby obtaining local frequency domain data.

[0229] Based on the conjugate symmetry of the Fourier transform, i.e., F*(u,v)=F(-u,-v), after obtaining the local frequency domain data, the image patch frequency domain data can be quickly obtained by leveraging the conjugate symmetry of the Fourier transform. This effectively reduces the computational load and improves computational efficiency.

[0230] Example 14

[0231] This embodiment provides an image processing system. The image processing method of this embodiment is largely the same as that of the image processing system in Embodiment 11 or Embodiment 12, except that the process of performing a shaping FFT transformation on the image block is different in this embodiment.

[0232] In this embodiment, the shaping FFT transform unit 102 first performs butterfly operations in parallel on each column in the column direction, and then performs butterfly operations in parallel on several target rows in the row direction to obtain local frequency domain data. Then, based on the local frequency domain data, the image block frequency domain data is obtained according to the conjugate symmetry of the Fourier transform.

[0233] In one optional implementation, the target rows are rows [1, N / 2+1] of the image block. That is, the target rows are rows 1 to (N / 2+1) of the image block. Specifically, when performing butterfly operations in the row direction, not all the data obtained from butterfly operations in the column direction is used; only local data is used to perform butterfly operations, thereby obtaining local frequency domain data.

[0234] Based on the conjugate symmetry of the Fourier transform, i.e., F*(u,v)=F(-u,-v), after obtaining the local frequency domain data, the image patch frequency domain data can be quickly obtained by leveraging the conjugate symmetry of the Fourier transform. This effectively reduces the computational load and improves computational efficiency.

[0235] Example 15

[0236] This embodiment provides an image processing system. The image processing system of this embodiment is largely the same as the image processing system of any one of the embodiments 11 to 14, except that the process of performing IFFT transform on the image block frequency domain processed data is different.

[0237] In this embodiment, the IFFT transform unit 104 performs butterfly operations row by row when performing the IFFT transform. During the butterfly operations row by row, the butterfly operations corresponding to each row are performed in parallel. Specifically, from 0 to k-1, after each butterfly operation corresponding to a stage is completed, the result of the butterfly operation in that stage is shifted one position to the right, until all k stages of butterfly operations have been completed.

[0238] After completing the butterfly operation in the row direction, the butterfly operation is performed in the column direction. During the butterfly operation in the column direction, the butterfly operations corresponding to each column are performed in parallel. In specific implementation, the butterfly operation corresponding to each stage is completed sequentially from 0 to k-1, until the butterfly operations of k stages are completed. Then, the total result of the butterfly operation is shifted right by k bits to obtain the result of the IFFT transform.

[0239] The image processing system in this embodiment effectively improves the parallelism of operations and increases processing speed; furthermore, by reasonably setting the data bit width during the operation, it effectively saves resources while ensuring the accuracy of the operation.

[0240] Example 16

[0241] This embodiment provides an image processing system. The image processing system of this embodiment is largely the same as the image processing system of any one of the embodiments 11 to 14, except that the process of performing IFFT transform on the image block frequency domain processed data is different.

[0242] In this embodiment, the IFFT transform unit 104 performs butterfly operations row by row when performing the IFFT transform. During the butterfly operations row by row, the butterfly operations corresponding to each row are performed in parallel. Specifically, from 0 to k-1, the butterfly operations corresponding to each stage are completed sequentially until all k stages of butterfly operations are completed. Then, the result obtained from the row-by-row butterfly operations is shifted right by k bits.

[0243] After performing the butterfly operation in the row direction, the butterfly operation is then performed in the column direction. During the column-wise butterfly operation, the butterfly operations for each column are performed in parallel. In practice, from 0 to k-1, after each stage of the butterfly operation is completed, the result of that stage's butterfly operation is shifted one position to the right, until all k stages of the butterfly operation are completed, thus obtaining the result of the IFFT transform.

[0244] The image processing system in this embodiment effectively improves the parallelism of operations and increases processing speed; furthermore, by reasonably setting the data bit width during the operation, it effectively saves resources while ensuring the accuracy of the operation.

[0245] Example 17

[0246] This embodiment provides an image processing system. The image processing system of this embodiment is largely the same as the image processing systems of any of the embodiments in Embodiments 11 and 14, except that the process of performing IFFT transform on the image block frequency domain processed data is different.

[0247] In this embodiment, the IFFT transform unit 104 performs butterfly operations row by row when performing the IFFT transform. During the butterfly operations row by row, the butterfly operations corresponding to each row are performed in parallel. Specifically, from 0 to k-1, the butterfly operations corresponding to one stage are completed sequentially until all k stages of butterfly operations are completed. Then, the result obtained from the row-by-row butterfly operations is shifted right by k bits.

[0248] After completing the butterfly operation in the row direction, the butterfly operation is performed in the column direction. During the butterfly operation in the column direction, the butterfly operations corresponding to each column are performed in parallel. In specific implementation, the butterfly operation corresponding to one stage is completed sequentially from 0 to k-1, until the butterfly operations of k stages are completed. Then, the total result of the butterfly operation is shifted right by k bits to obtain the result of the IFFT transform.

[0249] The image processing system in this embodiment effectively improves the parallelism of operations and increases processing speed; furthermore, by reasonably setting the data bit width during the operation, it effectively saves resources while ensuring the accuracy of the operation.

[0250] Example 18

[0251] Based on the image processing system of any one of Embodiments 11 to 17, this embodiment provides an image processing system. The image processing system of this embodiment is basically the same as the image processing systems in the above embodiments, except that the result image generation unit 105 is different.

[0252] In this embodiment, the result image generation unit 105 stitches together the IFFT transformation results corresponding to each image block according to the position of each image block in the original image, and uses the stitched image as an intermediate image.

[0253] Since the initial processing is based on image patches, and the image patches do not overlap, there may be discontinuities between the IFFT transform results of two adjacent image patches after the initial processing. That is, the IFFT transform results of two image patches may be discontinuous at the boundary.

[0254] In order to improve the continuity of pixel data at the boundary of adjacent image blocks with the surrounding related pixel data, the result image generation unit 105 performs smoothing filtering on the intermediate image to obtain the result image.

[0255] In one alternative implementation, the smoothing filtering process is performed based on all pixels of the intermediate image.

[0256] In another alternative implementation, the smoothing filtering process is performed based on pixels in the intermediate image corresponding to the boundaries of image patches, as well as several rows / columns of pixels near the boundaries of image patches. This effectively reduces the computational load of the smoothing filtering process and improves computational efficiency.

[0257] Smoothing filtering can improve the continuity of related pixel data at the edges of image blocks.

[0258] While specific embodiments of the present invention have been described above, those skilled in the art should understand that these are merely illustrative examples, and the scope of protection of the present invention is defined by the appended claims. Those skilled in the art can make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, but all such changes and modifications fall within the scope of protection of the present invention.

Claims

1. An image processing method, characterized in that, Includes the following steps: The image to be processed is divided into several image blocks; The image block is subjected to a shaping FFT transform to obtain the image block frequency domain data; The image block frequency domain data is processed in the frequency domain to obtain the image block frequency domain processed data; The image block frequency domain processed data is subjected to IFFT transformation to obtain the image block IFFT transformation result; The result image is obtained based on the IFFT transformation result of each image block corresponding to the image block; The steps for performing IFFT transform on the frequency domain processed data of the image patch include: The image block frequency domain processed data is subjected to conjugation processing to obtain conjugated data; Increase the bit width of the numerical portion of the conjugate data to a second target bit width to form extended conjugate data; Perform a butterfly operation on the extended conjugate data; The second target bit width is N_in_ifft+N_iup, where N_iup=32-2-N_in_ifft, N_in_ifft=N_in+2*k, N_in represents the bit width of the numerical part of the original data of the image block, and k represents the number of butterfly operation stages required by IFFT.

2. The image processing method as described in claim 1, characterized in that, The step of dividing the image to be processed into several image blocks includes: The image to be processed is divided into several image units, each image unit consisting of adjacent pixels, and no two image units overlap. The image units include a first type of image unit and a second type of image unit. The first type of image unit has N rows and N columns of pixels, where N is an integer power of 2. The second type of image unit is the image unit other than the first type of image unit. The second type of image unit is expanded into an expanded unit with N rows and N columns of pixels; The first type of image unit and the extended unit are used as the image block.

3. The image processing method as described in claim 2, characterized in that, The step of performing a shaping FFT transform on the image block includes: Construct a first integer array and a second integer array. The first integer array includes N first elements, each of which is used to store the real part of the input data. The second integer array includes N second elements, each of which is used to store the imaginary part of the input data. Increase the bit width of the numerical portion of the input data to a first target bit width to form extended data; Perform butterfly operations based on the extended data.

4. The image processing method as described in claim 3, characterized in that, N is less than or equal to 64.

5. The image processing method as described in claim 3, characterized in that, The first target bit width Wt = N_in_fft + N_up, where N_in_fft represents the bit width of the numerical part of the input data, N_up = 32 - 1 - N_in_fft - 2 * k, k = log2(N), and k represents the number of butterfly operation stages required by FFT.

6. The image processing method as described in claim 3, characterized in that, The steps for performing butterfly operations based on the extended data include: The weighting factors used in the butterfly operation are shaped.

7. The image processing method as described in claim 3, characterized in that, The steps for performing butterfly operations based on the extended data include: Perform butterfly operations first in the row direction, then perform butterfly operations in the column direction; Alternatively, perform a butterfly operation first along the column direction, and then perform a butterfly operation along the row direction.

8. The image processing method as described in claim 7, characterized in that, During the butterfly operation in the row direction, the butterfly operations corresponding to each row are performed in parallel; during the butterfly operation in the column direction, the butterfly operations corresponding to each column are performed in parallel.

9. The image processing method as described in claim 1, characterized in that, The step of performing a shaping FFT transform on the image block to obtain the image block frequency domain data includes: The image blocks are subjected to a shaping FFT transformation in parallel to obtain the frequency domain data of each image block. The step of performing IFFT transform on the frequency domain processed data of the image patch to obtain the IFFT transform result of the image patch includes: After frequency domain processing of each image block, the data is subjected to IFFT transformation in parallel to obtain the IFFT transformation result of each image block.

10. The image processing method as described in claim 1, characterized in that, The step of performing a butterfly operation on the conjugate data includes: Perform butterfly operations first in the row direction, then perform butterfly operations in the column direction.

11. The image processing method as described in claim 10, characterized in that, The step of performing butterfly operations first according to the row direction includes: After each butterfly operation is completed, the result of the butterfly operation in that stage is shifted one bit to the right until all the butterfly operations in k stages are completed; or, the butterfly operation corresponding to each stage is completed in sequence until all the butterfly operations in k stages are completed, and then the result of the butterfly operation is shifted k bits to the right. The step of performing butterfly operations in the column direction includes: After each butterfly operation is completed, the result of the butterfly operation in that stage is shifted right by one bit, until all butterfly operations in k stages are completed; or, the butterfly operation corresponding to each stage is completed sequentially, until all butterfly operations in k stages are completed, and then the result of the butterfly operation is shifted right by k bits.

12. The image processing method as described in claim 1, characterized in that, The step of obtaining the result image based on the IFFT transform result of each image block includes: The IFFT transformation results of each image block are stitched together to form an intermediate image. The intermediate image is smoothed and filtered to obtain the result image.

13. The image processing method as described in claim 3, characterized in that, The steps for performing butterfly operations based on the extended data include: First, butterfly operations are performed in parallel on each row in the row direction. Then, butterfly operations are performed in parallel on several target columns in the column direction to obtain local frequency domain data. Then, based on the local frequency domain data, the image block frequency domain data is obtained according to the conjugate symmetry of the Fourier transform. The several target columns are the [1, N / 2+1] columns of the image block. Alternatively, the step of performing butterfly operations based on the extended data includes: First, butterfly operations are performed in parallel on each column in the column direction. Then, butterfly operations are performed in parallel on several target rows in the row direction to obtain local frequency domain data. Then, based on the local frequency domain data, the image block frequency domain data is obtained according to the conjugate symmetry of the Fourier transform. The several target rows are the [1, N / 2+1] rows of the image block.

14. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the image processing method according to any one of claims 1-13.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the image processing method according to any one of claims 1-13.

16. An image processing system, characterized in that, It includes a partitioning unit, a shaping FFT transform unit, a frequency domain processing unit, an IFFT transform unit, and a result image generation unit; The partitioning unit is used to divide the image to be processed into several image blocks; The shaping FFT transform unit is used to perform a shaping FFT transform on the image block to obtain the image block frequency domain data; The frequency domain processing unit is used to perform frequency domain processing on the image block frequency domain data to obtain image block frequency domain processed data. The IFFT transform unit is used to perform IFFT transform on the image block frequency domain processed data to obtain the image block IFFT transform result; The result image generation unit is used to obtain the result image based on the IFFT transformation result of the image block corresponding to each image block; The IFFT transform unit is also used to perform conjugation processing on the image block frequency domain processed data to obtain conjugated data; The IFFT transform unit is also used to increase the bit width of the numerical part of the conjugate data to a second target bit width to form extended conjugate data; The IFFT transform unit is also used to perform butterfly operations on the extended conjugate data; The second target bit width is N_in_ifft+N_iup, where N_iup=32-2-N_in_ifft, N_in_ifft=N_in+2*k, N_in represents the bit width of the numerical part of the original data of the image block, and k represents the number of butterfly operation stages required by IFFT.

17. The image processing system as described in claim 16, characterized in that, The partitioning unit is further used to divide the image to be processed into several image units, each image unit consisting of adjacent pixels, and no two image units overlap; the image units include a first type of image unit and a second type of image unit, the first type of image unit having N rows and N columns of pixels, where N is an integer power of 2, and the second type of image unit being the image units other than the first type of image unit. The partitioning unit is also used to expand the second type of image unit into an expanded unit with N rows and N columns of pixels; The partitioning unit is also used to use the first type of image unit and the expanded unit as the image block.

18. The image processing system as described in claim 17, characterized in that, The shaped FFT transform unit is also used to construct a first integer array and a second integer array. The first integer array includes N first elements, each of which is used to store the real part of the input data. The second integer array includes N second elements, each of which is used to store the imaginary part of the input data. The shaping FFT transform unit is also used to increase the bit width of the numerical part of the input data to a first target bit width to form extended data; The shaped FFT transform unit is also used to perform butterfly operations based on the extended data.

19. The image processing system as described in claim 18, characterized in that, N is less than or equal to 64.

20. The image processing system as described in claim 18, characterized in that, The first target bit width Wt = N_in_fft + N_up, where N_in_fft represents the bit width of the numerical part of the input data, N_up = 32 - 1 - N_in_fft - 2 * k, k = log2(N), and k represents the number of butterfly operation stages required by FFT.

21. The image processing system as described in claim 18, characterized in that, The shaping FFT transform unit is also used to shape the weight factors used in the butterfly operation.

22. The image processing system as described in claim 18, characterized in that, The shaped FFT transform unit is also used to perform butterfly operations first in the row direction and then in the column direction; Alternatively, the shaped FFT transform unit is also used to perform butterfly operations first in the column direction and then in the row direction.

23. The image processing system as described in claim 22, characterized in that, During the butterfly operation in the row direction, the butterfly operations corresponding to each row are performed in parallel; during the butterfly operation in the column direction, the butterfly operations corresponding to each column are performed in parallel.

24. The image processing system as described in claim 16, characterized in that, The shaping FFT transform unit is also used to perform shaping FFT transform on each of the image blocks in parallel to obtain the image block frequency domain data corresponding to each of the image blocks; The IFFT transform unit is also used to perform IFFT transform on the frequency domain processed data of each of the image blocks in parallel to obtain the IFFT transform result of each image block.

25. The image processing system as described in claim 16, characterized in that, The IFFT transform unit is also used to perform butterfly operations first in the row direction and then in the column direction.

26. The image processing system as described in claim 25, characterized in that, When performing butterfly operations in the row direction, the IFFT transform unit is also used to shift the result of the butterfly operation of each stage to the right by one bit after completing the butterfly operation of each stage, until the butterfly operations of k stages have been completed; or, the IFFT transform unit is also used to complete the butterfly operation of each stage in sequence, until the butterfly operations of k stages have been completed, and then shift the result of the butterfly operation to the right by k bits. When performing butterfly operations in the column direction, the IFFT transform unit is also used to shift the result of the butterfly operation of each stage to the right by one bit after completing the butterfly operation of each stage, until the butterfly operations of k stages have been completed; or, the IFFT transform unit is also used to complete the butterfly operation of each stage in sequence, until the butterfly operations of k stages have been completed, and then shift the result of the butterfly operation to the right by k bits.

27. The image processing system as described in claim 16, characterized in that, The result image generation unit is also used to stitch together the IFFT transformation results of each image block to form an intermediate image; The result image generation unit is further configured to perform smoothing filtering on the intermediate image to obtain the result image.

28. The image processing system as described in claim 18, characterized in that, The shaping FFT transform unit first performs butterfly operations in parallel on each row in the row direction, and then performs butterfly operations in parallel on several target columns in the column direction to obtain local frequency domain data. Then, based on the local frequency domain data, the image block frequency domain data is obtained according to the conjugate symmetry of the Fourier transform. The several target columns are the [1, N / 2+1] columns of the image block. Alternatively, the shaping FFT transform unit first performs butterfly operations in parallel on each column in the column direction, and then performs butterfly operations in parallel on several target rows in the row direction to obtain local frequency domain data. Then, based on the local frequency domain data, the image block frequency domain data is obtained according to the conjugate symmetry of the Fourier transform. The several target rows are the [1, N / 2+1] rows of the image block.

Images