Image enhancement method and storage medium based on convolution operation

Abstract

A convolution-based image enhancement method and storage medium are disclosed. The method includes inputting an original image g(x, y) and caching the image, calculating a blurred image #imgabs0#, and calculating the result of the Laplacian operator on the blurred image; calculating a compensation coefficient a based on the current pixel coordinates and the horizontal and vertical resolutions of the image; and obtaining a restored image #imgabs1# based on an image restoration spatial domain expression. The convolution-based image restoration algorithm of the present invention improves the clarity of the image edges by reversing the causes of image blur and performing operations with different convolution kernels and enhancement factors at the image center and edges, thereby restoring the blurred image edges. Using different compensation coefficients for the image center and edges better enhances image edge information.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image enhancement method and storage medium based on convolution operation. Background Art

[0002] An imaging system is often affected by numerous degradation factors, such as atmospheric disturbances, optical aberrations, and relative motion between the camera and the object, resulting in varying degrees of image degradation. This invention primarily addresses the blurring phenomenon caused by lens field curvature. Field curvature (image curvature) is a common aberration in optical systems. When an optical system exhibits severe field curvature, it cannot produce a completely clear image of an object on a plane. If the center of the image is clearly focused, the edges will be blurred. Conversely, if the edges are clear, the center will be blurred.

[0003] For practical sampling systems, the causes of image degradation vary. If the point spread function of a degraded system does not vary with the object plane coordinates—that is, it is independent of the spatial position of the imaged object points—then the degraded system is shift-invariant. Image degradation can be described by a convolution process, and image restoration is a deconvolution problem.

[0004] Field curvature originates from the basic law of lens imaging. For the same lens, objects far away will be imaged close, and conversely, objects close to them will be imaged far away. Figure 1 As shown, points A and B are located on the same plane at different distances from the lens. Point A is imaged at point A', which is located on the sensor surface and produces a clear image. Point B is imaged at point B', which is located above the sensor surface and therefore not clearly imaged. Existing technologies cannot address the problem of blurred pixels at the edges of images caused by field curvature. Summary of the Invention

[0005] The present invention proposes an image enhancement method based on convolution operation, which can solve the above technical problems.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] The sensor inputs the original image g(x,y) and caches it inside the FPGA. The FPGA uses a pipeline processing method to restore the input original image in sequence. The FPGA processes the image in units of lines, and performs noise reduction on each line of the image to obtain the noise-reduced image. For each pixel, the compensation coefficient a is calculated according to the pixel position (x, y). Calculate its high-frequency components Finally calculate the restored image

[0008] Furthermore, an image buffer is opened inside the FPGA, which can store up to 16 lines of image data. The image buffer adopts the FIFO mechanism. Whenever the front-end sensor collects a new line of image data, it is placed at the end of the buffer and the earliest image data arriving in the FIFO is processed.

[0009] Furthermore, FPGA processes image data in units of rows. Each row of image is divided into 32 equal blocks, and a separate processing core is placed in each block of image for calculation.

[0010] Furthermore, the FPGA uses an anisotropic algorithm to perform image noise reduction on each block of image data. For each pixel point, the anisotropic algorithm uses its four adjacent pixels to participate in the calculation and only performs one iterative process.

[0011] Furthermore, the FPGA uses the Laplace operator to extract high-frequency components from the denoised image.

[0012] Furthermore, the expression of the Laplace operator is shown in formula (3):

[0013] Furthermore, the compensation coefficient a is calculated according to the position (x, y) of the current calculated pixel point.

[0014] Furthermore, the specific calculation method of the compensation coefficient a is as follows, where row is the image height, col is the image width, and β is the weight factor set according to the specific scenario;

[0015]

[0016] Furthermore, according to the compensation coefficient a and the high frequency component Calculate the restored image, where the restored image is On the other hand, the present invention further discloses a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of the above method.

[0017] As can be seen from the above technical solution, the convolution-based image enhancement method of the present invention is also a convolution-based image restoration algorithm. This algorithm reverses the causes of image blur and performs operations with different convolution kernels at the center and edges of the image to improve edge clarity. Based on the fundamental principles of field curvature, the algorithm of the present invention uses an inverse frequency domain transform to restore the blurred image edges. Different compensation coefficients are used for the image center and edges, with larger compensation coefficients applied closer to the edge, thereby enhancing image edge information.

[0018] Specifically, the main technical innovation of this invention lies in the use of an adaptive enhancement factor for convolution operations. For image edge regions, where field curvature is large and blur is high, a larger enhancement factor is used for image enhancement. For central regions, where field curvature is small and blur is low, a smaller enhancement factor or no enhancement is performed, thus ensuring consistency in overall image clarity. Furthermore, the method of first performing anisotropic filtering and then extracting high-frequency components effectively suppresses image noise, preventing image quality degradation caused by random noise.

[0019] On the other hand, this technical solution is based on FPGA implementation and can process multiple lines of image data in parallel. At the same time, each line of image is divided into 32 blocks, and each block of image data is handed over to a separate computing core for processing, which greatly reduces the image processing time and improves the system's concurrency. It is suitable for application scenarios with high-speed data transmission and image processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is the imaging optical path of the field curvature system;

[0021] Figure 2 is a flow chart of the method of the present invention;

[0022] Figure 3 This is the process of image processing by FPGA of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0024] like Figure 2 As shown in FIG, an image enhancement method based on convolution operation includes: the sensor inputs the original image g(x, y) and caches it inside the FPGA, the FPGA uses a pipeline processing method to restore the input original image in sequence; the FPGA processes the image in units of lines, and performs noise reduction processing on each line of the image to obtain the noise-reduced image For each pixel, the compensation coefficient a is calculated according to the pixel position (x, y). Calculate its high-frequency components Finally calculate the restored image

[0025] The following are specific instructions:

[0026] S1: An image buffer is created inside the FPGA, which can store up to 16 lines of image data. The image buffer uses a FIFO mechanism. Whenever the front-end sensor collects a new line of image data, it is placed at the end of the buffer and the earliest image data in the FIFO is processed. Figure 3 This demonstrates how FPGA processes image data. The image data collected by the camera is always placed at the end of the FIFO, and the FPGA retrieves data from the starting address of the FIFO for image enhancement.

[0027] S2: FPGA processes image data in units of rows. Each row of image is divided into 32 equal blocks, and a separate processing core is placed in each block for calculation.

[0028] S3: For each image, an anisotropic algorithm is used for noise reduction. The anisotropic algorithm uses the critical 4 pixel information of the current pixel. Considering the data processing characteristics of FPGA, only one iteration operation is used; at the same time, since FPGA cannot complete the more complex exponential operation process, the Taylor expansion method is used to replace the exponential operation in the original anisotropic algorithm. Specifically, as shown in formula (1), it is the basic formula of anisotropic filtering, and formulas (2) to (5) are methods of simplifying exponential operations using Taylor expansion. Where g(x,y) is the original grayscale value of the coordinate point (x,y), is the denoised image, and γ is the weight factor that needs to be set according to the specific scenario.

[0029]

[0030] k1=(1-(g(x,y-1)-g(x,y)) 2 / 5) 2 (2)

[0031] k2=(1-(g(x,y+1)-g(x,y)) 2 / 5) 2 (3)

[0032] k3=(1-(g(x+1,y)-g(x,y)) 2 / 5) 2 (4)

[0033] k4=(1-(g(x-1,y)-g(x,y)) 2 / 5) 2 (5);

[0034] Where k1 is the noise reduction coefficient in the horizontal East direction; k2 is the noise reduction coefficient in the horizontal West direction; k3 is the noise reduction coefficient in the vertical North direction; and k4 is the noise reduction coefficient in the vertical South direction. This formula first calculates the gradient in each of the South, North, East, and West directions, then calculates the compensation amount based on the noise reduction coefficient and gradient, thereby achieving the desired noise reduction effect.

[0035] S4: For the image after noise reduction The Laplace operator is used to extract the high-frequency component. The expression of the Laplace operator is shown in formula (6).

[0036]

[0037] S5: Calculate the compensation coefficient a based on the current calculated pixel position (x, y). The edge of the sensor may be more blurred than the central area due to factors such as field curvature, so the compensation coefficient a will increase accordingly. On the other hand, the closer to the center of the sensor, the smaller the compensation coefficient a.

[0038] The specific calculation method is shown below, where row is the image height, col is the image width, and β is the weight factor that needs to be set according to the specific scenario.

[0039]

[0040] S6: high-frequency component of the image calculated according to S4 The weight factor α calculated by S5 is used to obtain the final restored image according to formula (4):

[0041]

[0042] In summary, the convolution-based image enhancement method of the present invention is also a convolution-based image restoration algorithm. This algorithm reverses the causes of image blur and performs convolution operations with different kernels at the center and edges of the image to improve edge clarity. Based on the fundamental principles of field curvature, the algorithm uses an inverse frequency domain transform to restore image edge blur. Different compensation coefficients are used for the image center and edges, with larger compensation coefficients applied closer to the edge, thereby enhancing image edge information.

[0043] In another aspect, the present invention further discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of any of the above methods.

[0044] On the other hand, the present invention further discloses a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of any of the above methods.

[0045] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute the steps of any one of the methods in the above embodiments.

[0046] It is understandable that the system provided by the embodiment of the present invention corresponds to the method provided by the embodiment of the present invention, and the explanation, examples and beneficial effects of the relevant contents can refer to the corresponding parts of the above method.

[0047] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0048] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0049] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. An image enhancement method based on convolution operation, characterized in that: The sensor inputs the original image g(x,y) and caches it inside the FPGA. The FPGA uses a pipeline processing method to restore the input original image in sequence. The FPGA processes the image in units of lines, and performs noise reduction on each line of the image to obtain the noise-reduced image. , for each pixel point, the compensation coefficient a is calculated according to the pixel position (x, y). Calculate its high-frequency components , and finally calculate the restored image ; Calculate the compensation coefficient a according to the current calculated pixel position (x, y); The specific calculation method of the compensation coefficient a is as follows, where row is the image height, col is the image width, and β is the weight factor set according to the specific scenario; row_half = (row >> 1); col_half = (col >> 1); / / Calculate the radius of the pixel (x, y) when the image is centered if (y > row_half) row_delta = y - row_half; else row_delta = row_half – y; if (x > col_half) col_delta = x - col_half; else col_delta = col_half – x; d = row_delta row_delta + col_delta col_delta; α = (β d) >> 25 0.1； According to the compensation coefficient a and high frequency component Calculate the restored image; where the restored image is , .

2. The image enhancement method based on convolution operation according to claim 1, characterized in that: An image buffer is created inside the FPGA, which can store up to 16 lines of image data. The image buffer uses a FIFO mechanism. Whenever the front-end sensor collects a new line of image data, it is placed at the end of the buffer and the earliest image data in the FIFO is processed.

3. The image enhancement method based on convolution operation according to claim 1, characterized in that: FPGA processes image data in units of rows. Each row of image is divided into 32 equal blocks, and a separate processing core is placed in each block for calculation.

4. The image enhancement method based on convolution operation according to claim 1, characterized in that: FPGA uses an anisotropic algorithm to perform image noise reduction on each piece of image data. For each pixel, the anisotropic algorithm uses its four adjacent pixels to participate in the calculation and only performs one iteration.

5. The image enhancement method based on convolution operation according to claim 1, characterized in that: FPGA uses the Laplace operator to extract high-frequency components from the denoised image.

6. The image enhancement method based on convolution operation according to claim 5, characterized in that: The expression of the Laplace operator is shown in formula (3): .

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 6.

Images