An industrial camera image sharpening method, device, equipment and storage medium

Abstract

The application discloses an industrial camera image sharpening method, device, equipment and storage medium, which comprises the following steps: performing sharpening processing on an input original image to enhance the edge weight of the image, so as to obtain a preliminary sharpening image layer highlighting the image edge and an edge layer reflecting the image edge; performing a noise removal processing on the center pixel of a region in the edge layer according to the gray scale change degree of the region, so as to obtain a denoising edge layer; and performing a noise reduction sharpening weight processing on the denoising edge layer by an FPGA, and then performing a weighted fusion of the denoising edge layer and the preliminary sharpening image layer, so as to form a sharpening image which is sharpened only on the edge. The application removes isolated noise points by using edge determination, and further removes Poisson noise and Gaussian noise. Furthermore, the application reduces the influence of noise on the sharpening result by means of weighted fusion, and improves the image sharpening effect.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to an industrial camera image sharpening method, apparatus, device, and storage medium. Background Technology

[0002] Raw images from image sensors typically contain Poisson and Gaussian noise. Ordinary unsharp mask (USM) sharpening algorithms sharpen the noise along with the images, resulting in a loss of image information. Therefore, it is necessary to improve the algorithm so that it only enhances the edges without enhancing the noise.

[0003] Due to issues such as improper lens and focal length, raw images captured by camera sensors often contain transition pixels at the edges. Sharpening algorithms can be used to reduce the number of transition pixels. Traditional image sharpening algorithms include template operators such as the Laplacian operator and Unsharp Mask (USM) sharpening algorithm. These can enhance the edge information of an image, but they also enhance noise. Furthermore, due to the limited resources of a camera, complex sharpening algorithms cannot be applied to remove noise. Therefore, a simple sharpening algorithm is needed that enhances only the edges.

[0004] The closest existing solution to this invention is CN103763460A, which directly determines edge points by the difference between the current pixel and the surrounding pixels, and the enhancement magnitude of all edge points is fixed, without considering Gaussian noise and edge jaggedness.

[0005] Existing general USM sharpening algorithms extract high-frequency information from images through Gaussian filtering or differential operators, and then add weighted high-frequency components to the original image to obtain the sharpening result. They often do not take into account the Gaussian and Poisson noise problems in the image, resulting in sharpening noise while sharpening the edges. Summary of the Invention

[0006] The present invention proposes an industrial camera image sharpening method, which can at least solve one of the above-mentioned technical problems.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] An industrial camera image sharpening method includes:

[0009] The input original image is enhanced with edge weight sharpening to obtain a preliminary sharpening layer that highlights the image edges and an edge layer that reflects the image edges.

[0010] For the edge layer, noise removal processing is performed by assigning a fixed value to the center pixel of the region based on the degree of grayscale change of the region pixels, resulting in a denoised edge layer;

[0011] After reducing the noise and sharpening weights, the denoised edge layer is weighted and fused with the initial sharpening layer to form a sharpened image that only sharpens the edges.

[0012] Furthermore, based on the grayscale change standard of pixels in the noise edge region, noise regions are identified and removed, thereby selecting pixel block edges for recognition. The FPGA filters noise points based on the edge recognition results and performs Gaussian filtering on the resulting mask to remove Poisson and Gaussian noise. The grayscale change standard for pixels in the noise edge region is whether the pixel value at a certain location is significantly higher than the surrounding pixel values; if so, it is considered a noise point.

[0013] Furthermore, the original image, the preliminary sharpening layer, and the denoising edge layer are weighted and fused to obtain a sharpened image that only sharpens the edges.

[0014] Furthermore, the weighted fusion step involves determining the number of pixels greater than a threshold within an N*N (where N is an odd number and N is less than or equal to 9) small square centered on the current pixel in the edge layer.

[0015] Furthermore, the weighted fusion algorithm includes R = S*(1-B) + A*B; where S is the original image, R is the sharpened image with edge sharpening only, A is the initial sharpening layer, and B is the denoised edge layer.

[0016] Furthermore, the edge recognition includes the following logic: if the number of pixels greater than the threshold is less than N, then the edge point is determined to be 0; if the number of pixels greater than the threshold is greater than or equal to N, then the edge point is determined to be 1.

[0017] The present invention also discloses an image sharpening device, comprising:

[0018] The image sharpening module sharpens the original image to obtain a preliminary sharpening layer and the edge layer;

[0019] The noise removal module determines the edge points of the edge layer and assigns a fixed value to the center pixel of the region based on the degree of grayscale change of the region to remove noise, thereby obtaining a denoised edge layer.

[0020] The image fusion module performs weighted fusion on the original image, the preliminary sharpening layer, and the denoising edge layer to obtain a sharpened image that only sharpens the edges.

[0021] The present invention also discloses a computer device, including a processor and a memory.

[0022] Memory, used to store computer programs;

[0023] A processor is used to execute a computer program stored in the memory to implement the image sharpening method described above.

[0024] In another aspect, the present invention also discloses a storage medium that, when the program in the storage medium is executed by the processor, can implement the image sharpening method described above.

[0025] As can be seen from the above technical solution, the industrial camera image sharpening algorithm of the present invention sharpens the original image from the sensor while distinguishing between noise and edges, enhancing only the edges and leaving the noise unprocessed. To filter out noise points, the present invention adds an edge detection step to the original USM sharpening algorithm; to further reduce the influence of noise, the mask obtained from the edge detection is subjected to Gaussian filtering, which is then used to weight and fuse the result obtained from the traditional USM sharpening algorithm with the original image to obtain the final result.

[0026] This invention removes isolated noise by utilizing edge detection. If a pixel value at a certain location is much larger than the surrounding pixel values, it is reasonable to assume that this pixel value is caused by noise, since there are no other pixels that can form an edge with it. Furthermore, Gaussian filtering is applied to the image edges to reduce the weight of non-edge sharpening results. Finally, weighted fusion is used to further reduce the impact of noise on the sharpening results, which helps to improve the image sharpening effect. Attached Figure Description

[0027] Figure 1 This is a flowchart of a method according to an embodiment of the present invention;

[0028] Figure 2 This is a schematic diagram of an image sharpening device according to an embodiment of the present invention;

[0029] Figure 3 This is a schematic diagram of the pipeline mode image data processing of the FPGA internal image sharpening module in an embodiment of the present invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0031] The industrial camera image sharpening method described in this embodiment includes sharpening processing that enhances the edge weights of the input original image to obtain a preliminary sharpening layer that highlights the image edges and an edge layer that reflects the image edges.

[0032] The edge layer is subjected to noise removal processing by assigning a fixed value to the center pixel of the region based on the degree of grayscale change of the region's pixels, resulting in a denoised edge layer;

[0033] After the noise-reducing edge layer is processed with a sharpening weight to reduce noise, it is then weighted and fused with the initial sharpening layer to form a sharpened image that only sharpens the edges.

[0034] The following are examples, such as Figure 1 As shown:

[0035] First, input the original image S;

[0036] The second step is to perform Gaussian filtering on the original image S to obtain the first layer G1. Gaussian filtering will blur the edge parts of the image, while the flat areas will not be changed.

[0037] The third step is to subtract the original image S from the first layer G1 to obtain the second layer G2, i.e., G2 = S - G1. The original image S minus the Gaussian filtered image has only non-zero values ​​at the edges, while the flat parts become zero. In this way, the high-frequency part of the image can be extracted.

[0038] The fourth step is to add the original image S to the second layer G2 with a sharpening weight T to obtain the third layer G3. The sharpening weight T is set to 0.5 by default, that is, G3 = S + G2 * T. The original image S is added with the weighted high-frequency part to highlight the image edges.

[0039] The first to fourth steps above can be understood as sharpening the image edges by enhancing the image edge weights of the original input image, resulting in a preliminary sharpening layer that highlights the image edges and an edge layer that reflects the image edges.

[0040] The fifth step is to determine the edge points of the second layer G2 to obtain the fourth layer G4.

[0041] Specifically, for the 3x3 square centered on the current pixel in the second layer G2, determine the number of pixels greater than the threshold. If the number is less than 3, the value is 0; if it is greater than or equal to 3, the value is 1. The threshold is set using an adaptive threshold method. Specifically, the edge layer is divided into N*N squares, the grayscale standard deviation of each square is calculated, the standard deviations are sorted, and the grayscale values ​​of the first half of the standard deviations are selected. The average value of these standard deviations is then used as the threshold.

[0042] Due to the nature of discrete charges and the uncertainty of photons, images from CMOS sensors often contain noise that follows a Poisson distribution. This invention uses the degree of pixel clustering at edges to determine whether a pixel is an edge or noise. Specifically, if a pixel value at a certain location is significantly larger than the values ​​of its surrounding pixels, it can be reasonably assumed that this pixel value is due to noise, since no other pixels can form an edge with it; therefore, the pixel value at that location is an isolated noise point. In summary, after edge point determination, isolated noise points caused by image sensor noise can be removed.

[0043] The fifth step is equivalent to the noise removal process described above, which assigns a fixed value to the center pixel of the region based on the degree of grayscale change of the region's pixels, thus obtaining a denoised edge layer.

[0044] The sixth step is to perform Gaussian filtering on the fourth layer G4 to obtain the fifth layer G5; further Gaussian filtering on the edge distribution map can reduce the sharpening weight of noise.

[0045] Step 7: Use the fifth layer G5 to perform a weighted fusion of the original image S and the third layer G3 to obtain the final result R, i.e., R = S*(1-G5) + G3*G5. By using weighted fusion, the influence of previous noise sharpening can be removed, thus ensuring that the sharpening effect is applied only to the edges.

[0046] Steps six and seven are equivalent to the above-mentioned weighted fusion of the noise-reducing edge layer with the initial sharpening layer after the noise reduction sharpening weight processing, to form a sharpened image that only sharpens the edges.

[0047] The first four steps above are common steps in the USM general sharpening algorithm. Extracting the high-frequency part of the image can be done not only by Gaussian filtering, but also by operators such as Sobel, Roberts, Prewitt, Canny, and Laplacian. The fifth, sixth, and seventh steps are proposed in this invention to address the Poisson noise unique to image sensors, and are used to filter out the influence of noise on the sharpening effect.

[0048] Due to the discrete nature of electric charge and the uncertainty of photons, images from CMOS sensors often contain noise conforming to a Poisson distribution. Traditional sharpening algorithms simply compare pixel values ​​with a threshold to determine edges. However, this approach often leads to isolated noise points in the image being identified as edges and thus sharpened, resulting in partial image distortion. This invention uses the degree of pixel clustering to determine whether a pixel is an edge or noise. Isolated noise points are not processed; only the parts identified as edges are sharpened. Simultaneously, a Gaussian filter is applied to the edge distribution map to further reduce the sharpening weight of non-edge pixels.

[0049] Figure 2 This is a schematic diagram of an image sharpening device according to this method, which uses an FPGA to configure the CMOS sensor and acquire images. The FPGA internally includes I... 2 The module consists of a C module, an image acquisition module, an image sharpening module, and a GV interface module.

[0050] Among them, I 2 The C module receives configurations from the PC and imports them into the CMOS sensor; the image acquisition module receives commands from the PC and controls the image acquisition process of the CMOS sensor (such as exposure time and period) and the parsing of the sensor's raw data; the image sharpening module sharpens the raw image; and the GV interface module transmits the sharpened image data to the PC.

[0051] The workflow is as follows: First, the host computer or PC sends a command to I. 2 The C module configures the sensor parameters to ensure normal operation; the second step involves the host computer or PC controlling the image acquisition module to receive the raw image data from the CMOS sensor; the third step involves the image sharpening module receiving the data output by the image acquisition module and sharpening the image; and the final step involves the GV interface module receiving the sharpened image data and transmitting it to the host computer or PC via a network cable.

[0052] The FPGA's internal image sharpening module uses a pipelined approach to process image data; see [link to details]. Figure 3 Line 18 is a new line of image data output by the image acquisition module, waiting to be put into the FIFO buffer. At the same time, the FIFO stores 16 lines of image data. The earliest arriving Line 0 has been processed and can be output, while the processing core is sharpening Line 1.

[0053] FPGA processes image data in rows. Each row of the image is divided into 32 blocks, and each block is processed by a separate processing core.

[0054] In summary, to filter out noise points, this invention adds an edge detection step to the original Unsharp Mask (USM) sharpening algorithm. To further reduce the impact of noise, the mask obtained from the edge detection is subjected to Gaussian filtering. This Gaussian filtering is then used to weightedly fuse the result obtained from the traditional USM sharpening algorithm with the original image to obtain the final result. This invention removes isolated noise points through edge detection and further reduces the impact of noise on the sharpening result through weighted fusion.

[0055] Accordingly, embodiments of the present invention also disclose an image sharpening device, comprising:

[0056] The image sharpening module sharpens the original image to obtain a preliminary sharpening layer and the edge layer;

[0057] The noise removal module determines the edge points of the edge layer and assigns a fixed value to the center pixel of the region based on the degree of grayscale change of the region to remove noise, thereby obtaining a denoised edge layer.

[0058] The image fusion module performs weighted fusion on the original image, the preliminary sharpening layer, and the denoising edge layer to obtain a sharpened image that only sharpens the edges.

[0059] In another aspect, the present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of any of the methods described above.

[0060] In another aspect, the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of any of the methods described above.

[0061] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform the steps of any of the methods described in the above embodiments.

[0062] It is understood that the system provided in the embodiments of the present invention corresponds to the method provided in the embodiments of the present invention, and the explanation, examples and beneficial effects of the relevant content can be referred to the corresponding parts of the above methods.

[0063] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0064] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0065] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image sharpening method for industrial cameras, characterized in that, Includes the following steps, By enhancing the image edge weights of the input original image, a preliminary sharpening layer that highlights the image edges and an edge layer that reflects the image edges are obtained. Based on the standard of pixel grayscale change in the noise edge region, identify and remove noise regions; Using N×N small squares centered on the current pixel in the edge layer, determine the number of pixels greater than a threshold, where N is an odd number and N is less than or equal to 9; If the number of pixels greater than the threshold is less than N, the edge point is determined to be 0; if the number of pixels greater than the threshold is greater than or equal to N, the edge point is determined to be 1, thus selecting the pixel block edge recognition to obtain the noise-reduced edge layer. After the noise-reducing edge layer is processed with a sharpening weight to reduce noise, it is then weighted and fused with the initial sharpening layer to form a sharpened image that only sharpens the edges. The weighted fusion algorithm is: R = S × (1-B) + A × B; where S is the original image, R is the sharpened image with edge sharpening only, A is the initial sharpening layer, and B is the denoised edge layer.

2. The industrial camera image sharpening method according to claim 1, characterized in that, The threshold is calculated using an adaptive thresholding method, specifically including: Divide the edge layer into N×N small squares, calculate the grayscale standard deviation of each small square, sort the standard deviations, select the grayscale values ​​of the first half of the standard deviations, and calculate the average value as the threshold.

3. An image sharpening device, characterized in that, include: The image sharpening module sharpens the original image, resulting in a preliminary sharpening layer and an edge layer; The noise removal module determines the edge points of the edge layer and assigns a fixed value to the center pixel of the region based on the degree of grayscale change of the region to remove noise, thereby obtaining a denoised edge layer. Specifically, this includes: identifying and removing noise regions based on the grayscale change standard of pixels in the noise edge region; Using an N×N square centered on the current pixel in the edge layer, determine the number of pixels greater than a threshold, where N is an odd number and N is less than or equal to 9; if the number of pixels greater than the threshold is less than N, the edge point is determined to be 0; if the number of pixels greater than the threshold is greater than or equal to N, the edge point is determined to be 1, thereby selecting pixel block edge recognition to obtain a denoised edge layer. The image fusion module performs weighted fusion on the original image, the preliminary sharpening layer, and the denoising edge layer to obtain a sharpened image that only sharpens the edges. The weighted fusion algorithm is: R = S × (1-B) + A × B; where S is the original image, R is the sharpened image with edge sharpening only, A is the initial sharpening layer, and B is the denoised edge layer.

4. A computer device, characterized in that, include: Processor and memory, The memory is used to store computer programs; The processor is configured to execute the computer program stored in the memory to implement the image sharpening method as described in claim 1 or 2.

5. A storage medium, characterized in that, When the program in the storage medium is executed by a processor, the image sharpening method as described in claim 1 or 2 can be implemented.

Images