A pixel-level multi-focus image fusion method, storage medium and device

Abstract

The present application relates to a kind of pixel-level multi-focus image fusion method, storage medium and device, steps are as follows: the scene image of optical system is collected at different distances when multiple focusing points are clear;Multiple scene images are respectively separated to obtain background layer information and image sharpness information;The sharpness image of each scene image is respectively low-pass filtered to obtain filtered sharpness image;Local information fusion weight is calculated using the local information of multiple filtered sharpness images;Based on the fusion weight and sharpness image, the fused detail layer image is obtained;Finally, the background layer image and the fused detail layer image are added according to a certain proportion to obtain multi-focus image fusion image.After obtaining multi-focus image, clear image at different distances can be obtained after fusion, and the local brightness, contrast of image is excessively smooth, which is convenient for target detection, recognition, analysis and other back-end processing.The algorithm complexity is low, and it is easy to realize on different platforms.

Description

Technical Field

[0001] The present invention belongs to the technical field of image fusion, and in particular relates to a pixel-level multi-focus image fusion method, storage medium and device. Background Art

[0002] Due to the limitation of depth of field, the optical system can only clearly image the target within a certain distance range. However, in some special applications, it is necessary to obtain clear images at different distances.

[0003] There are generally two solutions to this problem:

[0004] 1) Obtain clear images at different distances by focusing. This method is convenient and fast, but it cannot obtain clear images at different distances simultaneously. 2) Add an image fusion algorithm based on method 1. This method can obtain clear images at different distances simultaneously, but the current methods are either computationally complex or have limited application scenarios. For example:

[0005] CN112702575A discloses a multi-focal image acquisition device and analysis method based on image fusion technology. This method uses two or more fixed-focus lenses for multi-focal image fusion. However, this method uses a fixed focal length lens and a relatively simple fusion mode, which can achieve good results in specific application scenarios. However, it cannot achieve good fusion when fusion between arbitrary focal lengths within a certain range is required.

[0006] CN115439376A discloses a model, method, and device for multifocal image fusion using a compound eye camera. This multifocal image fusion method, based on feature extraction and recovery, achieves good fusion results. However, this method is computationally complex and has a complicated process, making it difficult to implement on embedded platforms such as FPGAs. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to achieve smooth transition between different intervals while obtaining clear multi-focus images. To this end, the present invention provides a pixel-level multi-focus image fusion method, storage medium and device.

[0008] The technical solution adopted in the present invention is as follows:

[0009] A pixel-level multi-focus image fusion method comprises the following steps:

[0010] Step 1: According to the different focusing points of the optical system, collect scene images at different focusing points with clear targets at different distances. Suppose M focusing point images are collected, and the scene images at multiple different focusing points are denoted as II. II is expressed by the following formula:

[0011] II=[I1,I2,…I M ]

[0012] Furthermore, the size of M is determined according to the depth of field of the optical system and the actual target distance to be observed, and is generally between 2 and 20.

[0013] Step 2: Perform image separation on the scene images under multiple focus points to obtain background layer information and image clarity information. The image separation method can be layered using low-pass filters such as guided filtering, bilateral filtering, least square filtering, and Gaussian filtering. The separated images are:

[0014] II base =f([I1,I2,…I M ])=[I_base1,I_base2,…I_base M ]

[0015] II detail =II-II bas [I_detail1,I_detail2,…I_detail M ]

[0016] II base is the background layer image, II detail Image clarity image.

[0017] Whether it is guided filtering, bilateral filtering, least squares filtering or Gaussian filtering, a window size between 3 and 30 will generally produce better results.

[0018] Step 3: Perform low-pass filtering on the clarity image of each scene image to obtain the filtered clarity image. The image filtering method uses edge-preserving filters such as guided filtering, bilateral filtering, least squares filtering, etc. The filtered clarity image is:

[0019] II detail_Lf =Lf(II detail )=[I_detail_Lf1,I_detail_Lf2,…I_detail_Lf M ]

[0020] This step requires edge-preserving filtering of the clarity image in the spatial domain, and bilateral filtering achieves relatively good results.

[0021] Step 4: Calculate the fusion weight of local information using the local information of multiple filtered clarity images. The calculation formula is as follows:

[0022]

[0023] where w m(i, j) represents the detail information weight of the (i, j)th pixel in the mth focused image. Th is the threshold set for the information-free scene area, which is also the noise threshold. It needs to be determined based on the spatiotemporal threshold noise of the imaging device and is generally not greater than 15.

[0024] Step 5: Obtain the fused detail layer image based on the fusion weight and the clarity image; average multiple background layer images at the pixel level to obtain the fused background layer image. The calculation formulas are as follows:

[0025] Background layer calculation:

[0026]

[0027] Detailed layer calculation:

[0028]

[0029] Step 6: Add the fused background layer image and the fused detail layer image in a certain ratio to obtain the multi-focus image fusion image. The calculation formula is as follows:

[0030] I out =I base +α×I detail

[0031] Here, α is a coefficient that plays a role in multi-focus image fusion while achieving detail enhancement, and is generally between 0.8 and 2.

[0032] The beneficial effects of the present invention are:

[0033] 1) After acquiring the multi-focus image, this algorithm can simultaneously obtain clear images at different distances;

[0034] 2) The local brightness and contrast of the obtained fused image are smoothed, which is convenient for back-end processing such as target detection, recognition, and analysis;

[0035] 3) The algorithm complexity of this technology is low and it is easy to implement on different platforms such as FPGA, DSP, ARM, PC, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 Flowchart of the pixel-level multi-focus image fusion method of the present invention.

[0037] Figure 2 This is an effect diagram of an embodiment of the pixel-level multi-focus image fusion method of the present invention. DETAILED DESCRIPTION

[0038] The present invention is described in further detail below with reference to the embodiments.

[0039] Those skilled in the art will understand that the following examples are intended to illustrate the present invention only and should not be construed as limiting the scope of the present invention. Where specific techniques or conditions are not specified in the examples, the techniques or conditions described in the literature in the art or in the product specifications were used. Materials or equipment used without manufacturer identification are commercially available conventional products.

[0040] To illustrate the specific implementation process, it is assumed that the optical system has a multi-focal number M=3, with a value range of greater than or equal to 2; and the image collected is an infrared core with a resolution of 640×512:

[0041] Step 1: According to the different focusing points of the optical system, collect scene images at different focusing points with clear targets at different distances. Suppose M focusing point images are collected, and the scene images at multiple different focusing points are denoted as II. II is expressed by the following formula:

[0042] II=[I1,I2,I3]

[0043] Step 2: Perform image separation on the scene images under multiple focus points to obtain background layer information and image clarity information. The image separation method can be layered using low-pass filters such as guided filtering, bilateral filtering, least square filtering, and Gaussian filtering. The separated images are:

[0044] II base =f([I1,I2,I3])=[I_base1,I_base2,I_base3]

[0045] II detail =II-II base =[I_detail1,I_detail2,I_detail3]

[0046] II base is the background layer image, II detail Image clarity image.

[0047] Furthermore, the hierarchical algorithm uses guided filtering. The input image and guided image of the guided filtering algorithm are both original images. The calculation formula is as follows:

[0048] I_base i =a k I i +b k ; i=1,2,3

[0049]

[0050] where u k and σ k 2Indicates that I is in window w k The mean and variance in |w| represents the window w k The number of pixels, here the window is 7×7=49, ∈ is set to prevent the denominator from being too small or equal to 0. Generally, a better effect is achieved between 10000 and 50000, and 32768 can be used for calculation.

[0051] Step 3: Perform low-pass filtering on the clarity image of each scene image to obtain the filtered clarity image. The image filtering method uses edge-preserving filters such as guided filtering, bilateral filtering, least squares filtering, etc. The filtered clarity image is:

[0052] II detail_Lf =Lf(II detail )=[I_detail_Lf1,I_detail_Lf2,I_detail_Lf3]

[0053] This step requires edge-preserving filtering of the clarity image in the spatial domain, and bilateral filtering achieves relatively good results. The calculation formula for bilateral filtering is as follows:

[0054]

[0055] Among them G s and G r They are all standard Gaussian functions, defined as follows:

[0056]

[0057] w is the filter window, and the window size is 15×15. Take 25.

[0058] Furthermore, if the algorithm needs to be ported to FPGA, the exponential calculation of the edge-preserving filtering algorithm is replaced by a linear lookup table. Considering the experience of multiple embodiments, a maximum of 16,384 data points need to be stored to meet most exponential calculation requirements.

[0059] Step 4: Calculate the fusion weight of local information using the local information of multiple filtered clarity images. The calculation formula is as follows:

[0060]

[0061] where w m (i, j) represents the detail information weight of the (i, j)th pixel in the mth focused image. Th is the threshold set for the information-free scene area, which is also the noise threshold. It needs to be determined based on the spatiotemporal threshold noise of the imaging device. It is generally not greater than 15 and is taken as 5 here.

[0062] Furthermore, in a certain imaging system, by facing the imaging device to a uniform scene and continuously capturing X = [x1, x2, ... x L ] images to obtain the threshold, the calculation formula is as follows:

[0063]

[0064] Step 5: Obtain the fused detail layer image based on the fusion weight and the clarity image; average multiple background layer images at the pixel level to obtain the fused background layer image. The calculation formulas are as follows:

[0065] Background layer calculation:

[0066]

[0067] Detailed layer calculation:

[0068]

[0069] Step 6: Add the fused background layer image and the fused detail layer image in a certain ratio to obtain the multi-focus image fusion image. The calculation formula is as follows:

[0070] I out =I base +1.2×I detail

[0071] Among them, 1.2 is the detail gain coefficient, and the final effect is as follows Figure 2 shown.

[0072] The technical effects of this embodiment are as follows:

[0073] 1) After acquiring three images with different focal lengths, this algorithm can achieve clear images at different distances at the same time. Figure 2 shown.

[0074] 2) The local brightness and contrast of the final fused image are smoothed, which facilitates back-end processing such as target detection, recognition, and analysis.

[0075] 3) The algorithm complexity of this technology is low, making it easy to implement on different platforms such as FPGA, DSP, ARM, and PC. It also provides a solution for the exponential calculation involved in the algorithm.

[0076] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A pixel-level multi-focus image fusion method, characterized in that The steps include: Step 1: According to the different focusing points of the optical system, collect the scene images II=[I1,I2,…I M ], where M represents the number of focal points; Step 2: Perform image separation on the scene images at multiple focus points to obtain background layer information and image clarity information. The image separation method uses a low-pass filter for layering. The separated images are: II base =f([I1,I2,…I M ])=[I_base1,I_base2,…I_base M ]; II detail =II-II base =[I_detail1,I_detail2,…I_detail M ]; Among them: II base is the background layer image, II detail Image clarity image, f represents the image layering method, I_base1, I_base2, ... I_base M Represent the background layer images at the 1st, 2nd...Mth focus points, I_detail1, I_detail2,...I_detail M Represent the clarity images at the 1st, 2nd…Mth focal points respectively; Step 3: Perform low-pass filtering on the definition image of each scene image to obtain a filtered definition image; the filtered definition image is: II detail_Lf =Lf(II detail )=[I_detail_Lf1,I_detail_Lf2,…I_detail_Lf M ]; Among them: II detail_Lf Represents the clarity image after filtering, Lf represents the low-pass filter, I_detail_Lf1, I_detail_Lf2, ... I_detail_Lf M Represent the clarity images after filtering at the 1st, 2nd…Mth focus points respectively; Step 4: Calculate the fusion weight of local information using the local information of multiple filtered clarity images. The calculation formula is as follows: Where: w m (i, j) represents the detail information weight of the (i, j)th pixel in the mth focused image, Th is the threshold set for the uninformative scene area, and is also the noise threshold; Step 5: Obtain a fused detail layer image based on the fusion weight and the clarity image; average multiple background layer images at the pixel level to obtain a fused background layer image; Step 6: Add the fused background layer image and the fused detail layer image in a certain ratio to obtain a multi-focus image fusion image.

2. A pixel-level multi-focus image fusion method according to claim 1, characterized in that : The low-pass filter in step 2 includes guided filtering, bilateral filtering, least squares filtering or Gaussian filtering.

3. A pixel-level multi-focus image fusion method according to claim 1, characterized in that : The calculation formulas for the detail layer image and the background layer image in step 5 are as follows: Background layer calculation: Detailed layer calculation:

4. A pixel-level multi-focus image fusion method according to any one of claims 1 to 3, characterized in that : The multi-focus image fusion image in step 6 is calculated using the following formula: I out =I base +α×I detail Where: α is a coefficient, and its value is between 0.8 and 2, which plays the role of multi-focus image fusion and detail enhancement.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the steps of a pixel-level multi-focus image fusion method as described in any one of claims 1 to 4.

6. A pixel-level multi-focus image fusion device, characterized in that: The method comprises a processor and a computer-readable storage medium as claimed in claim 5 .

Images