Multi-focus image depth-of-field fusion method and system, computer readable storage medium and computer program product

Abstract

The invention relates to the technical field of image processing, and discloses a multi-focus image depth-of-field fusion method and system, a computer readable storage medium and a computer program product. The method comprises the following steps: aligning a plurality of input focusing images under the same coordinate system; carrying out the preprocessing of the plurality of aligned focusing images, and obtaining a plurality of denoised grayscale images; calculating the definition score of each pixel position on all gray level images, and generating a definition mask image; and according to the definition mask image, carrying out image fusion on the plurality of aligned focusing images. The invention provides an efficient and accurate multi-focus image depth-of-field fusion method, and solves the problems that the alignment precision is insufficient, the fusion effect is not ideal, the calculation complexity is high and the like. Especially when a complex scene is processed, the method can effectively solve the problems of image distortion, edge artifacts and the like.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, system, computer-readable storage medium and computer program product for multi-focus image depth-of-field fusion. Background Art

[0002] Multi-focus Image Fusion is one of the common technologies in the field of image processing. It aims to obtain a comprehensive image containing all the clear areas of the image by fusing multiple images with different focuses. This technology is widely used in many fields such as medical imaging, autonomous driving, monitoring, remote sensing imaging, etc., especially in generating clear panoramic images, enhancing image details and improving visual perception.

[0003] In practical applications, we often need to take a series of images with different focus positions to cover the focus of all objects in the scene. However, due to the different focal lengths of different images, the clarity of objects at different positions in the scene varies in different images. Usually, some objects may be clear in one image and blurry in other images. Therefore, when fusing these images, how to ensure the clarity of all objects and effectively remove unclear areas is a crucial problem. In addition, due to changes in shooting equipment, image focus, and imaging angle, different images taken may deviate in spatial position and direction. This difference will cause geometric transformation problems such as displacement, rotation, and scale change between images, thus affecting the fusion results.

[0004] Traditional fusion methods such as weighted average method and maximum response method, although simple and easy, cannot fully consider the detailed information of the image and the focus clarity of each area. Therefore, how to use the local information of the image, especially the clarity information, for effective fusion becomes the key to improving the fusion effect. In addition, in multi-focus image fusion, edge artifacts caused by focus conversion or alignment errors are often encountered. In addition, since the image fusion process involves the overlap of local areas of different images, how to avoid edge effects during image stitching and ensure the details and continuity of the fusion results is also a major challenge.

[0005] Existing multi-focus image fusion technologies can be roughly divided into the following categories:

[0006] 1) Pixel-based fusion method: This method mainly performs image fusion by weighted averaging or selecting the maximum response value at the pixel level. Although such methods are simple and easy to use, they ignore the complexity of local information in the image, especially the rough processing of complex textures and edges, which often leads to the loss of image details or the generation of artifacts.

[0007] 2) Feature-based fusion method: This method extracts image features (such as edges, corners, textures, etc.) and then fuses images based on the importance of the features. Although this method can preserve image details to a certain extent, it requires high accuracy in image registration, and may result in inaccurate feature matching and unsatisfactory fusion effects when processing more complex image scenes.

[0008] 3) Fusion methods based on wavelet transform or Laplace pyramid: This type of method decomposes the image into multiple frequency bands (such as low-frequency and high-frequency components), and then fuses them according to the clarity of different frequency bands. Wavelet transform and Laplace pyramid methods have good effects on image detail processing, but they also have the problem of high computational complexity and the need to decompose and reconstruct the image multiple times, which limits their application in scenarios with high real-time requirements.

[0009] 4) Deep learning methods: With the development of deep learning, neural network-based image fusion methods have gradually become mainstream. Deep learning methods can automatically extract features from data and fuse them by learning image fusion patterns in large-scale data sets. However, deep learning methods often require a large amount of training data and computing resources, and there is a certain delay and computing burden when applied to real-time processing.

[0010] In summary, most of the existing multi-focus image fusion methods have problems such as insufficient image alignment accuracy, unsatisfactory fusion effect, and high computational complexity. Especially when dealing with complex scenes, existing methods still cannot effectively solve challenges such as image distortion and edge artifacts. A new image fusion method is urgently needed to improve image clarity and fusion effect. Summary of the invention

[0011] The object of the present invention is to provide a method, system, computer-readable storage medium and computer program product for multi-focus image depth of field fusion, so as to solve or at least partially solve the technical problems mentioned in the above background technology.

[0012] To achieve this object, the present invention adopts the following technical solutions:

[0013] In a first aspect, the present invention provides a method for multi-focus image depth of field fusion, comprising:

[0014] Align multiple input focused images in the same coordinate system;

[0015] Preprocessing the aligned multiple focused images to obtain multiple grayscale images after denoising;

[0016] Calculate the clarity score of each pixel position on all grayscale images and generate a clarity mask image;

[0017] According to the clarity mask image, the aligned multiple focused images are fused.

[0018] Optionally, calculating the clarity score of each pixel position on all grayscale images to generate a clarity mask image specifically includes:

[0019] Calculate the clarity score of each pixel position on all grayscale images and construct a clarity curve for each pixel position;

[0020] The clarity mask corresponding to each pixel position is determined according to the clarity curve of each pixel position, and a clarity mask image is generated.

[0021] Optionally, the clarity score is calculated as:

[0022] For the pixel position (x, y), it is in the i-th grayscale image G i C on clarity rating i (x, y), set to the i-th grayscale image G i The average value of the clarity score of all pixels in the local area R (x, y) is expressed as:

[0023]

[0024] Where R(x, y) is a local area with a window size of k×k centered at the pixel position (x, y), k is a pre-set parameter value; (x′, y′) is a pixel in the local area R(x, y), and the Laplacian operator is the second-order derivative operator;

[0025] The clarity curve is expressed as:

[0026] {C′1(x,y),C′2(x,y),...,C′ n (x, y)}; where n is the total number of grayscale images.

[0027] Optionally, the determining the clarity mask corresponding to each pixel position according to the clarity curve of each pixel position and generating the clarity mask image specifically includes:

[0028] Finding a maximum value in the clarity curve, and calculating an average value of the clarity curve;

[0029] According to the comparison between the maximum value found and the average value, the pixel value of the sharpness mask image is determined, thereby generating the sharpness mask image;

[0030] Among them, the maximum value P = C' j (x, y), the following conditions must be met:

[0031] C′ j (x, y)>C′ j-1 (x, y), and C′ j (x, y)>C′ j+1 (x, y), 2≤j≤n-1;

[0032] The average value of the clarity curve P mean The calculation formula is:

[0033]

[0034] The step of determining the pixel value of the definition mask image based on the comparison between the maximum value found and the average value, and then generating the definition mask image, specifically includes:

[0035] Determine the clarity mask of the clarity mask image at each pixel position according to the comparison between the found maximum value and the average value;

[0036] Generate a clarity mask image according to the clarity mask at each pixel position of the clarity mask image;

[0037] Among them, the comparison process of the maximum value and the average value includes:

[0038] If the maximum value P>P mean If the number is equal to 0, the sharpness mask M(x, y) at the pixel position (x, y) is:

[0039]

[0040] If the maximum value P>P mean If the number of times is equal to 1, then the sequence number l of the image where the maximum value is located is taken as the pixel value of the clarity mask image at the position (x, y):

[0041] M(x, y) = 1;

[0042] If the maximum value P>P mean If the number of is greater than 1, the sequence number m of the image where the last maximum value is located is taken as the pixel value of the clarity mask image at position (x, y):

[0043] M(x, y) = m.

[0044] Optionally, the value range of k is [3-99].

[0045] Optionally, before fusing the aligned multiple focused images according to the clarity mask image, the method further includes:

[0046] The clarity mask image is subjected to smoothing filtering, which specifically includes:

[0047] First, median filtering is applied to the clarity mask image to obtain M med (x, y), the method is as follows:

[0048] M med (x, y)=median({M(x′,y′)|(x′,y′)∈W(x,y)});

[0049] Where W(x, y) is a k×k window centered at the pixel position (x, y), M(x′, y′) is the value of all pixels in the window, and median() represents the operation of selecting the median from the window;

[0050] Then apply Gaussian filtering to the result after median filtering to obtain a smoothed clarity mask image as follows:

[0051] M smooth (x, y) = ∑ (x′，y′)∈W(x，y) G(x′-x,y′-y)·M med (x′,y′);

[0052] The calculation formula of the Gaussian function weight is:

[0053]

[0054] Among them, M smooth (x, y) is the pixel value of the sharpness mask image at position (x, y); σ is the standard deviation of the Gaussian filter.

[0055] Optionally, the image fusion of the aligned multiple focused images according to the clarity mask image is specifically implemented as follows: according to the smoothed clarity mask image, the image fusion of the aligned multiple focused images is performed, and the clearest image at each pixel position is combined into a fused image; specifically including:

[0056] According to the pixel value of each pixel position of the smoothed clarity mask image, the corresponding image is selected for fusion; the fused image generation formula is:

[0057]

[0058] Where: F(x, y) is the pixel value of the fused image at position (x, y); M smooth (x, y) represents the serial number of the focused image corresponding to the clearest image at position (x, y); is the Mth in the aligned multifocus image smooth The pixel value of the (x, y) focused image at position (x, y).

[0059] Optionally, after fusing the aligned multiple focused images according to the clarity mask image, the method further includes:

[0060] Post-processing operations are performed on the fused image, including:

[0061] Calculate the minimum and maximum values ​​of the fused image; the calculation method is:

[0062]

[0063] Perform histogram stretching on each pixel F(x, y) to map the minimum and maximum values ​​of the fused image to the target range [L min , L max ];

[0064]

[0065] Among them: F stretch (x, y) is the pixel value of the stretched image, F(x, y) is the pixel value of the original fused image; F min and F max are the minimum and maximum values ​​of the original fused image, L min and L max are the minimum and maximum values ​​of the target range, respectively.

[0066] Optionally, aligning the input multiple focused images in the same coordinate system specifically includes:

[0067] Assume that the total number of input focused images is n, and the n focused images are denoted as: I1, I2, ..., I n , the pixel value of the i-th image at position (x, y) is represented by I i (x, y) indicates that (x, y) is the spatial coordinate of the focused image;

[0068] Select one of the n focused images as the reference image I ref ;

[0069] For each image I i , which is consistent with the reference image I ref The transformation between is expressed as an affine transformation model:

[0070] I i (x, y) = T i I ref (x′, y′);

[0071] Among them, T i is the transformation matrix, representing image I i The transformation relative to the reference image is expressed as:

[0072]

[0073] Among them, a, b, c, d represent transformation parameters, including rotation and scaling; tx, ty represent translation parameters;

[0074] The ECC alignment method is used to estimate the transformation matrix by optimizing the objective function. ref and I i , the ECC similarity measurement function is expressed as:

[0075]

[0076] Among them, T i (x, y) is the transformed image I i The pixel position in x，y It means summing all pixel positions;

[0077] Optimize the transformation matrix T through an iterative optimization algorithm i , so that the objective function ECC(T i ) maximize; transform parameter T i It will be updated step by step according to the alignment error between each image and the reference image until the similarity reaches the maximum. The final transformation matrix T i Will be applied to image I i , align it with the reference image, and obtain an image set after all focused images are aligned in the same coordinate system;

[0078] Among them, the pixel value of the focused image at position (x, y) in the aligned image set is expressed as:

[0079] I′ i (x, y) = T i I i (x, y).

[0080] Optionally, preprocessing the aligned multiple focused images to obtain multiple denoised grayscale images specifically includes:

[0081] For each aligned focused image, if its image type is a color image, it is converted into a grayscale image using the weighted average method; the grayscale value of the i-th grayscale image at position (x, y) is expressed as:

[0082] G i (x, y)=0.2989·R(x,y)+0.5870·G(x,y)+0.1140·B(x,y); where, for

[0083] The pixel values ​​of the red, green, and blue channels of the aligned focused image at position (x, y) are denoted as R(x, y), G(x, y), and B(x, y), respectively;

[0084] Perform mean filtering on each grayscale image to remove noise; the specific implementation is as follows:

[0085] Assuming that a filter window of size k×k is used, the mean filter formula is:

[0086]

[0087] Among them, (x+m, y+n) represents the neighborhood pixels within the filter window.

[0088] In a second aspect, the present invention provides a system for multi-focus image depth of field fusion, comprising:

[0089] The image processing module is used to align the input multiple focused images in the same coordinate system; and is also used to pre-process the aligned multiple focused images to obtain multiple grayscale images after denoising;

[0090] A mask image construction module, electrically connected to the image processing module, for calculating a clarity score of each pixel position on all grayscale images to generate a clarity mask image;

[0091] The image fusion module is electrically connected to the mask image construction module and the image processing module, and is used to perform image fusion on the aligned multiple focused images according to the clarity mask image.

[0092] Optionally, the mask image construction module is specifically used to calculate the clarity score of each pixel position on all grayscale images and construct a clarity curve for each pixel position; and is specifically used to determine the clarity mask corresponding to each pixel position according to the clarity curve of each pixel position, and generate a clarity mask image.

[0093] Optionally, the clarity score is calculated as:

[0094] For the pixel position (x, y), it is in the i-th grayscale image G i Clarity rating on C i(x,y), set to the i-th grayscale image G i The average value of the clarity score of all pixels in the local area R(x,y) is expressed as:

[0095]

[0096] Where R(x,y) is a local area with a window size of k×k centered at the pixel position (x,y), k is a pre-set parameter value; (x′, y′) is a pixel in the local area R(x,y), and the Laplace operator is the second-order derivative operator;

[0097] The clarity curve is expressed as:

[0098] {C′1(x,y),C′2(x,y),...,C′ n (x, y)}; where n is the total number of grayscale images.

[0099] Optionally, the determining the clarity mask corresponding to each pixel position according to the clarity curve of each pixel position and generating the clarity mask image specifically includes:

[0100] Finding a maximum value in the clarity curve, and calculating an average value of the clarity curve;

[0101] According to the comparison between the maximum value found and the average value, the pixel value of the sharpness mask image is determined, thereby generating the sharpness mask image;

[0102] Among them, the maximum value P = C' j (x, y), the following conditions must be met:

[0103] C′ j (x, y)>C′ j-1 (x, y), and C′ j (x, y)>C′ j+1 (x, y), 2≤j≤n-1;

[0104] The average value of the clarity curve P mean The calculation formula is:

[0105]

[0106] The step of determining the pixel value of the definition mask image based on the comparison between the maximum value found and the average value, and then generating the definition mask image, specifically includes:

[0107] Determine the clarity mask of the clarity mask image at each pixel position according to the comparison between the found maximum value and the average value;

[0108] Generate a clarity mask image according to the clarity mask at each pixel position of the clarity mask image;

[0109] Among them, the comparison process of the maximum value and the average value includes:

[0110] If the maximum value P>P meanIf the number is equal to 0, the sharpness mask M(x, y) at the pixel position (x, y) is:

[0111]

[0112] If the maximum value P>P mean If the number of times is equal to 1, then the sequence number l of the image where the maximum value is located is taken as the pixel value of the clarity mask image at the position (x, y):

[0113] M(x, y) = 1;

[0114] If the maximum value P>P mean If the number of is greater than 1, the sequence number m of the image where the last maximum value is located is taken as the pixel value of the clarity mask image at position (x, y):

[0115] M(x, y) = m.

[0116] Optionally, the value range of k is [3-99].

[0117] Optionally, the image fusion module is electrically connected to a smoothing filter module, and the smoothing filter module is electrically connected to a mask image construction module;

[0118] The smoothing filter module is used to perform smoothing filter processing on the clarity mask image; specifically comprising:

[0119] First, median filtering is applied to the clarity mask image to obtain M med (x, y), the method is as follows:

[0120] M med (x, y)=median({M(x′,y′)|(x′,y′)∈W(x,y)});

[0121] Where W(x,y) is a k×k window centered at the pixel position (x,y), M(x′, y′) is the value of all pixels in the window, and median() represents the operation of selecting the median from the window;

[0122] Then apply Gaussian filtering to the result after median filtering to obtain a smoothed clarity mask image as follows:

[0123] M smooth (x, y) = ∑ (x′，y′)∈W(x，y) G(x′-x,y′-y)·M med (x′, y′);

[0124] The calculation formula of the Gaussian function weight is:

[0125]

[0126] Among them, M smooth (x, y) is the pixel value of the sharpness mask image at position (x, y); σ is the standard deviation of the Gaussian filter.

[0127] Optionally, the image fusion module is specifically used to perform image fusion on the aligned multiple focused images according to the smoothed clarity mask image, and combine the clearest image at each pixel position into a fused image; specifically comprising:

[0128] According to the pixel value of each pixel position of the smoothed clarity mask image, the corresponding image is selected for fusion; the fused image generation formula is:

[0129]

[0130] Where: F(x, y) is the pixel value of the fused image at position (x, y); M smooth (x, y) represents the serial number of the focused image corresponding to the clearest image at position (x, y); is the Mth in the aligned multifocus image smooth The pixel value of the (x, y) focused image at position (x, y).

[0131] Optionally, the image fusion module is further electrically connected to an image post-processing module; the image post-processing module is used to perform post-processing operations on the fused image, specifically including:

[0132] Calculate the minimum and maximum values ​​of the fused image; the calculation method is:

[0133]

[0134] Perform histogram stretching on each pixel F(x, y) to map the minimum and maximum values ​​of the fused image to the target range [L min , L max ];

[0135]

[0136] Among them: F stretch (x, y) is the pixel value of the stretched image, F(x, y) is the pixel value of the original fused image; F min and F max are the minimum and maximum values ​​of the original fused image, L min and L max are the minimum and maximum values ​​of the target range, respectively.

[0137] Optionally, aligning the input multiple focused images in the same coordinate system specifically includes:

[0138] Assume that the total number of input focused images is n, and the n focused images are denoted as: I1, I2, ..., I n , the pixel value of the i-th image at position (x, y) is represented by I i (x, y) indicates that (x, y) is the spatial coordinate of the focused image;

[0139] Select one of the n focused images as the reference image I ref ;

[0140] For each image I i , which is consistent with the reference image I ref The transformation between is expressed as an affine transformation model:

[0141] I i (x, y) = T i I ref (x′, y′);

[0142] Among them, T i is the transformation matrix, representing image I i The transformation relative to the reference image is expressed as:

[0143]

[0144] Among them, a, b, c, d represent transformation parameters, including rotation and scaling; tx, ty represent translation parameters;

[0145] The ECC alignment method is used to estimate the transformation matrix by optimizing the objective function. ref and I i , the ECC similarity measurement function is expressed as:

[0146]

[0147] Among them, T i (x, y) is the transformed image I i The pixel position in x，y It means summing all pixel positions;

[0148] Optimize the transformation matrix T through an iterative optimization algorithm i , so that the objective function ECC(T i ) maximize; transform parameter T i It will be updated step by step according to the alignment error between each image and the reference image until the similarity reaches the maximum. The final transformation matrix T i Will be applied to image I i , align it with the reference image, and obtain an image set after all focused images are aligned in the same coordinate system;

[0149] Among them, the pixel value of the focused image at position (x, y) in the aligned image set is expressed as:

[0150] I′ i (x, y) = T i I i (x, y).

[0151] Optionally, preprocessing the aligned multiple focused images to obtain multiple denoised grayscale images specifically includes:

[0152] For each aligned focused image, if its image type is a color image, it is converted into a grayscale image using the weighted average method; the grayscale value of the i-th grayscale image at position (x, y) is expressed as:

[0153] G i (x, y)=0.2989·R(x,y)+0.5870·G(x,y)+0.1140·B(x,y); where, for

[0154] The pixel values ​​of the red, green, and blue channels of the aligned focused image at position (x, y) are denoted as R(x, y), G(x, y), and B(x, y), respectively;

[0155] Perform mean filtering on each grayscale image to remove noise; the specific implementation is as follows:

[0156] Assuming that a filter window of size k×k is used, the mean filter formula is:

[0157]

[0158] Among them, (x+m, y+n) represents the neighborhood pixels within the filter window.

[0159] In a third aspect, the present invention further provides a computer-readable storage medium, in which at least one instruction is stored, and the instruction is loaded and executed by a processor to implement the method for multi-focus image depth of field fusion as described above.

[0160] In a fourth aspect, the present invention further provides a computer program product, comprising a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, the above-mentioned method of multi-focus image depth of field fusion is implemented.

[0161] Compared with the prior art, the present invention has the following beneficial effects:

[0162] The present invention provides an efficient and accurate multi-focus image depth of field fusion method, which solves the problems of insufficient alignment accuracy, unsatisfactory fusion effect, and high computational complexity; especially when processing complex scenes, the method can effectively solve image distortion, edge artifacts, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0163] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0164] Figure 1 A flowchart of a method for multi-focus image depth of field fusion provided by an embodiment of the present invention.

[0165] Figure 2 A schematic diagram of the architecture of a system for multi-focus image depth-of-field fusion provided in an embodiment of the present invention.

[0166] Figure 3 A schematic diagram of multiple focused images to be aligned provided by an embodiment of the present invention.

[0167] Figure 4 A schematic diagram of multiple aligned focused images provided by an embodiment of the present invention.

[0168] Figure 5 An original image of a local area of ​​a focused image provided by an embodiment of the present invention.

[0169] Figure 6 For Figure 5 The image obtained after mean filtering 5*5.

[0170] Figure 7a A schematic diagram of a clarity curve provided by an embodiment of the present invention.

[0171] Figure 7b A schematic diagram of another clarity curve provided by an embodiment of the present invention.

[0172] Figure 8 A clarity mask image provided by an embodiment of the present invention.

[0173] Fig. 9 A definition mask image after smoothing filtering is provided in an embodiment of the present invention.

[0174] Fig.10 A schematic diagram of a fused image provided by an embodiment of the present invention.

[0175] Fig.11aA fused image after histogram stretching provided by an embodiment of the present invention.

[0176] Fig.11b This is the fused image obtained based on the traditional pixel fusion method.

[0177] In the figure:

[0178] 10. Image processing module; 20. Mask image construction module; 30. Image fusion module; 40. Smoothing filter module; 50. Image post-processing module. DETAILED DESCRIPTION

[0179] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0180] Embodiment 1:

[0181] Please refer to Figure 1 , Figure 1 A flowchart of a method for multi-focus image depth of field fusion provided by an embodiment of the present invention, the method specifically includes:

[0182] Step 110: Align the input multiple focused images in the same coordinate system.

[0183] Specifically, let the total number of input focused images be n, and the n focused images are sequentially recorded as: I1, I2, ..., I n , the pixel value of the i-th image at position (x, y) is represented by I i (x, y) indicates that (x, y) is the spatial coordinate of the focused image;

[0184] Select one of the n focused images as the reference image I ref ;

[0185] For each image I i , which is consistent with the reference image I ref The transformation between is expressed as an affine transformation model:

[0186] I i (x, y) = T i I ref (x′, y′);

[0187] Among them, T i is the transformation matrix, representing image Ii The transformation relative to the reference image is expressed as:

[0188]

[0189] Among them, a, b, c, d represent transformation parameters, including rotation and scaling; tx, ty represent translation parameters;

[0190] The ECC alignment method is used to estimate the transformation matrix by optimizing the objective function. ref and I i , the ECC similarity measurement function is expressed as:

[0191]

[0192] Among them, T i (x, y) is the transformed image I i The pixel position in x，y It means summing all pixel positions;

[0193] Optimize the transformation matrix T through an iterative optimization algorithm i , so that the objective function ECC(T i ) maximize; transform parameter T i It will be updated step by step according to the alignment error between each image and the reference image until the similarity reaches the maximum. The final transformation matrix T i Will be applied to image I i , align it with the reference image, and obtain an image set after all focused images are aligned in the same coordinate system;

[0194] Among them, the pixel value of the focused image at position (x, y) in the aligned image set is expressed as:

[0195] I′ i (x, y) = T i I i (x, y);

[0196] For example, please refer to Figure 3 and Figure 4 , Figure 3 A schematic diagram of multiple focused images to be aligned provided by an embodiment of the present invention, Figure 4 for Figure 3 The image set after the multiple focused images in are aligned.

[0197] Step 120: pre-process the aligned multiple focused images to obtain multiple denoised grayscale images.

[0198] Specifically, for each aligned focused image, if its image type is a color image, it is converted into a grayscale image using the weighted average method; the grayscale value of the i-th grayscale image at position (x, y) is expressed as:

[0199] G i (x, y)=0.2989·R(x,y)+0.5870·G(x,y)+0.1140·B(x,y); where, for

[0200] The pixel values ​​of the red, green, and blue channels of the aligned focused image at position (x, y) are denoted as R(x, y), G(x, y), and B(x, y), respectively;

[0201] Perform mean filtering on each grayscale image to remove noise; the specific implementation is as follows:

[0202] Assuming that a filter window of size k×k is used, the mean filter formula is:

[0203]

[0204] Among them, (x+m, y+n) represents the neighborhood pixels within the filter window;

[0205] For example, please refer to Figure 5 and Figure 6 , Figure 5 An original image of a local area of ​​a focused image provided in this embodiment, Figure 6 For Figure 5 The image obtained after mean filtering 5*5.

[0206] Step 130: Calculate the clarity score of each pixel position on all grayscale images to generate a clarity mask image.

[0207] Specifically, step 130 includes:

[0208] Step 131: Calculate the clarity score of each pixel position on all grayscale images and construct a clarity curve for each pixel position.

[0209] Step 132: Determine the clarity mask corresponding to each pixel position according to the clarity curve of each pixel position, and generate a clarity mask image.

[0210] In step 131, the clarity score is calculated as follows:

[0211] For the pixel position (x, y), it is in the i-th grayscale image G i The clarity score C′ i (x, y), set to the i-th grayscale image G iThe average value of the clarity score of all pixels in the local area R(x,y) is expressed as:

[0212]

[0213] Where R(x, y) is a local area with a window size of k×k centered at the pixel position (x, y), k is a pre-set parameter value; (x′, y′) is a pixel in the local area R(x, y), and the Laplacian operator is the second-order derivative operator;

[0214] The clarity curve is expressed as:

[0215] {C′1(x,y),C′2(x,y),...,C′ n (x, y)}; where n is the total number of grayscale images.

[0216] In this embodiment, the value range of k is [3-99].

[0217] For example, Figure 7a A schematic diagram of a clarity curve provided by an embodiment of the present invention, Figure 7b A schematic diagram of another clarity curve provided by an embodiment of the present invention.

[0218] Step 132 specifically includes:

[0219] Finding a maximum value in the clarity curve, and calculating an average value of the clarity curve;

[0220] According to the comparison between the maximum value found and the average value, the pixel value of the sharpness mask image is determined, thereby generating the sharpness mask image;

[0221] Among them, the maximum value P = C' j (x, y), the following conditions must be met:

[0222] C′ j (x, y)>C′ j-1 (x, y), and C′ j (x, y)>C′ j+1 (x, y), 2≤j≤n-1;

[0223] The average value of the clarity curve P mean The calculation formula is:

[0224]

[0225] The step of determining the pixel value of the definition mask image based on the comparison between the maximum value found and the average value, and then generating the definition mask image, specifically includes:

[0226] Determine the clarity mask of the clarity mask image at each pixel position according to the comparison between the found maximum value and the average value;

[0227] Generate a clarity mask image according to the clarity mask at each pixel position of the clarity mask image;

[0228] Among them, the comparison process of the maximum value and the average value includes:

[0229] If the maximum value P>P mean If the number is equal to 0, the sharpness mask M(x, y) at the pixel position (x, y) is:

[0230]

[0231] If the maximum value P>P mean If the number of times is equal to 1, then the sequence number l of the image where the maximum value is located is taken as the pixel value of the clarity mask image at the position (x, y):

[0232] M(x, y) = 1;

[0233] If the maximum value P>P mean If the number of is greater than 1, the sequence number m of the image where the last maximum value is located is taken as the pixel value of the clarity mask image at position (x, y):

[0234] M(x, y) = m.

[0235] For example, please refer to Figure 8 , Figure 8 A clarity mask image provided by an embodiment of the present invention.

[0236] Step 140: Perform smoothing filtering on the clarity mask image.

[0237] Specifically, firstly, median filtering is applied to the clarity mask image to obtain M med (x, y), the method is as follows:

[0238] M med (x, y)=median({M(x′,y′)|(x′,y′)∈W(x,y)});

[0239] Where W(x,y) is a k×k window centered at the pixel position (x,y), M(x′, y′) is the value of all pixels in the window, and median() represents the operation of selecting the median from the window;

[0240] Then apply Gaussian filtering to the result after median filtering to obtain a smoothed clarity mask image as follows:

[0241] M smooth (x, y) = ∑ (x′，y′)∈W(x，y) G(x′-x,y′-y)·M med (x′, y′);

[0242] The calculation formula of the Gaussian function weight is:

[0243]

[0244] Among them, M smooth (x, y) is the pixel value of the sharpness mask image at position (x, y); σ is the standard deviation of the Gaussian filter.

[0245] For example, please refer to Fig. 9 , Fig. 9 A definition mask image after smoothing filtering is provided in an embodiment of the present invention.

[0246] Step 150: perform image fusion on the aligned multiple focused images according to the smoothed clarity mask image.

[0247] Step 150 specifically includes:

[0248] According to the pixel value of each pixel position of the smoothed clarity mask image, the corresponding image is selected for fusion; the fused image generation formula is:

[0249]

[0250] Where: F(x, y) is the pixel value of the fused image at position (x, y); M smooth (x, y) represents the serial number of the focused image corresponding to the clearest image at the position (x, y); is the Mth in the aligned multifocus image smooth The pixel value of the (x, y) focused image at position (x, y);

[0251] Then the clearest images at each pixel position are combined into a fused image F.

[0252] For example, please refer to Fig.10 , Fig.10 A schematic diagram of a fused image provided by an embodiment of the present invention.

[0253] Step 160: Perform post-processing operations on the fused image.

[0254] Step 160 specifically includes:

[0255] Calculate the minimum and maximum values ​​of the fused image; the calculation method is:

[0256]

[0257] Perform histogram stretching on each pixel F(x, y) to map the minimum and maximum values ​​of the fused image to the target range [L min , L max ];

[0258]

[0259] Among them: F stretch (x, y) is the pixel value of the stretched image, F(x, y) is the pixel value of the original fused image; F min and F max are the minimum and maximum values ​​of the original fused image, L min and L max are the minimum and maximum values ​​of the target range, respectively.

[0260] For example, please refer to Fig.11a and Fig.11b ; Fig.11a A fused image after histogram stretching provided by an embodiment of the present invention, Fig.11b is the fused image obtained based on the traditional pixel fusion method (maximum response value);

[0261] By comparing the fusion effects of the two figures, it can be seen that the multi-focus image depth of field fusion method proposed in this embodiment has a better fusion effect.

[0262] In summary, this embodiment provides an efficient and accurate multi-focus image depth of field fusion method to solve the problems of insufficient alignment accuracy, unsatisfactory fusion effect, high computational complexity, etc. Especially when processing complex scenes, this method can effectively solve image distortion, edge artifacts, etc. It has at least the following beneficial effects:

[0263] 1) Improve image clarity and detail retention: Traditional multi-focus image fusion methods (pixel-based fusion methods) often cause the final fused image to lose details or become blurred in some areas due to inaccurate clarity assessment. This embodiment can effectively retain the details of each focused area through local clarity calculation and clarity mask generation, ensuring the optimal clarity of each area in the fused image, and avoiding the blur and detail loss common in traditional methods. Compared with methods such as wavelet transform and Laplace pyramid, the method of the present invention does not require multiple decomposition and reconstruction of the image, has low computational complexity, and can be applied in scenarios with high real-time requirements.

[0264] 2) Edge smoothing and natural fusion effect: Based on the clarity mask image, an edge smoothing algorithm is applied to reduce the transition mutation between focus areas, avoid hard boundaries or fusion artifacts, and ensure that the edges of the fused image are smooth and natural.

[0265] Embodiment 2:

[0266] Please refer to Figure 2 , Figure 2 The schematic diagram of the architecture of a system for multi-focus image depth fusion provided by an embodiment of the present invention. The system specifically includes:

[0267] The image processing module 10 is used to align the input multiple focused images in the same coordinate system; and is also used to pre-process the aligned multiple focused images to obtain multiple grayscale images after denoising;

[0268] The mask image construction module 20 is electrically connected to the image processing module 10 and is used to calculate the clarity score of each pixel position on all grayscale images to generate a clarity mask image;

[0269] The image fusion module 30 is electrically connected to the mask image construction module and the image processing module, and is used to perform image fusion on the aligned multiple focused images according to the clarity mask image.

[0270] Specifically, the mask image construction module 20 is specifically used to calculate the clarity score of each pixel position on all grayscale images and construct a clarity curve for each pixel position; and, specifically to determine the clarity mask corresponding to each pixel position according to the clarity curve of each pixel position, and generate a clarity mask image.

[0271] The image fusion module 30 is electrically connected to the smoothing filter module 40, and the smoothing filter module 40 is electrically connected to the mask image construction module 20;

[0272] The smoothing filter module 40 is used to perform smoothing filter processing on the clarity mask image;

[0273] The image fusion module 30 is specifically used to perform image fusion on the aligned multiple focused images according to the smoothed clarity mask image, and combine the clearest image at each pixel position into a fused image;

[0274] Specifically, the image fusion module 30 is also electrically connected to an image post-processing module 50 ; the image post-processing module 50 is used to perform post-processing operations on the fused image.

[0275] Since the first embodiment has disclosed the details of the relevant method execution steps and algorithms in detail, the same contents will not be repeated in this embodiment.

[0276] This embodiment provides an efficient and accurate multi-focus image depth of field fusion system to solve the problems of insufficient alignment accuracy, unsatisfactory fusion effect, high computational complexity, etc. Especially when processing complex scenes, this system can effectively solve image distortion, edge artifacts, etc.

[0277] Embodiment three:

[0278] This embodiment also provides a computer-readable storage medium, in which at least one instruction is stored. The instruction is loaded and executed by a processor to implement a method for multi-focus image depth of field fusion as described in the first embodiment.

[0279] Since the method for depth-of-field fusion of multi-focus images has been described in detail in the first embodiment, it will not be repeated in this embodiment.

[0280] Embodiment 4:

[0281] The present invention also provides a computer program product, including a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, a method for multi-focus image depth of field fusion as described in Example 1 is implemented.

[0282] Since the method for depth-of-field fusion of multi-focus images has been described in detail in the first embodiment, it will not be repeated in this embodiment.

[0283] Those skilled in the art can understand that all or part of the steps of the above embodiments can be completed by hardware, or by a program to instruct the relevant hardware to complete, and the program can be stored in a computer-readable storage medium, and the storage medium mentioned above can be a read-only memory, a disk or an optical disk, etc. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0284] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for multi-focus image depth of field fusion, characterized in that: include: Align multiple input focused images in the same coordinate system; Preprocessing the aligned multiple focused images to obtain multiple grayscale images after denoising; Calculate the clarity score of each pixel position on all grayscale images and generate a clarity mask image; According to the clarity mask image, the aligned multiple focused images are fused.

2. The method for multi-focus image depth of field fusion according to claim 1, characterized in that: The step of calculating the clarity score of each pixel position on all grayscale images to generate a clarity mask image specifically includes: Calculate the clarity score of each pixel position on all grayscale images and construct a clarity curve for each pixel position; The clarity mask corresponding to each pixel position is determined according to the clarity curve of each pixel position, and a clarity mask image is generated.

3. The method for multi-focus image depth of field fusion according to claim 2, characterized in that: The clarity score is calculated as: For the pixel position (x, y), it is in the i-th grayscale image G i The clarity score C′ i (x, y), set to the i-th grayscale image G i The average value of the clarity score of all pixels in the local area R(x,y) is expressed as: Where R(x,y) is a local area with a window size of k×k centered at the pixel position (x,y), and k is a preset parameter value; (x ′ ,y ′ ) is the pixel in the local region R(x,y), and the Laplacian operator is the second-order derivative operator; The clarity curve is expressed as: {C′1(x,y),C′2(x,y),...,C′ n (x, y)}; where n is the total number of grayscale images.

4. The method for multi-focus image depth of field fusion according to claim 3, characterized in that: The method of determining the definition mask corresponding to each pixel position according to the definition curve of each pixel position and generating a definition mask image specifically includes: Finding a maximum value in the clarity curve, and calculating an average value of the clarity curve; According to the comparison between the maximum value found and the average value, the pixel value of the sharpness mask image is determined, thereby generating the sharpness mask image; Among them, the maximum value P = C' j (x, y), the following conditions must be met: C′ j (x, y) > C′ j-1 (x, y), and C′ j (x, y) > C′ j+1 (x, y), 2 ≤ j ≤ n - 1; The average value of the clarity curve P mean The calculation formula is: The step of determining the pixel value of the definition mask image based on the comparison between the maximum value found and the average value, and then generating the definition mask image, specifically includes: Determine the clarity mask of the clarity mask image at each pixel position according to the comparison between the found maximum value and the average value; Generate a clarity mask image according to the clarity mask at each pixel position of the clarity mask image; Among them, the comparison process of the maximum value and the average value includes: If the maximum value P>P mean The number of times is equal to 0, then the sharpness mask M(x,y) at the pixel position (x,y) is: If the maximum value P>P mean If the number of is equal to 1, then take the sequence number l of the image where the maximum value is located as the pixel value of the clarity mask image at position (x, y): M(x,y)=l; If the maximum value P>P mean If the number of is greater than 1, the sequence number m of the image where the last maximum value is located is taken as the pixel value of the clarity mask image at position (x, y): M(x,y)=m.

5. The method for multi-focus image depth of field fusion according to claim 3, characterized in that: The value range of k is [3-99].

6. The method for multi-focus image depth of field fusion according to claim 4, characterized in that: Before the image fusion is performed on the aligned multiple focused images according to the clarity mask image, the method further includes: The clarity mask image is subjected to smoothing filtering, which specifically includes: First, median filtering is applied to the clarity mask image to obtain M med (x,y), as follows: M med (x, y)=median({M(x′, y′)|(x′, y′)∈W(x, y)}); Where W(x,y) is a k×k window centered at the pixel position (x,y), M(x′, y′) is the value of all pixels in the window, and median() represents the operation of selecting the median from the window; Then apply Gaussian filtering to the result after median filtering to obtain a smoothed clarity mask image as follows: M smooth (x,y)=∑ (x′，y′)∈W(x，y) G(x′-x,y′-y)·M med (x′,y′); The calculation formula of the Gaussian function weight is: Among them, M smooth (x, y) is the pixel value of the clarity mask image at position (x, y); σ is the standard deviation of the Gaussian filter.

7. The method for multi-focus image depth of field fusion according to claim 6, characterized in that: The method of performing image fusion on the aligned multiple focused images according to the clarity mask image is specifically implemented as follows: performing image fusion on the aligned multiple focused images according to the smoothed clarity mask image, and combining the clearest image at each pixel position into a fused image; specifically includes: According to the pixel value of the smoothed clarity mask image at each pixel position, the corresponding image is selected for fusion; the fused image generation formula is: Where: F(x,y) is the pixel value of the fused image at position (x,y); M smooth (x,y) represents the serial number of the focused image corresponding to the clearest image at position (x,y); is the Mth in the aligned multifocus image smooth The pixel value of the (x,y)th focused image at position (x,y).

8. The method for multi-focus image depth of field fusion according to claim 7, characterized in that: After the image fusion is performed on the aligned multiple focused images according to the clarity mask image, the method further includes: Post-processing operations are performed on the fused image, including: Calculate the minimum and maximum values ​​of the fused image; the calculation method is: Perform histogram stretching on each pixel F(x,y) to map the minimum and maximum values ​​of the fused image to the target range [L min ,L max ]; Among them: F stretch (x, y) is the pixel value of the stretched image, F(x, y) is the pixel value of the original fused image; F min and F max are the minimum and maximum values ​​of the original fused image, L min and L max are the minimum and maximum values ​​of the target range, respectively.

9. The method for multi-focus image depth of field fusion according to claim 1, characterized in that: The step of aligning the input multiple focused images in the same coordinate system specifically includes: Assume that the total number of input focused images is n, and the n focused images are denoted as: I1, I2, …, I n , the pixel value of the i-th image at position (x, y) is represented by I i (x,y) indicates that (x,y) is the spatial coordinate of the focused image; Select one of the n focused images as the reference image I ref ; For each image I i , which is consistent with the reference image I ref The transformation between is expressed as an affine transformation model: I i (x,y)=T i ·I ref (x′,y′); Among them, T i is the transformation matrix, representing image I i The transformation relative to the reference image is expressed as: Among them, a, b, c, d represent transformation parameters, including rotation and scaling; tx, ty represent translation parameters; The ECC alignment method is used to estimate the transformation matrix by optimizing the objective function. ref and I i , the ECC similarity measurement function is expressed as: Among them, T i (x,y) is the transformed image I i The pixel position in x，y It means summing all pixel positions; Optimize the transformation matrix T through an iterative optimization algorithm i , so that the objective function ECC(T i ) maximize; transform parameter T i It will be updated step by step according to the alignment error between each image and the reference image until the similarity reaches the maximum. The final transformation matrix T i Will be applied to image I i , align it with the reference image, and obtain an image set after all focused images are aligned in the same coordinate system; Among them, the pixel value of the focused image at position (x, y) in the aligned image set is expressed as: I′ i (x,y)=T i ·I i (x,y)。 10. The method for multi-focus image depth of field fusion according to claim 9, characterized in that: The preprocessing of the aligned multiple focused images to obtain multiple denoised grayscale images specifically includes: For each aligned focused image, if its image type is a color image, it is converted into a grayscale image using the weighted average method; the grayscale value of the i-th grayscale image at position (x, y) is expressed as: G i (x,y)=0.2989·R(x,y)+0.5870·G(x,y)+0.1140·B(x,y); where the pixel values ​​of the red, green and blue channels of the aligned focused image at position (x,y) are represented as R(x,y), G(x,y) and B(x,y), respectively; Perform mean filtering on each grayscale image to remove noise; the specific implementation is as follows: Assuming that a filter window of size k×k is used, the mean filter formula is: Among them, (x+m,y+n) represents the neighborhood pixels within the filter window.

11. A system for multi-focus image depth of field fusion, characterized in that: include: An image processing module is used to align multiple input focused images in the same coordinate system; It is also used to pre-process the aligned multiple focused images to obtain multiple grayscale images after denoising; A mask image construction module, electrically connected to the image processing module, for calculating a clarity score of each pixel position on all grayscale images to generate a clarity mask image; The image fusion module is electrically connected to the mask image construction module and the image processing module, and is used to perform image fusion on the aligned multiple focused images according to the clarity mask image.

12. The system for multi-focus image depth of field fusion according to claim 11, characterized in that: The mask image construction module is specifically used to calculate the clarity score of each pixel position on all grayscale images and construct a clarity curve for each pixel position; and is specifically used to determine the clarity mask corresponding to each pixel position according to the clarity curve of each pixel position and generate a clarity mask image.

13. The system for multi-focus image depth of field fusion according to claim 12, characterized in that: The clarity score is calculated as: For the pixel position (x, y), it is in the i-th grayscale image G i The clarity score C′ i (x, y), set to the i-th grayscale image G i The average value of the clarity score of all pixels in the local area R(x,y) is expressed as: Where R(x,y) is a local area with a window size of k×k centered at the pixel position (x,y), k is a pre-set parameter value; (x′,y′) is a pixel in the local area R(x,y), and the Laplacian operator is the second-order derivative operator; The clarity curve is expressed as: {C′1(x,y),C′2(x,y),C′ n (x, y)}; where n is the total number of grayscale images.

14. The system for multi-focus image depth of field fusion according to claim 13, characterized in that: The method of determining the definition mask corresponding to each pixel position according to the definition curve of each pixel position and generating a definition mask image specifically includes: Finding a maximum value in the clarity curve, and calculating an average value of the clarity curve; According to the comparison between the maximum value found and the average value, the pixel value of the sharpness mask image is determined, thereby generating the sharpness mask image; Among them, the maximum value P = C' j (x, y), the following conditions must be met: C′ j (x, y) > C′ j-1 (x, y), and C′ j (x, y) > C′ j+1 (x, y), 2 ≤ j ≤ n - 1; The average value of the clarity curve P mean The calculation formula is: The step of determining the pixel value of the definition mask image based on the comparison between the maximum value found and the average value, and then generating the definition mask image, specifically includes: Determine the clarity mask of the clarity mask image at each pixel position according to the comparison between the found maximum value and the average value; Generate a clarity mask image according to the clarity mask at each pixel position of the clarity mask image; Among them, the comparison process of the maximum value and the average value includes: If the maximum value P>P mean The number of times is equal to 0, then the sharpness mask M(x,y) at the pixel position (x,y) is: If the maximum value P>P mean If the number of is equal to 1, then take the sequence number l of the image where the maximum value is located as the pixel value of the clarity mask image at position (x, y): M(x,y)=l; If the maximum value P>P mean If the number of is greater than 1, the sequence number m of the image where the last maximum value is located is taken as the pixel value of the clarity mask image at position (x, y): M(x,y)=m.

15. The system for multi-focus image depth of field fusion according to claim 13, characterized in that: The value range of k is [3-99].

16. The system for multi-focus image depth of field fusion according to claim 14, characterized in that: The image fusion module is electrically connected to a smoothing filter module, and the smoothing filter module is electrically connected to a mask image construction module; The smoothing filter module is used to perform smoothing filter processing on the clarity mask image; specifically comprising: First, median filtering is applied to the clarity mask image to obtain M med (x,y), as follows: M med (x, y)=median({M(x′, y′)|(x′, y′)∈W(x, y)}); Where W(x,y) is a k×k window centered at the pixel position (x,y), M(x′, y′) is the value of all pixels in the window, and median() represents the operation of selecting the median from the window; Then apply Gaussian filtering to the result after median filtering to obtain a smoothed clarity mask image as follows: M smooth (x,y)=∑ (x′，y′)∈W(x，y) G(x′-x,y′-y)·M med (x′,y′); The calculation formula of the Gaussian function weight is: Among them, M smooth (x, y) is the pixel value of the clarity mask image at position (x, y); σ is the standard deviation of the Gaussian filter.

17. The system for multi-focus image depth of field fusion according to claim 16, characterized in that: The image fusion module is specifically used to perform image fusion on the aligned multiple focused images according to the smoothed clarity mask image, and combine the clearest image at each pixel position into a fused image; specifically includes: According to the pixel value of the smoothed clarity mask image at each pixel position, the corresponding image is selected for fusion; the fused image generation formula is: Where: F(x,y) is the pixel value of the fused image at position (x,y); M smooth (x,y) represents the serial number of the focused image corresponding to the clearest image at position (x,y); is the Mth in the aligned multifocus image smooth The pixel value of the (x,y)th focused image at position (x,y).

18. The system for multi-focus image depth of field fusion according to claim 17, characterized in that: The image fusion module is also electrically connected to an image post-processing module; the image post-processing module is used to perform post-processing operations on the fused image, specifically including: Calculate the minimum and maximum values ​​of the fused image; the calculation method is: Perform histogram stretching on each pixel F(x,y) to map the minimum and maximum values ​​of the fused image to the target range [L min ,L max ]; Among them: F stretch (x, y) is the pixel value of the stretched image, F(x, y) is the pixel value of the original fused image; F min and F max are the minimum and maximum values ​​of the original fused image, L min and L max are the minimum and maximum values ​​of the target range, respectively.

19. The system for multi-focus image depth of field fusion according to claim 11, characterized in that: The step of aligning the input multiple focused images in the same coordinate system specifically includes: Assume that the total number of input focused images is n, and the n focused images are denoted as: I1, I2, …, I n , the pixel value of the i-th image at position (x, y) is represented by I i (x,y) indicates that (x,y) is the spatial coordinate of the focused image; Select one of the n focused images as the reference image I ref ; For each image I i , which is consistent with the reference image I ref The transformation between is expressed as an affine transformation model: I i (x,y)=T i ·I ref (x′,y′); Among them, T i is the transformation matrix, representing image I i The transformation relative to the reference image is expressed as: Among them, a, b, c, d represent transformation parameters, including rotation and scaling; tx, ty represent translation parameters; The ECC alignment method is used to estimate the transformation matrix by optimizing the objective function. ref and I i , the ECC similarity measurement function is expressed as: Among them, T i (x,y) is the transformed image I i The pixel position in x，y It means summing all pixel positions; Optimize the transformation matrix T through an iterative optimization algorithm i , so that the objective function ECC(T i ) maximize; transform parameter T i It will be updated step by step according to the alignment error between each image and the reference image until the similarity reaches the maximum. The final transformation matrix T i Will be applied to image I i , align it with the reference image, and obtain an image set after all focused images are aligned in the same coordinate system; Among them, the pixel value of the focused image at position (x, y) in the aligned image set is expressed as: I′ i (x,y)=T i ·I i (x,y)。 20. The system for multi-focus image depth of field fusion according to claim 19, characterized in that: The preprocessing of the aligned multiple focused images to obtain multiple denoised grayscale images specifically includes: For each aligned focused image, if its image type is a color image, it is converted into a grayscale image using the weighted average method; the grayscale value of the i-th grayscale image at position (x, y) is expressed as: G i (x,y)=0.2989R(x,y)+0.5870·G(x,y)+0.1140·B(x,y); where the pixel values ​​of the red, green and blue channels of the aligned focused image at position (x,y) are represented as R(x,y), G(x,y) and B(x,y), respectively; Perform mean filtering on each grayscale image to remove noise; the specific implementation is as follows: Assuming that a filter window of size k×k is used, the mean filter formula is: Among them, (x+m,y+n) represents the neighborhood pixels within the filter window.

21. A computer-readable storage medium, wherein at least one instruction is stored in the storage medium, characterized in that: The instructions are loaded and executed by the processor to implement a method for multi-focus image depth of field fusion as described in any one of claims 1-10.

22. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, a method for multi-focus image depth of field fusion as described in any one of claims 1 to 10 is implemented.

Images