Image processing method based on adaptive weight multi-image fusion algorithm

Abstract

The invention discloses an image processing method based on an adaptive weight multi-image fusion algorithm, which relates to the technical field of image processing and comprises the following specific steps of: extracting a plurality of images; constructing an adaptive weight multi-image fusion algorithm to carry out fusion processing to obtain a fused image; the adaptive weight multi-image fusion algorithm specifically comprises the following steps: inputting an image sequence; convolution processing is carried out; performing position alignment on the image; screening overexposure and underexposure areas of the source image for shielding to obtain a convolution result image; carrying out merging processing to obtain a convolution image; performing normalization processing on the convolution image; and calculating the distance standard deviation from each pixel of the source image to the average gray value, and fusing a new image. According to the method, multiple images can be fused into one image with a soft surface and a luxurious edge, detail information in the multiple images is reserved, the noise of the images is reduced, and the local contrast of different areas in the images is enhanced, so that the quality of the images is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image processing method based on an adaptive weighted multi-image fusion algorithm. Background Art

[0002] An image is a carrier of visual information and a vivid reproduction or reproduction of the similarity of an objective object. It can be understood as a two-dimensional function in which the independent variable (usually spatial coordinates) and the dependent variable (usually color or grayscale) are representations of continuously changing physical quantities. In a computer, an image is usually represented as a two-dimensional array (or matrix), in which each element (or pixel) corresponds to a point in the image and contains information about the color, brightness or other visual attributes of the point.

[0003] Currently, after taking multiple images of the same scene, there are limitations in reflecting the scene through a single image. This is because the information contained in each image is different, and these images are subject to noise, blur or underexposure when shooting, which makes it difficult to clearly and accurately reflect the scene content, reducing the quality of the image.

[0004] In order to solve the above technical problems, an image processing method based on adaptive weighted multi-image fusion algorithm is proposed. Summary of the invention

[0005] The purpose of the present invention is to provide an image processing method based on an adaptive weighted multi-image fusion algorithm to solve the problems raised in the above background technology.

[0006] To achieve the above object, the present invention provides the following technical solution: an image processing method based on an adaptive weighted multi-image fusion algorithm, comprising the following specific steps:

[0007] Step 1: extract the information of multiple images to be fused and perform preprocessing;

[0008] Step 2: Utilize an adaptive weight multi-image fusion algorithm to fuse multiple images to obtain a fused image.

[0009] The adaptive weighted multi-image fusion algorithm specifically includes the following steps:

[0010] S1, input image sequence;

[0011] S2, performing convolution processing on all source images in the image sequence to obtain convolution edge results;

[0012] S3, aligning the image according to the convolution edge result obtained in step 2;

[0013] S4, screening all the images after position alignment, screening out all overexposed areas and underexposed areas of the source images, and using these areas as references to mask the convolution results to obtain the convolution result image;

[0014] S5, merging all the convolution result images to obtain a new convolution image;

[0015] S6, normalize the convolution image;

[0016] S7, using the normalized result of the convolution image in step 6 as a reference weight, calculate the standard deviation of the distance between each pixel of all source images and the average gray value, and fuse an image with a soft surface and sharp edges.

[0017] Preferably, in step 2, a convolution with a convolution kernel of 5×5 is performed on the source image, wherein the mathematical expression of the convolution edge result is specifically:

[0018]

[0019] Among them, the output image I′, the convolution kernel K(i,j) is the corresponding weight in the convolution kernel, I(x+i,y+j) is the pixel value in the 5×5 area centered on (x,y) in the input image, and I′(x,y) is a pixel value in the output image.

[0020] Preferably, in step three, aligning the image positions is specifically performed as follows:

[0021] Use the Sobel operator to perform convolution operation on the image to obtain the horizontal and vertical gradient images respectively;

[0022] Combine the horizontal and vertical gradient images and use the formula to calculate the gradient amplitude and gradient direction;

[0023] The position coordinates of the outer contour edge of sequence 1 image are used as the reference, and other sequence images are automatically aligned based on it.

[0024] Preferably, in the image position alignment process, the calculation formula for the gradient amplitude and direction is specifically:

[0025]

[0026]

[0027]

[0028]

[0029] In the formula, |G| is the gradient amplitude, G x is the horizontal coordinate of the image, G yis the vertical coordinate of the image, Θ is the gradient direction, and I is the image pixel value.

[0030] Preferably, in step 4, the screening and shielding of the position alignment image are specifically as follows:

[0031] Perform mean filtering on the aligned sequence images;

[0032] The feature area of ​​a single sequence image is calculated by automatic thresholding.

[0033] Preferably, the mean filter processing formula of the sequence image is:

[0034]

[0035] Where I(x+i,y+j) is the pixel value of the sequence image at the position (x,y), I′(x,y) is the pixel value of the filtered image at the position (x,y), and M×N is the size of the sequence image.

[0036] Preferably, in step 5, the normalization processing formula of the convolution image is:

[0037]

[0038] Preferably, in step seven, the fusion of an image with a soft surface and sharp edges is specifically as follows:

[0039] Perform mean filtering on the sequence images respectively to obtain the smoothed image I smooth ;

[0040] Calculate the original image and smoothed image I smooth The difference image I diff ;

[0041] Calculate the standard deviation σ of the pixel distance from the average gray value of the original image;

[0042] Use the normalized image results of each sequence as the weight factor ω to adjust I diff For each pixel value in , the normalized weight is obtained by dividing each pixel value by σ;

[0043] The weighted difference image ω·I diff Add back to I smooth The final image I is obtained final .

[0044] Preferably, the smoothed image I smooth The calculation formula is:

[0045] I diff (x,y)=I(x,y)-I smooth(x,y)

[0046] Where (x, y) represents the pixel position in the image;

[0047] The final image I final The calculation formula is:

[0048] I final =I smooth +ω·I diff

[0049] In the formula, I final represents the final image, ω represents the weight factor, I smooth represents a smooth image, I diff Represents the difference image.

[0050] Preferably, the standard deviation σ of the pixel distance from the average gray value of the original image is calculated as follows:

[0051] The color image is converted into a grayscale image through the conversion formula, where the conversion formula is:

[0052] Gray(x,y)=0.299R(x,y)+0.587G(x,y)+0.114B(x,y);

[0053] Where R(x,y), G(x,y), and B(x,y) represent the red, green, and blue channel values ​​of (x,y) respectively;

[0054] Calculate the average gray value, where the calculation formula for the average gray value is:

[0055]

[0056] Where M and N represent the width and height of the image respectively;

[0057] The standard deviation is calculated by the average gray value, where the standard deviation is calculated as:

[0058]

[0059] Where M and N represent the width and height of the image, Gray(x,y) represents the gray value, and u represents the average gray value.

[0060] Technical effects and advantages of the present invention:

[0061] The adaptive weighted multi-image fusion algorithm provided by the present invention can fuse multiple images into an image with a soft surface and sharp edges, retain the detail information in the multiple images, reduce the noise of the image, enhance the local contrast of different areas in the image, make the fused image have higher clarity, more vivid and lively, eliminate the limitations of a single image, so that a clear, lively and soft image can be obtained from multiple images, which helps to improve the quality of the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 It is a schematic diagram of the process of the adaptive weighted multi-image fusion algorithm of the present invention. DETAILED DESCRIPTION

[0063] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] The present invention provides Figure 1 An image processing method based on an adaptive weighted multi-image fusion algorithm is shown, comprising the following specific steps:

[0065] Step 1: extract the information of multiple images to be fused and perform preprocessing. Before inputting the image sequence, the corresponding multiple images need to be preprocessed. The preprocessing measures include but are not limited to size adjustment and color correction.

[0066] Step 2: Utilize an adaptive weight multi-image fusion algorithm to fuse multiple images to obtain a fused image.

[0067] The adaptive weighted multi-image fusion algorithm specifically includes the following steps:

[0068] S1, input image sequence, the example of this embodiment uses 4 sequences of images;

[0069] S2, performing convolution processing on all source images in the image sequence to obtain convolution edge results. The convolution processing can help extract image features and provide more effective information for the subsequent fusion process. It specifically includes the following steps:

[0070] S201, before performing the convolution process, it is necessary to ensure that the four images are aligned, because under different shooting conditions, the images may be slightly displaced or rotated. Alignment can ensure that all images are in the same spatial position, ensuring the consistency of subsequent image processing;

[0071] S202, define a 5×5 convolution kernel (here defined as a Sobel kernel) to help extract local features of the image;

[0072] S203, align the center of the convolution kernel with each pixel in the image, perform element-by-element multiplication of the convolution kernel and the corresponding area in the image, and then sum them to obtain the convolution edge result (convolution value); the specific calculation formula is as follows:

[0073]

[0074] Where I′(x,y) represents the pixel value of the output image at position (x,y), K(i,j) represents the corresponding weight of the convolution kernel at position (x,y), and I(x+i,y+j) is the pixel value in the 5×5 area centered at (x,y) in the input image;

[0075] Through convolution processing, the convolution edge result features of the four images can be extracted, providing more effective information for the subsequent fusion process;

[0076] S3, according to the convolution edge result obtained in step 2, the image is aligned. By extracting the convolution edge result of the image, the relative position relationship between the images can be determined to ensure that the images can be accurately superimposed during the fusion process. The alignment of the image position helps to improve the overall consistency and accuracy of the fused image. First, the Sobel operator is used to perform convolution operation on the image to obtain the horizontal and vertical gradient images respectively. Then, the horizontal and vertical gradient images are combined to calculate the gradient amplitude and gradient direction using the formula. Finally, the position coordinates of the outer contour edge of the sequence 1 image are used as the reference, and the other sequence images are automatically aligned with it. The specific example is:

[0077] Read four grayscale images;

[0078] Select the first image as the reference image and use the Sobel operator to extract the edge of the image;

[0079] Perform Sobel edge detection on the reference image and each image to be aligned;

[0080] Extract the convolution edge result of the reference image as the reference for alignment;

[0081] Use SIFT algorithm to detect edge key points of the reference image and each image to be aligned, use BFMatcher for feature matching, filter out the best matching point pairs through RANSAC algorithm, use cv2.findHomography function to calculate homography matrix, use cv2.warpPerspective function to transform each image into the coordinate system of the reference image, and use cv2.imshow to display the aligned images;

[0082] Through the above steps, multiple images can be effectively aligned based on the Sobel edge detection results to ensure that the positions of the images are consistent during the convolution and fusion process;

[0083] S4, screens all images after position alignment, screens out all overexposed and underexposed areas of the source images, and uses these areas as references to shield the convolution results to obtain the convolution result image. By screening out and shielding the overexposed and underexposed areas of the source images that contain less useful information, a convolution result image containing only valid area information is generated to ensure the accuracy and reliability of the convolution result. Since the underexposed areas are usually too dark and details are lost, and the overexposed areas are too bright and details are also lost, these areas contain less useful information and may even introduce noise. By screening these areas, it is possible to avoid losing important information during the image fusion process, improve the overall quality of the fused image, and make it more natural and realistic. Different lighting conditions will cause large changes in the brightness of certain areas in the image. By screening the underexposed and overexposed areas, the impact of these changes on the fusion results can be reduced, thereby improving the robustness of the algorithm. In addition, by shielding the underexposed and overexposed areas, the computational complexity of the convolution operation can be reduced, thereby improving the efficiency of the algorithm. Specifically:

[0084] First, the aligned sequence images are mean filtered, and then the feature area is calculated for a single sequence image by automatic thresholding, where the yellow area is the overexposed area and the blue area is the underexposed area. The pixel value of the overexposed area is close to 255, and the pixel value of the underexposed area is close to 0.

[0085] S5, merge all the convolution result images to obtain a new convolution image (multi-channel convolution image, one sequence corresponds to one image). By merging multiple convolution result images, the important feature information in the image can be enhanced, the saliency and robustness of the feature can be improved, the uncertainty of a single convolution result can be reduced, and the purpose of suppressing noise in the image can be achieved, the signal-to-noise ratio of the image can be improved, and the influence of noise on the image can be reduced, so that the fused new image can adapt to different scenes and task requirements, and the processing speed and effect of the image can be improved;

[0086] S6, normalizing the convolution image. Normalizing the convolution image synthesized in the above steps. By normalizing the convolution image, the pixel value of the convolution image can be adjusted to a specific range (for example, [0, 1] or [-1, 1]). When the range of the pixel value is too large, it may cause numerical overflow or underflow in the calculation process, thereby affecting the performance of the algorithm. Through normalization, this problem can be avoided and the stability and efficiency of the calculation can be improved. In addition, normalization can make the numerical values ​​of different images on the same feature dimension comparable, thereby more accurately reflecting the differences and similarities between images, which helps to optimize the distribution of weights, so that the fused image can better retain the key information and details in the original image, which helps to improve the quality of the fused image.

[0087] S7, using the normalized result of the convolution image in step 6 as a reference weight, calculate the standard deviation of the distance between each pixel of all source images and the average gray value, and fuse an image with a soft surface and sharp edges; the normalized result of the convolution image can reflect the key features and detail information in the image, and using it as a reference weight can more accurately allocate the contribution of different source images in the fusion process; by considering the standard deviation of the distance between the pixel and the average gray value, the algorithm can further distinguish the smooth area and the edge area in the image, so as to give different treatments during fusion; the specific process includes:

[0088] First, calculate the grayscale average value of each grayscale image as the benchmark grayscale value of the image;

[0089] For each pixel, calculate the distance between its grayscale value and the average grayscale value, and calculate the standard deviation of these distances. The standard deviation reflects the discreteness of the pixel grayscale value, that is, the softness of the image surface. The distance standard deviation here refers to the fluctuation degree of the pixel grayscale value relative to the average grayscale value. The smaller the fluctuation, the softer the surface, and the larger the fluctuation, the more edge information it contains.

[0090] Use the normalized convolution feature map as the reference weight, which can reflect the key features and detail information of the image. Then, with the standard deviation of the distance, assign an adaptive weight to each similarity of each source image. The weight distribution should make the fused image soft on the surface while keeping the edges sharp.

[0091] According to the distribution results of adaptive weights, the final fused image is generated. The fused image combines the key information and details in all source images, while presenting a visual effect with soft surface and sharp edges.

[0092] Through the above steps, multiple images can be fused into an image with a soft surface and sharp edges, retaining the detail information in the multiple images, reducing the noise of the image, enhancing the local contrast of different areas in the image, making the fused image have higher clarity, more vivid and lively, eliminating the limitations of a single image, so that a clear, vivid and soft image can be obtained from multiple images, thereby improving the image quality.

[0093] In step 3, in the image position alignment process, the calculation formula of the gradient amplitude and direction is specifically:

[0094]

[0095]

[0096]

[0097]

[0098] In the formula, |G| is the gradient amplitude, G x is the horizontal coordinate of the image, G y is the vertical coordinate of the image, Θ is the gradient direction, and I is the image pixel value.

[0099] In step 4, the screening and masking of the position-aligned image are specifically as follows:

[0100] Perform mean filtering on the aligned sequence images;

[0101] The feature area of ​​a single sequence image is calculated by automatic thresholding.

[0102] The mean filter processing formula for sequence images is:

[0103]

[0104] Where I(x+i,y+j) is the pixel value of the sequence image at the position (x,y), I′(x,y) is the pixel value of the filtered image at the position (x,y), and M×N is the size of the sequence image.

[0105] In step 5, the normalization formula of the convolution image is:

[0106]

[0107] In step 7, a soft-surface and sharp-edge image is fused as follows:

[0108] Perform mean filtering on the sequence images respectively to obtain the smoothed image I smooth ;

[0109] Calculate the original image and smoothed image I smooth The difference image I diff ;

[0110] Calculate the standard deviation σ of the pixel distance from the average gray value of the original image;

[0111] Use the normalized image results of each sequence as the weight factor ω to adjust I diff For each pixel value in , the normalized weight is obtained by dividing each pixel value by σ;

[0112] The weighted difference image ω·I diff Add back to I smooth The final image I is obtained final .

[0113] Smooth Image I smooth The calculation formula is:

[0114] I diff (x,y)=I(x,y)-I smooth (x,y)

[0115] Where (x, y) represents the pixel position in the image;

[0116] Final Image I final The calculation formula is:

[0117] I final =I smooth +ω·I diff

[0118] In the formula, I final represents the final image, ω represents the weight factor, I smooth represents a smooth image, I diff Represents the difference image.

[0119] The standard deviation σ of the pixel distance from the average gray value of the original image is calculated as follows:

[0120] The color image is converted into a grayscale image through the conversion formula, where the conversion formula is:

[0121] Gray(x,y)=0.299R(x,y)+0.587G(x,y)+0.114B(x,y);

[0122] Where R(x,y), G(x,y), and B(x,y) represent the red, green, and blue channel values ​​of (x,y) respectively;

[0123] Calculate the average gray value, where the calculation formula for the average gray value is:

[0124]

[0125] Where M and N represent the width and height of the image respectively;

[0126] The standard deviation is calculated by the average gray value, where the standard deviation is calculated as:

[0127]

[0128] Where M and N represent the width and height of the image, Gray(x,y) represents the gray value, and u represents the average gray value.

[0129] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An image processing method based on an adaptive weighted multi-image fusion algorithm, characterized in that: The specific steps include: Step 1: extract the information of multiple images to be fused and perform preprocessing; Step 2: Utilize an adaptive weight multi-image fusion algorithm to fuse multiple images to obtain a fused image. The adaptive weighted multi-image fusion algorithm specifically includes the following steps: S1, input image sequence; S2, performing convolution processing on all source images in the image sequence to obtain convolution edge results; S3, aligning the image according to the convolution edge result obtained in step 2; S4, screening all the images after position alignment, screening out all overexposed areas and underexposed areas of the source images, and using these areas as references to mask the convolution results to obtain the convolution result image; S5, merging all the convolution result images to obtain a new convolution image; S6, normalize the convolution image; S7, using the normalized result of the convolution image in step 6 as a reference weight, calculate the standard deviation of the distance between each pixel of all source images and the average gray value, and fuse an image with a soft surface and sharp edges.

2. The image processing method based on the adaptive weighted multi-image fusion algorithm according to claim 1, characterized in that: In the step 2, a convolution with a convolution kernel of 5×5 is performed on the source image, wherein the mathematical expression of the convolution edge result is specifically: Among them, the output image I′, the convolution kernel K(i,j) is the corresponding weight in the convolution kernel, I(x+i,y+j) is the pixel value in the 5×5 area centered on (x,y) in the input image, and I′(x,y) is a pixel value in the output image.

3. The image processing method based on the adaptive weighted multi-image fusion algorithm according to claim 2 is characterized in that: In step three, the image positions are aligned as follows: Use the Sobel operator to perform convolution operation on the image to obtain the horizontal and vertical gradient images respectively; Combine the horizontal and vertical gradient images and use the formula to calculate the gradient amplitude and gradient direction; The position coordinates of the outer contour edge of sequence 1 image are used as the reference, and other sequence images are automatically aligned based on it.

4. The image processing method based on the adaptive weighted multi-image fusion algorithm according to claim 3 is characterized in that: In the image position alignment process, the calculation formula of the gradient amplitude and direction is specifically: In the formula, |G| is the gradient amplitude, G x is the horizontal coordinate of the image, G y is the vertical coordinate of the image, Θ is the gradient direction, and I is the image pixel value.

5. The image processing method based on the adaptive weighted multi-image fusion algorithm according to claim 4 is characterized in that: In step 4, the screening and shielding of the position alignment image are specifically as follows: Perform mean filtering on the aligned sequence images; The feature area of ​​a single sequence image is calculated by automatic thresholding.

6. The image processing method based on the adaptive weighted multi-image fusion algorithm according to claim 5, characterized in that: The mean filter processing formula of the sequence image is: Where I(x+i,y+j) is the pixel value of the sequence image at the position (x,y), I′(x,y) is the pixel value of the filtered image at the position (x,y), and M×N is the size of the sequence image.

7. The image processing method based on the adaptive weighted multi-image fusion algorithm according to claim 6 is characterized in that: In step 5, the normalization processing formula of the convolution image is:

8. The image processing method based on the adaptive weighted multi-image fusion algorithm according to claim 7 is characterized in that: In step 7, a pair of images with soft surface and sharp edges are fused as follows: Perform mean filtering on the sequence images respectively to obtain the smoothed image I smooth ; Calculate the original image and smoothed image I smooth The difference image I diff ; Calculate the standard deviation σ of the pixel distance from the average gray value of the original image; Use the normalized image results of each sequence as the weight factor ω to adjust I diff For each pixel value in , the normalized weight is obtained by dividing each pixel value by σ; The weighted difference image ω·I diff Add back to I smooth The final image I is obtained final .

9. The image processing method based on the adaptive weighted multi-image fusion algorithm according to claim 8, characterized in that: The smoothed image I smooth The calculation formula is: I diff (x,y)=I(x,y)-I smooth (x,y) Where (x, y) represents the pixel position in the image; The final image I final The calculation formula is: I final =I smooth +ω·I diff In the formula, I final represents the final image, ω represents the weight factor, I smooth represents a smooth image, I diff Represents the difference image.

10. The image processing method based on the adaptive weighted multi-image fusion algorithm according to claim 9, characterized in that: The standard deviation σ of the pixel distance from the average gray value of the original image is calculated as follows: The color image is converted into a grayscale image through the conversion formula, where the conversion formula is: Gray(x,y)=0.299R(x,y)+0.587G(x,y)+0.114B(x,y); Where R(x,y), G(x,y), and B(x,y) represent the red, green, and blue channel values ​​of (x,y) respectively; Calculate the average gray value, where the calculation formula for the average gray value is: Where M and N represent the width and height of the image respectively; The standard deviation is calculated by the average gray value, where the standard deviation is calculated as: Where M and N represent the width and height of the image, Gray(x,y) represents the gray value, and u represents the average gray value.

Images