A Core Image Fusion Method for Optimizing Poisson Fusion
The Poisson fusion algorithm with weighted gradient operators addresses issues in existing methods by aligning and masking gradient matrices to eliminate noise and shadows, resulting in a clear and complete rock core image fusion.
Patent Information
- Application Number
- CN202111611340.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-27
- Publication Date
- 2025-07-15
- Estimated Expiration
- 2041-12-27
AI Technical Summary
In the existing core image scanning technology, traditional image fusion methods cannot effectively deal with blur and noise caused by depth of field differences, resulting in image information loss and ghosting, affecting the efficiency and accuracy of core observation.
The fusion algorithm based on Poisson fusion and weighted gradient operator is adopted, and the gradient matrix is calculated through the Sobel operator, image registration and Poisson fusion of low-frequency and high-frequency regions are performed, and ghosting is removed by using a pseudo-edge suppression algorithm to form a fully clear fusion image.
It realizes efficient fusion of core images, ensures image information integrity, reduces noise and ghosting, improves image clarity and similarity, and is suitable for petroleum exploration and geological research.
Smart Images
Figure CN115249255B_ABST
Abstract
Description
Technical Field
[0001] This article relates to the field of digital image processing technology. More specifically, it is a core image fusion method that optimizes Poisson fusion. Background Art
[0002] With the development of computer technology and industrial cameras, core image scanning technology has gradually become an important means for core image observation and analysis. It realizes the conversion from physical cores to permanent high-fidelity core digital images, which is of great significance for the development of fields such as oil exploration and geological research. During the scanning process, due to the limitation of the focal length of the camera lens, the images scanned by lenses with different object distances will produce a depth-of-field effect, that is, the images obtained outside the depth-of-field range are blurred and a complete and clear core image cannot be obtained.
[0003] Classifying and storing images according to object distance will not only cause redundancy of data, but also bring great inconvenience to the scientific research and observation of rock masses; simply stitching with image processing software cannot achieve ideal results in terms of efficiency and accuracy. Therefore, a multi-focus fusion algorithm needs to be adopted for the core acquisition images to fuse images with different object distances into a single fully clear picture, so as to completely express the image information.
[0004] Currently, traditional fusion methods include image fusion based on wavelet transform, Sobel high-pass fusion, gradient pyramid fusion, etc. However, wavelet transform can only capture information in three directions, and the weighted average processing in the low-frequency region is prone to information loss; Sobel high-pass fusion generates more gradient noise points and cannot perfectly handle the phenomenon of local ghosting distortion caused by RANSAC feature point purification; pyramid transform is prone to medium-frequency blur. Summary of the Invention
[0005] To solve the above technical problems, the present invention proposes a fusion algorithm based on Poisson fusion and weighted gradient operator.
[0006] The above technical objective of the present invention is achieved through the following technical solutions:
[0007] A core image fusion method that optimizes Poisson fusion includes the following steps:
[0008] Step S1, obtaining original images with different depths of field at the same position of the sample to be measured, using the first frame of the original images as the standard image, and performing image registration on other original images in terms of scale to form registered images;
[0009] Step S2, calculating the gradient matrix of the standard image and the first frame of the registered image through the Sobel operator;
[0010] Step S3: Denoise the obtained gradient matrix in the binary image, and use it as a mask to perform Poisson fusion on the standard image and the registered image in the low-frequency and high-frequency regions respectively;
[0011] Step S4: Remove the ghosting from the image after Poisson fusion to form the fused result image;
[0012] Step S5: Iteratively fuse according to Steps S2, S3, and S4 to obtain the multi-focus fused image.
[0013] As a further technical solution of the present invention, the method for obtaining the original images of the same position of the sample to be measured with different depths of field, using the first frame of the original image as the standard image, and performing image registration on other original images in terms of scale to form the registered image is as follows:
[0014] Adjust the object distance of the industrial camera, and mark the first frame of the original image at the standard object distance;
[0015] According to the standard object distance, set the object distance differences of other scanned images, and use them as the second frame, the third frame... the nth frame of the original image in sequence;
[0016] Use the first frame as the standard image, perform image registration on other original images in terms of scale with respect to it, and replace the second frame, the third frame... the nth frame of the original image.
[0017] As a further technical solution of the present invention, the method for calculating the gradient matrix by using the Sobel operator for the first frame of the standard image and the registered image specifically includes:
[0018] For the first frame of the standard image and the series of registered images, calculate the gradient matrix through the Sobel operator,
[0019]
[0020] In formula (1), Sx and Sy respectively correspond to the convolution kernels in the horizontal and vertical directions; the gradient calculation formulas in different directions are as shown in (2):
[0021]
[0022] P is the pixel value of the original image; finally, the approximate gradient G of the image can be obtained as shown in (3);
[0023]
[0024] After the local gradient is convolved with Gaussian weighting, the pixel value at any point on the matrix is as shown in (4):
[0025]
[0026] Where k is the gradient coefficient, eliminating the region of the middle depth.
[0027] As a further technical solution of the present invention, the deghosting of the Poisson-fused image to form a fused result image is specifically as follows:
[0028] The pseudo-edge suppression algorithm is adopted. Through corner detection, the edge region of the background replaces the ghost region of the fused image; the image after deghosting is the fused result image of the standard image and the first frame of the registered image.
[0029] As a further technical solution of the present invention, the pseudo-edge suppression algorithm is adopted. Through corner detection, the edge region of the background replaces the ghost region of the fused image; specifically: edge extraction is performed on the fused image. According to any point on the union of corners, it is determined whether there is a corner at the corresponding coordinate of the standard image and the first frame of the registered image, and whether the pixel point of this point is an edge region.
[0030] As a further technical solution of the present invention, the edge extraction of the fused image is performed. According to any point on the union of corners, it is determined whether there is a corner at the corresponding coordinate of the standard image and the first frame of the registered image, and whether the pixel point of this point is an edge region; specifically including:
[0031] If there are corners at this point in both the original image and the registered image, the pixel point with a larger absolute value of the gradient is taken as the edge region;
[0032] If there is a corner at this point in the original image and no corner in the registered image, the pixel point of this point in the original image is taken as the edge region;
[0033] If there is no corner at this point in the original image and there are corners of the original image in the neighborhood, that is, this point is a pseudo-edge, the point in the original image is taken as the flat region;
[0034] If there is no corner at this point in the original image and there are no corners of the original image in the neighborhood, then this point takes the registered image as the edge region.
[0035] The present invention has the following beneficial effects:
[0036] The present invention calculates the gradient matrices of two images with a local weighted gradient operator, performs denoising on the matrices and then uses them as masks for Poisson fusion. Finally, deghosting is performed through pseudo-edge suppression, and an ideal fusion effect can be obtained; in the Poisson fusion, an image fusion algorithm with an assigned weighted gradient operator is used. When processing core images with even dark tones, low contrast, and large sizes, it can still ensure that the core images are not distorted and has good image restoration performance, solving problems such as contour ghosting and gradient noise that are difficult to handle by traditional fusion methods, and is more conducive to actual application scenarios. Description of the Drawings
[0037] Figure 1Flowchart of a core image fusion method for optimizing Poisson fusion proposed by the present invention;
[0038] Figure 2 Schematic diagram of a core image fusion method for optimizing Poisson fusion proposed by the present invention;
[0039] Figure 3 Flowchart of a specific embodiment proposed by the present invention. Detailed implementation manners
[0040] This specification and the claims do not distinguish components by the difference in names, but by the difference in functions of the components. As mentioned throughout the specification and the claims, "comprising" is an open-ended term and should be interpreted as "comprising but not limited to". "Substantially" means within an acceptable error range. Those skilled in the art can solve the technical problems within a certain error range and basically achieve the technical effects.
[0041] The present invention will be further described in detail below with reference to the accompanying drawings, but it is not a limitation to this application.
[0042] Image acquisition: The core image to be fused refers to a 24-bit true color image with a resolution of at least 300 DPI. The acquisition platform is a German Basler industrial camera. The main process is to perform laser ranging and automatic lifting on the rock mass through a scanner to obtain several flat-swept images of the same area with different object distances.
[0043] Image registration: This application takes images as the research object. The primary task is to perform scale registration on the original images. This method first uses the SIFT-RANSAC algorithm on the original images to perform feature point matching and performs affine transformation according to the optimal matrix. Finally, scanned images with the same scale and different depths of field are obtained.
[0044] See Figure 1 and Figure 2 The present invention proposes a core image fusion method for optimizing Poisson fusion, including the following steps:
[0045] Step S1: Obtain original images of the same position of the sample to be measured with different depths of field, use the first frame of the original images as the standard image, and perform scale image registration on the other original images to form registered images;
[0046] Step S2: Calculate the gradient matrix of the standard image and the first frame of the registered image through the Sobel operator;
[0047] Step S3: Denoise the obtained gradient matrix to obtain a binary image, and use this as a mask to perform Poisson fusion on the low-frequency and high-frequency regions of the standard image and the registered image respectively;
[0048] Step S4: Remove the ghosting from the Poisson-fused image to form the fused result image;
[0049] Step S5: Iteratively fuse according to Steps S2, S3, and S4 to obtain the multi-focus fused image.
[0050] The present invention fuses core scan images with different object distances into a single fully clear picture. The method first uses the SIFT-RANSAC algorithm on the original image to perform feature point matching and performs an affine transformation according to the optimal matrix; then uses the Gaussian Sobel operator to construct a gradient matrix and completes Poisson fusion with this as a mask. Finally, a fully clear core image is obtained. To optimize the noise generated during the calculation process, the present invention introduces the 8-neighborhood algorithm and the connected component algorithm to denoise the binary image of the mask. Comparative analysis from six quantitative indicators such as average gradient, information entropy, gradient quality, structural similarity, mean square error, and peak signal-to-noise ratio shows that the algorithm is superior to other algorithms in each phase index and has obvious advantages.
[0051] In Step S1, obtain the original images of the same position of the sample to be measured with different depths of field, use the first frame of the original image as the standard image, and perform image registration on other original images in terms of scale to form the registered images. Specifically:
[0052] Adjust the object distance of the industrial camera and mark the first frame I1 of the original image at the standard object distance;
[0053] According to the standard object distance, set the object distance differences of other scanned images and use them as the second frame I2, the third frame I3... the nth frame In of the original image in sequence;
[0054] Use the first frame as the standard image, perform image registration on other original images in terms of scale, and replace the second frame I2, the third frame I3... the nth frame In of the original image.
[0055] In Step S2, calculate the gradient matrix of the standard image and the first frame of the registered image through the Sobel operator; specifically including:
[0056] For the standard image I1 and the first frame of the registered image series, calculate the gradient matrix through the Sobel operator,
[0057]
[0058] In formula (1), Sx and Sy correspond to the convolution kernels in the horizontal and vertical directions respectively; the gradient calculation formulas in different directions are as shown in (2):
[0059]
[0060] P is the pixel value of the original image; finally, the approximate gradient G of the image can be obtained as shown in (3);
[0061]
[0062] After the local gradient passes through Gaussian weighted convolution, the pixel value at any point on the matrix is as shown in (4):
[0063]
[0064] where k is the gradient coefficient, eliminating the area of intermediate depth.
[0065] In step S3, the gradient matrix obtained after the preliminary calculation in step S2 still has noise, ghosting, and "virtual images" with equal black and white parts caused by lens distortion. These "virtual images" are transitional regions generated because the sharpness of the two images is similar at the intermediate depth of the depth of field. To better optimize the quality of image fusion and ensure the overall visual effect of stitching, the present invention uses 8-neighborhood noise reduction to remove small noise points; and the connected domain denoising method to remove large noise points including contour ghosting.
[0066] In the fusion algorithm of the present invention, it is specifically divided into two parts: depth of field fusion and pseudo-edge suppression fusion.
[0067] The depth of field fusion specifically refers to extracting the clear part of the 42.5 cm object distance image through the denoised mask and performing Poisson fusion on the 45.5 cm object distance image, as Figure 2 shown.
[0068] To further optimize the ghosting caused by distortion, the deghosting of the image after Poisson fusion to form a fused result image is specifically as follows:
[0069] Adopting a pseudo-edge suppression algorithm, through corner detection, the edge region of the background replaces the ghosting region of the fused image; the image after deghosting is the fused result image of the standard image and the first frame of the registered image.
[0070] Among them, the adoption of the pseudo-edge suppression algorithm, through corner detection, makes the edge region of the background replace the ghosting region of the fused image; specifically: performing edge extraction on the fused image, and determining whether the pixel point of this point is an edge region according to whether there is a corner at the corresponding coordinate of any point on the union of corners in the standard image and the first frame of the registered image.
[0071] In the embodiment of the present invention, the edge extraction of the fused image, and determining whether the pixel point of this point is an edge region according to whether there is a corner at the corresponding coordinate of any point on the union of corners in the standard image and the first frame of the registered image; specifically includes:
[0072] If there are corners at this point in both the original image and the registered image, then take the pixel point with the larger absolute value of the gradient as the edge region;
[0073] If there is a corner point at this point in the original image but no corner point in the registered image, take this point as the pixel point of the original image as the edge region;
[0074] If there is no corner point at this point in the original image and there are corner points of the original image in the neighborhood, that is, this point is a pseudo-edge, take the original image point as the flat region;
[0075] If there is no corner point at this point in the original image and there are no corner points of the original image in the neighborhood either, then take the registered image at this point as the edge region.
[0076] To more objectively analyze the performance and effect of different fusion algorithms, the present invention uses Matlab 2018b as the platform to conduct image quality evaluation experiments on the fusion results. On the premise of mainly visual evaluation, six indicators, namely Average Gradient (AG), Information Entropy (IE), Gradient Quality (QAB / F), Structural Similarity (SSIM), Mean Square Error (MSE), and Peak Signal to Noise Ratio (PNSR), are selected as the criteria for quantitative analysis. Among them, Table 1 is the comparison of clarity evaluation indicators;
[0077] Table 1
[0078] Fusion algorithm AG IE QAB / F Sobel high-pass 2.7454 6.7434 0.3673 Gradient pyramid 2.6626 6.8808 0.1667 Wavelet fusion 2.7495 6.7725 0.5685 The algorithm of the present invention 2.8746 6.8903 0.5959
[0079] The average gradient is the clarity, which is used to reflect the detail contrast and texture change of the image. The larger the value, the richer the details; the information entropy is used to express the richness of information on the image. The larger the value, the more information it contains; QAB / F is an evaluation index of gradient quality. The larger the value, the more obvious the image edge. The above indicators can be used as the measurement criteria for image clarity.
[0080] Table 2 is the comparison of similarity evaluation indicators:
[0081] Table 2
[0082] Fusion algorithm SSIM MSE PNSR Sobel high-pass 0.9658 7.477×10-4 0.3673 Gradient pyramid 0.5841 59.0013×10-4 0.1667 Wavelet fusion 0.9521 8.7175×10-4 0.5685 The algorithm of the present invention 0.9697 7.2984×10-4 0.5959
[0083] The structural similarity can reflect the similarity degree between the fused image and the original image. The larger the value, the more similar; the mean square error represents the difference in pixel values between two images. The smaller the value, the smaller the difference; the peak signal to noise ratio represents the ratio of the maximum value signal to the noise. In the image, the higher the value, the less distortion and the better the fusion effect.
[0084] The algorithm of the present invention integrates the pixel value sources of images, mainly replacing local parts of the original image. Therefore, it has excellent data performance in terms of both clarity and similarity. As can be seen from the experimental results in Table 1 and Table 2, its various indicators are better than those of other algorithms.
[0085] The gradient operator fusion principle in the present invention is based on the precise replacement of the focused area, rather than resampling or some approximate expression. Therefore, whether subjectively or objectively, this algorithm has obvious advantages compared with traditional fusion algorithms in terms of effect. The conducted comparative experiments also prove that the algorithm of the present invention has a good evaluation result in terms of fusion effect, which is better than the fusion effects of other traditional algorithms.
[0086] The present invention proposes an image fusion algorithm that assigns and weights the gradient operator in Poisson fusion. When processing core images that are even dark in tone, low in contrast, and large in size, it can still ensure that the core images are not distorted and has good image restoration performance. It better solves problems such as contour ghosting and gradient noise that are difficult to handle by traditional fusion methods, and is more conducive to actual application scenarios.
[0087] This specific embodiment is only an interpretation of the present invention and is not a limitation thereof. Those skilled in the art can make modifications to this embodiment without creative contributions according to needs after reading this specification, but as long as it is within the scope of the claims of the present invention, it is protected by the patent law.
Claims
1. A core image fusion method for optimizing Poisson fusion, characterized in that, It includes the following steps: Step S1: Obtain the original images of the same position of the sample to be measured with different depths of field, use the first frame of the original images as the standard image, and perform image registration on the other original images in terms of scale to form registered images; Step S2: Calculate the gradient matrices of the standard image and the first frame of the registered images through the Sobel operator; Step S3: Denoise the obtained gradient matrices into binary images, and use them as masks to perform Poisson fusion on the low-frequency and high-frequency regions of the standard image and the registered images respectively; Step S4: Remove the ghosting from the images after Poisson fusion to form the fused result images; Step S5: Iteratively fuse according to Steps S2, S3, and S4 to obtain the multi-focus fused images.
2. The core image fusion method according to claim 1, wherein The obtaining of the original images of the same position of the sample to be measured with different depths of field, using the first frame of the original images as the standard image, and performing image registration on the other original images in terms of scale to form registered images is specifically as follows: Adjust the object distance of the industrial camera and mark the first frame of the original image at the standard object distance; According to the standard object distance, set the object distance differences of other scanned images and use them as the second frame, the third frame... the nth frame of the original images in turn; Use the first frame as the standard image, perform image registration on the other original images in terms of scale, and replace the second frame, the third frame... the nth frame of the original images.
3. An optimized Poisson fusion core image fusion method according to claim 1, characterized in that, The calculating of the gradient matrices of the standard image and the first frame of the registered images through the Sobel operator specifically includes: Calculate the gradient matrices of the standard image and the first frame of the registered image series through the Sobel operator, S in formula (1) x and S y correspond to the convolution kernels in the horizontal and vertical directions respectively; the gradient calculation formulas in different directions are shown in (2): P is the pixel value of the original image; finally, the approximate gradient G of the image can be obtained as shown in (3); After the local gradient is convolved with Gaussian weighting, the pixel value at any point on the matrix is as shown in (4): where k is the gradient coefficient to eliminate the area of the intermediate depth.
4. An optimized Poisson fusion core image fusion method according to claim 1, characterized in that The removing of the ghosting from the images after Poisson fusion to form the fused result images is specifically as follows: Adopt the pseudo-edge suppression algorithm, and through corner detection, replace the ghosting area of the fused image with the edge area of the background; the image after removing the ghosting is the fused result image of the standard image and the first frame of the registered image.
5. An optimized Poisson fusion core image fusion method according to claim 4, characterized in that The adopting of the pseudo-edge suppression algorithm, and through corner detection, replace the ghosting area of the fused image with the edge area of the background is specifically as follows: Perform edge extraction on the fused image, and determine whether the pixel point of this point is an edge area according to whether there is a corner at the corresponding coordinate of this point on the first frame of the standard image and the registered image.
6. The core image fusion method for optimizing Poisson fusion according to claim 5, wherein, The performing of edge extraction on the fused image, and determining whether the pixel point of this point is an edge area according to whether there is a corner at the corresponding coordinate of this point on the first frame of the standard image and the registered image specifically includes: If there are corners at this point in both the original image and the registered image, take the pixel point with the larger absolute value of the gradient as the edge area; If there is a corner at this point in the original image and no corner in the registered image, take the pixel point of this point in the original image as the edge area; If there is no corner at this point in the original image and there is a corner of the original image in the neighborhood, that is, this point is a pseudo-edge, take the point in the original image as the flat area; If there is no corner at this point in the original image and there is no corner of the original image in the neighborhood either, then take the registered image of this point as the edge area.