Multi-focus image fusion method based on SAM segmentation large model

Abstract

The invention relates to image fusion, in particular to a multi-focus image fusion method based on an SAM segmentation large model, and the method comprises the steps: carrying out the image preprocessing of a multi-focus image, and calculating the local contrast of the multi-focus image; based on the local contrast of the multi-focus image, performing image segmentation on the multi-focus image by using an SAM segmentation large model to obtain a segmentation mask of the multi-focus image; performing multi-focus image fusion according to the segmentation mask of the multi-focus image; according to the technical scheme provided by the invention, the defect that a multi-focus image fusion result is not clear enough in the prior art can be effectively overcome.

Description

Technical Field

[0001] The invention relates to image fusion, and in particular to a multi-focus image fusion method based on a SAM segmentation large model. Background Art

[0002] With the rapid development of computer vision technology, image fusion, as one of its important branches, has received more and more attention. Multi-focus image fusion technology, as a special image fusion method, can fuse multiple images taken with different focal lengths in the same scene into a full-clear image, thereby effectively solving the problem of some areas of the image being clear and some areas being blurred due to the depth of field limitation of the camera lens.

[0003] With the rapid development of deep learning technology, multi-focus image fusion methods based on deep learning have also received widespread attention. This type of method can achieve better image fusion effects than traditional methods by training deep neural networks to learn the laws of image fusion. However, when dealing with complex scenes, multi-focus image fusion methods based on deep learning often find it difficult to accurately identify and extract the focus area in the source image, resulting in artifacts and loss of edge contours in the fusion results. In addition, when dealing with large-scale image data, existing image fusion methods have high computational complexity and poor real-time performance, making it difficult to meet the needs of practical applications.

[0004] In general, the existing multi-focus image fusion methods have the following defects:

[0005] 1) Lack of ability to enhance the focused area and suppress the non-focused area: The existing multi-focus image fusion method does not specifically enhance the details of the focused area and suppress the details of the non-focused area, which may cause the fusion result to have blurred or distorted details, affecting the quality and clarity of the image;

[0006] 2) There are deficiencies in determining fusion weights: Existing multi-focus image fusion methods usually determine fusion weights based on the comparison of extracted features. This approach may not fully consider the importance and contribution of different regions in the image, resulting in inaccurate or unsatisfactory fusion results;

[0007] 3) Lack of effective control over object clarity: Existing multi-focus image fusion methods often lack an effective mechanism to control the clarity of objects in the fusion results, resulting in insufficient details in some focused areas or residual details in non-focused areas during the fusion process.

[0008] In recent years, the successful application of large models such as SAM (SegmentAnythingModel) in the field of image segmentation has provided new ideas for multi-focus image fusion. The SAM segmentation large model has powerful image segmentation capabilities and can accurately identify and segment different areas in the image, providing an effective solution for the extraction of focus areas in multi-focus image fusion. Summary of the invention

[0009] 1. Technical issues to be solved

[0010] In view of the above-mentioned shortcomings of the prior art, the present invention provides a multi-focus image fusion method based on the SAM segmentation large model, which can effectively overcome the defect of the prior art that the multi-focus image fusion result is not clear enough.

[0011] (II) Technical solution

[0012] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0013] A multi-focus image fusion method based on a SAM segmentation large model comprises the following steps:

[0014] S1, performing image preprocessing on the multi-focus image to calculate the local contrast of the multi-focus image;

[0015] S2, based on the local contrast of the multi-focus image, the multi-focus image is segmented using the SAM segmentation model to obtain a segmentation mask of the multi-focus image;

[0016] S3. Perform multi-focus image fusion according to the segmentation mask of the multi-focus image.

[0017] Preferably, performing image preprocessing on the multifocus image in S1 to calculate the local contrast of the multifocus image includes:

[0018] S11, using stationary wavelet transform to decompose each image into 2 layers, and obtain corresponding stationary wavelet coefficients {A, D};

[0019] S12, performing a stationary wavelet inverse transform on the corresponding image based on the stationary wavelet coefficients {A, γD} to obtain a local background component of each image;

[0020] S13, calculating the local contrast of each image according to the local background component of each image;

[0021] Among them, A is the scale coefficient, which reflects the low-frequency information of the image; D is the detail coefficient, which reflects the high-frequency information of the image; γ is the detail control factor, γ = 0.5;

[0022] Multi-focus images are multiple images taken at different focal lengths under the same scene.

[0023] Preferably, in S13, the local contrast of each image is calculated according to the local background component of each image, including:

[0024] Image I i The local contrast is calculated using the following formula:

[0025]

[0026] Among them, LC i (x,y) is the i-th image I i The contrast at the pixel (x, y), p i (x,y) is the i-th image I i The pixel value at the pixel point (x, y), LB i (x,y) is the i-th image I i The background component at the pixel point (x, y), i∈[1,K], K is the number of images contained in the multi-focus image.

[0027] Preferably, in S2, based on the local contrast of the multi-focus image, the multi-focus image is segmented using the SAM segmentation model to obtain a segmentation mask of the multi-focus image, including:

[0028] The local contrast of each image is input into the trained SAM segmentation model to obtain the segmentation mask of each image.

[0029] Preferably, obtaining the segmentation mask of each image includes:

[0030] Image I i The segmentation mask is expressed as follows:

[0031]

[0032] Among them, Mask i (x,y) is the i-th image I i Segmentation mask at pixel (x,y).

[0033] Preferably, performing multi-focus image fusion according to the segmentation mask of the multi-focus image in S3 includes:

[0034] S31, determine the focus area R of all images according to the segmentation mask of each image F and the non-focus area R NF ;

[0035] S32, performing multi-focus image fusion according to the segmentation mask of the multi-focus image to generate an all-focus image;

[0036] Among them, the focus area R F is the union of the focused areas in each image, and the non-focused area R NF =1-R F .

[0037] Preferably, in S32, performing multi-focus image fusion according to the segmentation mask of the multi-focus image to generate an all-focus image includes:

[0038] The following formula is used to perform multi-focus image fusion on all images to generate an all-focus image:

[0039]

[0040] Among them, I F (x,y) is the all-focus image I F The pixel value at the pixel point (x, y);

[0041] Represents the all-focus image I F Preserved and enhanced the focus areas of all images; Represents the all-focus image I F In the method, for pixels in the non-focused area of ​​all images, the average pixel value of the pixel in all images is used to ensure the stability of image quality.

[0042] (III) Beneficial effects

[0043] Compared with the prior art, the multi-focus image fusion method based on the SAM segmentation large model provided by the present invention can accurately identify the focused area and the non-focused area in the image by introducing the SAM segmentation large model, and generate corresponding segmentation masks, and then use these segmentation masks as fusion weights, which can more effectively measure the contribution of different areas in the image and reduce the interference of non-focused areas in the image, so as to generate a fully clear full-focus image, effectively solve the problem of blurred edge contours caused by incomplete information extraction in the multi-focus image fusion process, improve the quality and clarity of image fusion, and have broad application prospects in medical image analysis, security monitoring, target recognition and other fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0045] Figure 1 It is a schematic diagram of the process of the present invention;

[0046] Figure 2 It is a detailed schematic diagram of the process of the present invention;

[0047] Figure 3 It is a simplified schematic diagram of multi-focus image fusion when the multi-focus image only includes a near-focus image and a far-focus image (K=2) in the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] A multi-focus image fusion method based on the SAM segmentation model, such as Figure 1 and Figure 2 As shown, S1, image preprocessing is performed on the multi-focus image to calculate the local contrast of the multi-focus image, specifically including:

[0050] S11, using stationary wavelet transform to decompose each image into 2 layers, and obtain corresponding stationary wavelet coefficients {A, D};

[0051] S12, performing a stationary wavelet inverse transform on the corresponding image based on the stationary wavelet coefficients {A, γD} to obtain a local background component of each image;

[0052] S13, calculating the local contrast of each image according to the local background component of each image;

[0053] Among them, A is the scale coefficient, which reflects the low-frequency information of the image; D is the detail coefficient, which reflects the high-frequency information of the image; γ is the detail control factor, γ = 0.5;

[0054] Multi-focus images are multiple images taken at different focal lengths under the same scene.

[0055] Specifically, in S13, the local contrast of each image is calculated according to the local background component of each image, including:

[0056] Image I i The local contrast is calculated using the following formula:

[0057]

[0058] Among them, LC i (x,y) is the i-th image I iThe contrast at the pixel (x, y), p i (x,y) is the i-th image I i The pixel value at the pixel point (x, y), LB i (x,y) is the i-th image I i The background component at the pixel point (x, y), i∈[1,K], K is the number of images contained in the multi-focus image.

[0059] S2. Based on the local contrast of the multi-focus image, the multi-focus image is segmented using the SAM segmentation model to obtain a segmentation mask of the multi-focus image, which specifically includes:

[0060] The local contrast of each image is input into the trained SAM segmentation model to obtain the segmentation mask of each image.

[0061] Specifically, the segmentation mask of each image is obtained, including:

[0062] Image I i The segmentation mask is expressed as follows:

[0063]

[0064] Among them, Mask i (x,y) is the i-th image I i Segmentation mask at pixel (x,y).

[0065] S3, performing multi-focus image fusion according to the segmentation mask of the multi-focus image, specifically including:

[0066] S31, determine the focus area R of all images according to the segmentation mask of each image F and the non-focus area R NF ;

[0067] S32, performing multi-focus image fusion according to the segmentation mask of the multi-focus image to generate an all-focus image;

[0068] Among them, the focus area R F is the union of the focused areas in each image, and the non-focused area R NF =1-R F .

[0069] Specifically, in S32, multi-focus image fusion is performed according to the segmentation mask of the multi-focus image to generate an all-focus image, including:

[0070] The following formula is used to perform multi-focus image fusion on all images to generate an all-focus image:

[0071]

[0072] Among them, I F (x,y) is the all-focus image I F The pixel value at the pixel point (x, y);

[0073] Represents the all-focus image I F Preserved and enhanced the focus areas of all images; Represents the all-focus image I F In the method, for pixels in the non-focused area of ​​all images, the average pixel value of the pixel in all images is used to ensure the stability of image quality.

[0074] In the technical solution of the present application, by introducing the SAM segmentation large model, the focused area and non-focused area in the image can be accurately identified, and the corresponding segmentation masks can be generated. These segmentation masks are then used as fusion weights to more effectively measure the contribution of different areas in the image and reduce the interference of non-focused areas in the image, thereby generating a fully clear and full-focus image, effectively solving problems such as blurred edge contours caused by incomplete information extraction in the multi-focus image fusion process, improving the quality and clarity of image fusion, and having broad application prospects in medical image analysis, security monitoring, target recognition and other fields.

[0075] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-focus image fusion method based on a SAM segmentation model, characterized in that: The following steps are involved: S1, performing image preprocessing on the multi-focus image to calculate the local contrast of the multi-focus image; S2, based on the local contrast of the multi-focus image, the multi-focus image is segmented using the SAM segmentation model to obtain a segmentation mask of the multi-focus image; S3. Perform multi-focus image fusion according to the segmentation mask of the multi-focus image.

2. The multi-focus image fusion method based on the SAM segmentation large model according to claim 1 is characterized in that: In S1, the multi-focus image is preprocessed to calculate the local contrast of the multi-focus image, including: S11, using stationary wavelet transform to decompose each image into 2 layers to obtain corresponding stationary wavelet coefficients {A, D}; S12, performing a stationary wavelet inverse transform on the corresponding image based on the stationary wavelet coefficients {A, γD} to obtain a local background component of each image; S13, calculating the local contrast of each image according to the local background component of each image; Among them, A is the scale coefficient, which reflects the low-frequency information of the image; D is the detail coefficient, which reflects the high-frequency information of the image; γ is the detail control factor, γ = 0.5; Multi-focus images are multiple images taken at different focal lengths under the same scene.

3. The multi-focus image fusion method based on the SAM segmentation large model according to claim 2 is characterized in that: In S13, the local contrast of each image is calculated according to the local background component of each image, including: Image I i The local contrast is calculated using the following formula: Among them, LC i (x,y) is the i-th image I i The contrast at the pixel (x, y), p i (x,y) is the i-th image I i The pixel value at the pixel point (x, y), LB i (x,y) is the i-th image I i The background component at the pixel point (x, y), i∈[1,K], K is the number of images contained in the multi-focus image.

4. The multi-focus image fusion method based on the SAM segmentation large model according to claim 2 is characterized in that: In S2, based on the local contrast of the multi-focus image, the SAM segmentation model is used to segment the multi-focus image to obtain the segmentation mask of the multi-focus image, including: The local contrast of each image is input into the trained SAM segmentation model to obtain the segmentation mask of each image.

5. The multi-focus image fusion method based on the SAM segmentation large model according to claim 4 is characterized in that: The step of obtaining the segmentation mask of each image includes: Image I i The segmentation mask is expressed as follows: Among them, Mask i (x,y) is the i-th image I i Segmentation mask at pixel (x,y).

6. The multi-focus image fusion method based on the SAM segmentation large model according to claim 5 is characterized in that: In S3, multi-focus image fusion is performed according to the segmentation mask of the multi-focus image, including: S31, determine the focus area R of all images according to the segmentation mask of each image F and the non-focus area R NF ; S32, performing multi-focus image fusion according to the segmentation mask of the multi-focus image to generate an all-focus image; Among them, the focus area R F is the union of the focused areas in each image, and the non-focused area R NF =1-R F .

7. The multi-focus image fusion method based on the SAM segmentation large model according to claim 6 is characterized in that: In S32, multi-focus image fusion is performed according to the segmentation mask of the multi-focus image to generate an all-focus image, including: The following formula is used to perform multi-focus image fusion on all images to generate an all-focus image: Among them, I F (x,y) is the all-focus image I F The pixel value at the pixel point (x, y); Represents the all-focus image I F Preserved and enhanced the focus areas of all images; Represents the all-focus image I F In the method, for pixels in the non-focused area of ​​all images, the average pixel value of the pixel in all images is used to ensure the stability of image quality.

Images