Multi-focus image fusion method and system based on large selective kernel network

Abstract

The invention discloses a multi-focus image fusion method and system based on a large-scale selective kernel network, and the method comprises the steps: carrying out the processing of a multi-focus image through a large-scale selective kernel module of the large-scale selective kernel network, and extracting a multi-scale global feature; extracting local information through a combined differential convolution block of a large-scale selective kernel network according to the multi-scale global features; generating a focus decision diagram in combination with the global features and the local information; and according to the focus decision diagram, fusing the multi-focus image. According to the invention, a large-scale selective kernel network is utilized to process a multi-focus image, and the network can dynamically adjust a wide space receptive field, effectively expand the receptive field, capture multi-scale global information, and fully extract and integrate multi-scale backgrounds, so as to better identify various focusing areas in the image, and improve the image quality. The problem of insufficient feature extraction expressive force in a traditional method is solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of multi-focus image fusion, and in particular relates to a multi-focus image fusion method and system based on a large selective kernel network. Background Art

[0002] Multi-focus Image Fusion (MFIF) technology is an image enhancement method that aims to compensate for the limited depth of field of optical lenses. In many fields such as digital photography, biomedical imaging, and machine vision, MFIF technology generates a composite image that is clear throughout the entire image by merging source images with different focus information. Traditional MFIF methods are mainly divided into transform domain-based and spatial domain-based methods. Transform domain techniques decompose the image into multiple coefficients, then merge them using specific fusion rules, and finally obtain a fully focused image through inverse transformation. These techniques include using pyramid transform, wavelet transform, discrete cosine transform (DCT), and non-subsampled contourlet transform (NSCT). Spatial domain methods determine focus information pixel by pixel, measure the clarity of pixels according to certain standards, select clearer pixels, and then directly merge pixel values ​​to obtain a fully focused image.

[0003] Although traditional methods have made some progress in image fusion, they have some limitations. First, traditional methods usually use a unified transformation between different source images for feature extraction, which ignores the inherent differences in features between source images and may lead to insufficient expressiveness of the extracted features. Second, traditional feature fusion strategies are relatively simple, resulting in limited performance in terms of fusion effects. In addition, transform domain methods may have errors in pixel intensity during decomposition, fusion, and reconstruction, resulting in the fused image lacking the clarity in the original source image. Spatial domain methods may rely on accurate segmentation of the source image due to the choice of block size, resulting in blocking effects and blurred edge areas. In recent years, the development of deep learning technology has brought new research directions to the field of MFIF, but existing deep learning-based fusion methods have difficulties in capturing global features, resulting in the degradation of color and texture features, and challenges in distinguishing adjacent out-of-focus pixels from focused pixels. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes a multi-focus image fusion method and system based on a large selective kernel network to solve the problems existing in the above prior art.

[0005] To achieve the above objectives, in a first aspect, the present invention provides a multi-focus image fusion method based on a large selective kernel network, comprising:

[0006] The large selective kernel module of the large selective kernel network is used to process multi-focus images and extract multi-scale global features;

[0007] According to the multi-scale global features, local information is extracted through a combined differential convolution block of a large selective kernel network;

[0008] Combining the global features and the local information to generate a focus decision map;

[0009] The multi-focus images are fused according to the focus decision graph.

[0010] Preferably, the large selective kernel module is used to dynamically adjust the receptive field of the feature extraction backbone; wherein the large selective kernel module includes decomposing the serial components of the large convolution to adaptively construct long-distance dependencies.

[0011] Preferably, the combined differential convolution block is a module integrating spatial attention and gated linear units; wherein the combined differential convolution block includes: center differential convolution, angular differential convolution, horizontal differential convolution and vertical differential convolution.

[0012] Preferably, generating a focal decision diagram comprises:

[0013] The L1 loss function is used to calculate the loss during the training process. The global information of different images is captured through the loss to obtain the focus map.

[0014] The focus map is processed using the maximum selection strategy to obtain the initial decision map;

[0015] The initial decision graph is processed by using a small region removal method to obtain a final decision graph.

[0016] Preferably, fusing the multi-focus source images comprises:

[0017] According to the weight distribution diagram provided by the final decision diagram, weighted fusion is performed on the multi-focus source images to obtain a fused image; wherein the fusion of multiple source images is simplified into a fusion process of several two source images.

[0018] In a second aspect, the present invention further discloses a multi-focus image fusion system based on a large selective kernel network, comprising:

[0019] A first acquisition module is used for processing the multi-focus image using a large selective kernel module of a large selective kernel network to extract multi-scale global features;

[0020] A second acquisition module is used to extract local information through a combined differential convolution block of a large selective kernel network according to the multi-scale global features;

[0021] A decision diagram generation module, used to generate a focus decision diagram by combining the global features and local information;

[0022] An image fusion module is used to fuse the multi-focus images according to the focus decision map.

[0023] In a third aspect, the present invention further discloses a computer device, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in the first aspect.

[0024] In a fourth aspect, the present invention further discloses a computer-readable storage medium having a computer program stored thereon, which implements the steps of the method described in the first aspect when executed by a processor.

[0025] In a fifth aspect, the present invention further discloses a computer program product, including a computer program, which implements the steps of the method described in the first aspect when executed by a processor.

[0026] Compared with the prior art, the present invention has the following advantages and technical effects:

[0027] The present invention provides a multi-focus image fusion method based on a large selective kernel network. Firstly, a large selective kernel module of the large selective kernel network is used to process the multi-focus images to extract multi-scale global features. Secondly, according to the multi-scale global features, local information is extracted by a combined differential convolution block of the large selective kernel network. Then, a focus decision graph is generated by combining the global features and the local information. Finally, according to the focus decision graph, the multi-focus images are fused.

[0028] The present invention uses a large selective kernel network to process multi-focus images, which can dynamically adjust its wide spatial receptive field, effectively expand the receptive field and capture multi-scale global information. This method fully extracts and integrates multi-scale background to better identify various focus areas in the image, solving the problem of insufficient feature extraction expressiveness in traditional methods.

[0029] The present invention decomposes large convolutions through large selective kernel modules, uses spatial feature selection for feature aggregation, and then extracts local information by combining differential convolution blocks to enhance the reproduction capability and generate accurate decision graphs. This method solves the problem of simple feature fusion strategy and limited performance in traditional methods, and can reduce the impact of useless channel information and improve network performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0031] Figure 1 Schematic diagram of the framework of LKSN-MFIF according to an embodiment of the present invention; wherein (a) represents the composition of the entire network, (b) represents the large selective kernel module (LKSB) in the network structure, (c) represents the structure of the combined differential module (GDCB), and (d) represents the Monte Carlo attention mechanism;

[0032] Figure 2 A schematic diagram of the integration process of a training data set according to an embodiment of the present invention;

[0033] Figure 3 These are some images in the Lytro, MFFW and MFI-WHU datasets of the embodiments of the present invention. DETAILED DESCRIPTION

[0034] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0035] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0036] First, the technical terms involved in the following embodiments are explained.

[0037] (1) Multi-focus image refers to the process of combining multiple images with different focal points into a single all-focus image through depth of field fusion technology. This technology aims to solve the problem of blurring of certain areas of a single image due to depth of field limitation, thereby obtaining clear details of all focal areas in one image at the same time. Multi-focus image fusion technology is widely used in medical imaging, microscopy imaging, and machine vision to improve image clarity and detail.

[0038] Depth of field fusion technology achieves image synthesis through the following steps:

[0039] Multi-focus image acquisition: First, multiple images with different focus points are taken. Some areas in each image are clear, while other areas may be blurred.

[0040] Feature extraction: Perform feature extraction on each image to identify which areas are sharp. This is usually measured using gradient, Laplace transform or other edge detection methods.

[0041] Multi-focus image fusion technology is of great significance in many fields:

[0042] Medical imaging: In medical imaging, multi-focus image fusion can help doctors observe and analyze complex anatomical structures more clearly.

[0043] Microscope images: In microscope images, multi-focus image fusion technology can provide a wider and clearer field of view, helping researchers to better observe and analyze microscopic structures.

[0044] Machine vision: In machine vision applications, multi-focus image fusion can improve image clarity and detail, thereby enhancing the recognition and detection capabilities of automated systems.

[0045] (2) Large Selective Kernel Network (LSKNet) is a network architecture designed for remote sensing target detection. Its core advantage lies in its ability to dynamically adjust a large spatial receptive field to better capture the range context of different objects in remote sensing scenes. LSKNet works together through the Large Kernel Selection (LK Selection) sub-block and the Feed-Forward Network (FFN) sub-block to provide a wide range of contextual information while maintaining sensitivity to details, which is particularly important for remote sensing target detection.

[0046] The basic principle of LSKNet includes the following key components:

[0047] Large Kernel Selection (LK Selection) sub-block: This sub-block is able to dynamically adjust the network’s receptive field to capture contextual information of different scales as needed. This enables the network to adjust its processing capabilities to the different sizes and complexities of objects in remote sensing images.

[0048] Feedforward Network (FFN) sub-block: This sub-block is used for channel mixing and feature refinement. It consists of a fully connected layer, a depthwise convolution, a GELU activation function, and a second fully connected layer. These components work together to improve the quality of features and provide necessary information for classification and detection.

[0049] LSKNet achieves state-of-the-art performance in remote sensing classification, object detection, and semantic segmentation tasks. Experiments show that LSKNet has an average gain of 0.04-0.1 on datasets containing large and small object detection, and sometimes even no gain. Despite this, LSKNet is still considered an effective lightweight model for remote sensing object detection tasks.

[0050] In view of the technical problems existing in the prior art in the background technology, this embodiment provides a multi-focus image fusion method based on a large selective kernel network, including:

[0051] The large selective kernel module of the large selective kernel network is used to process multi-focus images and extract multi-scale global features;

[0052] According to the multi-scale global features, local information is extracted through a combined differential convolution block of a large selective kernel network;

[0053] Combining the global features and the local information to generate a focus decision map;

[0054] The multi-focus images are fused according to the focus decision graph.

[0055] like Figure 1 As shown, a large selective kernel network framework LKSN-MFIF for multi-focus image fusion is shown. LKSN-MFIF mainly consists of three LG blocks, each of which consists of a large selective kernel module (LKSB) and a combined differential convolution (GDCB). Specifically, this embodiment trains LKSN-MFIF to output a focus decision graph after processing a pair of multi-focus images.

[0056] LKSN-MFIF is divided into four components: feature fusion, feature extraction, discrimination task, and generation task. First, the two input source images are spliced ​​along the channel dimension using channel splicing, and then a 3×3 convolution operation is used to expand the input source image from 3×2 channels to 72 channels, and the activation function uses the rectified linear unit (ReLU). Figure 1 As shown, a large selective kernel module (LKSB) is designed to obtain multi-scale global features from different receptive fields. It consists of three large kernel convolutions and spatial feature selection, where most selective kernel networks use traditional global average pooling, which has limited ability to capture cross-scale correlations. Therefore, this embodiment uses the Monte Carlo attention mechanism instead of average pooling to enable the model to capture key information at different scales. After the LKSB module, this embodiment integrates simple spatial attention and gated linear units into the proposed differential convolution, which this embodiment calls combined differential convolution (GDCB) to reduce useless channel information and enhance the generalization ability of the model. These two modules constitute the main part of the network LG, where LKSB captures global information and GDCB ​​captures local features, working together to reduce network parameters and improve network performance. In addition, 3×3 convolution and activation functions are used to map the image to the range of 0-1. Finally, image fusion is performed according to the fusion strategy of this embodiment. Table 1 shows the comprehensive parameter configuration of each layer.

[0057] Table 1

[0058]

[0059]

[0060] The large selective kernel network framework specifically includes:

[0061] First, selective large nuclear block (LKSB);

[0062] As an additional implementation, LKSB implements modeling of a series of multiple long-range contexts for adaptive selection. Specifically, LKSB consists of a series of large kernel convolutions and a spatial kernel selection mechanism.

[0063] Given an input feature map X∈R C×H×W The large kernel convolution adaptively builds long-distance dependencies by decomposing the K×K convolution into three serial components: a (2d-1)×(2d-1) deep convolution DWConv(·), a The depth dilated convolution DWConv(·) and point-by-point convolution PConv(·) are shown in formula (1).

[0064] LK(X)=PConv(DWConv(DWDConv(X))) (1)

[0065] In order to obtain features with rich contextual information from different ranges of the input X, a series of decomposed depthwise convolutions with different receptive fields are applied. As shown in formula (3), the vector Decompose into and Then, we use the large kernel convolution formula (4) to obtain the features LK under different receptive fields. i , Norm(·) represents the normalization layer. First, such a design produces multiple features with various large receptive fields, which makes the subsequent kernel selection easier. Second, the sequential decomposition is more effective than simply applying a single larger kernel. Under the same theoretical receptive field, the decomposition of this embodiment greatly reduces the number of parameters compared with the standard large convolution kernel.

[0066]

[0067] This embodiment introduces Monte Carlo attention (MCAttn) instead of general average pooling, which learns to focus on the spatial information of the scene in a single stage by using an unknown pooling output size to generate attention maps of different scales. Specifically, MCattn generates scale-independent attention maps through a pooling operation based on random sampling, thereby enhancing the network's ability to capture multi-scale information and improving recognition performance. Specifically, MCattn randomly selects a 1×1 attention map from three scales of 3×3, 2×2, and 1×1 as the output (merged tensor). In traditional squeeze-excitation (SE) methods, global average pooling is usually used to generate a 1×1 output tensor to calibrate the interdependence between channels. However, this method has limitations in exploiting cross-scale correlations. To overcome this limitation, MCattn enhances long-term semantic interdependence by calculating attention maps from features at three different scales. Given an input tensor x, the output attention map of MCattn is denoted as Am x , as shown in formula (5).

[0068]

[0069] Where i represents the output size of the attention map, and f(x,i) represents the average pooling function. The association probability P1(x,i) of the attention map satisfies the condition and This ensures that the attention map is generated with agnostic and good generalizability. n represents the number of output pooling tensors, which is set to 3 in this embodiment.

[0070] In order to enable the network to obtain the most relevant spatial context information to detect the focus area, this embodiment uses a spatial selection mechanism to spatially select feature maps from large convolution kernels of different scales. First, as shown in formula (6), this embodiment connects the information features obtained from different receptive fields using different kernels.

[0071] U=[LK1;…;LK i ] (6)

[0072] Then, as shown in formula (7), a channel-based random sampling pooling operation is used to effectively extract the spatial relationship Am U Formula (8) uses convolution F C→N The pooled features (with C channels) are converted into an N-dimensional spatial attention map. For the spatial attention map SF, as shown in formula (9), σ(·) is applied to obtain a single spatial selection mask SF for each decomposed large kernel i, where σ(·) represents the sigmoid activation function. Formula (10) indicates that the feature map from the decomposed large kernel sequence is weighted by the corresponding spatial selection mask, and then fused through the convolution layer F(·) to obtain the attention feature S. Finally, the product of the input feature X and the selective feature S is shown in formula (11).

[0073]

[0074] SF=F C→N (Am U ) (8)

[0075] SF i =σ(SF i ) (9)

[0076]

[0077] Second, combine differential convolution blocks;

[0078] As an additional implementation, this embodiment integrates simple spatial attention and gated linear units into the proposed convolution block, solving the problem of large parameters of MLP with wide intermediate channels, especially for large input images. In order to enhance the representation and generalization capabilities, this embodiment integrates combined differential convolutions. Specifically, the center differential convolution (CDC), angular differential convolution (ADC), horizontal differential convolution (HDC) and vertical differential convolution (VDC) proposed by Chen et al. (Chen Z, He Z, Lu Z M. DEA-Net: Single image dehazing based on detail-enhanced convolution and content-guided attention [J]. IEEE Transactions on Image Processing, 2024.) are used to integrate traditional local descriptors into the convolution layer. Given densely transformed A and B, the key process of GDCB ​​is shown in formula (13).

[0079]

[0080] GCCB(A,B)=CCB(DWConv(A))·B (13)

[0081] Where CCB(·) represents the combined convolution operation, as shown in formula (12), K i The kernels represent CDC, ADC, HDC, and VDC respectively, which combines 4 differential convolutions in parallel through summation through spatial gating, thereby removing nonlinear layers and capturing local continuity in fairly complex situations.

[0082] Third, the loss function;

[0083] The loss function is one of the important indicators to measure the performance during the model training process. It can not only reflect the gap between the model prediction results and the actual situation, but also guide the model parameter update, thereby optimizing the model performance. In this study, this embodiment uses the L1 loss function shown in formula (14) to calculate the loss generated during the training process. Compared with other losses, the L1 loss is more robust to outliers and can prevent some extreme samples from having too much impact on the loss function. This feature is particularly important in image processing tasks, because image data usually contains complex and diverse content, and changes in different pixels may have a significant impact on model predictions. Through the L1 loss, the model can better capture the global information of different images while avoiding over-reliance on abnormal differences in local pixels. The formula is shown in (14).

[0084]

[0085] Among them, n represents the number of training samples in this batch, y i and They represent the pixel value at (x, y) of the reference image of the i-th image and the pixel value at (x, y) of the i-th predicted decision graph. By using L1 loss, this embodiment expects the network to be able to restore the image structure information more accurately and ultimately perform well in the target task.

[0086] Fourth, integration strategy;

[0087] Due to the limited representation capability of the neural network and the irregular noise in the image, the focus map generated by the training network will contain misjudged labels. In this embodiment, the "select the maximum" strategy is used to process the focus map. Specifically, since the values ​​in the focus map are between 0 and 1, as shown in formula (15), a threshold of 0.5 is used to segment the focus map, which will form a binary map containing only 0 and 1.

[0088]

[0089] At this time, the binary graph obtained has some small areas similar to "holes", which is called the initial decision graph I in this embodiment. D In this embodiment, a small area removal technique is used to deal with these "hole" problems. Specifically, in the binary image, the area smaller than the threshold area needs to be inverted. Generally, when the area of ​​a region in an image is smaller than 0.005×H×W, the region is a small region relative to the entire image. Among them, H and W are the height and width of each source image, respectively. Therefore, the region with an area size smaller than 0.005×H×W is identified as a small region, and the final decision graph F is formed by removing these small regions.D The calculation formula of small area filtering is shown in formula (16), where area(BM region ) represents the area of ​​a small region.

[0090]

[0091] After the final decision graph is obtained through the above two steps, the multi-focus source images are fused according to the weight distribution graph provided by the final decision graph. The specific fusion rule is shown in formula (17). A , S B , and the final fused image is F.

[0092] F(x,y)=S A (x,y)·F D (x,y)+S B (x,y)·(1-F D (x,y)) (17)

[0093] Simulation experiment:

[0094] (1) Training dataset;

[0095] As is well known, there is a lack of publicly available standard datasets in the field of multi-focus image fusion (MFIF). To alleviate this limitation, this embodiment uses the Pascal Visual Object Class 2012 (VOC2012) dataset, which contains 2913 images and their corresponding segmentation category masks. The synthesis method of the dataset used for training in this embodiment is summarized as follows: First, the source images in the VOC2012 dataset are converted to a size of 256×256 pixels. In order to replicate different degrees of defocus, three different Gaussian filters are applied to generate three layers of blurred images. Subsequently, the masks are converted to training labels through a binarization process, and then their complementary operations are reversely derived. Finally, the training set of this embodiment is synthesized using these two complementary masks, one mask generates an image focused on the foreground, and the other mask generates an image focused on the background. This embodiment divides the dataset created using this method into a training set and a validation set with a ratio of 9:1. Among them, 7866 pairs of 256×256 images are used for training the proposal network, and 874 pairs of images are used for validation. The process of synthesizing the training dataset of this embodiment is as follows Figure 2 shown.

[0096] (2) Test dataset;

[0097] The effectiveness of LKSN-MFIF is evaluated using three publicly available datasets: Figure 3As shown. Through the analysis of subjective visual effects and objective evaluation indicators, this embodiment confirms the superiority of LKSN-MFIF by comparing with other excellent LKSN-MFIF methods. The test datasets include: Lytro, consisting of 20 pairs of real multi-focus images from light field data; MFFW, the only dataset with strong DSE, containing 13 pairs of multi-focus images of different resolutions; and MFI-WHU, generated from blurred all-focus images, containing 30 pairs of synthetic multi-focus images of different resolutions.

[0098] (3) Implementation details;

[0099] The network training was performed in PyTorch. To improve the robustness of the model, the input images were randomly cropped to a size of 224×224, and horizontal and vertical flipping was applied to enhance the data. The batch size used for each epoch was 8 to facilitate weight adjustment during each epoch and ensure a more stable gradient descent process. The learning rate was initially set to a specific value of 2×10 -4 , and is planned to decay to 2×10-4×0.88 at the 8th epoch i-1 , where the epoch number is denoted by i. The network is optimized using the Adam optimizer. The experiments and ablation studies are performed on a computing platform equipped with an Intel(R) Core(TM) i5-14490F CPU running at 2.80GHz and an NVIDIA GeForce RTX 4070SUPER graphics card.

[0100] (4) Evaluation indicators;

[0101] The quality of the fusion result is the main performance criterion of the MFIF method. Given that there is no reference image in the test dataset, existing MFIF methods mainly use objective evaluation indicators for performance evaluation. In this study, this embodiment uses eight objective evaluation indicators to comprehensively evaluate the quality of the fusion result from different perspectives. These indicators include the metric Q based on normalized mutual information, NMI 、Tsallis entropy-based metric Q TE , select the measure Q of the entropy of nonlinear related information NICE , Gradient-based metric Q G , Phase consistency based metric Q P , the metric Q based on edge information AB / F , the structural similarity-based measure Q Y , a measure based on human visual perception Q CB Among them, Q NMI and Q TE It measures the interdependence between the source image and the fusion result, Q TEMore robust to noise. NICE By calculating the nonlinear correlation coefficient between the fusion result and each source image, the level of source information preservation can be better evaluated. G The image gradient information is mainly measured by the Sobel operator. A higher image gradient often indicates higher information richness and higher fusion performance. P Used to measure the correlation of gradient phase information. AB / F Pay more attention to marginal information, Q Y Reflects brightness, contrast and structure loss, further calibrating image distortion. CB The contrast of each source image converted to the fusion result is calculated to reflect the preservation of the source image information. For all indicators, higher values ​​indicate higher performance.

[0102] (4) Fusion methods;

[0103] As contemporary advances in MFIF mainly tend to be based on deep learning methods, this example compares and analyzes LSKN-MFIF with 9 typical deep learning-based methods. These include CNN, ECNN, MFF-GAN, DRPL, SESF, U2Fusion, IFCNN, SwinFusion, and MFFT. Among them, CNN, ECNN, SESF, and DRPL are decision graph-based techniques, while IFCNN, U2Fusion, MFFGAN, and SwinFusion belong to the end-to-end paradigm. MFFT combines two fusion methods to generate both decision graphs and fused images directly. All methods are tailored for specific MFIF tasks.

[0104] Beneficial effects of this embodiment:

[0105] This embodiment proposes a new multi-focus image fusion method, called LSKN-MFIF. This method uses large kernel decomposition technology to capture long-term dependencies between different scales, while dynamically adjusting the receptive field of the feature extraction backbone through a spatial selection mechanism. This method effectively assigns weights to features processed by a wide range of deep kernels and then integrates them spatially. By using a gating mechanism and a combined differential convolution mechanism, the representation and generalization capabilities are enhanced.

[0106] This embodiment uses the Monte Carlo attention mechanism instead of the traditional global average pooling, so that the model can capture key information at different scales, enhance the network's ability to capture multi-scale information, and improve recognition performance. This method solves the limitations of traditional methods in utilizing cross-scale correlations.

[0107] This embodiment uses the L1 loss function to calculate the loss during the training process. This loss function is more robust to outliers and can prevent certain extreme samples from having too much impact on the loss function. This helps the model better capture the global information of different images while avoiding over-reliance on abnormal differences in local pixels, solving the problem of complex and diverse content in image data processing having a significant impact on model prediction.

[0108] This embodiment uses the "maximum selection" strategy to process the focus map, and uses the small area removal technology to deal with the "hole" problem in the binary map to form the final decision map. This strategy solves the problem that the focus map generated by the neural network contains misjudged labels and improves the quality of the fused image.

[0109] Embodiment 2

[0110] Based on the same inventive concept, this embodiment also provides a multi-focus image fusion system based on a large selective kernel network, including:

[0111] A first acquisition module is used for processing the multi-focus image using a large selective kernel module of a large selective kernel network to extract multi-scale global features;

[0112] A second acquisition module is used to extract local information through a combined differential convolution block of a large selective kernel network according to the multi-scale global features;

[0113] A decision diagram generation module, used to generate a focus decision diagram by combining the global features and local information;

[0114] An image fusion module is used to fuse the multi-focus images according to the focus decision map.

[0115] The multi-focus image fusion system based on a large selective kernel network provided in this embodiment has all the advantages of the multi-focus image fusion method based on a large selective kernel network provided in the first embodiment.

[0116] Embodiment 3

[0117] This embodiment further discloses a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in the first embodiment.

[0118] Embodiment 4

[0119] This embodiment further discloses a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in the first embodiment are implemented.

[0120] Embodiment 5

[0121] This embodiment also discloses a computer program product, including a computer program, which implements the steps of the method described in the first embodiment when executed by a processor.

[0122] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A multi-focus image fusion method based on a large selective kernel network, characterized in that: The following steps are involved: The large selective kernel module of the large selective kernel network is used to process multi-focus images and extract multi-scale global features; According to the multi-scale global features, local information is extracted through a combined differential convolution block of a large selective kernel network; Combining the global features and the local information to generate a focus decision graph; The multi-focus images are fused according to the focus decision graph.

2. The method according to claim 1, characterized in that The large selective kernel module is used to dynamically adjust the receptive field of the feature extraction backbone; wherein the large selective kernel module includes decomposing the series components of the large convolution to adaptively construct long-distance dependencies.

3. The method according to claim 1, characterized in that The combined differential convolution block is a module that integrates spatial attention and gated linear units; wherein the combined differential convolution block includes: center differential convolution, angular differential convolution, horizontal differential convolution and vertical differential convolution.

4. The method according to claim 1, characterized in that: Generating a focal decision diagram includes: The L1 loss function is used to calculate the loss during training. The global information of different images is captured through the loss to obtain the focus map. The focus map is processed using the maximum selection strategy to obtain the initial decision map; The initial decision graph is processed by using a small region removal method to obtain a final decision graph.

5. The method according to claim 4, characterized in that Fusing the multi-focus source images includes: According to the weight distribution diagram provided by the final decision diagram, weighted fusion is performed on the multi-focus source images to obtain a fused image; wherein the fusion of multiple source images is simplified into a fusion process of several two source images.

6. A multi-focus image fusion system based on a large selective kernel network, characterized in that: A first acquisition module is used for processing the multi-focus image using a large selective kernel module of a large selective kernel network to extract multi-scale global features; A second acquisition module is used to extract local information through a combined differential convolution block of a large selective kernel network according to the multi-scale global features; A decision diagram generation module, used to generate a focus decision diagram by combining the global features and local information; An image fusion module is used to fuse the multi-focus images according to the focus decision map.

7. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to claims 1 to 5 are implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to claims 1 to 5 are implemented.

Images