A brain image fusion method based on convolution sparse representation

Abstract

The application discloses a brain image fusion method based on convolution sparse representation, which takes an adaptive convolution sparse algorithm as a fusion framework, takes brain images as training samples to obtain a pre-training dictionary, and takes the brain images as test samples to participate in a fusion process; a non-subsampled contourlet transform (NSCT) is adopted to construct a new activity measure in combination with sub-band correlation; finally, a fused brain image is obtained through two-scale reconstruction; through the above method, the influence of local mismatch and texture structure loss on convolution sparse brain image fusion can be effectively improved, and the time consumption of the algorithm can be significantly reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of brain function imaging, in particular to a brain image fusion method based on convolution sparse representation. BACKGROUND

[0002] Animal behavior, including human, involves a series of complex neural circuit functions such as motivation, perception, decision-making, motor control, feedback, etc. How to better understand the relationship between neural circuit activity and behavior of animals is the key goal of systems neuroscience. Taking the predatory behavior of fish as an example, the existing recording of the whole brain neural activity of fish is completed on the living fish with detection devices fixed on the head. The research on brain function structure helps to improve the understanding of the whole brain neural activity of fish. Brain image fusion in brain function imaging processing technology can effectively improve the stereoscopic perception of brain function structure information by fusing the complementary information of multi-modal brain function imaging about the same scene into a single image, thereby providing a necessary research basis for understanding the relationship between the whole brain neural activity and behavior of fish. Due to the different imaging mechanisms of multi-modal brain images, there are obvious bright-dark contrast, texture difference and low similarity between brain images. The imaging characteristics of brain images make the existing fusion methods not ideal, so it is necessary to study a fusion method suitable for the imaging characteristics of multi-modal brain images.

[0003] In recent years, researchers have conducted extensive research on multi-modal brain image fusion. According to the image transformation strategy adopted, such research mainly focuses on four aspects: multi-scale transformation-based methods, sparse representation-based methods, deep learning-based methods, and hybrid solutions of the three. Unlike multi-scale transformation based on basis function assumption, and unlike deep learning which relies on artificial design rules for deep feature selection, sparse representation learns a complete dictionary through training of different sample sets and is used for image fusion, which can realize more stable and flexible representation of source images, and the sparse representation method is more in line with the physiological mechanism of the human visual system. However, the traditional sparse representation has the problem of multi-value due to the block processing method for image reconstruction, and cannot realize the global optimization of the image. In the convolution sparse representation (CSR), the dictionary array and sparse coefficient can realize the purpose of global optimization of the image through convolution, and the reconstructed image has the characteristics of single value, which is conducive to the texture information in the fusion result and improves the robustness of image adaptation.

[0004] The performance of CSR-based brain-image fusion is mainly determined by two key factors: an overcomplete dictionary and an activity metric. The acquisition of an overcomplete dictionary typically falls into two categories: a fixed dictionary pre-constructed using methods such as discrete cosine transform and wavelet transform, and a learned dictionary trained on high-quality natural images. Existing research has shown that adaptive dictionaries trained on source images outperform fixed dictionaries in improving brain-image fusion performance. Based on this, researchers have proposed various efficient dictionary learning algorithms, such as joint sparse models, synchronous orthogonal matching pursuit algorithms, global dictionary learning algorithms, and gradient-based classification adaptive dictionary learning algorithms. Besides the overcomplete dictionary factor, the activity metric is mainly used to identify salient features of the source images during the fusion process. Inappropriate activity measurement strategies can reduce the weight accuracy of sparse coefficient fusion. For example, existing CSR-based brain image fusion methods use L1-norm local window averaging to construct activity measures. Because the L1 norm uses the same weights, sparse salient features cannot be effectively highlighted, easily leading to the loss of effective information in the fusion result. In addition, local window averaging weakens the translation invariance of CSR image transformations, easily causing ringing artifacts (pseudo-Gibbs phenomenon) in mismatched regions. Furthermore, manually controlled window sizes are difficult to achieve ideal robustness to registration. Therefore, considering the characteristics of multimodal brain imaging, it is essential to design a convolutional sparse brain image fusion method with low dictionary redundancy and high robustness to registration. Summary of the Invention

[0005] The problem addressed by this invention is to provide a brain image fusion method based on convolutional sparse representation, which has low dictionary redundancy and high robustness to registration. It effectively improves the impact of local mismatch and loss of texture information on brain image fusion based on convolutional sparse representation, so that the fusion result retains more complete texture structure information and obtains brain functional imaging that meets the requirements of fish predation behavior analysis.

[0006] To address the above problems, this invention provides a brain image fusion method based on convolutional sparse representation, comprising the following steps:

[0007] S1, Input Source Image I r ,r∈{1,...,R}, where R represents the number of image samples and r represents the number of the r-th image sample;

[0008] S2, Source Image I r The pre-trained dictionary array d is obtained as training samples through the convolutional sparse representation algorithm. m ,m∈{1,...,M}, where M represents the number of elements in the dictionary array, m represents the m-th element in the dictionary array, and the source image I r Two-scale decomposition is performed to obtain the base layer B. r and several detail layers D r ;

[0009] S3, each detail layer D r are sparsely coded to obtain sparse coefficient maps m and a fused sparse coefficient map is obtained by the activity measure constructed by NSCT and sub-band relevance

[0010] S4, base layer B r a base layer fusion result is obtained by arithmetic average method each detail layer D r is sparsely reconstructed to obtain a detail layer fusion result

[0011] S5, the base layer fusion result and the detail layer fusion result are two-scale reconstructed to obtain a fused brain image I F .

[0012] Further, the convolution sparse representation algorithm in step S2 includes the following steps:

[0013] S201, defining an initial dictionary array d m , m∈{1,...,M} and an auxiliary variable y m with a constraint, and constructing a sparse iterative optimization cost function satisfying the alternating direction multiplier method as follows:

[0014]

[0015] wherein, denotes the inner product operation of a vector, * denotes the convolution operation, λ denotes a regularization parameter,

[0016] S202, solving the auxiliary variable y m and the sparse coefficient s m in the sparse iterative optimization cost function, and iteratively optimizing and updating the auxiliary variable y m and the sparse coefficient s m by the alternating direction multiplier method;

[0017] S203, defining an auxiliary variable g m with a constraint, and constructing a dictionary iterative optimization cost function satisfying the alternating direction multiplier method as follows:

[0018] wherein, the dictionary norm constraint set C PN ={s∈R N :(I-PP T ​​s = 0 || s || 2 = 1}, which guarantees that the optimized dictionary array result has proper constraint support in the spatial domain, P represents a zero padding operation, I represents an identity matrix, and the constraint set C PN The indicator function is defined as:

[0019]

[0020] S204, iteratively optimizing the initial dictionary array d in the dictionary iteration cost function m and the auxiliary variable g m is solved, and the iterative optimization and dictionary array update of d m and the auxiliary variable g m are performed by the alternating direction multiplier method;

[0021] S205, when the auxiliary variable y m is equal to the sparse coefficient s m and the initial dictionary array d m is equal to the auxiliary variable g m , the iterative optimization and update are ended, and the pre-trained dictionary array d m is obtained.

[0022] Further, the formula of sparse coding in step S3 is as follows:

[0023]

[0024] The formula for NSCT to perform multi-scale and multi-directional decomposition on each sparse coefficient map is as follows:

[0025]

[0026] wherein, represents the K-layer NSCT decomposition operation. represents the low-frequency subband part, represents the high-frequency subband part with 2 K decomposition directions.

[0027] The subband correlation measure is defined as:

[0028]

[0029] wherein, and represent adjacent subbands in the k and k' directions, respectively; and represent the mean values of and , respectively; i and j represent the size of the multi-scale transform subband coefficient map.

[0030] ​Further, the source image I in step S2 r The base layer B is obtained by two-scale decomposition through Tikhonov regularization method r and the detail layer D r The formula is as follows:

[0031]

[0032] Wherein, T x = [-1 1] and T y = [-1 1] T respectively represent the gradient operator along the x direction and the y direction, and the base layer B is obtained r After that, the corresponding detail layer is finally obtained as:

[0033] D r = I r -B r .

[0034] Further, the formula of the arithmetic average method in step S4 is:

[0035]

[0036] The formula for obtaining the detail layer fusion result is:

[0037]

[0038] Further, the two-scale reconstruction in step S5 is

[0039] Further, the sparse coefficient s m in step S202 is solved as follows: the iterative optimization process of the sparse coefficient s m in the sparse iterative optimization cost function in step S202 is represented as:

[0040]

[0041] And simplified as:

[0042]

[0043] Wherein, let z m = y m -u m , define the linear operator D m , then D m s m = d m *s m At this time, the above formula can be converted through the DFT convolution theorem as:

[0044] Wherein, and are the discrete Fourier transforms of D m , s m , I and z m , respectively. The equation is transformed into a collective form as follows:

[0045]

[0046] wherein, the definitions are as follows:

[0047]

[0048] At this time, the linear form of the sparse coefficient solution is obtained as follows:

[0049] wherein, the matrix is composed of M series of R x R diagonal matrices.

[0050] Further, the auxiliary variable y m in step S202 is solved by a shrinkage soft threshold method, and the formula is as follows: wherein, the shrinkage threshold operator satisfies ⊙ represents the matrix element multiplication operation.

[0051] Further, the initial dictionary array d m in step S203 is solved as follows. The initial dictionary array d m in step S203 is iteratively optimized in the dictionary iteration optimization cost function. The iterative optimization process is represented as follows:

[0052]

[0053] and is simplified as follows:

[0054]

[0055] At this time, the above equation can be transformed into the following equation by the DFT convolution theorem:

[0056]

[0057] wherein, the definitions are as follows: and are the discrete Fourier transforms of D , I r and z m , respectively; the above equation is transformed into a collective form as follows:

[0058]

[0059] wherein, the definitions are as follows:

[0060]

[0061] At this time, the linear form of the dictionary array solution can be obtained:

[0062]

[0063] Further, the auxiliary variable g in step 204 m Solve by the prox function, and the formula is as follows:

[0064] And need to meet:

[0065]

[0066] The beneficial effects brought by the present application are that adaptive convolution sparse representation is adopted to reduce the time consumption of the algorithm and the uncertainty of the fusion performance; NSCT is combined with subband correlation to construct a new activity measure to improve the sensitivity of the fusion result to mismatch and retain more complete effective information. Through the above measures, the influence of local mismatch and texture structure loss on convolution sparse brain image fusion can be effectively improved, and the time consumption of the algorithm can be significantly reduced BRIEF DESCRIPTION OF DRAWINGS

[0067] Figure 1 It is an adaptive convolution sparse brain image fusion system framework based on NSCT and subband correlation;

[0068] Figure 2 It is a flowchart of the method of the present application;

[0069] Figure 3 It is a schematic diagram of the influence of the activity measure representation on the fusion performance;

[0070] Figure 4 It is a schematic diagram of the influence of local mismatch on the fusion performance;

[0071] Figure 5 It is a schematic diagram of the performance comparison of different fusion methods for CT / MRI brain image pairs;

[0072] Figure 6 It is a schematic diagram of the performance comparison of different fusion methods for CT / MR-T1 brain image pairs;

[0073] Figure 7 It is a schematic diagram of the performance comparison of different fusion methods for CT / MR-T2 brain image pairs;

[0074] Figure 8 It is a schematic diagram of the overall objective statistical results of different fusion methods;

[0075] Figure 9 It is a schematic diagram of the objective performance comparison of different fusion methods under three groups of brain image pairs;

[0076] Figure 10 CPU time consumption diagram for fusing two brain images with spatial resolution of 256x256 by different methods. DETAILED DESCRIPTION

[0077] In order to make the above objectives, characteristics and advantages of the present application more apparent, more comprehensible, specific embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0078] It should be noted that the following embodiments assume that the strict registration between the multi-modal brain images has been completed, which is a typical case for such fusion schemes and is not a special requirement of the present application; the first auxiliary variable y m and the second auxiliary variable g m are only used to distinguish the two auxiliary variables and do not represent that the two variables have a certain precedence; the alternating direction multiplier method is simply referred to as the ADMM algorithm, and the discrete Fourier transform convolution theorem is simply referred to as the DFT convolution theorem.

[0079] The present embodiment provides a brain image fusion method based on convolution sparse representation, comprising the following steps:

[0080] inputting a source image I r , r e {1,..., R}; wherein R represents the number of image samples, and r represents the rth image sample number;

[0081] inputting the source image I r as a training sample to obtain a pre-trained dictionary array d m , m e {1,..., M} by a convolution sparse representation algorithm;

[0082] performing two-scale decomposition on the source image I r to obtain a base layer B r and a detail layer D r by a Tikhonov regularization method;

[0083] each of the detail layers D r is obtained by sparse coding of the pre-trained dictionary array d m to obtain a sparse coefficient map and a fused sparse coefficient map is obtained by an activity measure constructed by the NSCT and the sub-band relevance;

[0084] the base layer B r is obtained by an arithmetic average method to obtain a base layer fusion result the detail layer D r is obtained by sparse reconstruction of the sparse coefficient map to obtain a detail layer fusion result the base layer fusion result and the results of layer-by-layer fusion Two-scale reconstruction was performed to obtain fused brain images I F .

[0085] The convolutional sparse representation algorithm includes the following steps:

[0086] Define an initial dictionary array D m , m∈{1,...,M} and the first auxiliary variable with constraints, M represents the number of elements in the dictionary array, m represents the m-th element, and a sparse iterative optimization cost function satisfying the alternating direction multiplier method is constructed;

[0087] The first auxiliary variable in the sparse iterative optimization cost function is solved by the shrinking soft thresholding method, the sparse coefficients are solved by the discrete Fourier transform convolution theorem, and the ADMM algorithm is used to iteratively optimize the first auxiliary variable and the sparse coefficients and update the sparse coefficients.

[0088] Define a dictionary array D m The second auxiliary variable with constraints is used to construct a dictionary iterative optimization cost function that satisfies the ADMM algorithm;

[0089] Iterative optimization of the dictionary array D in the cost function of the dictionary m Solving for the second auxiliary variable prox function using the Discrete Fourier Transform Convolution Theorem, and then applying the ADMN algorithm to D... m Iterative optimization and dictionary array updates are performed using the second auxiliary variable;

[0090] When the first auxiliary variable is equal to the sparse coefficient, the pre-trained dictionary array d is obtained. m ,m∈{1,...,M}.

[0091] It should also be noted that, for the sake of simplification, the following description uses the initial dictionary array D. m With the pre-trained dictionary array d m All use d m The specific implementation methods of the above algorithms are as follows:

[0092] Convolutional sparse representation algorithm:

[0093] Convolutional sparse representation algorithms use a set of sparse coefficient graphs and the corresponding dictionary array d m The form of the convolution sum of m∈{1,...,M} for brain images I r For the approximation of r∈{1,...,R}, the formula is as follows:

[0094]

[0095] in, denotes the inner product of vectors, * denotes the convolution operation, λ denotes the regularization parameter, M denotes the number of elements in the dictionary array, m denotes the mth element in the dictionary array, R denotes the number of image samples, and r denotes the rth image sample.

[0096] The sparse coding algorithm with a sparse penalty term ||s||1, i.e., the convolution basis pursuit denoising problem, is defined as:

[0097]

[0098] By introducing a first auxiliary variable y m The ADMM algorithm is used to iteratively solve the above optimization problem, and the cost function of sparse coding can be converted into:

[0099]

[0100] The corresponding variable iterative optimization process can be represented as:

[0101]

[0102]

[0103]

[0104] In the above iterative loop process, the variable y m The shrinkage soft threshold method can be used to solve:

[0105]

[0106] where the shrinkage threshold operator satisfies ⊙ denotes the matrix element multiplication operation.

[0107] The sparse coefficient s m The DFT convolution theorem can be used to solve, and formula (2) can be simplified as:

[0108]

[0109] where z m is defined as y m -u m . The linear operator D m is defined as D m s m =d m *s m , and formula (5) can be converted into:

[0110]

[0111] where and D m , s m , I and z m are the discrete Fourier transform. Transforming (6) into a collective form is:

[0112]

[0113] where the definition is:

[0114]

[0115] At this time, the linear form of the sparse coefficient solution can be obtained by (7):

[0116]

[0117] where the matrix is composed of M serial R x R diagonal matrices, M represents the number of elements in the dictionary array, and R represents the number of samples in the image . I represents the unit matrix.

[0118] Since the linear form in (8) meets the requirements of the Sherman-Morrison formula for solving linear systems, the Sherman-Morrison formula is used to quickly obtain the iterative sparse coefficient. Finally, the inverse discrete Fourier transform is used to obtain the iterative sparse coefficient. The iteration loop of (2)-(4) realizes the update of the sparse coefficient, and the iteration loop ends until the first auxiliary variable y m is equal to the sparse coefficient s m .

[0119] After obtaining the sparse coefficient through sparse coding, the corresponding dictionary update algorithm is the convolution form of the optimal direction method (MOD), which is defined as:

[0120]

[0121] where the dictionary norm constraint set C PN ={s∈R N :(I-PP T )s=0,||s||2=1} can guarantee that the optimized dictionary array result has appropriate constraint support in the spatial domain, and P represents the zero padding operation and I represents the unit matrix.

[0122] By introducing the second auxiliary variable g m with constraints, the ADMM algorithm is used to iteratively solve the above optimization problem, and at this time the dictionary update cost function can be transformed into:

[0123]

[0124] where the constraint set CPN Objective function is defined as:

[0125]

[0126] At this time, the variable iterative optimization process obtained by using the ADMM algorithm principle can be expressed as:

[0127]

[0128]

[0129]

[0130] In the above iterative loop process, the solution of variable g m satisfies the geometric relationship:

[0131]

[0132] Then the prox function can be directly used to solve the problem, satisfying:

[0133]

[0134] The solution of the dictionary array d m can be solved by the DFT convolution theorem, at this time (9) can be simplified as:

[0135]

[0136] Through the DFT convolution theorem, (12) can be converted to:

[0137]

[0138] Where, satisfy and are the discrete Fourier transforms of I r and z m , respectively. Converting (13) into a set form is:

[0139]

[0140] Where, define:

[0141]

[0142] At this time, the linear form of the dictionary array solution can be obtained through (14):

[0143]

[0144] Since the linear form in equation (15) meets the requirements of solving linear systems using the Sherman-Morrison formula, the Sherman-Morrison formula is used to quickly obtain the iterative dictionary array. Using the above DFT convolution theorem and repeating the Sherman-Morrison formula loop, the iterative dictionary array in equation (8) can be obtained.

[0145] Through the cyclic iteration of the above sparse coding and dictionary update, until the first auxiliary variable y... m Equal to the sparsity coefficient s m The iteration loop ends, and the pre-trained dictionary under the convolutional sparse representation is obtained.

[0146] Image two-scale decomposition:

[0147] Each source image I r The base layer B is obtained by decomposing r∈{1,...,R} using the following cost function. r and detail layer D r :

[0148]

[0149] Among them, T x =[-11] and T y =[-11] T Let represent the gradient operators along the x and y directions, respectively. This is a Tikhonov regularization problem, which can be efficiently solved using Fourier transform. The underlying layer B is obtained. r Afterwards, the corresponding detail layer can be obtained as follows:

[0150] D r =I r -B r .

[0151] Detail layer D r Fusion:

[0152] For each detail layer D r The corresponding sparse coefficient diagram This is obtained through equation (1) in sparse coding. Since multi-scale transformation typically uses low-resolution components to analyze approximate image features and high-resolution components to analyze detailed image features, the decomposition layers with different resolutions obtained through multi-scale and multi-directional decomposition contain different image features. Inspired by this, multi-scale transformation can be understood as a feature classification process, and the fusion of similar features within the same decomposition layer helps improve the accuracy of the fusion results. Therefore, NSCT is first used to perform multi-scale and multi-directional decomposition on each sparse coefficient map:

[0153]

[0154] wherein, denotes the K-layer NSCT decomposition operation. denotes the low-frequency subband part, denotes the high-frequency subband part with 2 K decomposition directions.

[0155] Low-frequency subband part of the source image corresponding sparse coefficient map Fusion is performed using an average strategy:

[0156]

[0157] High-frequency subband part The activity measure is characterized by a subband correlation measure (i.e. the correlation of adjacent subbands in the same scale of the sparse coefficient map), wherein the subband correlation measure is defined as:

[0158]

[0159] wherein, and denote the adjacent subbands in the k and k' directions, respectively. and denote the mean values of and , respectively. i and j denote the size of the multi-scale transform subband coefficient map. At this time, the fusion rule for the NSCT high-frequency subband of the corresponding sparse coefficient map is:

[0160]

[0161] After obtaining the fused NSCT low-frequency subband and high-frequency subband, the fused sparse coefficient map is obtained by NSCT inverse transform.

[0162]

[0163] Base layer B r fusion:

[0164] Due to the generally large differences in brightness and texture at the same position among multi-modal brain images, the traditional maximum fusion strategy may lead to visual inconsistency due to slight changes in local pixels. Therefore, the average strategy is used to obtain the fusion result of the base layer:

[0165]

[0166] Two-scale reconstruction:

[0167] obtained fused base layer and fused detail layer After that, the final fusion image can be obtained by two-scale reconstruction as follows:

[0168]

[0169] To verify the effectiveness of the activity measure in the proposed method on fusion performance, the experiment is designed to take human brain images as an example, and the experimental conditions are as follows: (1) the multi-modal brain images come from the Whole Brain Atlas Medical Image Database (WBAMI) created by Harvard Medical School, and the spatial resolution of the images is 256x256; (2) under the fusion framework of the present application, based on the idea of single variable control, the comparison activity measures include: sub-band correlation measure, measure combining NSCT and L1 norm, and the measure combining NSCT and sub-band correlation proposed in the present application; (3) the experiments are all carried out in the experimental environment of computer configuration of 3.3GHz CPU and 16.0GHz RAM, and MATLAB version of R2017a and Win7 64-bit operating system. The experiment of the influence of activity measure on fusion performance is shown in FIG. 6. As shown in FIG. 6, it can be seen that, compared with the fusion results of the activity measure proposed in the present application and the sub-band correlation measure, the fusion results of the activity measure proposed in the present application can retain more complete structural information, so NSCT is helpful to promote the retention of structural information. Compared with the fusion results of the activity measure proposed in the present application and the measure combining NSCT and L1 norm, the fusion results of the activity measure proposed in the present application can retain more complete texture information, so the sub-band correlation is more helpful to the retention of texture information than L1 norm. In summary, the activity measure in the fusion method proposed in the present application is helpful to retain more complete texture and structural information. Figure 3 Figure 3

[0170] To verify the robustness of the fusion results of the proposed method to mismatch and ensure the fairness of the experiment, the same multi-focus image pairs as in the robustness verification of the comparison method are used, and the experimental conditions are as follows: (1) the comparison methods include: the method based on sub-band correlation measure using the same fusion framework as the present application, the representative convolution sparse-based brain image fusion method CSMCA and CSR; (2) the spatial resolution of the multi-focus image pairs is 256x256; (3) the experiments are all carried out in the experimental environment of computer configuration of 3.3GHz CPU and 16.0GHz RAM, and MATLAB version of R2017a and Win7 64-bit operating system. The experiment of the influence of local mismatch on brain image fusion performance is shown in FIG. 7. As shown in FIG. 7, it can be seen that, compared with the fusion results of the activity measure proposed in the present application and the sub-band correlation measure, the fusion results of the activity measure proposed in the present application can retain more complete structural information, so NSCT is helpful to promote the retention of structural information. Compared with the fusion results of the activity measure proposed in the present application and the measure combining NSCT and L1 norm, the fusion results of the activity measure proposed in the present application can retain more complete texture information, so the sub-band correlation is more helpful to the retention of texture information than L1 norm. In summary, the activity measure in the fusion method proposed in the present application is helpful to retain more complete texture and structural information. Figure 4 Figure 4 ​​​It can be seen that the fusion results of CSMCA and CSR methods have obvious ringing artifacts at the edge of mismatched regions, i.e. pseudo-Gibbs phenomenon. This is because the local window average method adopted by CSMCA and CSR reduces the shift invariance of image transformation, which reduces the sensitivity of the fusion results of CSMCA and CSR to mismatch, and the artificially controlled window size is difficult to obtain the ideal robustness effect of registration. In the method based on sub-band correlation measure and the method proposed in the present application, the sub-band correlation uses global sub-band to ensure the shift invariance, which can effectively eliminate the pseudo-Gibbs phenomenon.

[0171] In order to verify the rationality of the fusion method proposed in the present application, the experiment is designed to take human brain image as an example, and the experimental conditions include: (1) 30 groups of multi-modal brain images are randomly selected from the "Acute stroke" data set, "Hypertensive encephalopathy" data set and "Multiple embolic infarctions" data set of the whole brain atlas brain image database created by Harvard Medical School, and the spatial resolution of the images is 256x256; (2) the comparison methods include: the method based on sub-band correlation measure with the same fusion framework as the present application, the representative convolution sparse-based brain image fusion method CSMCA and CSR; (3) algorithm parameter setting: the spatial size of each dictionary array is 8x8, the dictionary array is set to 32, and the regularization parameter λ is set to 0.01, and the above parameter settings are the same as those of CSR and CSMCA; (5) objective evaluation indexes include: entropy EN, joint entropy Q U , mutual information MI, structural similarity SSIM, and the larger the value of the above objective indexes, the better the fusion performance; (6) the experiment is carried out in the experimental environment of computer configuration of 3.3GHz CPU and 16.0GHz RAM, and MATLAB version of R2017a and Win7 64-bit operating system. In addition, the fusion performance analysis is carried out by combining subjective evaluation and objective evaluation. Among them, the fusion results of subjective evaluation are shown in Figs. 1-3, the performance indexes of objective evaluation are shown in Figs. 4-6 (the bold font indicates the optimal index, and the underlined font indicates the suboptimal index), and Figs. 7-9. Figure 5 Figure 6 Figure 7 Figure 9 Figure 8 Figure 10 Figure 5 Figure 6 Figure 7 ​​​​​​​​It can be seen that the fusion results of CSMCA method and CSR method are more complete in terms of texture definition and brightness information, and the continuity of edge texture is destroyed, but there is local detail loss. The method based on sub-band relevance measure is not advantageous in terms of detail texture definition and structural integrity, but it preserves more complete edge structural texture. In contrast, the fusion method of the present application preserves more complete structural and texture information. As shown in the accompanying Figure 9 、 Figure 8 and the accompanying Figure 10 It can be seen that the scores of the four objective indicators in the three brain image dataset experiments are relatively outstanding under the condition of low time consumption of the algorithm of the method of the present application. Among them, the advantage of the MI indicator of the method of the present application is most obvious, indicating that the method of the present application can not only preserve more complete texture structural information, but also maintain high algorithm execution efficiency.

[0172] Although the present disclosure is disclosed as above, the protection scope of the present disclosure is not limited to this. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present disclosure, and these changes and modifications will fall within the protection scope of the present application.

Claims

1. A method for brain image fusion based on convolutional sparse representation, characterized in that, The method comprises the following steps: S1, input source image I r r e {1, K, R}, R represents the number of image samples, and r represents the rth image sample number; S2, decompose the source image I r Pre-training dictionary array d is obtained by convolution sparse representation algorithm as training sample m , m∈{1, K, M}, M represents the number of elements in the dictionary array, m represents the mth element in the dictionary array, and the source image I r is decomposed into base layer B and several detail layers D by two-scale decomposition r r ;​ S3, each of the detail layers D r are sparse coded to obtain sparse coefficient maps m are sparse coded to obtain sparse coefficient maps and a fused sparse coefficient map is obtained by the activity measure constructed by the NSCT and the subband dependency S4, the base layer B r The base layer fusion result is obtained by the arithmetic mean method Each of the detail layers D r The sparse reconstruction is performed on the fused sparse coefficient map to obtain the detail layer fusion result The sparse reconstruction is performed on the fused sparse coefficient map to obtain the detail layer fusion result S5, fusing the base layer result and the detail layer result to obtain the fused brain image I F ; In the step S3, the NSCT decomposes each sparse coefficient map The formula for multi-scale multi-directional decomposition is: wherein, represents a K-layer NSCT decomposition operation, represents a low frequency subband portion, represents a high frequency subband portion with 2 K decomposition directions, the low frequency subband portion The formula for fusion using the average strategy is: High frequency subband portion The activity measure is characterized by a subband correlation measure (i.e. the correlation of neighboring directional subbands at the same scale of the sparse coefficient map), which is defined as: where, and denote the neighboring subbands in the kth and k'th direction, respectively, and denote the mean of and , i and j denote the size of the multi-scale transform subband coefficient map, in this case, the NSCT high frequency subband , the fusion rule is: After obtaining the fused NSCT low frequency subband and high frequency subband, the fused sparse coefficient map is obtained by NSCT inverse transform 2. The brain image fusion method based on convolution sparse representation according to claim 1, wherein, The convolution sparse representation algorithm in the step S2 comprises the following steps: S201、define an initial dictionary array d m , m ∈ {1,..., M} and the auxiliary variable y with constraints m and construct a sparse iterative optimization cost function that satisfies the alternating direction multiplier method as follows: wherein, denotes an inner product operation of vectors, * denotes a convolution operation, λ denotes a regularization parameter, I is the source image I r , s m are sparse coefficients; S202、to the auxiliary variable y in the sparse iterative optimization cost function m and sparse coefficients s m solving, and through the alternating direction multiplier method to the auxiliary variable y m and the sparse coefficients s m iterative optimization and sparse coefficient update; S203、Define the auxiliary variable g with constraints m The dictionary iteration optimization cost function satisfying the alternating direction multiplier method is constructed as follows: Among them, the dictionary norm constraint set C PN ={s∈R N :(I-PP T The formula s = 0, ||s||² = 1} guarantees that the optimized dictionary array result has appropriate constraint support in the spatial domain. P represents zero-padding, I represents the identity matrix, and s represents the dictionary array d. m The norm of the equation and the auxiliary variables are solved, and the iterative optimization constraint set C of the equation and the auxiliary variables is obtained by the alternating direction multiplier method. PN index function Defined as: S204, iterating over the dictionary to optimize the initial dictionary array d in the cost function m and the auxiliary variable g m solving, and iterating over d m and the auxiliary variable g m iterative optimization and dictionary array update; S205、when the auxiliary variable y m is equal to the sparse coefficient s m and the initial dictionary array d m is equal to the auxiliary variable g m , end the iterative optimization update, and obtain the pre-training dictionary array d m .

3. The brain image fusion method based on convolution sparse representation according to claim 1, wherein, The formula of the sparse coding in the step S3 is as follows: 4.The brain image fusion method based on convolution sparse representation according to claim 1, characterized in that, The source image I in the step S2 r The base layer B is obtained by two-scale decomposition through Tikhonov regularization method r And the detail layer D r The formula is as follows: where T x = [-11] and T y = [-11] T denote the gradient operators along the x and y directions, respectively, to obtain the base layer B r After that, the corresponding detail layer is finally obtained as: D r = I r - B r .

5. The brain image fusion method based on convolution sparse representation according to claim 1, wherein, The formula of the arithmetic average method in the step S4 is as follows: The formula of the detail layer fusion result is as follows:

6. The brain image fusion method based on convolution sparse representation according to claim 1, wherein, The bi-dimensional reconstruction in step S5 is such that 7.The brain image fusion method based on convolution sparse representation according to claim 2, characterized in that, The sparsity coefficient S in step S202 m The solution is obtained using the Discrete Fourier Transform Convolution Theorem. The specific process is as follows: The sparse coefficients s in the sparse iterative optimization cost function in step S202 are... m The iterative optimization process is represented as: And is simplified as: where z = y - u m = y m - u m , define a linear operator D m such that D m s m = d m s m , then the above equation can be transformed by DFT convolution theorem as follows: where and are the discrete Fourier transforms of D m , s m , I and z m , respectively, transforming the equation into a collective form as: Wherein, define: At this time, the linear form of the sparse coefficient solution is obtained: where the matrix is composed of M serially connected R x R diagonal matrices.

8. The brain image fusion method based on convolution sparse representation according to claim 2, wherein, The auxiliary variable y in the step S202 m By solving the shrinkage soft threshold method, the formula is as follows: where the shrinkage threshold operator satisfies e denotes a matrix element multiplication operation.

9. The brain image fusion method based on convolution sparse representation according to claim 2, wherein, The initial dictionary array d in the step S203 m The initial dictionary array d in the step S203 is solved by the discrete Fourier transform convolution theorem, and the specific process is as follows: m The iterative optimization process is represented as: And is simplified as: At this time, the above formula can be converted into: where and are respectively I r and z m the discrete Fourier transform; converting the above equation into a collective form is: Wherein, define: At this time, the linear form of the dictionary array solution is obtained: 10.The brain image fusion method based on convolution sparse representation according to claim 2, characterized in that, The auxiliary variable g in the step S204 m The solution by the prox function is given by the formula: and needs to satisfy:

Images