A multimodal brain image fusion method for analyzing fish predation behavior

Abstract

The present invention provides a multimodal brain image fusion method for analyzing fish predation behavior, relating to the field of image fusion technology. The method comprises steps S1: characterizing structural texture features by learning a sparse matrix composed of associated sparse coefficients based on a separable dictionary; S2: obtaining a pre-trained dictionary through iterative cycles using the orthogonal matching pursuit method and the conjugate gradient method on a Riemannian manifold; and S3: obtaining a fused brain image through sparse reconstruction using texture contrast and sparse saliency features and a constructed activity measure to preserve texture information. This method overcomes the drawbacks of the atomic dictionary update method that can easily lead to texture information loss and low overall dictionary update efficiency, as well as the drawback of texture information loss caused by the single activity measure representation.

Description

Technical Field

[0001] The present invention relates to the field of image fusion technology, and in particular to a multimodal brain image fusion method for analyzing fish predation behavior. Background Art

[0002] Animal behavior, including that of humans, involves a complex array of neural circuit functions, including motivation, perception, decision-making, motor control, and feedback. Understanding the connection between animal neural circuit activity and behavior is a key goal of systems neuroscience. For example, while existing recordings of whole-brain neural activity in fish are performed on live, head-fixed fish, research on brain function and structure can improve our understanding of whole-brain neural activity. Brain image fusion, a key area of ​​functional imaging processing, fuses complementary information from multimodal brain images of the same scene into a single image, effectively enhancing the three-dimensional perception of brain function and structure. This provides an essential research foundation for understanding the connection between whole-brain neural activity and behavior in fish. Due to the different imaging mechanisms of multimodal brain imaging, brain images often exhibit significant light-dark contrast, texture differences, and low similarity. These inherent characteristics of brain images make existing fusion methods less effective.

[0003] Currently, academic research on brain image fusion primarily focuses on three fusion levels: pixel-level, feature-level, and decision-level, and the research results are relatively comprehensive. Pixel-level fusion, by directly acting on pixels in the raw data layer, can more completely preserve the effective information in the source image. Because brain image fusion prioritizes the accuracy of the fusion algorithm over real-time performance, pixel-level fusion has been more widely used in brain image fusion. Common pixel-level fusion algorithms include spatial domain, transform domain, sparse representation, deep learning, and their hybrids. Compared to other algorithms, sparse representation can achieve the natural sparsity of the signal, facilitating the extraction of texture features. This principle is consistent with the physiological characteristics of the human visual system (HVS). Although traditional analytic dictionary learning methods based on sparse representation can achieve a simple and fast sparse representation of the signal, they generally have poor representation capabilities for complex textures. The dictionary update format reduces overall update efficiency. Dictionary atoms are updated in the form of column vectors, which can be understood as extracting texture information in only one dimension of the image. This reduces the structural and texture correlations between adjacent regions and can easily lead to information loss. In addition, the single activity measure representation of sparse features is also an important reason for the loss of texture information. Summary of the Invention

[0004] The problem solved by the present invention is how to overcome the shortcomings of the update form of dictionary atoms that easily causes loss of texture information and low overall dictionary update efficiency, as well as the defect of loss of texture information due to the single representation of activity measure.

[0005] To solve the above problems, the present invention provides a multimodal brain image fusion method for analyzing fish predation behavior, comprising the steps of:

[0006] S1: Based on the separation dictionary learning, a sparse matrix composed of associated sparse coefficients is used to represent the structural texture features;

[0007] S2: The pre-trained dictionary is obtained through cyclic iterations of the orthogonal matching pursuit method and the conjugate gradient method on the Riemannian manifold;

[0008] S3: Texture contrast and sparse saliency features are used together with the constructed activity measure to preserve texture information and obtain fused brain images through sparse reconstruction.

[0009] In this method, the atomic dictionary update method can easily lead to loss of texture information and low overall dictionary update efficiency. Separate dictionary learning is used to represent richer structural texture features through a sparse matrix composed of associated sparse coefficients. This efficiency is improved by synchronously updating the sub-dictionary matrix. To address the problem of texture information loss caused by the single activity measure representation, texture contrast and sparse saliency features are used, and a new activity measure is constructed to preserve more complete texture information. Finally, a fused brain image is obtained through sparse reconstruction.

[0010] Furthermore, the step S1 includes:

[0011] S11: Establish the initial dictionary atoms and the current residual, use the initial dictionary atoms and the current residual to determine the index of the most relevant dictionary atoms, and obtain the index set after multiple iterations. Use the least squares method to update the sparse coefficients, and calculate the partial derivative of the sparse coefficients to obtain the reconstructed samples under the current dictionary and the updated residual;

[0012] S12: Re-determine the index of the most relevant dictionary atom by combining the updated residual and the dictionary atom to complete the iterative cycle and obtain the updated sparse coefficient under the orthogonal matching pursuit method.

[0013] Furthermore, the step S2 includes:

[0014] S21: Project the group variables composed of the separation dictionary onto the Riemannian manifold, and move the step size through the geodesic linear search to obtain the Riemannian gradient of the current point in the tangent space and the corrected search direction;

[0015] S22: Determine the step size by linear search using the Riemannian gradient and the modified search direction, and obtain the updated dictionary array under the conjugate gradient method on the Riemannian manifold;

[0016] S23: Combine the orthogonal matching pursuit method and the conjugate gradient method on Riemannian manifold for cyclic iteration, and obtain a pre-trained dictionary after sample training.

[0017] Furthermore, the objective function of the dictionary learning is:

[0018]

[0019] in, represents the sparse coefficient; N represents the number of training samples Y; D A ∈R m×n (n>m) and D B ∈R m×n (n>m) represents a sub-dictionary;

[0020] The dictionary learning process includes two parts: sparse coding and separate dictionary update.

[0021] Furthermore, step S3 includes:

[0022] S31: A plurality of registered multimodal brain images are subjected to a sliding window technique to obtain corresponding overlapping blocks, and a pre-trained dictionary is used to perform a sparse representation on the overlapping blocks to obtain a corresponding sparse coefficient map.

[0023] Furthermore, the step S3 further includes:

[0024] S32: Extract spatially significant features based on overlapping blocks using texture contrast measurement;

[0025] S33: Based on the sparse coefficient graph, sparse saliency features and measures are used to extract transformation salient features;

[0026] S34: Combining spatial saliency and transformation saliency to construct a fusion activity measure, and using the activity measure to guide the fusion of sparse coefficients corresponding to multiple multimodal brain images;

[0027] S35: Obtain fused brain images through sparse reconstruction using the pre-trained dictionary.

[0028] The present invention adopts the above technical solution to achieve the following beneficial effects:

[0029] This method uses a separate dictionary learning approach to represent richer structural texture features through a sparse matrix composed of associated sparse coefficients, and improves dictionary update efficiency through synchronous updates in the form of sub-dictionary matrices. It uses texture contrast and sparse saliency features and constructs a new activity measure to retain more complete texture information. Through sparse reconstruction, a fused brain image is obtained, effectively mitigating the impact of texture information loss on dictionary learning brain image fusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 The multimodal brain image fusion method process for analyzing fish predation behavior provided by the embodiment of the present invention Figure 1 ;

[0031] Figure 2 The multimodal brain image fusion method process for analyzing fish predation behavior provided by the embodiment of the present invention Figure 2 ;

[0032] Figure 3 A brain image fusion framework for dictionary learning that combines spatial saliency and transformation saliency in a multimodal brain image fusion method for analyzing fish predation behavior provided in an embodiment of the present invention;

[0033] Figure 4 The multimodal brain image fusion method process for analyzing fish predation behavior provided by the embodiment of the present invention Figure 3 ;

[0034] Figure 5 The effect of variable n on brightness contrast in the multimodal brain image fusion method for analyzing fish predation behavior provided by an embodiment of the present invention;

[0035] Figure 6 This invention provides an embodiment of the present invention to analyze the effect of the variable η on the directional contrast in the multimodal brain image fusion method for analyzing fish predation behavior. DETAILED DESCRIPTION

[0036] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0037] The following are specific embodiments of the present invention and the accompanying drawings to further describe the technical solutions of the present invention, but the present invention is not limited to these embodiments.

[0038] Example

[0039] This embodiment provides a multimodal brain image fusion method for analyzing fish predation behavior. Figure 1 and Figure 2 As shown, the method includes the steps of:

[0040] S1: Based on the separation dictionary learning, a sparse matrix composed of associated sparse coefficients is used to represent the structural texture features;

[0041] S2: The pre-trained dictionary is obtained through cyclic iterations of the orthogonal matching pursuit method and the conjugate gradient method on the Riemannian manifold;

[0042] S3: Texture contrast and sparse saliency features are used together with the constructed activity measure to preserve texture information and obtain fused brain images through sparse reconstruction.

[0043] Specifically, the orthogonal matching pursuit method (OMP) and the conjugate gradient method on Riemannian manifold are used to obtain the pre-trained dictionary through cyclic iteration. Define the initial dictionary atom Di,j and the current residual R∈R m×m , use the two to determine the index (i, j) of the most relevant dictionary atom, and obtain the index set Λ after k iterations k ={(i1,j1),(i2,j2),…,(i k ,j k )}, on this basis, the sparse coefficient S is updated using the least squares method, and the Taking the partial derivative we get Thus, we can obtain the reconstructed samples under the current dictionary. and update the residual Combine the residual R with the dictionary atom D i,j Re-determine the index of the most relevant dictionary atom to implement an iterative loop until The iteration ends when , and the updated sparse coefficient S under the OMP method is obtained. At this time, the group variable V composed of the separation dictionary is A ,D B ) is projected onto the Riemann manifold M, and the optimal moving step η is linearly searched through the geodesic Γ to obtain the current point V (i+1) =Γ(V (i) ,H (i) ,η (i) ) and the point in the tangent space T V Riemann gradient G on M and modified search direction H. Use G and H to determine the optimal step size η through linear search (i) , and get the updated dictionary array V under the conjugate gradient method on the Riemann manifold. After a large number of sample training, the pre-trained dictionary {D A ,D B Then, texture contrast (TC) and sparse saliency feature (SSSF) are used to construct a fusion activity measure to guide the sparse coefficient fusion.

[0044] See Figure 3 Specifically, assume that there are K registered multimodal brain images I k ,k∈{1,2,K,K}, the kth brain image is obtained by sliding window technology to obtain the corresponding overlapping block And use the pre-trained dictionary {D A ,D B} for overlapping blocks Perform sparse representation to obtain the corresponding sparse coefficient graph For overlapping blocks TC measure is used to extract spatially significant features. The SSSF measure is used to extract transformation salient features. A new fusion activity measure is constructed by combining spatial saliency and transformation saliency. The activity measure is used to guide the fusion of sparse coefficients corresponding to K multimodal brain images. Finally, the pre-trained dictionary {D A ,D B The final fused brain image I is obtained through sparse reconstruction F .

[0045] Specifically, a separate dictionary learning approach is used to represent richer structural texture features through a sparse matrix composed of associated sparse coefficients. The efficiency of dictionary updates is improved through synchronous updates of sub-dictionary matrices. A new activity measure is constructed using TC and SSSF to preserve more complete texture information. These measures can effectively mitigate the impact of texture information loss on dictionary learning brain image fusion.

[0046] See Figure 2 , wherein step S1 includes:

[0047] S11: Establish the initial dictionary atoms and the current residual, use the initial dictionary atoms and the current residual to determine the index of the most relevant dictionary atoms, and obtain the index set after multiple iterations. Use the least squares method to update the sparse coefficients, and calculate the partial derivative of the sparse coefficients to obtain the reconstructed samples under the current dictionary and the updated residual;

[0048] S12: Re-determine the index of the most relevant dictionary atom by combining the updated residual and the dictionary atom to complete the iterative cycle and obtain the updated sparse coefficient under the orthogonal matching pursuit method.

[0049] Wherein, step S2 includes:

[0050] S21: Project the group variables composed of the separation dictionary onto the Riemann manifold, and move the step size through the geodesic linear search to obtain the Riemann gradient of the current point in the tangent space and the corrected search direction;

[0051] S22: Using the Riemannian gradient and the modified search direction to determine the step size through linear search, the update dictionary array under the conjugate gradient method on the Riemannian manifold is obtained;

[0052] S23: Combine the orthogonal matching pursuit method and the conjugate gradient method on Riemannian manifold for cyclic iteration, and obtain a pre-trained dictionary after sample training.

[0053] Among them, the objective function of dictionary learning is:

[0054]

[0055] in, represents the sparse coefficient; N represents the number of training samples Y; D A ∈R m×n (n>m) and D B ∈R m×n (n>m) represents a sub-dictionary;

[0056] The dictionary learning process includes two parts: sparse coding and separate dictionary update.

[0057] Specifically, D A ∈R m×n (n>m) and D B ∈R m×n (n>m) is a sub-dictionary, and both are obtained by Kronecker decomposition of the overcomplete dictionary D.

[0058] Among them, the OMP method is used for sparse coding process, and the manifold-based conjugate gradient method is used to update the separation dictionary.

[0059] The sparse coding process means obtaining the sparsest representation of the signal under the current dictionary, so its objective function can be transformed into:

[0060]

[0061] Specifically, the most relevant dictionary atom is selected: in the OMP method, the dictionary atom D is used i,j and the current residual R∈R m×m The maximum absolute value of the inner product determines the index of the most relevant dictionary atom as (i, j)←argmax| <R,D i,j >|.

[0062] Specifically, update the support set: the kth iteration support set index sequence is (i k ,j k ), so the support set of the corresponding separation dictionary is and a i It's D A The i-th column of the index set Λ k ={(i1,j1),(i2,j2),…,(i k ,j k )}.

[0063] Specifically, update the sparse matrix S: use the least squares method to update the sparse matrix as follows:

[0064]

[0065] in, is the index set Λ in the sparse matrix S k A vector consisting of elements on , and satisfying:

[0066]

[0067] For the sparse matrix S Taking the partial derivative and setting the result equal to 0, we get:

[0068] Specifically, update the residual: use the sparse matrix S in the updated sparse matrix S to obtain the reconstructed sample under the current dictionary Then the updated residual after k iterations is

[0069] Update the residual and return to selecting the most relevant dictionary atom to iterate until The iteration ends when , and the optimal sparse matrix S is obtained.

[0070] After sparse coding obtains the sparse matrix S, the objective function of the dictionary update process is expressed as:

[0071]

[0072] Combined with the constraints of the dictionary, that is, the l2 norm of atoms in the dictionary is 1 and there is an independent characteristic between atoms, the log function is used to fit the full rank and column independence of the dictionary. At this time, the objective function of the dictionary update can be rewritten as:

[0073]

[0074] Among them, λ and β are fitting parameters and satisfy:

[0075]

[0076]

[0077] The separation dictionary D A ,D B The group variable V=(D A ,D B ) is projected onto the Riemann manifold, the objective function of dictionary update can be further rewritten as:

[0078] Where M represents a Riemannian manifold and V is a point on M. The conjugate gradient method is used as the optimization algorithm to derive the objective function f(V). First, the point V of the previous iteration of the geodesic Γ is obtained. (i) Starting point along H (i) Move, the optimal moving step length of linear search is: and And satisfy:

[0079]

[0080] So we can get the current point V (i+1) =Γ(V (i) ,H (i) ,η (i) ), and the current point V(i+1) In the tangent space T V M, that is, the Riemannian gradient on the tangent space of the Riemann manifold M at point V:

[0081]

[0082] At this time, the previous iteration point H is obtained (i) Corrected search direction at:

[0083] H (i+1) =-G (i+1) +δτ(H (i) ,V (i) ,H (i) ,η (i) )

[0084] Among them, τ(H (i) ,V (i) ,H (i) ,η (i) ) satisfies τ(ξ,d,h,t)=ξ-(ξ T h / ||h||2)(ξ||h||2sin(t||h||2)+h(1-cos(t||h||2))).

[0085] Substitute the obtained Riemann gradient G and search direction H back into the linear search optimal moving step η (i) In the iterative loop, the updated dictionary {D A ,D B}.

[0086] See Figure 2 , wherein step S3 includes:

[0087] S31: A plurality of registered multimodal brain images are subjected to a sliding window technique to obtain corresponding overlapping blocks, and a pre-trained dictionary is used to perform a sparse representation on the overlapping blocks to obtain a corresponding sparse coefficient map.

[0088] Wherein, step S3 further includes:

[0089] S32: Extract spatially significant features based on overlapping blocks using texture contrast measurement;

[0090] S33: Based on the sparse coefficient graph, sparse saliency features and measures are used to extract transformation salient features;

[0091] S34: Combining spatial saliency and transformation saliency to construct a fusion activity measure, and using the activity measure to guide the fusion of sparse coefficients corresponding to multiple multimodal brain images;

[0092] S35: Obtain fused brain images through sparse reconstruction using the pre-trained dictionary.

[0093] Brain image fusion based on sparse representation often constructs fusion activity measures through the transform domain. The detail information in the spatial domain is important for maintaining image clarity, so the fusion activity measure is constructed by combining the spatial domain and the transform domain.

[0094] Among them, TC is used to represent spatial saliency, and SSSF is used to represent transformation saliency. The visual system can identify salient areas from the retina to the visual cortex. One of the early information captured by the visual system on the retina is LC information, while in the visual cortex, OC participates in higher-level context understanding. Inspired by the formation process of the visual system, TC uses a combination of LC and OC to represent spatial saliency. LC is defined as:

[0095]

[0096] The variable n increases the dynamic range of suppressing the proportion of high-contrast non-significant textures in the background. Indicates overlapping blocks , Φ' represents the 3×3 adjacent area centered on the pixel point (x, y), and M represents the size of the area.

[0097] To further illustrate the effect of the variable n on brightness contrast in the present invention, and to adjust the random parameters to optimize the performance of the overall fusion algorithm, the experimental design uses human brain images as an example, and the experimental setting conditions are as follows:

[0098] The multimodal brain images were all from the whole-brain atlas brain imaging database created by Harvard Medical School, and the spatial resolution of the images was 256×256;

[0099] The spatial size of the sliding window is 8×8, the sliding window step is 1, the size of the sub-dictionary is set to 8×16, and the reconstruction error ε is set to 0.1;

[0100] All experiments were conducted on a computer with a 3.3GHz CPU and 16.0GHz RAM, using MATLAB R2017a and a Windows 7 64-bit operating system. The optimal value of variable n was determined by measuring how the brightness contrast of brain images changes with the variable n.

[0101] See Figure 5 As shown in the figure, increasing the variable n effectively highlights high-contrast salient regions, but also results in significant information loss. Therefore, the value of variable n affects not only brightness contrast but also the degree of information retention in the subsequent brain image fusion results. Here, a variable n = 3 is reasonable.

[0102] In addition to brightness contrast, directional contrast is used to characterize local brain image structure through a weighted structure tensor. The weighted structure tensor can more effectively summarize the dominant direction of each significant feature and the energy along that direction by weighting the gradients of each brain image. It is defined as:

[0103]

[0104] in, and Respectively represent the gradient along the x and y directions based on a given pixel point (x, y). k (x,y) represents the weight, which is defined as:

[0105]

[0106] in, Represents a local saliency measure, which reflects the importance of a pixel by calculating the sum of the intensities around the pixel point (x, y) and is defined as:

[0107]

[0108] In which, |·| represents the absolute value operation. In order to characterize the structural features of the local image, due to the weighted structure tensor matrix The semi-positive definiteness of , and the singular value decomposition of , can be obtained:

[0109]

[0110] The directional contrast defined by the singular values ​​β1 and β2 of the weighted structure tensor matrix is ​​expressed as:

[0111]

[0112] Among them, η>-1, this parameter represents the relative importance of directional contrast to diagonal structures.

[0113] In order to further illustrate the effect of the variable η on the directional contrast in the present invention, and to adjust the random parameters to optimize the performance of the overall fusion algorithm, the experimental setting conditions are as follows:

[0114] The multimodal brain images were all from the whole-brain atlas brain imaging database created by Harvard Medical School, and the spatial resolution of the images was 256×256;

[0115] The spatial size of the sliding window is 8×8, the sliding window step is 1, the size of the sub-dictionary is set to 8×16, and the reconstruction error ε is set to 0.1;

[0116] All experiments were conducted on a computer with a 3.3GHz CPU and 16.0GHz RAM, using MATLAB R2017a and a Windows 7 64-bit operating system. The optimal value of variable n was determined by measuring how the brightness contrast of brain images changes with the variable n.

[0117] See Figure 6 The optimal value of variable η is determined by measuring the change in brain image directional contrast with variable η. As variable η increases, the texture structure in the brain image becomes clearer, which is more conducive to extracting directional contrast information from the brain image. Here, variable η = 0.5 is reasonable.

[0118] At this time, the texture contrast TC defined by combining the brightness contrast LC and the direction contrast OC is expressed as: and will obtain Normalized to the interval [0,255] for grayscale representation.

[0119] Based on the texture contrast that represents spatial saliency, the transformation saliency is represented by sparse saliency features and SSSF. First, the pre-trained dictionary {D A ,D B} for overlapping blocks Perform sparse coding to obtain the corresponding sparse coefficient graph Then, through the sparse coefficient graph The difference between adjacent regions in the sum is used to highlight the significant sparse features, which is expressed as:

[0120]

[0121] Where P and Q represent the size of the sparse matrix. The local sparse saliency feature LSSF represents the sparse saliency difference between adjacent pixels and is defined as:

[0122]

[0123] Where Φ represents overlapping blocks A sliding window centered on the sparse coefficient corresponding to the pixel point (x, y).

[0124] Combining texture contrast TC with sparse saliency features and SSSF, the fusion activity measure is defined as:

[0125]

[0126] in, Indicates source overlapping blocks Then, the sparse coefficient fusion is performed by taking the maximum measurement value rule, which is expressed as:

[0127]

[0128] Through sparse reconstruction, the fusion result of the rth overlapping block can be obtained as follows:

[0129]

[0130] The final fused brain image I is obtained by fusion of block images F .

[0131] See Figure 4 Specifically, start inputting source image I k ,k∈{1,2,K,K}, the sliding window technique obtains overlapping block images Define brightness contrast via the intensity property of a pixel: Directional contrast is defined by a weighted structure tensor: Texture contrast is defined by combining brightness contrast and directional contrast: Combining texture contrast and sparse saliency features and constructing a fusion activity measure: Use maximum measure value planning to perform sparse coefficient fusion:

[0132] Sparse coding obtains the corresponding sparse coefficient graph pass The differences between adjacent regions define local sparse salient features: pass The difference between adjacent regions and the definition of sparse saliency features and: Combining texture contrast and sparse saliency features and constructing a fusion activity measure:

[0133] The sparse coding part uses the orthogonal matching pursuit method to update the sparse coefficients; provides the updated sparse coefficients S; provides the current sub-dictionary {D A ,D B The dictionary update part uses the manifold-based conjugate gradient method to update the sub-dictionary; the separation dictionary learning algorithm provides a pre-trained dictionary, and finally the sparse reconstruction is used to obtain the fused image: Finish.

[0134] This method can represent richer structural texture features through a sparse matrix composed of associated sparse coefficients via separate dictionary learning, and improves dictionary update efficiency through synchronous updates in the form of sub-dictionary matrices. Texture contrast and sparse saliency features are used, and a new activity measure is constructed to retain more complete texture information. Sparse reconstruction is used to obtain fused brain images, effectively alleviating the impact of texture information loss on dictionary learning brain image fusion. This method can effectively improve the texture information loss deficiency of sparse representation in multimodal brain image fusion, allowing the fusion result to retain more complete texture structure information and obtain brain functional imaging that meets the needs of complex behavioral analysis.

[0135] Although the present disclosure is disclosed as above, the protection scope of the present disclosure is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present disclosure, and these changes and modifications will fall within the protection scope of the present invention.

Claims

1. A multimodal brain image fusion method for analyzing fish predation behavior, characterized in that: Including steps: S1: Based on the separation dictionary learning, a sparse matrix composed of associated sparse coefficients is used to represent the structural texture features; S2: Pre-trained dictionary is obtained through cyclic iteration through orthogonal matching pursuit method and conjugate gradient method on Riemann manifold ; S3: Using texture contrast and sparse saliency features and construct activity measures to preserve texture information, a fused brain image is obtained through sparse reconstruction. Wherein, the step S3 includes: S31: Multiple registered multimodal brain images are subjected to a sliding window technique to obtain corresponding overlapping blocks, and a pre-trained dictionary is used to perform sparse representation on the overlapping blocks to obtain a corresponding sparse coefficient map; S32: Extract spatially significant features based on overlapping blocks using texture contrast measurement; S33: Based on the sparse coefficient graph, sparse saliency features and measures are used to extract transformation salient features; S34: Combining spatial saliency and transformation saliency to construct a fusion activity measure, and using the activity measure to guide the fusion of sparse coefficients corresponding to multiple multimodal brain images; S35: Obtain fused brain images through sparse reconstruction using pre-trained dictionaries; Wherein, the steps S31 to S35 specifically include: Input source image , ; Sliding window technology to obtain overlapping block images , defining brightness contrast via the intensity attribute of a pixel: ； Directional contrast is defined by a weighted structure tensor: ; Use the pre-trained dictionary obtained through dictionary learning For overlapping blocks Perform sparse coding to obtain the corresponding sparse coefficient graph ; Through the sparse coefficient graph The difference between adjacent regions in the image is used to highlight the significant sparse features, which is defined as the sparse significant feature sum and is expressed as: ; in, P and Q Indicates the size of the sparse matrix; The local sparse saliency feature LSSF represents the sparse saliency difference between adjacent pixels and is defined as: ； in, Indicates overlapping blocks Medium pixel The sliding window centered on the corresponding sparse coefficient; Combining texture contrast TC with sparse saliency features and SSSF, the fusion activity measure is defined as: ； in, Indicates source overlapping blocks Corresponding measurement results; The sparse coefficient fusion is performed using the maximum measurement value rule, which is expressed as: ； Through sparse reconstruction, we can obtain r The fusion result of overlapping blocks is: ; The final fused brain image is obtained by fusing the block images .

2. The multimodal brain image fusion method for analyzing fish predation behavior according to claim 1, characterized in that: The step S1 comprises: S11: Establish the initial dictionary atoms and the current residual, use the initial dictionary atoms and the current residual to determine the index of the most relevant dictionary atoms, and obtain the index set after multiple iterations. Use the least squares method to update the sparse coefficients, and calculate the partial derivative of the sparse coefficients to obtain the reconstructed samples under the current dictionary and the updated residual; S12: Re-determine the index of the most relevant dictionary atom by combining the updated residual and the dictionary atom to complete the iterative cycle and obtain the updated sparse coefficient under the orthogonal matching pursuit method.

3. The multimodal brain image fusion method for analyzing fish predation behavior according to claim 2, characterized in that: The step S2 comprises: S21: Project the group variables composed of the separation dictionary onto the Riemannian manifold, and move the step size through the geodesic linear search to obtain the Riemannian gradient of the current point in the tangent space and the corrected search direction; S22: Using the Riemannian gradient and the modified search direction to determine the step size through linear search, the update dictionary array under the conjugate gradient method on the Riemannian manifold is obtained; S23: Combine the orthogonal matching pursuit method and the conjugate gradient method on Riemannian manifold for cyclic iteration, and obtain a pre-trained dictionary after sample training.

4. The multimodal brain image fusion method for analyzing fish predation behavior according to claim 3, characterized in that: The objective function of the dictionary learning is: ； in, represents the sparse coefficient; N Represents training samples Y The number of and Represents a sub-dictionary; The dictionary learning process includes two parts: sparse coding and separate dictionary update.

Images