Advanced multimodal medical image fusion system

Abstract

A multimodal medical image fusion system consisting of: an input module comprising a storage unit and a user interface, wherein the input module is configured to allow the medical images to be fed into the system as input, wherein the storage unit is configured to store the input medical images, and the user interface enables user interaction, thereby facilitating uploading of the input medical images; a central processing unit having a memory configured to perform adaptive decomposition-based image fusion of the input medical images, the central processing unit comprising: a signal decomposition module configured to decompose input medical images using Hilbert vibration decomposition (HVD) to generate instantaneous image amplitudes, wherein the image matrices are converted into one-dimensional vectors that are further reorganized to generate instantaneous image amplitudes; an edge preservation module operatively connected to the signal decomposition module and comprising a guided filter configured to receive the instantaneous image amplitudes and preserve image edges while filtering the instantaneous image amplitudes; and a fusion module operating on a Laplacian pyramid-based fusion mechanism and configured to generate pyramid levels of the filtered instantaneous image amplitudes, apply a maximum selection fusion rule to individual pyramid levels, and combine the fused instantaneous image amplitudes to generate a final fused medical image; an output module configured to display the final fused medical images, wherein the output module is connected to the user interface for displaying the fused medical images via a display; and a communication interface configured to enable data communication between different components of the system, the communication interface facilitating the transmission of the input image from the input module to the central processing unit, the transmission of data between different modules of the central processing unit, and the transmission of data from the central processing unit to the output module.

Description

FIELD OF THE INVENTIONThe present disclosure relates to an advanced multimodal medical image fusion system, in particular a multimodal medical image fusion system for medical applications. In particular, the present invention relates to a system based on adaptive decomposition with guided filtering and laplace pyramid-based image fusion for medical applications.BACKGROUND OF THE INVENTIONMedical image fusion is a key advance in digital image processing. It overcomes the basic limitations of single image display by combining information from multiple imaging methods. In the medical context, this technique integrates various imaging methods such as computed tomography (CT), magnetic resonance tomography (MRI), positron emission tomography (PET), and x-rays into a comprehensive single image. This improves the diagnostic possibilities and supports complex clinical decision processes.Conventional image fusion methods are faced with considerable challenges, including spatial resolution, edge conservation, and computational complexity constraints. Existing methods can typically be classified into the categories "space" or "transform domain", each approach having specific disadvantages. Spatial domain methods require less computing power, but often impair image quality. Transformation range methods, on the other hand, provide better subjective results, but with increased computing effort. Therefore, there is a need to overcome these technological limitations.The foregoing discussion clearly demonstrates that a system is needed that enables image fusion without sacrificing quality. The present invention proposes a system using several transformation techniques. The proposed system combines Hilbert vibration decomposition (HVD), guided filtering and Laplace pyramid fusion and eliminates existing critical restrictions in medical image processing. It provides adaptive decomposition, superior edge conservation, and efficient fusion mechanisms that minimize computational effort while maximizing diagnostic information extraction.SUMMARY OF THE INVENTIONThe present disclosure relates to an advanced multimodal medical image fusion system for medical applications. The system is configured to break down medical images into instantaneous amplitudes, preserve critical edge details through guided filtering, and integrate these components using a sophisticated Laplace pyramid approach. The system includes an adaptive image decomposition module that uses Hilbert vibration decomposition (HVD) to decompose the image into different energy components. The desired features of the decomposed image components are passed through a guided filter for edge conservation. The system also comprises an image fusion module using a Laplace pyramid fusion mechanism that integrates the filtered parts according to the "Choose-Max" rule. The subjective results of this system for various publicly available medical image datasets are clear and better than those of the various high-volume systems. In addition, the values obtained for various objective evaluation metrics such as information information entropy (IE): 7.6943, 5.9737, mean value: 110.6453, 54.6346, standard deviation (SD): 85.5376, 61.8129, average slope (AG): 109.2818, 64.6451, spatial frequency (SF): 0.1475, 0.1100 and edge metric (QHK / S): 0.5400, 0.6511 demonstrate the comparison with other systems. This system is characterized by improved spatial resolution, reduced block effects and minimal color distortions and is therefore particularly suited for real-time medical imaging applications. The system represents a significant advance in medical image fusion and provides researchers and clinicians with a more precise and computationally efficient system for analyzing complex medical image data.An object of the present disclosure is to provide a multimodal medical image fusion system for medical applications. The system comprises: an input module having a storage unit and a user interface, the input module configured to allow medical images to be input to the system, the storage unit configured to store the input medical images, the user interface allowing user interaction and facilitating uploading of the input medical images; a centralized processing unit having a memory configured to perform adaptive decomposition-based image fusion of the input medical images, the centralized processing unit comprising: a signal decomposition module configured to decompose input medical images by Hilbert Vibration Decomposition (HVD) to generate instantaneous image amplitudes, wherein the image matrices are converted into one-dimensional vectors that are further reorganized to generate instantaneous image amplitudes, an edge obtaining module operatively connected to the signal decomposition module and comprising a guided filter configured to receive the current image amplitudes and obtain image edges while filtering the current image amplitudes, and a fusion module operating on a Laplace pyramid-based fusion mechanism configured to generate pyramid levels of the filtered current image amplitudes, apply a maximum selection fusion rule to individual pyramid levels, and combine the fused current image amplitudes to generate a final fused medical image; an output module configured to display the final fused medical images, wherein the output module is connected via a display to the user interface for displaying the fused medical images; and a communication interface configured to enable data communication between different components of the system, wherein the communication interface facilitates transmission of the input image from the input module to the central processing unit, data transmission between different modules of the central processing unit, and data transmission from the central processing unit to the output module.An object of the present disclosure is to provide an advanced multimodal medical image fusion system for medical applications.Another object of the present disclosure is a system integrating Hilbert vibration decomposition (HVD) and Laplace pyramid (LP), in which energy components (instantaneous image amplitudes) obtained by HVD are integrated using pyramids.Another object of the present disclosure is to provide better spatial details in the output image and to protect edges using a guided filter (GF) applied to the image amplitudes.Another object of the present disclosure is to reduce the blocking effects and other subjective disadvantages arising from down sampling and blurring effects in the pyramid process.Another object of the present disclosure is to provide a more precise and computationally efficient system for analyzing complex medical image data.In order to further clarify the advantages and features of the present disclosure, the invention will be explained in more detail with reference to specific embodiments that are illustrated in the accompanying drawings. These drawings illustrate only typical embodiments of the invention and are therefore not to be considered as limiting the scope thereof. The invention will be described and explained in more detail with reference to the accompanying drawings.BRIEF DESCRIPTION OF THE FIGURESThese and other features, aspects, and advantages of the present disclosure will become more fully understood when the following detailed description is read with reference to the accompanying drawings, in which like characters represent like parts throughout. The following applies here: FIG. 1 is a block diagram of a multi-mode medical image fusion system for medical applications according to an embodiment of the present disclosure; and FIG. 2 is a diagram showing the operation of the proposed multimodal medical image fusion system according to an embodiment of the present disclosure.Those skilled in the art will also appreciate that the elements in the drawings are shown for simplicity and are not necessarily to scale. For example, the flowcharts illustrate the method using the key steps to improve understanding of aspects of the present disclosure. In addition, regarding the construction of the apparatus, individual or multiple components of the apparatus may be represented by conventional symbols in the drawings. The drawings may only show the specific details relevant to understanding the embodiments of the present disclosure in order not to obscure the drawings with details readily apparent to those skilled in the art after the present description.DETAILED DESCRIPTION:In order to aid in the understanding of the principles of the invention, reference will now be made to the embodiment illustrated in the drawings and will be described in an comprehensible manner. However, the scope of the invention is not limited thereby. Changes and further modifications of the illustrated system, as well as further applications of the principles of the invention, are possible, as would normally occur to a person skilled in the art.It will be understood by those skilled in the art that the foregoing general description and the following detailed description are exemplary and explanatory of the invention and are not intended to be limiting thereof.References throughout this specification to "one aspect," "another aspect," or similar language mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. Thus, the phrases "in one embodiment," "in another embodiment," and similar phrases in this specification may or may not refer to the same embodiment.The terms "comprises," "comprising," or other variations thereof are intended to cover a non-exclusive inclusion, such that a process or method comprising a list of steps may include not only those steps, but also other steps not expressly listed or inherent in that process or method. Likewise, the phrase "comprises... for" one or more devices, subsystems, elements, structures, or components does not exclude, without further limitations, the existence of other devices, subsystems, elements, structures, components, or additional devices, subsystems, elements, structures, or components.Unless otherwise defined, all technical and scientific terms used herein have the same meaning as understood by one of ordinary skill in the art. The systems, methods, and examples provided herein are for illustrative purposes only and are not to be considered limiting.Embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.The functional units described in this specification are referred to as devices. A device may be implemented in programmable hardware devices such as processors, digital signal processors, central processing units, field programmable gate arrays, programmable array logic systems, programmable logic devices, cloud processing systems, or the like. The devices may also be implemented in software for execution by various types of processors. An identified device may include executable code and may consist, for example, of one or more physical or logical blocks of computer instructions, which may be organized, for example, as an object, procedure, function, or other construct. However, the executable of an identified device need not be physically stored at the same location, but may consist of different instructions stored at different locations that, logically linked, form the device and serve its purpose.Device or module executable code may consist of one or more instructions and even be distributed over multiple code segments, different applications, and multiple storage devices. Likewise, operational data may be identified and displayed within the device and presented in any form and data structure. The operational data may be acquired as a single data set or distributed across different storage devices and may be at least partially present as electronic signals in a system or network.References throughout this specification to "a selected embodiment," "an embodiment," or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the disclosed subject matter. Therefore, the terms "a selected embodiment," "in one embodiment," or "in one embodiment" in various places throughout this specification do not necessarily refer to the same embodiment.Moreover, the described features, structures, or characteristics may be combined in any manner in one or more embodiments. The following description contains numerous specific details to provide a thorough understanding of the embodiments of the disclosed subject matter. However, those skilled in the art will appreciate that the disclosed subject matter may be practiced without one or more of the specific details or with other methods, components, materials, etc. In other instances, well-known structures, materials, or operations are not shown or described in detail in order not to obscure aspects of the disclosed subject matter.According to the example embodiments, the disclosed computer programs or modules may be executed in a variety of ways, such as an application in a device's memory or a hosted application on a server that communicates with the device application or browser via various standard protocols such as TCP / IP, HTTP, XML, SOAP, REST, JSON, and other suitable protocols. The disclosed computer programs may be written in example programming languages that execute from the memory of the device or from a hosted server, such as BASIC, COBOL, C, C++, Java, Pascal, or scripting languages such as JavaScript, Python, Ruby, PHP, Perl, or other suitable programming languages.Some of the disclosed embodiments include or otherwise involve data transfer over a network, for example, the transfer of various inputs or files over the network. The network may include, for example, the Internet, wide area networks (WANs), local area networks (LANs), analog or digital wired and wireless telephone networks (e.g., PSTN, Integrated Services Digital Network (ISDN), cellular networks and digital subscriber line (xDS)), radio, television, cable, satellite, and / or other transmission or tunneling mechanisms for data transmission. The network may comprise multiple networks or sub-networks, each including, for example, a wired or wireless data path. The network may comprise a circuit switched voice network, a packet switched data network or other network for transferring electronic communication. For example, the network may comprise networks based on Internet Protocol (IP) or Asynchronous Transfer Mode (ATM) and support voice, for example, via VoIP, voice over ATM, or other comparable protocols for voice data communication. In one implementation, the network includes a cellular network configured to exchange text or SMS messages.Examples of the network include a personal area network (PAN), a storage area network (SAN), a home area network (HAN), a campus area network (CAN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a virtual private network (VPN), an enterprise private network (EPN), the Internet, a global area network (GAN), etc.FIG. 1 shows a block diagram of a multi-mode medical image fusion system for medical applications according to an embodiment of the present disclosure.Referring to FIG. 1, the multi-mode medical image fusion system (100) includes: an input module (102) having a storage unit (102a) and a user interface (102b), the input module (102) configured to allow the medical images to be input to the system (100), the storage unit (102a) configured to store the input medical images, and the user interface (102b) allowing user interaction and facilitating uploading of the input medical images; a central processing unit (104) having a memory configured to perform adaptive decomposition-based image fusion of the input medical images, the central processing unit (104) including: a signal decomposition module (106) configured to: decompose input medical images using Hilbert Vibration Decomposition (HVD), to generate instantaneous image amplitudes, wherein the image matrices are transformed into one-dimensional vectors that are further reorganized to generate instantaneous image amplitudes, an edge obtaining module (108) operatively connected to the signal decomposing module (106) and comprising a guided filter (108a) configured to receive the instantaneous image amplitudes and obtain image edges while filtering the instantaneous image amplitudes, and a fusion module (110) operating on a Laplace pyramid-based fusion mechanism configured to generate pyramid levels of the filtered instantaneous image amplitudes, apply a maximum selection fusion rule to individual pyramid levels, and apply the fused instantaneous image amplitudes to generate a final fused medical image; an output module (112) configured to display the final fused medical images, the output module (112) connected to the user interface (102b) for displaying the fused medical images via a display; and a communication interface (114) configured to enable data communication between various components of the system, the communication interface (114) enabling the transmission of the input image from the input module (102) to the central processing unit (104), the transmission of data between various modules of the central processing unit (104), and the transmission of data from the central processing unit (104) to the output module (112).In one embodiment, the signal decomposition module (106) is further configured to decompose multi-component non-stationary medical images into simpler components, extract components with slowly varying instantaneous amplitudes and frequencies, and generate instantaneous amplitudes based on the harmonic or quasiharmonic functions of the signal. Additionally, the signal decomposition module ( 106) is also configured to process multiple image components simultaneously, extract inherent signal components across different time scales, and generate instantaneous amplitudes based on discrete signal processing techniques.In one embodiment, the guided filter (108a) of the edge conservation module (108) is also configured to preserve sharp edges during image decomposition, operate independently of filter size, and prevent edge distortion or obscuring during image processing.In one embodiment, the fusion module (110) is further configured to generate pyramid levels by iterative blurring and down sampling, use a Gaussian window to blurring the components, and apply a maximum selection rule to merge the image components on each pyramid level.In one embodiment, the fusion module (110) is also configured to restore the fused image by: expanding downscaled image versions and summing expanded components across multiple pyramid levels.In one embodiment, fusion module (110) is also configured to reduce blocking effects inherent in conventional pyramid-based fusion methods; retain spatial details of source medical images; and generate a fused image with enhanced diagnostic information.In one embodiment, the system (100) is specifically configured for medical imaging applications including: integration of soft tissue images; integration of hard tissue images; and enhancement of computer-assisted diagnostic functions, and wherein the system (100) is configured to integrate information from multiple medical imaging modalities with enhanced diagnostic precision.In one embodiment, the system (100) further comprises: a computing power optimization module (116) coupled to the central processing unit and configured to minimize processing time, reduce color distortions, and minimize unwanted pixel intensity variations.The present invention relates to an advanced multimodal medical image fusion system for medical applications. The system integrates multiple medical imaging methods with non-example precision and efficiency. The core of the system is an innovative combination of Hilbert-Vibration Decomposition (HVD), guided filtering and Laplace pyramid fusion to transform medical image analysis and interpretation. The first stage involves Hilbert-Vibration Decomposition (HVD), which decomposes medical images into their instantaneous image amplitudes, and extracts complex image components with considerable spatial resolution. After decomposition, the system utilizes a guided filtering mechanism which preserves critical edge details and spatial information and thus ensures that the most important diagnostic features are retained throughout the image processing. The guided filter acts as an advanced preservation tool that prevents edge distortions and maintains the integrity of complex medical image features. After the filtering stage, the Laplace pyramid fusion mechanism configures the system so that the filtered image components are integrated by a complex multi-stage approach. This fusion strategy applies a maximum selection rule to individual pyramid levels and ensures that the most important information from each source image is retained in the final output. The result is a synthesized image with improved diagnostic capabilities, improved spatial resolution, reduced noise, and minimal color distortions. The architecture of the system allows multiple medical imaging procedures to be processed simultaneously, thus providing clinicians a more comprehensive survey of diagnostic information. By combining advanced signal processing techniques, the invention overcomes traditional medical image fusion limitations and provides a pathbreaking solution for more precise and differentiated medical diagnostics.FIG. 2 is a diagram showing the operation of the proposed multimodal medical image fusion system according to an embodiment of the present disclosure.Referring to Figure 2, the operation of the system is described as follows: (i) source images are decomposed by HVD into two current image amplitudes, all image matrices are converted to 1-D format by row or column sorting, and then reorganization is performed to generate current image amplitudes, (ii) current image amplitudes are passed through a Guided Filter (GF) for better feature extraction and edge conservation, (iii) filtered current image amplitudes are merged by a Laplace pyramid based fusion, and (iv) the merged current image amplitudes are combined to generate the merged image.In one embodiment, the proposed system utilizes Hilbert Vibration Decomposition (HVD), a signal processing method that configures the system to extract components (modes) from broadband non-stationary signals. A non-stationary, multicomponent signal is broken down into simpler components with slowly varying instantaneous amplitudes and frequencies. The first extracted component represents the varying maximum amplitude, while the remainder contains components with lower amplitudes. The decomposition is based on three assumptions: (i) The signal consists of symmetrical quasiharmonic functions, (ii) each oscillating component has different envelopes, and (iii) each component comprises several periods of the slowest components. Global time domain analysis of the instantaneous frequency (IF) of the signal allows extraction of inherent components across different time scales. A signal is represented as a combination of harmonic or quasiharmonic functions and is broken down into mono-components, each component being expressed as a multiplication of its instantaneous amplitude and cosine of its instantaneous phase. This concept is extended to two input images, with the pixels of both images being rearranged into row / column vectors and their instantaneous amplitudes calculated using a discretized equation. The calculated amplitudes are then converted again into 2D matrices, so-called instantaneous image amplitudes. An edge-maintaining guided filter (GF) is applied to the instantaneous image amplitudes to avoid sharp edge distortions. The filtered instantaneous image amplitudes of both images are then fused, resulting in fused instantaneous image amplitudes. These fused amplitudes are summed to produce the final fused image. The fusion of the system is based on the Laplace pyramid (LP). Iterative blurring and down sampling make a compact representation of the image. The filtered instantaneous image amplitudes are soft-drawn and down-sampled on several levels, resulting in different levels of the Laplace pyramid. The fuzzy versions are subtracted from the originally filtered amplitudes to generate quasiband information at each level. A window function determines the blur size and the LP planes are mathematically represented. Select-max fusion is applied at individual levels, thereby generating the fused instantaneous image amplitudes. The original image is reconstructed by expanding and summing the down-sampled versions. The final fused image is obtained by summing the fused instantaneous image amplitudes across all planes.In one embodiment, a simulation of the image fusion system is performed. Based on the simulation, subjective and objective evaluations of the system are performed. The parameters obtained from the evaluations are compared with various novel techniques and the computational power of the system is evaluated. To evaluate the system, quantitative metrics are calculated including mean (average pixel value), standard deviation, information entropy, spatial frequency, average gradient, and edge information metric. The system is compared to the current state of the art to evaluate its efficiency compared to existing solutions. The results showed that the obtained values of various objective evaluation metrics such as information entropy (IE): 7.6943, 5.9737, mean value: 110.6453, 54.6346, standard deviation (SD): 85.5376, 61.8129, average gradient (AG): 109.2818, 64.6451, spatial frequency (SF): 0.1475, 0.1100 and edge metric (QHK / S): 0.5400, 0.6511 demonstrate the efficiency and comparison with other existing solutions. The evaluation of the system also showed that the runtime of the system is 0.161244 seconds, which indicates a high computing power. The outputs of the system show a clearer image with less noise artifacts. In addition, the fusion metrics are also better or comparable to existing solutions. The high computing power of the proposed system showed its superiority and made it suitable for real-time processing applications. However, there are still some limitations such as the requirement of the pixel array in 1D format. The fusion system is also suitable for medical images and some examples of multifocal images. Therefore, there is a future need to develop a generalized system for other applications of image fusion. Moreover, the development of a 2D HVD can reduce computing time and be more effective for real-time applications.The drawings and the foregoing description show examples of embodiments. Those skilled in the art will appreciate that one or more of the described elements may well be combined into a single functional element. Alternatively, certain elements may be divided into multiple functional elements. Elements of one embodiment may be added to another embodiment. For example, the order of the processes described herein may be changed and is not limited to the manner described herein. Moreover, the actions of a flow chart need not be performed in the order shown; nor do all actions necessarily need to be performed. Also, actions that are not dependent on other actions may be performed in parallel with the other actions. The scope of the embodiments is by no means limited by these specific examples. Numerous variations, whether or not explicitly stated in the specification, such as differences in structure, dimensions, and material use, are possible. The scope of the embodiments is at least as broad as recited in the following claims.Advantages, other advantages and solutions to problems have been described above with reference to specific embodiments. However, the advantages, merits, solutions to problems and any components that may result in an advantage, merit or solution being introduced or enhanced are not to be understood as critical, required or essential features or components of individual or all claims.REFERENCES100 A Multimodal Medical Image Fusion System For Medical Applications. 102 Input module 102 a Speichereinheit unit 102 b Benutzer 104 Central processing unit 106 Signal separation module 108 Edge obtaining module 108 a Geführt filter 110 Fusion module 112 Output module 114 Communication interface 116 Module for optimizing the computing power 202 Source image 1 204 Source image 2 206 Row / column sorting 208 HVD With rearrangement 210 Current image amplitudes 212 Guided filtering 214 Fusion based on 216 Fused image amplitudes 218 Fused image

Claims

A multi-mode medical image fusion system comprising: an input module comprising a storage unit and a user interface, the input module configured to allow the medical images to be fed as input to the system, the storage unit configured to store the entered medical images, and the user interface to allow user interaction, thereby facilitating uploading the entered medical images; a central processing unit having a memory configured to perform adaptive decomposition-based image fusion of the input medical images, the central processing unit comprising: a signal decomposition module configured to decompose input medical images using Hilbert Vibration Decomposition (HVD) to generate instantaneous image amplitudes, wherein the image matrices are converted into one-dimensional vectors that are further reorganized to generate instantaneous image amplitudes; an edge obtaining module operatively connected to the signal decomposition module and comprising a guided filter configured to receive the instantaneous image amplitudes and retain image edges while filtering the instantaneous image amplitudes; and a fusion module operating on a Laplace pyramid-based fusion mechanism and configured to generate pyramid levels of the filtered instantaneous image amplitudes, apply a fusion rule to individual pyramid levels for maximum selection, and combine the fused instantaneous image amplitudes to generate a final fused medical image; an output module configured to display the final fused medical images, the output module being connected to the user interface for displaying the fused medical images via a display; and a communication interface configured to enable data communication between different components of the system, the communication interface facilitating transmission of the input image from the input module to the central processing unit, data transmission between different modules of the central processing unit, and data transmission from the central processing unit to the output module.The system of claim 1, wherein the signal decomposition module is further configured to decompose multi-component non-stationary medical images into simpler components, extract components with slowly varying instantaneous amplitudes and frequencies, and generate instantaneous amplitudes based on the harmonic or quasiharmonic functions of the signal, and wherein the decomposition module is further configured to process multiple image components simultaneously, extract inherent signal components across different time scales, and generate instantaneous amplitudes based on discrete signal processing techniques.The system of claim 1, wherein the guided filter of the edge conservation module is further configured to obtain sharp edges during image decomposition, operate independently of filter size, and prevent edge distortion or obscuring during image processing.The system of claim 1, wherein the fusion module is further configured to generate pyramid levels by iterative blurring and down sampling, use a Gaussian window to blurring the components, and apply a maximum selection rule to merge the image components on each pyramid level.The system of claim 1, wherein the fusion module is further configured to restore the fused image by: expanding down-sampled image versions and summing expanded components across multiple pyramid levels.The system of claim 1, wherein the fusion module is further configured to reduce blocking effects occurring in conventional pyramid-based fusion methods; retain spatial details of source medical images; generate a fused image with augmented diagnostic information.The system of claim 1, wherein the system is specifically configured for medical imaging applications including: integration of soft tissue images; integration of hard tissue images; and enhancement of computer-assisted diagnostic functions, and wherein the system is configured to integrate information from multiple medical imaging modalities with enhanced diagnostic precision.The system of claim 1, further comprising: a computing power optimization module coupled to the central processing unit and configured to minimize processing time, reduce color distortions, and minimize unwanted pixel intensity variations.

Images