A Deep Learning-Based SPECT / CT Image Fusion Method and System

Abstract

The present invention discloses a SPECT / CT image fusion method and system based on deep learning, including: acquiring images respectively through SPECT technology and CT technology; constructing a multi-modal deep learning feature extraction network to extract features of SPECT and CT images respectively; generating a fusion strategy for the extracted features; and fusing SPECT and CT according to the fusion strategy. By constructing a multi-modal feature extraction network through deep learning, precise fusion of SPECT and CT images is achieved, fully retaining the functional information of SPECT and the anatomical details of CT, improving the clarity and contrast of the images, enhancing the detection ability for abnormal regions, significantly improving the diagnostic accuracy, and providing more comprehensive support for clinical decision-making.

Description

Technical Field

[0001] The present invention relates to the technical field of image fusion, and particularly to a SPECT / CT image fusion method and system based on deep learning. Background Art

[0002] In the field of medical imaging, single photon emission computed tomography (SPECT) and computed tomography (CT) are two important imaging technologies, each with unique advantages. SPECT can provide in-vivo functional information, such as metabolic activity, blood flow status, and organ function, etc. By injecting a radioactive tracer, it shows the radioactive distribution in a specific area and is widely used in functional detection of heart diseases, tumors, neurological diseases, etc. However, the spatial resolution of SPECT images is relatively low, and the image edges are blurred, which is not conducive to accurate anatomical localization.

[0003] As a high-resolution structural imaging technology, CT can clearly display anatomical details, such as bones, soft tissues, and blood vessels, etc. Due to its characteristics of fast scanning and clear details, CT images are widely used in fields such as emergency, trauma, and tumor detection. However, CT can only provide anatomical information and lacks the reflection of tissue function status, which limits its application in evaluating metabolic and physiological changes.

[0004] In order to improve the diagnostic accuracy, in recent years, the SPECT / CT fusion imaging technology has emerged. By combining the functional information of SPECT with the anatomical structure information of CT, it realizes the synchronous display of function and anatomy, providing a more comprehensive reference for clinical diagnosis and treatment planning. However, traditional fusion methods usually rely on simple image superposition or manual adjustment, which not only makes it difficult to ensure the precise alignment of the two images, but also the fusion effect is easily interfered by artifacts and noise, resulting in an unsatisfactory diagnostic effect.

[0005] With the development of deep learning technology, researchers have begun to explore the use of advanced algorithms such as convolutional neural networks (CNNs) to extract and fuse features of multimodal images in order to achieve more precise image registration and fusion. Deep learning methods can automatically learn the complex feature relationships of images and have high adaptability and robustness, providing a new way to improve the quality and efficiency of SPECT / CT image fusion. However, how to retain the key features of each image during the fusion process and maximize the diagnostic value is still a technical problem that needs to be solved urgently. Summary of the Invention

[0006] In view of the above existing problems, the present invention is proposed.

[0007] To solve the above technical problems, the present invention provides the following technical solution: A SPECT / CT image fusion method based on deep learning, including:

[0008] Obtain images through SPECT technology and CT technology respectively;

[0009] Construct a multi-modal deep learning feature extraction network to extract features of SPECT and CT images respectively;

[0010] Generate a fusion strategy for the extracted features;

[0011] Fuse SPECT and CT according to the fusion strategy.

[0012] As a preferred solution of the SPECT / CT image fusion method based on deep learning according to the present invention, wherein: functional image data is obtained through the SPECT technology; at the same time, high-resolution anatomical structure image data is obtained through the CT technology.

[0013] As a preferred solution of the SPECT / CT image fusion method based on deep learning according to the present invention, wherein: the image data of SPECT and CT are normalized to unify the pixel value range;

[0014] And the positions of SPECT and CT images are matched to establish a mapping relationship; for the same frame of image data, the mapping results of any two pixel points are different;

[0015] The specific mapping relationship is as follows:

[0016] When obtaining SPECT image data, generate the mapping relationship of each pixel point in the three-dimensional entity:

[0017]

[0018] SP represents the set of pixel points of SPECT image data, represents the mapping relationship, and SW represents the set of positions of the three-dimensional entity; wherein, the i-th pixel point in the set SP can be mapped to the position in the three-dimensional entity of SW;

[0019] When obtaining CT image data, generate the mapping relationship of each pixel point in the three-dimensional entity:

[0020]

[0021] C represents the set of pixel points of CT image data; wherein, the j-th pixel point in the set C can be mapped to the position in the three-dimensional entity of SW;

[0022] According to the position in the three-dimensional entity, establish a mapping relationship between the pixels of the two images respectively;

[0023] Among them, the mapping relationship from SPECT to CT specifically includes: First, using the mapping relationship between SPECT image data and the three-dimensional entity, the position corresponding to each pixel point is obtained, and the obtained position label; during the mapping process of the CT image data and the three-dimensional entity, when the label position appears, the pixel point of the CT image data is obtained , and at the same time, when the label is generated, the pixel point of the SPECT image data is obtained , and the mapping relationship between and is established; all pixel points are completed in sequence by cycling, and the mapping from SPECT to CT is completed;

[0024] The mapping result is expressed as: ;

[0025] Among them, the mapping relationship from CT to SPECT is opposite to the mapping relationship from SPECT to CT; the mapping result is expressed as: .

[0026] As a preferred solution of the SPECT / CT image fusion method based on deep learning according to the present invention, wherein: the construction of the multi-modal deep learning feature extraction network includes a two-stream convolutional neural network, the network includes two independent input streams, which respectively process SPECT and CT image data, and uses a deep convolutional neural network for abnormal feature extraction and position detection;

[0027] Through the training set and the validation set respectively, the deep convolutional neural network of the two independent channels is learned, so that the neural network of each channel can identify the abnormal area of the input image;

[0028] The overall architecture of the deep convolutional neural network includes:

[0029] Input layer: Input size I S = 128×128×1, single-channel image data;

[0030] Convolutional layer 1: Convolution kernel size 3×3, number 32, stride 1, activation function ReLU, output size 128×128×32;

[0031] Pooling layer 1: Max pooling 2×2, stride 2, output size 64×64×32;

[0032] Convolutional layer 2: Convolution kernel size 3×3, number 64, stride 1, activation function ReLU, output size 64×64×64;

[0033] Pooling layer 2: Max pooling 2×2, stride 2, output size 32×32×64;

[0034] Anomaly detection layer: A 3×3 convolutional kernel is used to output a single-channel anomaly feature map with a size of 32×32×1, which is used to mark the positions of anomaly features in the SPECT image.

[0035] As a preferred solution of the SPECT / CT image fusion method based on deep learning according to the present invention, wherein: the extraction of features from the SPECT and CT images respectively includes detecting anomaly features in the SPECT and CT image data through two independent channels of the dual-stream convolutional neural network, and outputting two anomaly feature position maps and , which are the anomaly positions detected in the SPECT and CT images respectively;

[0036] is the anomaly feature position map extracted from the SPECT image, which is an image corresponding to the original image and contains the positions of the anomaly features marked in the SPECT image; is the anomaly feature position map extracted from the CT image, which is an image corresponding to the original CT image and contains the positions of the abnormal anatomical features marked in the CT image;

[0037] In the image, the positions of the anomaly features will be displayed with different visual marks.

[0038] As a preferred solution of the SPECT / CT image fusion method based on deep learning according to the present invention, wherein: the fusion strategy includes that through the interactive selection on the front-end interface, the user can select the fusion strategy;

[0039] Strategy 1: The fusion result with the functional image as the main body, using the CT image to clarify the local contour of the SPECT image;

[0040] Strategy 2: The fusion result with the high-resolution anatomical structure image as the main body, using the SPECT image to mark and locally enhance the CT image.

[0041] As a preferred solution of the SPECT / CT image fusion method based on deep learning according to the present invention, wherein: specifically, the Strategy 1 includes using an edge detection algorithm to extract the anatomical structure contour of the CT image , copying the structural contour of the CT image, and according to the mapping relationship between the SPECT and CT, mapping the extracted CT structural contour to the SPECT image to form a corresponding contour mark ;

[0042] Taking the pixel points of the contour edge as the center, expanding the width by The area forms the inner area of the detection areas on both sides and ;

[0043] Calculate the average radioactivity intensity of the inner area and the outer area and denote them as and ;

[0044] Calculate the radioactivity performance difference degree D:

[0045]

[0046] Set the difference degree threshold , if , it is determined that the radioactivity performances of the inner and outer sides are similar and no processing is performed; if , the radioactivity performances of the inner and outer sides are quite different and edge enhancement is performed; define the edge pixel intensity , and adjust the edge pixel value according to the inner-outer difference:

[0047]

[0048] wherein, represents the enhanced edge pixel intensity; is the weighting coefficient, which is adjusted according to the actual application to control the enhancement intensity;

[0049] The specific content of Strategy 2 includes obtaining the abnormal feature position map extracted from the SPECT image , and according to the positions of the marked abnormal features in it and the mapping relationship between SPECT and CT, locate the abnormal position WZ1 in the CT image;

[0050] Obtain the abnormal feature position map extracted from the CT image the position WZ2 of the marked abnormal features in it;

[0051] Mark WZ1 and WZ2 in the CT image, and through the mapping relationship between SPECT and CT, extract all the marked positions in the CT image in the SPECT image, and copy the functional image of the extracted marked positions to the marked positions in the CT image to obtain the fusion result of Fusion Strategy 2.

[0052] A deep learning-based SPECT / CT image fusion system adopting the method of the present invention, wherein:

[0053] An acquisition unit, which obtains images respectively through SPECT technology and CT technology;

[0054] An extraction unit constructs a multi-modal deep learning feature extraction network to extract features from SPECT and CT images respectively;

[0055] A fusion unit generates a fusion strategy for the extracted features;

[0056] An execution unit fuses SPECT and CT according to the fusion strategy.

[0057] A computer device includes: a memory and a processor; the memory stores a computer program, wherein: when the processor executes the computer program, the steps of the method described in any one of the present inventions are implemented.

[0058] A computer-readable storage medium stores a computer program thereon, wherein: when the computer program is executed by a processor, the steps of the method described in any one of the present inventions are implemented.

[0059] Advantages of the present invention: The SPECT / CT image fusion method based on deep learning provided by the present invention constructs a multi-modal feature extraction network through deep learning to achieve precise fusion of SPECT and CT images, fully retaining the functional information of SPECT and the anatomical details of CT, improving the clarity and contrast of the images, enhancing the detection ability of abnormal regions, significantly improving the diagnostic accuracy, and providing more comprehensive support for clinical decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0061] Figure 1 It is the overall flowchart of a SPECT / CT image fusion method based on deep learning provided by the first embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0062] To make the above objects, features, and advantages of the present invention more obvious and understandable, the specific embodiments of the present invention will be described in detail below with reference to the drawings of the specification. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0063] Example 1, refer to Figure 1, which is an embodiment of the present invention, provides a SPECT / CT image fusion method based on deep learning, including:

[0064] S1: Obtain images through SPECT technology and CT technology respectively.

[0065] Through the SPECT technology, functional image data is obtained; at the same time, through the CT technology, high-resolution anatomical structure image data is obtained. By using SPECT and CT technologies to collect data simultaneously, the same spatial position during shooting of the two images is ensured, which means that the two devices work together and share the same scanning bed or device, thereby ensuring that the data sources of SPECT and CT images are exactly the same. Since the two images are obtained at the same position and time, it is ensured that the same anatomical position corresponds in SPECT and CT images, thereby improving the accuracy and alignment degree during the fusion of the two images and avoiding the deviation and influence on diagnosis caused by different spatial positions.

[0066] Furthermore, the image data of SPECT and CT are normalized to unify the pixel value range; and the positions of SPECT and CT images are matched to establish a mapping relationship; ensure that SPECT and CT images strictly correspond spatially, and the positions of each pixel point in the two images are mapped to the same three-dimensional entity position, thereby ensuring the precise superposition of functional information and anatomical information. For the same frame of image data, the mapping results of any two pixel points are different, making the mapping relationship of each pixel point unique, ensuring that for any two pixel points in the same frame of image data, their mapping results are different, avoiding overlapping or repeated mapping, and guaranteeing the registration accuracy.

[0067] It should be noted that based on mapping the pixel points of SPECT and CT images to the corresponding positions of the three-dimensional entity, the two image data are strictly corresponding. By using the three-dimensional spatial information obtained during the image shooting process, the positions of each pixel point of SPECT and CT can be accurately calibrated, ensuring the scientific nature of the mapping relationship. By associating SPECT and CT images with the positions of the three-dimensional entity respectively, the corresponding pixels of the functional image (SPECT) and the anatomical image (CT) are accurately aligned, enabling the precise matching of their respective image features during the fusion process and realizing the efficient combination of functional and anatomical information.

[0068] The specific mapping relationship is as follows:

[0069] When obtaining SPECT image data, generate the mapping relationship of each pixel point in the three-dimensional entity:

[0070]

[0071] Generating the mapping relationship of each pixel point in the SPECT image data to the three-dimensional entity is a key step in achieving accurate image alignment and fusion. This process mainly involves image registration, spatial calibration, and coordinate transformation techniques to ensure that each SPECT pixel point can be accurately mapped to the actual three-dimensional position.

[0072] Let SP represent the set of pixel points in the SPECT image data, represent the mapping relationship, and SW represent the set of positions in the three-dimensional entity; among them, the i-th pixel point in the set SP can be mapped to the position in the three-dimensional entity of SW; in the mapping between SPECT and the three-dimensional entity, the small region (voxel) corresponding to the pixel point is a reasonable representation of its actual spatial position, fully considering the imaging resolution and the diffusion of radioactive signals. This design meets the actual needs of image fusion, ensures the accurate reflection of functional information, and provides a more accurate image fusion effect in clinical applications.

[0073] When acquiring CT image data, generate the mapping relationship of each pixel point to the three-dimensional entity:

[0074]

[0075] Generating the mapping relationship of each pixel point in the CT image data to the three-dimensional entity is to ensure that the anatomical information in the CT image can be accurately corresponding to the actual three-dimensional spatial position. This process involves steps such as coordinate transformation, spatial reconstruction, and equipment calibration to achieve the accurate registration of the CT image and the actual physical space.

[0076] Let C represent the set of pixel points in the CT image data; among them, the j-th pixel point in the set C can be mapped to the position in the three-dimensional entity of SW.

[0077] According to the positions in the three-dimensional entity, establish the mapping relationship between the pixels of the two images respectively. Among them, the mapping relationship from SPECT to CT specifically includes, first, using the mapping relationship between the SPECT image data and the three-dimensional entity to obtain the position corresponding to each pixel point, and taking the obtained position label; during the mapping process of the CT image data to the three-dimensional entity, when the label position appears, obtain the pixel point of the CT image data and at the same time obtain the pixel point of the SPECT image data when the label is generated , and establish and mapping relationship; loop through all pixel points in turn to complete the mapping from SPECT to CT.

[0078] The mapping result is expressed as: ; wherein, the mapping relationship from CT to SPECT is opposite to the mapping relationship from SPECT to CT; the mapping result is expressed as: 。

[0079] It should be noted that by marking and mapping the positions in the three-dimensional entity, it is ensured that each pixel point of the SPECT and CT images can be accurately aligned. This design overcomes the alignment difficulties caused by the different resolutions and imaging principles of the two images, and ensures the accurate matching of functional information and anatomical information during the fusion process. SPECT provides functional information, while CT provides anatomical structures. By establishing this two-way mapping relationship, the accurate fusion of the two images in space can be achieved, so that the abnormal functional positions of SPECT can be displayed in the CT image, and the anatomical details of CT can be clarified in the SPECT image, forming a more intuitive and diagnostically valuable fused image.

[0080] Traditional fusion methods are prone to information misalignment or overlap, resulting in confusion of anatomical and functional information. Through the clear mapping relationship, each functional information and anatomical information has a unique registration object, thus ensuring the accuracy of the information after image fusion.

[0081] S2: Construct a multi-modal deep learning feature extraction network to extract the features of SPECT and CT images respectively.

[0082] Furthermore, the construction of the multi-modal deep learning feature extraction network includes a two-stream convolutional neural network. The network contains two independent input streams, which process SPECT and CT image data respectively, and uses a deep convolutional neural network for abnormal feature extraction and position detection.

[0083] Respectively, through the training set and the validation set, the deep convolutional neural networks of the two independent channels are learned, so that the neural network of each channel can identify the abnormal regions of the input images.

[0084] The overall architecture of the deep convolutional neural network includes:

[0085] Input layer: Input size I S = 128×128×1, single-channel image data.

[0086] Convolutional layer 1: Convolution kernel size 3×3, number 32, stride 1, activation function ReLU, output size 128×128×32.

[0087] Pooling layer 1: Max pooling 2×2, stride 2, output size 64×64×32.

[0088] Convolutional layer 2: Convolution kernel size 3×3, number 64, stride 1, activation function ReLU, output size 64×64×64.

[0089] Pooling layer 2: Max pooling 2×2, stride 2, output size 32×32×64.

[0090] Anomaly detection layer: Use a 3×3 convolutional kernel to output a single-channel anomaly feature map with a size of 32×32×1 for marking the positions of anomaly features in SPECT images.

[0091] Detect anomaly features in SPECT and CT image data respectively through two independent channels of the dual-stream convolutional neural network, and output two anomaly feature position maps and , which are the anomaly positions detected in SPECT and CT images respectively.

[0092] is the anomaly feature position map extracted from the SPECT image, which is an image corresponding to the original image and contains the positions of anomaly features marked in the SPECT image; is the anomaly feature position map extracted from the CT image, which is an image corresponding to the original CT image and contains the positions of abnormal anatomical features marked in the CT image. In the image, the positions of anomaly features will be displayed with different visual markers.

[0093] It should be noted that SPECT images provide functional information and can show abnormalities in physiological activities such as metabolism and blood flow in the body; CT images provide high-resolution anatomical information and can accurately show the details and abnormalities of anatomical structures. By detecting anomaly features in the two types of images respectively, functional lesions and anatomical abnormalities can be clearly identified, providing more intuitive support for clinical diagnosis. The independent channels of the dual-stream convolutional neural network are used to process SPECT and CT data respectively, and are optimized according to the characteristics of each type of image. The SPECT stream focuses on detecting abnormal regions of metabolism or radioactive distribution, and the CT stream focuses on detecting anatomical structure abnormalities (such as masses, fractures, etc.). This independent processing strategy enables the network to focus more on the extraction of specific features and improves the accuracy of anomaly detection. The output anomaly feature position maps mark the positions of abnormal regions in the images, providing a clear reference for subsequent image fusion. These position maps can not only be used to assist doctors' manual interpretation, but also be further used as the input for automated analysis and diagnostic models.

[0094] S3: Generate a fusion strategy for the extracted features; according to the fusion strategy, fuse SPECT and CT.

[0095] Through the interactive selection of the front-end interface, the user can select the fusion strategy. Strategy 1: The fusion result with functional images as the main body, using CT images to clarify the local contours of SPECT images. Strategy 2: The fusion result with high-resolution anatomical structure images as the main body, using SPECT images to label and locally enhance CT images.

[0096] Further, the specific steps of Strategy 1 include using an edge detection algorithm to extract the anatomical structure contours of CT images , copying the structural contours of CT images, and according to the mapping relationship between SPECT and CT, mapping the extracted CT structural contours into SPECT images to form corresponding contour markings .

[0097] Taking the pixel points of the contour edge as the center, expand the regions with a width of towards the inside and outside respectively to form the detection regions on both sides, the inner region and .

[0098] Calculate the average radioactive intensity of the inner region and the outer region , and record them as and respectively.

[0099] Calculate the radioactive performance difference degree D:

[0100]

[0101] Set the difference degree threshold . If , it is determined that the radioactive performances on the inside and outside are similar and no processing is performed; if , the radioactive performances on the inside and outside are quite different and edge enhancement is performed; define the edge pixel intensity , and adjust the edge pixel value according to the inside-outside difference:

[0102]

[0103] Among them, represents the enhanced edge pixel intensity; is the weighting coefficient, which is adjusted according to actual applications to control the enhancement intensity.

[0104] It should be noted that the anatomical structure contours in CT images are extracted through edge detection algorithms, and these contours are accurately mapped into SPECT images, enabling the functional images to have clear structural markings. These markings make the originally blurred boundaries in SPECT images clearer, providing more intuitive structural information references for clinical diagnosis. The enhancement of the structural contours enables doctors to more clearly observe the correspondence between functional abnormalities and anatomical structures. For example, in tumor diagnosis, the contour markings of CT can help accurately identify the edges of lesions, making the localization of functional abnormalities in SPECT more reliable.

[0105] By calculating the average radioactive intensity of the regions inside and outside the contour, the performance difference degree of functional information at the contour edge is determined. When the difference degree is large, enhancement processing is performed on the contour edge to highlight the radioactive characteristics at the edge position. This enhancement strategy helps improve the contrast of the image, making the functional abnormal regions more obvious at the contour edge, and enhancing the visibility and recognition rate of the abnormal regions.

[0106] A difference degree threshold is set to ensure that enhancement is only performed when there is an obvious difference in radioactive performance between the inside and outside. For regions with similar radioactive performance, unnecessary processing is avoided. This can not only reduce artifacts and noise but also maintain the original authenticity of the image and avoid interference from artificial enhancement to the diagnosis. This design focuses on processing the contour edge region, which can effectively improve the visibility of local details. Especially in the case where the distribution of functional abnormalities is uneven or the boundary is unclear, edge enhancement can help highlight important diagnostic features and provide a more definite abnormal location. Strengthening the edge region with a large radioactive difference makes the edge features of functional abnormalities more prominent, helps refine the manifestation of the lesion, helps doctors more accurately identify the lesion boundary, and improves the diagnostic accuracy.

[0107] The second strategy specifically includes obtaining the abnormal feature position map extracted from the SPECT image , and according to the positions of the marked abnormal features in it and the mapping relationship between SPECT and CT, the abnormal position WZ1 is located in the CT image. Obtain the abnormal feature position map extracted from the CT image the position WZ2 of the marked abnormal features in it.

[0108] Mark WZ1 and WZ2 in the CT image, and through the mapping relationship between SPECT and CT, extract all the marked positions in the CT image in the SPECT image, and copy the functional image at the extracted marked positions to the marked positions in the CT image to obtain the fusion result of the second fusion strategy.

[0109] It should be noted that the specific locations of functional abnormalities (such as metabolic abnormal regions) and anatomical structure abnormalities (such as masses, lesions) are respectively marked through the abnormal feature position maps extracted from SPECT and CT images. The abnormal position WZ1 of SPECT and the abnormal position WZ2 of CT are mapped to each other and precisely aligned through the mapping relationship, facilitating the observation and analysis of the correspondence between functional abnormalities and anatomical abnormalities. By copying the information of the functional abnormal region in the SPECT image to the marked position in the CT image and combining with the anatomical structure of CT, the functional abnormal region becomes more intuitive and prominent in the CT image. This design helps to better combine functional information and anatomical information, assisting doctors in clarifying the nature of the lesion. This design enables the simultaneous presentation of functional abnormalities and anatomical abnormalities in the same image. Especially in the diagnosis of complex lesions, the fused image can more clearly display the morphology and functional performance of the abnormal region, providing important references for clinical diagnosis.

[0110] Furthermore, when copying the extracted functional image to the marked position in the CT image, data tags are inserted to record the following content: the current intensity of the abnormal region, the average intensity inside and outside the abnormal region respectively, as well as the standard deviation, range, etc. of the response intensity inside and outside the region. The recording and analysis of the average intensity, standard deviation, range, and current intensity inside and outside the region can provide rich and detailed functional performance information for clinical practice, making the nature and activity status of the lesion more transparent. Through the fine copying and analysis of the functional image at the CT marked position, the performance of the fused image in terms of functional performance and anatomical structure becomes more intuitive and clear, providing more lesion details and judgment bases for clinical practice, and enhancing the clinical application value of the fused image.

[0111] On the other hand, this embodiment also provides a SPECT / CT image fusion system based on deep learning, which includes:

[0112] An acquisition unit that obtains images through SPECT technology and CT technology respectively.

[0113] An extraction unit that constructs a multi-modal deep learning feature extraction network to extract the features of SPECT and CT images respectively.

[0114] A fusion unit that generates a fusion strategy for the extracted features.

[0115] An execution unit that fuses SPECT and CT according to the fusion strategy.

[0116] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0117] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a definite sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or in combination with these instruction execution systems, apparatus, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

[0118] More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection part with one or more wirings (electronic device), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber device, and portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, then editing, interpreting, or otherwise processing it as appropriate, and then storing it in a computer memory.

[0119] It should be understood that each part of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0120] Example 2, an embodiment of the present invention, provides a SPECT / CT image fusion method based on deep learning. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0121] The test subjects included 20 patients known to have different degrees of functional and anatomical abnormalities, including 10 tumor patients and 10 cardiovascular disease patients. All patients underwent SPECT and CT image acquisitions under the same conditions. The test environment was a standardized imaging diagnosis room, and the equipment included the latest model of dual-modal SPECT / CT system equipped with an optimized deep learning feature extraction network.

[0122] Test preparation:

[0123] Patient preparation: All patients fasted for 6 hours in advance to reduce gastrointestinal interference, and standardized radioactive tracer injections were used to ensure accurate functional reflection of SPECT images.

[0124] Equipment calibration: The SPECT / CT system was calibrated to ensure that SPECT and CT images could be accurately acquired in the same coordinate system.

[0125] Data acquisition: First, functional images were obtained through SPECT technology, and then CT scans were immediately performed to obtain high-resolution anatomical structure images.

[0126] Implementation process:

[0127] Multi-modal feature extraction: The constructed two-stream convolutional neural network was used to process SPECT and CT images respectively to extract the abnormal feature positions of each image. The neural network detected the functional and anatomical abnormalities of the two images through independent channels and output two abnormal feature position maps.

[0128] Image Registration and Mapping: The data of SPECT and CT images are normalized. After unifying the pixel value range, the system - built algorithm is used to map the functional abnormal positions of SPECT into the anatomical structure of CT, and the abnormal regions WZ1 and WZ2 are marked. All marked positions are corrected by the automatic image registration system of the images to ensure the precise correspondence of the pixel points of the two images.

[0129] Fusion Strategy Application:

[0130] Strategy 1: Extract the edges of the CT image. Use the edge detection algorithm to copy and map the anatomical structure contour of the CT image into the SPECT image to generate precise contour markings. Calculate the difference in radioactivity performance on both sides inside and outside the contour edge. If the difference is significant, enhance the edge to highlight the abnormal region.

[0131] Strategy 2: Compare the abnormal feature position map in the SPECT image with the abnormal position map in the CT image. Copy the functional abnormal region to the corresponding marked position in the CT image, and synthesize a fused image to highlight the abnormal contrast between function and structure.

[0132] Data Recording and Analysis: After the fused image is generated, the system automatically records the radioactivity intensity at each marked position, including the average intensity, standard deviation, range, and current intensity inside and outside the region. The fused images of each patient are blindly evaluated by radiologists, and the scoring indicators include the visibility of the abnormal region, diagnostic clarity, and image quality.

[0133] Table 1: Experimental Data of SPECT / CT Image Fusion

[0134]

[0135] It can be seen from the experimental data that through the SPECT / CT image fusion method, the image quality and the visibility of abnormal features are significantly improved. The fusion strategy effectively enhances the functional performance of the abnormal region, reflecting the complexity and heterogeneity of the lesion, which cannot be presented by CT images alone.

[0136] Compared with traditional methods, existing technologies usually can only simply overlay images or align images manually, resulting in problems such as inaccurate fusion and unprominent details. The present invention conducts intelligent detection and precise registration through deep - learning algorithms, and combines contrast optimization strategies to accurately mark and enhance abnormal features. It not only overcomes the problem of information loss in traditional methods, but also significantly improves the clarity and diagnostic feasibility of images. Especially in the boundary recognition of complex lesions and the functional abnormal manifestations, the fused images provide a more detailed and intuitive performance, providing a more reliable diagnostic basis for clinical practice, and having obvious innovation and practical value.

[0137] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered by the scope of the claims of the present invention.

Claims

1. A SPECT / CT image fusion method based on deep learning, characterized in that, Including: Obtain images through SPECT technology and CT technology respectively; Construct a multi-modal deep learning feature extraction network to extract features of SPECT and CT images respectively; Generate a fusion strategy for the extracted features; Fuse SPECT and CT according to the fusion strategy; The fusion strategy includes Strategy 1 and Strategy 2; through the interactive selection of the front-end interface, the user can select the fusion strategy; to enhance the edge to highlight the abnormal area, then select Strategy 1; to highlight the abnormal contrast between function and structure, then select Strategy 2; Strategy 1: For the fusion result with the functional image as the main body, use the CT image to clarify the local contour of the SPECT image; Strategy 2: For the fusion result with the high-resolution anatomical structure image as the main body, use the SPECT image to mark and locally enhance the CT image; The specific content of Strategy 1 includes using an edge detection algorithm to extract the anatomical structure contours of CT images , copying the structural contours of the CT images, and according to the mapping relationship between SPECT and CT, mapping the extracted CT structural contours into the SPECT images to form corresponding contour markers ; Set the contour edge as the center of the pixel points, and expand the width by towards the inside and outside respectively to form the inner regions of the detection regions on both sides and ; Calculate the average radioactivity intensity of the inner region and the outer region respectively, denoted as and ; Calculate the radioactive manifestation difference degree D: Set the difference threshold , if , it is determined that the medial and lateral radioactive manifestations are similar and no processing is performed; if , the medial and lateral radioactive manifestations are quite different, and edge enhancement is performed; define the edge pixel intensity , and adjust the edge pixel value according to the medial-lateral difference: Among them, represents the enhanced edge pixel intensity; is the weighting coefficient, which is adjusted according to actual applications to control the enhancement intensity; The second strategy specifically includes obtaining an abnormal feature position map extracted from the SPECT image , and based on the positions of marked abnormal features and the mapping relationship between SPECT and CT, locating the abnormal position WZ1 in the CT image; Obtain the abnormal feature location map extracted from the CT image Mark the position WZ2 of the abnormal feature therein; Mark WZ1 and WZ2 in the CT image, and through the mapping relationship between SPECT and CT, extract all the marked positions in the CT image in the SPECT image, and copy the functional image at the extracted marked positions to the marked positions in the CT image to obtain the fusion result of Strategy 2.

2. The SPECT / CT image fusion method based on deep learning according to claim 1, wherein: Obtain functional image data through the SPECT technology; at the same time, obtain high-resolution anatomical structure image data through the CT technology.

3. The SPECT / CT image fusion method based on deep learning according to claim 2, characterized in that: Normalize the image data of SPECT and CT to unify the pixel value range; Match the positions of SPECT and CT images to establish a mapping relationship; for the same frame of image data, the mapping results of any two pixel points are different; The specific mapping relationship is: When obtaining SPECT image data, generate the mapping relationship of each pixel point in the three-dimensional entity: SP represents the set of pixel points of SPECT image data, represents the mapping relationship, and SW represents the set of positions of the three-dimensional entity; among them, the i-th pixel point in the set SP can be mapped to the position in the three-dimensional entity of SW; When obtaining CT image data, generate the mapping relationship of each pixel point in the three-dimensional entity: C represents the set of pixel points of CT image data; among them, the j-th pixel point in the set C can all be mapped to the position in the three-dimensional entity of SW; According to the positions in the three-dimensional entity, establish mapping relationships between the pixels of the two images respectively; Among them, the mapping relationship from SPECT to CT specifically includes: First, using the mapping relationship between SPECT image data and the three-dimensional entity, obtain the position corresponding to each pixel point, and label the obtained positions; during the mapping process of the CT image data and the three-dimensional entity, when the labeled positions appear, obtain the pixel points of the CT image data , and at the same time, when generating the labels, obtain the pixel points of the SPECT image data , establish and the mapping relationship; loop through all pixel points in sequence to complete the mapping from SPECT to CT; The mapping result is represented as: ; Among them, the mapping relationship from CT to SPECT is opposite to the mapping relationship from SPECT to CT; the mapping result is expressed as: .

4. The SPECT / CT image fusion method based on deep learning according to claim 3, wherein: The construction of the multi-modal deep learning feature extraction network includes a two-stream convolutional neural network, and the network contains two independent input streams, which process SPECT and CT image data respectively, and use a deep convolutional neural network for abnormal feature extraction and position detection; Learn the deep convolutional neural networks of the two independent channels through the training set and the validation set respectively, so that the neural network of each channel can identify the abnormal areas of the input images; The overall architecture of the deep convolutional neural network includes: Input layer: Input size I S = 128×128×1, single-channel image data; Convolutional layer 1: The convolutional kernel size is 3×3, the number is 32, the stride is 1, the activation function is ReLU, and the output size is 128×128×32; Pooling layer 1: Max pooling 2×2, stride 2, output size 64×64×32; Convolutional layer 2: The convolutional kernel size is 3×3, the number is 64, the stride is 1, the activation function is ReLU, and the output size is 64×64×64; Pooling layer 2: Max pooling 2×2, stride 2, output size 32×32×64; Anomaly detection layer: Use a 3×3 convolutional kernel to output a single-channel anomaly feature map with a size of 32×32×1, which is used to mark the positions of anomaly features in the SPECT image.

5. The SPECT / CT image fusion method based on deep learning according to claim 4, characterized in that: The features of separately extracting SPECT and CT images include detecting abnormal features of SPECT and CT image data respectively through two independent channels of the dual-stream convolutional neural network, and outputting two abnormal feature position maps and , which are the abnormal positions detected in the SPECT and CT images respectively; It is a map of the abnormal feature positions extracted from SPECT images, which is an image corresponding to the original image and contains the positions of the abnormal features marked in the SPECT images; It is a map of the abnormal feature positions extracted from CT images, which is an image corresponding to the original CT image and contains the positions of the abnormal anatomical features marked in the CT images; In the image, the positions of anomaly features will be displayed with different visual markings.

6. A deep learning-based SPECT / CT image fusion system adopting the method according to any one of claims 1-5, characterized in that: An acquisition unit, which acquires images through SPECT technology and CT technology respectively; An extraction unit, which constructs a multi-modal deep learning feature extraction network to extract the features of SPECT and CT images respectively; A fusion unit, which generates a fusion strategy for the extracted features; An execution unit, which fuses SPECT and CT according to the fusion strategy.

7. A computer device, comprising: A memory and a processor; The memory stores a computer program, characterized in that: when the processor executes the computer program, the steps of the method according to any one of claims 1-5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, the steps of the method according to any one of claims 1-5 are implemented.

Images