SPECT / CT image fusion method and system based on deep learning

Abstract

The invention discloses an SPECT / CT image fusion method and system based on deep learning, and the method comprises the steps: obtaining an image through an SPECT technology and a CT technology; constructing a multi-modal deep learning feature extraction network, and respectively extracting the features of the SPECT and CT images; generating a fusion strategy for the extracted features; and according to the fusion strategy, carrying out fusion on the SPECT and the CT. The multi-modal feature extraction network is constructed through deep learning, precise fusion of SPECT and CT images is achieved, functional information of SPECT and anatomical details of CT are fully reserved, the definition and contrast ratio of the images are improved, the detection capacity of abnormal areas is enhanced, the diagnosis accuracy is remarkably improved, and more comprehensive support is provided for clinical decision making.

Description

Technical Field

[0001] The present invention relates to the field of image fusion technology, and specifically to a SPECT / CT image fusion method and system based on deep learning. Background Art

[0002] In the field of medical imaging, single photon emission computed tomography (SPECT) and computed tomography (CT) are two important imaging technologies, each with its own unique advantages. SPECT can provide functional information in the body, such as metabolic activity, blood flow status, and organ function, and can display the distribution of radioactivity in a specific area by injecting radioactive tracers. It is widely used in functional detection of heart disease, tumors, neurological diseases, etc. However, the spatial resolution of SPECT images is low, and the image edges are blurred, which is not conducive to accurate anatomical positioning.

[0003] As a high-resolution structural imaging technology, CT can clearly display anatomical details such as bones, soft tissues and blood vessels. CT images are widely used in emergency, trauma, tumor detection and other fields due to their rapid scanning and clear details. However, CT can only provide anatomical information and lacks reflection of the functional status of tissues, which limits its application in evaluating metabolic and physiological changes.

[0004] In order to improve the accuracy of diagnosis, SPECT / CT fusion imaging technology has emerged in recent years. By combining the functional information of SPECT with the anatomical structure information of CT, the synchronous display of function and anatomy is achieved, providing a more comprehensive reference for clinical diagnosis and treatment planning. However, traditional fusion methods usually rely on simple image superposition or manual adjustment, which not only makes it difficult to ensure the precise alignment of the two images, but also the fusion effect is easily affected by artifacts and noise, resulting in unsatisfactory diagnostic results.

[0005] With the development of deep learning technology, researchers have begun to explore the use of advanced algorithms such as convolutional neural networks (CNN) to extract and fuse multimodal images to achieve more accurate image registration and fusion. Deep learning methods can automatically learn the complex feature relationships of images, are highly adaptable and robust, and provide a new way to improve the quality and efficiency of SPECT / CT image fusion. However, how to retain the key features of each image during the fusion process and maximize the diagnostic value is still a technical problem that needs to be solved urgently. Summary of the invention

[0006] In view of the above-mentioned problems, the present invention is proposed.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions: a SPECT / CT image fusion method based on deep learning, comprising: Images were acquired using SPECT and CT techniques respectively; Construct a multimodal deep learning feature extraction network to extract features of SPECT and CT images respectively; Generate fusion strategies based on the extracted features; According to the fusion strategy, SPECT and CT are fused.

[0008] As a preferred solution of the deep learning-based SPECT / CT image fusion method described in the present invention, functional image data is acquired through the SPECT technology; and high-resolution anatomical structure image data is acquired through the CT technology.

[0009] As a preferred solution of the deep learning-based SPECT / CT image fusion method of the present invention, the image data of SPECT and CT are normalized to unify the pixel value range; The positions of SPECT and CT images are matched to establish a mapping relationship; for the same frame of image data, the mapping results of any two pixels are different; The mapping relationship is specifically: When acquiring SPECT image data, the mapping relationship of each pixel in the three-dimensional entity is generated: SP represents the pixel set of SPECT image data. represents the mapping relationship, SW represents the position set of the three-dimensional entity; among them, the i-th pixel point in the set SP All can be mapped to the positions in the three-dimensional entity of SW; When acquiring CT image data, the mapping relationship of each pixel point in the three-dimensional entity is generated: C represents the pixel point set of CT image data; the jth pixel point in the set C is All can be mapped to the positions in the three-dimensional entity of SW; According to the positions in the three-dimensional entity, mapping relationships are established between the pixels of the two images; The mapping relationship from SPECT to CT specifically includes: first, using the mapping relationship between SPECT image data and three-dimensional entities, the position corresponding to each pixel point is obtained, and the obtained position label is labeled; in the mapping process between CT image data and three-dimensional entities, if a label position appears, the pixel point of the CT image data is obtained. , and at the same time obtain the pixel points of the SPECT image data when generating the label ,Establish and Mapping relationship; loop through all pixels in sequence to complete the mapping from SPECT to CT; The mapping result is expressed as: ; The mapping relationship from CT to SPECT is opposite to the mapping relationship from SPECT to CT; the mapping result is expressed as: .

[0010] As a preferred solution of the deep learning-based SPECT / CT image fusion method described in the present invention, wherein: the construction of a multimodal deep learning feature extraction network includes a two-stream convolutional neural network, the network includes two independent input streams, respectively processing SPECT and CT image data, and using a deep convolutional neural network to extract abnormal features and detect positions; The deep convolutional neural networks of two independent channels are learned through the training set and the validation set respectively, so that the neural network of each channel can identify the abnormal areas of the input image; The overall architecture of the deep convolutional neural network includes: Input layer: Input size I S = 128×128×1, single-channel image data; Convolutional layer 1: convolution kernel size 3×3, number 32, stride 1, activation function ReLU, output size 128×128×32; Pooling layer 1: max pooling 2×2, stride 2, output size 64×64×32; Convolutional layer 2: convolution kernel size 3×3, number 64, stride 1, activation function ReLU, output size 64×64×64; Pooling layer 2: max pooling 2×2, stride 2, output size 32×32×64; Anomaly detection layer: Use a 3×3 convolution kernel to output a single-channel abnormal feature map with a size of 32×32×1, which is used to mark the abnormal feature locations in the SPECT image.

[0011] As a preferred solution of the deep learning-based SPECT / CT image fusion method of the present invention, wherein: the extracting of the features of the SPECT and CT images respectively includes detecting abnormal features of the SPECT and CT image data respectively through two independent channels of the dual-stream convolutional neural network, and outputting two abnormal feature position maps and , are the abnormal locations detected in SPECT and CT images, respectively; It is an abnormal feature location map extracted from the SPECT image, which is an image corresponding to the original image and contains the locations of abnormal features in the SPECT image; It is an abnormal feature location map extracted from the CT image, which is an image corresponding to the original CT image and contains the locations of abnormal anatomical features in the CT image; In the image, the location of abnormal features is indicated by different visual markers.

[0012] As a preferred solution of the deep learning-based SPECT / CT image fusion method of the present invention, wherein: the fusion strategy includes that the user can select the fusion strategy through interactive selection on the front-end interface; Strategy 1: Taking the functional image as the main fusion result, the CT image is used to clarify the local contour of the SPECT image; Strategy 2: Based on the fusion results of high-resolution anatomical structure images, SPECT images are used to mark and locally enhance CT images.

[0013] As a preferred solution of the SPECT / CT image fusion method based on deep learning described in the present invention, wherein: the strategy 1 specifically includes using an edge detection algorithm to extract the anatomical structure contour of the CT image , copy the structural contour of the CT image, and extract the CT structural contour according to the mapping relationship between SPECT and CT. Mapped into SPECT images to form corresponding contour markers ; Set the contour edge The pixel point is the center, and the width is expanded to the inside and outside respectively. area, forming the inner area of ​​the detection area on both sides and ; Calculate the medial area and outer areas The average radioactivity intensity is denoted as and ; Calculate the radioactivity difference D: Set the difference threshold , like , then the radiological manifestations inside and outside are judged to be similar and no treatment is performed; if , then the difference in radioactivity between the inner and outer sides is large, so edge enhancement is performed; define the edge pixel intensity , adjust the edge pixel value according to the difference between the inside and outside: in, Represents the enhanced edge pixel intensity; It is the weighting factor, which is adjusted according to the actual application to control the enhancement strength; The second strategy specifically includes obtaining an abnormal feature location map extracted from the SPECT image. ,according to The position of abnormal features and the mapping relationship between SPECT and CT are marked in the image, and the abnormal position WZ1 is located in the CT image; Obtaining abnormal feature location maps extracted from CT images The position of the abnormal feature is marked in WZ2; WZ1 and WZ2 are marked in the CT image, and all the marked positions in the CT image are extracted in the SPECT image through the mapping relationship between SPECT and CT. The functional image extracted to the marked position is copied to the marked position in the CT image to obtain the fusion result of fusion strategy two.

[0014] A SPECT / CT image fusion system based on deep learning using the method according to any one of claims 1 to 7, wherein: The acquisition unit acquires images by using SPECT technology and CT technology respectively; The extraction unit constructs a multimodal deep learning feature extraction network to extract the features of SPECT and CT images respectively; A fusion unit generates a fusion strategy based on the extracted features; The execution unit fuses SPECT and CT according to the fusion strategy.

[0015] A computer device comprises: a memory and a processor; the memory stores a computer program, wherein: the processor implements the steps of any one of the methods of the present invention when executing the computer program.

[0016] A computer-readable storage medium stores a computer program, wherein: when the computer program is executed by a processor, the steps of any one of the methods of the present invention are implemented.

[0017] Beneficial effects of the present invention: The SPECT / CT image fusion method based on deep learning provided by the present invention constructs a multimodal feature extraction network through deep learning, realizes the precise fusion of SPECT and CT images, fully retains the functional information of SPECT and the anatomical details of CT, improves the clarity and contrast of the image, enhances the detection capability of abnormal areas, significantly improves the diagnostic accuracy, and provides more comprehensive support for clinical decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0019] Figure 1 An overall flow chart of a SPECT / CT image fusion method based on deep learning provided for the first embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0021] Example 1, reference Figure 1 , as an embodiment of the present invention, provides a SPECT / CT image fusion method based on deep learning, comprising: S1: Obtain images using SPECT and CT technology respectively.

[0022] Functional imaging data is acquired through the SPECT technology; high-resolution anatomical structure imaging data is acquired through the CT technology. By using SPECT and CT technologies to collect data simultaneously, the same spatial position of the two images is ensured when they are taken, which means that the two devices work together and share the same scanning bed or device, thereby ensuring that the data sources of the SPECT and CT images are completely consistent. Since the two images are acquired at the same position and time, the same anatomical position is ensured to correspond in the SPECT and CT images, thereby improving the accuracy and alignment of the two images when they are fused, and avoiding deviations caused by different spatial positions and the impact on diagnosis.

[0023] Furthermore, the image data of SPECT and CT are normalized to unify the pixel value range; the positions of SPECT and CT images are matched to establish a mapping relationship; it is ensured that the SPECT and CT images are strictly spatially corresponding, and the position of each pixel in the two images is mapped to the same three-dimensional entity position, thereby ensuring the accurate superposition of functional information and anatomical information. For the same frame of image data, the mapping results of any two pixels are different, making the mapping relationship of each pixel unique, ensuring that the mapping results of any two pixels in the same frame of image data are different, avoiding overlapping or repeated mapping, and ensuring the accuracy of registration.

[0024] It is important to say that the pixels of SPECT and CT images are mapped to the corresponding positions of three-dimensional entities to ensure that the two types of image data are strictly corresponding. Using the three-dimensional spatial information obtained during the image shooting process, the position of each pixel of SPECT and CT can be accurately calibrated to ensure the scientific nature of the mapping relationship. By associating SPECT and CT images with the positions of three-dimensional entities respectively, the corresponding pixels of functional images (SPECT) and anatomical images (CT) are accurately aligned, so that the respective image features can be accurately matched during the fusion process, realizing the efficient combination of functional and anatomical information.

[0025] The mapping relationship is specifically: When acquiring SPECT image data, the mapping relationship of each pixel in the three-dimensional entity is generated: Generating the mapping relationship of each pixel point of SPECT image data in a three-dimensional entity is a key step in achieving accurate image alignment and fusion. This process mainly involves image registration, spatial calibration and coordinate transformation technology to ensure that each SPECT pixel point can be accurately mapped to the actual three-dimensional position.

[0026] SP represents the pixel set of SPECT image data. represents the mapping relationship, SW represents the position set of the three-dimensional entity; among them, the i-th pixel point in the set SP All of them can be mapped to the position of the 3D entity of SW; in the mapping of SPECT and 3D entity, the small area (voxel) corresponding to the pixel is a reasonable expression of its actual spatial position, which fully considers the imaging resolution and the diffusion of radioactive signals. This design meets the actual needs of image fusion, ensures the accurate reflection of functional information, and provides more accurate image fusion effect in clinical applications.

[0027] When acquiring CT image data, the mapping relationship of each pixel point in the three-dimensional entity is generated: The mapping relationship between each pixel of CT image data and the three-dimensional entity is generated to ensure that the anatomical information in the CT image can accurately correspond to the actual three-dimensional space position. This process involves steps such as coordinate transformation, space reconstruction and equipment calibration to achieve accurate registration of CT images with actual physical space.

[0028] C represents the pixel point set of CT image data; the jth pixel point in the set C is All can be mapped to positions in the three-dimensional entity of SW.

[0029] According to the position in the three-dimensional entity, a mapping relationship is established between the pixels of the two images. Among them, the mapping relationship from SPECT to CT specifically includes, first, using the mapping relationship between SPECT image data and the three-dimensional entity, obtaining the position corresponding to each pixel point, and labeling the obtained position; in the mapping process between the CT image data and the three-dimensional entity, if a label position appears, the pixel point of the CT image data is obtained. , and at the same time obtain the pixel points of the SPECT image data when generating the label ,Establish and The mapping relationship of SPECT to CT is completed by looping through all pixels in sequence.

[0030] The mapping result is expressed as: ; Wherein, the mapping relationship from CT to SPECT is opposite to the mapping relationship from SPECT to CT; the mapping result is expressed as: .

[0031] It should be noted that by marking and mapping the positions in the three-dimensional entity, it is ensured that every pixel of the SPECT and CT images can be accurately aligned. This design overcomes the alignment difficulties caused by the different resolutions and imaging principles of the two images, ensuring the accurate matching of functional information and anatomical information during the fusion process. SPECT provides functional information, while CT provides anatomical structure. By establishing this two-way mapping relationship, the precise fusion of the two images in space can be achieved, so that the functional abnormality position of SPECT can be displayed in the CT image, and the anatomical details of CT can be clarified in the SPECT image, forming a more intuitive and more diagnostically valuable fusion image.

[0032] Traditional fusion methods are prone to information misalignment or overlap, leading to confusion between anatomical and functional information. Through a clear mapping relationship, each functional information and anatomical information has a unique registration object, thus ensuring the accuracy of information after image fusion.

[0033] S2: Construct a multimodal deep learning feature extraction network to extract the features of SPECT and CT images respectively.

[0034] Furthermore, the construction of the multimodal deep learning feature extraction network includes a two-stream convolutional neural network, which contains two independent input streams, respectively processing SPECT and CT image data, and using a deep convolutional neural network to extract abnormal features and detect positions.

[0035] The deep convolutional neural networks of two independent channels are learned through training sets and validation sets respectively, so that the neural network of each channel can identify abnormal areas of the input image.

[0036] The overall architecture of the deep convolutional neural network includes: Input layer: Input size I S = 128×128×1, single-channel image data.

[0037] Convolutional layer 1: convolution kernel size 3×3, number 32, stride 1, activation function ReLU, output size 128×128×32.

[0038] Pooling layer 1: max pooling 2×2, stride 2, output size 64×64×32.

[0039] Convolutional layer 2: convolution kernel size 3×3, number 64, stride 1, activation function ReLU, output size 64×64×64.

[0040] Pooling layer 2: max pooling 2×2, stride 2, output size 32×32×64.

[0041] Anomaly detection layer: Use a 3×3 convolution kernel to output a single-channel abnormal feature map with a size of 32×32×1, which is used to mark the abnormal feature locations in the SPECT image.

[0042] Through the two independent channels of the dual-stream convolutional neural network, the abnormal features of SPECT and CT image data are detected respectively, and two abnormal feature location maps are output. and , are the abnormal locations detected in SPECT and CT images, respectively.

[0043] It is an abnormal feature location map extracted from the SPECT image, which is an image corresponding to the original image and contains the locations of abnormal features in the SPECT image; It is an abnormal feature location map extracted from a CT image. It is an image corresponding to the original CT image and contains the locations of abnormal anatomical features in the CT image. In the image, the locations of abnormal features are displayed with different visual markers.

[0044] It is important to know that SPECT images provide functional information and can show abnormalities in physiological activities such as metabolism and blood flow in the body; CT images provide high-resolution anatomical information and can accurately show the details and abnormalities of anatomical structures. By detecting the abnormal features in the two images separately, functional lesions and anatomical abnormalities can be clearly identified, providing more intuitive support for clinical diagnosis. The independent channels of the two-stream convolutional neural network are used to process SPECT and CT data separately, and the characteristics of each image are optimized. The SPECT stream focuses on detecting abnormal areas of metabolism or radioactive distribution, and the CT stream focuses on detecting anatomical abnormalities (such as masses, fractures, etc.). This independent processing strategy enables the network to focus more on the extraction of specific features and improve the accuracy of abnormality detection. The output abnormal feature location map marks the location of the abnormal area in the image, providing a clear reference for subsequent image fusion. These location maps can not only be used to assist doctors in manual interpretation, but can also be further used as input for automated analysis and diagnostic models.

[0045] S3: Generate a fusion strategy for the extracted features; and fuse SPECT and CT according to the fusion strategy.

[0046] Through interactive selection on the front-end interface, users can select fusion strategies. Strategy 1: Fusion results based on functional images, using CT images to clarify the local contours of SPECT images. Strategy 2: Fusion results based on high-resolution anatomical images, using SPECT images to mark and locally enhance CT images.

[0047] Furthermore, the strategy 1 specifically includes extracting the anatomical structure contour of the CT image using an edge detection algorithm , copy the structural contour of the CT image, and extract the CT structural contour according to the mapping relationship between SPECT and CT. Mapped into SPECT images to form corresponding contour markers .

[0048] Set the contour edge The pixel point is the center, and the width is expanded to the inside and outside respectively. area, forming the inner area of ​​the detection area on both sides and .

[0049] Calculate the medial area and outer areas The average radioactivity intensity is denoted as and .

[0050] Calculate the radioactivity difference D: Set the difference threshold , like , then the radiological manifestations inside and outside are judged to be similar and no treatment is performed; if , then the difference in radioactivity between the inner and outer sides is large, so edge enhancement is performed; define the edge pixel intensity , adjust the edge pixel value according to the difference between the inside and outside: in, Represents the enhanced edge pixel intensity; It is the weighting factor, which can be adjusted according to the actual application to control the enhancement strength.

[0051] It is important to note that by extracting the contours of anatomical structures in CT images through edge detection algorithms and accurately mapping these contours to SPECT images, functional images have clear structural markers. This marker makes the originally blurred boundaries in SPECT images clearer, providing a more intuitive structural information reference for clinical diagnosis. The enhancement of structural contours enables doctors to more clearly observe the correspondence between functional abnormalities and anatomical structures. For example, in tumor diagnosis, CT contour markers can help accurately identify the edges of lesions, making the positioning of functional abnormalities in SPECT more reliable.

[0052] By calculating the average radioactivity intensity of the inner and outer areas of the contour, the difference in the performance of functional information at the edge of the contour is determined. When the difference is large, the edge of the contour is enhanced to highlight the radioactivity characteristics at the edge. This enhancement strategy helps to improve the contrast of the image, making the functional abnormal area more obvious at the edge of the contour, and improving the visibility and recognition rate of the abnormal area.

[0053] Set a difference threshold to ensure that enhancement is performed only when there is a significant difference in the radiological manifestations between the inside and outside. Avoid unnecessary processing for areas with similar radiological manifestations, which can not only reduce artifacts and noise, but also maintain the original authenticity of the image and avoid interference with diagnosis caused by artificial enhancement. This design focuses on processing the contour edge area, which can effectively improve the visibility of local details. Especially in cases where functional abnormalities are unevenly distributed or the boundaries are unclear, edge enhancement can help highlight important diagnostic features and provide clearer abnormality positioning. Strengthening edge areas with large radiological differences makes the edge features of functional abnormalities more prominent, which helps to refine the manifestation of lesions, help doctors more accurately identify lesion boundaries, and improve diagnostic accuracy.

[0054] The second strategy specifically includes obtaining an abnormal feature location map extracted from the SPECT image. ,according to Mark the position of abnormal features and the mapping relationship between SPECT and CT in the image, and locate the abnormal position WZ1 in the CT image. Obtain the abnormal feature position map extracted from the CT image The location of the abnormal feature is marked in WZ2.

[0055] WZ1 and WZ2 are marked in the CT image, and all the marked positions in the CT image are extracted in the SPECT image through the mapping relationship between SPECT and CT. The functional image extracted to the marked position is copied to the marked position in the CT image to obtain the fusion result of fusion strategy two.

[0056] It should be said that the specific locations of functional abnormalities (such as metabolic abnormalities) and anatomical abnormalities (such as masses and lesions) are marked respectively through the abnormal feature position maps extracted from SPECT and CT images. The abnormal position WZ1 of SPECT and the abnormal position WZ2 of CT are mapped to each other, and the mapping relationship is precisely aligned to facilitate the observation and analysis of the correspondence between functional abnormalities and anatomical abnormalities. By copying the information of the functional abnormality area in the SPECT image to the marked position in the CT image, combined with the anatomical structure of CT, the functional abnormality area is more intuitive and prominent in the CT image. This design helps to better combine functional information and anatomical information, helping doctors to clarify the nature of the lesion. This design enables functional abnormalities and anatomical abnormalities to be presented simultaneously in the same image, especially in the diagnosis of complex lesions. The fused image can more clearly show the morphology and functional performance of the abnormal area, providing an important reference for clinical diagnosis.

[0057] Furthermore, when the functional image extracted to the marked position is copied to the marked position in the CT image, a data tag is inserted to record the following contents: the current intensity of the abnormal area, the average intensity inside and outside the abnormal area, and the standard deviation and range of the reaction intensity inside and outside the area. The recording and analysis of the average intensity, standard deviation, range and current intensity inside and outside the area can provide rich and detailed functional performance information for the clinic, making the nature and activity status of the lesion more transparent. Through the fine replication and analysis of the functional image at the marked position of the CT, the performance of the fused image in terms of functional performance and anatomical structure is more intuitive and clear, providing more lesion details and judgment basis for the clinic, and enhancing the clinical application value of the fused image.

[0058] On the other hand, this embodiment also provides a SPECT / CT image fusion system based on deep learning, which includes: The acquisition unit acquires images through SPECT technology and CT technology respectively.

[0059] Extraction unit, builds a multimodal deep learning feature extraction network to extract the features of SPECT and CT images respectively.

[0060] The fusion unit generates a fusion strategy based on the extracted features.

[0061] The execution unit fuses SPECT and CT according to the fusion strategy.

[0062] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0063] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0064] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0065] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0066] Example 2 is an embodiment of the present invention, which provides a SPECT / CT image fusion method based on deep learning. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0067] The trial subjects included 20 patients with known functional and anatomical abnormalities of varying degrees, including 10 cancer patients and 10 cardiovascular disease patients. All patients underwent SPECT and CT imaging under the same conditions in a standardized imaging diagnostic room, with the latest dual-modality SPECT / CT system equipped with an optimized deep learning feature extraction network.

[0068] Test preparation: Patient preparation: All patients fasted for 6 hours in advance to reduce gastrointestinal disturbances, and used standardized radiotracer injection to ensure accurate functional reflection of SPECT images.

[0069] Equipment calibration: Calibrate the SPECT / CT system to ensure that SPECT and CT images can be accurately acquired in the same coordinate system.

[0070] Data Acquisition: Functional images are first acquired using SPECT technology, followed immediately by a CT scan to obtain high-resolution anatomical images.

[0071] Implementation process: Multimodal feature extraction: The constructed two-stream convolutional neural network is used to process SPECT and CT images respectively to extract the abnormal feature location of each image. The neural network detects the functional and anatomical abnormalities of the two images through independent channels and outputs two abnormal feature location maps.

[0072] Image registration and mapping: After normalizing the data of SPECT and CT images and unifying the pixel value range, the system's built-in algorithm is used to map the functional abnormality of SPECT to the anatomical structure of CT, marking the abnormal areas WZ1 and WZ2. All marked positions are corrected by the automatic image registration system to ensure that the pixels of the two images correspond accurately.

[0073] Fusion strategy application: Strategy 1: Extract the edge of the CT image, use the edge detection algorithm to copy and map the anatomical structure contour of the CT image to the SPECT image, and generate accurate contour markings. Calculate the difference in radiological performance on both sides of the contour edge. If the difference is significant, enhance the edge to highlight the abnormal area.

[0074] Strategy 2: Compare the abnormal feature location map in the SPECT image with the abnormal location map in the CT image, copy the functional abnormal area to the corresponding marked position of the CT image, and synthesize a fused image to highlight the abnormal contrast between function and structure.

[0075] Data Recording and Analysis: After the fusion image is generated, the system automatically records the radioactivity intensity of each marked position, including the average intensity, standard deviation, range, and current intensity inside and outside the area. Each patient's fusion image is blindly evaluated by a radiologist, and the scoring indicators include visibility of abnormal areas, diagnostic clarity, and image quality.

[0076] Table 1: SPECT / CT image fusion experimental data It can be seen from the experimental data that the image quality and visibility of abnormal features are significantly improved through the SPECT / CT image fusion method. The fusion strategy effectively enhances the functional performance of the abnormal area, reflecting the complexity and heterogeneity of the lesions, which cannot be presented by CT images alone.

[0077] Compared with traditional methods, the existing technology can usually only simply superimpose images or manually adjust and align images, which has the problems of inaccurate fusion and unclear details. The present invention uses a deep learning algorithm for intelligent detection and precise registration, and combines contrast optimization strategies to accurately mark and enhance abnormal features, which not only overcomes the problem of information loss in traditional methods, but also significantly improves the clarity and diagnostic feasibility of images. Especially in terms of boundary recognition and functional abnormality of complex lesions, fused images provide more detailed and intuitive performance, providing a more reliable diagnostic basis for clinical practice, and have obvious innovation and practical value.

[0078] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A SPECT / CT image fusion method based on deep learning, characterized in that: include: Images were acquired using SPECT and CT techniques respectively; Construct a multimodal deep learning feature extraction network to extract features of SPECT and CT images respectively; Generate fusion strategies based on the extracted features; According to the fusion strategy, SPECT and CT are fused.

2. The SPECT / CT image fusion method based on deep learning according to claim 1, characterized in that: Functional imaging data is acquired through the SPECT technology; high-resolution anatomical imaging data is acquired through the CT technology.

3. The SPECT / CT image fusion method based on deep learning according to claim 2, characterized in that: Normalize the SPECT and CT image data to unify the pixel value range; The positions of SPECT and CT images are matched to establish a mapping relationship; for the same frame of image data, the mapping results of any two pixels are different; The mapping relationship is specifically: When acquiring SPECT image data, the mapping relationship of each pixel in the three-dimensional entity is generated: SP represents the pixel set of SPECT image data. represents the mapping relationship, SW represents the position set of the three-dimensional entity; among them, the i-th pixel point in the set SP All can be mapped to the positions in the three-dimensional entity of SW; When acquiring CT image data, the mapping relationship of each pixel point in the three-dimensional entity is generated: C represents the pixel point set of CT image data; the jth pixel point in the set C is All can be mapped to the positions in the three-dimensional entity of SW; According to the positions in the three-dimensional entity, mapping relationships are established between the pixels of the two images; The mapping relationship from SPECT to CT specifically includes: first, using the mapping relationship between SPECT image data and three-dimensional entities, the position corresponding to each pixel point is obtained, and the obtained position label is labeled; in the mapping process between CT image data and three-dimensional entities, if a label position appears, the pixel point of the CT image data is obtained. , and at the same time obtain the pixel points of the SPECT image data when generating the label ,Establish and Mapping relationship; loop through all pixels in sequence to complete the mapping from SPECT to CT; The mapping result is expressed as: ; The mapping relationship from CT to SPECT is opposite to the mapping relationship from SPECT to CT; the mapping result is expressed as: .

4. The SPECT / CT image fusion method based on deep learning as claimed in claim 3, characterized in that: The multimodal deep learning feature extraction network is constructed, comprising a two-stream convolutional neural network, the network comprising two independent input streams, processing SPECT and CT image data respectively, and using a deep convolutional neural network to extract abnormal features and detect positions; The deep convolutional neural networks of two independent channels are learned through the training set and the validation set respectively, so that the neural network of each channel can identify the abnormal areas of the input image; The overall architecture of the deep convolutional neural network includes: Input layer: input size I S = 128×128×1, single-channel image data; Convolutional layer 1: convolution kernel size 3×3, number 32, stride 1, activation function ReLU, output size 128×128×32; Pooling layer 1: max pooling 2×2, stride 2, output size 64×64×32; Convolutional layer 2: convolution kernel size 3×3, number 64, stride 1, activation function ReLU, output size 64×64×64; Pooling layer 2: max pooling 2×2, stride 2, output size 32×32×64; Anomaly detection layer: Use a 3×3 convolution kernel to output a single-channel abnormal feature map with a size of 32×32×1, which is used to mark the abnormal feature locations in the SPECT image.

5. The SPECT / CT image fusion method based on deep learning according to claim 4, characterized in that: The extracting of features of the SPECT and CT images respectively includes detecting abnormal features of the SPECT and CT image data respectively through two independent channels of the dual-stream convolutional neural network, and outputting two abnormal feature position maps. and , are the abnormal locations detected in SPECT and CT images, respectively; It is an abnormal feature location map extracted from the SPECT image, which is an image corresponding to the original image and contains the locations of abnormal features in the SPECT image; It is an abnormal feature location map extracted from the CT image, which is an image corresponding to the original CT image and contains the locations of abnormal anatomical features in the CT image; In the image, the location of abnormal features is indicated by different visual markers.

6. The SPECT / CT image fusion method based on deep learning according to claim 5, characterized in that: The fusion strategy includes that the user can select the fusion strategy through interactive selection on the front-end interface; Strategy 1: Taking the functional image as the main fusion result, the CT image is used to clarify the local contour of the SPECT image; Strategy 2: Based on the fusion results of high-resolution anatomical structure images, SPECT images are used to mark and locally enhance CT images.

7. The SPECT / CT image fusion method based on deep learning according to claim 6, characterized in that: The first strategy specifically includes extracting the anatomical structure contour of the CT image using an edge detection algorithm , copy the structural contour of the CT image, and extract the CT structural contour according to the mapping relationship between SPECT and CT. Mapped into SPECT images to form corresponding contour markers ; Set the contour edge The pixel point is the center, and the width is expanded to the inside and outside respectively. area, forming the inner area of ​​the detection area on both sides and ; Calculate the medial area and outer areas The average radioactivity intensity is denoted as and ; Calculate the radioactivity difference D: Set the difference threshold , like , then the radiological manifestations inside and outside are judged to be similar and no treatment is performed; if , then the difference in radioactivity between the inner and outer sides is large, so edge enhancement is performed; define the edge pixel intensity , adjust the edge pixel value according to the difference between the inside and outside: in, Represents the enhanced edge pixel intensity; It is the weighting factor, which is adjusted according to the actual application to control the enhancement strength; The second strategy specifically includes obtaining an abnormal feature location map extracted from the SPECT image. ,according to The position of abnormal features and the mapping relationship between SPECT and CT are marked in the image, and the abnormal position WZ1 is located in the CT image; Obtaining abnormal feature location maps extracted from CT images The position of the abnormal feature is marked in WZ2; WZ1 and WZ2 are marked in the CT image, and all the marked positions in the CT image are extracted in the SPECT image through the mapping relationship between SPECT and CT. The functional image extracted to the marked position is copied to the marked position in the CT image to obtain the fusion result of fusion strategy two.

8. A SPECT / CT image fusion system based on deep learning using the method according to any one of claims 1 to 7, characterized in that: The acquisition unit acquires images by using SPECT technology and CT technology respectively; The extraction unit constructs a multimodal deep learning feature extraction network to extract the features of SPECT and CT images respectively; A fusion unit generates a fusion strategy based on the extracted features; The execution unit fuses the SPECT and CT according to the fusion strategy.

9. A computer device comprising: Memory and processor; The memory stores a computer program, characterized in that when the processor executes the computer program, the steps of the SPECT / CT image fusion method based on deep learning are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the SPECT / CT image fusion method based on deep learning are implemented.

Images