A PET image lesion detection device based on photoelectric structure

Abstract

The present invention discloses a PET image lesion detection device based on an optoelectronic structure. The device comprises a feature extraction component, an FPGA, a detection component, and a preprocessing module. The feature extraction component receives the PET image as input, converts the optical signal into an electrical signal through optical convolution in the feature extraction component, and transmits the signal to the FPGA. The FPGA deploys a convolutional neural network to predict a CT image, which is then output to the detection component. The detection component receives the predicted CT image output by the FPGA, combines it with a lesion location reference image input by the preprocessing module, and, after optical transformation, outputs a detection result image. The present invention first introduces an optoelectronic structure to improve the efficiency and accuracy of image processing. Second, a CNN is deployed on the FPGA to rapidly generate CT images. Finally, optical circuits are used for lesion detection, improving disease detection capabilities and significantly accelerating the process.

Description

Technical Field

[0001] The present invention belongs to the field of photoelectric imaging and computer technology, and in particular relates to a PET image lesion detection device based on a photoelectric structure. Background Art

[0002] PET (Positron Emission Computed Tomography) is a functional imaging technology that uses the injection of FDG (Fludeoxyglucose). It has the characteristics of low resolution and easy observation of lesions, so it is commonly used in the detection of tumor lesions. However, since it cannot accurately locate the tumor, it needs to be combined with CT (Computed Tomography) to achieve precise positioning. Therefore, tumor patients often require simultaneous imaging of CT and PET, which will cause the patient to receive a large radiation dose. PET can assist in observing the condition of the lesion, but CT images are more used for precise positioning. Since PET and CT images need to be observed simultaneously to achieve lesion detection, it also increases the workload of medical staff. Therefore, a fast and automated lesion detection method is also one of the means to improve the efficiency of medical staff.

[0003] Existing lesion detection technologies include AI-based image analysis systems and methods for PET / SPECT (Single-Photon Emission Computed Tomography) lesion detection and characterization. These use machine learning to detect hotspots of potentially cancerous lesions in subjects, determine lesion locations, and segment the lesion area. They also include systems for assisting in the diagnosis of primary lesions and metastases. These primarily employ deep learning methods to fuse PET and CT images, segment lesions, and construct 3D (three-dimensional) models of the lesion area to identify primary lesions and metastases. Currently, common automated lesion detection technologies are based on machine learning and deep learning strategies. First, they require a large amount of training data, which limits algorithm performance. These data volumes may not be sufficient, resulting in a low detection success rate. Second, they require high computing power, resulting in lengthy training times and complex parameter adjustments. Finally, given the medical nature of lesion detection, these two training models are typically black-box models, making the training mechanisms and decision-making processes difficult to explain, leaving medical professionals and patients with a sense of unreliability. Summary of the Invention

[0004] The present invention proposes a PET image lesion detection device based on an optoelectronic structure. On the one hand, the optoelectronic structure is used to extract features from images, achieving high processing speed and rapidly acquiring key information from the images, preparing for subsequent input into an FPGA (Field Programmable Gate Array). On the other hand, PET-CT pre-training is used to generate pseudo-CT images from PET images, thereby reducing the number of CT scans required for patients. Lesions are then detected using a joint Fourier transform correlator (JTC), which enables real-time processing and immediate lesion detection with high sensitivity and specificity.

[0005] Noun Description:

[0006] PET stands for Positron Emission Tomography

[0007] FDG stands for fluorodeoxyglucose

[0008] CT stands for computed tomography

[0009] SPECT stands for single photon emission computed tomography

[0010] 2D means two-dimensional

[0011] 3D means three-dimensional

[0012] FPGA stands for Field Programmable Gate Array

[0013] JTC stands for Joint Transform Correlator

[0014] BSO crystal is bismuth silicate crystal

[0015] CCD is a charge coupled device

[0016] CNN stands for Convolutional Neural Network

[0017] SLM stands for spatial light modulator

[0018] PyTorch is a framework for building deep learning models.

[0019] To achieve the above-mentioned purpose, the present invention provides a PET image lesion detection device based on a photoelectric structure, comprising a feature extraction component, an FPGA, a detection component and a preprocessing module, wherein:

[0020] The input of the feature extraction component is the PET image. After optical convolution of the feature extraction component, the optical signal is converted into an electrical signal and transmitted to the FPGA.

[0021] The FPGA deploys a convolutional neural network to predict the CT image and output it to the detection component;

[0022] The detection component receives the predicted CT image output by the FPGA, combines it with the lesion position reference image input by the preprocessing module, and outputs the detection result image after optical transformation.

[0023] Preferably, the feature extraction component includes a first Fourier lens, a grating, a phase modulator and a photosensitive detector, all of which are at the same horizontal position to form an optical path, f is the focal length of the first Fourier lens, the distance between the PET image input to the first Fourier lens and the first Fourier lens is f, the distance between the first Fourier lens and the grating is f, the phase modulator is placed behind the grating, the distance between the phase modulator and the photosensitive detector is f, the photosensitive detector is placed at the end of the optical path, receives and converts the optical signal into an electrical signal, and transmits it to the FPGA for processing.

[0024] Preferably, the phase modulator changes the phase distribution of light and performs phase adjustment on the optical convolution kernel, specifically including:

[0025] Determine the convolution kernel function, and assume that the convolution kernel function is:

[0026]

[0027]

[0028] Or use express;

[0029] Among them, P represents the number of neighborhood pixels, which is 8 here; R represents the neighborhood radius, g i represents the gray value of the i-th neighborhood pixel, g c Represents the grayscale value of the center pixel,

[0030] Set the phase modulation mode and encode it into the phase modulator. Assume that the weight range of the convolution kernel is [w_min, w_max] and the range of the phase modulation mode is [phase_min, phase_max].

[0031] phase=(weight-w_min)×(phase_max-phase_min) / (w_max-w_min)+phase_min;

[0032] Among them, phase represents the phase value in the phase modulation mode, and weight represents the weight value in the convolution kernel;

[0033] The input signal is phase modulated, and the input signal that needs to be convolved is passed through the phase modulator to interfere with the phase modulation pattern to achieve the convolution effect.

[0034] Preferably, the photosensitive detector converts the signal input from the phase modulator into an electrical signal and records it. The output of the photosensitive detector is the light intensity distribution after optical convolution.

[0035] Preferably, the FPGA deployment of the convolutional neural network includes defining and training the model using the deep learning framework PyTorch; converting the CNN model into a deployable form, including conversion of weight parameters and network topology; mapping the CNN model into the FPGA; passing the results of the feature extraction component to the FPGA, using the CNN model to perform image prediction on the FPGA, and outputting a predicted CT image.

[0036] Preferably, the detection component includes a laser light source, a beam expander, a collimator, a compensator, a second Fourier lens, a spatial light modulator, a third Fourier lens, a holographic plate and a CCD. All nine are at the same horizontal position to form an optical path. f is the focal length of the second Fourier lens and also the focal length of the third Fourier lens. The second Fourier lens, the spatial light modulator, the third Fourier lens and the holographic plate are arranged in sequence, and the front and back intervals are all f; the beam expander is located at the front focal plane position of the collimator, and the compensator is located at the rear focal plane position of the collimator.

[0037] Preferably, the compensating element is a conjugate mirror or a four-wave mixing optical path.

[0038] Preferably, when the compensating element is a conjugate mirror, assuming that the conjugate mirror consists of two surfaces, the medium refractive index is n, the incident angle is θ1, and the exit angle is θ2, ideally, it can be obtained:

[0039]

[0040] The compensation amount is C = -2arcsin(n×sin(θ2)).

[0041] Preferably, when the compensating element is a four-wave mixing optical path, the four-wave mixing optical path specifically includes a beam splitter BS1, a beam splitter BS2, a beam splitter BS3, a reflector M1, a reflector M2, a reflector M3, a reflector M4, a reflector M5, a BSO crystal, a quarter glass slide, a second Fourier lens and a spatial light modulator; it is divided into three paths, the first path: a reference light is formed by the beam splitter BS1, the reflector M1 and the BSO crystal; the second path: a reference light is formed by the beam splitter BS2, mirror M2, and beam splitter BS3 are then irradiated onto the image group in the spatial light modulator and then imaged onto the BSO crystal through the second Fourier lens to form the object light; the third path: through the mirror M3, mirror M4, mirror M5, 1 / 4 glass slide, and BSO crystal to form the readout light; the object light and the reference light are coherently superimposed to form a phase grating, and the readout light diffracts on the phase grating to form a phase conjugate beam of the object light, achieving the compensation effect; assuming the current phase distortion is Imaging onto a spatial light modulator reconstructs the light wave into The light wave passing through the spatial light modulator is O'(x,y)O(x,y)exp[iθ(x,y)]; where O(x,y) is the light wave of the measured object, O'(x,y) is the light wave reflected by the BSO crystal, and i is an imaginary number. is the inverse transformation of the light wave phase, and exp[iθ(x,y)] is the phase term provided by the crystal.

[0042] Preferably, if the CCD detects the presence of an optical correlation peak exceeding a preset value, it indicates that the matching is successful, and the area where the CCD outputs the correlation spot is bright is the location of the lesion.

[0043] Compared with the prior art, the present invention has at least the following beneficial effects:

[0044] (1) Using optoelectronic structures to extract features from images, the processing speed is fast, and key information in the image can be quickly obtained to prepare for subsequent entry into the FPGA (Field Programmable Gate Array);

[0045] (2) Tumor patients often require simultaneous CT and PET imaging, which results in a high radiation dose for the patient. PET can assist in observing the condition of the lesion, but CT images are more used for precise positioning. If there is a way to appropriately reduce the number of CT operations, it will effectively reduce the radiation dose received by the patient. In the present invention, FPGA deploys CNN (Convolutional Neural Networks), uses feature extraction, and the CNN system outputs pseudo-CT images, which can generate pseudo-CT images from PET images to reduce the number of CT imaging times for patients;

[0046] (3) Use JTC (Joint Fourier Transform Correlator) to detect lesions. JTC can realize real-time processing, detect lesions immediately and has high sensitivity and specificity. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to make the purpose, technical solutions and beneficial effects of the present invention more clear, the present invention provides the following drawings for illustration:

[0048] Figure 1 This is a structural block diagram of a PET image lesion detection device based on a photoelectric structure according to an embodiment of the present invention;

[0049] Figure 2 This is a structural diagram of a feature extraction component of a PET image lesion detection device based on a photoelectric structure according to an embodiment of the present invention;

[0050] Figure 3 This is a structural diagram of a detection component of a PET image lesion detection device based on a photoelectric structure according to an embodiment of the present invention;

[0051] Figure 4 This is a schematic diagram of the input PET image, the CT image generated by PET prediction, and the lesion location display of the PET image lesion detection device based on the optoelectronic structure according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0053] See also Figure 1 The present invention provides a PET image lesion detection device based on a photoelectric structure, comprising a feature extraction component 10, an FPGA 20, a detection component 30 and a pre-processing module 42, wherein:

[0054] The input of the feature extraction component 10 is the PET image ( Figure 1 The input PET image is displayed on the screen 41, and after optical convolution of the feature extraction component 10, the optical signal is converted into an electrical signal and transmitted to the FPGA 20;

[0055] FPGA 20 deploys a convolutional neural network to predict the CT image and outputs it to the detection component 30;

[0056] The detection component 30 receives the image group 60, which includes the predicted CT image output by the FPGA 20 and the lesion position reference image output by the pre-processing module 42, and outputs the detection result image after optical transformation.

[0057] See also Figure 2The feature extraction component 10 includes a first Fourier lens 11, a grating 12, a phase modulator 13 and a photosensitive detector 14. The four are all at the same horizontal position to form an optical path. f is the focal length of the first Fourier lens 11. The distance between the PET image input to the first Fourier lens 11 and the first Fourier lens 11 is f, and the distance between the first Fourier lens 11 and the grating 12 is f. The phase modulator 13 is placed behind the grating 12. The distance between the phase modulator 13 and the photosensitive detector 14 is f. The photosensitive detector 14 is placed at the end of the optical path, receives and converts the optical signal into an electrical signal, and transmits it to the FPGA 20 for processing.

[0058] The optical convolution method is used to achieve the purpose of feature extraction. The screen 41 provides input to the first Fourier lens 11, and the first Fourier lens 11 performs Fourier transform to convert the spatial frequency distribution in the light field into the corresponding focal plane spatial frequency distribution. The grating 12 plays a role of spatial filtering on the focal plane due to diffraction, realizing optical convolution (the period and direction of the grating 12 can select a specific spatial frequency to pass through the light field, and diffract and eliminate components of other frequencies, realizing selective filtering of specific frequency components and achieving the effect of optical convolution). The filtered light wave enters the phase modulator 13, which changes the phase distribution of the light and performs phase adjustment on the optical convolution kernel, specifically including:

[0059] Determine the convolution kernel function, and assume that the convolution kernel function is:

[0060]

[0061]

[0062] Or use express;

[0063] Among them, P represents the number of neighborhood pixels, which is 8 here; R represents the neighborhood radius, g i represents the gray value of the i-th neighborhood pixel, g c Represents the grayscale value of the center pixel,

[0064] Set the phase modulation mode and encode it to the phase modulator 13. Assume that the weight range of the convolution kernel is [w_min, w_max] and the range of the phase modulation mode is [phase_min, phase_max].

[0065] phase=(weight-w_min)×(phase_max-phase_min) / (w_max-w_min)+phase_min;

[0066] Among them, phase represents the phase value in the phase modulation mode, and weight represents the weight value in the convolution kernel;

[0067] The input signal is phase modulated, and the input signal that needs to be convolved passes through the phase modulator 13 so as to interfere with the phase modulation pattern to achieve the convolution effect.

[0068] The photodetector 14 may be a CCD sensor (Charge-coupled Device). The photodetector 14 converts the signal input from the phase modulator 13 into an electrical signal and records it. The output of the photodetector 14 is the light intensity distribution after optical convolution.

[0069] FPGA20 deploys CNN (Convolutional Neural Networks), including defining and training the model using the deep learning framework PyTorch; converting the CNN model into a deployable form, including conversion of weight parameters and network topology; mapping the CNN model to FPGA20; passing the results of the feature extraction component 10 to FPGA20, using the CNN model to perform image prediction on FPGA20, and outputting a predicted CT image.

[0070] See also Figure 3 The detection component 30 includes a laser light source 31, a beam expander 32, a collimator 33, a compensator 34, a second Fourier lens 35, a spatial light modulator 36, a third Fourier lens 37, a holographic plate 38 and a CCD 39. All nine are at the same horizontal position to form an optical path. f is the focal length of the second Fourier lens 35 and also the focal length of the third Fourier lens 37. The second Fourier lens 35, the spatial light modulator 36, the third Fourier lens 37 and the holographic plate 38 are arranged in sequence, and the front and back intervals are all f; the beam expander 32 is located at the front focal plane of the collimator 33, and the compensator 34 is located at the back focal plane of the collimator 33.

[0071] Laser source 31 provides a stable, high-brightness laser beam. Beam expander 32 diffuses the beam, increasing its diameter. Collimator 33 converges the beam to a focal point, minimizing the beam angle and achieving optimal focusing. Because spatial light modulator 36 introduces additional phase distortion, which can affect optical recognition efficiency and accuracy, a four-wave mixing optical path or conjugate mirror is added to compensate.

[0072] In a specific embodiment, the compensating element 34 may be a conjugate mirror or a four-wave mixing optical path.

[0073] When the compensating element 34 is a conjugate mirror, assuming that the conjugate mirror consists of two surfaces, the medium refractive index is n, the incident angle is θ1, and the exit angle is θ2, ideally, the following can be obtained:

[0074]

[0075] The compensation amount is C = -2arcsin(n×sin(θ2)).

[0076] When the compensation component 34 is a four-wave mixing optical path, it specifically includes a beam splitter BS1, a beam splitter BS2, a beam splitter BS3, a reflector M1, a reflector M2, a reflector M3, a reflector M4, a reflector M5, a BSO crystal (bismuth silicate crystal), a 1 / 4 glass (Q) and a subsequent second Fourier lens and a spatial light modulator. It is divided into three paths. The first path: the reference light is formed through BS1, M1, and BSO; the second path: after passing through BS2, M2, and BS3, it is irradiated onto the image group in the spatial light modulator and then imaged onto the BSO crystal through the second Fourier lens to form the object light; the third path: the readout light is formed through M3, M4, M5, Q, and the BSO crystal. The object light and the reference light are coherently superimposed to form a phase grating. The diffracted beam of the readout light on the phase grating forms a phase conjugate beam of the object light, thereby achieving the compensation effect. Assuming that the current phase distortion is Imaging onto a spatial light modulator reconstructs the light wave into The light wave passing through the spatial light modulator is O'(x,y)O(x,y)exp[iθ(x,y)]; where O(x,y) is the light wave of the measured object, O'(x,y) is the light wave reflected by the BSO crystal, and i is an imaginary number. is the inverse transformation of the light wave phase, and exp[iθ(x,y)] is the phase term provided by the crystal.

[0077] The laser light source 31 is expanded by the beam expander 32 and then enters the collimator 33 to form a collimated expanded beam. The beam then passes through the compensator 34 to remove phase distortion and enters the spatial light modulator 36 .

[0078] The predicted CT image (from the output of FPGA 20) and the reference image (from pre-processing module 42) are input to the spatial light modulator 36. The third Fourier lens 37 performs a joint transformation. The output result is displayed on the holographic plate 38, and then detected by the CCD 39 and finally displayed on the computer 50. If a large optical correlation peak is detected, it means that the match is successful. At this time, the area with bright output correlation light spot is the location of the lesion. Figure 4 , the left side is the input PET image, and the right side is the predicted CT image. The circled arrow in the predicted CT image on the right side points to the lesion location.

[0079] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A PET image lesion detection device based on a photoelectric structure, characterized in that: It includes feature extraction components, FPGA, detection components and preprocessing modules, among which, The input of the feature extraction component is the PET image, which is converted into an electrical signal through optical convolution of the feature extraction component and then input into the FPGA; The FPGA deploys a convolutional neural network to predict the CT image and output it to the detection component; The detection component receives the predicted CT image output by the FPGA, combines it with the lesion position reference image input by the preprocessing module, and outputs the detection result image after optical transformation; The feature extraction component includes a first Fourier lens, a grating, a phase modulator, and a photosensitive detector, all of which are located at the same horizontal position to form an optical path. f is the focal length of the first Fourier lens. The distance between the PET image input to the first Fourier lens and the first Fourier lens is f, and the distance between the first Fourier lens and the grating is f. The phase modulator is placed behind the grating. The distance between the phase modulator and the photosensitive detector is f. The photosensitive detector is placed at the end of the optical path, receives and converts the optical signal into an electrical signal, and transmits it to the FPGA for processing. The phase modulator changes the phase distribution of light and adjusts the phase of the optical convolution kernel, specifically including: Determine the convolution kernel function, and assume that the convolution kernel function is: ； ； Or use express; Among them, P represents the number of neighborhood pixels, which is 8 here; R represents the neighborhood radius, g i Represents the gray value of the i-th neighborhood pixel, g c Represents the grayscale value of the center pixel, ; Set the phase modulation mode and encode it into the phase modulator, assuming that the weight range of the convolution kernel is , the range of the phase modulation mode is ; ; Among them, phase represents the phase value in the phase modulation mode, and weight represents the weight value in the convolution kernel; The input signal is phase modulated, and the input signal that needs to be convolved is passed through the phase modulator to interfere with the phase modulation pattern to achieve the convolution effect.

2. The device according to claim 1, characterized in that The photosensitive detector converts the signal transmitted by the phase modulator into an electrical signal and records it. The output of the photosensitive detector is the light intensity distribution after optical convolution.

3. The device according to claim 1, characterized in that The FPGA deployment of the convolutional neural network includes defining and training the model using the deep learning framework PyTorch; converting the CNN model into a deployable form, including conversion of weight parameters and network topology; The CNN model is mapped to the FPGA; the results of the feature extraction component are passed to the FPGA, and the CNN model is used to perform image prediction on the FPGA to output the predicted CT image.

4. The device according to claim 1, characterized in that The detection assembly includes a laser light source, a beam expander, a collimator, a compensator, a second Fourier lens, a spatial light modulator, a third Fourier lens, a holographic plate and a CCD. All nine are located at the same horizontal position to form an optical path. f is the focal length of the second Fourier lens and also the focal length of the third Fourier lens. The second Fourier lens, the spatial light modulator, the third Fourier lens and the holographic plate are arranged in sequence, and the front and back intervals are all f; the beam expander is located at the front focal plane of the collimator, and the compensator is located at the rear focal plane of the collimator.

5. The device according to claim 4, characterized in that The compensating element is a conjugate mirror or a four-wave mixing optical path.

6. The device according to claim 5, characterized in that When the compensating element is a conjugate mirror, it is assumed that the conjugate mirror consists of two surfaces, the medium refractive index is n, and the incident angle is , the exit angle is ,get: ； The compensation amount is .

7. The device according to claim 5, characterized in that When the compensating element is a four-wave mixing optical path, the four-wave mixing optical path specifically includes a beam splitter BS1, a beam splitter BS2, a beam splitter BS3, a reflector M1, a reflector M2, a reflector M3, a reflector M4, a reflector M5, a BSO crystal, a quarter glass slide, a second Fourier lens, and a spatial light modulator; the four-wave mixing optical path is divided into three paths, the first path: a reference light is formed by the beam splitter BS1, the reflector M1, and the BSO crystal; The second path: After passing through the beam splitter BS2, the reflector M2, and the beam splitter BS3, it is irradiated to the image group of the spatial light modulator and then imaged by the second Fourier lens to the BSO crystal to form the object light; The third path: the readout light is formed through the reflector M3, reflector M4, reflector M5, 1 / 4 glass slide, and BSO crystal; the object light and the reference light are coherently superimposed to form a phase grating, and the readout light diffracts on the phase grating to form a phase conjugate beam of the object light, achieving a compensation effect; assuming that the current phase distortion is , imaging onto the spatial light modulator to reconstruct the light wave , the light wave passing through the spatial light modulator is ;in, is the light wave of the measured object, is the light wave reflected by the BSO crystal, i is an imaginary number, is the inverse transformation of the light wave phase, is the phase term provided by the crystal.

8. The device according to claim 4, characterized in that If the CCD detects an optical correlation peak that exceeds a preset value, it indicates that the matching is successful, and the area where the CCD outputs the correlation light spot is bright is the location of the lesion.

Images