Multimodal molecular imaging systems and methods

Abstract

The present disclosure belongs to the technical field of biomedical imaging, and provides a multi-modal molecular imaging system and method. The system comprises: an optical imaging camera, whose optical axis is directed towards a to-be-imaged object, acquires a molecular luminescent signal and forms a molecular luminescent image; an ultrasonic probe is supported by an adjusting support, so that the ultrasonic probe is directed towards a target imaging area of the to-be-imaged object; an acoustic action area of the ultrasonic probe at least partially overlaps with an optical imaging field of view of the optical imaging camera in the target imaging area; an ultrasonic imaging device forms an ultrasonic structural image for determining the target imaging area based on ultrasonic echoes received by the ultrasonic probe, and controls the ultrasonic probe to excite ultrasonic response luminescent molecules to generate a molecular luminescent signal; and an image processing device controls the ultrasonic imaging device to perform ultrasonic excitation based on a preset task timing, and controls the optical imaging camera to acquire a molecular luminescent image. Without transferring the to-be-imaged object, ultrasonic structural imaging, ultrasonic excitation and molecular luminescent imaging of the to-be-imaged object can be completed.

Description

Technical Field

[0001] This disclosure belongs to the field of biomedical imaging technology, and in particular relates to a multimodal molecular imaging system and method. Background Technology

[0002] Ultrasound imaging can obtain information on the structure, boundaries, and depth of tissues, while molecular luminescence imaging can highly sensitively reflect the distribution and dynamic changes of probes in vivo; the two are complementary. Among related technologies, the feasibility of ultrasound-induced luminescence for in vivo imaging has been verified, but research mainly focuses on the luminescence mechanism and luminescent materials. Ultrasound excitation and optical acquisition are usually simply combined using separate devices. Ultrasound imaging and optical molecular imaging are typically used in separate devices with simple superposition, making it difficult to achieve coordinated, integrated imaging. Summary of the Invention

[0003] To address the aforementioned issues, this disclosure provides a multimodal molecular imaging system and method.

[0004] This disclosure is achieved through the following technical solution: In a first aspect, embodiments of this disclosure provide a multimodal molecular imaging system, the system comprising: an ultrasonic transducer and an optical imaging camera, an ultrasonic imaging device and an image processing device; The optical imaging camera's optical axis is oriented towards the object to be imaged, acquiring molecular luminescence signals and forming a molecular luminescence image; The ultrasonic transducer assembly includes an ultrasonic probe and an adjustment bracket. The ultrasonic probe is supported by the adjustment bracket so that the ultrasonic probe faces the target imaging area of ​​the object to be imaged. The acoustic action area of ​​the ultrasonic probe and the optical imaging field of view of the optical imaging camera at least partially overlap in the target imaging area. The ultrasonic imaging device is communicatively connected to the ultrasonic probe, and forms an ultrasonic structural image for determining the target imaging area based on the ultrasonic echo received by the ultrasonic probe, and controls the ultrasonic probe to excite ultrasonic luminescent molecules to generate molecular luminescence signals. The image processing device is communicatively connected to the optical imaging camera and the ultrasonic imaging device. Based on a preset task timing, it controls the ultrasonic imaging device to perform ultrasonic excitation and controls the optical imaging camera to acquire molecular luminescence images.

[0005] Furthermore, The system also includes: a target support platform and a three-dimensional adjustment mechanism; The target support platform is fixed to one side of the three-dimensional adjustment mechanism; the target support platform is used to support the object to be imaged. The adjustment bracket is used to adjust the height and tilt angle of the ultrasonic probe, and the three-dimensional adjustment mechanism is used to adjust the position of the target support platform in three-dimensional space; wherein, the overlapping area of ​​the acoustic action area of ​​the ultrasonic probe and the optical imaging field of view of the optical imaging camera covers the target imaging area.

[0006] Furthermore, The longitudinal axis of the ultrasonic probe forms an angle with the optical axis of the optical imaging camera, and the orthographic projection of the ultrasonic probe onto the bearing surface of the target platform does not coincide with the orthographic projection of the target imaging area onto the bearing surface of the target platform.

[0007] Furthermore, The ultrasound imaging device is configured to: during the ultrasound imaging phase, control the ultrasound probe to emit imaging ultrasound toward the object to be imaged, and receive the ultrasound echo of the object to be imaged; form an ultrasound structural image based on the ultrasound echo, wherein the ultrasound structural image is used to determine the location, depth, boundary and size of the target tissue of the object to be imaged, and to determine the target imaging area; During the ultrasonic excitation phase, in response to the acquisition of the excitation command, the ultrasonic probe is controlled to apply excitation ultrasound to the target imaging region to excite the ultrasonic response luminescent molecules in the target imaging region to generate molecular luminescence signals; wherein, the ultrasonic probe switches between the ultrasonic imaging working state and the ultrasonic excitation working state, and uses the same probe coordinate system when performing ultrasonic imaging and ultrasonic excitation.

[0008] Furthermore, The ultrasonic probe includes an ultrasonic imaging unit and an ultrasonic excitation unit. The ultrasonic imaging unit is used to emit imaging ultrasound to the object to be imaged and to receive the ultrasonic echo of the object to be imaged. The ultrasonic excitation unit is used to apply excitation ultrasound to the target imaging area.

[0009] Furthermore, The image processing device is configured to: after determining the target imaging area, send an excitation command to the ultrasound imaging device; and control the optical imaging camera to acquire molecular emission images in real time and acquire molecular emission images with a delay during the corresponding period.

[0010] Furthermore, The image processing device is further configured to: establish a coordinate transformation relationship based on ultrasound image coordinates, target platform coordinates, and optical image coordinates; preprocess the molecular luminescence image; and register and fuse the ultrasound structural image with the molecular luminescence image according to the coordinate transformation relationship; and determine and display the anatomical structure, tissue depth, and molecular luminescence distribution of the target imaging region.

[0011] Secondly, based on the same inventive concept, this disclosure also provides a multimodal molecular imaging method applied to a multimodal molecular imaging system, the system comprising: an ultrasound transducer and an optical imaging camera, an ultrasound imaging device and an image processing device; the ultrasound transducer includes an ultrasound probe, the ultrasound imaging device is communicatively connected to the ultrasound probe, and the image processing device is communicatively connected to the optical imaging camera and the ultrasound imaging device; the multimodal molecular imaging method includes: The ultrasonic imaging device forms an ultrasonic structural image based on the ultrasonic echo received by the ultrasonic probe. Based on the ultrasonic structural image, the target imaging area of ​​the object to be imaged is determined, at least by adjusting the orientation of the ultrasonic probe, so that the acoustic action area of ​​the ultrasonic probe and the optical imaging field of view of the optical imaging camera at least partially overlap in the target imaging area. The image processing device controls the ultrasonic imaging device to perform ultrasonic excitation according to a preset task sequence, and controls the optical imaging camera to acquire molecular luminescence images.

[0012] Furthermore, The image processing device controls the ultrasound imaging device to perform ultrasound excitation according to a preset task sequence, and controls the optical imaging camera to acquire molecular luminescence images, including: After determining the target imaging area, the image processing device sends an excitation command to the ultrasound imaging device; In response to the acquisition of the excitation command, the ultrasonic imaging device controls the ultrasonic probe to apply excitation ultrasound to the target imaging area, so as to excite the ultrasonic response luminescent molecules in the target imaging area to generate molecular luminescence signals. The image processing device controls the optical imaging camera to acquire molecular emission images in real time and at a delay during the corresponding period.

[0013] Furthermore, The method further includes: The image processing device establishes a coordinate transformation relationship based on ultrasound image coordinates, target platform coordinates, and optical image coordinates; preprocesses the molecular luminescence image, and registers and fuses the ultrasound structural image with the molecular luminescence image according to the coordinate transformation relationship; and determines and displays the anatomical structure, tissue depth, and molecular luminescence distribution of the target imaging area.

[0014] The multimodal molecular imaging system in this embodiment selects and locates the target imaging region requiring ultrasonic excitation based on the ultrasonic structural image. This allows ultrasonic excitation and molecular luminescence imaging to act on the same target tissue or area of ​​the object being imaged, reducing the bias caused by estimating the location of deep targets solely based on surface landmarks of the object and improving the targeting and repeatability of the excitation. Simultaneously, ultrasonic structural imaging, ultrasonic excitation, and molecular luminescence imaging of the object are performed at the same location within the dark chamber, eliminating the need to transfer the object between different devices. This reduces spatial errors and signal fluctuations caused by changes in the object's posture, repetitive positioning, and imaging time delays.

[0015] Other features and advantages of this disclosure will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the disclosure. The objects and other advantages of this disclosure may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A schematic diagram of a multimodal molecular imaging system provided in this disclosure embodiment; Figure 2 This is a schematic diagram of a dark box and its internal structure with the door open, provided by an embodiment of the present disclosure. Figure 3 This is a schematic diagram of a dark box and its internal structure when the door is closed, provided as an embodiment of the present disclosure. Figure 4 This is a schematic diagram of an adjustable support structure provided in an embodiment of the present disclosure; Figure 5 This is a schematic diagram of a target support platform and a three-dimensional adjustment mechanism provided in an embodiment of the present disclosure; Figure 6 A schematic diagram of an anesthesia device provided in an embodiment of this disclosure; Figure 7 This is a flowchart of a multimodal molecular imaging method provided in an embodiment of the present disclosure.

[0018] Explanation of reference numerals in the attached figures: 1. Dark box; 2. Optical imaging camera; 3. Ultrasonic probe; 4. Adjustment bracket; 41. Fixed mounting part; 42. Adjusting arm; 421. Limiting groove; 43. Rotating joint; 44. Probe clamping part; 45. Linear guide part; 46. Locking part; 5. Target support platform; 6. Three-dimensional adjustment mechanism; 7. Ultrasonic imaging equipment; 8. Anesthesia device; 9. External anesthesia tubing; 10. Anesthesia tubing box wall interface; 11. Internal anesthesia tubing; 12. Animal anesthesia nose mask; 13. Image processing computer; 14. Optical imaging display; 15. Dark box door; 16. Ultrasonic probe connection cable; 17. Ultrasonic connection cable box wall interface; 18. Temperature maintaining plate; 19. X-axis linear adjustment assembly; 20. Y-axis linear adjustment assembly; 21. Z-axis lifting adjustment assembly. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0020] Firstly, Figure 1 This is a schematic diagram of a multimodal molecular imaging system provided in an embodiment of the present disclosure, as shown below. Figure 1 As shown, this disclosure provides a multimodal molecular imaging system, which includes: an ultrasound transducer and optical imaging camera 2, an ultrasound imaging device 7, and an image processing device.

[0021] The optical axis of the optical imaging camera 2 is oriented towards the object to be imaged, acquiring molecular luminescence signals and forming a molecular luminescence image. The optical imaging camera 2 can be installed on the top surface of the interior space of the dark chamber 1, or on the upper half of the inner surface of the side wall of the dark chamber 1. The optical imaging camera 2 can be directly fixed inside the dark chamber 1, or it can be installed inside the dark chamber 1 via an adjustable connector to adjust the orientation of its optical axis. The object to be imaged is located in the lower region of the interior space of the dark chamber 1, and the optical axis of the optical imaging camera 2 is oriented towards the object below it. The optical imaging field of view of the optical imaging camera 2 at least partially covers the object to be imaged.

[0022] The ultrasonic transducer assembly includes an ultrasonic probe 3 and an adjustment bracket 4. The ultrasonic probe 3 is supported by the adjustment bracket 4, so that the ultrasonic probe 3 faces the target imaging area of ​​the object to be imaged. The acoustic action area of ​​the ultrasonic probe 3 and the optical imaging field of view of the optical imaging camera 2 at least partially overlap in the target imaging area. The adjustment bracket 4 in the ultrasonic transducer assembly can adjust the height, tilt angle, and extension length of the ultrasonic probe 3 so that the acoustic action area of ​​the ultrasonic probe 3 and the optical imaging field of view of the optical imaging camera 2 at least partially overlap in the target imaging area. This allows for the acquisition of ultrasonic structural images and molecular luminescence images of the target imaging area without moving the object to be imaged, improving the accuracy and stability of positioning and repeatable imaging. The adjustment bracket 4 can be fixed to the inner surface of the darkroom 1. The specific fixing position can be determined according to the size and structure of the darkroom 1 and the setting position of the optical imaging camera 2.

[0023] The ultrasound imaging device 7 is communicatively connected to the ultrasound probe 3. Based on the ultrasound echo received by the ultrasound probe 3, it forms an ultrasound structural image to determine the target imaging region and controls the ultrasound probe 3 to excite ultrasound-responsive luminescent molecules to generate molecular luminescence signals. According to the ultrasound structural image of the object to be imaged, the target imaging region of the object to be imaged can be accurately located, which can reduce the deviation caused by estimating the location of deep targets based solely on animal surface landmarks, and improve the specificity and repeatability of exciting ultrasound-responsive luminescent molecules.

[0024] The image processing device is communicatively connected to the optical imaging camera 2 and the ultrasonic imaging device 7. Based on a preset task timing sequence, it controls the ultrasonic imaging device 7 to perform ultrasonic excitation and controls the optical imaging camera 2 to acquire molecular luminescence images. The image processing device can control the timing of the ultrasonic imaging device 7 performing ultrasonic excitation and the optical imaging camera 2 acquiring molecular luminescence images to coordinate the excitation and acquisition timing, ensure that the ultrasonic excitation and molecular luminescence signal acquisition times are matched, and obtain synchronously corresponding ultrasonic and molecular luminescence imaging data.

[0025] It should be noted that the ultrasound imaging device 7 can be integrated with the image processing device in the same device to form an integrated multimodal imaging control system, or they can be set up separately; the ultrasound imaging device 7 and the ultrasound probe 3, and the image processing device and the optical imaging camera 2 and the ultrasound imaging device 7 can be connected by wired or wireless communication.

[0026] The multimodal molecular imaging system in this embodiment places an optical imaging camera 2 and an ultrasonic probe 3 inside a dark box 1, and places the object to be imaged inside the dark box 1. The ultrasonic imaging device 7 controls the ultrasonic probe 3 to emit imaging ultrasound towards the object and receives the ultrasonic echo from the object, forming an ultrasonic structural image based on the ultrasound echo; based on the ultrasonic structural image, the target imaging region of the object is determined. The adjustment bracket 4 supporting the ultrasonic probe 3 is adjusted so that the ultrasonic probe 3 is tilted and points towards the target imaging region, with the acoustic action area of ​​the ultrasonic probe 3 at least partially overlapping with the optical imaging field of view of the optical imaging camera 2 in the target imaging region. The target imaging region requiring ultrasonic excitation is selected and located according to the ultrasonic structural image, so that ultrasonic excitation and molecular luminescence imaging act on the same target tissue or area of ​​the object, reducing the deviation caused by estimating the location of deep targets solely based on surface landmarks of the object, and improving the targeting and repeatability of the excitation. Simultaneously, the ultrasound structural imaging, ultrasound excitation, and molecular luminescence imaging of the object to be imaged are all performed at the same location within the dark chamber 1, eliminating the need to transfer the object between different devices. This reduces spatial errors and signal fluctuations caused by changes in the object's posture, repetitive positioning, and imaging time delays. The objects to be imaged include animal experimental subjects or tissue simulation samples. The obtained image data is used for scientific observation and quantitative analysis, and is not used to directly make disease diagnoses or treatment decisions.

[0027] In this embodiment of the disclosure, Figure 2 This is a schematic diagram of a dark box and its internal structure with the door open, provided by an embodiment of the present disclosure. Figure 3 This is a schematic diagram of a dark box and its internal structure when the door is closed, as provided in an embodiment of this disclosure. Figure 2 , 3 As shown, due to the weak molecular luminescence signal, low-light optical imaging requires the acquisition of molecular luminescence images within a light-shielding space. The dark box 1 constitutes the light-shielding space for low-light optical imaging, facilitating the acquisition of molecular luminescence images. The dark box 1 has an openable dark box door 15. At least a labyrinth-style light-shielding structure or an elastic sealing strip is provided between the dark box door 15 and the dark box door frame; a combination of both can also be used. The dark box door 15 is located on one side of the dark box 1. When closed, its circumferential edge fits against the dark box door frame, forming a continuous light-shielding and sealing boundary. The inner surface of the dark box 1 can be treated with low reflection, light-absorbing, or light-dissipating treatments to reduce internal scattered light and reflected stray light. When the dark box door 15 is open, the object to be imaged and all parts requiring adjustment can be operated; when the dark box door 15 is closed, it effectively provides light shielding.

[0028] Continue to refer to Figure 2 , 3The dark box 1 is equipped with at least an optical imaging camera 2, an ultrasound probe 3, and an adjustment bracket 4. Outside the dark box 1, an ultrasound imaging device 7 and an image processing device are installed. The image processing device includes an image processing computer 13 and an optical imaging display 14. The image processing computer 13 is communicatively connected to the optical imaging camera 2 and the ultrasound imaging device 7, and is also communicatively connected to the optical imaging display 14. The optical imaging display 14 is used to display the anatomical structure, tissue depth, and molecular luminescence distribution of the target imaging area. The optical imaging display 14 can also be used to intuitively control the overall system and configure preset task sequences.

[0029] In this embodiment, the optical imaging camera 2 is positioned at the top of the dark box 1, with its optical axis pointing towards the object to be imaged below it. The optical axis of the optical imaging camera 2 is a straight line passing through the center of curvature of the spherical surfaces of all the lenses inside the camera, extending outwards from the camera's interior, representing the central observation direction of the image. The optical imaging camera 2 can be of various types, including cooled CCD (charge-coupled device) cameras, electron-multiplying CCD cameras, scientific-grade CMOS (complementary metal-oxide-semiconductor) cameras, or near-infrared array cameras. Depending on the emission band of the ultrasound-responsive luminescent molecules, a fixed-focus or zoom lens, a bandpass filter, a long-pass filter, or a multi-channel filter assembly can be configured on the light-incident side of the optical imaging camera 2. The optical imaging display 14 can be used to display optical preview, acquisition status, ultrasound-luminescence fusion images, and quantitative analysis results. The ultrasound-responsive luminescent molecules can generate emission signals under ultrasonic excitation; these molecules can be fabricated as imaging probes, i.e., ultrasound-responsive luminescent probes. These molecules can be introduced into animal subjects or distributed within isolated tissue samples to achieve targeted or non-targeted enrichment in animals or tissues. Under ultrasonic excitation, they generate characteristic luminescent signals, which, combined with ultrasonic structural images, enable molecular-level information detection and imaging within tissues.

[0030] It should be noted that the specific chemical structure, nanomaterial composition, emission band and emission lifetime of the ultrasound-responsive luminescent molecules and their corresponding ultrasound-responsive luminescent probes in this disclosure can be adjusted and selected according to the specific animal experimental subjects or target simulated samples, and no further specific limitations are made here.

[0031] In this disclosure, reference continues to be made to... Figure 2 , 3The ultrasonic transducer assembly includes an ultrasonic probe 3 and an adjustment bracket 4. The adjustment bracket 4 is fixed to the inner surface of the side wall of the darkroom 1. The ultrasonic probe 3 is supported by the adjustment bracket 4, so that the ultrasonic probe 3 is tilted and points towards the object to be imaged. After determining the target imaging area of ​​the object to be imaged, the adjustment bracket 4 can be adjusted so that the ultrasonic probe 3 is tilted and points towards the object to be imaged, thereby making the acoustic action area of ​​the ultrasonic probe 3 at least partially overlap with the optical imaging field of view of the optical imaging camera 2 in the target imaging area. The ultrasonic probe 3 is connected to the ultrasonic imaging device 7 via the ultrasonic probe connection cable 16. When setting up, the adjustment bracket 4 used to support the ultrasonic probe 3 can be set on the inner surface of any three side walls except the darkroom door, but not on the inner surface of the darkroom door 15, to prevent the function of the ultrasonic transducer assembly from being affected or the ultrasonic probe 3 from being damaged during the opening and closing of the darkroom door. When specifically setting up the ultrasonic transducer assembly, it can be selected from the inner surface of any three side walls except the darkroom door, according to the overall size of the darkroom 1, the placement method of the object to be imaged, and the optical axis of the optical imaging camera 2.

[0032] Continue to refer to Figure 2 , 3 The ultrasonic probe connection cable 16 passes through the wall of the darkroom 1 via the ultrasonic connection cable enclosure interface 17, and communicates with the ultrasonic probe 3. The ultrasonic connection cable enclosure interface 17 can be composed of one or more of the following: a through-plate connector, a flange, a sleeve, a compression seal, and a light-shielding labyrinth, to provide a fixed, sealed, and light-shielding structure, enabling the interface to secure the cable and prevent external light from entering the darkroom 1 through the cable gap. The ultrasonic probe connection cable 16 does not need to be laid through the door gap of the darkroom door 15, thus avoiding interference with the normal opening and closing of the darkroom door 15 and its light-shielding sealing.

[0033] In some examples, Figure 4 This is a schematic diagram of an adjustable support structure provided in an embodiment of the present disclosure, as shown below. Figure 4As shown, the adjustment bracket 4 includes a fixed mounting part 41, at least two adjusting arms 42, multiple rotating joints 43, a probe clamping part 44, and a linear guide part 45. The rotating joints 43 can be used to rotatably connect the fixed mounting part 41 to at least one adjusting arm 42, to rotatably connect the adjusting arms 42 to each other, and to rotatably connect the probe clamping part 44 to at least one adjusting arm 42. When the adjusting arms 42 are connected to each other via the rotating joints 43, it can be a rotatable connection between the ends of two adjusting arms 42, or a rotatable connection between the ends and the middle of two adjusting arms 42. When the ends and the middle of the adjusting arms 42 are rotatably connected, a groove can be cut in the middle of the adjusting arm 42, allowing the corresponding positions of the ends of the adjusting arms 42 to slide within the limiting groove 421 in the middle of the adjusting arm 42, thereby achieving more angles and greater amplitude adjustments to the ultrasonic probe 3. The limiting groove 421 can be completely hollowed out, partially hollowed out, or not hollowed out. The rotating joint 43 can be adjusted and locked by loosening or tightening the locking member 46, which includes a locking screw and a knob. The probe clamping part 44 is used to clamp the ultrasonic probe 3. The probe clamping part 44 is provided with the locking member 46. The ultrasonic probe 3 can be separated from the probe clamping part 44 by loosening the locking member 46, and the ultrasonic probe 3 can be locked by tightening the locking member 46. The fixed mounting part 41 is slidably connected to the linear guide part 45. The length direction of the linear guide part 45 is perpendicular to the ground direction. The linear guide part 45 can be a slide rail, so that the fixed mounting part 41 can be adjusted in height along the slide rail, thereby allowing the ultrasonic probe 3 to move along the slide rail to approach or move away from the target imaging area. The fixed mounting part 41 is provided with a locking member 46. The fixed mounting part 41 can slide on the slide rail by loosening the locking member 46, and the fixed mounting part 41 can be locked and positioned on the linear guide part 45 by tightening the locking member 46. The adjustment arm 42 and the fixed mounting part 41, the two adjustment arms 42, the adjustment arm 42 and the probe clamping part 44, the probe clamping part 44 and the ultrasonic probe 3, and the linear guide part 45 and the fixed mounting part 41 can all be in an active or locked state. This improves the flexibility of the ultrasonic probe 3 position adjustment, allowing adjustment of the height, tilt angle, and extension length of the ultrasonic probe 3 so that the acoustic action area of ​​the ultrasonic probe 3 at least partially overlaps with the optical imaging field of view of the optical imaging camera 2 in the target imaging area; it also maintains the set position, angle, and coupling state of the ultrasonic probe 3, improving the targeting and repeatability of the excitation of ultrasonic response luminescent molecules, as well as the accuracy and stability of positioning and repeated imaging.

[0034] It should be noted that the adjustment bracket 4 can be a manual joint bracket, an electric multi-axis bracket, or a robot positioning mechanism. Only one example is given here, and no specific limitation is made.

[0035] In some examples, Figure 5This is a schematic diagram of a target support platform and a three-dimensional adjustment mechanism provided in an embodiment of this disclosure, as shown below. Figure 5 As shown, the dark box 1 of the multimodal molecular imaging system also includes a target support stage 5 and a three-dimensional adjustment mechanism 6. The three-dimensional adjustment mechanism 6 is typically fixed to the bottom of the dark box 1, and the target support stage 5 is fixed to one side of the three-dimensional adjustment mechanism 6, specifically to the side of the three-dimensional adjustment mechanism 6 furthest from the ground. The target support stage 5 is used to support the object to be imaged. The adjustment bracket 4 is used to adjust the height and tilt angle of the ultrasound probe 3, and the three-dimensional adjustment mechanism 6 is used to adjust the position of the target support stage 5 in three-dimensional space. The overlapping area of ​​the acoustic action area of ​​the ultrasound probe 3 and the optical imaging field of view of the optical imaging camera 2 covers the target imaging area. By adding the target support stage 5 and the three-dimensional adjustment mechanism 6, both the ultrasound probe 3 and the target support stage 5 supporting the object to be imaged can be adjusted simultaneously to increase the overlapping area of ​​the acoustic action area of ​​the ultrasound probe 3 and the optical imaging field of view of the optical imaging camera 2, ensuring that the overlapping area covers the target imaging area. The three-dimensional adjustment mechanism 6 includes an X-axis linear adjustment component 19, a Y-axis linear adjustment component 20, and a Z-axis lifting adjustment component 21, which can precisely move the target imaging area to the area where the acoustic action area and the optical imaging field of view coincide. The X-axis linear adjustment component 19, the Y-axis linear adjustment component 20, and the Z-axis lifting adjustment component 21 of the three-dimensional adjustment mechanism 6 can be set with scales or markings to record and restore the three-dimensional spatial position of the target support stage 5.

[0036] Furthermore, a temperature holding plate 18 is provided between the target support platform 5 and the three-dimensional adjustment mechanism 6 to maintain the temperature of the experimental object. The temperature holding plate 18 can be a heating plate responsible for heat generation and temperature control, or a heat-conducting plate for heat transfer. The target support platform 5 can also be equipped with an ultrasonic coupling structure, which has an ultrasonic coupling contact surface. The ultrasonic coupling contact surface is in direct contact with the object to be imaged, and when set, it at least partially overlaps with the object to be imaged, and thus overlaps with the target imaging area, or it can cover the entire target support platform 5. The ultrasonic coupling structure can adopt replaceable contact parts, coupling medium containment structure, or local coupling pad. The relevant materials of the ultrasonic coupling structure preferably have low self-emission and low scattering characteristics in the target emission band.

[0037] It should be noted that the fixation, anesthesia support, and ultrasound parameters of the specific object to be imaged shall be determined by qualified personnel in accordance with the approved experimental protocol, equipment safety boundaries, and applicable standards; no specific anesthetic medium or probe delivery protocol is limited here.

[0038] In some examples, the ultrasonic probe 3 is located below the optical imaging camera 2, and typically, the ultrasonic probe 3 is closer to the target stage than the optical imaging camera 2. The ultrasonic probe 3 is supported by an adjustment bracket 4 and is tilted towards the target imaging area on the target stage 5. The longitudinal axis of the ultrasonic probe 3 forms an angle with the optical axis of the optical imaging camera 2, which can be between 20° and 70°. The orthographic projection of the ultrasonic probe 3 onto the bearing surface of the target stage 5 does not coincide with the orthographic projection of the target imaging area onto the bearing surface of the target stage 5. The longitudinal axis is the longitudinal geometric center of the probe housing and handle of the ultrasonic probe 3, and is the mechanical axis of the ultrasonic probe 3. By adjusting the angle between the longitudinal axis of the ultrasonic probe 3 and the optical axis of the optical imaging camera 2, the acoustic action area of ​​the ultrasonic probe 3 and the imaging field of view of the optical imaging camera 2 at least partially overlap in the target imaging area, while ensuring that the probe tip and body avoid the effective light-receiving cone leading from the target imaging area to the optical imaging camera 2.

[0039] Furthermore, the angle between the longitudinal axis of the ultrasonic probe 3 and the optical axis of the optical imaging camera 2 is preferably 30° to 60°; this expands the overlap between the acoustic action area of ​​the ultrasonic probe 3 and the imaging field of view of the optical imaging camera 2 in the target imaging area. The ultrasonic probe 3 is tilted from the side of the optical axis toward the target imaging area, avoiding the effective light-receiving path while maintaining stable acoustic coupling.

[0040] In some examples, such as Figure 6 As shown, the multimodal molecular imaging system also includes an anesthesia device 8; the anesthesia device 8 is located outside the dark chamber 1, and its output end is connected to an external anesthesia tubing 9. The external anesthesia tubing 9 is connected to an internal anesthesia tubing 11 inside the dark chamber 1 via an anesthesia tubing box wall interface 10, and the end of the internal anesthesia tubing 11 is connected to an animal anesthesia nasal mask 12 located at the target support stage 5. Thus, the anesthesia tubing does not need to cross over or be sandwiched between the dark chamber door 15 and the door frame from the front of the equipment, and can provide continuous anesthesia support for the experimental subject without affecting the light-shielding and sealing of the dark chamber 1. The anesthesia tubing box wall interface 10 can be a combination of detachable connectors at both ends, through-plate sealing seats, airtight gaskets, and bent light-shielding channels. The internal anesthesia tubing 11 is arranged along the inner wall of the dark chamber 1 and the edge of the target support stage, avoiding entry into the main field of view of the optical imaging camera 2 or the movement range of the interference adjustment bracket. The target support stage also includes an animal restraint structure to maintain the animal's posture, which serves as a positioning reference for repeated imaging.

[0041] In some examples, the darkroom 1 of the multimodal molecular imaging system can also be reserved with through-wall interfaces for temperature sensors, ECG monitoring lines, respiratory monitoring lines, trigger lines, position sensors, and other scientific imaging accessories. Each through-wall interface preferably employs a uniform light-shielding and sealing method and is positioned so as not to affect the opening and closing of the darkroom door 15 or the adjustment bracket 4 in the ultrasound transducer assembly for adjusting the position of the ultrasound probe 3 and the optical imaging field of view. The ultrasound probe 3 connection cable and anesthesia tubing both enter the darkroom 1 through dedicated wall interfaces, which reduces probe obstruction, tubing interference, and stray light, improving the stability and signal-to-noise ratio of weak light emission acquisition.

[0042] In some examples, the ultrasound imaging device 7 is configured to: during the ultrasound imaging phase, control the ultrasound probe 3 to emit imaging ultrasound towards the object to be imaged and receive the ultrasound echo from the object; form an ultrasound structural image based on the ultrasound echo, the ultrasound structural image being used to determine the location, depth, boundary, and size of the target tissue of the object to be imaged, and to determine the target imaging region; the target imaging region can also be determined by combining the spatial relationship between the target tissue and surrounding tissues in the ultrasound structural image; the ultrasound structural image can be a two-dimensional or three-dimensional image, and the type of ultrasound structural image includes B-mode images, and optionally includes Doppler images, contrast ultrasound images, elastography images, or combinations thereof. During the ultrasound excitation phase, in response to the acquisition of an excitation command, control the ultrasound probe 3 to apply excitation ultrasound to the target imaging region to excite the ultrasound-responsive luminescent molecules in the target imaging region to generate molecular luminescence signals; wherein, the excitation command can be generated by the image processing device and sent to the ultrasound imaging device 7 after the target imaging region is determined. The ultrasonic imaging stage and ultrasonic excitation stage of the ultrasonic imaging device 7 can be switched manually or automatically in response to the instructions of the image processing device; the ultrasonic probe 3 is in ultrasonic imaging working state in the ultrasonic imaging stage and in ultrasonic excitation working state in the ultrasonic excitation stage. The ultrasonic probe 3 switches between ultrasonic imaging working state and ultrasonic excitation working state, and uses the same probe coordinate system when performing ultrasonic imaging and ultrasonic excitation.

[0043] In this embodiment, the ultrasound probe 3, in ultrasound imaging mode, is tilted and pointed at the object to be imaged. If the target imaging area is pre-marked on the object's surface, the probe can directly point at the target imaging area. The ultrasound imaging device 7 controls the ultrasound probe 3 to send imaging ultrasound and receive the ultrasound echo from the object to be imaged. An ultrasound structural image of the object is formed based on the ultrasound echo. The location, depth, boundary, and size of the target tissue are determined in the ultrasound structural image by the operator or an ultrasound image receiving and target selection program in the image processing device, and the target imaging area is selected. The target tissue can be the tissue under study, and the target imaging area can be a region of interest for ultrasound excitation. The ultrasound structural image is not only used to observe tissue structure but also to determine the spatial target for subsequent ultrasound excitation. After determining the target imaging area, the ultrasonic imaging device 7 switches from ultrasonic imaging mode to ultrasonic excitation mode, ensuring that the acoustic action area of ​​the ultrasonic probe 3 and the optical imaging field of view of the optical imaging camera 2 at least partially overlap in the target imaging area. This is achieved by adjusting the support bracket 4 supporting the ultrasonic probe 3 and the three-dimensional adjustment mechanism 6 fixing the target platform 5, ensuring that the acoustic action area of ​​the ultrasonic probe 3 and the optical imaging field of view of the optical imaging camera 2 at least partially overlap in the target imaging area, or that the overlapping area completely covers the target imaging area. After positioning the ultrasonic probe 3, optical imaging camera 2, and target platform 5, while maintaining the spatial relationship between the object to be imaged, the target platform 5, the ultrasonic probe 3, and the optical imaging camera 2, the ultrasonic probe 3 applies excitation ultrasound to the target imaging area. The excitation ultrasound can be an ultrasonic excitation sequence to excite the ultrasonic response luminescent molecules in the target imaging area to generate molecular luminescence signals. Since ultrasonic imaging and ultrasonic excitation share the same probe coordinate system, the selected target imaging area in the ultrasonic structural image can directly establish a spatial correspondence with the ultrasonic excitation action area. Therefore, after determining the target imaging area, ultrasonic structural images of the target imaging area can be repeatedly acquired.

[0044] In some examples, the ultrasonic probe 3 includes an ultrasonic imaging unit and an ultrasonic excitation unit. The ultrasonic imaging unit is used to emit imaging ultrasound to the object to be imaged and to receive the ultrasonic echo from the object to be imaged. The ultrasonic excitation unit is used to apply excitation ultrasound to the target imaging area.

[0045] Furthermore, the ultrasound imaging device 7 may include an ultrasound transmission and reception control unit, an echo signal processing unit, a beamforming unit, and an ultrasound image display or output unit. The ultrasound transmission and reception control unit can be used to control the ultrasound probe 3 to transmit imaging ultrasound and receive ultrasound echoes; the echo signal processing unit can be used to process the received ultrasound echo signals; the beamforming unit can be used to perform beamforming on the processed echo signals to obtain scanning data for imaging. The ultrasound image display or output unit can be used to display the beamforming results as an ultrasound structural image and / or output it to an image processing device.

[0046] In this embodiment, the ultrasonic imaging unit can be an ultrasonic imaging probe, and the ultrasonic excitation unit can be an ultrasonic excitation transducer. Ultrasonic structural imaging and ultrasonic excitation can be performed by the ultrasonic imaging probe and an independent ultrasonic excitation transducer, respectively. The ultrasonic imaging probe and the ultrasonic excitation transducer are mounted on the same adjustment bracket 4, the same base, or different brackets with position sensors, and a fixed or calculable spatial transformation relationship is established through mechanical dimensions, position sensor readings, or an acousto-optic calibration body. Thus, the selected target imaging area in the ultrasonic structural image can be converted into the action position for ultrasonic excitation by the ultrasonic excitation transducer.

[0047] In some examples, the image processing device is configured to: after determining the target imaging region, send an excitation command to the ultrasound imaging device 7; the excitation command is used to cause the ultrasound imaging device 7 to apply excitation ultrasound to the target imaging region to excite the ultrasound-responsive luminescent molecules in the target imaging region to generate molecular luminescence signals; control the optical imaging camera 2 to acquire molecular luminescence images in real time and with a delay during a corresponding period; specifically, control the optical imaging camera 2 to acquire molecular luminescence images in real time during ultrasound excitation, and / or to acquire molecular luminescence images with a preset delay time after ultrasound excitation. Real-time acquisition and delayed acquisition of molecular luminescence images can be used individually or in combination. The image processing device can also control the optical imaging camera 2 to acquire an optical baseline image in a state without ultrasound excitation after determining the target imaging region and before performing ultrasound excitation, for subsequent preprocessing of the molecular luminescence images.

[0048] In some examples, the image processing device is also configured to: establish a coordinate transformation relationship based on ultrasound image coordinates, target stage coordinates, and optical image coordinates; preprocess the molecular luminescence image, and register and fuse the ultrasound structural image with the molecular luminescence image according to the coordinate transformation relationship; wherein the ultrasound structural image registered and fused with the molecular luminescence image can be one or more previously acquired ultrasound structural images, or one or more newly acquired ultrasound structural images, and one or more molecular luminescence images can also be used during the registration and fusion process; after the ultrasound structural image and the molecular luminescence image are registered and fused, the anatomical structure, tissue depth, and molecular luminescence distribution of the target imaging region are determined, and the anatomical structure, tissue depth, and molecular luminescence distribution of the target imaging region are displayed.

[0049] The establishment of coordinate transformation relationships includes: to establish the spatial correspondence between ultrasound structural images and molecular luminescence images, an acousto-optic calibration body with known marker positions can be placed on the target stage 5. Ultrasonic coordinates of the target marker positions are obtained using the ultrasound probe 3, and optical coordinates of the target marker positions or marker positions with known geometric relationships to the target marker positions are obtained using the optical imaging camera 2. Based on multiple ultrasound and optical coordinates, the image processing equipment calculates the coordinate transformation relationship between the ultrasound image coordinates, the target stage coordinates, and the optical image coordinates. This transformation relationship can be represented in matrix form. Establishing coordinate transformation relationships facilitates longitudinal observation of the same object, comparison between different objects, and repeated experiments across batches. After ultrasound structural imaging and molecular luminescence imaging, the coordinate transformation relationship can be used to map at least the target boundary, excitation region, or depth information in the ultrasound structural image to the molecular luminescence image. The ultrasound structural image and the molecular luminescence image of the same target imaging region are registered and fused, with the ultrasound image providing anatomical structure and tissue depth reference, and the molecular luminescence image providing high-sensitivity molecular functional information, achieving complementary imaging of the two types of information.

[0050] In this embodiment, the image processing device can run an ultrasound image receiving and target selection program, an optical acquisition program, and an acousto-optic registration and fusion program. The ultrasound image receiving and target selection program is used to receive ultrasound structural images and determine the target imaging object based on the ultrasound structural images. The optical acquisition program is used to control the optical imaging camera 2 to acquire molecular emission images and can acquire optical baseline images in the absence of ultrasound excitation for subsequent preprocessing of the molecular emission images. The acousto-optic registration and fusion program is used to preprocess the molecular emission images, including dark field correction, flat field correction, background subtraction, bad pixel processing, normalization, region of interest extraction, and temporal sequencing. The analysis includes background subtraction, which uses the optical baseline image acquired without ultrasound excitation as the background to subtract the baseline and background signals from the molecular luminescence image to remove the influence of system background noise and ambient light. Based on the pre-established coordinate transformation relationship between the ultrasound image coordinates, the target stage coordinates and the optical image coordinates, the ultrasound structural image and the molecular luminescence image are registered and fused. The anatomical structure, tissue depth and molecular luminescence distribution of the target tissue are jointly displayed on the optical imaging display. At the same time, the total luminescence intensity, average luminescence intensity, peak intensity, luminescence centroid position and luminescence intensity change over time in the region of interest are calculated.

[0051] In addition to the aforementioned procedures, the image processing device may also include a task instruction program or interface for sending instructions to the ultrasound imaging device 7 and the optical imaging camera 2 to control them. The operation can be performed manually by an operator based on a preset task sequence, or it can be controlled by receiving hardware trigger signals, software trigger signals, or a unified clock timestamp from an independent controller to perform ultrasound structural imaging, ultrasound excitation, and molecular luminescence image acquisition within a preset time window according to the preset task sequence. The ultrasound imaging device 7 and the optical imaging camera 2 are time-correlated through at least one of hardware triggering, software triggering, a unified clock, or a timestamp.

[0052] While maintaining the positioning reference of the experimental subject relative to the target support platform 5, ultrasonic structural imaging, ultrasonic excitation, and molecular luminescence image acquisition can be repeatedly performed according to different ultrasonic excitation parameters, different excitation sequences, and / or different observation times. This allows for longitudinal observation of the same experimental subject, comparison between different experimental subjects, and / or parameter screening of ultrasonic-responsive luminescence probes. The target support platform 5 can be configured with one or more animal positioning areas as needed, accommodating imaging of single or multiple small animals. The obtained image data is used for scientific research observation and quantitative analysis.

[0053] Secondly, Figure 7 A flowchart of a multimodal molecular imaging method provided in this disclosure is shown below. Figure 7 As shown, based on the same inventive concept, this disclosure also provides a multimodal molecular imaging method, applied to a multimodal molecular imaging system, the system including: an ultrasonic transducer and an optical imaging camera 2, an ultrasonic imaging device 7 and an image processing device; the ultrasonic transducer includes an ultrasonic probe 3, the ultrasonic imaging device 7 is communicatively connected to the ultrasonic probe 3, and the image processing device is communicatively connected to the optical imaging camera 2 and the ultrasonic imaging device 7.

[0054] Before imaging, the multimodal molecular imaging system is prepared. Specifically, before the device is put into use, the darkroom door is closed and the external equipment is put into standby mode. The optical imaging camera 2 acquires dark-field images, and the darkroom 1 and its wall interfaces are checked for light leakage. The wall interfaces include the ultrasound connection cable interface and the anesthesia tubing interface. If necessary, a uniform low-light reference source is used to acquire flat-field images. The animal experimental subjects given ultrasound-responsive luminescent probes, or tissue simulation samples containing ultrasound-responsive luminescent probes, are placed on the target support stage 5 and positioned to establish a positioning reference.

[0055] The multimodal molecular imaging method in this disclosure includes: S1: The ultrasonic imaging device 7 forms an ultrasonic structural image based on the ultrasonic echo received by the ultrasonic probe 3.

[0056] Specifically, the ultrasound imaging device 7 is turned on and put into ultrasound imaging operation mode. The ultrasound probe 3 is controlled to emit imaging ultrasound towards the object to be imaged and to receive the ultrasound echo from the object. The ultrasound imaging device 7 forms an ultrasound structural image of the object based on the ultrasound echo. The ultrasound structural image can be a two-dimensional or three-dimensional image, and its type includes B-mode images, and optionally includes Doppler images, contrast ultrasound images, elastography images, or combinations thereof. The ultrasound structural image is not only used to observe tissue structures, but also to determine the spatial targets for subsequent ultrasound excitation.

[0057] S2: Based on the ultrasonic structural image, determine the target imaging area of ​​the object to be imaged, at least by adjusting the orientation of the ultrasonic probe 3, so that the acoustic action area of ​​the ultrasonic probe 3 and the optical imaging field of view of the optical imaging camera 2 at least partially overlap in the target imaging area.

[0058] Specifically, the operator or the ultrasonic image receiving and target selection program in the image processing equipment determines the location, depth, boundary, and internal structure of the target tissue of the object to be imaged based on the ultrasonic structural image, and selects the region of interest for which ultrasonic excitation will be performed as the target imaging region. At least by adjusting the orientation of the ultrasonic probe 3, or simultaneously adjusting the three-dimensional adjustment mechanism 6, the acoustic action area of ​​the ultrasonic probe 3 and the optical imaging field of view of the optical imaging camera 2 are at least partially overlapped in the target imaging region, or the overlapping area of ​​the acoustic action area and the optical imaging field of view covers the target imaging region. Simultaneously, the probe pose and the position parameters of the adjustment mechanism can be saved or recorded to maintain the spatial relationship between the object to be imaged, the ultrasonic probe 3, and the optical imaging camera 2. The object to be imaged is located inside the dark box 1.

[0059] S3: The image processing device controls the ultrasonic imaging device 7 to perform ultrasonic excitation according to the preset task sequence, and controls the optical imaging camera 2 to acquire molecular luminescence images.

[0060] Specifically, after determining the target imaging area, the image processing device sends an excitation command to the ultrasound imaging device 7 to control the ultrasound imaging device 7 to perform ultrasound excitation. In response to the acquisition of the excitation command, the ultrasound imaging device 7 controls the ultrasound probe 3 to apply excitation ultrasound to the target imaging area, thereby exciting the ultrasound-responsive luminescent molecules in the target imaging area to generate molecular luminescence signals. The image processing device controls the optical imaging camera 2 to acquire molecular luminescence images in real time during ultrasound excitation and / or to acquire molecular luminescence images at a preset delay time after the ultrasound excitation ends. Real-time acquisition of molecular luminescence images and delayed acquisition of molecular luminescence images can be used individually or in combination.

[0061] In some examples, the image processing device controls the ultrasound imaging device 7 to perform ultrasound excitation according to a preset task sequence, and controls the optical imaging camera 2 to acquire molecular luminescence images, including: S31: After determining the target imaging area, the image processing device sends an excitation command to the ultrasound imaging device 7.

[0062] Specifically, after determining the target imaging area, the image processing device generates an excitation command and sends it to the ultrasound imaging device 7 to control the ultrasound imaging device 7 to perform ultrasound excitation. The excitation command is used to cause the ultrasound imaging device 7 to apply excitation ultrasound to the target imaging area, thereby exciting the ultrasound-responsive luminescent molecules in the target imaging area to generate molecular luminescence signals.

[0063] S32: In response to the acquisition of the excitation command, the ultrasonic imaging device 7 controls the ultrasonic probe 3 to apply excitation ultrasound to the target imaging area, so as to excite the ultrasonic response luminescent molecules in the target imaging area to generate molecular luminescence signals.

[0064] Specifically, in response to the acquisition of the excitation command, the ultrasonic imaging device 7 controls the ultrasonic probe 3 to apply excitation ultrasound to the target imaging area. The excitation ultrasound can be an ultrasonic excitation sequence to excite the ultrasonic response luminescent molecules in the target imaging area to generate molecular luminescence signals. Since ultrasonic imaging and ultrasonic excitation share the same probe coordinate system, the selected target imaging area in the ultrasonic structural image can directly establish a spatial correspondence with the ultrasonic excitation area, ensuring that the excitation ultrasound is accurately applied to the target imaging area. The ultrasonic probe 3 may include an ultrasonic imaging unit and an ultrasonic excitation unit, with the ultrasonic excitation unit applying excitation ultrasound to the target imaging area; the ultrasonic excitation unit can be an ultrasonic excitation transducer.

[0065] S33: The image processing device controls the optical imaging camera 2 to acquire molecular luminescence images in real time and acquire molecular luminescence images with a delay during the corresponding period.

[0066] Specifically, the image processing device controls the optical imaging camera 2 to acquire molecular emission images in real time during ultrasonic excitation, and / or to acquire molecular emission images at one or more preset delay times after ultrasonic excitation. Real-time and delayed acquisition of molecular emission images can be used individually or in combination within the same experiment. After the target image is determined, before applying excitation ultrasound to the target imaging region, the image processing device can also control the optical imaging camera 2 to acquire an optical baseline image in the absence of ultrasonic excitation, which will be used for subsequent preprocessing of the molecular emission image.

[0067] In some examples, multimodal molecular luminescence imaging methods also include: S4: The image processing device establishes a coordinate transformation relationship based on the ultrasound image coordinates, the target platform coordinates, and the optical image coordinates; it preprocesses the molecular luminescence image and registers and fuses the ultrasound structural image with the molecular luminescence image according to the coordinate transformation relationship; it determines and displays the anatomical structure, tissue depth, and molecular luminescence distribution of the target imaging area.

[0068] Specifically, establishing coordinate transformation relationships includes: to establish the spatial correspondence between the ultrasonic structural image and the molecular luminescence image, an acousto-optic calibration body with known marker positions can be placed on the target support stage 5. The ultrasonic coordinates of the target marker positions are obtained using the ultrasonic probe 3, and the optical coordinates of the target marker positions or marker positions with known geometric relationships to the target marker positions are obtained using the optical imaging camera 2. The image processing device calculates the coordinate transformation relationship between the ultrasonic image coordinates, the target support stage coordinates, and the optical image coordinates based on multiple ultrasonic and optical coordinates. This transformation relationship can be represented in matrix form. Preprocessing of the molecular luminescence image includes dark field correction, flat field correction, background subtraction, bad pixel processing, normalization, region of interest extraction, and time series analysis. Background subtraction uses the optical baseline image acquired without ultrasonic excitation as the background background, subtracting the baseline and background signals from the molecular luminescence image to remove system background noise and the influence of ambient background light. By utilizing the pre-established coordinate transformation relationship between ultrasound image coordinates, target stage coordinates, and optical image coordinates, the target boundary, excitation region, or depth information in the ultrasound structural image is mapped to the molecular luminescence image. The ultrasound structural image and the molecular luminescence image are then registered and fused according to the coordinate transformation relationship. After the ultrasound structural image and the molecular luminescence image are registered and fused, the anatomical structure, tissue depth, and molecular luminescence distribution of the target imaging region are determined. The anatomical structure, tissue depth, and molecular luminescence distribution of the target tissue are then jointly displayed on the optical imaging monitor. At the same time, the total luminescence intensity, average luminescence intensity, peak intensity, luminescence centroid position, and the change of luminescence intensity over time within the region of interest are calculated.

[0069] Although the present disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. A multimodal molecular imaging system, characterized in that, The system includes: an ultrasonic transducer and an optical imaging camera, an ultrasonic imaging device and an image processing device. The optical imaging camera's optical axis is oriented towards the object to be imaged, acquiring molecular luminescence signals and forming a molecular luminescence image; The ultrasonic transducer assembly includes an ultrasonic probe and an adjustment bracket. The ultrasonic probe is supported by the adjustment bracket so that the ultrasonic probe faces the target imaging area of ​​the object to be imaged. The acoustic action area of ​​the ultrasonic probe and the optical imaging field of view of the optical imaging camera at least partially overlap in the target imaging area. The ultrasonic imaging device is communicatively connected to the ultrasonic probe, and forms an ultrasonic structural image for determining the target imaging area based on the ultrasonic echo received by the ultrasonic probe, and controls the ultrasonic probe to excite ultrasonic luminescent molecules to generate molecular luminescence signals. The image processing device is communicatively connected to the optical imaging camera and the ultrasonic imaging device. Based on a preset task timing, it controls the ultrasonic imaging device to perform ultrasonic excitation and controls the optical imaging camera to acquire molecular luminescence images.

2. The system according to claim 1, characterized in that, The system also includes: a target support platform and a three-dimensional adjustment mechanism; The target support platform is fixed to one side of the three-dimensional adjustment mechanism; the target support platform is used to support the object to be imaged. The adjustment bracket is used to adjust the height and tilt angle of the ultrasonic probe, and the three-dimensional adjustment mechanism is used to adjust the position of the target support platform in three-dimensional space; wherein, the overlapping area of ​​the acoustic action area of ​​the ultrasonic probe and the optical imaging field of view of the optical imaging camera covers the target imaging area.

3. The system according to claim 1 or 2, characterized in that, The longitudinal axis of the ultrasonic probe forms an angle with the optical axis of the optical imaging camera, and the orthographic projection of the ultrasonic probe onto the bearing surface of the target platform does not coincide with the orthographic projection of the target imaging area onto the bearing surface of the target platform.

4. The system according to claim 1, characterized in that, The ultrasound imaging device is configured to: during the ultrasound imaging phase, control the ultrasound probe to emit imaging ultrasound toward the object to be imaged, and receive the ultrasound echo of the object to be imaged; form an ultrasound structural image based on the ultrasound echo, wherein the ultrasound structural image is used to determine the location, depth, boundary and size of the target tissue of the object to be imaged, and to determine the target imaging area; During the ultrasonic excitation phase, in response to the acquisition of the excitation command, the ultrasonic probe is controlled to apply excitation ultrasound to the target imaging region to excite the ultrasonic response luminescent molecules in the target imaging region to generate molecular luminescence signals; wherein, the ultrasonic probe switches between the ultrasonic imaging working state and the ultrasonic excitation working state, and uses the same probe coordinate system when performing ultrasonic imaging and ultrasonic excitation.

5. The system according to claim 4, characterized in that, The ultrasonic probe includes an ultrasonic imaging unit and an ultrasonic excitation unit. The ultrasonic imaging unit is used to emit imaging ultrasound to the object to be imaged and to receive the ultrasonic echo of the object to be imaged. The ultrasonic excitation unit is used to apply excitation ultrasound to the target imaging area.

6. The system according to claim 1, characterized in that, The image processing device is configured to: after determining the target imaging area, send an excitation command to the ultrasound imaging device; and control the optical imaging camera to acquire molecular emission images in real time and acquire molecular emission images with a delay during the corresponding period.

7. The system according to claim 6, characterized in that, The image processing device is further configured to: establish a coordinate transformation relationship based on ultrasound image coordinates, target platform coordinates, and optical image coordinates; preprocess the molecular luminescence image; and register and fuse the ultrasound structural image with the molecular luminescence image according to the coordinate transformation relationship; and determine and display the anatomical structure, tissue depth, and molecular luminescence distribution of the target imaging region.

8. A multimodal molecular imaging method, characterized in that, An application is made in a multimodal molecular imaging system, the system comprising: an ultrasound transducer and an optical imaging camera, an ultrasound imaging device, and an image processing device; the ultrasound transducer includes an ultrasound probe, the ultrasound imaging device is communicatively connected to the ultrasound probe, and the image processing device is communicatively connected to the optical imaging camera and the ultrasound imaging device; the multimodal molecular imaging method includes: The ultrasonic imaging device forms an ultrasonic structural image based on the ultrasonic echo received by the ultrasonic probe. Based on the ultrasonic structural image, the target imaging area of ​​the object to be imaged is determined, at least by adjusting the orientation of the ultrasonic probe, so that the acoustic action area of ​​the ultrasonic probe and the optical imaging field of view of the optical imaging camera at least partially overlap in the target imaging area. The image processing device controls the ultrasonic imaging device to perform ultrasonic excitation according to a preset task sequence, and controls the optical imaging camera to acquire molecular luminescence images.

9. The method according to claim 8, characterized in that, The image processing device controls the ultrasound imaging device to perform ultrasound excitation according to a preset task sequence, and controls the optical imaging camera to acquire molecular luminescence images, including: After determining the target imaging area, the image processing device sends an excitation command to the ultrasound imaging device; In response to the acquisition of the excitation command, the ultrasonic imaging device controls the ultrasonic probe to apply excitation ultrasound to the target imaging area, so as to excite the ultrasonic response luminescent molecules in the target imaging area to generate molecular luminescence signals. The image processing device controls the optical imaging camera to acquire molecular emission images in real time and at a delay during the corresponding period.

10. The method according to claim 9, characterized in that, The method further includes: The image processing device establishes a coordinate transformation relationship based on ultrasound image coordinates, target platform coordinates, and optical image coordinates; preprocesses the molecular luminescence image, and registers and fuses the ultrasound structural image with the molecular luminescence image according to the coordinate transformation relationship; and determines and displays the anatomical structure, tissue depth, and molecular luminescence distribution of the target imaging area.

Images