Bimodal real-time three-dimensional imaging system and method based on LED light source

Abstract

The invention discloses a bimodal real-time three-dimensional imaging method based on an LED light source, which is applied to a bimodal real-time three-dimensional imaging system based on the LED light source, and comprises the following steps: step 1, calibrating a calibration plate on an ultrasonic transducer, and obtaining a transformation matrix from a two-dimensional image coordinate system to a calibration plate coordinate system; step 2, moving an ultrasonic transducer to scan an imaging target, synchronously acquiring and reconstructing a two-dimensional ultrasonic image and a two-dimensional photoacoustic image, and tracking and recording pose information of the calibration plate in real time in an optical positioning mode, namely a transformation matrix from a calibration plate coordinate system to a camera coordinate system; step 3, based on the transformation matrix, mapping a two-dimensional ultrasonic imaging result and a two-dimensional photoacoustic imaging result obtained in real time to the same three-dimensional space through coordinate transformation; and 4, respectively adjusting the color and transparency parameters of the ultrasonic and opto-acoustic two modal images, fusing and rendering in real time, and displaying the bimodal three-dimensional image.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional imaging, and in particular to a dual-modal real-time three-dimensional imaging system and method based on an LED light source. Background Technology

[0002] Compared to two-dimensional ultrasound imaging, three-dimensional ultrasound imaging can provide more comprehensive information on the location, morphology, and volume of lesions, and has broad application prospects. Achieving three-dimensional reconstruction typically relies on positioning technologies such as optical, mechanical, electromagnetic, or acoustic techniques to obtain probe pose information, and then stitching two-dimensional tomographic images into three-dimensional data. Among these, optical positioning technology is widely used in handheld probe-based three-dimensional ultrasound imaging systems due to its good portability and low cost. Photoacoustic imaging, as an emerging medical imaging technology, combines the advantages of high resolution and safety and reliability, and is suitable for imaging various surface organs and tissues. Its imaging principle is as follows: after pulsed light irradiates the sample, the tissue absorbs the light energy and generates a photoacoustic effect, thereby exciting a sound pressure signal; by periodically receiving the acoustic signal corresponding to each light pulse through an ultrasound transducer, a photoacoustic image can be reconstructed. Based on this mechanism, photoacoustic tomography systems are easily integrated with ultrasound imaging systems to form multimodal imaging systems, enabling the simultaneous acquisition of anatomical and functional information of tissues. Currently, high-resolution photoacoustic imaging systems mostly use pulsed lasers as excitation sources. However, these laser devices are expensive and have limited pulse repetition frequencies, which restricts their application in real-time imaging. In contrast, LED pulsed light sources have advantages such as low cost and high repetition frequency, making them more suitable for supporting the realization of real-time photoacoustic imaging. Summary of the Invention

[0003] Purpose of the invention: The technical problem to be solved by the present invention is to provide a dual-modal real-time three-dimensional imaging system and method based on LED light source, which addresses the shortcomings of the prior art.

[0004] To address the aforementioned technical problems, this invention discloses a dual-modal real-time three-dimensional imaging system based on an LED light source. The system includes an ultrasonic transducer, an LED array, a calibration board, an optical camera, an FPGA, and a pulse drive circuit.

[0005] The calibration plate and the LED array are fixed to the ultrasonic transducer;

[0006] The optical camera is mounted on a fixed tripod;

[0007] The pulse driving circuit is connected to the LED array;

[0008] The FPGA is connected to the pulse driving circuit and the optical camera, respectively.

[0009] The computer controls the programmable ultrasound research platform and the FPGA respectively.

[0010] The calibration plate is printed with an asymmetrical circular calibration pattern and four ArUco codes located in the corners.

[0011] The LED array consists of LEDs of the same model, with high rated power, rise and fall times (i.e., the time required to light up and turn off) in the nanosecond range, high radiation intensity, and emission wavelengths within the wavelength range that can produce photoacoustic phenomena. All LEDs are uniformly encapsulated in a light-transmitting and waterproof acrylic 3D printed structure.

[0012] The pulse driving circuit has a repetition frequency on the order of kHz and a peak pulse output voltage greater than 55V. It drives the LED array circuit to emit pulse light at a specified frequency. In each working cycle, the LED is continuously on for nanoseconds and is off at other times. All LEDs are simultaneously on and off.

[0013] A dual-modal real-time three-dimensional imaging method based on an LED light source, the method being applied to a dual-modal real-time three-dimensional imaging system based on an LED light source as described in claim 1, comprising the following steps:

[0014] Step 1: Calibrate the calibration plate on the ultrasonic transducer and obtain the transformation matrix Rt from the two-dimensional image coordinate system to the calibration plate coordinate system. i2w This matrix can be decomposed into the corresponding rotation matrix R. i2w Translation matrix t i2w This is used for subsequent real-time calculation of the pose information of the two-dimensional image.

[0015] Step 2: Move the ultrasonic transducer to scan the imaging target, simultaneously acquire and reconstruct two-dimensional ultrasonic images and two-dimensional photoacoustic images, and use optical positioning to track and record the pose information Rt of the calibration plate in real time. w2c ;

[0016] Step 3, based on the transformation matrix Rt i2w and Rt w2c By transforming coordinates, the real-time acquired two-dimensional ultrasound imaging results and two-dimensional photoacoustic imaging results are mapped to the same three-dimensional space;

[0017] Step 4: Adjust the color and transparency parameters of the ultrasound and photoacoustic images respectively, and then fuse, render and display the dual-modal 3D image in real time.

[0018] The method of moving the ultrasonic transducer in step 2 is freehand handheld, and the acquisition steps include:

[0019] Step 2-1: Move the ultrasonic transducer according to the target area of ​​the sample. The sample is the target of the required three-dimensional imaging.

[0020] Step 2-2: During the movement, the optical camera takes pictures of the calibration board at a set frequency, calculates and records the calibration board's pose information in real time, and simultaneously transmits a trigger signal.

[0021] Steps 2-3: After receiving the trigger signal, the FPGA controls the ultrasonic transducer to perform one acquisition to reconstruct a two-dimensional ultrasonic image.

[0022] In steps 2-4, the FPGA then sends a drive signal to the pulse drive circuit to drive the LED array to emit light. The light illuminates the sample, the sample absorbs the light energy and generates a photoacoustic effect to obtain a photoacoustic signal. The FPGA controls the ultrasonic transducer to start receiving the photoacoustic signal synchronously. The photoacoustic signal is reconstructed to obtain a two-dimensional photoacoustic image. The specific reconstruction method is a delay superposition algorithm. By calculating the distance from the target point to each receiving unit of the ultrasonic transducer, and combining the speed of the transmission medium and the acquisition frequency of the ultrasonic transducer array, the receiving time and signal strength corresponding to each unit are determined. Finally, all signals are superimposed to obtain the reconstructed signal strength of the target point.

[0023] The specific reconstruction method of the two-dimensional ultrasound image described in steps 2-3 is a beamforming technique based on synthetic aperture. The array elements of the ultrasound transducer sequentially transmit ultrasound pulses. After each transmission, all array elements are used to receive the complete original radio frequency data. After all array elements have transmitted, a frame of two-dimensional ultrasound image is reconstructed through a delay superposition algorithm.

[0024] The optical positioning method described in step 2 involves real-time tracking, including: detecting the ArUco code in the image captured by the camera, comparing its position and pixel size, and then obtaining the length, width, center distance, and other features of the calibration plate through proportional transformation, so as to calculate the pose of the calibration plate relative to the optical camera.

[0025] Specifically, in step 1, when moving the ultrasonic transducer, it is necessary to ensure that the calibration plate is always within the camera's field of view and is not obstructed. The optical camera continuously captures images of the calibration plate to record changes in the ultrasonic transducer's position. First, the asymmetric circular checkerboard points on the calibration plate are detected in the initial frame image, and the center coordinates of each point are obtained. If the detection fails, the next frame is read and the attempt is repeated. After successful detection, intermediate frame detection begins. Using the center points obtained from the initial frame as the center, multiple square regions are extracted (usually 28, the circular calibration plate is 4*7, with 4 circles per row and 7 circles per column, and a square is extracted at the center of each circle). Within each square region, the center coordinates are precisely located using connected component analysis. After successfully obtaining all center coordinates, the camera's extrinsic parameters, i.e., the rotation matrix R from the calibration plate to the camera, are calculated in real time. w2c Translation matrix t w2c .

[0026] The specific process of coordinate transformation in step 3 is as follows: based on the transformation matrix R from the pre-calibrated two-dimensional image coordinate system to the calibration plate coordinate system. ti2w And the transformation matrix Rt from the calibration board to the camera coordinate system acquired in real time. w2c The algorithm transforms all pixels of the two-dimensional ultrasound image and the two-dimensional photoacoustic image into the camera coordinate system, and assigns values ​​to the three-dimensional reconstruction voxels in real time through the reconstruction algorithm to realize the three-dimensional reconstruction of the target.

[0027] Specifically, the acquired ultrasonic tomographic and photoacoustic tomographic results are mapped into three-dimensional space through coordinate transformation, i.e., using the transformation matrix Rt. i2w With Rt w2c Transform the 2D image points to the camera coordinate system. Let the rotation matrix of the calibration plate relative to the camera for the j-th acquired image be... Translation matrix A pixel in the image has coordinates [u, v, 0]. T Then the coordinates of that point in the camera coordinate system are [x] c ,y c ,z c ] T Determined by the following formula:

[0028]

[0029] After converting all the pixels of the two-dimensional image to the camera coordinate system in the manner described above, the three-dimensional reconstruction voxels are assigned values ​​in real time through the reconstruction algorithm to achieve three-dimensional reconstruction of the target.

[0030] The dual-modal 3D image rendering mentioned in step 4 refers to the independent rendering of 3D voxels of ultrasound and photoacoustic modes using different colors and transparency, so as to achieve simultaneous, real-time, and clear display of dual-modal images.

[0031] Beneficial effects:

[0032] This invention proposes a dual-modal real-time three-dimensional imaging method based on LED light source. Combining the advantages of low cost and high repetition frequency of LED pulse light source with optical positioning technology, the method achieves real-time, dual-modal, three-dimensional reconstruction of the imaging target by scanning with a handheld ultrasonic transducer. This effectively overcomes the limitations of incomplete information in single-modal imaging and poor real-time performance of laser light source, providing intuitive and three-dimensional imaging results and making up for the shortcomings of existing technologies. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the dual-modal real-time three-dimensional imaging process of the present invention.

[0034] Figure 2 This is a schematic diagram of the dual-modal real-time three-dimensional imaging system involved in the present invention.

[0035] Figure 3 This is a schematic diagram of the calibration plate involved in the present invention.

[0036] Figure 4 This is a schematic diagram of the timing control of the dual-modal real-time three-dimensional imaging system involved in this invention.

[0037] Figure 5 This is a schematic diagram of coordinate transformation for 3D reconstruction.

[0038] Figure 6 This is a schematic diagram of the dual-modal three-dimensional imaging results of the phantom in one embodiment.

[0039] in, Figure 2 The numbers in the diagram are as follows: 1 represents the imaging target, 2 represents the ultrasonic transducer, 3 represents the LED array, 4 represents the calibration board, 5 represents the optical camera, 6 represents the FPGA, 7 represents the pulse drive circuit, 8 represents the Verasonics programmable ultrasonic research platform, and 9 represents the computer. Detailed Implementation

[0040] This invention discloses a dual-modal real-time three-dimensional imaging method based on an LED light source: as follows Figure 2 As shown:

[0041] The system includes an ultrasonic transducer, an LED array, a calibration board, an optical camera, an FPGA, and a pulse drive circuit.

[0042] The calibration plate and the LED array are fixed to the ultrasonic transducer;

[0043] The optical camera is mounted on a fixed tripod;

[0044] The pulse driving circuit is connected to the LED array;

[0045] The FPGA is connected to the pulse driving circuit and the optical camera, respectively.

[0046] The computer controls the programmable ultrasound research platform and the FPGA respectively.

[0047] like Figure 1 As shown, this embodiment demonstrates the application of the present invention in the imaging of superficial lymphatic vessels and blood vessels in the human body, including:

[0048] Step 1: Calibrate the calibration plate on the ultrasonic transducer and obtain the transformation matrix Rt from the two-dimensional image coordinate system to the calibration plate coordinate system. i2w This matrix can be decomposed into the corresponding rotation matrix R. i2w Translation matrix t i2w This is used for subsequent real-time calculation of the pose information of the two-dimensional image;

[0049] In this example, the ultrasound transducer uses a one-third annular concave array ultrasound transducer (aperture angle approximately 110°), containing 64 array elements, with a radius of 7.5 cm. The transducer has a center frequency of 2 MHz, a bandwidth of 400 kHz, dimensions of 12 mm × 2.048 mm, and is made of PZT. The concave array ultrasound transducer has a large contact surface and a large imaging depth, making it ideal for covering the undulating human body surface and scanning the underlying vascular and lymphatic networks.

[0050] In this example, the calibration plate measures 60mm × 85mm and is 3mm thick. The diameter of the circular calibration plate pattern is 4mm, and the center-to-center distance between the circular patterns in the same row is 12mm. The calibration plate is tightly fixed to the ultrasonic transducer using a 3D-printed bracket, ensuring that its relative position to the ultrasonic transducer remains unchanged during the scanning process.

[0051] After the calibration board is installed, it needs to be calibrated. The calibration method is N-line calibration, which obtains the transformation matrix Rt from the two-dimensional image coordinate system to the calibration board coordinate system. i2w This matrix can be decomposed into the corresponding rotation matrix R. i2w Translation matrix t i2w , where represents the corresponding rotation and translation amounts, used for subsequent real-time calculation of the pose information of the 2D image. This step only needs to be performed once.

[0052] In this example, a high-resolution camera (MV-CH650-90XM, Hikrobot, China) is used and fixed at a suitable focusing distance from the scanning area to ensure that the calibration board can be continuously and unobstructedly photographed during the scanning process.

[0053] The number and arrangement of LEDs in the LED array can be adjusted according to the light intensity requirements. In this example, the LED array is fixed in an arc shape on the side of the concave ultrasonic transducer to ensure that the illumination field can cover the entire imaging area of ​​the probe. It consists of four parallel LEDs, each with 50 LEDs connected in series. The LED array is simultaneously turned on and off, emitting pulsed light. The LED array emits light at a wavelength of 940nm, which has good optical absorption characteristics for hemoglobin and carbon nanotubes. The divergence angle is 70°, the size of the emitting area is 0.73mm × 0.73mm, the rise time is 20ns, the fall time is 16ns, the forward instantaneous conduction voltage drop is 2.2V, the maximum forward instantaneous current can reach 2A, and the radiation intensity is 680mW / sr.

[0054] The pulse driving circuit is connected to the LED array via a high-frequency signal transmission line, which has a shielding layer to reduce interference to the pulse signal. To drive the LED array and ensure high-resolution photoacoustic imaging, the peak pulse voltage output by the driving circuit should be greater than 55V, the pulse width should be 50ns, and the repetition frequency should be 10kHz.

[0055] Step 2: Move the ultrasonic transducer to scan the imaging target, simultaneously acquire and reconstruct two-dimensional ultrasonic images and two-dimensional photoacoustic images, and use optical positioning to track and record the pose information Rt of the calibration plate in real time. w2c ;

[0056] In this case, with the approval of the ethics committee and informed consent from the volunteers, the lower leg, forearm, or neck area to be scanned was exposed. To enhance the contrast of lymphatic vessel imaging, a small amount of nano-carbon contrast agent was injected subcutaneously at the distal end before scanning.

[0057] The operator holds the installed ultrasonic transducer, applies coupling gel to the target area (e.g., lower leg) of the volunteer, and then moves freely to scan. During the scan, the calibration plate is kept within the camera's field of view at all times.

[0058] The camera continuously captures images of the calibration board at a frequency of 15Hz. The ArUco calibration block detection algorithm is used to detect the ArUco code and dot pattern in real time, and the precise pose of the calibration board relative to the camera at each moment is calculated using the rotation matrix R. w2c Translation matrix t w2c express.

[0059] Each time the camera captures a frame, it sends a trigger signal to the FPGA. Upon receiving the signal, the FPGA immediately controls the ultrasonic transducer to perform acquisition. The specific method for reconstructing the two-dimensional ultrasonic image at this time is a beamforming technique based on synthetic aperture. The system controls all array elements to emit ultrasonic pulses sequentially. After each emission, all array elements receive the complete original radio frequency data, and a frame of two-dimensional ultrasonic image is reconstructed through a delay superposition algorithm.

[0060] After completing an ultrasound acquisition, the FPGA immediately sends a drive signal to the pulse drive circuit. The drive circuit drives the entire LED array to emit a nanosecond-level light pulse. The light pulse is absorbed by biological tissue (hemoglobin in blood vessels or carbon nanotubes in lymphatic vessels) to generate a photoacoustic signal, which is then received by the same concave array transducer. After receiving 500 photoacoustic signals, the signals are averaged, and then a two-dimensional photoacoustic image is reconstructed in real time using a delay superposition algorithm.

[0061] The above process is performed in real time and repeatedly according to the camera frame rate, and its timing logic is as follows: Figure 4 As shown.

[0062] Step 4, based on the transformation matrix Rt i2w and Rt w2c By transforming coordinates, the real-time acquired two-dimensional ultrasound imaging results and two-dimensional photoacoustic imaging results are mapped to the same three-dimensional space;

[0063] All acquired two-dimensional ultrasound and photoacoustic images are correlated with pose information at the same time. and Association, based on pre-obtained R i2w and t i2w According to the coordinate transformation formula in the instruction manual, each pixel in each frame of two-dimensional image is mapped to a common three-dimensional coordinate system and filled into a predefined three-dimensional volume data grid of size 200×200×400, forming three-dimensional ultrasonic volume data and three-dimensional photoacoustic volume data respectively. The three-dimensional volume data is continuously updated.

[0064] Step 4: Adjust the color and transparency parameters of the ultrasound and photoacoustic images respectively, and then fuse, render and display the dual-modal 3D image in real time.

[0065] The three-dimensional ultrasound data is rendered in grayscale and semi-transparent mode to clearly display anatomical structures (such as muscles and tissue boundaries) and blood vessels; at the same time, the three-dimensional photoacoustic data is rendered in color (such as red or green) and highlighted to highlight the strong photoacoustic signals generated by the lymphatic vessels injected with nano-carbon and the photoacoustic signals of the blood vessels.

[0066] Ultimately, the dual-modal 3D images are displayed in real time on the screen as a fused, 360-degree rotatable 3D model, clearly showing the spatial distribution, orientation, and positional relationship of blood vessels and lymphatic vessels with surrounding tissues. Furthermore, the images from both modalities can distinguish between lymphatic vessels and blood vessels, aiding in lymphatic vessel localization and providing rich and intuitive imaging information for assessing lymphedema and peritumoral angiogenesis.

[0067] Figure 6 The imaging results of the phantom model obtained by the method of the present invention are shown. The phantom model was prepared by placing three polyethylene tubes with an inner diameter of 0.5 mm in an agar block, two of which were injected with a 1:50 concentration of nano-carbon solution. Figure 6 The target tube morphology is clearly visible as continuous, complete, and three-dimensional, with a high signal-to-noise ratio and almost no background noise interference. Furthermore, the red photoacoustic data distinguishes the carbon nanotubes exhibiting photoacoustic effects, while the grayscale ultrasonic data represents the target tube without contents. The results demonstrate that the method of this invention can effectively and simultaneously display images of both ultrasonic and photoacoustic modalities, achieving perfect spatial matching and providing more comprehensive information than a single-modal approach, thus enabling clear localization and observation of the target structure.

[0068] This invention provides a dual-modal real-time three-dimensional imaging system and method based on an LED light source. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A dual-modal real-time three-dimensional imaging system based on an LED light source, characterized in that, The system includes an ultrasonic transducer, an LED array, a calibration board, an optical camera, an FPGA, a pulse drive circuit, a programmable ultrasonic research platform, and a computer. The calibration plate and the LED array are fixed to the ultrasonic transducer; The optical camera is mounted on a fixed tripod; The pulse driving circuit is connected to the LED array; The FPGA is connected to the pulse driving circuit and the optical camera, respectively. The computer controls the programmable ultrasound research platform and the FPGA respectively.

2. The dual-modal real-time three-dimensional imaging system based on an LED light source according to claim 1, characterized in that, The calibration plate is printed with an asymmetrical circular calibration pattern and an ArUco code located in the corner.

3. The dual-modal real-time three-dimensional imaging system based on an LED light source according to claim 1, characterized in that, The LEDs in the LED array are of the same type, and their rise and fall times (i.e., the time required to light up and turn off) are in the nanosecond range. They have high radiation intensity and their emission wavelengths are within the wavelength range that can produce photoacoustic phenomena.

4. A dual-modal real-time three-dimensional imaging system based on an LED light source according to claim 3, characterized in that, The pulse driving circuit has a repetition frequency on the order of kHz, and the pulse output voltage is higher than the threshold. It drives the LED array circuit to emit pulse light at a specified frequency. In each working cycle, the continuous conduction time of the LED is on the order of nanoseconds, and it is in the off state at other times. All LEDs are simultaneously on and off.

5. A dual-modal real-time three-dimensional imaging method based on an LED light source, characterized in that, The method, applied to a dual-modal real-time three-dimensional imaging system based on an LED light source as described in claim 1, includes the following steps: Step 1: Calibrate the calibration plate on the ultrasonic transducer and obtain the transformation matrix Rt from the two-dimensional image coordinate system to the calibration plate coordinate system. i2w ; Step 2: Move the ultrasonic transducer to scan the imaging target, simultaneously acquire and reconstruct two-dimensional ultrasonic images and two-dimensional photoacoustic images, and use optical positioning to track and record the pose information of the calibration plate in real time, i.e., the transformation matrix Rt from the calibration plate coordinate system to the camera coordinate system. w2c ; Step 3, based on the transformation matrix Rt i2w and transformation matrix Rt w2c By transforming coordinates, the real-time acquired two-dimensional ultrasound imaging results and two-dimensional photoacoustic imaging results are mapped to the same three-dimensional space; Step 4: Adjust the color and transparency parameters of the ultrasound and photoacoustic images respectively, and then fuse, render and display the dual-modal 3D image in real time.

6. The dual-modal real-time three-dimensional imaging method based on an LED light source according to claim 5, characterized in that, The method of moving the ultrasonic transducer in step 2 is freehand handheld, and the acquisition steps include: Step 2-1: Move the ultrasonic transducer according to the target area of ​​the sample. Step 2-2: During the movement, the optical camera takes pictures of the calibration board at a set frequency, calculates and records the calibration board's pose information in real time, and simultaneously transmits a trigger signal. Steps 2-3: After receiving the trigger signal, the FPGA controls the ultrasonic transducer to perform one acquisition to reconstruct a two-dimensional ultrasonic image. In steps 2-4, the FPGA then sends a drive signal to the pulse drive circuit to drive the LED array to emit light. The light illuminates the sample, the sample absorbs the light energy and generates a photoacoustic effect to obtain a photoacoustic signal. The FPGA controls the ultrasonic transducer to start receiving the photoacoustic signal synchronously. The photoacoustic signal is reconstructed to obtain a two-dimensional photoacoustic image. The specific reconstruction method is a delay superposition algorithm. By calculating the distance from the target point to each receiving unit of the ultrasonic transducer, and combining the speed of the transmission medium and the acquisition frequency of the ultrasonic transducer array, the receiving time and signal strength corresponding to each unit are determined. Finally, all signals are superimposed to obtain the reconstructed signal strength of the target point.

7. The dual-modal real-time three-dimensional imaging method based on an LED light source according to claim 6, characterized in that, The specific reconstruction method of the two-dimensional ultrasound image described in steps 2-3 is a beamforming technique based on synthetic aperture. The array elements of the ultrasound transducer sequentially emit ultrasound pulses. After the emission, all array elements are used to receive the complete original radio frequency data. After all array elements have emitted, a frame of two-dimensional ultrasound image is reconstructed through a delay superposition algorithm.

8. The dual-modal real-time three-dimensional imaging method based on an LED light source according to claim 5, characterized in that, The optical positioning method described in step 2 involves real-time tracking, including: detecting the ArUco code in the image captured by the camera, comparing its position and pixel size, and then obtaining the length, width, and center distance features of the calibration plate through proportional transformation, so as to calculate the pose of the calibration plate relative to the optical camera.

9. A dual-modal real-time three-dimensional imaging method based on an LED light source according to claim 5, characterized in that, The specific process of coordinate transformation described in step 3 is as follows: based on the transformation matrix Rt from the pre-calibrated two-dimensional image coordinate system to the calibration plate coordinate system. i2w And the transformation matrix Rt from the calibration board to the camera coordinate system acquired in real time. w2c The algorithm transforms all pixels of the two-dimensional ultrasound image and the two-dimensional photoacoustic image into the camera coordinate system, and assigns values ​​to the three-dimensional reconstruction voxels in real time through the reconstruction algorithm to realize the three-dimensional reconstruction of the target.

10. A dual-modal real-time three-dimensional imaging method based on an LED light source according to claim 5, characterized in that, The dual-modal 3D image rendering mentioned in step 4 refers to the independent rendering of 3D voxels of ultrasound and photoacoustic modes using different colors and transparency, so as to achieve simultaneous and real-time display of dual-modal images.

Images