Handheld rapid three-dimensional fundus blood supply imaging system and method

Abstract

The invention discloses a handheld rapid three-dimensional fundus blood supply imaging system and method.The system comprises an upper computer, a laser excitation module, a probe, a data collection module and a space positioning module, the laser excitation module, the probe, the data collection module and the space positioning module are connected with the upper computer, and the laser excitation module conducts wavelength conversion on input pump laser through the stimulated Raman scattering principle to generate excitation light; the exciting light is then transmitted to the probe for high-speed scanning, the probe receives real-time pose feedback of the space positioning module at the same time so as to eliminate the influence of operation jitter, the data acquisition module pre-processes a photoacoustic signal acquired by the probe and then transmits the photoacoustic signal to the upper computer, and a distribution diagram is generated through an image reconstruction algorithm and analyzed. According to the invention, four clinical troubles of poor molecular specificity, depth-resolution imbalance, low clinical adaptability and deficiency of quantitative standards in the existing imaging technology are fundamentally solved, and a universal tool is provided for early screening of eye ground ischemic diseases.

Description

Technical Field

[0001] This invention belongs to the field of photoacoustic imaging technology, specifically relating to a handheld rapid three-dimensional fundus blood supply imaging system and method. Background Technology

[0002] In the early diagnosis of fundus vascular diseases, non-invasive quantitative analysis of hemodynamic parameters of the retina and choroid (blood oxygen saturation, arterial / venous flow velocity) has irreplaceable clinical value. According to statistics from the World Health Organization in 2025, among the approximately 160 million patients with diabetic retinopathy worldwide, the rate of blindness due to missed diagnosis of choroidal ischemia is as high as 30%. However, current mainstream technologies have significant limitations: fluorescence angiography (FFA) requires the injection of contrast agents and carries invasive operational risks; optical coherence tomography angiography (OCTA) cannot achieve quantitative blood oxygenation analysis; and near-infrared spectroscopy (NIRS) has insufficient penetration ability into deep choroidal vessels (depth > 800 μm) (signal-to-noise ratio attenuation > 90%).

[0003] While photoacoustic imaging technology combines the advantages of optical contrast and ultrasonic penetration depth, current patents and academic research still face three major bottlenecks: First, in terms of hardware design and signal acquisition, the signal-to-noise ratio of the choroidal layer in ultrasonic transducer arrays is less than 3dB due to edge acoustic field attenuation, which cannot meet the needs of clinical diagnosis; although common-path multimodal probes integrate OCT and photoacoustic imaging, laser scattering causes local corneal temperature rise of >2°C, posing a safety risk, and the single imaging time is limited to within 3 minutes; more importantly, more than 90% of existing technologies can only rely on speckle signals generated by venous blood flow to achieve flow velocity measurement. Because the spatial distribution of red blood cells in arterial blood flow is uniform, it cannot generate effective photoacoustic Doppler frequency shift, resulting in blind spots in the assessment of diseases such as arterial embolism. Second, in terms of algorithms and functions, deep learning reconstruction schemes take more than 8 minutes to process a single frame of 3D image, which is difficult to meet the needs of real-time intraoperative navigation; while traditional blood oxygen quantification models assume uniform distribution of tissue light flux, in reality, due to scleral scattering, the light energy attenuation in the choroidal layer reaches more than 85%, and the blood oxygen saturation measurement error exceeds 25%. Third, in terms of clinical translation, adaptive optics and MEMS scanning modules have driven up the system cost to over 2 million yuan, and the adoption rate in primary hospitals is less than 1%. At the same time, existing equipment is not capable of compensating for micro-movements of the living eye (speed > 50 μm / s), and motion artifacts cause vascular positioning deviations to exceed the resolution limit, which seriously affects the accuracy of dynamic blood flow monitoring for diseases such as glaucoma.

[0004] In summary, the industry urgently needs a novel photoacoustic imaging technology for the eye to overcome the following technological gaps: 1. Innovation in arterial blood flow detection mechanism: Overcoming signal loss caused by uniform red blood cell distribution and achieving simultaneous quantification of arteries and veins; 2. Millisecond-level eye movement compensation: Developing a low-cost optical correction module (target cost reduction of 70%) to suppress physiological motion artifacts; 3. Penetration depth-resolution balance: Compressing blood oxygenation measurement error to <5% while ensuring 5μm resolution in the choroidal layer; and addressing the shortcomings of existing photoacoustic imaging technologies such as slow mechanical scanning speed, severe motion artifacts, and low system integration. Summary of the Invention

[0005] The main objective of this invention is to provide a handheld rapid three-dimensional fundus blood supply imaging system and system to overcome the shortcomings of the prior art.

[0006] To achieve the above-mentioned objectives, the present invention adopts the following technical solution: An embodiment of the present invention provides a handheld rapid three-dimensional fundus blood supply imaging system, comprising: Host computer; The laser excitation module, connected to the host computer, is used to generate excitation light with a wavelength range of 1530nm~1540nm and input the excitation light to the probe; The probe, connected to both the laser excitation module and the host computer, is used to dynamically scan the eye with the input excitation light and generate photoacoustic signals. The data acquisition module is connected to both the probe and the host computer. It is used to preprocess the photoacoustic signal acquired by the probe and transmit it to the host computer. The host computer generates a distribution map through an image reconstruction algorithm and analyzes the distribution map. The spatial positioning module is connected to the probe, the data acquisition module and the host computer, and is used to provide real-time position and pose feedback to the probe and to correct the probe's position and pose in conjunction with the host computer. The central control unit, connected to the probe and the spatial positioning module, is used to receive scanning instructions from the host computer and pose feedback from the spatial positioning module, and generate synchronous trigger signals to the laser excitation module, the probe and the data acquisition unit to synchronously perform laser emission, probe scanning and data acquisition respectively.

[0007] In a preferred embodiment, the laser excitation module includes a stimulated Raman scattering module, which is used to convert the wavelength of the input pump laser with a wavelength shorter than that of the excitation light through stimulated Raman scattering to generate the excitation light.

[0008] In a preferred embodiment, the probe includes a housing and a galvanometer, a laser collimator, an optical lens, a separator, and an ultrasonic transducer integrated within the housing. The separator divides the interior of the housing into an upper sealed space and a lower sealed space. The galvanometer, laser collimator, and optical lens are located in the upper sealed space, and the ultrasonic transducer is located in the lower sealed space. The excitation light enters the housing through the laser collimator, is reflected by the galvanometer to the optical lens, and finally undergoes signal conversion by the ultrasonic transducer to output the photoacoustic signal.

[0009] In a preferred embodiment, the probe further includes a water bladder membrane located at the bottom of the housing, and the lower sealed space is filled with liquid during operation.

[0010] In a preferred embodiment, the image reconstruction algorithm includes generating the distribution map by coupling the spatial trajectory position relationship of the photoacoustic signal emitted by the probe; and / or, the image reconstruction algorithm is a two-dimensional imaging algorithm, which specifically includes preprocessing the three-dimensional photoacoustic signal of the probe based on a delay superposition algorithm to obtain a phased array imaging unit with a certain equivalent channel, and outputting the delayed superposition two-dimensional distribution map through pixel mapping of the image; and / or, the host computer automatically separates features through a deep convolutional network and outputs quantitative parameters for diagnostic analysis, the quantitative parameters including orientation angle and crosslinking density.

[0011] In a preferred embodiment, the spatial positioning module acquires the position and attitude information of the probe in space in real time, and performs spatial transformation on the two-dimensional image by matching it with a pre-established coordinate system; and / or, the spatial positioning module includes establishing a coordinate system, acquiring the position and attitude information of the probe in space through real-time data acquisition, performing data conversion, converting the acquired data to a pre-established coordinate system for matching, and performing spatial transformation on the two-dimensional image to obtain a three-dimensional image.

[0012] On the other hand, one embodiment of the present invention provides a handheld rapid three-dimensional fundus blood supply imaging method, comprising: S1, the laser excitation module generates excitation light with a wavelength range of 1530nm~1540nm, and inputs the excitation light to the probe; S2, the probe uses the excitation light to dynamically scan the eye and generate photoacoustic signals, while receiving real-time pose feedback from the spatial positioning module and correcting its pose in conjunction with the host computer. S3, the data acquisition module preprocesses the photoacoustic signal acquired by the probe and transmits it to the host computer. The host computer generates a distribution map through an image reconstruction algorithm and analyzes the distribution map. S4, the central control unit receives the scanning command from the host computer and the pose feedback from the spatial positioning module, and generates a synchronization trigger signal to the laser excitation module, the probe and the data acquisition unit to synchronously perform laser emission, probe scanning and data acquisition respectively.

[0013] In a preferred embodiment, in step S1, the laser excitation module converts the wavelength of the input pump laser, which has a wavelength shorter than that of the excitation light, through stimulated Raman scattering to generate the excitation light.

[0014] In a preferred embodiment, in step S3, the image reconstruction algorithm includes generating the distribution map by coupling the spatial trajectory position relationship of the photoacoustic signal emitted by the probe; and / or, the image reconstruction algorithm is a two-dimensional imaging algorithm, which specifically includes preprocessing the three-dimensional photoacoustic signal of the probe based on a delay superposition algorithm to obtain a phased array imaging unit with a certain equivalent channel, and outputting the delayed superposition two-dimensional distribution map through pixel mapping of the image; and / or, the host computer automatically separates features through a deep convolutional network and outputs quantitative parameters for diagnostic analysis, the quantitative parameters including orientation angle and crosslinking density.

[0015] In a preferred embodiment, in step S2, the spatial positioning module acquires the position and attitude information of the probe in space in real time, performs spatial transformation on the two-dimensional image by matching it with a pre-established coordinate system; and / or, the spatial positioning module includes establishing a coordinate system, acquiring the position and attitude information of the probe in space through real-time data acquisition, performing data conversion, converting the acquired data to a pre-established coordinate system for matching, and performing spatial transformation on the two-dimensional image to obtain a three-dimensional image.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention constructs a handheld three-dimensional photoacoustic imaging system for the eye: It utilizes a 1535nm pulsed laser based on stimulated Raman scattering to achieve specific targeted excitation of molecules; a handheld MEMS scanning imaging probe overcomes the volume limitations of traditional mechanical scanning; a free-space positioning device eliminates the impact of operational jitter on image quality; and a quantitative imaging algorithm based on convolutional neural networks analyzes pathological information at the molecular level. These technologies, through the interdisciplinary integration of optics, acoustics, mechanics, and artificial intelligence, fundamentally solve the four major clinical dilemmas of existing imaging technologies: poor molecular specificity, depth-resolution imbalance, low clinical adaptability, and lack of quantitative standards. This provides a universal tool for the early screening of ischemic retinal diseases.

[0017] 2. This invention employs a dual-axis MEMS galvanometer to achieve millisecond-level scanning (≤0.5 seconds / frame), increasing speed by 60 times and meeting the requirements for real-time monitoring of hemodynamics in living eyes. Furthermore, the non-contact deflection of the galvanometer, through rigid optical path coupling and a real-time eye-tracking algorithm (accuracy ±2μm), compresses the vascular positioning error to 1 / 5 of the theoretical resolution, completely eliminating heartbeat and respiratory artifacts. In addition, this invention integrates the galvanometer into the confocal photoacoustic excitation-detection path, reducing the probe volume by 70% and eliminating the need for additional calibration, laying the foundation for the development of handheld fundus imaging devices. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of a handheld rapid three-dimensional fundus blood supply imaging system according to one embodiment of this application; Figure 2a This is a schematic diagram of the internal structure of the handheld probe in one embodiment of this application. Figure 2b This is a schematic diagram of the external structure of the handheld probe in one embodiment of this application; Figure 3 This is a schematic diagram of the delay superposition algorithm in one embodiment of this application; Figure 4 This is an address mapping diagram of the delay superposition algorithm in one embodiment of this application; Figure 5 This is a flowchart illustrating a handheld rapid three-dimensional fundus blood supply imaging method according to one embodiment of this application; Figure 6 This is a photoacoustic microscopy image of the mouse brain and cecum in one embodiment of this application. Figures 7a-7d These are, respectively, two-dimensional maximum projection images of blood vessels in the mouse ear, two-dimensional maximum projection images of blood vessels in the brain, three-dimensional images of blood vessels in the ear, and three-dimensional images of blood vessels in the brain.

[0020] Figure label: 1. Outer shell, 2. Galvanometer, 3. Laser collimator, 4. Optical lens, 5. Separator, 6. Ultrasonic transducer, 7. Water bladder membrane. Detailed Implementation

[0021] The invention will be more fully understood through the following detailed description, which should be read in conjunction with the accompanying drawings. Detailed embodiments of the invention are disclosed herein; however, it should be understood that the disclosed embodiments are merely exemplary of the invention, which may be embodied in various forms. Therefore, the specific functional details disclosed herein should not be construed as limiting, but rather as the basis for the claims and as intended to teach those skilled in the art to employ the representative basis of the invention in different ways in any suitable detailed embodiment.

[0022] like Figure 1 As shown in the figure, an embodiment of the present invention discloses a handheld rapid three-dimensional fundus blood supply imaging system that achieves efficient and accurate detection through the collaborative operation of multiple modules. Specifically, it includes a host computer and a laser excitation module, a probe, a data acquisition module, and a spatial positioning module connected to the host computer. The laser excitation module first converts the wavelength of the input pump laser using stimulated Raman scattering to generate excitation light. The excitation light is then transmitted to the probe, which performs high-speed scanning while simultaneously receiving real-time pose feedback from the spatial positioning module to eliminate the effects of operational jitter. The data acquisition module preprocesses the photoacoustic signal acquired by the probe and transmits it to the host computer, where an image reconstruction algorithm generates a distribution map and analyzes it. The entire process forms a closed loop of "optical excitation - dynamic scanning - intelligent analysis," requiring only 5 seconds for a single detection, with a cost controlled at $20, providing high-resolution (50μm) molecular-level pathological information for clinical use.

[0023] Specifically, in this embodiment, the laser excitation module is based on 1535nm pulsed laser for stimulated Raman scattering to achieve specific targeted excitation of molecules. It includes a stimulated Raman scattering module, which specifically inputs a 1μm band pump laser (such as a 1064nm pump laser) into the stimulated Raman scattering module. Through the principle of stimulated Raman scattering, wavelength conversion is performed, which efficiently converts the light into the peak wavelength of specific absorption (such as the main peak 1535nm±5nm) to excite the light. This process makes the photoacoustic signal intensity reach 8.3 times that of the traditional 1064nm system.

[0024] Specifically, the quantum energy level transition process of the physical mechanism of stimulated Raman scattering is as follows: when the pump light (photon energy) of 1064 nm... When incident on a Raman-active medium, molecules absorb photons and transition to a virtual energy level, subsequently releasing energy to a Stokes level via phonon coupling. The energy conversion relationship is as follows: ; in, The difference in molecular vibrational energy levels (in hydrogen gas) ), corresponding wavelength conversion: .

[0025] The excitation light is then transmitted to the probe. This invention's probe overcomes the size limitations of traditional mechanical scanning, integrating a MEMS electromagnetic dual-mirror array and a miniature ultrasonic transducer array for high-speed scanning. Simultaneously, it receives real-time pose feedback from a spatial positioning module to eliminate operational jitter, ensuring that the deformation error of the curved surface scan is less than 0.1 mm. Specifically, in this embodiment, the probe is a handheld probe, combined with... Figure 2a and Figure 2b As shown, it specifically includes a housing 1 and a galvanometer 2, a laser collimator 3, an optical lens 4, a separator 5, and an ultrasonic transducer 6 integrated within the housing 1. The separator 5 divides the interior of the housing 1 into an upper sealed space and a lower sealed space. In this embodiment, the separator 5 is a glass plate. The galvanometer 2, laser collimator 3, and optical lens 4 are all fixed within the housing 1 and located within the upper sealed space. In this embodiment, the mirror surface of the galvanometer 2 is inclined to the horizontal plane, specifically at a 45° angle, although other angles are possible as needed. The laser collimator 3 is used to input excitation light, which is horizontal, meaning its laser direction forms a 45-degree angle with the galvanometer 2. The optical lens 4 is located downstream of the laser collimator 3 and the galvanometer 2, specifically in the middle of the housing 1, and receives the excitation light reflected from the galvanometer 2. The ultrasonic transducer 6 is located within the lower sealed space. During operation, the excitation light enters the outer shell 1 via the laser collimator 3, is reflected by the galvanometer 2 to the optical lens 4, and finally outputs a photoacoustic signal after signal conversion by the ultrasonic transducer 6. Preferably, the probe also includes a water sac membrane 7. In this embodiment, the water sac membrane 7 is located at the bottom of the outer shell 1, and the lower sealed space is filled with liquid (specifically water) during operation. Compared with the existing separate structure of the ultrasonic transducer and probe scanning mechanism, this invention integrates the galvanometer into the confocal photoacoustic excitation-detection path, breaking through the volume limitations of traditional mechanical scanning, reducing the probe volume by 70%, and achieving 100Hz high-speed scanning and 50μm longitudinal resolution with a weight of 800g, without the need for additional calibration, laying the foundation for the development of handheld fundus imaging devices.

[0026] The data acquisition module is connected to both the probe and the host computer. It preprocesses the photoacoustic signals acquired by the probe and transmits them to the host computer. The host computer then generates a distribution map using an image reconstruction algorithm and analyzes the map. Specifically, in this embodiment, the data acquisition module includes a data acquisition unit connected to the probe and the host computer. This unit preprocesses the photoacoustic signals acquired by the probe and transmits them to the host computer. The host computer then generates a distribution map using an image reconstruction algorithm and analyzes the map. In this embodiment, the image reconstruction algorithm includes generating the distribution map by coupling the spatial trajectory position relationship of the photoacoustic signals emitted by the probe; and / or, the image reconstruction algorithm is a two-dimensional imaging algorithm, specifically including preprocessing the three-dimensional photoacoustic signals from the probe based on a delay-overlay algorithm to obtain a phased array imaging unit with a certain equivalent channel, and outputting the delayed-overlay two-dimensional distribution map through pixel mapping of the image. The host computer automatically separates features using a deep convolutional network and outputs quantitative parameters for diagnostic analysis. These quantitative parameters include orientation angle and crosslinking density, achieving a diagnostic consistency of over 95%.

[0027] Specifically, image reconstruction begins with the obtained photoacoustic signal, and through coupling spatial trajectory positional relationships, ultimately achieves large-scale three-dimensional imaging. First, this invention introduces a two-dimensional imaging algorithm for a single image. This two-dimensional imaging algorithm is based on the traditional delay-summing domain, preprocessing the photoacoustic signal to obtain a phased array imaging unit with a certain number of equivalent channels. The core of the delay-summing algorithm (DAS) lies in the calculation of delay. The principle of the delay-summing algorithm is to calculate the delay of echo data from different channels, then perform delay compensation on the data of each channel to achieve time-domain alignment, and finally superimpose the aligned data to make the beam directional, maximizing the energy at the target point, such as... Figure 3 As shown.

[0028] Figure 4 This demonstrates the address mapping of the delay superposition algorithm. For the echo signal s received by the ultrasonic transducer, a three-dimensional array can be used. The letters represent the e-th transmission and the j-th signal received by the k-th array element, respectively. The algorithm's input is the three-dimensional echo signal, and the output is the superimposed radio frequency signal RF (i.e., the probe's photoacoustic signal). The number of transmitting array elements in the ultrasonic transducer is defined as N, and the number of transmitting array elements in the algorithm is the same as the number of receiving array elements. The magnitude of the radio frequency signal is defined as RF[W][H], where W represents the number of signal columns, i.e., the number of echo signal beams, and its magnitude is the same as the number of receiving array elements (W=N), with a one-to-one correspondence, also known as the scan line. This represents the number of sampling points or the sampling depth, indicating the echo amplitude at different depth locations within the scanned area. This allows for pixel mapping of the image through overlay.

[0029] The spatial positioning module is connected to the aforementioned probe, data acquisition module, and host computer. It provides real-time pose feedback to the probe and, in conjunction with the host computer, corrects the probe's pose. In this embodiment, the spatial positioning module acquires the probe's position and orientation information in space in real time, performs spatial transformation on the two-dimensional image by matching it with a pre-established coordinate system; and / or, the spatial positioning module includes establishing a coordinate system, acquiring the probe's position and orientation information in space through real-time data acquisition, performing data conversion, converting the acquired data to a pre-established coordinate system for matching, and performing spatial transformation on the two-dimensional image to obtain a three-dimensional image.

[0030] Specifically, the spatial positioning module uses data from built-in positioning sensors (such as electromagnetic sensors) to perform spatial positioning correction on the generated two-dimensional photoacoustic image. The electromagnetic sensor acquires the probe's position and attitude information in space in real time. By matching this information with a pre-established coordinate system, the two-dimensional image is spatially transformed to provide accurate spatial positioning information. This avoids image deviations caused by changes in probe position and angle, improving image accuracy. The process of spatial positioning correction and coordinate system matching includes establishing the coordinate system, acquiring sensor data, data conversion, image spatial transformation, and verification and adjustment. First, when establishing the coordinate system, a reference coordinate system needs to be selected, usually a fixed world coordinate system containing three axes (X, Y, Z) and an origin position, ensuring that the units of the coordinate system are consistent with the electromagnetic sensor output, such as meters or millimeters. Then, through real-time data acquisition, the electromagnetic sensor acquires the probe's position and attitude information in space, ensuring that the data is output in a processable format. Next, data conversion is performed, transforming the sensor data from the sensor coordinate system to the pre-established coordinate system. This typically involves translation and rotation operations to adjust the coordinates to the world coordinate system. The transformed coordinates can be obtained by applying the rotation matrix and calculating the translation vector. The image space transformation stage applies the transformed probe position to the generated two-dimensional photoacoustic image and uses interpolation methods to handle the blank areas caused by the spatial transformation.

[0031] The specific process of spatial 3D imaging includes: first, capturing the eye contour. This invention employs a deep learning pose estimation model, specifically the Heatmap method. The core of this network consists of multiple stacked "hourglass" modules, each capable of downsampling and upsampling the input image. During training, the model generates heatmaps of hand joints, with peak positions corresponding to joint locations. For example, for each joint, a heatmap of the same size as the input image (or scaled proportionally) is generated. Each pixel value in the heatmap represents the probability that the location is a joint, and the joint's position is determined by finding the maximum value in the heatmap. This method can quickly locate the various key points of the eye, connecting these feature points to form a grid shape, and simultaneously creating a two-dimensional trajectory—Path(u, v). This is a two-dimensional pixel path with no real-world meaning; to map this path to a real-world path, this invention needs to combine it with the corresponding depth map of the RGB image.

[0032] This invention uses the RealSense D435i camera, equipped with a high-resolution RGB camera and an infrared camera, capable of simultaneously acquiring color and depth images. The RGB camera has a resolution of up to 1920×1080. This invention acquires the RGB and depth images of the hand at the same moment, and combines this with the camera's intrinsic parameters to solve for the path in the actual environment for this two-dimensional pixel path transformation. The formula for the transformation of a single pixel is as follows:

[0033]

[0034]

[0035] Here, X, Y, and Z represent the actual position information of the pixel based on the camera coordinate system, (u, v) is the pixel in the image, and depth is the pixel in the depth image corresponding to (u, v). cx and cy are the camera's principal point, and fx and fy are the camera's focal lengths. These four parameters are obtained through camera calibration.

[0036] The above formula illustrates the transformation of a single pixel into the actual environment. However, for an entire path, this invention needs to discretize the path formed by these 20 feature points and create a uniform list of path points. The number of points determines the number of images acquired in this invention. This invention now demonstrates a method for discretizing a path formed by two feature points into N points. Assuming this invention has two feature points P1(x1,y1) and P2(x2,y2), then this invention needs to first calculate the path length and discretize it, as shown in the following formula:

[0037] By calculating the distance between two points, this invention can then obtain the spacing between discrete points:

[0038] Thus, the pixel coordinates of each discrete point can be obtained:

[0039]

[0040] This invention processes all pixels to obtain a path with real-world significance in the camera coordinate system, where each path point contains only (x, y, z). The handheld probe requires not only spatial position information but also end-effector attitude information (provided by the gyroscope) during operation. This invention introduces a homogeneous transformation matrix:

[0041] Here, P refers to the location information of the path point, while R is a 3x3 identity orthogonal matrix. The first to third columns represent the unit direction components of the three coordinate axes of the coordinate system about the base coordinates, which also represent the pose of the point. For the eye, the path of this invention is generally the centerline of the region with the highest probability. In this case, the pose of the point is generally vertically downward, and the default pose can be directly assigned to each path point. However, when this invention scans the eye, the scanning range of scanning the centerline alone is small, and it is necessary to scan both sides to ensure a more complete image. Therefore, the default direction is no longer the optimal scanning direction. The optimal scanning direction should be perpendicular to the hand-eye direction. Here, this invention introduces point cloud data because the point cloud data not only contains the location information of each point, but also the normal vector information of the plane formed by the point and the surrounding points. In this way, this invention can obtain the normal vector direction of the path point. This invention can use the KD-tree method to combine the actual location points obtained by combining the above RGB image with the depth map and the point cloud data to find the path in the point cloud. The coordinate system of a point cloud is generally inconsistent with the camera coordinate system. An extrinsic parameter matrix is ​​needed to transform the 3D coordinates in the camera coordinate system to the point cloud coordinate system in order to find the corresponding points in the point cloud.

[0042] A path point has a normal vector, but a single normal vector cannot represent the orientation of the probe tip. This invention requires a three-axis coordinate system, while the normal vector only has one axis. Therefore, this invention uses the normal vector as the Z-axis component of the rotation matrix R, and defines the direction components of the X and Y axes. First, this invention needs to normalize the normal vector:

[0043] Select another one with Non-parallel unit vectors The cross product yields the X-axis components:

[0044] Then, this invention will compare the x-axis and z-axis ( The y-axis component can be obtained by cross-product of the components:

[0045] Finally, this invention combines all the components into a single rotation matrix R: .

[0046] This forms a rotation matrix that can fix the posture of the robotic arm's end effector. However, in the above-mentioned selection of vectors for cross product calculation of the x-axis component, the present invention uses random selection, which leads to the inability of the present invention to control the RZ posture of the robotic end effector. Therefore, the present invention can be slightly improved by drawing a unit circle at the origin of the coordinate system and setting a variable th to represent the radius vector of the circle:

[0047] Thus, this invention obtains a unit vector in any direction of 360 degrees on the XY plane. However, the probe tip of this invention cannot run perpendicular to the Z-axis direction, so all vectors are not parallel to the normal vector. The introduced variable th can control the direction of the probe tip RZ axis, allowing the probe to scan at any angle.

[0048] Based on the path points obtained above, this invention can transform all the path points into a list of homogeneous transformation matrices. This path list refers to the path in the camera coordinate system. Now, this invention needs to transform the path into the base coordinate system to convert the two-dimensional image into three dimensions.

[0049] In this embodiment, the data acquisition module also includes a central control unit, which serves as the core of the system scheduling. The central control unit is connected to the probe and the spatial positioning module and is used to receive scanning instructions from the host computer and pose feedback from the spatial positioning module. It generates a synchronization trigger signal with sub-millisecond precision to the laser excitation module, the probe, and the data acquisition unit, so as to synchronously perform laser emission, probe scanning, and data acquisition, and coordinate the timing of laser emission, probe scanning, and data acquisition.

[0050] like Figure 5 As shown, corresponding to the above system, the handheld rapid three-dimensional fundus blood supply imaging method disclosed in this invention specifically includes the following steps: S1, the laser excitation module generates excitation light with a wavelength range of 1530nm~1540nm, and inputs the excitation light to the probe; S2, the probe uses the excitation light to dynamically scan the eye and generate photoacoustic signals, while receiving real-time pose feedback from the spatial positioning module and correcting its pose in conjunction with the host computer. S3, the data acquisition module preprocesses the photoacoustic signal acquired by the probe and transmits it to the host computer. The host computer generates a distribution map through an image reconstruction algorithm and analyzes the distribution map. S4, the central control unit receives the scanning command from the host computer and the pose feedback from the spatial positioning module, and generates a synchronization trigger signal to the laser excitation module, the probe and the data acquisition unit to synchronously perform laser emission, probe scanning and data acquisition respectively.

[0051] The specific working process of steps S1 to S4 can be referred to the descriptions in the above modules, and will not be repeated here.

[0052] According to the above-described invention, Figure 6 This image shows the photoacoustic microscopic imaging effect of the mouse brain and cecum in one embodiment of this application, reflecting the imaging capability of the probe. The overall image is divided into two parts: a three-dimensional image generated by the system and a two-dimensional image generated by the system. Figures A, B, C, and D show the maximum two-dimensional projections of the system in the brain, cecum, kidney, and liver, respectively. Figures E and F show the three-dimensional images of blood vessels in the ear and brain. The three sub-figures I-III in Figures AD represent the actual image, structural projection, and blood oxygen content projection, respectively. Figures 7a-7d These are photoacoustic microscopic imaging results of the mouse ear and brain in one embodiment of this application, showing the two-dimensional maximum projection of blood vessels in the mouse ear, the two-dimensional maximum projection of blood vessels in the brain, a three-dimensional image of blood vessels in the ear, and a three-dimensional image of blood vessels in the brain. These experimental images demonstrate that the structure proposed in this invention performs well in multi-scene photoacoustic imaging applications.

[0053] This invention has the following technical advantages: 1. This invention constructs a handheld three-dimensional photoacoustic imaging system for the eye: 1535nm pulsed laser based on stimulated Raman scattering enables specific targeted excitation of molecules; the handheld MEMS scanning imaging probe overcomes the volume limitations of traditional mechanical scanning; the free-space positioning device eliminates the impact of operational jitter on image quality; and the quantitative imaging algorithm based on convolutional neural networks analyzes pathological information at the molecular level. These technologies, through the interdisciplinary integration of optics, acoustics, mechanics, and artificial intelligence, fundamentally solve the four major clinical dilemmas of existing imaging technologies: poor molecular specificity, depth-resolution imbalance, low clinical adaptability, and lack of quantitative standards, providing a universal tool for the early screening of ischemic diseases of the fundus. 2. This invention employs a dual-axis MEMS galvanometer to achieve millisecond-level scanning (≤0.5 seconds / frame), increasing speed by 60 times. This meets the real-time monitoring requirements of hemodynamics in living eyes, solving the problem that traditional mechanical translation stages take >30 seconds / frame to scan and cannot capture dynamic blood flow. Furthermore, the non-contact deflection of the galvanometer, through rigid optical path coupling and a real-time eye-tracking algorithm (accuracy ±2μm), compresses the vascular positioning error to 1 / 5 of the theoretical resolution, completely eliminating heartbeat and respiratory artifacts. Additionally, this invention integrates the galvanometer into the confocal photoacoustic excitation-detection path, reducing the probe volume by 70% and eliminating the need for additional calibration, laying the foundation for the development of handheld fundus imaging devices.

[0054] Although the invention has been described with reference to illustrative embodiments, those skilled in the art will understand that various other changes, omissions, and / or additions can be made without departing from the spirit and scope of the invention, and that elements of the described embodiments can be substituted with substantially equivalents. Furthermore, many modifications can be made without departing from the scope of the invention to adapt particular situations or materials to the teachings of the invention. Therefore, this document is not intended to limit the invention to the specific embodiments disclosed for carrying out the invention, but rather to include all embodiments falling within the scope of the appended claims.

Claims

1. A hand-held rapid three-dimensional fundus blood supply imaging system, characterized in that, The system comprises: a host computer; a laser excitation module connected to the host computer, configured to generate excitation light with a wavelength ranging from 1530 nm to 1540 nm and input the excitation light into a probe; the probe connected to the laser excitation module and the host computer, configured to perform dynamic scanning on an eye part using the input excitation light and generate photoacoustic signals; a data acquisition module connected to the probe and the host computer, configured to transmit the photoacoustic signals obtained by the probe to the host computer after preprocessing, and the host computer generates a distribution map through an image reconstruction algorithm and analyzes the distribution map; a spatial positioning module connected to the probe, the data acquisition module and the host computer, configured to provide real-time pose feedback to the probe and correct the pose of the probe in combination with the host computer; a central control unit connected to the probe and the spatial positioning module, configured to receive scanning instructions from the host computer and pose feedback from the spatial positioning module, and generate a synchronous trigger signal to the laser excitation module, the probe and the data acquisition unit to perform laser emission, probe scanning and data acquisition respectively.

2. The handheld rapid three-dimensional fundus blood supply imaging system according to claim 1, characterized in that: The laser excitation module comprises a stimulated Raman scattering module, configured to convert the wavelength of the input pump laser with a wavelength less than the excitation light through the principle of stimulated Raman scattering to generate the excitation light.

3. The handheld rapid three-dimensional fundus blood supply imaging system according to claim 1, characterized in that: The probe comprises a shell, a galvanometer, a laser collimator, an optical lens, a partition sheet and an ultrasonic transducer integrated in the shell, the partition sheet divides the shell into an upper airtight space and a lower airtight space, the galvanometer, the laser collimator and the optical lens are located in the upper airtight space, and the ultrasonic transducer is located in the lower airtight space; the excitation light enters the shell through the laser collimator, is reflected to the optical lens by the galvanometer, and is finally output as the photoacoustic signal after signal conversion by the ultrasonic transducer.

4. The handheld rapid three-dimensional fundus blood supply imaging system according to claim 3, characterized in that: The probe further comprises a water sac membrane, which is located at the bottom of the shell, and the lower airtight space is filled with liquid during operation.

5. The handheld rapid three-dimensional fundus blood supply imaging system according to claim 1, characterized in that: The image reconstruction algorithm comprises generating the distribution map through the coupling space trajectory position relationship of the photoacoustic signals emitted by the probe; and / or, the image reconstruction algorithm is a two-dimensional imaging algorithm, which specifically comprises preprocessing the three-dimensional photoacoustic signals of the probe based on a delay-and-sum algorithm, obtaining a phased array imaging unit with a certain equivalent channel, and outputting a two-dimensional distribution map after delay-and-sum through pixel mapping of an image; and / or, the host computer automatically separates features through a deep convolution network, and outputs quantitative parameters for diagnostic analysis, the quantitative parameters comprising an orientation angle and a crosslinking density.

6. The handheld rapid three-dimensional fundus blood supply imaging system according to claim 1, characterized in that: The spatial positioning module obtains position and attitude information of the probe in space in real time, matches with a pre-established coordinate system, and performs spatial transformation on a two-dimensional image; and / or, the spatial positioning module comprises establishing a coordinate system, obtaining position and attitude information of the probe in space through real-time data acquisition, converting data, matching the converted data with a pre-established coordinate system, and performing spatial transformation on a two-dimensional image to obtain a three-dimensional image.

7. A handheld rapid three-dimensional fundus blood supply imaging method, characterized in that, The method comprises: S1, the laser excitation module generates excitation light with a wavelength range of 1530nm-1540nm, and inputs the excitation light into the probe; S2, the probe uses the excitation light to dynamically scan the eye and generate photoacoustic signals, and simultaneously receives real-time pose feedback of the spatial positioning module and corrects the pose in combination with the host computer; S3, the data acquisition module transmits the photoacoustic signals obtained by the probe after preprocessing to the host computer, and the host computer generates a distribution map by an image reconstruction algorithm and analyzes the distribution map; S4, the central control unit receives the scanning instruction of the host computer and the pose feedback of the spatial positioning module, generates a synchronous trigger signal to the laser excitation module, the probe and the data acquisition unit respectively to synchronize laser emission, probe scanning and data acquisition.

8. The handheld rapid three-dimensional fundus blood supply imaging method according to claim 7, characterized in that: In S1, the laser excitation module converts the input pump laser with a wavelength less than the excitation light through stimulated Raman scattering principle to generate the excitation light.

9. The handheld rapid three-dimensional fundus blood supply imaging method according to claim 7, characterized in that: In S3, the image reconstruction algorithm includes generating the distribution map by coupling the spatial trajectory position relationship of the photoacoustic signals emitted by the probe; and / or, the image reconstruction algorithm is a two-dimensional imaging algorithm, which specifically includes preprocessing the three-dimensional photoacoustic signals of the probe based on the delay and superposition algorithm to obtain a phased array imaging unit with a certain equivalent channel, and outputting a two-dimensional distribution map after delay and superposition through image pixel mapping; and / or, the host computer automatically separates features through a deep convolution network to output quantitative parameters for diagnostic analysis, the quantitative parameters including orientation angle and crosslinking density.

10. The handheld rapid three-dimensional fundus blood supply imaging method according to claim 7, characterized in that: In S2, the spatial positioning module obtains the position and attitude information of the probe in space in real time, matches with the pre-established coordinate system, and performs spatial transformation on the two-dimensional image; and / or, the spatial positioning module includes establishing a coordinate system, obtaining the position and attitude information of the probe in space through real-time data acquisition, performing data conversion, converting the obtained data into the pre-established coordinate system for matching, and performing spatial transformation on the two-dimensional image to obtain a three-dimensional image.

Images