Quantitative laser speckle blood flow imaging method based on neuromorphic visual sensor

Abstract

The invention discloses a quantitative laser speckle blood flow imaging method based on a neuromorphic visual sensor, and relates to the technical field of biomedical engineering. The method solves the problem that static scattered light caused by physiological behaviors and electronic noise of imaging equipment have adverse effects on blood flow imaging quality in a traditional laser speckle contrast imaging method, and comprises the following steps: step 1, using a helium-neon laser as a light source, and irradiating a scattering medium with an expanded laser beam to form a speckle pattern; 2, a microfluidic injector is adopted, two parameters of the inner diameter and the flow of the injector are set, and the flow speed range is controlled; 3, asynchronously sensing the change of the light intensity of the speckle pattern by using an event camera, and shooting the speckle pattern by using a frame camera; and 4, acquiring event stream data by the event camera, and processing the data by using an algorithm. Real-time and high-resolution blood flow imaging and flow velocity calculation are achieved, and the inherent limitation of a traditional method is overcome.

Description

Technical Field

[0001] The present invention relates to the field of biomedical engineering technology, and in particular to a quantitative laser speckle blood flow imaging method based on a neuromorphic visual sensor. Background Art

[0002] The existing Laser Speckle Contrast Imaging (LSCI) method can achieve full-field blood flow imaging, but there is a problem that is difficult to balance between temporal and spatial resolution, real-time and quantitative capabilities. The traditional LSCI method analyzes blood flow by calculating speckle contrast. This method requires a lot of time and computing resources and cannot meet the clinical needs of real-time and accurate blood flow measurement. In addition, due to the use of frame-based acquisition, the temporal resolution of the traditional LSCI method is limited by the camera frame rate and cannot accurately capture rapid blood flow changes, which poses a challenge to the application scenarios of blood flow imaging with high dynamic changes.

[0003] Among the existing LSCI technologies, common methods include spatial laser speckle contrast imaging (sLSCI), temporal laser speckle contrast imaging (tLSCI), and spatial-temporal laser speckle contrast imaging (stLSCI). These methods have improved the quality of blood flow imaging to a certain extent, but they still cannot take into account both high spatial resolution and high temporal resolution. For example, although the spatial laser speckle contrast imaging method has good spatial resolution, its temporal resolution is low and it cannot accurately describe rapidly changing blood flow; the temporal laser speckle contrast imaging method has good temporal resolution, but its spatial resolution is poor and it is difficult to obtain detailed blood flow images.

[0004] In addition, the existing LSCI method also has limitations in quantitative blood flow measurement. In order to achieve quantitative measurement of blood flow velocity, it is usually necessary to use a multiple exposure method to establish a nonlinear model to solve the relationship between speckle autocorrelation time and blood flow velocity through the change of speckle contrast under different exposure times. This method requires a lot of computing resources and has high errors, making it difficult to meet high-precision requirements in real-time applications. Especially during clinical surgery, doctors need to understand the blood flow changes in the lesion area in real time in order to adjust the surgical strategy in time. Traditional methods are difficult to provide sufficient real-time support.

[0005] The noise problem is also a major difficulty in the traditional LSCI method. During the imaging process, static scattered light caused by physiological behaviors (such as heartbeat and breathing) and the electronic noise of the imaging device itself will have an adverse effect on the final blood flow imaging quality. These noises not only reduce the signal-to-noise ratio of the image, but also affect the accuracy of quantitative measurements. Although some studies have proposed noise reduction methods based on image post-processing, these methods usually increase the computational complexity and cannot be applied in real time during the imaging process; therefore, they do not meet the existing needs. In this regard, we propose a quantitative laser speckle blood flow imaging method based on a neuromorphic visual sensor. Summary of the invention

[0006] The purpose of the present invention is to provide a quantitative laser speckle blood flow imaging method based on a neuromorphic visual sensor to solve the problem that static scattered light caused by physiological behavior and the electronic noise of the imaging device itself in the traditional LSCI method proposed in the above background technology will have an adverse effect on the final blood flow imaging quality.

[0007] To achieve the above object, the present invention provides the following technical solution: a quantitative laser speckle blood flow imaging method based on a neuromorphic visual sensor, comprising the following steps:

[0008] Step 1: Use a He-Ne laser as a light source, and use the expanded laser beam to irradiate the scattering medium to form a speckle pattern;

[0009] Step 2: Use a microfluidic syringe to set the two parameters of the syringe inner diameter and flow rate to control the flow rate range;

[0010] Step 3: Use an event camera to asynchronously sense changes in the light intensity of the speckle pattern. Whenever the speckle pattern changes, the camera continuously outputs a pulse signal to form a pulse data stream to detect high-speed changes in blood flow information. Use a "frame" camera to capture the speckle pattern, calculate the speckle pixel size n in the image frame, calibrate the pixel physical size k, and calculate the average speckle size m, the formula is m = n × k;

[0011] Step 4: Start the microfluidic syringe to move the fluid in the pipe. The event camera collects event stream data. Combining the position and polarity information in the event with the event trigger threshold set in advance, the grayscale value of a specific pixel at a specific time is determined. Then, interpolation is performed at a time interval of 20 μs to obtain a curve of the grayscale value of the point over time and calculate the blood flow velocity.

[0012] Preferably, the wavelength of the helium-neon laser is 632 nm.

[0013] Preferably, the scattering medium includes hydrogel, capillary glass flakes, diamond slurry, or blood.

[0014] Preferably, the event camera is a camera based on a neuromorphic vision sensor.

[0015] Preferably, the flow rate range of the microfluidic syringe is 0.001 μL / min to 43.349 ml / min, and the flow rate range in step 2 is 0.1 to 2.0 mm / s.

[0016] Preferably, the blood flow velocity is the speckle moving velocity, and the calculation formula of the blood flow velocity v is:

[0017]

[0018] Where m is the average speckle size, τ C is the speckle autocorrelation time.

[0019] Preferably, after the autocorrelation operation is performed after interpolation in step 4 to obtain the autocorrelation image, the middle peak of the image is taken, and the speckle autocorrelation time is the half-maximum full width of the peak.

[0020] Preferably, the calculation formula for the average speckle size m is:

[0021] m = n × k;

[0022] Where n is the speckle pixel size and k is the physical pixel size.

[0023] Preferably, the speckle pixel size is calculated by taking image frames by a “frame” camera, and the pixel physical size is obtained by calibrating the “frame” camera.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] 1. The present invention combines event cameras with laser speckles to achieve accurate capture of dynamic changes in blood flow, breaking through the technical bottleneck of traditional frame cameras that are difficult to describe rapid dynamic changes under frame rate restrictions. The present invention can asynchronously collect speckle change data with a time resolution of microseconds, directly respond to brightness changes caused by blood cell movement, and shield static areas that are not involved in changes, thereby effectively reducing data redundancy. In the process of blood flow imaging, the present invention utilizes the characteristics of the event camera outputting event data streams to generate event responses only for pixels with brightness changes, significantly reducing the computational requirements for data storage and processing, and improving the response speed and processing efficiency of the system while maintaining system performance. In specific applications, the technical solution provides accurate blood flow dynamic information for clinical surgery by generating high temporal and spatial resolution blood flow imaging data in real time, which can support doctors to quickly adjust surgical strategies and improve the safety and accuracy of surgery. In addition, the event camera has the advantage of a high dynamic range and can maintain good imaging effects under low light conditions, ensuring its applicability under a variety of clinical and experimental conditions;

[0026] 2. The present invention can realize real-time full-field imaging of blood flow under limited transmission bandwidth through the asynchronous perception, low latency and low data redundancy characteristics of event cameras. This is very helpful for understanding the blood flow behavior in complex vascular networks. For example, in evaluating the blood supply of tissues and studying the occurrence and development of vascular lesions (such as cerebral infarction, varicose veins, hemangiomas, etc.), more comprehensive information can be obtained through the visualization of blood flow distribution. In addition, quantitative measurement can also provide reliable data support for disease diagnosis and efficacy evaluation; BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a schematic diagram of the imaging principle of the event camera;

[0028] Figure 2 This is a schematic diagram of the laser speckle blood flow imaging experimental platform system structure;

[0029] Figure 3 This is a flow chart of event stream data processing of the present invention;

[0030] Figure 4 A graph showing the gray value of the event stream data processing result of a specific pixel point over time;

[0031] Figure 5 It is a graph of the autocorrelation calculation result of the event stream data processing result of a specific pixel point according to the present invention;

[0032] Figure 6 This is a schematic diagram of the laser speckle blood flow imaging verification experiment of the present invention;

[0033] Figure 7 This is a diagram showing the flow imaging result of diamond polishing liquid by the event camera of the present invention;

[0034] Figure 8 This is a diagram showing the blood flow imaging result of the event camera of the present invention;

[0035] Fig. 9 It is a schematic diagram of the front side of the hydrogel;

[0036] Fig.10 This is the result of imaging the flow of diamond slurry covered by hydrogel using an event camera;

[0037] Fig.11 Speckle field map generated for frosted glass;

[0038] Fig.12 The original data and fitting curve diagram of the time-varying speed of the frosted glass driven by the electric translation stage;

[0039] Fig.13 The original data and fitting curve diagram of the correlation time of blood flow in the microfluidic chip changing with the speed;

[0040] Fig.14 This is a graph showing the correlation time of blood flow in the mouse ear as a function of laser irradiation time. DETAILED DESCRIPTION

[0041] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0042] See also Figures 1 to 14 The present invention provides an embodiment: a quantitative laser speckle blood flow imaging method based on a neuromorphic visual sensor, comprising the following steps:

[0043] Step 1: Build a laser speckle blood flow imaging experimental platform system, such as Figure 2 As shown, a helium-neon laser (Thorlabs, HNL210L, 21 mW) is used as a light source, the wavelength of the helium-neon laser is 632 nm, and the laser beam after beam expansion is used to irradiate the scattering medium to form a speckle pattern;

[0044] Step 2: Use a microfluidic syringe, the microfluidic syringe is a microfluidic syringe of Baoding Dichuang Electronic Technology Co., Ltd., model LSP01-1A, set the two parameters of syringe inner diameter and flow rate, the flow range of the microfluidic syringe is 0.001μL / min~43.349ml / min, and the control flow rate range is 0.1~2.0mm / s;

[0045] Step 3: Use an event camera to asynchronously sense changes in the light intensity of the speckle pattern. The event camera model is CeleX5-MP from Shanghai OmniVision, with a resolution of 1280×800, a pixel size of 9.8μm×9.8μm, and a photosensitive area of ​​12.5mm×7.8mm. Each pixel of the camera works independently and can asynchronously sense changes in light intensity. Whenever the speckle pattern changes, the camera continuously outputs a pulse signal to form a pulse data stream to detect high-speed changes in blood flow information. Use a "frame" camera to capture the speckle pattern. The camera model is MV-CA020-10UM from Hikvision Robotics, with a resolution of 1624×1240 and a pixel size of 4.5μm×4.5μm. Calculate the speckle pixel size n in the image frame, calibrate the pixel physical size k, and calculate the average speckle size m. The formula is m=n×k;

[0046] Step 4: Start the microfluidic syringe to make the fluid in the pipe move, and the event camera collects the event stream data. Next, process the data. The processing flow chart is as follows: Figure 3As shown in Figure 1, the grayscale value of a specific pixel at a specific time is determined by combining the position and polarity information of the event with the event trigger threshold set in advance. Then, interpolation is performed at a time interval of 20 μs to obtain a graph showing the grayscale value of the point over time, as shown in Figure 1. Figure 4 As shown, Figure 4 The horizontal axis is time, in microseconds, and the vertical axis is the gray value of the pixel. After interpolation, autocorrelation operation is performed, and the result is as follows: Figure 5 As shown, take the middle peak of the image, the full width at half maximum of the peak is also the speckle autocorrelation time τ C , calculate the blood flow velocity, the blood flow velocity is the speckle moving speed, the calculation formula of blood flow velocity v is:

[0047]

[0048] Where m is the average speckle size, and the calculation formula for the average speckle size m is:

[0049] m = n × k;

[0050] Where n is the speckle pixel size, k is the physical pixel size, the speckle pixel size is calculated by shooting the image frame with the “frame” camera, and the physical pixel size is obtained by calibrating the “frame” camera.

[0051] The above-mentioned scattering medium includes hydrogel, capillary glass, diamond polishing liquid and blood.

[0052] Figure 1 The diagram shows the principle of imaging a moving object by an event camera. Compared with the data recorded by a frame-based camera, the data recorded by the event camera has better continuity on the time axis and low data redundancy. Event cameras have significant advantages over traditional cameras in real-time data processing and long-term monitoring. For example, at a resolution of 1 million pixels, a high-speed camera with a frame rate of 1,000 frames per second generates 1GB of data per second, which makes it impossible to operate for a long time. However, at the same spatial resolution and higher temporal resolution, the data volume of the event camera is about 40MB / s, which can be processed and stored in real time on a computer through a USB interface. Therefore, event cameras are very suitable for long-term monitoring in industrial and biological fields.

[0053] Use the event camera to image the fluid and verify its imaging performance: To observe the imaging effect of the event camera on the fluid, a Figure 6 The experimental setup is shown in Figure 1. The laser generated by the helium-neon laser is expanded by a plano-convex lens. The laser beam is irradiated on the capillary to form a speckle pattern. The microfluidic syringe pushes diamond abrasive liquid with a particle diameter of about 10 μm through the capillary glass tube to simulate blood flow. The event camera captures the change of the speckle field. The measurement results are shown in Figure 1. Figure 7 As shown;

[0054] Testing the imaging effect of real blood flow, such as Figure 8 As shown, it can be seen that the proposed method can well image the fluid containing scatterers. Compared with the traditional method, the imaging method proposed by the present invention does not need to calculate the speckle contrast, can be used for real-time imaging, and the imaging frame rate can reach up to 1000 frames per second.

[0055] The diamond slurry covered by the hydrogel was imaged using an event camera to verify its imaging performance in complex situations: In order to simulate the occlusion and scattering effect of the skin on blood vessels, we used agar powder mixed with Intralipid to prepare a hydrogel with scattering properties similar to those of the skin. Fig. 9 As shown, the hydrogel is covered on the capillary glass, and the imaging results are shown Fig.10 It can be seen that the proposed method can still complete the imaging of diamond slurry well in the case of hydrogel coverage.

[0056] The event camera is used to image the speckle field formed by the capillary glass sheet and calculate the speed to verify the feasibility of the speed measurement method proposed in the present invention: an experimental platform is built, a piece of frosted glass is fixed on the electric translation stage, and the electric translation stage drives the frosted glass to move at a uniform speed. The speed range of the electric translation stage is set to 0.1-2.0 mm / s, with an interval of 0.1 mm / s. A total of 20 sets of data are measured. The results are as follows Fig.12 As shown. The fitting curve is obtained by fitting the results. It can be seen that the fitting effect is good and there is no point with large deviation. By shooting image frames, such as Fig.11 As shown in the figure, the speckle pixel size of the speckle field is calculated to be 3.2 pixels, and the physical size of the pixel is calibrated to be 5.1667 μm / pixel, so the theoretical average speckle size is 16.533 μm, and the actual average speckle size obtained from the fitting curve is 16.8 μm, with an error of 1.61%. This experiment verifies the feasibility of our proposed method.

[0057] The event camera is used to image the fluid in the microfluidic chip and calculate the flow rate to verify the feasibility of the velocity measurement method proposed in the present invention: To further simulate the flow of blood in capillaries, we use microfluidic chips to simulate capillaries and observe the flow of blood in tiny channels. The channel width of the microfluidic chip is 0.2mm and the channel depth is 30μm. The results are as follows Fig.13 As shown, it can be seen that the fitting effect is good. We use R 2 To measure the fitting effect, calculate R 2 =0.9984. This indicates that the fitting curve is very consistent with the calculation formula of blood flow velocity v and satisfies the inverse function relationship.

[0058] An event camera was used to image the mouse ears, and thrombus formation was induced in the ears using the optical thrombosis method. The changes in blood flow before and after were observed to verify the application potential of the method proposed in the present invention in biomedical engineering: The chemical reagent used in this embodiment is Bengal rose red, also known as tiger red. 1.5% tiger red (2ml / kg) was injected through the tail vein of the mouse, and the mouse ears were illuminated after waiting for 3 minutes. The illuminated area was the mouse ears, and the laser light source used was a small semiconductor laser from Daheng Optoelectronics, model GCI-07-520-20, with an output power ≥20mW and a central wavelength of 520±10nm. The correlation time reflects the relative change in blood flow, such as Fig.14 As shown, 5 minutes after laser irradiation, the blood flow dropped to 37.80% of the original level.

[0059] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A quantitative laser speckle blood flow imaging method based on a neuromorphic visual sensor, characterized in that: The steps include: Step 1: Use a He-Ne laser as a light source, and use the expanded laser beam to irradiate the scattering medium to form a speckle pattern; Step 2: Use a microfluidic syringe to set the two parameters of the syringe inner diameter and flow rate to control the flow rate range; Step 3: Use an event camera to asynchronously sense changes in the light intensity of the speckle pattern. Whenever the speckle pattern changes, the camera continuously outputs pulse signals to form a pulse data stream to detect high-speed changes in blood flow information. Use a "frame" camera to capture the speckle pattern, calculate the speckle pixel size n in the image frame, calibrate the pixel physical size k, and calculate the average speckle size m, the formula is m = n × k; Step 4: Start the microfluidic syringe to move the fluid in the pipe. The event camera collects event stream data. Combining the position and polarity information in the event with the event trigger threshold set in advance, the grayscale value of a specific pixel at a specific time is determined. Then, interpolation is performed at a time interval of 20 μs to obtain a curve of the grayscale value of the point over time and calculate the blood flow velocity.

2. The quantitative laser speckle blood flow imaging method based on neuromorphic visual sensor according to claim 1, characterized in that: The wavelength of the helium-neon laser is 632 nm.

3. The quantitative laser speckle blood flow imaging method based on neuromorphic visual sensor according to claim 1, characterized in that: The scattering medium includes hydrogel, capillary glass flakes, diamond polishing fluid and blood.

4. The quantitative laser speckle blood flow imaging method based on neuromorphic visual sensor according to claim 1, characterized in that: The event camera is a camera based on a neuromorphic vision sensor.

5. The quantitative laser speckle blood flow imaging method based on neuromorphic visual sensor according to claim 1, characterized in that: The flow rate range of the microfluidic syringe is 0.001 μL / min to 43.349 ml / min, and the flow rate range in step 2 is 0.1 to 2.0 mm / s.

6. The quantitative laser speckle blood flow imaging method based on neuromorphic visual sensor according to claim 1, characterized in that: The blood flow velocity is the speckle moving speed, and the calculation formula of the blood flow velocity v is: Where m is the average size of speckle, τ C is the speckle autocorrelation time.

7. The quantitative laser speckle blood flow imaging method based on neuromorphic visual sensor according to claim 5, characterized in that: After the autocorrelation operation is performed after interpolation in step 4 to obtain the autocorrelation image, the middle peak of the image is taken, and the speckle autocorrelation time is the half-maximum full width of the peak.

8. The quantitative laser speckle blood flow imaging method based on neuromorphic visual sensor according to claim 1, characterized in that: The calculation formula of the average speckle size m is: m = n × k; Where n is the speckle pixel size and k is the physical pixel size.

9. The quantitative laser speckle blood flow imaging method based on neuromorphic visual sensor according to claim 1, characterized in that: The speckle pixel size is calculated by shooting image frames with a "frame" camera, and the pixel physical size is obtained by calibrating the "frame" camera.

Images