Motion compensated assisted OCT system, method and method of operation

Abstract

The application relates to a motion compensation auxiliary optical coherence tomography system, method and working method, which can reduce motion artifacts, improve imaging quality, avoid image blurring caused by physiological motions such as respiration, and obtain a clear image accurately reflecting the real situation of a lesion. An optical distance sensor is used as a visual servo system of a robot, a 7-joint cooperative mechanical arm is fixed at the end of the OCT imaging probe and holds the OCT imaging probe, the optical distance sensor is integrated and installed at the end of the probe, the position of a laser point is recognized based on the principle of triangulation, the accurate distance from the probe to the sample is calculated, the position of the probe is controlled in real time, the axial motion of the sample is compensated, a communication channel based on TCP / IP is established between a host and a robot controller, and a motion compensation control loop is established. The system is simple in structure, convenient to operate, high in stability, and suitable for various clinical and scientific research application fields.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of biomedical engineering, in particular to a motion compensation assisted OCT system and method, and also relates to the working principle and method of the motion compensation assisted OCT system. BACKGROUND

[0002] Optical coherence tomography (OCT) is a high-resolution, non-destructive, non-invasive optical three-dimensional imaging technology. OCT is very suitable for human skin imaging. It is based on the principle of low coherence light interference to obtain two-dimensional tomographic images and three-dimensional structure images of the tissue with micron-level (1-15 μm) resolution at a depth of about 2 mm below the skin surface. Many skin diseases only affect the skin surface within a few millimeters, and OCT imaging can distinguish abnormal tissues from normal tissues in the body, which makes OCT a useful diagnostic tool for diagnosing early skin cancer and other skin diseases.

[0003] However, in clinical practice, OCT scanning is point-by-point scanning, and the entire scanning process takes a certain amount of time, and the imaging depth is small. Thus, when imaging the skin in the clinic, the patient's involuntary movements such as breathing and heartbeat, as well as the inevitable physiological tremor of the operator's hand, will cause image motion artifacts, affect image quality, and even cause OCT signal loss, making it very difficult to obtain stable and continuous OCT images.

[0004] The limitations of the handheld OCT probe commonly used in the clinic include: 1. Due to the obstruction of the probe, the operator cannot clearly observe the imaging area, making it difficult to accurately aim the scanning area, resulting in unstable imaging and low image quality. 2. Fatigue and discomfort: In order to accurately assess the risk and development of the patient's skin disease, the physician operator needs to work continuously for more than several hours to complete the OCT scanning examination of the patient's skin, which is inconvenient, laborious and time-consuming, and is prone to fatigue and discomfort. 3. It is difficult to control hand tremor, and the operator's hand tremor during the process is inevitable, and the lack of direct perception of the distance from the probe to the skin surface in non-contact imaging makes it difficult to keep the distance between the probe and the skin tissue surface stable, causing image motion artifacts, image blurring and signal loss, which is not conducive to clinical diagnosis. The low image quality caused by such technical problems of existing handheld OCT devices has affected the doctor's accurate judgment of skin diseases, limiting its widespread application in clinical practice. SUMMARY

[0005] To overcome the defects of the prior art, the technical problem to be solved by the present application is to provide a motion compensation assisted OCT system which can avoid image blurring caused by movement, greatly improve the imaging quality, and obtain clear images that accurately reflect the true situation of the lesion, which is more conducive to the diagnosis of diseases by doctors.

[0006] The technical scheme of the present application is: the motion compensation auxiliary OCT system comprises a specified spectral domain OCT system (11), a probe (2), a 7-joint collaborative mechanical arm (10), an optical distance sensor (1) and a host computer (5);

[0007] The optical distance sensor is used as the visual servo system of the robot, the probe is fixed and clamped at the end of the 7-joint collaborative mechanical arm, and the optical distance sensor is integrally installed at the end of the probe; the optical distance sensor comprises a laser pen (12) for emitting a red dot laser and a micro high-frame-rate CCD camera (13); the CCD camera simultaneously collects RGB color images of the imaging area and the laser dot, and transmits the RGB color images to the host computer in real time; the host computer processes the images, identifies the position of the laser dot, and obtains the position information of the laser dot on the surface of the sample (3); according to the identification and positioning result, the accurate distance from the probe to the sample is calculated based on the principle of triangulation, the host computer sends a control instruction to the mechanical arm, and the mechanical arm controller (4) is used to make the mechanical arm clamping the probe follow the axial movement of the sample surface, so as to compensate for the axial movement of the sample; a communication channel based on TCP / IP is established between the host computer and the robot controller, and a motion compensation control loop is established.

[0008] The optical distance sensor is used as the visual servo system of the robot, the probe is fixed and clamped at the end of the 7-joint collaborative mechanical arm, and the optical distance sensor is integrally installed at the end of the probe; the optical distance sensor comprises a laser pen (12) for emitting a red dot laser and a micro high-frame-rate CCD camera (13); the CCD camera simultaneously collects RGB color images of the imaging area and the laser dot, and transmits the RGB color images to the host computer in real time; the host computer processes the images, identifies the position of the laser dot, and obtains the position information of the laser dot on the surface of the sample (3); according to the identification and positioning result, the accurate distance from the probe to the sample is calculated based on the principle of triangulation, the host computer sends a control instruction to the mechanical arm, and the mechanical arm controller (4) is used to make the mechanical arm clamping the probe follow the axial movement of the sample surface, so as to compensate for the axial movement of the sample; a communication channel based on TCP / IP is established between the host computer and the robot controller, and a motion compensation control loop is established.

[0009] A motion compensation auxiliary OCT method is also provided, which comprises the following steps:

[0010] (I) the optical distance sensor is used as the visual servo system of the robot, the probe is fixed and clamped at the end of the 7-joint collaborative mechanical arm, and the optical distance sensor is integrally installed at the end of the probe;

[0011] (II) The CCD camera simultaneously acquires RGB color images of the imaging area and the laser point, and transmits the RGB color images to the host in real time;

[0012] (III) The host computer processes the image, identifies the position of the laser point, and obtains the position information of the laser point on the sample surface;

[0013] (IV) Based on the principle of triangulation, the precise distance from the probe to the sample is calculated according to the laser point identification and positioning results, and the host computer issues control commands to the robotic arm.

[0014] The robotic arm controller enables the robotic arm to move axially along the sample surface while holding the probe, thus compensating for the axial movement of the sample.

[0015] (V) Establish a TCP / IP-based communication channel between the host computer and the robot controller.

[0016] Establish a motion compensation control loop.

[0017] A method for operating a motion-compensated assisted OCT system is also provided, which includes the following steps:

[0018] (1) Begin;

[0019] (2) The CCD camera simultaneously acquires RGB color images of the imaging area and the laser point;

[0020] (3) The host computer processes the image, identifies the position of the laser point, and obtains the position information of the sample surface;

[0021] (4) When the detected laser point position changes by more than 2 pixels compared with the y-direction coordinate of the center point pixel of the image, the robotic arm motion compensation function is activated; specifically, when the laser point coordinate is less than the center point coordinate, it indicates that the distance between the sample and the probe is too far, and the probe moves downward to reduce the distance; conversely, the probe moves upward to increase the distance, so that the distance between the probe and the sample is always kept near the ideal position. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the motion compensation-assisted OCT system according to the present invention.

[0023] Figure 2 This is a schematic diagram of the probe of the motion compensation-assisted OCT system according to the present invention.

[0024] Figure 3 This is a schematic diagram illustrating the principle of an optical sensor measuring probe based on the triangulation principle to measure the distance between the sample surface and the sample surface, according to an embodiment of the present invention.

[0025] Figure 4 is an example control flow diagram of a mechanical arm assisted motion compensation system according to an embodiment of the present application.

[0026] Figure 5 Surface errors of the motion compensation assisted OCT system according to the present application are shown when the system is not turned on and turned on. DETAILED DESCRIPTION

[0027] As shown in the motion compensation assisted OCT system, it comprises a spectral domain OCT system 11, a probe 2, a 7-joint collaborative robot arm 10, an optical distance sensor 1 and a host computer 5. Figure 1 The optical distance sensor is used as the visual servo system of the robot, the probe is fixed and clamped at the end of the 7-joint collaborative robot arm, and the optical distance sensor is integrated at the end of the probe. The optical distance sensor comprises a laser pen 12 emitting a red dot laser and a micro high-frame-rate CCD camera 13. The CCD camera simultaneously collects RGB color images of the imaging area and the laser dot and transmits the RGB color images to the host computer in real time. The host computer processes the images, identifies the position of the laser dot, and obtains the position information of the laser dot on the surface of the sample 3. Based on the principle of triangulation, the accurate distance from the probe to the sample is calculated according to the laser recognition and positioning result, the host computer sends control instructions to the mechanical arm, and the mechanical arm controller 4 moves the mechanical arm to follow the sample movement to compensate for the axial movement of the sample. A communication channel based on TCP / IP is established between the host computer and the robot controller to establish a motion compensation control loop.

[0028]

[0029] ​The application uses an optical distance sensor as a visual servo system of a robot, fixes and holds a probe at the end of a 7-joint collaborative robot arm, and integrates the optical distance sensor at the end of the probe. The optical distance sensor comprises a laser pen emitting red dot laser and a miniature high-frame-rate CCD camera. Based on the principle of triangulation, the CCD camera simultaneously collects RGB color images of the imaging area and the laser dot, and transmits the RGB color images to the host computer in real time. The host computer processes the images, identifies the position of the laser dot, and obtains the position information of the laser dot on the sample surface. The accurate distance from the probe to the sample is calculated according to the identification and positioning result, and the host computer sends control instructions to the robot. The robot controller makes the robot holding the probe move together with the sample to compensate for the axial movement of the sample. A communication channel based on TCP / IP is established between the host computer and the robot controller, a motion compensation control loop is established, the distance between the current probe and the sample surface is calculated in real time, and the calculated control instructions are sent to the robot controller to control and guide the robot holding the OCT probe to move in the direction perpendicular to the sample surface, so that the axial motion compensation is realized. The application has the advantages of reasonable structure, accurate surface distance detection, high efficiency, reliable performance and low cost, so that the image blur caused by movement can be avoided, the imaging quality is greatly improved, and clear images accurately reflecting the real situation of the lesion are obtained. The system of the application has simple structure, convenient operation and high stability, and is suitable for various clinical and scientific research application fields. Preferably, the spectral domain OCT system comprises a superluminescent diode light source 7 with a central wavelength of 1310 nm, a short-wave infrared spectrometer 6, and a 50 / 50 optical fiber coupler 8 having a reference arm 9.

[0030] Preferably, the probe scans the light beam using a micro-electro-mechanical system scanning galvanometer (MEMS) 14, the scanning range is 4*4 mm, the optical device of the probe is fixed in an aluminum shell, and the aluminum adapter is connected with the end of the collaborative robot arm; the working distance of the probe is 28 mm, and the probe diameter is 18 mm.

[0031] Preferably, an ultra-compact laser pen emitting red dot laser is fixed at the end of the objective lens barrel of the probe, the laser optical axis forms an angle of 60° with the OCT imaging optical axis, and a small high-speed RGB camera 13 is installed on the other side of the objective lens barrel, the camera optical axis forms an angle of 60° with the OCT optical axis, so that the optical axes of the camera, the laser and the OCT all intersect with the imaging plane of the target.

[0032] Further provided is a motion compensation assisted OCT method, which comprises the following steps:

[0033] (I) using an optical distance sensor as a visual servo system of a robot, fixing and holding a probe at the end of a 7-joint collaborative robot arm, and integrating the optical distance sensor at the end of the probe;

[0034] (II) CCD camera simultaneously collects RGB color images of the imaging area and the laser point, and transmits the RGB color images to the host computer in real time;

[0035] (III) The host computer completes image processing, identifies the position of the laser point, and obtains the position information of the laser point on the sample surface;

[0036] (IV) Based on the principle of triangulation, the accurate distance from the probe to the sample is calculated according to the laser recognition and positioning result, the host computer sends control instructions to the robot controller, and the robot controller controls the robot to move the probe to follow the axial movement of the sample to compensate for the axial movement of the sample;

[0037] (V) A communication channel based on TCP / IP is established between the host computer and the robot controller,

[0038] A motion compensation control loop is established.

[0039] As shown in Figure 4 , a working method of a motion compensation assisted OCT system is also provided, which comprises the following steps:

[0040] (1) Start;

[0041] (2) CCD camera simultaneously collects RGB color images of the imaging area and the laser point;

[0042] (3) The host computer completes image processing, identifies the position of the laser point, and obtains the position information of the sample surface;

[0043] (4) When the detected laser point position changes more than 2 pixels compared with the pixel y direction coordinate of the center point of the image, the motion compensation function of the robot is activated; when the laser point coordinate is less than the center point coordinate, it indicates that the distance from the sample to the probe is too far, and the probe moves downward to reduce the distance, otherwise, the probe moves upward to increase the distance, so that the distance between the probe and the sample is always maintained near the ideal position.

[0044] The specific embodiments of the present application are described in detail below. Some embodiments of the present application can provide a mechanical arm assisted OCT system based on an optical distance sensor, which can track the surface and compensate for the motion, to reduce the motion of the skin surface caused by the unconscious physiological activities of the patient such as breathing and heartbeat in clinical and intraoperative skin OCT imaging, and to cause motion artifacts and image blur of the OCT image even signal loss.

[0045] Figure 1is a schematic diagram of an optical sensor based robotic arm assisted motion compensation OCT system according to an embodiment of the present application, comprising an optical distance sensor 1, a custom made OCT probe 2, a sample skin 3 for imaging, a robotic arm controller 4, a host computer 5, a collaborative robotic arm 10 and a spectral domain OCT system (SD-OCT) 11.

[0046] The sample surface variations are precisely sensed by the optical distance sensor consisting of a laser pointer 12 and a CCD camera 13. Specifically, the distance between the probe and the sample surface is determined from the position of the laser spot identified in the RGB image captured by the CCD camera. And according to the feedback from the optical distance sensor, the laser spot position is identified by image processing by the workstation 5 (Delta workstation T3630) and the probe to surface distance is calculated. The calculated control commands are then sent to the robotic arm controller to control the multi-degree of freedom collaborative robotic arm 10 to move the probe 2 to compensate for the axial motion of the sample. The sample surface can be the surface 3 of a patient's skin, or subcutaneous tissue.

[0047] The spectral domain OCT system (SD-OCT) comprises a superluminescent diode light source 7 (Santec S5FC1021S: single mode fiber-coupled benchtop SLD light source center wavelength 1310 nm, optical power 12.5 mW, 85 nm bandwidth) with a center wavelength of 1310 nm, a short wave infrared spectrometer 6 (C1300-1298 / 245-76-SG2K (Wasatch Photonics Inc): maximum imaging speed 76 kHz, pixel number 2048, center wavelength 1298 nm, wavelength range 245 nm), and a 50 / 50 fiber coupler 8 (Santec: TM105R5F2A).

[0048] In the example of Figure 2 The custom made micro OCT probe is scanned by a micro-electro-mechanical system (MEMS, Micro-Electro-Mechanical System) scanning mirror 14 (A7B1.1 (Mirrorcle Technologies, Inc.). Dual axis scanning. Scan mirror diameter: 3.6 mm). The OCT scanning range is 4x4 mm. The optics of the OCT probe are fixed in an aluminum housing, which is connected to the end of the collaborative robotic arm through an aluminum adapter. The probe working distance is 28 mm, and the probe diameter is smaller, at 18 mm, which makes it easier to apply to narrow working spaces without interfering with any surgical procedures, making it more versatile in a clinical setting.

[0049] Figure 3A schematic diagram of the principle of a triangulation-based optical sensor measurement probe measuring the distance to a sample surface according to an embodiment of the present application is shown. At the end of the OCT probe 2 objective lens tube, a super-compact (4mm diameter) laser pointer 12 is fixed, which can emit a red dot of laser light. The laser optical axis is at an angle of 60° to the OCT imaging optical axis. Similarly, a small high-speed RGB camera 13 is mounted on the other side of the objective lens tube. The camera optical axis is at an angle of 60° to the OCT optical axis. Due to the precise design, the camera optical axis, the laser optical axis and the OCT optical axis all intersect the target imaging plane.

[0050] Figure 3 The Z2 plane in FIG. 13 is the target imaging plane, and Z1 and Z3 are the sample surfaces that are too close and too far from the probe, respectively. The a, b, c lines represent how the laser light reflected from the surfaces at different distances enters the camera sensor. When the distance between the probe and the measured object surface changes, the position of the detected laser spot on the sensor also changes. From these positions, the distance between the probe and the measured object surface can be inferred in reverse. For laser point recognition, the HSV color space threshold processing and morphological processing method is used to obtain the pixel position of the laser point center.

[0051] Figure 4 An example control flowchart of a mechanical arm-assisted motion compensation system according to an embodiment of the present application is shown. This is a very simple closed-loop control method. When the detected laser spot position changes more than 2 pixels compared to the image center point pixel y direction coordinate (240), the motion compensation function is activated, and the mechanical arm moves at a fixed step (0.1 mm). After each movement by one step, the next movement direction is determined by the feedback of the optical position sensor, forming a closed-loop control. When the laser point coordinate is less than the center point coordinate, the probe moves downward to reduce the distance between the probe and the sample, and vice versa. That is, once the change value reaches the threshold, the mechanical arm will drive the probe to reduce the change.

[0052] An example implementation of surface tracking and motion compensation testing is shown. A balloon with a rhythmic inflation and deflation (16 times per second) is used to simulate the skin surface motion caused by respiratory motion. The probe is directed vertically to the target surface, which reciprocally moves up and down from the initial position. The optical sensor monitors the motion and feeds back to the host computer, which calculates the surface position and sends a motion instruction to the collaborative mechanical arm, which holds the probe to move with the surface to adjust the distance between the probe and the surface, maintaining a constant distance, i.e., the working distance of the probe (28 mm). As shown in FIG. 14, the probe is kept at a constant distance from the surface, and the surface moves up and down, and the probe moves up and down with the surface, keeping the distance between the probe and the surface constant. Figure 5The error of the distance between the target surface position and the probe is recorded over time. By comparing the surface error values between the compensation on and compensation off, the effect of compensation is obvious. The results of using the surface tracking and motion compensation system are obviously improved. Such a system can be widely applied to clinical skin OCT imaging and can improve the accuracy of clinical diagnosis.

[0053] The above is only the preferred embodiment of the present application, not any form of the present application, any simple modification, equivalent change and modification of the above embodiment according to the technical essence of the present application are still within the protection scope of the technical solution of the present application.

Claims

1. A motion-compensated assisted OCT system, characterized by: It comprises: The designated spectral domain OCT system (11), the probe (2), the 7 joint cooperative mechanical arm (10), the optical distance sensor (1) and the host computer (5); The optical distance sensor is used as the visual servo system of the robot, the probe is fixed and held at the end of the 7 joint cooperative mechanical arm, the optical distance sensor is integratedly installed at the end of the probe, the optical distance sensor comprises: a laser pen (12) emitting red dot laser, a miniature high frame rate CCD camera (13), the CCD camera simultaneously collects RGB color images of the imaging area and the laser dot, and transmits the RGB color images to the host computer in real time; the host computer completes image processing, identifies the position of the laser dot on the sample surface, calculates the real-time distance from the sample (3) surface to the OCT probe based on the principle of triangulation; the host computer sends control instructions to the mechanical arm, and the mechanical arm controller (4) makes the mechanical arm holding the probe follow the sample axial movement to compensate for the axial movement of the sample; a communication channel based on TCP / IP is established between the host computer and the robot controller, and a motion compensation control loop is established; An ultra-compact laser pen emitting red dot laser is fixed at the end of the objective lens barrel of the probe, the laser optical axis is at an angle of 60° with the OCT imaging optical axis, and a miniature high frame rate CCD camera (13) is installed on the other side of the objective lens barrel, the camera optical axis is at an angle of 60° with the OCT optical axis, so that the optical axes of the camera, the laser and the OCT all intersect with the imaging plane of the target.

2. The motion-compensated assisted OCT system of claim 1, wherein: The designated spectral domain OCT system comprises an ultraviolet diode light source (7) with a central wavelength of 1310 nm, a short-wave infrared spectrometer (6), a 50 / 50 optical fiber coupler (8), and the optical fiber coupler has a reference arm (9).

3. The motion-compensated assisted OCT system of claim 2, wherein: The probe scans the light beam by using a micro-electro-mechanical system scanning galvanometer (14), the scanning range is 4*4mm, the optical device of the probe is fixed in an aluminum shell, and the aluminum adapter is connected with the end of the cooperative mechanical arm; the working distance of the probe is 28mm, and the probe diameter is 18mm.

4. The method of operating a motion-compensated assisted OCT system of claim 1, wherein: It comprises the following steps: (I) The optical distance sensor is used as the visual servo system of the robot, the probe is fixed and held at the end of the 7 joint cooperative mechanical arm, and the optical distance sensor is integratedly installed at the end of the probe; (II) The CCD camera simultaneously collects RGB color images of the imaging area and the laser dot, and transmits the RGB color images to the host computer in real time; (III) The host computer completes image processing, identifies the position of the laser dot, and obtains the position information of the laser dot on the sample surface; (IV) Based on the principle of triangulation, the real-time distance from the sample surface to the OCT probe is calculated according to the position of the laser on the sample surface, the host computer sends control instructions to the mechanical arm, and the mechanical arm controller makes the mechanical arm holding the probe follow the sample axial movement to compensate for the axial movement of the sample; (V) A communication channel based on TCP / IP is established between the host computer and the robot controller, and a motion compensation control loop is established.

5. The method of operating a motion-compensated assisted OCT system of claim 4, wherein: It comprises the following steps: (1) Start; (2) The CCD camera simultaneously collects RGB color images of the imaging area and the laser dot; (3) The host computer completes the image processing, identifies the position of the laser point, and obtains the sample surface position information; (4) When the detected laser point position changes more than 2 pixels compared with the y direction coordinate of the image center point, the mechanical arm motion compensation function is activated; specifically, when the laser point coordinate is less than the center point coordinate, it indicates that the probe is too far away from the sample, and the probe moves downward to reduce the distance, otherwise, the probe moves upward to increase the distance, so that the distance between the probe and the sample is always kept near the ideal position.

6. The method of operating a motion-compensated assisted OCT system of claim 5, wherein: In step (3), the pixel position of the laser point center is obtained by using the HSV color space threshold processing and morphological processing method.

Images