Adaptive endoscope system and method based on photoacoustic image structural feature recognition

By adopting photoacoustic image structure feature recognition technology and adaptive zoom lens in the photoacoustic endoscope system, combined with photoacoustic and white light imaging technology, the shortcomings of the existing photoacoustic endoscope in deep narrow cavity channels and open cavity environments are solved, real-time high-resolution confocal imaging and multi-dimensional data detection are achieved, and the diagnostic capabilities of the endoscope are enhanced.

CN119302588BActive Publication Date: 2025-03-28SOUTH CHINA NORMAL UNIV +1
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411847178.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-12-16
Publication Date
2025-03-28
Estimated Expiration
2044-12-16

AI Technical Summary

Technical Problem

The existing photoacoustic endoscopes are limited in size when penetrated into the narrow cavity of the human body and cannot penetrate deep; and are not applicable in open cavity environments, and they are not able to achieve real-time zoom positioning and quantitative analysis of photoacoustic image characteristics.

Method used

Adaptive endoscopic system based on feature recognition of photoacoustic image structures, including control system, light source system, endoscopic probe, feedback system, signal processing system and computing terminal, the blood vessel structure is identified through feature recognition algorithms, and the zoom lens is adjusted to achieve adaptive adjustment of the focal length. Combined with photoacoustic imaging and white light imaging technology, real-time high-resolution confocal imaging is achieved.

Benefits of technology

It realizes the integration of adaptive photoacoustic and optical imaging methods, can perform real-time high-resolution confocal imaging, is suitable for in-situ diagnosis of a variety of malignant tumors, provides multi-dimensional data detection devices and methods, and enhances the depth detection ability of the endoscope and the recognition ability of abnormal tissues.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119302588B_ABST
    Figure CN119302588B_ABST
Patent Text Reader

Abstract

The invention discloses an adaptive endoscope system and method based on photoacoustic image structural feature recognition, which relates to the technical field of photoacoustic endoscopic imaging. The system comprises a control system, a light source system, an endoscope probe, a feedback system, a signal processing system, and a computing terminal; the control system synchronously triggers the light source system and the signal processing system, the light source system comprises a white light source and a laser, the laser excitation light enters the endoscope probe through a single-mode optical fiber, the inside of the probe is scanned by a micro-electromechanical system (micro-electromechanical system scanning micro-mirror) scanning micro-mirror, the scanning light is incident on a focusing lens group via a relay lens, and irradiates the surface of the tissue to be imaged through a liquid lens, the echo signal is collected and processed by a signal processing system and the photoacoustic image reconstruction is completed, the feedback system is connected to the liquid lens inside the probe, and the optical image is collected and reconstructed by the eyepiece at the top of the endoscope and the matching CMOS.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of photoacoustic endoscopic imaging, and more particularly to an adaptive endoscope system and method based on photoacoustic image structural feature recognition. Background Art

[0002] A medical endoscope is an endoscopic tool used for minimally invasive medical treatment of the human body. It enters the human body through natural cavities or small surgical incisions. By guiding the endoscope into the tissues and organs to be examined, high-resolution optical images can be obtained, which can effectively diagnose and treat the patient's internal organs. Different types of endoscopic probes are suitable for different clinical application scenarios, expanding the application field of optical imaging. The provision of tissue structure information has brought convenience to doctors' diagnosis and treatment.

[0003] Photoacoustic imaging is an emerging medical imaging technology that combines the high resolution of optical imaging with the high penetration depth of ultrasound imaging. When a pulsed laser is irradiated into biological tissue, the light-absorbing medium within the tissue absorbs the laser energy and is stimulated to generate an ultrasonic signal. The rapidly heated and expanded medium produces a photoacoustic signal that carries information about the tissue's light absorption characteristics. An ultrasonic transducer receives the photoacoustic signal and reconstructs an image of the corresponding light absorption distribution within the tissue. Photoacoustic imaging reflects differences in light absorption within biological tissue absorbers, eliminating the need for exogenous probes.

[0004] In recent years, many teams at home and abroad have proposed many photoacoustic endoscope solutions. Patent No. 201210186582.8 proposes a forward-looking photoacoustic endoscope for intracavitary tissue imaging. The probe scanning structure is a piezoelectric scanning structure, which limits the probe size and cannot penetrate deep into the narrow cavities of the human body. At the same time, multiple scans are required for reconstruction; Application No. 201710845921.1 proposes a photoacoustic endoscope based on adaptive focusing of a translucent and sound-transmitting water bag and its implementation method. The overall structure of the endoscope is a soft mirror, which requires a motor to control the scanning area and angle. It is not suitable for open intracavitary environments such as the abdominal cavity. At the same time, the patent realizes adaptive focusing based on the boundary of the water bag, does not perform quantitative analysis of the photoacoustic image characteristics, and cannot realize real-time zoom positioning during surgery. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the present invention aims to provide an adaptive endoscope system and method based on photoacoustic image structural feature recognition.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] An adaptive endoscope system based on photoacoustic image structural feature recognition includes a control system, a light source system, an endoscope probe, a feedback system, a signal processing system, and a computing terminal. The endoscope probe is connected to the light source system, the control system, the feedback system, and the signal processing system. The feedback system, the signal processing system, and the control system are respectively connected to the computing terminal.

[0008] The control system generates a pulse sequence according to the signal timing, controls the light emission of the first pulse laser and the second pulse laser, triggers the signal acquisition card, and moves the MEMS scanning micromirror;

[0009] The light source system includes a first pulse laser, a second pulse laser, and a white light source. The first pulse laser, the second pulse laser, and the white light source control terminal are all connected to the FPGA development board. The outputs of the first pulse laser and the second pulse laser are connected to optical fibers through a fiber coupler and connected to an endoscope probe.

[0010] The endoscope probe includes an ultrasonic transducer, a CMOS, an eyepiece, a spectroscope, a MEMS scanning micromirror, a first collimator, a second collimator, a reflector, a focusing lens, a dichroic mirror, a second optical fiber bundle, a relay lens, a focusing lens group, a liquid lens, and a light-transmitting reflector. The output light of the first pulse laser and the second pulse laser enters the endoscope probe through the first collimator and the second collimator, is focused by the focusing lens, and then enters the reflector. After being reflected, it enters the surface of the MEMS scanning micromirror. The scanning light is reflected by the spectroscope and then enters the relay lens. The output light of the relay lens passes through the focusing lens group, enters the light-transmitting reflector through the liquid lens, and then enters the tissue surface to be imaged.

[0011] The feedback system is composed of a zoom lens driving device. The feedback system receives feedback signals from a computing terminal. The feedback signals are obtained by the computing terminal identifying the blood vessel structure through a feature recognition algorithm, extracting and analyzing the blood vessel features, and adjusting the voltage of the zoom lens driving device to change the curvature of the liquid inside the liquid lens, thereby achieving adaptive adjustment of the focal length.

[0012] The signal processing system includes a signal amplifier and a signal acquisition card. When a pulsed laser is incident on the tissue surface, the surface is stimulated to generate ultrasonic waves, which are received by the ultrasonic transducer on the side wall of the endoscope probe. The transducer converts the ultrasonic signal into an electrical signal. The electrical signal is amplified by the signal amplifier with noise reduction and then enters the signal acquisition card. The signal acquisition card performs analog-to-digital conversion on the photoacoustic signal and sends the original data to the computing terminal.

[0013] Furthermore, the adaptive endoscopy method based on photoacoustic image structural feature recognition includes the following steps:

[0014] Photoacoustic imaging: The control system generates a timing signal to control the first pulse laser and the second pulse laser to generate lasers. The output light of the first pulse laser and the second pulse laser enters the endoscope probe through the first collimator and the second collimator, is focused by the focusing lens, and then enters the reflector. After reflection, it enters the surface of the MEMS scanning micromirror. The scanning light is reflected by the spectrometer and then enters the relay lens. The output light of the relay lens passes through the focusing lens group and enters the light-transmitting reflector by the liquid lens, and then enters the surface of the tissue to be imaged. The photoacoustic signal obtains a photoacoustic blood vessel image through the signal processing system. The feature recognition network extracts the vascular network skeleton through the image segmentation algorithm, identifies the vascular structure, and calculates the structural information and functional information. The abnormal tissue area is identified by comparing the difference in photoacoustic images in different areas, and the zoom lens is adjusted for continuous zoom. The continuous zoom refers to the zooming of the light focus within the range of 1 to 2 mm from the probe surface with an accuracy of 0.02 mm. The zooming is used to enlarge or reduce the scanning area.

[0015] White light imaging: The control system generates a timing signal to control the white light source. The light outlet is connected to the light cone of the endoscope probe through the first optical fiber bundle. The light emitted from the light cone is irradiated to the tissue surface through the second optical fiber bundle on the side wall. The reflected light from the tissue surface is collected by the focusing lens group, relayed to the eyepiece through the relay lens, and then enters the CMOS. The CMOS converts the incident light signal into an electric charge output, and the electrical signal is input into the computing terminal through the data line.

[0016] Vascular feature recognition based on photoacoustic images: The vessel diameter, tortuosity, and number of vessels are identified through photoacoustic images, the blood flow velocity is calculated through the correlation between adjacent frames, and the blood oxygen saturation is obtained through the imaging intensity analysis algorithm.

[0017] Furthermore, structural information includes blood vessel diameter, tortuosity, and number of blood vessels, and functional information includes blood flow rate and blood oxygen saturation.

[0018] Furthermore, the blood vessel diameter is calculated as follows: the photoacoustic image is binarized, and the edge of the grayscale image corresponds to the blood vessel skeleton. The value of the center line of the blood vessel skeleton is assigned a high value to distinguish the skeleton from the blood vessel centerline. The slope of the skeleton and the perpendicular line at each position are calculated. The blood vessel diameter is calculated by taking the highest slope between two adjacent points.

[0019] Furthermore, the blood vessel tortuosity is calculated as follows: binarize the photoacoustic image, mark the connected blood vessel area, find the first element of the connected area, find the area where the connected area is smaller than two points, refine the center line of the connected area, find the endpoints of the connected area, calculate the linear distance between the two ends of the blood vessel and the actual length of the blood vessel, traverse each connected area, and store the sequential tortuosity of each connected area.

[0020] Furthermore, the number of blood vessels is calculated as follows: the photoacoustic image is binarized, the grayscale image area is the area occupied by the blood vessels, the number of pixels inside the area is calculated, and the number is divided by the entire imaging area to obtain the blood vessel area ratio.

[0021] Furthermore, the blood flow velocity is calculated as follows: by performing multiple B-scans on the same section, different pixels in multiple images are retained, and pixels without differences are removed, and the correlation is calculated.

[0022] Furthermore, blood oxygen saturation is calculated as follows: imaging intensity is proportional to the product of the substance's absorption coefficient of light and the substance concentration. 532 nm and 558 nm correspond to the absorption peaks of oxyhemoglobin and deoxyhemoglobin, respectively. The corresponding absorption intensity is the sum of the products of the absorption coefficients of different substances and the substance concentration. Solving the absorption intensity equation can obtain blood oxygen saturation.

[0023] Furthermore, the feature recognition network uses an image segmentation algorithm to extract the vascular network skeleton, identify the vascular structure, calculate the vascular diameter, tortuosity, number of blood vessels, blood flow rate, and blood oxygen saturation. Through feature recognition, a database of differences between normal tissue and abnormal areas is established. During the photoacoustic imaging process, the image feature differences are compared. After identifying the abnormal tissue, the zoom lens is driven to enlarge the scanning area to obtain a photoacoustic image with more feature points.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] 1. The present invention realizes the integration of adaptive photoacoustic and optical imaging methods, and can perform real-time high-resolution confocal imaging. The photoacoustic imaging technology combines the characteristics of high resolution of optical imaging and deep depth of acoustic imaging. It can image the tumor microvascular network, determine the depth of tumor infiltration, and provide an important reference for determining tumor staging. It can also be used for postoperative screening, imaging the microstructure morphology of the marginal area, and determining whether there is residual at the resection margin. Compared with traditional white light endoscopes, the present invention has the characteristics of information integration, tomographic imaging, high resolution, and deep imaging depth. It is suitable for in vivo diagnosis of various malignant tumors. It can combine the advantages of various imaging devices to obtain medical images of tissues and organs, and provide a multi-dimensional data detection device and method for the field of medical imaging.

[0026] 2. The combination of a light-transmitting acoustic mirror and a rod lens within the endoscopic probe of the present invention enables optical beam combining and light path multiplexing, enabling real-time, high-resolution confocal imaging of the target tissue region, and simultaneous acquisition of photoacoustic and white-light images. The photoacoustic excitation light is focused by the rod lens onto the tissue to be imaged, extending the working distance of the endoscope. The photoacoustic excitation light enters the biological tissue, and the echo signal is reflected by the light-transmitting acoustic mirror and received by a lateral transducer.

[0027] 3. The endoscopic probe of this invention achieves dual-wavelength confocality through the use of an adjustable collimator. The different wavelengths of the photoacoustic excitation light cause dispersion when passing through the same lens, reducing image resolution. By using an adjustable collimator, the position of the incident light spot on the tissue is changed, achieving confocality of photoacoustic images at different wavelengths and improving image resolution.

[0028] 4. The internal scanning structure of the multimodal endoscope probe of the present invention adopts a micro-electromechanical system (MEMS) scanning micromirror. Compared with galvanometer scanning, the micro-electromechanical system (MEMS) scanning micromirror can reduce the size of the endoscope and reduce the weight of the endoscope, while having a high reflectivity to the incident light.

[0029] 5. The internal lens structure of the multimodal endoscopic probe of the present invention can change the focal position of the photoacoustic system and extend the working distance of the multimodal imaging system. The endoscopic probe can penetrate deep into narrow cavity organs for in-vivo detection. By adjusting the number of lens groups, the system working distance can be changed, thereby achieving high-resolution confocal endoscopic imaging of cavity organ tissues at different depths. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a system module diagram of an adaptive endoscope system based on photoacoustic image structural feature recognition;

[0031] Figure 2 Schematic diagram of the structure of an endoscope based on photoacoustic image structural feature recognition;

[0032] Figure 3 Schematic diagram of the structure of the endoscope probe.

[0033] Explanation of the accompanying drawings: 1. First pulse laser; 2. First fiber coupler; 3. Second pulse laser; 4. Second fiber coupler; 5. Endoscope probe; 6. Ultrasonic transducer; 7. Signal acquisition card; 8. White light source; 9. First fiber bundle; 10. Zoom lens drive device; 11. FPGA development board; 12. Computing terminal; 13. CMOS; 14. Eyepiece; 15. Spectrometer; 16. MEMS scanning micromirror; 17. First collimator; 18. Second collimator; 19. Reflector; 20. Focusing lens; 21. Dichroic mirror; 22. Second fiber bundle; 23. Relay lens; 24. Focusing lens group; 25. Liquid lens; 26. Transparent reflector. DETAILED DESCRIPTION

[0034] Reference Figure 1-Figure 3 As shown, this embodiment discloses an adaptive endoscope system based on photoacoustic image structural feature recognition of the present invention, including a control system, a light source system, an endoscope probe 5, a feedback system, a signal processing system, and a computing terminal 12.

[0035] Among them, the control system uses the FPGA development board 11 as the control module, generates a pulse sequence according to the signal timing to control the first pulse laser 1, the second pulse laser 3, the signal acquisition card 7, and the MEMS scanning micromirror 16; the light source system includes the first pulse laser 1, the second pulse laser 3, and the white light source 8; the endoscope probe 5 is composed of an integrated endoscope, including CMOS 13, an eyepiece 14, a spectrometer 15, a first collimator 17, a second collimator 18, a reflector 19, a focusing lens 20, a MEMS scanning micromirror 16, a dichroic mirror 21, a first optical fiber bundle 9, a relay lens 23, a focusing lens group 24, a liquid lens 25, a light-transmitting reflector 26, an ultrasonic transducer 6, and an endoscope housing; the data acquisition and reconstruction system includes a signal amplifier, a signal acquisition card 7, and a computing terminal 12; and the feedback system is composed of a zoom lens drive device 10.

[0036] The control system uses an FPGA development board 11 as a control module, generates a pulse sequence according to the signal timing to control the first pulse laser 1, and the laser output light is coupled into the single-mode optical fiber via the first optical fiber coupler 2, and the single-mode optical fiber is connected to the first collimator 17 in the endoscopic probe 5; generates a pulse sequence to control the second pulse laser 3, and the laser output light is coupled into the single-mode optical fiber via the second optical fiber coupler 4, and the single-mode optical fiber is connected to the second collimator 18 in the endoscopic probe 5; generates a pulse sequence to control the scanning trajectory of the MEMS scanning micromirror 16; and generates a pulse sequence to control the acquisition of the control signal acquisition card 7.

[0037] In the endoscopic probe 5, the light emitted from the first collimator 17 is reflected by the dichroic mirror 21 and combined with the incident light from the second collimator 18. The combined light is focused by the focusing lens 20 and then enters the reflective mirror 19, changing the direction of the light to enter the MEMS scanning micromirror 16. The MEMS scanning micromirror 16 controls the trajectory of the scanning light and reflects it through the beam splitter 15 to enter the relay lens 23. The light emitted from the lens passes through the focusing lens group 24 and is incident on the light-transmitting reflective mirror 26 by the liquid lens 25 to reach the surface of the tissue to be imaged. The ultrasonic signal generated on the tissue surface is received by the ultrasonic transducer 6 on the side wall of the endoscope, and enters the computing terminal 12 after passing through the signal acquisition card 7 to complete the reconstruction of the photoacoustic image.

[0038] The addition of the dichroic mirror 21 allows photoacoustic excitation lights of different wavelengths to share the same optical path. Short-wavelength pulsed light is incident vertically on the dichroic mirror 21, while long-wavelength pulsed light is incident horizontally on the dichroic mirror 21, thereby achieving excitation light beam combination.

[0039] The addition of the first collimator 17 and the second collimator 18 enables the confocal focus of dual-wavelength photoacoustic imaging. Lasers of different wavelengths disperse when passing through the same lens. By adjusting the adjustable focus collimator, the confocal focus of the imaging is achieved, allowing clear images to be obtained at the same location under different imaging modes.

[0040] The addition of the first pulsed laser 1 and the second pulsed laser 3 enables photoacoustic blood oxygen functional imaging. Lasers of different wavelengths correspond to the absorption differences between hemoglobin and deoxyhemoglobin. By calculating the changes in photoacoustic image intensity at different wavelengths, the blood oxygen saturation in the imaging area can be calculated.

[0041] In this embodiment, the optical fiber on the side wall of the endoscopic probe 5 is used to transmit the illuminating white light, and the light reflected from the tissue surface passes through the beam splitter 15 via the relay lens 23 .

[0042] In this embodiment, the laser focusing light path is focused and reflected by the reflector to the surface of the MEMS scanning micromirror 16. The control system controls the deflection of the MEMS scanning micromirror 16 to adjust the scanning range. Compared with the galvanometer scanning, the MEMS scanning micromirror 16 can reduce the volume of the endoscope and reduce the quality of the endoscope, while having a high reflectivity to the incident light.

[0043] The pulsed laser inside the endoscopic probe 5 is focused through the laser focusing optical path and then reflected by the reflector 19 to the surface of the MEMS scanning micromirror 16. The control system controls the deflection of the MEMS scanning micromirror 16 to adjust the scanning range. Compared with the galvanometer scanning, the MEMS scanning micromirror 16 reduces the volume of the endoscope and reduces the quality of the endoscope. At the same time, the surface of the reflector 19 is coated with an aluminum film, which has a high reflectivity to the incident light.

[0044] The dichroic mirror 21 is a long-wavelength dichroic mirror with a cutoff wavelength of 535-555 nm.

[0045] The driving process of the zoom lens driving device 10 is as follows: the zoom lens inside the endoscopic probe 5 is connected to the zoom lens driving device 10 through a control line. The zoom lens driving device 10 realizes continuous zooming of the zoom lens by continuously changing the voltage according to the signal feedback from the computing terminal 12. According to the corresponding relationship between the driving voltage and the focal length of the zoom lens, the voltage range in which the focus is 1~2 mm away from the probe surface is selected as a continuous zoom range, where the zoom accuracy is 0.02 mm.

[0046] In this embodiment, white light guides the probe to the tissue area to be imaged for scanning, and the endoscopic probe uses a forward scanning mode. The use of a rod lens can extend the working distance. The addition of a light-transmitting reflector 26 at the front end of the probe reflects the photoacoustic echo signal, which is received by the high-frequency ultrasonic transducer 6 on the side wall. The high-frequency ultrasonic transducer 6 has a main frequency of 40 MHz and a bandwidth of 80%.

[0047] In this embodiment, the signal amplifier has an amplification gain of 60 dB.

[0048] In this embodiment, the endoscope housing is a metal housing with biocompatibility, the working distance is 20 cm, and the endoscope diameter is 10 mm.

[0049] The present invention utilizes an adaptive endoscopy method based on photoacoustic image structural feature recognition, comprising the following steps:

[0050] Photoacoustic imaging: The control system generates a timing signal to control the pulsed laser to generate laser light. The pulsed laser relays the optical path through the rod lens inside the endoscope. The emitted focused light is refocused by the focusing lens group 24. The focused light enters the liquid lens 25 and reaches the surface of the tissue to be imaged. The photoacoustic signal is processed by the signal processing system to obtain a photoacoustic vascular image. The feature recognition network uses an image segmentation algorithm to extract the vascular network skeleton, identify the vascular structure, and calculate structural information such as vessel diameter, tortuosity, and density, as well as functional information such as blood flow rate and blood oxygen saturation. Abnormal tissue areas are identified by comparing the differences in photoacoustic images in different areas. The zoom lens is adjusted for continuous zooming. Continuous zooming refers to zooming the optical focus within a range of 1 to 2 mm from the probe surface with an accuracy of 0.02 mm. This zooming can enlarge or reduce the scanning area.

[0051] White light imaging: The control system generates a timing signal to control the white light source 8. The light outlet is connected to the light cone part of the endoscope through the first optical fiber bundle 9. The light emitted from the light cone is irradiated to the tissue surface through the first optical fiber bundle 9 on the side wall. The reflected light on the tissue surface is collected by the lens group, relayed to the eyepiece 14 through the relay lens 23 and enters the CMOS13. The CMOS13 can convert the incident light signal into an electric charge output and enter the electrical signal into the computer through the data line.

[0052] Vascular feature recognition based on photoacoustic images: The photoacoustic image is used to identify vessel diameter, tortuosity, density, and other information. Blood flow velocity is calculated through correlation between adjacent frames, and blood oxygen saturation is obtained using an imaging intensity analysis algorithm. To calculate vessel diameter, the photoacoustic image is binarized. The edges of the grayscale image correspond to the vessel skeleton. The centerline of the vessel skeleton is assigned a high value to distinguish the skeleton from the vessel centerline. The slope of the skeleton and the perpendicular line at each location are calculated. The vessel diameter is calculated by taking the slope of the two adjacent points. To calculate vessel tortuosity, the photoacoustic image is binarized, connected regions (vessel regions) are marked, the first element of the connected region is found, and regions with a connected region smaller than two points are found. The centerline of the connected region is thinned, the endpoints of the connected region are found, and the linear distance between the two ends of the vessel and the actual length of the vessel are calculated. Each connected region is traversed, and the sequential tortuosity of each connected region is stored. To calculate the number of vessels, the photoacoustic image is binarized, and the grayscale image region represents the area occupied by the vessel. The number of pixels within the region is calculated and divided by the total imaging area to obtain the vessel area percentage. Calculating blood flow velocity: Multiple B-scans are performed on the same section, retaining pixels (blood flow) that differ across the images and removing pixels (tissue) that do not differ. Correlation is then calculated. Blood oxygen saturation: Imaging intensity is proportional to the product of a substance's absorption coefficient and its concentration. 532nm and 558nm correspond to the absorption peaks of oxyhemoglobin and deoxyhemoglobin, respectively. The corresponding absorption intensity is the sum of the product of the absorption coefficients of the different substances and their concentrations. Solving the absorption intensity equation yields blood oxygen saturation.

[0053] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0054] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. An adaptive endoscope system based on photoacoustic image structural feature recognition, characterized in that: It comprises a control system, a light source system, an endoscope probe (5), a feedback system, a signal processing system, and a computing terminal (12), wherein the endoscope probe (5) is connected to the light source system, the control system, the feedback system, and the signal processing system, and the feedback system, the signal processing system, and the control system are respectively connected to the computing terminal (12); The control system generates a pulse sequence according to the signal timing, controls the light emission of the first pulse laser (1), the second pulse laser (3), the triggering of the signal acquisition card (7), and the movement of the MEMS scanning micromirror (16); The light source system comprises a first pulse laser (1), a second pulse laser (3), and a white light source (8); the control ends of the first pulse laser (1), the second pulse laser (3), and the white light source (8) are all connected to an FPGA development board (11); the outputs of the first pulse laser (1) and the second pulse laser (3) are connected to an optical fiber via an optical fiber coupler and connected to an endoscope probe (5); The endoscope probe (5) comprises an ultrasonic transducer (6), a CMOS (13), an eyepiece (14), a spectroscope (15), a MEMS scanning micromirror (16), a first collimator (17), a second collimator (18), a reflector (19), a focusing lens (20), a dichroic mirror (21), a second optical fiber bundle (22), a relay lens (23), a focusing lens group (24), a liquid lens (25), and a light-transmitting reflector (26); a first pulse laser (1) and a second pulse laser (1) are connected to each other. The output light of the laser (3) enters the endoscope probe (5) via the first collimator (17) and the second collimator (18), is focused by the focusing lens (20), and then enters the reflective mirror (19), and then enters the surface of the MEMS scanning micromirror (16), the scanning light is reflected by the beam splitter (15), and then enters the relay lens (23), the output light of the relay lens (23) passes through the focusing lens group (24), and enters the light-transmitting reflective mirror (26) via the liquid lens (25), and then enters the surface of the tissue to be imaged; The feedback system is composed of a zoom lens driving device (10), and the feedback system receives a feedback signal from a computing terminal (12), wherein the feedback signal is obtained by the computing terminal (12) identifying the blood vessel structure through a feature recognition algorithm, extracting blood vessel features for analysis, and adjusting the voltage of the zoom lens driving device (10) to change the curvature of the liquid inside the liquid lens (25), thereby realizing adaptive adjustment of the focal length; The signal processing system comprises a signal amplifier and a signal acquisition card (7). When a pulsed laser is incident on a tissue surface, the surface is stimulated to generate ultrasonic waves which are received by an ultrasonic transducer (6) on the side wall of an endoscope probe (5). The transducer converts the ultrasonic signal into an electrical signal. The electrical signal is amplified by the signal amplifier for noise reduction and then enters the signal acquisition card (7). The signal acquisition card (7) performs analog-to-digital conversion on the photoacoustic signal and then sends the original data to a computing terminal (12). The system works in the following steps: Photoacoustic imaging: The control system generates a timing signal to control the first pulse laser (1) and the second pulse laser (3) to generate laser light. The output light of the first pulse laser (1) and the second pulse laser (3) enters the endoscope probe (5) through the first collimator (17) and the second collimator (18). After being focused by the focusing lens (20), the light enters the reflection mirror (19). After being reflected, the light enters the surface of the MEMS scanning micromirror (16). The scanning light is reflected by the beam splitter (15) and enters the relay lens (23). The output light of the relay lens (23) passes through the focusing lens group (2 4) The liquid lens (25) is incident on the light-transmitting reflector (26), and then on the surface of the tissue to be imaged. The photoacoustic signal is processed by the signal processing system to obtain a photoacoustic vascular image. The feature recognition network extracts the skeleton of the vascular network through an image segmentation algorithm, identifies the vascular structure, and calculates the structural information and functional information. The abnormal tissue area is identified by comparing the difference in the photoacoustic images in different areas, and the zoom lens is adjusted to continuously zoom. The continuous zoom refers to the light focus being zoomed with an accuracy of 0.02 mm within a range of 1 to 2 mm from the probe surface, and the scanning area is enlarged or reduced by zooming; White light imaging: the control system generates a timing signal to control the white light source (8), the light outlet is connected to the light cone part of the endoscope probe (5) through the first optical fiber bundle (9), the light cone output light is irradiated to the tissue surface through the second optical fiber bundle (22) on the side wall, the light reflected from the tissue surface is collected by the focusing lens group (24), relayed to the eyepiece (14) through the relay lens (23), and then enters the CMOS (13), the CMOS (13) converts the incident light signal into an electric charge output, and inputs the electric signal into the computing terminal (12) through the data line; Vascular feature recognition based on photoacoustic images: Identify vascular diameter, curvature, and number of blood vessels through photoacoustic images, calculate blood flow velocity through the correlation between adjacent frames, and obtain blood oxygen saturation through imaging intensity analysis algorithm; Structural information includes blood vessel diameter, tortuosity, and number of blood vessels, while functional information includes blood flow rate and blood oxygen saturation.

2. An adaptive endoscopy method based on photoacoustic image structural feature recognition, characterized in that: The adaptive endoscope system based on photoacoustic image structural feature recognition as described in claim 1 is characterized in that the blood vessel diameter is calculated as follows: the photoacoustic image is binarized, the edge of the grayscale image corresponds to the blood vessel skeleton, the value of the center line of the blood vessel skeleton is assigned a high value to distinguish the skeleton from the blood vessel center line, the slope of the skeleton and the perpendicular line at each position are calculated, and the blood vessel diameter is calculated by taking the slope of the two adjacent points as the maximum slope.

3. The adaptive endoscopy method based on photoacoustic image structural feature recognition according to claim 2, characterized in that: The calculation method of blood vessel tortuosity is as follows: binarize the photoacoustic image, mark the connected blood vessel area, find the first element of the connected area, find the area where the connected area is smaller than two points, refine the center line of the connected area, find the endpoints of the connected area, calculate the linear distance between the two ends of the blood vessel and the actual length of the blood vessel, traverse each connected area, and store the sequential tortuosity of each connected area.

4. The adaptive endoscopy method based on photoacoustic image structural feature recognition according to claim 2, characterized in that: The number of blood vessels is calculated as follows: binarize the photoacoustic image, the grayscale image area is the area occupied by the blood vessels, the number of pixels inside the area is calculated, and then divided by the entire imaging area to obtain the blood vessel area ratio.

5. The adaptive endoscopy method based on photoacoustic image structural feature recognition according to claim 2, characterized in that: The blood flow velocity is calculated as follows: through multiple B-scans of the same section, the pixels with differences in multiple images are retained, and the pixels with no differences are removed, and the correlation is calculated.

6. The adaptive endoscopy method based on photoacoustic image structural feature recognition according to claim 2, characterized in that: The calculation method of blood oxygen saturation is as follows: the imaging intensity is proportional to the product of the absorption coefficient of the substance to light and the substance concentration. 532nm and 558nm correspond to the absorption peaks of oxygenated hemoglobin and deoxygenated hemoglobin, respectively. The corresponding absorption intensity is the sum of the products of the absorption coefficients of different substances and the substance concentration. Solving the absorption intensity equation can obtain the blood oxygen saturation.

7. The adaptive endoscopy method based on photoacoustic image structural feature recognition according to claim 2, characterized in that: The feature recognition network extracts the vascular network skeleton through the image segmentation algorithm, identifies the vascular structure, calculates the vascular diameter, tortuosity, number of blood vessels, blood flow rate, and blood oxygen saturation. Through feature recognition, a database of differences between normal tissue and abnormal areas is established. During the photoacoustic imaging process, the image feature differences are compared. After identifying the abnormal tissue, the scanning area is enlarged by driving the zoom lens to obtain a photoacoustic image with more feature points.

Citation Information

Patent Citations

  • Forward-looking photoacoustic endoscope

    CN103462645B

  • Self-adaptive focusing photoacoustic endoscope based on light-transmittable sound-transmittable water sac and implementation method thereof

    CN107638168A

  • Electronic endoscope system

    CN102247115A

  • Variable-focus 3-D capsule endoscope system based on liquid lens

    CN104939793A

  • Arthroscope device integrating optical-photoacoustic multi-mode imaging and imaging method thereof

    CN118986260A