AI robot assisted esophageal ultrasonic detection system and method

Abstract

The invention discloses an AI robot assisted esophageal ultrasonic detection system and method. The system comprises a control type mechanical arm and a multi-mode sensing probe. The AI robot control module is configured to control the insertion depth, the deflection angle and the contact pressure of the multi-mode sensing probe through the control type mechanical arm; the signal processing module is used for carrying out synchronous acquisition and noise reduction processing on the temperature data, the ultrasonic imaging data, the optical image data, the contact force data, the impedance data and the blood flow signals; and the AI engine module is based on multi-modal data fusion, is provided with a pre-trained deep learning model, and is used for generating an esophageal wall structure in real time, extracting hemodynamics and focus features and optimizing an examination path. Through a multi-mode sensing probe, intelligent robot control and a multi-source data fusion AI algorithm, accurate evaluation of the esophageal structure and function is realized. The accuracy, the safety and the efficiency of digestive tract disease diagnosis are remarkably improved by combining adaptive path planning and a lesion automatic classification technology.

Description

Technical Field

[0001] The present invention relates to the technical field of computer-aided artificial intelligence medical robots, and particularly to an AI robot-assisted esophageal ultrasound detection system and method. Background Art

[0002] Early diagnosis of esophageal diseases is crucial for improving the prognosis of patients, especially for malignant lesions such as esophageal cancer. Currently, ultrasound examination and endoscopic examination are the main means for screening esophageal lesions, but traditional methods have significant limitations. The interpretation of ultrasound images highly depends on the technical level of the operator, and the diagnostic consistency among doctors with different experiences varies greatly, which may lead to missed diagnosis or misdiagnosis. For example, the differential diagnosis of esophageal leukoplakia highly depends on the subjective judgment of endoscopic doctors due to the high similarity of lesion characteristics, resulting in the risk of unnecessary biopsies or missed diagnosis. The lack of operation standardization greatly affects the acquisition quality of ultrasound images by the doctor's manipulation. Especially in the scenario of telemedicine, grass-roots doctors may cause images not to meet the standards due to insufficient technology, affecting the judgment of consultation experts. The traditional ultrasound examination process takes a long time, and the resources of high-level doctors are limited, making it difficult to meet the grass-roots medical needs.

[0003] The existing esophageal detection technologies have the following problems. First, single-modal limitation: traditional ultrasound probes cannot synchronously detect tissue mechanical properties and blood flow information. Second, traditional endoscopic probes are limited by space constraints and are difficult to meet multi-modal imaging and navigation. Third, operation depends on experience: manual intubation is likely to cause mucosal damage and it is difficult to accurately locate the lesion area. Fourth, lack of intelligence: lack the ability of multi-modal data fusion and real-time pathological analysis. Summary of the Invention

[0004] Aiming at the deficiencies of the prior art, the present invention provides an AI (Artificial Intelligence) robot-assisted esophageal ultrasound detection system and method, which provides a solution for medical image diagnosis through artificial intelligence AI technology. The AI-assisted system can significantly improve the diagnostic accuracy by analyzing ultrasound images through deep learning.

[0005] To achieve the above object, the present invention provides the following technical solutions.

[0006] On the one hand, the present invention provides an AI robot-assisted esophageal ultrasound detection system, and the system includes: a manipulative robotic arm, which is equipped with an end effector and a multi-degree-of-freedom motion mechanism;

[0007] A multi-modal sensing probe connected to the end effector, including a temperature sensor array, an ultrasound transducer, an optical imaging module, a contact force sensor, an impedance measurement unit, and a Doppler blood flow detection module;

[0008] The AI robot control module is configured to control the insertion depth, deflection angle, and contact pressure of the multimodal sensing probe through the manipulative robotic arm;

[0009] The signal processing module synchronously collects and denoises temperature data, ultrasonic imaging data, optical image data, contact force data, impedance data, and blood flow signals;

[0010] The AI engine module, based on multimodal data fusion, is configured with a pre-trained deep learning model to generate esophageal wall structure, hemodynamics, lesion feature extraction, and examination path optimization in real time.

[0011] In some embodiments, the structure of the multimodal sensing probe is a three-layer flexible package: the outer layer is a biocompatible silicone sleeve with multiple scale marks on the surface; the middle layer is a flexible circuit board integrating a sensor signal conditioning circuit and a micro heat pipe heat dissipation structure; the inner layer is a coaxial cable and a guide wire;

[0012] The multimodal sensing probe has a terminal structure, and the terminal structure includes an ultrasonic transducer and a zoom lens. The ultrasonic transducer is arranged on one side of the outer periphery of the zoom lens, and the optical path and the acoustic path of the multimodal sensing probe are perpendicular.

[0013] In some embodiments, the temperature sensor array is used to monitor the surface temperature distribution of the multimodal sensing probe in real time; the impedance measurement unit is used to detect the contact impedance characteristics between the multimodal sensing probe and the esophageal wall; the contact force sensor is used to ensure stable contact between the multimodal sensing probe and the tissue; the Doppler blood flow detection module works in cooperation with the ultrasonic transducer through frequency division multiplexing.

[0014] In some embodiments, the AI robot control module includes:

[0015] The tactile feedback unit dynamically adjusts the motion state of the multimodal sensing probe according to the magnitude and distribution of the contact force of the contact force sensor;

[0016] Dynamically adjusting the motion state of the multimodal sensing probe according to the magnitude and distribution of the contact force of the contact force sensor includes: executing a tactile feedback control logic, and the tactile feedback control logic includes one or more of a global pressure control logic, an absolute pressure control logic, and a comparative pressure control logic.

[0017] In some embodiments, the AI engine module includes:

[0018] The multimodal data fusion network fuses temperature data, ultrasonic image data, optical image data, contact force data, blood flow signals, and impedance data based on the Transformer architecture;

[0019] A pathological classification model that identifies esophagitis, Barrett's esophagus, and early cancer through transfer learning;

[0020] A real-time visualization interface that outputs a pseudo-color blood flow superimposed image and a risk probability heat map.

[0021] In some embodiments, the AI engine module updates the path data every 1 s. Combining with the actual motion data of the robot control module, when the contact force is greater than 1.5 N, local replanning and bypassing are initiated.

[0022] In some embodiments, the signal processing module uses: a lock-in amplification technique to extract the quadrature I / Q components in the Doppler signal; a wavelet transform algorithm to suppress motion artifacts and electromagnetic interference; a timestamp synchronization mechanism to align ultrasound image frames with multi-modal sensor data.

[0023] In some embodiments, the global pressure control logic includes: during the movement of the multi-modal sensing probe, when the pressure gradient ▽F is within the threshold range, when the average contact force F avg <0.3 N, the multi-modal sensing probe accelerates; when 0.3 N < F avg <0.8 N, the multi-modal sensing probe maintains a constant speed; when F avg > 1.2 N, an emergency brake is triggered and the multi-modal sensing probe retreats;

[0024] The absolute pressure control logic includes: during the movement of the multi-modal sensing probe, when the contact force F of any contact force sensor is greater than 1.5 N, an emergency brake is triggered and the multi-modal sensing probe retreats; when the pressure gradient ▽F > 0.5 N / mm, the multi-modal sensing probe decelerates and fine-tunes the angle of the multi-modal sensing probe to relieve local stress concentration.

[0025] In some embodiments, the contact force sensor includes at least a first contact force sensor and a second contact force sensor. The first contact force sensor is arranged on the probe contact surface, and the second contact force sensor is arranged on the other side opposite to the probe contact surface;

[0026] The comparison pressure control logic includes: during the movement of the multi-modal sensing probe, if ultrasonic imaging is required, the magnitudes of the contact force F1 collected by the first contact force sensor and the contact force F2 collected by the second contact force sensor are judged. If F1 < F2, the angle of the multi-modal sensing probe is adjusted to deflect the multi-modal sensing probe in the direction of the probe contact surface.

[0027] On the other hand, the present invention provides an AI robot-assisted esophageal ultrasound detection method, which includes: constructing a patient-specific esophageal path atlas through preoperative imaging data; completing automatic probe placement by combining real-time force feedback and AI visual guidance; using multi-modal data fusion technology to achieve dynamic tracking of the esophageal wall structure; and automatically generating a structured inspection report according to the abnormal detection results.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] The present invention discloses an AI (Artificial Intelligence) robot-assisted esophageal ultrasound detection system, which adopts a multi-modal sensing probe with a special structure, makes full use of the limited space at the end of the probe structure, integrates optical imaging and ultrasonic imaging, and realizes precise tactile feedback control logic through innovative contact force sensor design and distribution. Through the multi-modal sensing probe, intelligent robot control and multi-source data fusion AI algorithm, precise evaluation of the esophageal structure and function is achieved. The system integrates functions of ultrasonic imaging, contact mechanics, impedance measurement and blood flow detection, combines adaptive path planning and automatic lesion classification technology, and significantly improves the accuracy, safety and efficiency of the diagnosis of digestive tract diseases. The system combines AI image analysis and miniaturized robot operation to realize the full-process automation from image acquisition, lesion recognition to report generation. Through the optimization of the adaptive path optimization algorithm, a feature fusion model is designed for multi-modal ultrasound data such as white light endoscopy and narrow band imaging to improve the ability to distinguish esophageal lesions. Through force feedback and real-time visual guidance, the accuracy and safety of the manipulator operation are optimized to support the collaborative diagnosis and treatment between primary medical institutions and superior hospitals. Description of the Drawings

[0030] Figure 1 Schematic diagram of an AI robot-assisted esophageal ultrasound detection system provided in some embodiments of the present invention.

[0031] Figure 2 Block diagram of an AI robot-assisted esophageal ultrasound detection system provided in some embodiments of the present invention.

[0032] Figure 3 Overall layout diagram of the multi-modal sensing probe of the system provided in some embodiments of the present invention.

[0033] Figure 4 Block diagram of the end structure of the multi-modal sensing probe of the system provided in some embodiments of the present invention.

[0034] Figure 5 Schematic diagram of the end structure of the multi-modal sensing probe of the system provided in some embodiments of the present invention.

[0035] Figure 6In some embodiments of the present invention Figure 5 Schematic diagram of cross-section A-A of the end structure

[0036] Figure 7 Schematic diagram of a zoom lens in a zoom state in some embodiments of the present invention

[0037] Figure 8 Schematic diagram of a zoom lens in another zoom state in some embodiments of the present invention

[0038] Figure 9 Flow chart of the AI robot-assisted esophageal ultrasound detection method in some embodiments of the present application Detailed implementation manners

[0039] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention

[0040] It should be noted that the terms used herein are only for describing specific implementation manners and are not intended to limit the exemplary implementation manners according to the present application. As used herein, unless otherwise clearly specified in the context, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "include" and / or "comprise" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof

[0041] Figure 1 Schematic diagram of an AI robot-assisted esophageal ultrasound detection system provided in some embodiments of the present invention Figure 2 Block diagram of an AI robot-assisted esophageal ultrasound detection system provided in some embodiments of the present invention. Refer to Figure 1 and Figure 2, in some embodiments, an AI (Artificial Intelligence) robot-assisted esophageal ultrasound detection system is provided. The system includes: a manipulative robotic arm 1000, which is equipped with an end effector 101, multi-degree-of-freedom motion mechanisms 102a, 102b, etc., a multi-modal sensing probe 2000 connected to the end effector, and a robot base 103. The multi-modal sensing probe 104 includes a temperature sensor array, an ultrasonic transducer, an optical imaging module, a contact force sensor, an impedance measurement unit, and a Doppler blood flow detection module; an AI robot control module 3000, which controls the insertion depth, deflection angle, and contact pressure of the multi-modal sensing probe by manipulating the robotic arm; a signal processing module 4000, which synchronously collects and denoises temperature data, ultrasonic imaging data, optical image data, contact force data, impedance data, and blood flow signals; an AI engine module 5000, which is configured with a pre-trained deep learning model based on multi-modal data fusion to generate esophageal wall structure, hemodynamics, lesion feature extraction, and examination path optimization in real time.

[0042] In the embodiments of the present application, the AI robot-assisted esophageal ultrasound detection system integrates ultrasonic imaging, contact mechanics, impedance measurement, and blood flow detection functions, and combines adaptive path planning and automatic lesion classification technologies to significantly improve the accuracy, safety, and efficiency of the diagnosis of digestive tract diseases.

[0043] In some embodiments, as Figure 3 shown is the overall layout diagram of the multi-modal sensing probe in this embodiment. The overall structure of the multi-modal sensing probe is a three-layer flexible package: the outer layer is a biocompatible silica gel sleeve 203 with multiple scale marks on the surface; the middle layer is a flexible circuit board 202, which integrates a sensor signal conditioning circuit and a micro heat pipe heat dissipation structure; the inner layer is a coaxial cable and a guide wire 201. In this multi-modal composite miniaturized probe, the silica gel sleeve on the outer layer has insulation and flexibility, the flexible circuit board in the middle layer integrates electrodes, a pre-amplification circuit, a signal conditioning circuit, and a micro heat pipe heat dissipation structure, the coaxial cable on the inner layer is used for signal transmission, and the guide wire on the inner layer plays a mechanical support role.

[0044] In the embodiments of the present application, through the overall structure of the three-layer flexible package of the multi-modal sensing probe, it is ensured that it can reliably enter the esophagus, provides stable signal transmission for the end structure of the multi-modal sensing probe, and ensures the stable operation of the multi-modal sensing probe.

[0045] In some embodiments, as Figure 4The following is a block diagram of the end structure of the multi-modal sensing probe of this embodiment. The end structure of the multi-modal sensing probe includes an optical imaging module 301, a contact force sensor 302, an ultrasonic transducer 303, a Doppler blood flow detection module 304, a temperature sensor array 305, and an impedance measurement unit 306. In the embodiment of the present application, the optical imaging module 301 is used to collect optical image data, the contact force sensor 302 is used to collect contact force data, the ultrasonic transducer 303 is used to collect ultrasonic imaging data, the Doppler blood flow detection module 304 is used to collect blood flow signals, the temperature sensor array 305 is used to collect temperature data, and the impedance measurement unit 306 is used to collect impedance data.

[0046] In some embodiments, Figure 5 is a schematic diagram of the end structure of the multi-modal sensing probe of the system provided in some embodiments of the present invention. Figure 6 is Figure 5 a schematic cross-sectional view of the end structure of. Refer to Figure 5 and Figure 6 , the optical imaging module forms an optical path 3011, and the ultrasonic transducer 303 forms an acoustic path 3031. The optical path 3011 and the acoustic path 3031 are generally perpendicular. In the embodiment of the present application, generally perpendicular means that the center line of the acoustic path 3031 (the direction indicated by the double arrow Y in the figure) is perpendicular to the center line of the optical path 3011 (the direction indicated by the double arrow X in the figure). The end structure generally adopts an asymmetric structure. In the embodiment of the present application, the end structure generally adopting an asymmetric structure means that the center lines of the optical path 3011 and the acoustic path 3031 do not coincide.

[0047] Specifically, the optical imaging module at least includes an end fixed lens 3012, a zoom lens 3013, an internal lens 3014, and an optoelectronic module 3015 arranged in sequence along the axis, and the axis is parallel to the optical path 3011. The axis can be the connection line of the centers of the fixed lens 3012, the zoom lens 3013, and the internal lens 3014. In some embodiments, the zoom lens 3013 uses a liquid medium for zooming. Specifically, the zoom lens 3013 includes a first zoom medium 3013-1 and a second zoom medium 3013-2. The first zoom medium 3013-1 and the second zoom medium 3013-2 are respectively arranged in a first zoom cavity and a second zoom cavity, and there is a flexible transparent film 3013-3 between the first zoom cavity and the second zoom cavity. The first zoom medium 3013-1 and the second zoom medium 3013-2 are transparent liquid media with different refractive indices. In the embodiment of the present application, by using the zoom lens 3013 with liquid medium zooming, optical zooming can be achieved. On the one hand, it can perform imaging of the distal esophagus to meet real-time visual guidance, and on the other hand, it can perform precise macro imaging of the esophageal wall to perform multi-modal imaging of optical images and ultrasonic images to meet the diagnostic requirements.

[0048] In some embodiments, Figure 6 Figure 5 The optical imaging module 301 further includes hydraulic conduits for changing the volumes of the first zoom cavity corresponding to the first zoom medium 3013-1 and the second zoom cavity corresponding to the second zoom medium 3013-2, so as to achieve precise zooming. Specifically, the hydraulic conduits include a first hydraulic conduit 3016 and a second hydraulic conduit 3017. The first hydraulic conduit 3016 communicates with the first zoom cavity, and the second hydraulic conduit 3017 communicates with the second zoom cavity. The other ends of the first hydraulic conduit 3016 and the second hydraulic conduit 3017 are connected to a hydraulic device which is arranged outside the body and is used to provide hydraulic force and achieve precise control of the hydraulic force.

[0049] In some embodiments, Figure 7 FIG. [X] is a schematic diagram of the zoom lens in one zoom state. Figure 8 FIG. [Y] is a schematic diagram of the zoom lens in another zoom state. Referring to Figure 7 and Figure 8 , in some embodiments, by respectively and real-time synchronously controlling the hydraulic pressures of the first zoom medium 3013-1 and the second zoom medium 3013-2 in the first zoom cavity and the second zoom cavity, the volumes of the first zoom cavity and the second zoom cavity are synchronously changed to achieve liquid medium zooming.

[0050] In some embodiments, the ultrasonic transducer 303 is arranged on one side of the outer periphery of the zoom lens 3013. The other side of the outer periphery of the zoom lens 3013 opposite to the ultrasonic transducer 303 is the probe contact surface 3018. There are multiple contact force sensors 302. In some implementations, the contact force sensors 302 at least include a first contact force sensor 302-1 and a second contact force sensor 302-2. The first contact force sensor 302-1 is arranged with its end structure on one side of the probe contact surface 3018, and the second contact force sensor 302-2 is arranged with its end structure on the other side opposite to the probe contact surface 3018, that is, the side corresponding to the ultrasonic transducer 303. In the embodiments of the present application, by comprehensively analyzing and comparing the magnitudes of the feedback forces of the first contact force sensor 302-1 and the second contact force sensor 302-2, the contact state between the probe contact surface and the esophageal wall can be judged more precisely.

[0051] In an embodiment of the present application, the ultrasonic transducer 303 is disposed on one side of the outer periphery of the zoom lens 3013. The acoustic path 3031 passes through the zoom lens 3013 and the probe contact surface 3018. The end structure as a whole adopts an asymmetric structure, and the optical path 3011 is perpendicular to the acoustic path 3031 as a whole. Through the asymmetric structure setting of the present application, the internal space layout of the lens barrel is ingeniously utilized, and the liquid medium of the zoom lens 3013 is perfectly utilized, avoiding the interference of air on the acoustics in the acoustic path 3031, and the imaging is more accurate. In an embodiment of the present application, the first zoom medium 3013-1 and the second zoom medium 3013-2 can be water and dielectric oil with little acoustic interference respectively. In some embodiments, the flexible transparent film 3013-3 is a deformable film with little acoustic interference, and the influence of the flexible transparent film can be removed by calibrating the image during ultrasonic imaging.

[0052] In some embodiments, specifically, the first hydraulic conduit 3016, the second hydraulic conduit 3017, and the ultrasonic transducer 303 are all disposed on the outer periphery of the zoom lens 3013, and the first hydraulic conduit 3016 and the second hydraulic conduit 3017 are disposed on both sides in the circumferential direction of the ultrasonic transducer 303, making full use of the axial and circumferential spaces, avoiding mutual interference, and minimizing the radial dimension as much as possible.

[0053] In some embodiments, the space outside the end fixed lens 3012 can also be fully utilized, and a light source 307 is disposed outside the end fixed lens 3012 to provide sufficient brightness for optical imaging and smoothly collect optical images. In some embodiments, the transmission line of the light source can be disposed in the space between the ultrasonic transducer 303 and the zoom lens 3013 and arranged in parallel with the first hydraulic conduit 3016 or the second hydraulic conduit 3017. In addition, to suppress crosstalk between light and sound, a band-pass filter is added to the optical receiving end to shield the stray light generated when the ultrasonic transducer works, and an acoustic absorption material is filled on the surface of the ultrasonic transducer 303 opposite to the optoelectronic module 3015 to reduce the interference of the backward sound wave on the optical module. In an embodiment of the present application, a moving component 3019 can also be axially disposed on the side of the optoelectronic module 3015 opposite to the internal lens 3014. The moving component 3019 is connected to the optoelectronic module 3015, and the optoelectronic module 3015 is connected to the internal lens 3014. The moving component 3019 can drive the optoelectronic module 3015 and the internal lens 3014 to move to adapt to the zoom imaging of the zoom lens. In some embodiments, the moving component 3019 can adopt mechanical drive or electromagnetic drive. The axial setting of the moving component 3019 can make full use of the axial space and have no impact on the radial space.

[0054] In some embodiments, the contact force sensor 302 employs an optical fiber force sensor. In the multimodal sensing probe, the optical fiber force sensor (such as an FBG grating) is embedded in the probe, and a conductive polymer is sprayed on the surface to achieve tactile and temperature sensing. The layout of the contact force sensor can meet the requirements of functional synergy, space limitation, signal isolation, and data fusion efficiency. The contact force sensor is used to ensure stable contact between the multimodal sensing probe and the tissue. In the embodiments of the present application, the contact force sensor 302 includes a first contact force sensor 302-1 and a second contact force sensor 302-2. By comparing and analyzing the magnitudes of the feedback forces of the first contact force sensor 302-1 and the second contact force sensor 302-2, the contact state between the probe contact surface and the esophageal wall can be judged more accurately. In some embodiments, the contact force sensor 302 further includes a third contact force sensor 302-3. The third contact force sensor 302-3 is disposed on the outer periphery of the end fixed lens 3012 and is coaxial with the external optical imaging module. The third contact force sensor 302-3 is used to judge the magnitude and distribution of the pressure at the front end of the multimodal sensing probe, so as to further control the forward speed and angle adjustment of the probe.

[0055] In some embodiments, for the layout of the ultrasonic transducer 303 and the Doppler blood flow detection module 304, in this embodiment, the ultrasonic transducer, the Doppler blood flow detection module, and a signal processing chip (not shown) are stacked. The ultrasonic transducer is located at the bottom layer, the Doppler module is located at the middle layer, and the signal processing chip is located at the top layer in a 3D stacked package. In the embodiments of the present application, a 100-μm aerogel thermal insulation layer is provided between the layers. Through this layout, the goals of structural miniaturization, high blood flow sensitivity, and real-time imaging are achieved, and the problems of interlayer thermal management and signal crosstalk are solved. In some embodiments, the transducer frequency is 5-20 MHz, the size of the Doppler blood flow detection module is less than or equal to 5 mm × 5 mm. The Doppler blood flow detection module integrates a transceiver circuit and a microstrip antenna. The ultrasonic imaging uses the fundamental frequency (such as 10 MHz), and the Doppler blood flow detection uses the sideband frequency (such as ±1 MHz offset). The signals are separated by a filter. In this embodiment, the ultrasonic transducer array is used to transmit / receive ultrasonic waves to generate a B-mode image and a Doppler blood flow signal, and the Doppler blood flow detection module works in cooperation with the ultrasonic transducer through frequency division multiplexing.

[0056] In some embodiments, the temperature sensor array 305 is located in the outermost layer of the three-layer structure of the overall three-layer flexible package of the multi-modal sensing probe, directly contacting the tissue, and is distributed in a ring shape. Micro temperature sensors (such as MEMS thermocouples) are evenly arranged along the circumference of the multi-modal sensing probe. There are 4-8 micro temperature sensors in each ring, and the axial spacing is set to 2-3 mm. A 50-μm thick thermal conductive silica gel is covered on the outer layer of the temperature sensor array to ensure the heat transfer efficiency. It should be noted that a 100-μm aerogel thermal insulation layer is provided between the outer layer and the middle layer to block the heat generated by the internal circuit. In this embodiment, the temperature sensor array is used to monitor the surface temperature distribution of the multi-modal sensing probe in real time.

[0057] In some embodiments, the impedance measurement unit 306 is in a ring array, with 4-8 pairs of electrodes arranged circumferentially on the multi-modal sensing probe, with a spacing of 2-5 mm to achieve 360° contact detection. The electrode material uses biocompatible materials, such as gold-plated / platinum electrodes with low polarization or Ag / AgCl electrodes with good low-frequency stability. The surface is a nanoporous coating to increase the effective area and reduce the contact impedance. In this way, when detecting the contact impedance characteristics between the detection probe and the esophageal wall, the impedance measurement unit can meet the requirements of biocompatibility, safety, high precision and miniaturization. In this embodiment, the impedance measurement unit is used to detect the contact impedance characteristics between the probe and the esophageal wall.

[0058] In some embodiments, the signal processing module 4000 includes the signal processing chip. The signal processing module uses the lock-in amplification technique to extract the quadrature I / Q components in the Doppler signal, suppresses motion artifacts and electromagnetic interference through the wavelet transform algorithm, and synchronizes and aligns the ultrasonic image frames and multi-modal sensor data based on the time stamp. In the embodiments of the present application, during the process of synchronizing and aligning the ultrasonic image frames and multi-modal sensor data based on the time stamp, a high-precision timer is used at the software level to attach a unique time stamp label to each ultrasonic image frame and sensor data point. For high-frequency data, downsampling is performed according to the ultrasonic frame rate for alignment, and for low-frequency data, upsampling is performed to match the ultrasonic time axis. For example, the 1000-Hz contact force is downsampled according to the 30-Hz ultrasonic frame rate, and for the 1-Hz data, it is upsampled to 30 Hz for interpolation. The multi-modal data is transmitted through the ROS protocol to ensure that the time stamp is bound to the data packet. During the synchronization process, the DTW algorithm is used to match the time series characteristics of the sensor data and the ultrasonic image, align the probe position and the contact force sensor, and achieve precise navigation. The synchronization and alignment of the ultrasonic image frames and multi-modal sensor data based on the time stamp in the embodiments of the present application can achieve time synchronization with millimeter-level accuracy.

[0059] In some embodiments, the AI robot control module 3000 controls the insertion depth, deflection angle, and contact pressure of the probe through a manipulator. The AI robot control module includes a tactile feedback unit, which dynamically adjusts the motion state of the multi-modal sensing probe according to the magnitude and distribution of the contact force of the contact force sensor. Specifically, the magnitudes of the contact forces collected by the first contact force sensor 302-1, the second contact force sensor 302-2, and the third contact force sensor 302-3 are F1, F2, and F3 respectively. The contact force array data is collected at 200 Hz, and the contact force data is mapped into a two-dimensional pressure heat map, and the following parameters are calculated: average contact force F avg , pressure gradient ▽F, average contact force F avg represents the global pressure mean value, F avg = (F1 + F2 + F3) / 3, and the target range is 0.3 - 0.8 N. The pressure gradient ▽F represents the contact force change rate, and ▽F includes the change rates of F1, F2, and F3 collected by the first contact force sensor 302-1, the second contact force sensor 302-2, and the third contact force sensor 302-3 respectively. In the embodiments of the present application, the contact force change rate adopts the contact force change value per unit displacement. In some embodiments, the threshold value of ▽F, ▽Flim ≤ 0.5 N / mm, is used to prevent the probe from moving too fast and causing danger due to excessive displacement. In the embodiments of the present application, the tactile feedback unit executes a tactile feedback control logic, and the tactile feedback control logic includes one or more of a global pressure control logic, an absolute pressure control logic, and a comparative pressure control logic. In some embodiments, the global pressure control logic includes: during the movement of the multi-modal sensing probe, when the pressure gradient ▽F is within the threshold range, when the average contact force F avg < 0.3 N, the multi-modal sensing probe accelerates; when 0.3 N < F avg < 0.8 N, the multi-modal sensing probe maintains a constant speed; when F avg > 1.2 N, an emergency brake is triggered and the multi-modal sensing probe retreats. Specifically, the global pressure control logic includes: during the movement of the multi-modal sensing probe, when ▽F is within the threshold range, when F avg < 0.3 N, it accelerates and advances to increase the speed to 3 - 5 mm / s; when 0.3 N < F avg < 0.8 N, it maintains a constant speed of 1 - 2 mm / s to ensure stable image acquisition; when F avg > 1.2 N, an emergency brake is triggered to retreat the tube 2 - 3 mm to prevent the risk of perforation.

[0060] In some embodiments, the absolute pressure control logic includes: during the movement of the multimodal sensing probe, when the contact force F of any one of the contact force sensors is greater than 1.5 N, an emergency brake is triggered and the multimodal sensing probe retreats; when the pressure gradient ▽F > 0.5 N / mm, the multimodal sensing probe decelerates, and the angle of the multimodal sensing probe is finely adjusted to relieve local stress concentration. Specifically, the absolute pressure control logic includes: during the movement of the multimodal sensing probe, when any one of F1, F2, and F3 is greater than 1.5 N, an emergency brake is triggered to retract the tube by 1 - 2 mm to prevent the risk of perforation; when the ▽F corresponding to F1, F2, and F3 respectively collected by the first contact force sensor 302-1, the second contact force sensor 302-2, and the third contact force sensor 302-3 is greater than 0.5 N / mm, the speed is reduced to 0.5 mm / s, and the angle of the multimodal sensing probe is finely adjusted to relieve local stress concentration.

[0061] In some embodiments, the contact force sensor includes at least a first contact force sensor and a second contact force sensor. The first contact force sensor is disposed on the contact surface of the probe, and the second contact force sensor is disposed on the other side opposite to the contact surface of the probe.

[0062] The comparative pressure control logic includes: during the movement of the multimodal sensing probe, if ultrasonic imaging is required, the magnitudes of the contact force F1 collected by the first contact force sensor and the contact force F2 collected by the second contact force sensor are judged. If F1 < F2, the angle of the multimodal sensing probe is adjusted to deflect the multimodal sensing probe in the direction of the contact surface of the probe. Specifically, the comparative pressure control logic includes: during the movement of the multimodal sensing probe, if ultrasonic imaging is required, the magnitudes of F1 and F2 are judged. If F1 < F2, the angle of the multimodal sensing probe is adjusted to deflect the end structure of the multimodal sensing probe in the direction of the probe contact surface 3018 until F1 > F2, and ultrasonic imaging is performed while ensuring F1 > F2.

[0063] In some embodiments, one or more of the global pressure control logic, the absolute pressure control logic, and the comparative pressure control logic can be adopted as needed, or all of them can be adopted. In the embodiments of the present application, through the global pressure control logic, the absolute pressure control logic, and the comparative pressure control logic, the movement of the multimodal sensing probe can be ensured to be stable and safe, and the imaging quality can be improved.

[0064] In some embodiments, the AI robot control module 3000 further includes an adaptive path planning algorithm configured to construct a three-dimensional esophageal model by combining preoperative CT or MRI images, perform dynamic trajectory optimization based on reinforcement learning, and avoid narrow areas in real time. Specifically, first, a three-dimensional model is constructed. Preoperatively, high-resolution (0.5 - 1 mm slice thickness) esophageal images covering the entire anatomical structure from the pharynx to the gastroesophageal junction are obtained through CT or MRI. The two-dimensional image data is converted into a three-dimensional model using the medical image processing software 3D Slicer, and key areas such as stenosis, diverticulum, or tumor are manually corrected to ensure that the model accuracy error is ≤ 0.3 mm. A mesh model is generated based on the segmentation results, and parameters such as the diameter and bending angle of the narrow area are marked. Areas with a diameter < 8 mm and a curvature radius less than 20 mm are marked as high-risk areas. At the same time, based on the three-dimensional model, an initial path is generated using a path planning algorithm to avoid known narrow or high-risk areas. Then, real-time dynamic adjustment is performed. Real-time data such as contact force, ultrasonic images, optical images, and impedance measurements are integrated to monitor the dynamic changes of the esophagus, such as peristalsis and spasm. Through simultaneous localization and mapping technology, the three-dimensional model is updated in real time to make up for the static limitations of preoperative images. When an unforeseen obstacle, such as a temporary stenosis, is detected, a local path optimization algorithm is triggered to quickly generate a detour path. Then, a reinforcement learning algorithm is designed. The state space and action space of reinforcement learning are designed. The features of the state space include geometric features, mechanical parameters, and image features. The geometric features include the current position of the probe, the distance to the nearest narrow area, and the path curvature. The mechanical parameters include the mean / gradient of the contact force, impedance value, and temperature distribution entropy. The image features include ultrasonic image texture and optical image texture. The features of the action space include the advancement speed (0 - 5 mm / s), yaw angle (±15°), pitch angle (±10°), emergency tube withdrawal (2 mm), start flushing, and switch imaging mode. A reward function also needs to be designed. The reward function R is as follows:

[0065] R = -α|F avg - 0.5| + β·D safe - γ·T step + δ·I SNR (1)

[0066] Where D safe represents the minimum Euclidean distance between the probe and the narrow area, F avg represents the average contact force, T step represents the time, I SNRIt represents the signal-to-noise ratio of the ultrasonic image and the visible light image, where α = 0.3, β = 0.4, γ = 0.2, and δ = 0.1. The network architecture selects the Actor network, which includes 3 layers of LSTM and 2 layers of fully connected layers, and outputs the action probability distribution. The Actor network converts the action probability distribution into an action instruction, and then the AI robot control module converts the action instruction into a motor propulsion / steering control signal. The adaptive path planning algorithm also includes a safety protection mechanism and a risk warning mechanism. The safety protection mechanism triggers an emergency stop when the impedance value exceeds the threshold or the contact force is greater than 5N. The risk warning unit avoids the vascular plexus and nerve bundle regions based on the prediction results of the machine learning model.

[0067] In some embodiments, the AI engine module 5000 is configured with a pre-trained deep learning model based on multi-modal data fusion for real-time generation of esophageal wall structure, hemodynamics, lesion feature extraction, and examination path optimization. Specifically, the AI engine module includes a multi-modal data fusion network, a lesion classification model, and a real-time visualization interface. The multi-modal data fusion network fuses temperature data, ultrasonic images, optical images, mechanical data, blood flow velocity, and impedance spectrum features based on a multi-task Transformer architecture; the lesion detection head of the lesion classification model outputs the lesion location, size, and type through Faster R-CNN, and can identify esophagitis, Barrett esophagus, and early cancer through transfer learning; the real-time visualization interface outputs a pseudo-color blood flow overlay image and a risk probability heat map.

[0068] In some embodiments, the AI engine module performs real-time lesion feature extraction and examination path optimization, specifically including the following.

[0069] Synchronously collect data and preprocess it. Real-time synchronously collect ultrasonic image data, visible light images (i.e., optical image data), contact force data, impedance data, temperature data, blood flow signals, etc., and then preprocess the collected data. For temperature data and blood flow signals, extract the peak blood flow velocity through spectral analysis. For contact force data and impedance data, generate pressure and conductivity distribution heat maps respectively, and input them after normalization.

[0070] Deep learning model design and training process. Input layer: Ultrasound image data and optical image data are input into a 3D CNN encoder (ResNet-18 3D), contact force data and impedance data are input into a GNN encoder, and temperature data and blood flow signals are input into an LSTM encoder. Fusion layer: Cross-modal attention module. Output layer: The structure segmentation head outputs the esophageal wall stratification mask (mucosa / muscular layer / adventitia) for the U-Net decoder, the hemodynamic head outputs the peak velocity (PSV) and resistance index RI through a fully connected layer, and the lesion detection head outputs the lesion location, size, and type (early cancer / ulcer / varices) through Faster R-CNN. In the embodiments of the present application, the AI engine module includes a pathological classification model that can identify esophagitis, Barrett's esophagus, and early canceration through transfer learning. The path optimization head outputs the propulsion direction / speed through the reinforcement learning policy network PPO.

[0071] The pre-training tasks include self-supervised learning and multi-modal alignment pre-training. Self-supervised learning uses unlabeled ultrasound data to extract general features through contrastive learning. Multi-modal alignment pre-training trains a cross-modal association model on the publicly available dataset MICCAI 2023 multi-modal esophageal dataset. In the embodiments of the present application, the pre-trained deep learning model is obtained through the above design and training.

[0072] In the embodiments of the present application, the AI engine module is configured with a pre-trained deep learning model, and uses a multi-modal data fusion network based on the multi-task Transformer architecture to perform real-time hemodynamic analysis, extract the peak blood flow velocity, and fuse temperature data, ultrasound image data, optical image data, contact force data, blood flow signals, and impedance data. At the same time, it can generate the esophageal wall structure in real time, and perform lesion feature extraction and dynamic examination path optimization.

[0073] In the embodiments of the present application, real-time lesion feature extraction and dynamic examination path optimization include the following content.

[0074] Lesion feature extraction includes extracting lesion morphological features and functional features. In the embodiments of the present application, the pathological classification model can identify esophagitis, Barrett's esophagus, and early canceration through transfer learning. Specifically, the mucosal layer thickness is calculated from the ultrasound segmentation result. The normal mucosal layer thickness is 0.5 - 1.2 mm. If it thickens, it indicates a lesion, which may be esophagitis. For the irregularity of the lesion shape, if the fractal dimension is greater than 1.6, it indicates malignancy and may be a tumor. If the blood flow velocity PSV is greater than 80 cm / s, it indicates tumor neovascularization and may be early canceration. An impedance drop rate Δσ > 40% indicates an increase in extracellular fluid and may be inflammation or canceration, including Barrett's esophagus.

[0075] The process of dynamic inspection path optimization is as follows: The first step: Define the state space and action space. The state space includes the probe position, real-time lesion distribution, contact force gradient, and historical path. The action space includes a speed of 0 - 5 mm / s, a deflection angle of ±15°, and a rotation direction of 0 - 360°. The second step: Encode the state space and normalize the action space. In addition, a reward function for path dynamic optimization needs to be designed. The reward function is: R = 0.5 × number of newly covered lesions - 0.3 × number of times of contact force exceeding the limit + 0.2 × path smoothness. The third step: Real-time collect data such as ultrasonic images, contact force, and pose, update the state vector, locate the lesions through a target detection network such as YOLOv7, calculate the malignancy probability, and obtain a risk probability heat map. Input the state vector into a reinforcement learning policy network such as the PPO Actor network to output the action probability distribution. Sample the action and decode it into actual speed, angle, and rotation control instructions. In this process, the AI engine module updates the path data every 1 s, preferentially covering high-confidence lesion areas. In case of a narrow area, combine the actual motion data of the robot control module. For example, when the contact force is greater than 1.5 N, initiate local replanning to bypass.

[0076] In an embodiment of the present application, the AI engine module further includes a real-time visualization interface, which can output a pseudo-color blood flow overlay image and a risk probability heat map in real time based on a multi-modal data fusion network and a pathological classification model.

[0077] In an embodiment of the present application, in the AI robot-assisted esophageal ultrasound detection system, the main task of the AI engine module is to generate a global inspection path recommendation based on multi-modal data fusion to optimize the lesion coverage rate and detection efficiency. The main task of the robot control module is to dynamically adjust the probe trajectory based on the preoperative model and real-time feedback to ensure safe obstacle avoidance and stable contact. In terms of path planning, the AI engine generates a global path recommendation before surgery, and the robot control module pre-computes potential risk points. During the intraoperative stage, the AI engine module outputs a path adjustment recommendation every 1 second, such as "a new lesion is found, and a circular scan needs to be added at the current position". The robot control module fine-tunes the probe movement every 100 ms according to the real-time sensor data to execute safety constraints. The robot module and the AI engine module also include a cooperation mechanism. The path recommendation of the AI engine needs to pass the safety verification of the robot control module, such as the verification of contact force exceeding the limit. If the path recommendation of the AI engine module is not feasible, such as the path curvature exceeding the mechanical limit, the robot module replans autonomously. In addition, the robot control module feeds back the actual motion data to the AI engine for online model fine-tuning.

[0078] Through multi-modal fusion network design, multi-task joint training, and reinforcement learning path optimization, the AI engine module achieves precise structure analysis with a mucosal stratification accuracy of sub-millimeter level. The AI engine realizes dynamic function assessment, enabling multi-parameter joint diagnosis of blood flow, impedance, temperature, ultrasound, visible light images, and contact force. The intelligent path planning of the AI engine maximizes the lesion detection efficiency under the premise of safety. This system upgrades esophageal ultrasound detection from experience-dependence to data-driving, significantly improving the early cancer detection rate and operation safety.

[0079] In some embodiments, the AI robot-assisted esophageal ultrasound detection system further includes a human-machine interaction control interface, which realizes the overlay display of real-time ultrasound images and virtual navigation markers, provides tactile feedback by simulating the tissue resistance characteristics during the probe movement process, and automatically terminates the operation when abnormal movement or sudden change in physiological parameters is detected.

[0080] In some embodiments, the AI robot-assisted esophageal ultrasound detection system supports a wireless transmission mode, including: a low-power Bluetooth module for transmitting compressed multi-modal data streams; and a cloud AI coprocessor for performing high-complexity calculations and returning diagnostic instructions.

[0081] Figure 9 The following shows a flowchart of the AI robot-assisted esophageal ultrasound detection method in some embodiments of the present application. Refer to Figure 9 In an embodiment of the present application, an AI robot-assisted esophageal ultrasound detection method is further provided. The method is based on the AI robot-assisted esophageal ultrasound detection system in any of the above embodiments and includes: S1, constructing a patient-specific esophageal path atlas through preoperative imaging data; S2, automatically inserting a multi-modal sensing probe by combining real-time force feedback and AI visual guidance; S3, using multi-modal data fusion technology to achieve dynamic tracking of the esophageal wall structure; S4, automatically generating a structured inspection report according to the abnormal detection results.

[0082] It can be understood that all functions of the above AI robot-assisted esophageal ultrasound detection system can be applied to the AI robot-assisted esophageal ultrasound detection method of the present application.

[0083] In summary, an AI robot-assisted esophageal ultrasound detection system and method proposed by the present invention uses a multi-modal sensing probe with a special structure, makes full use of the limited space at the end of the probe structure, integrates optical imaging and ultrasound imaging, and realizes precise tactile feedback control logic through an innovative design and distribution of contact force sensors. Through the multi-modal sensing probe, intelligent robot control, and multi-source data fusion AI algorithm, precise evaluation of the esophageal structure and function is achieved. The system integrates functions of ultrasound imaging, contact mechanics, impedance measurement, and blood flow detection, combines adaptive path planning and automatic lesion classification technology, and significantly improves the accuracy, safety, and efficiency of the diagnosis of digestive tract diseases.

[0084] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0085] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0086] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner described in the specification, such that the instructions stored in the computer-readable memory produce a manufactured article including an instruction device that implements the functions specified in one process Figure 1 one process or multiple processes and / or boxes Figure 1 or multiple boxes specified in the box.

[0087] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner described in the specification, such that the instructions stored in the computer-readable memory produce a manufactured article including an instruction device that implements the functions specified in one process Figure 1 one process or multiple processes and / or boxes Figure 1 or multiple boxes specified in the box.

[0088] It should be noted that the technical features in the above embodiments can be combined arbitrarily, and the combined technical solutions all fall within the protection scope of this application. And in this text, terms such as "including", "comprising" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or device. Without more limitations, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or device including the said element.

[0089] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An AI robot-assisted esophageal ultrasound detection system, characterized in that, The system includes: a manipulative robotic arm equipped with an end effector and a multi-degree-of-freedom motion mechanism; A multi-modal sensing probe connected to the end effector, including a temperature sensor array, an ultrasonic transducer, an optical imaging module, a contact force sensor, an impedance measurement unit, and a Doppler blood flow detection module; An AI robot control module configured to control the insertion depth, deflection angle, and contact pressure of the multi-modal sensing probe through the manipulative robotic arm; A signal processing module that synchronously collects and denoises temperature data, ultrasonic imaging data, optical image data, contact force data, impedance data, and blood flow signals; An AI engine module, based on multi-modal data fusion, configured with a pre-trained deep learning model to generate esophageal wall structure, hemodynamics, and lesion feature extraction, as well as inspection path optimization in real time.

2. The AI robot-assisted esophageal ultrasound detection system according to claim 1, wherein The structure of the multi-modal sensing probe is a three-layer flexible package: the outer layer is a biocompatible silicone sleeve with multiple scale marks on the surface; the middle layer is a flexible circuit board integrating a sensor signal conditioning circuit and a micro heat pipe heat dissipation structure; the inner layer is a coaxial cable and a guide wire; The multi-modal sensing probe has an end structure, and the end structure includes an ultrasonic transducer and a zoom lens. The ultrasonic transducer is arranged on one side of the outer periphery of the zoom lens, and the optical path of the multi-modal sensing probe is perpendicular to the acoustic path.

3. An AI robot-assisted esophageal ultrasound detection system according to claim 1, characterized in that, The temperature sensor array is used to monitor the surface temperature distribution of the multi-modal sensing probe in real time; the impedance measurement unit is used to detect the contact impedance characteristics between the multi-modal sensing probe and the esophageal wall; the contact force sensor is used to ensure stable contact between the multi-modal sensing probe and the tissue; the Doppler blood flow detection module works in cooperation with the ultrasonic transducer through frequency division multiplexing.

4. An AI robot-assisted esophageal ultrasound detection system according to claim 1, characterized in that, The AI robot control module includes: A tactile feedback unit that dynamically adjusts the motion state of the multi-modal sensing probe according to the magnitude and distribution of the contact force of the contact force sensor; Dynamically adjusting the motion state of the multi-modal sensing probe according to the magnitude and distribution of the contact force of the contact force sensor includes: executing a tactile feedback control logic, and the tactile feedback control logic includes one or more of a global pressure control logic, an absolute pressure control logic, and a comparative pressure control logic.

5. An AI robot-assisted esophageal ultrasound detection system according to claim 1, characterized in that, The AI engine module includes: A multi-modal data fusion network that fuses temperature data, ultrasonic imaging data, optical image data, contact force data, blood flow signals, and impedance data based on the Transformer architecture; A pathological classification model that identifies esophagitis, Barrett esophagus, and early canceration through transfer learning; A real-time visualization interface that outputs a pseudo-color blood flow overlay image and a risk probability heat map.

6. An AI robot-assisted esophageal ultrasound detection system according to claim 5, wherein, The AI engine module updates the path data every 1s. Combining the actual motion data of the robot control module, when the contact force is greater than 1.5N, local replanning and bypassing are started.

7. An AI robot-assisted esophageal ultrasound detection system according to claim 1, characterized in that, The signal processing module adopts: a lock-in amplification technique to extract the quadrature I / Q components in the Doppler signal; a wavelet transform algorithm to suppress motion artifacts and electromagnetic interference; a timestamp synchronization mechanism to align ultrasonic image frames with multi-modal sensor data.

8. An AI robot-assisted esophageal ultrasound detection system according to claim 4, characterized in that The global pressure control logic includes: during the movement of the multimodal sensing probe, when the pressure gradient ▽F is within the threshold range, when the average contact force F avg < 0.3 N, the multimodal sensing probe accelerates; when 0.3 N < F avg < 0.8 N, the multimodal sensing probe maintains a constant speed; when F avg > 1.2 N, an emergency brake is triggered and the multimodal sensing probe retreats; The absolute pressure control logic includes: during the movement of the multi-modal sensing probe, when the contact force F of any contact force sensor is greater than 1.5 N, an emergency brake is triggered and the multi-modal sensing probe retreats; when the pressure gradient ▽F > 0.5 N / mm, the multi-modal sensing probe decelerates and the angle of the multi-modal sensing probe is finely adjusted to relieve local stress concentration.

9. The AI robot-assisted esophageal ultrasound detection system according to claim 4, wherein The contact force sensor includes at least a first contact force sensor and a second contact force sensor. The first contact force sensor is arranged on the probe contact surface, and the second contact force sensor is arranged on the other side opposite to the probe contact surface; The comparative pressure control logic includes: during the movement of the multi-modal sensing probe, if ultrasonic imaging is required, the magnitudes of the contact force F1 collected by the first contact force sensor and the contact force F2 collected by the second contact force sensor are judged. If F1 < F2, the angle of the multi-modal sensing probe is adjusted to deflect the multi-modal sensing probe in the direction of the probe contact surface.

10. A transesophageal echocardiography method based on the system according to any one of claims 1-9, characterized in that, The method includes: constructing a patient-specific esophageal path map through preoperative imaging data; completing the automatic placement of the multi-modal sensing probe by combining real-time force feedback and AI vision guidance; using multi-modal data fusion technology to achieve dynamic tracking of the esophageal wall structure; automatically generating a structured inspection report according to the abnormal detection results.

Images