An ultrasonic endoscope probe, an ultrasonic endoscope system, and a method of operation thereof

Abstract

This invention relates to an ultrasound endoscope probe, an ultrasound endoscope system, and its working method. The ultrasound endoscope probe includes multiple convex array transducers arranged in an array on a mounting plate. The mounting plate has a retracted position and an extended position. When the mounting plate is in the retracted position, the connecting surface is abutted against the mating inclined surface. When the mounting plate is in the extended position, the working surface is perpendicular to the central axis of the endoscope catheter. The ultrasound endoscope system includes a signal synchronization module, a data acquisition module, a data processing module, and a display module. By constructing a mapping relationship between the probe coordinate system and the BEV global coordinate system, a three-dimensional voxel mesh is generated to segment blood vessels and tumor tissue, ultimately achieving three-dimensional reconstruction. This invention overcomes the shortcomings of traditional ultrasound imaging, such as low vascular discrimination and large three-dimensional reconstruction errors, effectively improving imaging and segmentation accuracy, clearly displaying the relationship between lesions and adjacent blood vessels, shortening operation time, improving the safety of minimally invasive operations, reducing the burden of diagnosis and treatment for patients, and demonstrating significant clinical application value.

Description

Technical Field

[0001] This invention relates to the field of medical imaging equipment technology, specifically to an ultrasonic endoscope probe, an ultrasonic endoscope system, and its working method. Background Technology

[0002] Ultrasonic endoscopy is a minimally invasive medical device that integrates ultrasound detection and endoscopic examination. Its ultrasound probe is placed at the tip of the endoscope. After the device is inserted into the body cavity, it can directly observe lesions inside the cavity through the endoscope, while acquiring two-dimensional ultrasound images of tissue cross sections through the ultrasound probe. Based on the sequence of images, it can complete the three-dimensional reconstruction of the hierarchical structure of internal organs. It is the core equipment for intraoperative navigation in minimally invasive abdominal surgeries such as hepatobiliary surgery, and provides key anatomical structural references for clinical surgical decisions.

[0003] However, existing ultrasound endoscopes have many insurmountable limitations in clinical applications. Firstly, the accuracy of 3D reconstruction is insufficient and highly operator-dependent. Sequential image acquisition relies entirely on manual probe manipulation, leading to inconsistent spatial scanning paths and resulting in voids and errors in the reconstructed 3D anatomical model, failing to meet the high-precision requirements of surgical navigation. Furthermore, imaging results are highly dependent on the operator's clinical experience, with poor consistency between different operators and severely limited clinical repeatability. Secondly, the accuracy of blood vessel and tumor identification is low. Ultrasound imaging is easily affected by factors such as organ peristalsis, respiratory movements, and uneven tissue echoes, resulting in complex background noise. Traditional ultrasound segmentation algorithms have limited ability to identify blood vessel edges under complex noise, leading to generally low segmentation accuracy for vascular structures and large errors in tumor contour reconstruction, failing to provide precise boundary guidance for surgery. Thirdly, real-time performance is insufficient. Image processing delays for 3D model reconstruction and ultrasound image segmentation typically exceed 100ms, causing a spatiotemporal misalignment between the navigation image and the actual anatomical structure, failing to meet the dynamic navigation requirements of surgery and affecting the timeliness of surgical decisions.

[0004] In summary, due to the limitations of existing ultrasound endoscopy, surgeons are prone to accidentally rupture critical blood vessels such as the hepatic artery and cystic artery during minimally invasive hepatobiliary surgery, which may force conversion to open surgery. This is a common clinical problem faced by hepatobiliary surgeons when performing minimally invasive surgery. Summary of the Invention

[0005] In view of the deficiencies of the prior art described above, the technical problem to be solved by the present invention is to provide an ultrasonic endoscope probe, an ultrasonic endoscope system and its working method. Through the digital fusion development of the ultrasonic endoscope probe, the position of the artery can be displayed simultaneously, guiding the surgical operation and effectively avoiding accidental damage to the artery during the operation.

[0006] To achieve the above objectives, the present invention provides an ultrasonic endoscope probe, comprising:

[0007] An endoscopic catheter, wherein the distal end face of the endoscopic catheter is an inclined mating bevel;

[0008] An extension structure includes a handle, a slider switch, an axial push rod, a radial linkage rod, and a mounting plate. The handle has an inner cavity, and the proximal end of the endoscope catheter is fixed to the distal end of the handle. A through groove extending axially is formed on the outer peripheral surface of the handle, and the slider switch is slidably engaged in the through groove. The axial push rod is disposed in the inner cavity of the endoscope catheter and the handle. One end of the radial linkage rod is hinged to the axial push rod, and the other end is fixed to the slider switch. The mounting plate has a connecting surface and a working surface arranged opposite to each other. The distal end of the axial push rod is hinged to the connecting surface, and the connecting surface is hinged to the tip of the mating inclined surface.

[0009] A linear array transducer, wherein the linear array transducer is disposed on the working surface;

[0010] Multiple convex array transducers are arranged in an array on the working surface;

[0011] The mounting plate has a retracted position and an extended position. When the mounting plate is in the retracted position, the connecting surface is in contact with the mating inclined surface. When the mounting plate is in the extended position, the working surface is perpendicular to the central axis of the endoscope catheter.

[0012] The cable is disposed in the inner cavity of the endoscope catheter and the handle, and is electrically connected to the linear array transducer and the plurality of convex array transducers, respectively, for transmitting ultrasonic echo data of the linear array transducer and the convex array transducer.

[0013] Furthermore, it also includes a water spray structure, which includes a water pipe and multiple water nozzles. The water pipe is located in the inner cavity of the endoscope guide tube and the handle. The multiple water nozzles are all located on the working surface and are respectively located close to multiple convex array transducers. The multiple water nozzles are respectively connected to the water pipe.

[0014] The present invention also provides an ultrasonic endoscope system, comprising:

[0015] The aforementioned ultrasonic endoscope probe is used to scan the target area and acquire ultrasonic echo data of the target area.

[0016] The signal synchronization module is used to perform spatial position calibration on the multiple convex array transducers of the ultrasonic endoscope probe and output the synchronization timing control signal of the multiple convex array transducers.

[0017] The data acquisition module is used to acquire ultrasonic echo data output by multiple convex array transducers, and to perform preprocessing on the acquired ultrasonic echo data to obtain the triaxial coordinate values ​​of each voxel in the target region.

[0018] The data processing module is used to calculate the weighting coefficients of multi-channel ultrasound echo data; perform weighted fusion on the preprocessed ultrasound echo data to generate a three-dimensional voxel mesh; perform feature extraction and feature fusion on the three-dimensional voxel mesh to complete the segmentation of blood vessels and tumor tissue in the target area; and perform three-dimensional reconstruction on the segmentation results to generate a three-dimensional reconstruction model.

[0019] The display module is used to display the three-dimensional reconstructed model.

[0020] Furthermore, the preprocessing performed by the data acquisition module includes: performing noise reduction processing on the ultrasonic echo data using a wavelet denoising algorithm, and compressing the dynamic range of the ultrasonic echo data to 30dB-300dB.

[0021] Furthermore, the data processing module calculates the weighting coefficients of the multi-channel ultrasonic echo data, specifically by dynamically calculating the weighting coefficients w of the multi-channel ultrasonic echo data based on the spatial positions and scanning angles of the multiple convex array transducers. i And the weighting coefficient w i Normalization is performed to ensure that the sum of the weighting coefficients of the voxels corresponding to all convex array transducers is 1, wherein the weighting coefficient w i The calculation formula is as follows:

[0022] (1)

[0023] In the formula, d i θ represents the perpendicular distance from the voxel to the central axis of the i-th convex transducer; i Let be the cosine of the angle between the scanning angle of the i-th convex array transducer and the main scanning direction.

[0024] Furthermore, the data processing module performs weighted fusion of multiple ultrasonic echo data to generate a three-dimensional voxel mesh. Specifically, it fuses each voxel in the target region using a weighted average formula, maps the preprocessed ultrasonic echo data to the BEV global coordinate system, and generates a three-dimensional voxel mesh. The weighted average formula is as follows:

[0025] (2)

[0026] In the formula, N is the total number of convex array transducers; I i (x|y|z) represents the three-axis coordinates of the voxel acquired by the i-th convex array transducer; I fused (x|y|z) represents the three-axis coordinates of the corresponding voxel in the three-dimensional voxel mesh in the BEV global coordinate system.

[0027] Furthermore, the data processing module performs feature extraction and feature fusion on the three-dimensional voxel mesh to complete the segmentation of blood vessels and tumor tissue in the target region. Specifically, based on the improved U-Net++ network, dilated convolutions with different dilation rates are introduced into different depth layers of the three-dimensional voxel mesh to extract multi-scale features of the blood vessel wall and tumor capsule; at the encoder end of the U-Net++ network, the global features are analyzed based on the EfficientNet-B0 classifier to output a blood vessel density map and achieve blood vessel differentiation; based on fractal morphological analysis methods, tumor contour features are extracted to complete the segmentation of blood vessels and tumor tissue in the target region.

[0028] Furthermore, the data processing module performs three-dimensional reconstruction on the segmentation results to generate a three-dimensional reconstruction model. Specifically, it uses a diffusion model to perform three-dimensional reconstruction on the segmentation results, including: generating an initial three-dimensional volume based on the segmentation results of blood vessels and tumor tissue in the target region; gradually adding noise to the initial three-dimensional volume to generate a diffusion path; recovering the three-dimensional structural model from the noise volume through a reverse diffusion process; quantifying the reconstruction confidence of each voxel in the three-dimensional structural model; retaining only voxels with a confidence > 0.9; and generating a three-dimensional reconstruction model.

[0029] This invention also discloses a method for operating an ultrasonic endoscope system, wherein the method is implemented using the ultrasonic endoscope system described above, and includes the following steps:

[0030] S1. Position calibration and timing synchronization: The spatial position of multiple convex array transducers is calibrated by the signal synchronization module to eliminate the spatial deviation between the multiple convex array transducers. At the same time, the synchronization timing control signal of multiple convex array transducers is output to ensure that the timing error of ultrasonic transmission and echo reception of each convex array transducer is ≤1μs.

[0031] S2. Data Acquisition and Preprocessing: The ultrasonic echo data output by multiple convex array transducers is acquired synchronously through the data acquisition module, and the sampling rate is controlled to be ≥300MHz. The ultrasonic echo data is denoised by wavelet denoising algorithm, and the dynamic range of the ultrasonic echo data is compressed to 30dB-300dB.

[0032] S3. Calculate the weighting coefficient: Using the data processing module, for each voxel in the target region, calculate the vertical distance d from that voxel to the central axis of the corresponding convex array transducer. i Calculate the cosine value θ of the angle between the scanning angle of the convex array transducer and the main scanning direction. i The weighting coefficient wi of the multi-channel ultrasonic echo data is dynamically calculated by formula (1), and the weighting coefficient wi is normalized to ensure that the sum of the weighting coefficients of the voxels corresponding to all convex array transducers is 1.

[0033] S4. Generate a three-dimensional voxel mesh: Through the data processing module, according to formula (2), for each voxel in the target area, the weighted average formula is used to fuse them, and the preprocessed ultrasonic echo data is mapped to the BEV global coordinate system to generate a three-dimensional voxel mesh with a unified spatial dimension.

[0034] S5. Feature Extraction and Target Segmentation:

[0035] S5.1 Input a three-dimensional voxel mesh. Through the data processing module, based on the improved U-Net++ network, introduce dilated convolutions with different dilation rates into different depth network layers of the three-dimensional voxel mesh to extract multi-scale features of blood vessel walls and tumor capsules.

[0036] S5.2 Introduce a gating mechanism in the skip connections of the U-Net++ network. Through the gating skip connections, multi-scale features are fused to the decoder, while sharing the underlying feature extraction network to simultaneously predict the blood vessel segmentation mask and tumor contour.

[0037] S5.3 Extract global features at the encoder end of the U-Net++ network and input them into the EfficientNet-B0 classifier. Analyze the global features through the EfficientNet-B0 classifier and output a blood vessel density map.

[0038] S5.4 Based on the blood vessel density map, an adaptive threshold segmentation algorithm is used to extract the blood vessel region, and a fractal morphological analysis method is used to extract the tumor contour features to generate the segmentation results of blood vessels and tumor tissue in the target region. The blood vessel segmentation mask, tumor contour features and blood vessel density map are output simultaneously.

[0039] S6. 3D Model Reconstruction:

[0040] S6.1. Through the data processing module, the segmentation results of blood vessels and tumor tissues are input into the diffusion model to generate an initial three-dimensional volume as the starting point for reconstruction.

[0041] S6.2. Gradually add noise to the initial three-dimensional volume to generate diffusion paths;

[0042] S6.3. The three-dimensional structural model is recovered from the noise volume through the reverse diffusion process;

[0043] S6.4 During the posterior sampling process of the diffusion model, the reconstruction confidence of each voxel in the three-dimensional structural model is quantified, and only voxels with a confidence of >0.9 are retained to generate a three-dimensional reconstruction model.

[0044] S7. Real-time navigation display: The three-dimensional reconstruction model is displayed in real time through the display module, realizing real-time intraoperative navigation of ultrasound endoscopy.

[0045] As described above, the ultrasonic endoscope probe, ultrasonic endoscope system, and working method of the present invention have the following beneficial effects:

[0046] 1. The ultrasound endoscope probe, through its extension structure, can significantly reduce the resistance when the ultrasound endoscope probe enters the body cavity, reducing cavity stimulation; through the combined use of linear array transducers and convex array transducers, it can achieve the complementary advantages of flat close-range imaging and fan-shaped wide-angle imaging. The linear array transducer is responsible for high-precision imaging of the superficial mucosa and small blood vessels, while the convex array transducer is responsible for large-field fan-shaped scanning of deep tissues and the overall infiltration range of tumors, adapting to different morphological cavities in the human body and significantly improving the comprehensiveness of detection;

[0047] 2. The endoscopic ultrasound system and its working method, through systematic improvements in core technologies such as multi-convex array transducer collaborative imaging, BEV global spatial registration, dynamic weighted data fusion, multi-scale intelligent segmentation, and high-confidence three-dimensional reconstruction, can reduce processing latency to less than 50ms and increase the frame rate to more than 25fps, meeting the real-time requirements of surgical navigation. It can provide accurate and reliable three-dimensional visualization models for real-time navigation of minimally invasive hepatobiliary surgery, and present the three-dimensional anatomical structures that cannot be intuitively presented by traditional two-dimensional ultrasound sections to the doctor in a fully visualized manner, providing effective anatomical decision-making basis. Attached Figure Description

[0048] Figure 1 This is a schematic diagram of the structure of the ultrasonic endoscope probe in this invention.

[0049] Figure 2 for Figure 1 Enlarged view of point A in the middle.

[0050] Figure 3 for Figure 1 Enlarged view of point B in the middle.

[0051] Figure 4 A schematic diagram of the working surface of the mounting plate in this invention.

[0052] Figure 5 This is a schematic diagram of the ultrasonic endoscope system in this invention.

[0053] Explanation of icon numbers

[0054] 1. Endoscope catheter; 201. Handle; 202. Slider switch; 203. Axial push rod; 204. Radial linkage rod; 205. Mounting plate; 251. Connecting surface; 252. Working surface; 3. Linear array transducer; 4. Convex array transducer; 501. Water pipe; 502. Water nozzle; 6. Cable. Detailed Implementation

[0055] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. These embodiments are for illustrative purposes only and are not intended to limit the scope of the invention.

[0056] In the description of this invention, it should be noted that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0057] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0058] Furthermore, in the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0059] See Figures 1 to 4 This invention provides an ultrasonic endoscope probe (see...) Figure 1 ),include:

[0060] Endoscopic catheter 1, the distal end face of endoscopic catheter 1 is an inclined mating bevel (see...). Figure 2 );

[0061] The extension structure includes a handle 201, a slider switch 202, an axial push rod 203, a radial linkage rod 204, and a mounting plate 205. The handle 201 has an inner cavity, and the proximal end of the endoscope catheter 1 is fixed to the distal end of the handle 201. Specifically, the inner cavity of the endoscope catheter 1 communicates with the inner cavity of the handle 201 (see...). Figure 1 A through groove extending axially is provided on the outer peripheral surface of the handle 201, penetrating the side wall of the handle 201. The slider switch 202 is slidably engaged with the through groove. The axial push rod 203 is disposed in the inner cavity of the endoscope guide tube 1 and the handle 201. One end of the radial linkage rod 204 is hinged to the axial push rod 203, and the other end is fixed to the slider switch 202 (see...). Figure 2Mounting plate 205 has a connecting surface 251 and a working surface 252 that are arranged opposite to each other. The distal end of the axial push rod 203 is hinged to the connecting surface 251, and the connecting surface 251 is hinged to the tip of the mating inclined surface (see...). Figure 1 );

[0062] Linear array transducer 3 (see Figure 4 Specifically, the linear array transducer 3 is an existing technology. Multiple array elements of the linear array transducer 3 are arranged sequentially along a straight line at equal intervals. All array elements are on the same plane. The specific structure will not be described in detail here. The linear array transducer 3 is located on the working surface 252. The linear array transducer 3 is used in conjunction with the convex array transducer 4 to achieve the complementary advantages of flat close-range imaging and fan-shaped wide-angle imaging, effectively expanding the imaging field of view, improving local resolution, reducing detection blind spots, adapting to different morphological cavity tissues of the human body, and significantly improving the comprehensiveness of detection and imaging quality.

[0063] Multiple convex array transducers 4 (see Figure 2 , Figure 4 Specifically, the convex array transducer 4 is the prior art. The multiple array elements of the convex array transducer 4 are arranged in an arc-shaped curved surface with equal spacing in the circumferential direction and are convex as a whole. The specific structure will not be described in detail here. Multiple convex array transducers 4 are arranged in an array on the working surface 252. In this embodiment, there are three convex array transducers 4, which are arranged in sequence along a straight line with equal spacing.

[0064] Mounting plate 205 has a retracted position and an extended position (see...) Figure 2 When the mounting plate 205 is in the retracted position, the connecting surface 251 is in contact with the mating inclined surface; when the mounting plate 205 is in the extended position, the working surface 252 is perpendicular to the central axis of the endoscope tube 1.

[0065] Cable 6 is located within the lumen of the endoscope catheter 1 and the handle 201 (see...). Figure 3 Cable 6 is electrically connected to linear array transducer 3 and multiple convex array transducers 4 respectively (see...). Figure 2 ), used to transmit ultrasonic echo data from linear array transducer 3 and convex array transducer 4;

[0066] Preferably, it also includes a water spray structure, which includes a water pipe 501 and multiple water nozzles 502. The water pipe 501 is disposed in the inner cavity of the endoscope guide tube 1 and the handle 201 (see...). Figure 3 Multiple water nozzles 502 are located on the working surface 252 and are positioned close to multiple convex array transducers 4 (see [reference]). Figure 2Multiple water nozzles 502 are connected to water pipes 501. Specifically, multiple connecting holes are provided on the mounting plate 205, and the water nozzles 502 are located in the connecting holes. One end of the water nozzle 502 extends out of the working surface 252, and the other end of the water nozzle 502 is connected to the water pipe 501. By injecting sterile saline into the water pipe 501, the sterile saline can be sprayed out through the water nozzles 502, so that a continuous liquid coupling medium can be formed between the convex array transducer 4 and human tissue, thereby dispelling air, ensuring normal penetration of ultrasound signals, and effectively eliminating ultrasound attenuation and imaging artifacts caused by gas.

[0067] The basic working principle of the ultrasonic endoscope probe involved in this invention is as follows: By setting an extension structure, the handle 201 is easy for the operator to grip; the slider switch 202 can slide axially along the through groove of the handle 201; the radial linkage rod 204 can convert the movement of the slider switch 202 into the axial movement of the axial push rod 203; the mounting plate 205, together with the fixed hinge point of the mating inclined surface tip and the moving hinge point of the mounting plate 205 and the axial push rod 203, form a double-hinged structure, which can convert the axial movement of the axial push rod 203 into the rotation of the mounting plate 205 around the fixed hinge point, allowing the mounting plate 205 to switch between a retracted position and an extended position. When the mounting plate 205 is in the retracted position, it facilitates the ultrasonic endoscope probe... When the head enters the body cavity, the linear array transducer 3 and the convex array transducer 4 are in the extended position, facilitating scanning of the target area by the linear array transducer 3 and the convex array transducer 4. The endoscopic catheter 1, with its inclined surface, provides a support surface when the mounting plate 205 retracts. Simultaneously, the inclined surface significantly reduces resistance when the ultrasound endoscope probe enters the body cavity, minimizing cavity irritation. The combined use of the linear array transducer 3 and the convex array transducer 4 achieves a complementary advantage between flat, near-field imaging and fan-shaped wide-angle imaging, effectively expanding the imaging field of view, improving local resolution, reducing blind spots, adapting to different morphological cavities in the human body, and significantly improving the comprehensiveness of detection and imaging quality. Data transmission from the convex array transducer 4 is achieved via the cable 6.

[0068] See Figure 5 The present invention also provides an ultrasonic endoscope system, comprising:

[0069] The aforementioned ultrasonic endoscope probe is used to scan the target area and acquire ultrasonic echo data of the target area.

[0070] Signal synchronization module (see) Figure 5The system is used to calibrate the spatial position of multiple convex array transducers 4 of the ultrasonic endoscope probe and output synchronous timing control signals for the multiple convex array transducers 4; preferably, the spatial position of the multiple convex array transducers 4 of the ultrasonic endoscope probe is calibrated by an electromagnetic tracking system or calibration points CPs; preferably, the signal synchronization module is implemented using an FPGA (Field-Programmable Gate Array) platform.

[0071] Data acquisition module (see) Figure 5 ), used to acquire ultrasonic echo data output by multiple convex array transducers 4, and perform preprocessing on the acquired ultrasonic echo data to obtain the triaxial coordinate values ​​of each voxel in the target area;

[0072] Preferably, the preprocessing includes: denoising the ultrasonic echo data using a wavelet denoising algorithm. In other embodiments, the ultrasonic echo data can also be denoised using algorithms such as deep learning denoising, and the dynamic range of the ultrasonic echo data is compressed to 30dB-300dB. In this embodiment, a dynamic range compression algorithm is used to reduce the strong signal of the ultrasonic echo data and enhance its weak signal.

[0073] Data acquisition module (see) Figure 5 ), used to acquire ultrasonic echo data output by multiple convex array transducers 4, and perform preprocessing on the acquired ultrasonic echo data to obtain the triaxial coordinate values ​​of each voxel in the target area;

[0074] Data processing module (see Figure 5 The data processing module is used to calculate the weighting coefficients of multi-channel ultrasound echo data; to perform weighted fusion on the preprocessed ultrasound echo data to generate a three-dimensional voxel mesh; to perform feature extraction and feature fusion on the three-dimensional voxel mesh to complete the segmentation of blood vessels and tumor tissues in the target area; and to perform three-dimensional reconstruction on the segmentation results to generate a three-dimensional reconstruction model. Preferably, the data processing module is implemented using an embedded GPU platform.

[0075] Preferably, the data processing module calculates the weighting coefficients of the multi-channel ultrasonic echo data, specifically by dynamically calculating the weighting coefficients w of the multi-channel ultrasonic echo data based on the spatial position and scanning angle of the multiple convex array transducers 4. i And the weighting coefficient w i Normalization is performed to ensure that the sum of the weighting coefficients of the corresponding voxels for all convex array transducers 4 is 1, effectively suppressing noise and improving the clarity of blood vessel and tumor edges. The weighting coefficient w... i The calculation formula is as follows:

[0076] (1)

[0077] In the formula, di θ represents the perpendicular distance from the voxel to the central axis of the i-th convex transducer 4; i Let be the cosine of the angle between the scanning angle of the i-th convex array transducer 4 and the main scanning direction;

[0078] Preferably, the data processing module performs weighted fusion of multiple ultrasonic echo data to generate a three-dimensional voxel mesh. Specifically, it fuses each voxel in the target region using a weighted average formula, maps the preprocessed ultrasonic echo data to the BEV global coordinate system, and generates a three-dimensional voxel mesh. The weighted average formula is as follows:

[0079] (2)

[0080] In the formula, N is the total number of convex array transducers 4; I i (x|y|z) The three-axis coordinate values ​​of the voxel collected by the i-th convex array transducer (4); I fused (x|y|z) represents the three-axis coordinates of the corresponding voxel in the three-dimensional voxel mesh in the BEV global coordinate system.

[0081] Preferably, the data processing module performs feature extraction and feature fusion on the three-dimensional voxel mesh to complete the segmentation of blood vessels and tumor tissue in the target region. Specifically, based on the improved U-Net++ network, dilated convolutions with different dilation rates are introduced into different depth layers of the three-dimensional voxel mesh to extract multi-scale features of the blood vessel wall and tumor capsule; at the encoder end of the U-Net++ network, the global features are analyzed based on the EfficientNet-B0 classifier to output a blood vessel density map and achieve blood vessel differentiation; based on fractal morphological analysis methods, tumor contour features are extracted to complete the segmentation of blood vessels and tumor tissue in the target region.

[0082] Preferably, the data processing module performs three-dimensional reconstruction on the segmentation results to generate a three-dimensional reconstruction model. Specifically, it uses a diffusion model to perform three-dimensional reconstruction on the segmentation results, including: generating an initial three-dimensional volume based on the segmentation results of blood vessels and tumor tissue in the target region; gradually adding noise to the initial three-dimensional volume to generate a diffusion path; recovering the three-dimensional structural model from the noise volume through a reverse diffusion process; quantifying the reconstruction confidence of each voxel in the three-dimensional structural model; retaining only voxels with a confidence > 0.9; and generating a three-dimensional reconstruction model.

[0083] Display module (see Figure 5 This is used to display the three-dimensional reconstruction model and highlight the location and course of key identified blood vessels for doctors' reference.

[0084] See Figure 5The present invention also discloses a method for operating an ultrasonic endoscope system, the method being implemented using the ultrasonic endoscope system described above, and including the following steps:

[0085] S1. Position calibration and timing synchronization: The spatial position of multiple convex array transducers 4 is calibrated by the signal synchronization module to eliminate the spatial deviation between multiple convex array transducers 4. At the same time, the synchronization timing control signal of multiple convex array transducers 4 is output to ensure that the timing error of ultrasonic transmission and echo reception of each convex array transducer 4 is ≤1μs.

[0086] S2. Data Acquisition and Preprocessing: The ultrasonic echo data output by multiple convex array transducers 4 is acquired synchronously through the data acquisition module, and the sampling rate is controlled to be ≥300MHz. The ultrasonic echo data is denoised by wavelet denoising algorithm, and the dynamic range of the ultrasonic echo data is compressed to 30dB-300dB.

[0087] S3. Calculate the weighting coefficient: Using the data processing module, calculate the vertical distance d from each voxel in the target region to the central axis of the corresponding convex array transducer 4. i Calculate the cosine value θ of the angle between the scanning angle of the convex array transducer 4 and the main scanning direction. i The weighting coefficients wi of the multi-channel ultrasonic echo data are dynamically calculated using formula (1), and the weighting coefficients wi are normalized to ensure that the sum of the weighting coefficients of the voxels corresponding to all convex array transducers 4 is 1, that is, ;

[0088] S4. Generate a three-dimensional voxel mesh: Through the data processing module, according to formula (2), for each voxel in the target area, the weighted average formula is used to fuse them, and the preprocessed ultrasonic echo data is mapped to the BEV global coordinate system to generate a three-dimensional voxel mesh with a unified spatial dimension.

[0089] S5. Feature Extraction and Target Segmentation:

[0090] S5.1 Input a three-dimensional voxel mesh, wherein the voxels are resampled to a fixed size (0.35×0.35×0.35mm), and histograms are applied to equalize them, reducing tissue contrast differences. The voxel intensity is normalized to the range of [0,1]. Through the data processing module, based on the improved U-Net++ network, dilated convolutions with different dilation rates are introduced into the network layers at different depths of the three-dimensional voxel mesh to extract multi-scale features of the blood vessel wall and tumor capsule. The dilation rates of the dilated convolutions are 1, 2, 4, 8, and 16, respectively.

[0091] S5.2 In the skip connections of the U-Net++ network, a gating mechanism is introduced to fuse multi-scale features into the decoder through gated skip connections, while sharing the underlying feature extraction network to simultaneously predict blood vessel segmentation masks and tumor contours. Here, a 1×1 convolution of the Sigmoid activation function is used as the gating mechanism.

[0092] S5.3 Extract global features at the end of the encoder of the U-Net++ network and input them into the EfficientNet-B0 classifier. Analyze the global features using the EfficientNet-B0 classifier and output a blood vessel density map. The EfficientNet-B0 classifier includes an encoder and a decoder. The encoder consists of 5 downsampling layers, each containing 3 convolutional kernels, with a size of 3×3 and a stride of 2. The number of output channels increases from 16 to 512. The decoder consists of 5 upsampling layers, each containing 3 convolutional kernels, with a size of 3×3 and a stride of 2. The number of output channels decreases from 512 to 16.

[0093] S5.4 Based on the blood vessel density map, an adaptive threshold segmentation algorithm is used to extract the blood vessel region, and a fractal-based morphological analysis method is used to extract the tumor contour features. Morphological operations (such as opening and closing operations) are applied to smooth the segmentation results, and majority voting and self-refining algorithms are used to optimize the segmentation boundary. The segmentation results of blood vessels and tumor tissue in the target region are generated, and the blood vessel segmentation mask, tumor contour features and blood vessel density map are output simultaneously.

[0094] S6. 3D Model Reconstruction:

[0095] S6.1. Through the data processing module, the segmentation results of blood vessels and tumor tissues are input into the diffusion model to generate an initial three-dimensional volume as the starting point for reconstruction. The diffusion model parameters are set as follows: diffusion steps are 1000, noise scheduling is linear scheduling, and its range is

[10] . -5 10 -2 The learning rate is 10. -4 The training rounds are 100 epochs, the batch size is 8, and the hardware configuration is Nvidia V100 GPU (32GB memory).

[0096] S6.2. Gradually add noise to the initial three-dimensional volume to generate diffusion paths;

[0097] S6.3. The three-dimensional structural model is recovered from the noise volume through the reverse diffusion process;

[0098] S6.4 During the posterior sampling process of the diffusion model, the reconstruction confidence of each voxel in the three-dimensional structural model is quantified. Only voxels with confidence > 0.9 are retained, and low-confidence regions are discarded to reduce errors and generate a three-dimensional reconstruction model.

[0099] S7. Real-time navigation display: The display module displays the three-dimensional reconstruction model in real time, realizing real-time intraoperative navigation of ultrasound endoscopy; preferably, the display module can also provide the functions of rotation, scaling and transparency adjustment of the three-dimensional reconstruction model, and render the three-dimensional reconstruction model, highlight the location and course of the identified key blood vessels, and generate visualization results that can be used for surgical navigation.

[0100] The principle of the ultrasonic endoscope system and its working method involved in this invention is as follows: The ultrasonic endoscope system of this invention uses a unified global feature space to carry all the original information by first fusing, then sensing and imaging, skipping the intermediate links of single-channel independent processing, thus avoiding information loss and error accumulation at the source. For multi-channel ultrasonic echo data, cross-source fusion is completed first, and then subsequent tasks such as imaging, segmentation, recognition, and 3D reconstruction are performed based on the unified feature body after fusion. This is completely different from the traditional ultrasonic endoscope that first independently senses and images, and then fuses the results, only merging the results in the final stage. At the same time, during the pre-fusion process, the ultrasonic endoscope system of this invention uniformly converts the ultrasonic echo data of the convex array transducer 4, which are scattered in different local perspectives and coordinate systems, into a BEV (Bird's Echo) with no perspective distortion and absolute spatial alignment. The EyeView (bird's-eye view) global feature space enables a complete and continuous representation of the target region's topological structure, spatial location, and semantic information. Therefore, through data processing modules, weight coefficients are calculated and three-dimensional voxel meshes are generated. Essentially, this establishes a quantifiable mapping relationship between the local view features of multiple convex array transducers 4 and the global world coordinate system of the BEV. This integrates the originally scattered local features into a unified global feature carrier that can be directly used for segmentation, reconstruction, and navigation. In the three-dimensional voxel mesh, each voxel corresponds to a unique physical spatial coordinate in the BEV global coordinate system. There is no perspective distortion where near objects appear larger than far objects. Spatial positions can be directly quantified and calculated without secondary coordinate transformation. This perfectly adapts to the precise measurement of the relative positions of surgical instruments and anatomical structures during surgery and completely preserves the three-dimensional spatial topological relationships of the target region, such as the branching direction of the biliary artery and the adjacent relationship between blood vessels and tumors. At the same time, the three-dimensional reconstruction model can be updated in real time with changes in ultrasound echo data. The feature map at each moment is strictly synchronized with the current intraoperative physical space, with no cumulative registration error and end-to-end processing delay, fully meeting the needs of intraoperative dynamic navigation.

[0101] Clinical validation of the endoscopic ultrasound system and its working method:

[0102] 1. Experimental Design:

[0103] A prospective comparative study was conducted on 30 patients with cholecystitis.

[0104] A double-blind, controlled design was employed, with two experienced surgeons performing scans using the system of this invention and a conventional system, respectively.

[0105] The accuracy of vascular segmentation and contour reconstruction was evaluated using MRI as the standard.

[0106] 2. Experimental Results:

[0107] Vascular segmentation coefficient: 91.8±2.1% for the endoscopic ultrasound system of this invention, and 74.5±3.2% for the conventional endoscopic ultrasound system;

[0108] Three-dimensional reconstructed Chamfer distance: 0.48±0.07 mm for the ultrasonic endoscope system of this invention, and 0.95±0.12 mm for the conventional ultrasonic endoscope;

[0109] Intra-group correlation coefficient (ICC): 0.977 for the ultrasonic endoscope system of this invention, and 0.927 for the conventional ultrasonic endoscope.

[0110] Verification results show that the Dice coefficient of the vascular detection system of the present invention reaches over 92%, and the sensitivity and specificity both exceed 90%, which is a significant improvement over traditional ultrasound endoscopy, significantly improving the accuracy of vascular identification. The Chamfer distance of the three-dimensional reconstruction model is reduced from 0.95±0.12mm in traditional ultrasound endoscopy to 0.48±0.07mm, greatly improving geometric accuracy and effectively solving the problems of holes and artifacts in traditional three-dimensional reconstruction. The ultrasound endoscopy system of the present invention significantly reduces the dependence on operator skill, improves the consistency of results among different operators, and the intragroup correlation coefficient (ICC) reaches over 0.977, which is a significant improvement over traditional ultrasound endoscopy.

[0111] As described above, the ultrasound endoscopy system and ultrasound imaging method based on pre-fusion BEV feature maps of the present invention have the following beneficial effects:

[0112] 1. The ultrasound endoscope probe, through its extension structure, can significantly reduce the resistance when the ultrasound endoscope probe enters the body cavity, reducing cavity stimulation; through the combined use of linear array transducer 3 and convex array transducer 4, it can achieve the complementary advantages of flat close-range imaging and fan-shaped wide-angle imaging. Linear array transducer 3 is responsible for high-precision imaging of the superficial mucosa and small blood vessels, while convex array transducer 4 is responsible for large-field fan-shaped scanning of deep tissues and the overall infiltration range of tumors, adapting to different morphological cavity tissues in the human body and improving the comprehensiveness of detection;

[0113] 2. The endoscopic ultrasound system and its working method, through systematic improvements in core technologies such as multi-convex array transducer 4-synergistic imaging, BEV global spatial registration, dynamic weighted data fusion, multi-scale intelligent segmentation, and high-confidence 3D reconstruction, meet the real-time requirements of surgical navigation. It can provide accurate and reliable 3D visualization models for real-time navigation of minimally invasive hepatobiliary surgery, and present the 3D anatomical structures that cannot be intuitively presented by traditional 2D ultrasound sections to doctors in a fully visualized manner, providing effective anatomical decision-making basis.

[0114] In summary, this invention effectively overcomes the various shortcomings of the prior art and has high industrial application value.

[0115] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. An ultrasonic endoscope probe, characterized in that: include: Endoscopic catheter (1), the distal end face of the endoscopic catheter (1) is an inclined mating bevel; The extension structure includes a handle (201), a slider switch (202), an axial push rod (203), a radial linkage rod (204), and a mounting plate (205); the handle (201) has an inner cavity, and the proximal end of the endoscope catheter (1) is fixed to the distal end of the handle (201); a through groove extending axially is provided on the outer peripheral surface of the handle (201), and the slider switch (202) is slidably engaged in the through groove; the axial push rod (203) Located within the cavities of the endoscope catheter (1) and the handle (201), one end of the radial linkage rod (204) is hinged to the axial push rod (203), and the other end is fixed to the slider switch (202); the mounting plate (205) has a connecting surface (251) and a working surface (252) arranged opposite to each other, the distal end of the axial push rod (203) is hinged to the connecting surface (251), and the connecting surface (251) is hinged to the tip of the mating inclined surface; Linear array transducer (3), the linear array transducer (3) is disposed on the working surface (252); Multiple convex array transducers (4) are arranged in an array on the working surface (252); The mounting plate (205) has a retracted position and an extended position. When the mounting plate (205) is in the retracted position, the connecting surface (251) is in contact with the mating inclined surface. When the mounting plate (205) is in the extended position, the working surface (252) is perpendicular to the central axis of the endoscope catheter (1). Cable (6), the cable (6) is located in the inner cavity of the endoscope catheter (1) and the handle (201), the cable (6) is electrically connected to the linear array transducer (3) and the plurality of convex array transducers (4) respectively, and is used to transmit the ultrasonic echo data of the linear array transducer (3) and the convex array transducer (4).

2. The ultrasonic endoscope probe according to claim 1, characterized in that: It also includes a water spray structure, which includes a water pipe (501) and multiple water nozzles (502). The water pipe (501) is located in the inner cavity of the endoscope guide tube (1) and the handle (201). The multiple water nozzles (502) are all located on the working surface (252) and are respectively located close to multiple convex array transducers (4). The multiple water nozzles (502) are respectively connected to the water pipe (501).

3. An ultrasonic endoscopy system, characterized in that: include: The ultrasonic endoscope probe as described in claim 1 is used to scan a target area and acquire ultrasonic echo data of the target area; The signal synchronization module is used to perform spatial position calibration on the multiple convex array transducers (4) of the ultrasonic endoscope probe and output the synchronization timing control signal of the multiple convex array transducers (4). The data acquisition module is used to acquire the ultrasonic echo data output by multiple convex array transducers (4), and to perform preprocessing on the acquired ultrasonic echo data to obtain the triaxial coordinate values ​​of each voxel in the target area. The data processing module is used to calculate the weighting coefficients of multi-channel ultrasound echo data; perform weighted fusion on the preprocessed ultrasound echo data to generate a three-dimensional voxel mesh; perform feature extraction and feature fusion on the three-dimensional voxel mesh to complete the segmentation of blood vessels and tumor tissue in the target area; and perform three-dimensional reconstruction on the segmentation results to generate a three-dimensional reconstruction model. The display module is used to display the three-dimensional reconstructed model.

4. The ultrasonic endoscope probe according to claim 3, characterized in that: The preprocessing performed by the data acquisition module includes: denoising the ultrasonic echo data using a wavelet denoising algorithm and compressing the dynamic range of the ultrasonic echo data to 30dB-300dB.

5. The ultrasonic endoscope probe according to claim 3, characterized in that: The data processing module calculates the weight coefficient of the multi-channel ultrasonic echo data, specifically: according to the spatial positions and scanning angles of the plurality of convex array transducers (4), the weight coefficient w of the multi-channel ultrasonic echo data is dynamically calculated i , and the weight coefficient w i is normalized to ensure that the sum of the weight coefficients of all convex array transducers (4) corresponding to the voxel is 1, and the calculation formula of the weight coefficient w i is as follows: （1） In the formula, d i θ represents the vertical distance from the voxel to the central axis of the i-th convex array transducer (4); i Let be the cosine of the angle between the scanning angle of the i-th convex array transducer (4) and the main scanning direction.

6. The ultrasonic endoscope probe according to claim 3, characterized in that: The data processing module performs weighted fusion of multiple ultrasonic echo data to generate a three-dimensional voxel mesh. Specifically, it fuses each voxel in the target region using a weighted average formula, maps the preprocessed ultrasonic echo data to the BEV global coordinate system, and generates a three-dimensional voxel mesh. The weighted average formula is as follows: （2） In the formula, N is the total number of convex array transducers (4); I i (x|y|z) The three-axis coordinate values ​​of the voxel collected by the i-th convex array transducer (4); I fused (x|y|z) represents the three-axis coordinates of the corresponding voxel in the three-dimensional voxel mesh in the BEV global coordinate system.

7. The ultrasonic endoscope probe according to claim 3, characterized in that: The data processing module performs feature extraction and feature fusion on the three-dimensional voxel mesh to segment blood vessels and tumor tissue in the target region. Specifically, based on the improved U-Net++ network, dilated convolutions with different dilation rates are introduced into different depth layers of the three-dimensional voxel mesh to extract multi-scale features of the blood vessel wall and tumor capsule. At the encoder end of the U-Net++ network, the global features are analyzed based on the EfficientNet-B0 classifier to output a blood vessel density map, thereby distinguishing blood vessels. Based on fractal morphological analysis methods, tumor contour features are extracted to complete the segmentation of blood vessels and tumor tissue in the target region.

8. The ultrasonic endoscope probe according to claim 3, characterized in that: The data processing module performs three-dimensional reconstruction on the segmentation results to generate a three-dimensional reconstruction model. Specifically, it uses a diffusion model to perform three-dimensional reconstruction on the segmentation results, including: generating an initial three-dimensional volume based on the segmentation results of blood vessels and tumor tissue in the target region; gradually adding noise to the initial three-dimensional volume to generate a diffusion path; recovering the three-dimensional structural model from the noise volume through a reverse diffusion process; quantifying the reconstruction confidence of each voxel in the three-dimensional structural model; retaining only voxels with a confidence > 0.9; and generating a three-dimensional reconstruction model.

9. A method for operating an ultrasonic endoscope system, characterized in that: The working method is implemented using the ultrasonic endoscope system as described in any one of claims 3-8, and includes the following steps: S1. Position calibration and timing synchronization: The spatial position calibration of multiple convex array transducers (4) is performed through the signal synchronization module to eliminate the spatial deviation between multiple convex array transducers (4), and at the same time, the synchronization timing control signal of multiple convex array transducers (4) is output to ensure that the timing error of ultrasonic wave transmission and echo reception of each convex array transducer (4) is ≤1μs. S2. Data acquisition and preprocessing: The ultrasonic echo data output by multiple convex array transducers (4) is acquired synchronously through the data acquisition module. The sampling rate is controlled to be ≥300MHz. The ultrasonic echo data is denoised by wavelet denoising algorithm and the dynamic range of ultrasonic echo data is compressed to 30dB-300dB. S3. Calculate the weighting coefficient: Through the data processing module, for each voxel in the target area, calculate the vertical distance d from the voxel to the central axis of the corresponding convex array transducer (4). i Calculate the cosine value θ of the angle between the scanning angle of the convex array transducer (4) and the main scanning direction. i The weighting coefficient wi of the multi-channel ultrasonic echo data is dynamically calculated by formula (1), and the weighting coefficient wi is normalized to ensure that the sum of the weighting coefficients of the voxels corresponding to all convex array transducers (4) is 1. S4. Generate a three-dimensional voxel mesh: Through the data processing module, according to formula (2), for each voxel in the target area, the weighted average formula is used to fuse them, and the preprocessed ultrasonic echo data is mapped to the BEV global coordinate system to generate a three-dimensional voxel mesh with a unified spatial dimension. S5. Feature Extraction and Target Segmentation: S5.1 Input a three-dimensional voxel mesh. Through the data processing module, based on the improved U-Net++ network, introduce dilated convolutions with different dilation rates into different depth network layers of the three-dimensional voxel mesh to extract multi-scale features of blood vessel walls and tumor capsules. S5.2 Introduce a gating mechanism in the skip connections of the U-Net++ network. Through the gating skip connections, multi-scale features are fused to the decoder, while sharing the underlying feature extraction network to simultaneously predict the blood vessel segmentation mask and tumor contour. S5.3 Extract global features at the encoder end of the U-Net++ network and input them into the EfficientNet-B0 classifier. Analyze the global features through the EfficientNet-B0 classifier and output a blood vessel density map. S5.4 Based on the blood vessel density map, an adaptive threshold segmentation algorithm is used to extract the blood vessel region, and a fractal morphological analysis method is used to extract the tumor contour features to generate the segmentation results of blood vessels and tumor tissue in the target region. The blood vessel segmentation mask, tumor contour features and blood vessel density map are output simultaneously. S6. 3D Model Reconstruction: S6.

1. Through the data processing module, the segmentation results of blood vessels and tumor tissues are input into the diffusion model to generate an initial three-dimensional volume as the starting point for reconstruction. S6.

2. Gradually add noise to the initial three-dimensional volume to generate diffusion paths; S6.

3. The three-dimensional structural model is recovered from the noise volume through the reverse diffusion process; S6.4 During the posterior sampling process of the diffusion model, the reconstruction confidence of each voxel in the three-dimensional structural model is quantified, and only voxels with a confidence of >0.9 are retained to generate a three-dimensional reconstruction model. S7. Real-time navigation display: The three-dimensional reconstruction model is displayed in real time through the display module, realizing real-time intraoperative navigation of ultrasound endoscopy.

Images