An appendicitis endoscopic surgery navigation system based on multi-modal image fusion

Abstract

The application discloses a kind of based on multi-modal image fusion appendicitis endoscopic surgery navigation system, the present application relates to surgical navigation system technical field.The based on multi-modal image fusion appendicitis endoscopic surgery navigation system, comprising: multi-modal image fusion modeling unit, topological navigation parameter pre-configuration unit, real-time pose and video access unit, local occlusion feature completion unit, dynamic safety path generation unit, micro recursive emergency navigation unit, intraoperative fusion rendering unit, dual-mode seamless switching unit;The present scheme is directed to the exclusive anatomical characteristics of appendix, and the whole chain exclusive rules from preoperative topological annotation, intraoperative motion constraint to emergency boundary check are customized, which is different from the generalization parameters of general laparoscopic navigation, and all constraint logic is completely consistent with the clinical operation scene of appendiceal endoscope, and the native surgical field is not occluded throughout the translucent color label superposition rendering, and can be highlighted when deviating from safety path Warning.

Description

Technical Field

[0001] This invention relates to the field of surgical navigation system technology, specifically to a surgical navigation system for appendicitis endoscopy based on multimodal image fusion. Background Technology

[0002] Existing surgical navigation systems are based on multimodal medical imaging data such as CT and MRI. Through technologies such as three-dimensional reconstruction and optical / electromagnetic positioning, they match the patient's anatomical structure and the position of surgical instruments in real time, assisting doctors in accurately planning surgical paths and monitoring the operation process. They have expanded from early applications in neurosurgery to multiple fields such as orthopedics, ophthalmology, and thoracic surgery. Domestic equipment has achieved technological breakthroughs in AR navigation, portability, low cost, and sub-millimeter precision, which significantly reduces surgical trauma and operational risks, and promotes the development of minimally invasive surgery towards a more precise and safer direction.

[0003] As disclosed in prior art CN115645050, a surgical navigation system includes a working module, an intraoperative image acquisition module, a display module, and an execution module. The execution module includes a calibration unit and an execution unit. The calibration unit is used to connect with a preset location of the object to unify the object into a preset coordinate system. The intraoperative image acquisition module acquires intraoperative images of a specified location of the object. The working module establishes a surgical planning scheme based on preset CT image information, then registers the preset CT image information with the intraoperative image information, and transmits the preset CT image and surgical planning scheme to the display module. The execution unit transmits the coordinate position information of the execution unit to the working module. The display module is used to display the preset CT image, the surgical planning scheme, and the coordinate position information of the execution unit.

[0004] As shown in the above technology, existing general surgical navigation systems are mostly based on image modeling to achieve instrument positioning and guidance. However, when applied to the special luminal environment of the appendix, which is long and thin-walled, there are some obvious limitations. For example, there are many mucosal folds in the appendix lumen. If the endoscopic field of view is partially obscured by mucosal folds or mucus, conventional visual matching is prone to losing effective features, resulting in navigation tracking interruption. Furthermore, if a small amount of bleeding completely covers the field of view during the operation, the system has no emergency navigation mechanism and will directly lose its guiding ability. Moreover, the appendix lumen is narrow, and blind operation is not safe enough. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a surgical navigation system for appendectomy based on multimodal image fusion, which solves the problems existing in the application of existing surgical navigation systems to appendectomy.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a multimodal image fusion-based endoscopic appendectomy navigation system, comprising: The multimodal image fusion modeling unit is used to complete the acquisition, registration and fusion of preoperative CT and endoscopic ultrasound images to generate a three-dimensional visualization model of the entire appendix area. The topology navigation parameter pre-configuration unit is used to complete the intracavitary feature annotation and topology relationship construction in the 3D model, and generate a dedicated topology feature library and biomechanical deformation prior constraints. The real-time pose and video access unit is used for low-latency intraoperative access to high-definition endoscopic video streams and optical positioning pose data, and caches the full trajectory information of the first 3 frames. The partial occlusion feature completion unit is used to calculate the coordinates of missing features based on topological relationships when the field of view is partially occluded by mucus or wrinkles, thus maintaining the continuity of matching. The dynamic safety path generation unit is used to generate a dynamic guide line by combining the real-time position of the lens with the target point of the lesion, and simultaneously display the remaining distance. The micro-recursive emergency navigation unit is used to complete visual recursion based on cached pose and step size when the full field of view bleeding features are lost, and the constrained trajectory does not go out of the lumen. The intraoperative fusion rendering unit is used to overlay pre-marked lesions, blood vessels, and danger zones onto the native endoscopic field of view with semi-transparent color marks, and to provide lens safety warnings.

[0007] Preferably, the working steps of the multimodal image fusion modeling unit are as follows: before the operation, the patient undergoes a thin-slice abdominal CT scan to acquire complete three-dimensional tomographic images of the appendix and its surrounding blood vessels and mesangial tissues; simultaneously, endoscopic ultrasound scan is performed to obtain detailed image data of each layer of the appendix wall structure and lesions within the lumen. Then, a rigid registration algorithm is used to achieve global spatial coarse alignment between CT images and endoscopic ultrasound images. Then, non-rigid registration is used to correct spatial errors caused by patient position and natural organ deformation. Finally, a complete three-dimensional visualization model covering the appendix body, intraluminal lesions, surrounding blood vessels, and surgical risk areas is generated.

[0008] Preferably, the working steps of the topology navigation parameter pre-configuration unit are as follows: In the completed 3D visualization model, AI automatic annotation combined with manual verification is used to mark the precise spatial coordinates of all feature points of each mucosal fold and each identifiable microvascular in the appendix lumen, and the adjacent connection relationship and branch direction topological association information between each feature point are recorded simultaneously, and the recorded data is stored in the appendix-specific topology feature library; at the same time, each topology feature point is bound with a lumen axial depth label starting from the 0mm reference position at the appendix opening, and an axial anchoring system covering the entire lumen is constructed; considering the extremely short stroke characteristic of the appendix of only a few centimeters, micro-recursive running parameters with a step size of 0.1mm are pre-customized, the pre-constructed appendix-specific biomechanical deformation prior library is entered, and exclusive motion constraint rules adapted to the slender lumen are preset, and the inner wall contour of the appendix lumen extracted from the preoperative 3D model is set as the recursive hard boundary, and all invalid motion trajectories exceeding the range of the appendix lumen are automatically filtered during operation.

[0009] Preferably, the working steps of the real-time pose and video access unit are as follows: the high-definition video signal output by the intraoperative endoscope is accessed to the navigation system through a low-latency transmission link, and the spatial pose and absolute position information of the endoscope lens are tracked in real time through the optical positioning module; the system background continuously caches the most recent 3 frames of accurate pose data of the endoscope, and fully records the full trajectory information including the lens's moving speed, rotation angle, and direction of movement.

[0010] Preferably, the working steps of the dynamic safety path generation unit are as follows: the system combines the real-time position of the current lens with the pre-set target point for lesion resection, automatically calculates and generates a dynamic three-dimensional path guide line, and simultaneously displays the real-time remaining distance value from the lens to the target point in the corner area of ​​the field of view.

[0011] Preferably, the working steps of the intraoperative fusion rendering unit are as follows: the system performs rapid feature point matching between the real-time acquired endoscopic images and the preoperative 3D model, and uses a lightweight deep learning real-time fusion algorithm to overlay and render the preoperatively marked lesion boundaries, blood vessel patterns, and dangerous areas in the native endoscopic field of view in the form of semi-transparent color marks; during the entire operation, when the endoscopic lens deviates from the preset safe path and approaches the preoperatively marked dangerous area, the system pops up a bright deflection prompt at the edge of the endoscopic field of view.

[0012] Preferably, the working steps of the local occlusion feature completion unit are as follows: when the endoscopic field of view is partially obscured by mucosal folds and thin mucus, axial depth anchoring is completed by the remaining visible feature points in the field of view to quickly lock the specific depth position of the current lens in the appendix lumen; based on the topological association relationship of the matched visible feature points, the spatial coordinates of all feature points in the obscured area are calculated by geometric constraints along the inherent direction of the folds and blood vessels stored in the preoperative data; during the calculation process, the cross-validation logic of folds and blood vessels is started simultaneously to eliminate false feature points caused by mucus reflection and mucosal bleeding points, and then combined with the real-time acquired lens traction posture, the offset of the local feature coordinates is finely adjusted by calling the appendix biomechanical deformation prior library, and the feature matching is maintained continuously without interruption throughout the process.

[0013] Preferably, the working steps of the micro-progression emergency navigation unit are as follows: when a small amount of bleeding during the operation completely covers the endoscopic field of view and visual features are completely lost, the system automatically switches to the pre-configured 0.1mm step size micro-progression emergency mode, directly retrieves the cached first 3 frames of accurate pose data, and starts the inter-frame motion inertia lock-up mechanism to strictly constrain the motion parameters in the subsequent progression process within the reasonable range of clinical operation of appendectomy; after each 0.1mm position calculation is completed, the system automatically checks whether the current calculation point falls within the hard boundary of the appendiceal lumen and corrects invalid trajectories that exceed the boundary.

[0014] Preferably, the system also includes a dual-mode seamless switching unit: when the bleeding gradually subsides and the endoscopic field of view reveals enough effective feature points again, the system fixes the position of the last frame output of the micro-recursion mode as the reference anchor point, and completes the rapid rematch of visual features in a local range of millimeters around the anchor point. Through progressive pose calibration, the system completes the zero-error seamless switching between the two modes of micro-recursion emergency navigation and conventional visual fusion navigation.

[0015] Preferably, the specific steps for seamless switching between the two modes with zero error are as follows: The system continuously performs pixel-level semantic recognition on the real-time endoscopic images at 10ms intervals, calls a pre-trained appendix-specific bleeding semantic segmentation model, and counts the proportion of effective mucosal features in the images in real time. When the number of effective feature points of mucosal texture, blood vessels, and folds not covered by blood is ≥5 in 3 consecutive frames, and the distribution of these feature points covers at least 2 different quadrants of the image, the system determines that the bleeding occlusion state has been officially lifted and automatically starts the navigation mode switching process. Immediately freeze all pose data of the last frame of the micro-recursive mode, and solidify the six types of parameters of the camera at this moment, including spatial three-dimensional coordinates, rotation angle, axial depth label, and motion speed, as the sole reference anchor point for visual rematching; at the same time, retrieve the feature coordinate set of all pre-labeled mucosal folds and microvessels within the axial depth ±1mm range from the preoperative topological feature library to limit the search range of subsequent visual matching. The five or more newly exposed effective feature points in the current endoscopic image are quickly matched with the preoperative topological feature library within a limited range, and feature alignment is completed based on the cross-validation rule of folds and blood vessels. Automatically compare the new pose obtained from visual rematch with the reference anchor pose, and perform pose calibration based on different deviation thresholds and corresponding gradients. After calibration is completed, the system automatically exits the micro-push emergency mode and switches back to the regular real-time visual fusion navigation state. Simultaneously, relevant data on bleeding occlusion is recorded in the background and stored in the surgical log.

[0016] This invention provides a navigation system for endoscopic appendectomy based on multimodal image fusion. Compared with existing technologies, it has the following advantages: 1. Most existing general laparoscopic navigation systems directly reuse abdominal CT images for global registration. However, for the appendix, a slender tube with a diameter of only a few millimeters and a very short travel distance, there are common problems such as image fusion misalignment and omission of annotations for small blood vessels and mucosal folds. This solution uses a two-stage registration process of "rigid coarse alignment + non-rigid fine correction" between CT and endoscopic ultrasound, combined with a dedicated topological feature library constructed using AI annotation and manual verification. For the first time, it achieves full feature spatial coordinate coverage of every mucosal fold and every identifiable small blood vessel in the appendix lumen. The modeling accuracy is improved from 2-3mm in conventional navigation to 0.1mm level, fundamentally solving the industry pain points of low anatomical feature recognition in the appendix lumen and large deviations between preoperative models and actual intraoperative anatomical morphology.

[0017] 2. Traditional endoscopic navigation loses pose tracking and navigation disconnection when the field of view is obstructed by mucus or mucosal folds due to insufficient effective feature points. This solution relies on a pre-constructed topological association set R to quickly locate the current lumen position of the lens using an axial depth anchoring formula. It directly calculates the feature coordinates of the obstructed area by following the inherent topological orientation of folds and blood vessels. Then, it uses dual-feature cross-validation to eliminate false features caused by mucus reflection and bleeding points, achieving continuous and uninterrupted feature matching throughout the entire process in partially obstructed scenarios, significantly improving navigation stability under complex fields of view.

[0018] 3. Existing endoscopic navigation systems fail when intraoperative bleeding completely covers the field of view, resulting in the loss of visual features. Surgeons are forced to rely on experience and operate blindly, increasing the risk of the lens penetrating the appendix wall or injuring surrounding blood vessels. This proposed solution uses an inter-frame motion inertial locking constraint formula to strictly limit recursive motion parameters within a reasonable range for clinical appendectomy. Combined with a formula for verifying the legality of the lumen's hard boundary, it performs micro-recursive calculations in 0.1mm steps, maintaining sub-millimeter-level navigation accuracy within a 2-second emergency window. This completely avoids the problem of complete navigation failure during full-field bleeding.

[0019] 4. Conventional navigation often experiences pose jumps and sudden misalignments of superimposed color markers when visual tracking resumes after bleeding subsides, which can easily disrupt the surgeon's rhythm. This solution uses a dual-mode switching progressive calibration formula to align the micro-recursive final frame anchor point with the visual rematch coordinates through a smooth 5-step transition, keeping the maximum offset within 0.2mm throughout. This completely eliminates the screen jump misalignment problem during mode switching, and there are no perceptible gaps during the switching between the two navigation modes, ensuring uninterrupted surgeon continuity.

[0020] 5. This solution is tailored to the unique anatomical characteristics of the appendix, namely its narrow lumen, short travel distance, and thin wall. It features a complete set of exclusive rules, from preoperative topological annotation and intraoperative motion constraints to emergency boundary verification. Unlike the generalized parameters of universal laparoscopic navigation, all constraint logic is perfectly aligned with the clinical operation scenario of appendectomy. The semi-transparent color mark overlay rendering does not obstruct the original surgical field of view throughout the entire process. Combined with the high-brightness warning when deviating from the safe path, it not only preserves the operating habits of doctors in routine surgical observation but also provides a safety net for the entire process, significantly reducing the learning curve threshold for young doctors to perform complex appendectomy. Attached Figure Description

[0021] Figure 1 This is a system module block diagram of the present invention; Figure 2 This is a schematic diagram of the overall surgical procedure of the present invention; Figure 3 This is a detailed flowchart of the seamless dual-mode switching of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] See Figures 1-3This invention discloses a navigation system for endoscopic appendectomy based on multimodal image fusion, and provides the following three technical solutions: First implementation method: includes: The multimodal image fusion modeling unit is used to perform a thin-slice CT scan of the abdomen before surgery to acquire complete three-dimensional tomographic images of the appendix and its surrounding blood vessels and mesangial tissues, denoted as image set A; at the same time, endoscopic ultrasound scan is performed to obtain detailed image data of each layer of the appendix wall structure and lesions in the lumen, denoted as image set B, to provide multi-dimensional raw materials for subsequent modeling. First, a rigid registration algorithm is used to achieve global spatial coarse alignment between image set A and image set B. Then, non-rigid registration is used to correct spatial errors caused by patient position and natural organ deformation. Finally, a complete three-dimensional visualization model covering the appendix body, intraluminal lesions, surrounding blood vessels, and surgical risk areas is generated, denoted as model M.

[0024] The topology navigation parameter pre-configuration unit is used to accurately annotate the spatial coordinates of all feature points of every mucosal fold and every identifiable microvessel within the appendix lumen in the completed model M using AI automatic annotation combined with manual verification. The set of all feature points is denoted as P. Simultaneously, the unit records the topological association information of the adjacent connections and branch directions between each feature point, and the set of topological associations is denoted as R. Sets P and R are jointly stored in the appendix-specific topological feature library L. At the same time, each topological feature point is bound to a luminal axial depth marker starting from the 0mm reference position at the appendix opening. The axial depth of a single feature point is denoted as d, and the set of axial depth labels for the entire lumen is denoted as D, constructing an axial anchoring system covering the entire lumen. Considering the extremely short stroke of the appendix (only a few centimeters), micro-recursive running parameters with a step size of s=0.1mm are pre-defined, and a pre-constructed biomechanical deformation prior library K specific to the appendix is ​​entered. Motion constraint rules adapted to this slender lumen are preset, and the inner wall contour of the appendix lumen extracted from the preoperative 3D model is set as a recursive hard boundary. The set of all spatial boundary coordinate points is denoted as C. During operation, all invalid motion trajectories exceeding the set C are automatically filtered out.

[0025] The real-time pose and video access unit is used to connect the high-definition video signal output by the endoscope during the operation to the navigation system through a low-latency transmission link. Simultaneously, the optical positioning module tracks the spatial attitude and absolute position information of the endoscope lens in real time. The pose data of a single frame is denoted as F. The system background continuously caches the most recent 3 frames of accurate pose data of the endoscope, and the cache set is denoted as Q. It fully records the full trajectory information such as the lens's moving speed v, rotation angle θ, and motion direction vector V.

[0026] The dynamic safety path generation unit is used by the system to automatically calculate and generate a dynamic three-dimensional path guide line by combining the real-time position of the current lens with the pre-set target point T for lesion resection. Simultaneously, the real-time remaining distance l from the lens to the target point T is displayed in the corner area of ​​the field of view.

[0027] Intraoperative fusion rendering unit: It is used to quickly match feature points of the real-time acquired endoscopic images with model M. Through a lightweight deep learning real-time fusion algorithm, the pre-annotated lesion boundaries, blood vessel patterns and danger areas are superimposed and rendered in the form of semi-transparent color marks on the native endoscopic field of view, without obstructing the doctor's normal surgical observation line of view throughout the operation. During the operation, when the endoscope lens deviates from the preset safe path and approaches the pre-annotated danger area, the system will pop up a bright deflection prompt at the edge of the endoscopic field of view.

[0028] The partial occlusion feature completion unit is used to complete axial depth anchoring by using the remaining visible feature points in the field of view and the axial depth anchoring matching formula when the endoscopic field of view is partially obscured by mucosal folds or thin mucus. This allows the lens to pinpoint the specific depth position of the lens within the appendiceal lumen. ; Where n is the number of visible feature points that have been successfully matched in the current field of view, and d i The axial depth label pre-stored for the i-th visible feature point, Δd offset The measured axial offset from the lens end face to the feature point is d, which is then calculated. cur This refers to the precise axial depth of the lens within the cavity; Based on the topological association set R of the already matched visible feature points, following the pre-stored folds and inherent directions of blood vessels, the precise spatial coordinates of all feature points in the occluded area are directly calculated through geometric constraints. During the calculation process, the logic of cross-validation of folds and blood vessels is simultaneously activated to eliminate false feature points caused by mucus reflection and mucosal bleeding points. Then, combined with the real-time acquired lens traction posture, the deformation prior library K is called to fine-tune the offset Δp of the local feature coordinates, maintaining continuous and uninterrupted feature matching throughout the process.

[0029] The micro-push emergency navigation unit is used to automatically switch to the pre-configured micro-push emergency mode with a step size of s=0.1mm when a small amount of bleeding during the operation completely covers the endoscopic field of view and visual features are completely lost. The specific operation is as follows: The system directly retrieves the first three frames of precise pose data from the cache set Q, activates the inter-frame motion inertia lock constraint formula, and strictly constrains the motion parameters v, θ, and V in the subsequent recursive process within the specified range for clinical appendectomy: ; Among them, v cur Let θ be the camera movement speed obtained through recursion.cur and θ pre V represents the camera rotation angles of the current frame and the previous frame, respectively. cur V is the current motion direction vector of the camera. axis (d cur ) represents the current axial depth d cur The direction vector of the central axis of the cavity corresponding to the position. All recursive results that exceed this constraint are directly judged as illegal and are smoothed by the parameters of the previous legal frame. After each step s of position estimation, the system automatically verifies whether the current estimated point falls within the set C of hard boundaries of the appendix lumen using the lumen hard boundary validity check formula: dist(P cur C)≤R(d) cur ); Among them, P cur The current recursive obtained 3D coordinates of the lens, dist() is the function to calculate the shortest distance from a spatial point to the set, d cur R(d) represents the axial depth of the cavity corresponding to the current point. cur The radius of the appendix lumen corresponding to this axial depth position is set to the preset radius. If the formula is invalid, the recursive point is directly determined to be invalid, and it is automatically corrected to the valid range along the central axis of the lumen. Within the 2-second emergency window, sub-millimeter-level navigation accuracy is maintained to avoid complete navigation interruption.

[0030] Dual-mode seamless switching unit: When bleeding gradually subsides and sufficient effective feature points are re-exposed in the endoscopic field of view, the system fixes the position of the last frame output of the micro-recursive mode as the reference anchor point O, and completes rapid rematching of visual features within a millimeter-level local range around the anchor point O. Through the dual-mode switching progressive calibration formula, it achieves zero-error seamless switching between the micro-recursive emergency navigation and conventional visual fusion navigation modes. ; Among them, P ins P represents the coordinates of the last frame anchor point output by the micro-recursion. vis For the precise coordinates obtained from visual rematching, P k It is the intermediate transition anchor point coordinate of step k in the dual-mode switching calibration process. Through 5 steps of progressive offset within 0.2mm each, the two coordinates are finally smoothly aligned, completely avoiding screen jump misalignment.

[0031] The second implementation method differs from the first in that the specific steps for seamless switching between the two modes with zero error are as follows: The system continuously performs pixel-level semantic recognition on real-time endoscopic images at 10ms intervals (t0=10ms), calling a pre-trained semantic segmentation model specifically for appendiceal bleeding, and statistically analyzing the proportion of effective mucosal features in the images in real time. The system counts the number n effective feature points of mucosal textures, blood vessels, and folds not covered by blood in three consecutive frames. val When there are ≥5 features, and the distribution of these feature points covers at least 2 different quadrants of the screen, the system determines that the bleeding occlusion state has been officially lifted and automatically starts the navigation mode switching process to avoid accidentally triggering the switch due to the brief exposure of sporadic features in a local area of ​​the screen. The system immediately freezes all pose data of the last frame in the micro-recursive mode and sets the spatial three-dimensional coordinates P of the camera at that moment. ins Rotation angle θ ins Axial depth label d ins , velocity v ins A total of 6 types of parameters are fully fixed, serving as the unique set of benchmark anchor points for visual rematching. set Simultaneously, retrieve the preoperative topological feature library L, using d ins The axial depth range centered on [d] ins 1mm, d ins The set of feature coordinates of all pre-labeled mucosal folds and microvessels within a range of +1mm is denoted as the local feature subset P. loc This limits the search scope for subsequent visual matching, reducing the global search to a millimeter-level local search, and significantly reducing matching time. The system denotes five or more newly revealed valid feature points in the current endoscopic image as the real-time feature set P′, and the local feature subset P′ as the local feature subset P′. loc For fast matching, the "cross-validation of fold and blood vessel features" rule is used: the fold topological continuity criterion R is defined. f Criterion for consistency of vascular branch morphology R v If R is satisfied at the same time f =1、R v =1, directly complete feature alignment; if a small number of feature matching conflicts occur, automatically call the appendix biomechanical deformation prior library K, and finely adjust the offset of local feature coordinates Δp′ in combination with the current traction posture of the lens to eliminate false feature interference caused by residual bleeding and ensure that the matching result fully conforms to the real anatomical morphology. The system automatically compares the new pose F obtained from visual rematching. vis And the previously solidified micro-recursive final state reference anchor point O set Corresponding pose F insThe system calculates the 3D coordinate deviation Δe between the two. If the deviation Δe ≤ 0.2mm, the trajectory accuracy of the entire micro-recursion is directly determined to be qualified, and no screen jump correction is performed. If the deviation is between 0.2mm < Δe ≤ 0.5mm, the system performs a 5-frame progressive smooth transition with a step size of 0.05mm, gradually correcting the positions of the navigation overlay color marks and guide lines to completely avoid screen jumps and misalignments. If the deviation Δe > 0.5mm, a second local topology feature verification is automatically triggered to check for any uncleared bleed pseudo-features, and the transition is performed again after accurate matching is completed. After completing pose calibration, the system automatically exits the micro-push emergency mode and smoothly switches back to the normal real-time visual fusion navigation state, while simultaneously recording the duration t of the bleeding occlusion in the background. b Micro-recursive trajectory deviation value e avg Rematching time t m The three types of data are stored in the surgical log to provide data support for parameter optimization in similar scenarios in the future. The entire switching process takes ≤30ms, and the doctor cannot perceive the switching of navigation modes in the field of vision, achieving a truly zero-error seamless connection.

[0032] The main difference between the third and second implementation methods is that the motion constraint rules preset by the topology navigation parameter pre-configuration unit to adapt to the slender lumen of the appendix include: Axial movement range constraint rules: The appendiceal opening marked in the preoperative 3D model is taken as the starting point (0mm) and denoted as the axial starting point coordinate S. The end of the appendix is ​​taken as the ending point and denoted as the axial ending point coordinate E. The total axial length of the appendiceal lumen is denoted as L. total The axial movement range of the endoscope lens is strictly limited to 0 ≤ d. cur ≤ L total Within a total appendiceal lumen length range of ≤30mm; simultaneously limiting the maximum axial movement speed v of the endoscope along the lumen. axial ≤2mm / s, perfectly matching the actual advancement speed of clinicians operating appendectomyoscopes, filtering out abnormal recursive trajectories that exceed this speed range; Radial range of motion constraint rule: based on each axial depth d in the preoperative 3D model. cur Using the central axis of the corresponding appendiceal lumen as a reference, the radial offset distance r of the endoscope lens is limited. offset Satisfying r offset ≤ 0.8 × R(d cur The radial offset of the lens cannot exceed 80% of the actual radius of the appendix lumen at that location; for example, if the diameter of a certain appendix lumen is 3mm, the radial offset of the lens is limited to no more than 1.2mm, so as to avoid invalid results where the recursive trajectory passes through the appendix wall from the source. Posture angle change constraint rules: limit the maximum change of the pitch and rotation angle of the endoscope lens in a single frame to Δθ≤3°, and at the same time, the angle between the lens axis and the central axis of the appendix lumen at the current position can never exceed 20°; fully adapt to the slender lumen of the appendix and filter out large-angle deflection recursive results that do not conform to the clinical operation logic.

[0033] Furthermore, any content not described in detail in this specification is existing technology known to those skilled in the art.

[0034] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0035] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An appendicitis endoscopic surgery navigation system based on multi-modal image fusion, characterized in that, include: The multimodal image fusion modeling unit is used to complete the acquisition, registration and fusion of preoperative CT and endoscopic ultrasound images to generate a three-dimensional visualization model of the entire appendix area. The topology navigation parameter pre-configuration unit is used to complete the intracavitary feature annotation and topology relationship construction in the 3D model, and generate a dedicated topology feature library and biomechanical deformation prior constraints. The real-time pose and video access unit is used for low-latency intraoperative access to high-definition endoscopic video streams and optical positioning pose data, and caches the full trajectory information of the first 3 frames. The dynamic safety path generation unit is used to generate a dynamic guide line by combining the real-time position of the lens with the target point of the lesion, and simultaneously display the remaining distance. The intraoperative fusion rendering unit is used to overlay pre-marked lesions, blood vessels, and danger areas onto the native endoscopic field of view with semi-transparent color marks, and to provide lens safety warnings. The partial occlusion feature completion unit is used to calculate the coordinates of missing features based on topological relationships when the field of view is partially occluded by mucus or wrinkles, thus maintaining the continuity of matching. The micro-recursive emergency navigation unit is used to complete visual recursion based on cached pose and step size when the full field of view bleeding features are lost, and to constrain the trajectory to not go out of the lumen.

2. The endoscopic appendectomy navigation system based on multimodal image fusion according to claim 1, characterized in that: The working steps of the multimodal image fusion modeling unit are as follows: before the operation, the patient undergoes a thin-slice abdominal CT scan to acquire complete three-dimensional tomographic images of the appendix and its surrounding blood vessels and mesangial tissues; simultaneously, endoscopic ultrasound scan is performed to obtain detailed image data of each layer of the appendix wall and lesions within the lumen. Then, a rigid registration algorithm is used to achieve global spatial coarse alignment between CT images and endoscopic ultrasound images. Then, non-rigid registration is used to correct spatial errors caused by patient position and natural organ deformation. Finally, a complete three-dimensional visualization model covering the appendix body, intraluminal lesions, surrounding blood vessels, and surgical risk areas is generated.

3. The endoscopic appendectomy navigation system based on multimodal image fusion according to claim 2, characterized in that: The specific working steps of the topology navigation parameter pre-configuration unit are as follows: In the completed 3D visualization model, AI automatic annotation combined with manual verification is used to accurately annotate the spatial coordinates of all feature points of every mucosal fold and every identifiable microvascular in the appendix lumen, and simultaneously record the topological association information of the adjacent connection relationship and branch direction between each feature point, and store the recorded data in the appendix-specific topology feature library; at the same time, bind the axial depth label of the lumen starting from the 0mm reference position at the appendix opening to each topology feature point, and construct an axial anchoring system covering the entire lumen; considering the extremely short stroke characteristic of the appendix of only a few centimeters, micro-recursive running parameters with a step size of 0.1mm are pre-customized, the pre-constructed appendix-specific biomechanical deformation prior library is entered, and exclusive motion constraint rules adapted to the slender lumen are preset, and the inner wall contour of the appendix lumen extracted from the preoperative 3D model is set as the recursive hard boundary, and all invalid motion trajectories that exceed the range of the appendix lumen are automatically filtered during operation.

4. The endoscopic appendectomy navigation system based on multimodal image fusion according to claim 1, characterized in that: The specific working steps of the real-time pose and video access unit are as follows: the high-definition video signal output by the intraoperative endoscope is connected to the navigation system through a low-latency transmission link, and the spatial attitude and absolute position information of the endoscope lens are tracked in real time through the optical positioning module. The system background continuously caches the most recent 3 frames of accurate pose data of the endoscope, and fully records the complete trajectory information including the lens's movement speed, rotation angle, and direction of movement.

5. The endoscopic appendectomy navigation system based on multimodal image fusion according to claim 1, characterized in that: The specific working steps of the dynamic safety path generation unit are as follows: the system combines the real-time position of the current lens with the pre-set target point for lesion resection, automatically calculates and generates a dynamic three-dimensional path guide line, and simultaneously displays the real-time remaining distance value from the lens to the target point in the corner area of ​​the field of view.

6. The endoscopic appendectomy navigation system based on multimodal image fusion according to claim 1, characterized in that: The specific steps of the intraoperative fusion rendering unit are as follows: The system performs rapid feature point matching between the real-time acquired endoscopic images and the preoperative 3D model. Through a lightweight deep learning real-time fusion algorithm, the preoperatively marked lesion boundaries, blood vessel patterns, and dangerous areas are superimposed and rendered in the native endoscopic field of view in the form of semi-transparent color marks. During the entire operation, when the endoscopic lens deviates from the preset safe path and approaches the preoperatively marked dangerous area, the system pops up a highlighted deflection prompt at the edge of the endoscopic field of view.

7. The endoscopic appendectomy navigation system based on multimodal image fusion according to claim 1, characterized in that: The specific steps of the local occlusion feature completion unit are as follows: When the endoscopic field of view is partially obscured by mucosal folds and thin mucus, axial depth anchoring is completed by the remaining visible feature points in the field of view to quickly lock the specific depth position of the current lens in the appendix lumen; based on the topological association relationship of the matched visible feature points, the spatial coordinates of all feature points in the obscured area are calculated by geometric constraints along the inherent direction of the folds and blood vessels stored in the preoperative data; during the calculation process, the cross-validation logic of folds and blood vessels is started simultaneously to eliminate false feature points caused by mucus reflection and mucosal bleeding points, and then combined with the real-time acquired lens traction posture, the offset of the local feature coordinates is fine-tuned by calling the appendix biomechanical deformation prior library, and the feature matching is maintained continuously without interruption throughout the process.

8. The endoscopic appendectomy navigation system based on multimodal image fusion according to claim 1, characterized in that: The specific working steps of the micro-progression emergency navigation unit are as follows: when a small amount of bleeding during the operation completely covers the endoscopic field of view and visual features are completely lost, the system automatically switches to the pre-configured 0.1mm step size micro-progression emergency mode, directly retrieves the cached first 3 frames of accurate pose data, and starts the inter-frame motion inertia lock-up mechanism to strictly constrain the motion parameters in the subsequent progression process within the reasonable range of clinical operation of appendectomy; after each 0.1mm position calculation, the system automatically checks whether the current calculation point falls within the hard boundary of the appendiceal lumen and corrects invalid trajectories that exceed the boundary.

9. The endoscopic appendectomy navigation system based on multimodal image fusion according to claim 1, characterized in that: The system also includes a dual-mode seamless switching unit: when the bleeding gradually subsides and the endoscopic field of view reveals enough effective feature points again, the system fixes the position of the last frame output of the micro-recursion mode as the reference anchor point, and completes the rapid rematch of visual features in a local range of millimeters around the anchor point. Through progressive pose calibration, it completes the zero-error seamless switching between the two modes of micro-recursion emergency navigation and conventional visual fusion navigation.

10. A navigation system for endoscopic appendectomy based on multimodal image fusion according to claim 9, characterized in that: The specific steps for seamless switching between the two modes with zero error are as follows: The system continuously performs pixel-level semantic recognition on the real-time endoscopic images at 10ms intervals, calls a pre-trained semantic segmentation model for appendiceal bleeding, and calculates the proportion of effective mucosal features in the images in real time. When there are ≥5 valid feature points of mucosal texture, blood vessels, and folds that are not covered by blood in 3 consecutive frames, and the distribution of these feature points covers at least 2 different quadrants of the image, the system determines that the bleeding occlusion state has been officially lifted and automatically starts the navigation mode switching process. Freeze all pose data of the last frame of the micro-recursive mode, and solidify the six types of parameters of the camera at this moment, including spatial three-dimensional coordinates, rotation angle, axial depth label, and motion speed, as the sole reference anchor point for visual rematch; at the same time, retrieve the feature coordinate set of all pre-annotated mucosal folds and microvessels within the axial depth ±1mm range of the last frame from the preoperative topological feature library to limit the search range of subsequent visual matching. The five or more newly exposed effective feature points in the current endoscopic image are quickly matched with the preoperative topological feature library within a limited range, and feature alignment is completed based on the cross-validation rule of folds and blood vessels. Automatically compare the new pose obtained from visual rematch with the reference anchor pose, and perform pose calibration based on different deviation thresholds and corresponding gradients. After calibration is completed, the system automatically exits the micro-push emergency mode and switches back to the regular real-time visual fusion navigation state, while simultaneously recording the data of bleeding occlusion in the background and storing it in the surgical log.

Images