Method for identifying and marking gallbladder triangle in laparoscopic surgery

Abstract

The invention relates to a gallbladder triangular identification and labeling method in a laparoscopic surgery, which comprises the following steps of: fusing a standard surgical field image with light source angle change data in a micro-operation process of an operator so as to enhance a tissue level separation contrast ratio; dynamic textures including gallbladder venation pulsation and liver surface breathing rhythm in the intraoperative image are extracted to serve as spatial reference marks, and anatomical boundary reduction is assisted; through time sequence image difference calculation, a regional saliency heat map is established frame by frame, so that the distinction degree of a gallbladder triangular region relative to surrounding tissues is displayed; an anatomical deformation track and a boundary transfer function are established, and the direction and angle of the gall bladder traction operation of the operator are associated with the local tissue deformation trend; identifying the variation trend of the included angle between the gallbladder duct and the common bile duct in the continuous frames, and reasoning the formation rule of the triangular closed boundary; the gallbladder triangle is divided into a plurality of structure segments including an included angle between a gallbladder duct and a common bile duct and a gallbladder artery intersection; and independently evaluating the identification confidence of each fragment.

Description

Technical Field

[0001] The present invention relates to a method for identifying and marking the Calot triangle, specifically a method for identifying and marking the Calot triangle in laparoscopic surgery. Background Art

[0002] Currently, the methods for identifying and marking the Calot triangle in laparoscopic surgery mainly include visual image enhancement, structural edge extraction, and manual experience-assisted recognition. Although basic anatomical assistance can be achieved in some routine cases, there are still many obvious deficiencies and drawbacks in complex surgical field scenarios and variant anatomical structures, which limit their universal application ability in high-complexity surgeries.

[0003] Firstly, most of the existing methods rely on the static feature recognition of single-frame images and often use traditional edge detection, color segmentation, or brightness contrast enhancement-based methods to assist surgeons in judging the boundaries of the Calot triangle structure. However, the images during laparoscopic surgery generally have characteristics such as occlusion, tissue adhesion, uneven illumination, and frequent changes in the surgeon's perspective, resulting in a significant decline in the recognition ability of these methods when the tissue boundaries are unclear or the background interference is strong, often leading to misrecognition or non-recognition problems. Especially when the gallbladder overlaps with the lower edge of the liver, the common bile duct, and the cystic artery, the edge features in a single-frame image are extremely likely to be lost or obscured by noise, thus affecting the overall structural judgment accuracy. Secondly, the existing methods lack the effective modeling ability for the intraoperative dynamic process and cannot utilize the natural evolution of tissue morphology and the influence of the surgeon's operation behavior on the structure exposure during the surgical process, resulting in the system's recognition relying entirely on the structure signals visible in the current image, while ignoring the critical stage of "the structure is not yet fully exposed but is taking shape", thus missing the possibility of early recognition and risk prediction. In addition, although some algorithms introduce deep learning models or structural segmentation networks to assist in recognition, most of these models are trained on static images or ideal datasets with perfect annotations and lack the learning ability for intraoperative real operation behaviors, camera jitter, and light source changes. Their recognition results are highly sensitive to image quality, with poor stability and insufficient robustness in dynamic scenarios.

[0004] Furthermore, most current systems are unable to perform dynamic updates and validations after the structural annotation is completed, lacking an effective boundary correction mechanism. During the operation, if the gallbladder changes its position due to traction or the viewing angle shifts, the original annotation cannot be adjusted in real time, which is extremely likely to cause the surgeon to mistakenly think that the structural boundary is fixed and result in incorrect cutting. Another common practice is to manually assist with lines or template prompts for the location of the Calot's triangle based on the surgeon's operating experience. However, this method highly depends on the individual experience of the surgeon, has strong operation subjectivity, and cannot adapt to individual differences and anatomical variations, especially posing great safety hazards in young surgeons or complex cases. Finally, existing recognition systems generally lack a closed-loop verification mechanism, that is, the recognition results cannot be effectively verified and self-corrected through the surgeon's behavior or tissue feedback. Once the recognition results deviate, the system cannot automatically prompt or adjust, and can only rely on the surgeon to discover it actively, which is extremely likely to result in recognition omissions or risk misguidance under the high-intensity operation pressure during the operation. Summary of the Invention

[0005] The object of the present invention is to provide a method for identifying and annotating the Calot's triangle in laparoscopic surgery, so as to solve some of the drawbacks and deficiencies pointed out in the background technology.

[0006] The following technical solutions adopted by the present invention to solve its above technical problems include the following steps:

[0007] S1. Adopt a mechanism for visualizing the surgical field structure by fusing multi-source visual information:

[0008] S1.1. Fuse the standard surgical field image with the data on the change of the light source angle during the micro-operation process of the surgeon to enhance the contrast of tissue layer separation;

[0009] S1.2. Extract the dynamic textures including the pulsation of the gallbladder choroid and the respiratory rhythm of the liver surface in the intraoperative image as spatial reference marks to assist in restoring the anatomical boundary; calculate the difference between sequential images, and establish a regional saliency heat map frame by frame to visualize the distinctiveness of the Calot's triangle area relative to the surrounding tissues;

[0010] S2. Adopt a decision model for the structural boundary of the Calot's triangle by morphological evolution reasoning:

[0011] S2.1. Establish an anatomical deformation trajectory and a boundary transfer function to associate the direction and angle of the surgeon's traction operation on the gallbladder with the local tissue deformation trend;

[0012] S2.2. Identify the change trend of the included angle between the cystic duct and the common bile duct in consecutive frames, and infer the formation law of the closed boundary of the triangle;

[0013] S3. Adopt a multi-level annotation strategy for the Calot's triangle driven by regional trust:

[0014] S3.1. Divide the Calot triangle into multiple structural segments including the angle between the cystic duct and the common bile duct and the confluence point of the cystic artery;

[0015] S3.2. Independently evaluate the recognition confidence of each segment to form a structural confidence map;

[0016] S3.3. The annotation system adjusts the color, border clarity or annotation shape according to the confidence level to make the credible area highlighted and the non-credible area weakly displayed during the operation;

[0017] S4. Adopt a fusion mechanism of intraoperative annotation intervention and structural closure criterion:

[0018] S4.1. Verify the effectiveness of the structural annotation through the spatial response relationship between the operator's operation behavior and the recognized structure;

[0019] S4.2. If the recognized structure fails to form a geometrically closed shape, a warning of unclosed structure is issued; if the operator further traction or electrocision leads to continuous exposure of the structure, re-inference and update of the annotation boundary are carried out.

[0020] Furthermore, the construction method of the surgical field structure visualization mechanism for multi-source visual information fusion includes:

[0021] Adopt an identification path based on the response behavior of micro-dynamic texture. Use the dynamic rhythms that are invisible during the operation but physiologically exist, including gallbladder pulsation and liver surface rhythm, as spatial anchors for anatomical structures to inversely infer the true boundary of tissues; and capture dynamic texture features, introduce a perturbed texture response function to sense sub-pixel-level tissue activities from continuous image frames; the function Ω is defined as follows:

[0022]

[0023] where x, y represent the two-dimensional pixel coordinates of a point in the image, and t represents the current time frame; I t (x, y) is the pixel gray value or color intensity of the point at time t; δ i is a pixel-level displacement vector used to simulate minute physiological jitters to perturb the original image spatially; I t+i (x + δ i , y + δ i ) represents the pixel value at the perturbed position in the future i-th frame; γ is a non-linear enhancement exponential parameter used to enhance the response value of minute texture changes; N is the window size of the number of frames participating in the comparison, used to control the time sensitivity.

[0024] Furthermore, the construction method of the surgical field structure visualization mechanism for multi-source visual information fusion includes:

[0025] Construct the micro-dynamic behavior into a time-series response feature map to manifest the persistence of organizational behavior; introduce a time-weighted regional saliency heat map construction function R(x, y), as follows:

[0026]

[0027] Among them, R(x, y) represents the dynamic saliency heat value of the pixel point (x, y) in the image; φ t (x, y) is the dynamic response intensity of the point at time t; is the rate of change of the response value with respect to time; β is the change suppression coefficient, used to enhance stable regions and suppress noise disturbances; the integration range [t0, t n represents the analysis time period.

[0028] Furthermore, the construction method of the surgical field structure manifestation mechanism for multi-source visual information fusion includes:

[0029] Adopt a reverse structure reconstruction mechanism based on the organizational behavior trajectory, and infer the boundary evolution trend of the invisible structure through the spatial distribution of the movement direction, frequency, and acceleration within continuous time; adopt a custom boundary inversion function Θ, as follows:

[0030]

[0031] Among them, Θ(x, y) represents the probability response value that the pixel point (x, y) is judged as the structure boundary; K is the number of tissue types considered, including the gallbladder, cystic duct, and common bile duct; Ψ k (x, y, t) represents the local motion vector field of the k-th type of tissue at time t; represents the motion gradient information of the tissue, the directionality and degree of change of the motion; μ k (x, y) is the spatial coupling weight of the point with the k-th type of tissue, representing the similarity or correlation degree; ∈ is the perturbation scale for the theoretical limit, and α is the boundary sensitivity coefficient, used to adjust the response sensitivity of the system to boundary blurring or gradual change.

[0032] Furthermore, the construction method of the gallbladder triangle structure boundary determination model for morphological evolution inference:

[0033] By introducing a structure modeling mechanism driven by the operator's behavior, adopt an identification mechanism centered on operation vector modeling, angle evolution inference, and boundary dynamic correction; construct a spatial mapping model between the operator's operation behavior and the tissue response; by recording the force application direction and amplitude during the operator's operation, and combining the response deformation trajectory of the local tissue in the image, enable the quantitative expression of action → structure change; introduce a response mapping function expression:

[0034]

[0035] Among them, Λ(x, y, t) represents the intensity of the tissue deformation effect experienced by the pixel position (x, y) in the image at time t; is the pulling action vector of the operator at this moment; represents the displacement response vector of this pixel point at time t, reflecting the actual deformation trend of the tissue in this area; is the gradient of the deformation vector in the image space; the symbol represents the coupling tensor mapping operation between behavior and structure, and is used to calculate the actual influence degree of the operator's operation on the tissue morphology change.

[0036] Further, the method for constructing the boundary determination model of the Calot triangle structure for morphological evolution inference:

[0037] Based on the judgment of whether the Calot triangle structure enters the formed state, taking the angle between the cystic duct and the common bile duct as the key structural parameter, continuously tracking the change trend of this angle in the time dimension to judge whether the triangular area is spatially closed; for this purpose, an angle evolution trend function is constructed:

[0038]

[0039] Among them, η(t) is the angle evolution index at time t, which is a trend index for judging the closing degree of the triangular area; θ(s) represents the actual angle between the cystic duct and the common bile duct observed at any time s; τ is the length of the historical analysis time window, which is used to control the time scale of the trend judgment; κ is the sensitivity parameter, which determines the amplification response degree of the system to the initial change of the angle.

[0040] Further, the method for constructing the boundary determination model of the Calot triangle structure for morphological evolution inference:

[0041] Introduce a boundary transfer function based on time series tracking, which is used to dynamically adjust the position and contour of the annotation area; taking the displacement speed of the edge trajectory and the change of the edge sharpness in the image as the input, construct as follows:

[0042]

[0043] Among them, Δ(x, y, t) represents the boundary adjustment intensity of the pixel point (x, y) in the image at time t; ξ(x, y, t) is the function of the spatial coordinates of the point on the boundary line changing with time, which is used to measure its displacement speed; represents the real-time drift speed of the boundary point; ζ(x, y, s) is the edge sharpness index of this point at time point s; t0 represents the starting frame time; the whole function is used to dynamically correct the position of the boundary point. When the boundary becomes blurred, drifts or is occluded, compensation is made according to the historical sharpness change trajectory to make the annotation line always fit the anatomical structure.

[0044] The method for identifying and marking the Calot triangle in laparoscopic surgery provided by the present invention has the following remarkable beneficial effects:

[0045] First, by introducing a multi-source visual information fusion mechanism, combining intraoperative light source changes, micro-dynamic texture responses (such as gallbladder pulsation, liver surface rhythm), and time-series heat map analysis, the visualization ability of anatomical structures under complex operative field conditions is significantly enhanced, enabling the accurate identification of potential areas of the Calot triangle even in the case of tissue occlusion, adhesion, or bleeding.

[0046] Second, a morphological evolution reasoning model is constructed, innovatively establishing a mapping relationship between the operator's operation behavior vector and the tissue response deformation trajectory in the image, and using the structural deformation and included angle evolution trend reasoning to realize the real-time judgment of the structural formation state, so that the identification no longer depends on the completely exposed boundary, improving the predictability and intraoperative adaptability of the system.

[0047] Third, a boundary dynamic update mechanism is introduced, constructing a boundary transfer function based on the edge trajectory speed and clarity changes, which can achieve high-response and low-latency adaptive correction of the boundary annotation under the conditions of structural drift or occlusion, and keep the annotation area always highly consistent with the real anatomical structure. Description of the Drawings

[0048] Figure 1 It is a flow chart of the method for identifying and marking the Calot triangle in laparoscopic surgery of the present invention.

[0049] Figure 2 It is a flow chart of the method for constructing the operative field structure visualization mechanism of multi-source visual information fusion of the present invention.

[0050] Figure 3 It is a flow chart of the method for constructing the boundary determination model of the Calot triangle structure by morphological evolution reasoning of the present invention. Detailed Embodiments

[0051] The following will give a detailed description of the specific embodiments of the present invention with reference to the drawings.

[0052] By integrating the image information of the surgical field with the physical change data related to the operator's operation behavior, a multi-dimensional dynamic analysis mechanism is established to improve the accuracy and stability of anatomical structure recognition. First, in the intraoperative image processing stage, not only the standard surgical field images obtained by traditional high-definition laparoscopes are used, but also the changes in the light source angle caused by the operator during the operation are introduced as additional dynamic parameters for analysis. Since the subtle changes in the light source angle will produce reflection differences and shadow transitions on the organ surface, these changes can be used to enhance the brightness contrast and layering between tissues through image superposition processing. Especially between the loose connective tissues and duct structures in the hepatobiliary area, whose light reflection characteristics are greatly affected by the light angle, this integration method can significantly improve the separation degree of the Calot triangle area from the surrounding tissues in the image and enhance the basis for boundary recognition.

[0053] On this basis, the system further introduces the concept of dynamic texture, and carefully extracts the physiological rhythm signals shown in the intraoperative image frame sequence, such as the local periodic texture fluctuations brought by the pulsation of the fine veins on the gallbladder wall, or the slow undulating movement of the liver surface due to breathing. These micro-changes in consecutive frames are usually not easily recognizable by the naked eye, but through means such as texture tracking, pixel displacement estimation, and optical flow analysis, stable dynamic texture patterns can be extracted. Since these physiological activities are highly related to specific anatomical structures, they can be regarded as "spatial reference marks" to assist the system in judging the true position distribution of tissue boundaries under visual blurring or occlusion conditions, thereby improving the boundary restoration ability. Finally, the system analyzes the dynamic differences in each region of the image sequence frame by frame, establishes a temporal saliency response model, assigns saliency weights to the tissue regions in each frame of the image, and constructs a saliency heat map that evolves over time. This heat map can automatically highlight those regions that maintain stable dynamic behavior over time or are most affected by surgical operations. For example, the region continuously exposed due to traction at the junction of the cystic duct and the common bile duct will show higher brightness or color contrast in the heat map, thus clearly distinguishing the Calot triangle from the relatively static or non-rhythmic tissues around it.

[0054] A dynamic response relationship is established between the operator's operation behavior and tissue morphological changes, enabling structural recognition to not only rely on the visual results in the image, but also understand and utilize the deformation process caused by intraoperative operations for reasoning and judgment. In actual operation, the actions of the operator to pull, abduct, or lift the gallbladder will be applied to the tissue surface through instruments such as clamping or probing, bringing about relative displacement, rotation, or deformation of the tissues around the gallbladder, especially between the cystic duct, common bile duct, and cystic artery. This method first quantifies and models these operation directions and amplitudes to obtain the operator's force vector, and then combines the real-time displacement response of tissue points in the image to generate an anatomical deformation trajectory that evolves over time. This trajectory not only reflects the dynamic movement pattern of the anatomical structure in three-dimensional space, but also records the evolution process of its boundary line during the surgical operation, forming a mapping channel that reflects "structure deformation guided by actions".

[0055] On this basis, in order to judge whether the hepatoduodenal triangle has formed an anatomical closed state, the system further analyzes the change trend of the angle between the cystic duct and the common bile duct in consecutive image frames. This angle is not constant during the surgical process, but gradually opens or closes with the pulling operation, and the rate of change and stability of its angle can be used to judge whether the structure is in the "shaping" stage. When the system recognizes that the angle has reached a certain stable closed state or is close to the critical value, and there is an obvious stable trend in the edge deformation trajectory, it can be inferred that the three sides of the hepatoduodenal triangle have been completed, thus triggering the annotation module to perform structure highlighting or navigation prompts. During the whole process, the introduction of the boundary transfer function is used to track and dynamically correct the position of the tissue edge in the image to ensure continuous adjustment with the pulling angle and tissue deformation.

[0056] By performing fine-grained structural division and confidence expression on the Calot triangle region, intraoperative intelligent auxiliary prompts with greater hierarchy and credibility are achieved. First, instead of identifying the Calot triangle as a whole unified structure, this method refines it into multiple key segments with anatomical significance, such as the angle region between the cystic duct and the common bile duct, the intersection point of the cystic artery and the cystic duct, and the adjacent area between the cystic duct and the liver boundary. This segmentation method enables the system to independently analyze and identify each structural unit, thereby improving robustness in complex, occluded, or morphologically variant surgical scenarios. After the division is completed, the system evaluates the recognition confidence of each segment. This process is based on multi-dimensional indicators such as image clarity, dynamic texture consistency, and structural deformation stability to assess whether the segment in the current frame image has sufficient anatomical identifiability, thus constructing a spatial structure confidence map. Each region of this map represents the degree to which it is "seen clearly" by the system during the operation. Next, the system uses the confidence map results to guide the visualization annotation expression. By adjusting the color depth, border sharpness, line type style, or transparency of the annotation graphics, the high-confidence regions are displayed in a higher brightness and clear boundary form, while the low-confidence regions adopt weak display strategies such as blurring, semi-transparency, or fuzzy boundaries, enabling the surgeon to clearly identify at a glance during the operation which anatomical relationships in which regions are highly confirmed by the system and which parts still require careful discrimination or manual intervention.

[0057] By introducing the spatial response relationship between the operator's operation behavior and the system recognition structure, a real-time interactive feedback mechanism is established to dynamically verify the effectiveness and rationality of the structure annotation. Specifically, after the system identifies the key structures of the Calot triangle, instead of directly fixing the annotation result, it continuously monitors the operator's operation behavior, such as the direction of clamping and pulling, the change in force, the path of instrument pushing, or the electrocision action, etc. At the same time, it observes in real time whether these operations produce the expected spatial response to the recognized structure area in the image. For example, whether the angle between the cystic duct and the common bile duct further opens with the pulling, or whether the previously occluded cystic artery is gradually exposed due to electrocision. If the recognized structure area cannot form a complete geometric closed shape in multiple frames of images, or the boundary of the triangle area does not meet the requirements of anatomical continuity, clarity, and boundary intersection characteristics, the system will automatically judge that the structure annotation is not closed and issue an intraoperative prompt or warning of "structure not closed" to guide the operator to further operate to promote the clear exposure of the structure. At this time, if the operator continues to perform operations such as abduction traction, rotation of the gallbladder, or fine dissection, resulting in the gradual exposure of local tissues and the formation of a stable structural boundary continuity, the system will re-run the structure recognition algorithm based on the newly exposed area, compare the previously marked boundary with the current newly obtained image information, and perform boundary inference and update, so as to optimize and correct the marked area in real time, ensuring that the finally presented Calot triangle annotation not only conforms to the anatomical logic but also dynamically responds to the actual situation during the operation, forming a closed-loop recognition and annotation mechanism of "recognition-verification-feedback-update", greatly improving the practicability, safety, and reliability of the system in complex surgical fields.

[0058] Example 1:

[0059] During the laparoscopic cholecystectomy procedure, especially in the crucial step of identifying and protecting the Calot triangle area, the operator often faces complex situations such as visual occlusion, anatomical variations, or tissue adhesions. Traditional recognition methods relying on static edges or color contrast are difficult to provide reliable support. Therefore, the proposed mechanism for visualizing the surgical field structure by fusing multi-source visual information provides a solution idea. The core is to identify the micro-dynamic texture behaviors that, although invisible in the image, actually exist physiologically, such as the pulsation of the gallbladder or the rhythmic movement of the liver surface with breathing. These tissue movement patterns with periodicity and local regularity are used as spatial anchor points for identifying the Calot triangle, and a reverse inference path for tissue boundaries is constructed based on this. The specific implementation of this process depends on a computational model called the perturbed texture response function, and its form is:

[0060]

[0061] where x and y are the coordinates of the target pixel point, t is the current time frame, and I t (x,y) represents the original image gray value of this pixel point at time t, and δ iIt is a tiny pixel displacement that simulates physiological jitter in space, usually ranging from [-1.0, +1.0] pixels. t+i (x+δ i ,y+δ i ) is the pixel value of the disturbed position of the point in the future i-th frame, N is the frame window size, which is used to adjust the temporal analysis depth. The general value range is 3≤N≤10, and γ is the response enhancement coefficient, which controls the sensitivity to slight changes in texture. In practical applications, it is recommended to set it to 1.2≤γ≤2.5. When γ=1, the response is close to linear. When γ>2, the system tends to highlight very subtle but continuous changes.

[0062] To verify the feasibility of this method, we use the following surgical example to illustrate: The patient, Mr. Li, a 42-year-old male, was diagnosed with chronic cholecystitis before surgery, accompanied by mild adhesion between the gallbladder and the lower edge of the liver. Before gallbladder dissection during surgery, the surgeon placed the clamp on the gallbladder body and gently lifted it up. However, because the local tissue adhesion was not completely dissected, the boundary of the gallbladder triangle area was difficult to directly reveal in the surgical field image. At this time, the traditional image recognition method failed to judge due to the blurred boundary;

[0063] The system automatically starts the dynamic texture tracking mechanism, records the surgical field image sequence at a rate of 30 frames per second, calculates the texture response of the gallbladder surrounding area, sets the frame window number N = 6, and the nonlinear enhancement parameter γ = 2.0. At the same time, the perturbation response function calculation is performed for multiple points. A reference pixel point (x = 120, y = 96) located in the expected area of ​​the gallbladder duct is selected, and the perturbation vector δ is superimposed in 6 consecutive frames. i ;

[0064] Take (0.5, 0.5), (-0.3, 0.2), (0.1, -0.4), (-0.6, -0.6), (0.3, 0.1), (-0.2, 0.4) respectively. The system automatically reads the pixel value at each disturbance position and compares it with the initial frame pixel intensity I t By comparing (x, y) = 126, the disturbance response of each frame is [4.2, 3.1, 2.7, 4.6, 3.3, 2.9], and the absolute value is taken and then raised to the square. The cumulative average is Ω(120, 96, t) ≈ 10.8. After dynamic threshold comparison, it is found that the response value is about 3 times higher than the average value of the static tissue area. The system automatically determines that there is regular texture beating in this area and judges it as a gallbladder pulsation signal. Then, a spatial reference anchor point is constructed for this point and the surrounding 5x5 pixel range. In subsequent images, the system uses the dynamic behavior characteristics of this area to continuously track the boundary, and infers the angle between the cystic duct and the common bile duct. Finally, it realizes the accurate marking of the boundary of the gallbladder triangle without fully exposing the anatomical structure. After completing the anatomical traction and exposure, the overlap between the system's marked area and the true anatomical boundary reaches 92%.

[0065] Construct a temporal response feature map and generate a time-weighted saliency heat map R(x, y). The core objective is to distinguish, in the time dimension, regions with stable activities and clear anatomical significance from misidentified regions caused by short-term sporadic noise or tissue occlusion. The specific approach is to first track the dynamic response intensity of each pixel point in consecutive image frames, denoted as and calculate the dynamic response change rate by analyzing its change trend. Then, fuse the response intensity with its change rate to form a time-weighted response curve, and finally construct the following saliency function:

[0066]

[0067] where Rx,y is the dynamic saliency heat of a certain pixel point in the time period t0, t n The dynamic saliency heat within, represents the dynamic response value of this point at time t, and its value range is usually between 0 and 255. is its time change rate, reflecting the fluctuation speed of the response value over time. β is the change suppression factor, and its selectable range in the actual system is from 1.0 to 3.0. A larger β value will more strongly suppress instantaneous changes and emphasize response stability;

[0068] To achieve the calculation, the system acquires images at 30 frames per second and analyzes a 60-frame sequence within the past 2 seconds. Set the integration interval t0 = 0, t n = 2 seconds, and select a typical point x = 122, y = 98 in the Calot triangle area. Its response value shows a slight periodic fluctuation within 2 seconds. The system records the sequence of φ value changes of this point in 60 frames as 128, 130, 127, 129, 131, 126,... Calculate its average response as 128.3 and the average change rate dφ / dt as 0.45. Substitute into the formula and set β = 2.0 to obtain the weighted response per frame as Integrate and sum this value on the 60-frame time axis. Finally, the Rx,y of this pixel point ≈ 38.8 × 2 = 77.6 (in unit of normalized heat value);

[0069] The system performs the same operation on the entire image area and maps all Rx,y results into a heat map form. Finally, in the surgeon's main view, those tissue areas with stable dynamic textures within the Calot triangle area are highlighted in the form of brightness colors, significantly prominent compared to the background tissue. This heat map also superimposes a transparency attenuation mechanism to downweight low-response areas, enabling the surgeon to immediately identify which areas are the highly confident structural boundaries of the system and which are still in a dynamically unstable or occluded state. Through this mechanism, the intraoperative visual guidance efficiency is significantly improved. After postoperative verification, the heat map highlighted areas and the postoperative anatomy Figure 1The hit rate reaches 89.4%, indicating that this significance mechanism not only has good visualization effects.

[0070] This embodiment further enables the third stage in the multi-source visual information fusion mechanism: the reverse structure reconstruction mechanism based on tissue behavior trajectories. By tracking the spatial evolution trends of the tissue movement directions, frequencies, and accelerations within continuous time, the true positions and deformation trends of the structural boundaries are deduced under invisible conditions. Specifically, a custom boundary inversion function Θ(x,y) is used to model the boundary response values, and its definition is as follows:

[0071]

[0072] Among them, Θ(x,y) represents the probability response value that the pixel point (x,y) in the image is determined as the structural boundary. K represents the number of tissue categories analyzed by the system simultaneously. In this example, K = 3 is set, which are the gallbladder (k = 1), cystic duct (k = 2), and common bile duct (k = 3); Ψ k (x,y,t) is the local motion vector field of the k-th type of tissue at time t, recording the movement direction and amplitude of each pixel caused by traction or instrument pushing in consecutive frames. Its gradient is used to measure the motion mutation trend of this tissue per unit time; μ k (x,y) represents the spatial coupling degree between the pixel point (x,y) and the k-th type of tissue, which is calculated through the structural prior map or deep learning feature embedding in the system, reflecting the morphological association strength between this point and the target tissue; ε is the perturbation scale, simulating the influence of tiny perturbations near the boundary on the response in the simulation inference. In actual calculations, its value ranges from 0.01 to 0.05; α is the boundary sensitivity coefficient, used to control the reaction sensitivity of the system to fuzzy edges, and its range is usually between 1.5 and 3.0. A larger value is used to identify gradually changing transitional boundaries, and a smaller value is suitable for abrupt boundaries; in this example, α = 2.0 is selected.

[0073] In practical applications, the system extracts the dynamic response fields Ψ1, Ψ2, Ψ3 within the gallbladder region from the intraoperative image sequence, establishes a time vector sequence of 30 frames for each type of tissue respectively, calculates the motion gradient values of each pixel point in the 3 types of tissues. Assuming at the pixel position x = 125, y = 102, the local vector field gradient of the cystic duct is The coupling weight μ2 = 0.82. Substituting it into the formula and setting ε = 0.02, α = 2.0, then the boundary contribution of the cystic duct to this point is 0.46 × 0.82 × (0.02) 2 = 0.00015088; similarly, the contribution of the common bile duct to this point is 0.39 × 0.79 × (0.02) 2 = 0.00012324, and the contribution of the gallbladder tissue is 0.28 × 0.74 × (0.02) 2= 0.00008288, and the sum is Θ(125, 102) ≈ 0.000357. After being normalized by the boundary response, this value is higher than the system boundary recognition threshold of 0.0003. Therefore, the system marks this pixel point as a structural edge point and continues the diffusion calculation near the edge line of this area to form a continuous speculative boundary line. Combining the previous texture anchor points and heat map response information, the closed-loop boundary reconstruction of the Calot triangle is finally completed. After being verified by postoperative pathology and video playback, the overlap rate between the reconstructed boundary of this structure and the real anatomical structure reaches 91.7%, which proves that even under the influence of visual occlusion or adhesion, this method can still perform boundary inversion on the invisible area through tissue behavior trajectories and mathematical modeling, providing stronger structural recognition support and risk prevention capabilities during clinical surgery, and reflecting the practical operability and clinical value of the multi-source fusion and dynamic response mechanism in the laparoscopic intelligent recognition system.

[0074] Example 2:

[0075] Continuing the story of Mr. Li's laparoscopic cholecystectomy in Example 1, the system has previously gradually established a dynamic recognition foundation for the Calot triangle area through the micro-dynamic texture response Ω, the temporal heat map Rx,y, and the boundary inversion function Θx,y. However, due to the adhesion at the lower pole of the gallbladder during the operation, the surgeon had to repeatedly pull and adjust the gallbladder body. Especially when trying to expose the junction relationship between the cystic duct and the common bile duct, the traditional visual recognition method was difficult to stably obtain boundary information due to severe tissue deformation. Therefore, the system further enabled the morphological evolution reasoning mechanism in the present invention to achieve structural dynamic recognition through the structural modeling method driven by the surgeon's behavior, that is, introducing the spatial mapping relationship between the operation behavior vector and the tissue response in the surgical image recognition process, thereby establishing a structural change reasoning model guided by actions. Specifically, the response mapping function Λ(x, y, t) is used for modeling, and the expression is as follows:

[0076]

[0077] Where Λx,y,t represents the intensity of the tissue deformation effect caused by the surgeon's operation on the pixel point x, y in the image at time t, is the pulling action vector executed by the surgeon at this time point, including the direction component (such as vertical upward traction, right front abduction, etc.) and the amplitude component (recorded by the intraoperative instrument force sensor, unit: N); is the tissue displacement response vector of this point, and its tissue pixel displacement path can be obtained by calculating the optical flow field of consecutive image frames, is the gradient of this displacement vector in the image space, used to describe the deformation rate and the degree of directional mutation in the surrounding area of this point, and the symbol represents the tensor mapping between the operation behavior and the tissue response, used to quantify the driving effect of the surgeon on the formation or exposure of the structure;

[0078] In actual operation, the operator's operation of pulling the gallbladder upward and outward lasts for about 1.2 seconds, and the system records the vector average value is 3.2, 1.5 N, indicating that the main pulling direction is 30 degrees to the upper right, and the force strength is about 3.5 N; at the same time, in the image frame, the key pixel points x = 118, y = 107 in the speculated area of the cystic duct are selected, and the system obtains the continuous frame displacement of this point as 1.4, 0.6 pixels through optical flow estimation, forming a response vector Its gradient is estimated by convolution to be about 1.85 and substituted into the formula for calculation (the unit is the normalized deformation driving force strength), the system sets the influence strength threshold to 5.0, and if it exceeds this value, it is considered that the operator's current operation has successfully caused the structural deformation and response in this area, and cross-validation is carried out in combination with the previous heat map Rx,y and the boundary response Θx,y. If high responses are obtained in the same area, it is determined that the structure has entered the active state of morphological evolution.

[0079] Subsequently, the system activates the boundary dynamic correction module, uses the deformation trend of this area as the main direction of structural evolution, fine-tunes and matches the preliminary structural boundary line, and finally forms an updated cystic triangle contour that changes in real time with the operator's operation. It is found in the postoperative review that the overlap rate of this contour with the postoperative sectional anatomical structure is 93.2%, which is significantly better than 86.5% of the static image recognition model alone, thus verifying that this morphological evolution inference model introduces the "operator's action" into the "image recognition" process.

[0080] The key step in the next judgment of whether the cystic triangle structure has completed the closed boundary in this embodiment is to use the included angle between the cystic duct and the common bile duct as the key dynamic structure parameter. The system continuously monitors the change trend of this included angle in the time dimension through visual tracking and inter-frame analysis of the image, and combines the real-time data to judge whether the triangular area has reached the anatomically recognizable closed state. For this purpose, the system constructs an included angle evolution trend function η(t), which is defined as follows:

[0081]

[0082] where η(t) represents the quantization index of the closing trend of the triangular area at time t, θs is the included angle between the cystic duct and the common bile duct observed by the system at time s, the unit is radian, cosθ(s) can reflect the structural convergence trend, τ is the time window length, which controls the influence range of the historical angle change trend, usually set τ between 1.0 and 3.0 seconds, and the actual value depends on the surgical rhythm. In this example, τ = 2.0 seconds; κ is the change sensitivity parameter, which determines the amplification degree of the system to the initial convergence change of the included angle. The recommended value range is 1.0 to 4.0, and κ = 2.5 is taken under the operating conditions without violent disturbance.

[0083] In a specific surgical scenario, the surgeon continuously abducts and lifts the gallbladder to the right and outward by clamping, expecting to expose the structures within the Calot triangle. The system automatically annotates the central axes of the cystic duct and the common bile duct from the intraoperative images, extracts the included angle θs in images at 30 frames per second, and records the θ values in 60 frames of the past 2 seconds as: 87°, 84°, 81°, 79°, 77°, 75°, 74°, 73.5°, 73.2°... gradually stabilizing and closing frame by frame. Convert it to radians and calculate cosθs. The corresponding cos values are 0.0523, 0.1045, 0.1564, 0.1908, 0.2249, 0.2588, 0.2756, 0.2837, 0.2864 respectively. Substitute them into the formula for integral approximation estimation. The system calculates [1 - cosθ(s)] κ Evaluate and accumulate the values frame by frame and then take the average to obtain η(t)≈0.0478. According to the system-set threshold η c rit = 0.04 (exceeding this value is regarded as the formation of the triangle shape). The system determines that the current triangle structure has formed a spatial geometric closed relationship, and highlights and outlines the triangle boundary in the visual guidance interface and locks the structure to prevent accidental resection of the bile duct area.

[0084] Finally, under the system prompt, the surgeon successfully completed the separation of the gallbladder and the dissection of the triangle. The postoperative playback data shows that the system's early recognition time for the prediction of the included angle change is 1.6 seconds, that is, before the structure is fully displayed, the system can predict the possibility of its closure based on the evolution trend of the included angle, providing an intelligent assistance mechanism for the surgeon to make an earlier intervention judgment, and also improving the forward-looking and accuracy of recognition as a whole, verifying that the included angle evolution trend function η(t) designed in the present invention has strong adaptability, computability and predictability in the dynamic intraoperative scenario.

[0085] When the laparoscopic cholecystectomy of patient Mr. Li continued to progress to a critical stage, the system had already determined through the included angle evolution trend function η(t) that the Calot triangle was in a formed state, and highlighted the boundary of the preliminarily annotated structural area. However, since the surgeon needed to further pull the neck of the gallbladder during the subsequent separation process, and this action might cause slight drifts in the relative positions of the cystic duct, the common bile duct and the cystic artery, resulting in a deviation between the original boundary annotation and the actual tissue edge. Therefore, the system activates the third step of the morphological evolution reasoning model in the present invention: the boundary time-series tracking and dynamic correction mechanism, that is, the annotation area is adjusted in real time through the boundary transfer function Δx,y,t to maintain a high degree of fit with the actual anatomical boundary. This function takes the real-time displacement speed of the edge trajectory and the evolution process of the edge clarity in the image over time as inputs, and the form is as follows:

[0086]

[0087] Where Δ(x, y, t) is the boundary adjustment intensity of the pixel point (x, y) at time t, reflecting the necessary degree of the current system to correct the position of this point. ξ(x, y, t) is the trajectory function of the spatial position of this pixel point on the boundary line changing with time, and dξ / dt is its real-time velocity component, which depends on the displacement degree of this boundary point in consecutive frames. ζx,y,s is the edge sharpness index of this point in the image at time s, which can be calculated from local gray-scale gradients, edge responses, or Canny edge detection response values, reflecting whether this point maintains good structural visibility in the image. t0 is the starting frame of tracking, generally set as the starting time of the current stage of operation. In actual operation, the system sets the value range of ζ to 0, 1, where 0 represents complete blur and 1 represents the state with the clearest edge. The larger the output value of Δ, the more strongly the system needs to adjust this boundary point to fit the current anatomical reality.

[0088] Taking the intraoperative time point t = 12.4 seconds as an example, the surgeon performs a further abduction and traction action on the lower right of the gallbladder body, causing the edge structure of the cystic duct to drift about 2.6 pixels in the image. The system dynamically tracks the key edge points x = 116, y = 104, and gets dξ(x, y, t) / dt ≈ 2.6 pixels / s. At the same time, among the 72 frames from the starting frame t0 = 10.0 seconds to t = 12.4 seconds, the values of ζ(x, y, s) are obtained through the edge detection module, and the average value is 0.63. The integral is approximately ζ integral value ≈ 0.63×2.4 ≈ 1.512. Then, substituting into the calculation, we get Δ116,104,12.4 ≈ 2.6×1 + 1.512 ≈ 2.6×2.512 ≈ 6.531, which exceeds the set value of the system boundary drift correction threshold of 3.5. Therefore, the system immediately updates the boundary position of this point;

[0089] The boundary line is pushed about 2.1 pixels in the new response direction of this point, and at the same time, local interpolation is performed with reference to the Δ values of adjacent points to make the whole boundary line show fluidity and smooth adjustment without breaks or jumps. After continuous operation and correction, the intraoperative boundary line can continuously move with the anatomical structure and dynamically fit the tissue contour. Finally, after the complete separation of the Calot triangle, the system and the surgeon confirm that the coincidence rate of the automatically marked boundary and the actual tissue boundary reaches 94.3%, the boundary update time delay is less than 100 milliseconds, and no recognition interruption or error amplification occurs, fully verifying that the real-time correction mechanism based on the boundary transfer function Δx,y,t has high responsiveness, boundary fitting, and recognition stability, effectively solving the problem that the traditional static boundary model cannot adapt when drift occurs during intraoperative operation, and reflecting the core innovation ability of the present invention to maintain the continuous accuracy of anatomical structure recognition in the dynamic intraoperative environment.

[0090] The foregoing has shown and described the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments, and what is described in the above embodiments and the specification is only to illustrate the principle of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements fall within the scope of the present invention claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for identifying and marking the Calot triangle in laparoscopic surgery, characterized in that Including the following steps: S1. Adopt the operative field structure visualization mechanism based on multi-source visual information fusion: S1.

1. Fuse the standard operative field image with the data of the light source angle change during the micro-operation of the operator to enhance the contrast of tissue layer separation; S1.

2. Extract the dynamic textures including the gallbladder choroid pulsation and the liver surface respiration rhythm in the intraoperative image as spatial reference marks to assist in restoring the anatomical boundary; Calculate the difference between sequential images, and establish a regional saliency heat map frame by frame to visualize the distinctiveness of the Calot triangle region relative to the surrounding tissues; S2. Adopt the Calot triangle structure boundary determination model based on morphological evolution reasoning: S2.

1. Establish the anatomical deformation trajectory and the boundary transfer function, and correlate the direction and angle of the operator's pulling operation on the gallbladder with the local tissue deformation trend; S2.

2. Identify the change trend of the angle between the cystic duct and the common bile duct in consecutive frames, and infer the formation law of the closed boundary of the triangle; S3. Adopt the multi-level annotation strategy of the Calot triangle driven by regional confidence: S3.

1. Divide the Calot triangle into multiple structural segments including the angle between the cystic duct and the common bile duct and the confluence point of the cystic artery; S3.

2. Independently evaluate the recognition confidence of each segment to form a structural confidence map; S3.

3. The annotation system adjusts the color, border clarity or annotation shape according to the confidence level, and adopts the intraoperative expression strategy of highlighting the credible area and weakly displaying the non-credible area; S4. Adopt the mechanism of fusing intraoperative annotation intervention and structural closure criterion: S4.

1. Verify the effectiveness of the structural annotation through the spatial response relationship between the operator's operation behavior and the recognized structure; S4.

2. If the recognized structure fails to form a geometric closed shape, a warning of unclosed structure is issued; If the operator further pulls or electrocuts, resulting in the continuous exposure of the structure, the annotation boundary is re-inferred and updated.

2. The method for identifying and marking the Calot triangle in laparoscopic surgery according to claim 1, wherein The construction method of the operative field structure visualization mechanism based on multi-source visual information fusion includes: Adopt the recognition path based on the micro-dynamic texture response behavior, and use the dynamic rhythms including gallbladder pulsation and liver surface rhythm that are invisible during the operation but physiologically present as spatial anchor points of anatomical structures to infer the true boundary of tissues in reverse; Capture the dynamic texture features and introduce the perturbation texture response function to perceive the sub-pixel level tissue activities from consecutive image frames.

3. The method for identifying and marking the Calot triangle in laparoscopic surgery according to claim 2, wherein The construction method of the operative field structure visualization mechanism based on multi-source visual information fusion includes: Construct the micro-dynamic behavior into a sequential response feature map to visualize the persistence of tissue behavior; Introduce the time-weighted regional saliency heat map construction function R(x, y), as follows: Among them, R(x, y) represents the dynamic saliency heat value of the pixel point (x, y) in the image; φ t (x, y) is the dynamic response intensity of the point at time t; is the change rate of the response value with respect to time; β is the change suppression coefficient, which is used to enhance the stable region and suppress the noise disturbance; the integration range is [t0, t n represents the analysis time period.

4. The method for identifying and marking the Calot triangle in laparoscopic surgery according to claim 3, wherein The construction method of the operative field structure visualization mechanism based on multi-source visual information fusion includes: Adopt the reverse structure reconstruction mechanism based on the tissue behavior trajectory, and speculate the boundary evolution trend of the invisible structure through the spatial distribution of the movement direction, frequency and acceleration within continuous time; Adopt the custom boundary inversion function Θ, as follows: Among them, Θ(x, y) represents the probability response value that the pixel point (x, y) is judged as a structural boundary; K is the number of tissue types considered, including the gallbladder, cystic duct, and common bile duct; Ψ k (x, y, t) represents the local motion vector field of the k-th type of tissue at time t; represents the motion gradient information of the tissue, the directionality and degree of change of the motion; μ k (x, y) is the spatial coupling weight of the point with the k-th type of tissue, representing the similarity or correlation degree; ∈ is the perturbation scale for the theoretical limit, and α is the boundary sensitivity coefficient, which is used to adjust the response sensitivity of the system to boundary blur or gradual change.

5. The method for identifying and marking the Calot triangle in laparoscopic surgery according to claim 1, characterized in that The construction method of the Calot triangle structure boundary determination model based on morphological evolution reasoning: By introducing a surgeon behavior-driven structural modeling mechanism, an identification mechanism centered on operation vector modeling, angle evolution reasoning, and boundary dynamic correction is adopted; a spatial mapping model between the surgeon's operation behavior and tissue response is constructed; by recording the force application direction and amplitude during the surgeon's operation and combining with the response deformation trajectory of local tissues in the image, a quantitative expression of action → structural change is achieved.

6. The method for identifying and marking the Calot triangle in laparoscopic surgery according to claim 5, characterized in that The method for constructing a boundary determination model of the Calot triangle structure based on morphological evolution reasoning: Based on the judgment of whether the Calot triangle structure enters the formed state, with the angle between the cystic duct and the common bile duct as the key structural parameter, continuously track the change trend of this angle in the time dimension to determine whether the triangular area is spatially closed; For this purpose, an angle evolution trend function is constructed: Among them, η(t) is the angle evolution index at time t, which is a trend index for judging the closing degree of the triangular area; θ(s) represents the actual angle between the cystic duct and the common bile duct observed at any time s; τ is the length of the historical analysis time window, which is used to control the time scale of trend judgment; κ is the sensitivity parameter, which determines the amplification response degree of the system to the initial change of the angle.

7. The method for identifying and marking the Calot triangle in laparoscopic surgery according to claim 1, characterized in that The method for constructing a boundary determination model of the Calot triangle structure based on morphological evolution reasoning: Introduce a boundary transfer function based on time series tracking, which is used to dynamically adjust the position and contour of the annotation area; with the displacement speed of the edge trajectory and the change of edge clarity in the image as the input, construct as follows: Among them, Δ(x, y, t) represents the boundary adjustment intensity of the pixel point (x, y) in the image at time t; ξ(x, y, t) is the function of the spatial coordinates of the point on the boundary line changing with time, which is used to measure its displacement speed; represents the real-time drift speed of the boundary point; ζ(x, y, s) is the edge sharpness index of the point at time point s; t0 represents the starting frame time; the whole function is used to dynamically correct the position of the boundary point. When the boundary is blurred, drifted or occluded, compensation is made according to the historical sharpness change trajectory, so that the annotation line always fits the anatomical structure.

Images