Intraoperative image fusion and dynamic tissue recognition processing method

Abstract

The present application relates to intraoperative image fusion and dynamic tissue recognition processing method, comprising: acquiring continuous intraoperative image frames, dividing the continuous intraoperative image frames into multiple image regions, extracting the boundary features, texture features and motion features of the image regions in adjacent intraoperative image frames;Calculate the global motion component according to the motion features of the multiple image regions, correct the motion features of the image regions based on the global motion component, obtain the local motion features, and determine the registration fracture state according to the local motion feature difference, boundary change information and texture continuity information of adjacent image regions;When the fracture determination threshold is met, set the propagation blocking mark, determine the registration correction amount of the image regions on both sides of the fracture respectively and execute local registration processing, generate intraoperative fusion image data and dynamic tissue recognition mark data, thereby improving the boundary consistency, local stability and recognition reliability, and reducing the influence of fusion misplacement and mark drift.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, specifically to a method for intraoperative image fusion and dynamic tissue recognition. Background Technology

[0002] In existing technologies, intraoperative image fusion and tissue recognition typically rely on the registration relationship between preoperative medical images, intraoperative two-dimensional or three-dimensional images, surgical instrument poses, and endoscopic or laparoscopic images. This maps image information from different sources to the same surgical field coordinates or display space to assist doctors in determining the location of lesions, instruments, and key tissues. For example, Chinese Patent Publication No. CN114073579A discloses a surgical navigation method, device, electronic device, and storage medium. It acquires first two-dimensional medical images of the target object and surgical instruments, combines the pose data of the capturing device and the pose data of the surgical instruments, determines the coordinate transformation relationship, and then registers and fuses the three-dimensional medical images with the two-dimensional medical images for display, thereby improving navigation accuracy. This type of solution can establish a correspondence between preoperative or intraoperative images and real-time operation scenarios in surgical navigation scenarios. However, its technical focus is mainly on the overall registration and fusion display between two-dimensional and three-dimensional medical images and instrument poses. For regional registration breaks caused by tissue traction, tissue deformation, local slippage, instrument contact or relative movement of local tissues in continuous intraoperative image frames, it lacks a special judgment and processing mechanism for the discontinuous motion state between adjacent image regions.

[0003] Existing technologies also include solutions that utilize artificial intelligence models for intraoperative anatomical structure identification and risk warning. For example, Chinese patent CN114299072A discloses an AI-based method and system for identifying and warning of anatomical variations. This method acquires real-time laparoscopic images, uses laparoscopic organ segmentation models and instrument keypoint detection models to obtain real-time organ segmentation data and instrument keypoint data, and combines this with preoperative imaging images to determine the location of variant structures, providing warnings when instruments enter the area of ​​variant anatomical structures. This type of solution can identify and warn of organs, instruments, or variant structures in the surgical field of view, and is suitable for improving the perception of surgical risk areas. However, its identification results usually rely on the model's segmentation, detection, and matching of target structures in the current frame or current field of view. It does not fully address the problem of mispropagation caused by local motion differences between different tissue regions during continuous frame fusion, and it is also difficult to avoid incorrectly transmitting the registration correction amount of one region to adjacent regions with different actual motion states.

[0004] Therefore, during intraoperative image processing, when consecutive image frames are divided into multiple image regions, if fusion is performed solely based on global motion or uniform registration correction, it is prone to problems such as boundary misalignment, texture stretching, tissue identification marker drift, or local recognition instability when there are issues like tissue traction, respiratory fluctuations, bleeding obstruction, instrument obstruction, or local tissue slippage. This is especially problematic in dynamic tissue recognition scenarios, where tissue boundaries, texture continuity, and local motion characteristics often change over time. Relying solely on global registration, organ segmentation, or instrument keypoint detection makes it difficult to promptly determine whether registration breaks have occurred between adjacent image regions, and also difficult to block the propagation of registration correction at the break boundaries. Therefore, an intraoperative image fusion and dynamic tissue recognition method is needed that can distinguish between global and local motion based on the boundary, texture, and motion characteristics of consecutive intraoperative image frames, determine the registration break status of adjacent regions, and perform local registration processing at the break points. Summary of the Invention

[0005] The purpose of this invention is to provide a method for intraoperative image fusion and dynamic tissue recognition, thereby addressing some of the drawbacks and shortcomings pointed out in the background art.

[0006] The present invention adopts the following technical solution to solve the above-mentioned technical problems:

[0007] A method for intraoperative image fusion and dynamic tissue recognition, the method being executed by a computer device, comprising:

[0008] Acquire consecutive intraoperative image frames, divide the consecutive intraoperative image frames into multiple image regions, and extract the boundary features, texture features and motion features of the image regions in adjacent intraoperative image frames;

[0009] Global motion components are calculated based on the motion characteristics of multiple image regions. The motion characteristics of the image regions are corrected based on the global motion components to obtain local motion characteristics. The registration break state between adjacent image regions is determined based on the difference in local motion characteristics, boundary change information, and texture continuity information of adjacent image regions.

[0010] When the registration breakage condition meets the breakage judgment threshold, a propagation blocking marker is set between the corresponding adjacent image regions. Based on the propagation blocking marker, the registration correction amount of the image regions on both sides of the registration breakage is determined and local registration processing is performed to generate intraoperative fused image data and dynamic tissue identification marker data.

[0011] Furthermore, when determining the registration break state between adjacent image regions, continuous registration assumption processing and separate registration assumption processing are performed on the adjacent image regions respectively; the continuous registration assumption processing uses the same registration correction amount, and the separate registration assumption processing uses different registration correction amounts; when the reduction of the boundary misalignment amount obtained by the separate registration assumption processing relative to the boundary misalignment amount obtained by the continuous registration assumption processing reaches a preset boundary threshold, and the reduction of the texture residual amount reaches a preset texture threshold, it is determined that there is a registration break between the adjacent image regions.

[0012] Furthermore, when setting propagation blocking markers between corresponding adjacent image regions, blocking pixel bands are formed by the boundary pixels of the adjacent image regions, and the width of the blocking pixel bands is adjusted according to the boundary change information, texture continuity information and local motion feature difference within the blocking pixel bands; when performing local registration processing, the blocking pixel bands are used as the termination area for the transmission of registration correction amounts.

[0013] Furthermore, before determining the registration break state between adjacent image regions, quasi-break boundaries and occlusion boundaries are assigned according to the motion correlation degree of newly added boundaries in the continuous intraoperative image frames; when the motion correlation degree between the newly added boundary and the local motion features of one side image region reaches a first motion threshold, and the motion correlation degree with the local motion features of the other side image region is lower than a second motion threshold, the newly added boundary is used as a candidate registration break boundary; when the newly added boundary moves with the occlusion region and disappears after the occlusion region disappears, setting a propagation blocking marker based on the newly added boundary is prohibited.

[0014] Furthermore, when adjusting the width of the blocking pixel band, the texture continuity matching degree of the image regions on both sides of the blocking pixel band after extending into the blocking pixel band is calculated respectively; when the difference between the texture continuity matching degrees on both sides exceeds a preset difference threshold, the blocking pixel band is extended to the side with the lower texture continuity matching degree value according to a preset expansion step size.

[0015] Furthermore, during the local registration process, the pixels within the blocking pixel band are set as isolated pixels; the isolated pixels participate in the determination of the boundary positions of adjacent image regions and are excluded from the calculation samples of the registration correction amount, so as to prevent the registration correction amount from being transmitted through the blocking pixel band.

[0016] Furthermore, the same blocking pixel band is continuously verified in consecutive intraoperative image frames; when the width change of the blocking pixel band within a preset number of frames is less than a preset width change threshold, and the motion difference between the position change of the blocking pixel band and the weighted local motion features of the image regions on both sides of the blocking pixel band is less than a preset position threshold, the blocking pixel band is retained; otherwise, the corresponding propagation blocking identifier is canceled.

[0017] Furthermore, when distinguishing between registration fracture boundaries and occlusion boundaries, the visible area changes of the image regions on both sides of the newly added boundary in consecutive intraoperative image frames are obtained; when the visible area change rate of one image region within a preset number of frames is less than zero and the cumulative decrease exceeds a preset area threshold, and the visible area change rate of the other image region within the same frame interval is greater than zero and the cumulative increase exceeds a preset increment threshold, the newly added boundary is used as an occlusion boundary; when the visible areas of the image regions on both sides of the newly added boundary are both greater than a preset visible area threshold, and the motion correlation between local motion features is lower than a preset synchronization threshold, the newly added boundary is used as a registration fracture candidate boundary.

[0018] Furthermore, it is prohibited to perform attribution recovery judgment on the corresponding image region after the occlusion area disappears after setting the propagation blocking mark based on the newly added boundary; when the boundary similarity, texture similarity and motion similarity between the boundary features, texture features and local motion features of the corresponding image region and the image region before occlusion reach the preset boundary recovery threshold, preset texture recovery threshold and preset motion recovery threshold respectively, the attribution of the image region before occlusion is restored, and the local registration processing result determined before occlusion occurs is maintained.

[0019] Furthermore, when determining the attribution of the corresponding image region after the occlusion disappears, the corresponding image region is reverse-mapped to the intraoperative image frame before occlusion based on the local motion characteristics of the corresponding image region, and the boundary overlap and texture consistency between the reverse-mapped region and the image region before occlusion are calculated; when the boundary overlap and texture consistency reach the preset boundary overlap threshold and preset texture consistency threshold respectively, the attribution of the image region before occlusion is restored; when the boundary overlap or texture consistency does not reach the corresponding threshold, the corresponding image region is marked as a newly appearing image region.

[0020] This invention acquires continuous intraoperative image frames and divides them into multiple image regions. It then extracts the boundary features, texture features, and motion features of each image region in adjacent intraoperative image frames. This enables intraoperative image processing to no longer rely solely on the uniform registration relationship of the entire image. Instead, it uses image regions as the basic processing objects to analyze the motion differences and structural changes in different regions, thereby improving the adaptability to local deformation, relative slippage, and boundary changes of intraoperative tissues.

[0021] This invention calculates global motion components based on the motion characteristics of multiple image regions, and corrects the motion characteristics of image regions based on global motion components to obtain local motion characteristics. This can effectively distinguish between global motion caused by factors such as camera movement and changes in overall field of view, and real local motion caused by local tissue activity. Furthermore, by combining the difference in local motion characteristics, boundary change information, and texture continuity information of adjacent image regions, the registration break state between adjacent image regions is determined, which helps to improve the accuracy of registration break judgment and reduce the situation of misjudging occlusion, noise, or overall motion as tissue break.

[0022] When the registration breakage condition meets the breakage judgment threshold, this invention sets propagation blocking markers between corresponding adjacent image regions. Based on these markers, it determines the registration correction amounts for the image regions on both sides of the registration breakage and then performs local registration processing. This prevents the registration correction amounts from propagating erroneously across the breakage boundary, allowing the regions on both sides of the breakage to complete registration correction according to their own motion states. The resulting intraoperative fused image data and dynamic tissue identification marker data exhibit better boundary consistency, local stability, and recognition reliability, reducing the adverse effects of fusion misalignment and marker drift on intraoperative observation and auxiliary judgment. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating the intraoperative image fusion and dynamic tissue recognition processing method of the present invention.

[0024] Figure 2 This is a local motion feature difference determination diagram in Embodiment 1 of the present invention.

[0025] Figure 3 This is an evaluation chart showing the improvement of continuous registration and separate registration in Embodiment 1 of the present invention.

[0026] Figure 4 This is a persistent verification diagram of the blocking pixel band in Embodiment 1 of the present invention.

[0027] Figure 5 This is a newly added boundary motion correlation determination diagram in Embodiment 2 of the present invention.

[0028] Figure 6 This is a diagram showing the occlusion boundary in Embodiment 2 of the present invention, distinguishing the visible area changes.

[0029] Figure 7 This is the similarity determination diagram for attribution recovery after the occlusion disappears in Embodiment 2 of the present invention. Detailed Implementation

[0030] As attached Figure 1As shown, in one specific embodiment, the computer device acquires intraoperative image frames continuously output by the intraoperative image acquisition device and divides each frame into multiple image regions according to a preset region division rule. Region division can be based on pixel position, tissue boundaries, grayscale changes, color changes, or texture distribution to ensure high local consistency of image content within the same image region. The computer device extracts boundary features, texture features, and motion features of each image region between adjacent intraoperative image frames. Boundary features characterize the positional changes of tissue contours or instrument edges; texture features characterize the grayscale distribution, color distribution, and texture direction of the tissue surface; and motion features characterize the displacement, direction, and velocity changes of the image region between adjacent frames. Subsequently, the computer device calculates a global motion component based on the motion features of multiple image regions. This global motion component reflects the common motion caused by lens movement, endoscope posture changes, or overall field of view drift. The computer device corrects the motion features of each image region based on the global motion component, thereby removing the influence of overall motion and obtaining local motion features that reflect tissue deformation, traction, or relative displacement. The computer device further compares the differences in local motion features between adjacent image regions and determines the registration breakage state between them by combining boundary change information and texture continuity information. Boundary change information can be obtained from changes in boundary position, boundary direction, and boundary shape in adjacent frames, while texture continuity information can be obtained from the texture direction, texture intensity, and texture matching degree at the boundary of adjacent image regions. When the registration breakage state meets the breakage judgment threshold, the computer device sets a propagation blocking marker between the corresponding adjacent image regions to prevent the registration correction amount from being directly transmitted on both sides of the breakage boundary. Based on the propagation blocking marker, the computer device determines the registration correction amount of the image regions on both sides of the registration breakage and performs local registration processing on both sides of the image regions respectively, so that tissue regions under different motion states can complete image fusion according to their respective local motion features. After the above processing, the computer device generates intraoperative fused image data and generates dynamic tissue identification marker data based on the local registration results, boundary change results, and region motion results, thereby distinguishing image regions with different motion states or tissue affiliations in the intraoperative images.

[0031] In the above specific implementation, the preset region division rules can be pre-stored in the computer device. The computer device can first perform denoising, brightness normalization, and invalid edge removal on the intraoperative image frames, and then form multiple candidate regions based on tissue boundaries, gray-level abrupt changes, color differences, texture direction consistency, and pixel spatial connectivity. For candidate regions that are too small, lack sufficient texture information, or whose differences from adjacent regions are lower than the preset merging conditions, the computer device can merge them with adjacent image regions; for candidate regions whose internal gray-level changes, color changes, or motion changes exceed the preset splitting conditions, the computer device can further split them. In this way, the image regions can maintain local consistency while avoiding overly fragmented region division due to noise points or local reflections.

[0032] When extracting boundary features, texture features, and motion features, the computer device can first determine the effective pixel set within each image region, excluding overexposed pixels, shadowed pixels, blurred pixels, and pixels occluded by equipment. The computer device can determine boundary features based on edge position, contour direction, boundary continuity, and boundary curvature variations within the effective pixel set; determine texture features based on grayscale distribution, color distribution, texture direction, texture coarseness, and texture repeatability; and determine motion features based on feature point matching, block matching, or motion tracking results between adjacent frames. For image regions with low matching confidence, the computer device can reduce the weight of that region in subsequent global motion component determination and registration breakage judgment, or mark that region as an area to be confirmed.

[0033] When determining global motion components, the computer device can select image regions from multiple image regions that exhibit consistent motion direction, stable texture matching, and do not belong to instrument or occlusion areas as global motion reference regions. The computer device can determine global motion components based on the common displacement, common rotation trend, and common scale change of the global motion reference regions between adjacent frames, and discard image regions whose motion trends are significantly inconsistent with those of most reference regions. This method reduces the interference of local tissue traction, instrument contact, or temporary occlusion on global motion components. When the computer device corrects the motion characteristics of each image region based on global motion components, it can remove the common motion effects caused by camera movement or field of view drift from the original motion changes of each image region, preserving the local motion changes of each image region relative to the overall field of view.

[0034] Based on the above specific implementation, when determining the registration break state between adjacent image regions, the computer device can establish continuous registration assumptions and separate registration assumptions for adjacent image regions respectively. The continuous registration assumption indicates that adjacent image regions belong to the same continuous tissue region or have a continuously transferable motion relationship. Under this assumption, the computer device performs registration calculations on adjacent image regions using the same registration correction amount, and obtains the corresponding boundary misalignment amount and texture residual amount. The separate registration assumption indicates that there are differences in tissue deformation, traction, or relative motion between adjacent image regions. Under this assumption, the computer device performs registration calculations on adjacent image regions using different registration correction amounts, and obtains the corresponding boundary misalignment amount and texture residual amount. The computer device compares the boundary misalignment amount obtained by the separate registration assumption with the boundary misalignment amount obtained by the continuous registration assumption, and calculates the reduction between the two. Simultaneously, it compares the texture residual amount obtained by the separate registration assumption with the texture residual amount obtained by the continuous registration assumption, and calculates the reduction between the two. When the reduction in boundary misalignment reaches a preset boundary threshold and the reduction in texture residual reaches a preset texture threshold, it indicates that the boundary misalignment and texture residual can be significantly reduced after the adjacent image regions are separated and registered. Based on this, the computer device determines that there is a registration break between adjacent image regions.

[0035] In the aforementioned continuous registration and separate registration assumptions, the boundary misalignment can be determined based on the boundary position offset, boundary direction deviation, and degree of boundary morphology inconsistency at the boundary between adjacent image regions. The texture residual can be determined based on the grayscale difference, color difference, texture direction difference, and degree of texture matching failure within the boundary zone between adjacent image regions. Preset boundary thresholds and preset texture thresholds can be pre-set based on preoperative calibration data, historical intraoperative image data, registration results of stable tissue regions in the current image, or manually confirmed samples. They can also be updated during the same surgical procedure based on the statistical results of stable frames. This approach avoids directly determining registration failure based solely on a single boundary or texture change.

[0036] Based on the above specific implementation, after determining that a registration break exists between adjacent image regions, the computer device can set a propagation blocking marker between the corresponding adjacent image regions. The propagation blocking marker is used to indicate that the registration correction amount stops being transmitted at the corresponding boundary. The computer device can use the boundary pixels of adjacent image regions as an initial pixel set and form a blocking pixel band based on this initial pixel set. The position of the blocking pixel band corresponds to the boundary position between adjacent image regions, and the width of the blocking pixel band can be adjusted according to the boundary change information, texture continuity information, and local motion feature difference within the blocking pixel band. When the boundary change amplitude increases, the texture continuity decreases, or the local motion feature difference increases, the computer device can increase the width of the blocking pixel band so that the blocking pixel band covers the area affected by the registration break. When the boundary change amplitude decreases, the texture continuity increases, or the local motion feature difference decreases, the computer device can decrease the width of the blocking pixel band so that the blocking pixel band matches the actual break area. During local registration processing, the computer device uses the blocking pixel band as the termination area for the transmission of the registration correction amount, so that the registration correction amount cannot be transmitted from one side of the blocking pixel band to the other side, thereby avoiding mutual interference between image regions with different motion states.

[0037] When determining the registration correction amount for the image regions on both sides of the registration break based on the propagation blocking marker, the computer device can select effective registration samples on both sides of the blocking pixel band. Effective registration samples can include pixels or feature points with stable textures, clear boundaries, motion tracking confidence meeting preset requirements, and that are not occluded. The computer device determines the registration correction amount for the corresponding side based on the effective registration samples on both sides, and ensures that the registration correction amount is only transmitted within the same side image region defined by the propagation blocking marker. If the number of effective registration samples on one side is insufficient, the computer device can supplement it using stable features of adjacent image regions on that side, stable features of the same tissue region in adjacent frames before and after, or local registration results confirmed before occlusion, to maintain the continuity of local registration processing.

[0038] When adjusting the width of the blocking pixel band, the computer device can acquire the texture distribution of the image regions on both sides of the blocking pixel band, and extend and match the image regions on both sides into the blocking pixel band to obtain the corresponding texture continuity matching degree. The texture continuity matching degree is used to characterize the degree of matching between the texture of one side of the image region after it is extended into the blocking pixel band and the texture within the blocking pixel band. When the difference in texture continuity matching degree between the two sides exceeds a preset difference threshold, it indicates that the blocking pixel band has shifted relative to the actual breakage position. The computer device extends the blocking pixel band towards the side with the lower texture continuity matching degree value according to a preset extension step size, so that the blocking pixel band can cover the area on the side with poor texture continuity. In this way, the position and width of the blocking pixel band can be adaptively adjusted according to tissue boundaries, texture breaks, and local motion differences.

[0039] During local registration processing, the computer device can set pixels within a blocking pixel band as isolated pixels. These isolated pixels can participate in determining the boundary positions of adjacent image regions, improving the accuracy of break boundary localization. Simultaneously, isolated pixels are excluded from the calculation sample of registration correction amounts, ensuring that the estimation of registration correction amounts is unaffected by pixels in mixed textures, boundary transitions, or break regions within the blocking pixel band. Since isolated pixels do not participate in the calculation of registration correction amounts, and the blocking pixel band serves as the termination region for the transmission of registration correction amounts, it prevents the transmission of registration correction amounts between adjacent image regions through the blocking pixel band. This allows the image regions on either side of the registration break to complete local registration according to their respective local motion characteristics.

[0040] In continuous intraoperative image frames, the computer device can also perform continuous verification of the same blocking pixel band. Within a preset number of frames, the computer device tracks the width and position changes of the blocking pixel band and calculates the motion difference between the positional change of the blocking pixel band and the weighted local motion features of the image regions on either side of the blocking pixel band. When the width change of the blocking pixel band within the preset number of frames is less than a preset width change threshold, and the motion difference is less than a preset position threshold, it indicates that the blocking pixel band has a stable spatial position and motion relationship, and the computer device retains the blocking pixel band and its corresponding propagation blocking identifier. When the width change of the blocking pixel band does not meet the preset width change threshold, or the motion difference does not meet the preset position threshold, it indicates that the blocking pixel band may be formed by noise, transient occlusion, or local texture disturbance, and the computer device cancels the corresponding propagation blocking identifier. Continuous verification can reduce false blocking caused by temporary image changes and improve the stability of intraoperative image fusion and dynamic tissue recognition.

[0041] During continuous verification, the computer equipment can assign band identifiers to blocked pixel bands and track the same blocked pixel band based on the boundary overlap, position proximity, and motion continuity of the image regions on both sides between adjacent frames. Weighted local motion features can be determined based on the visible area, texture stability, motion tracking confidence, and boundary clarity of the image regions on both sides of the blocked pixel band, giving stable and reliable image regions a greater influence on motion relationship judgment. This method avoids continuously identifying short-lived reflective boundaries, blood-covered boundaries, or instrument edge disturbances as registration breaks.

[0042] Based on the above specific implementation, before determining the registration break state between adjacent image regions, the computer device can first determine the source of newly added boundaries appearing in consecutive intraoperative image frames. Newly added boundaries can be newly formed tissue contours, instrument edges, occlusion edges, or boundaries formed by relative movement between tissues after comparison of adjacent frames. The computer device acquires the local motion features of the image regions on both sides of the newly added boundary and calculates the motion correlation between the newly added boundary and the image regions on both sides. The motion correlation can be determined based on the difference in motion direction, displacement, velocity, or motion trajectory. When the motion correlation between the newly added boundary and the local motion features of one image region reaches a first motion threshold, and the motion correlation with the local motion features of the other image region is lower than a second motion threshold, it indicates that the newly added boundary is closer to the motion state of one image region, while there is motion separation between it and the other image region. In this case, the computer device uses this newly added boundary as a candidate boundary for registration break. If the newly added boundary moves along with the occlusion region and disappears synchronously after the occlusion region disappears, it indicates that the newly added boundary is mainly caused by occlusion, rather than by a true registration break between tissue regions. At this point, computer equipment is prohibited from setting propagation blocking markers based on the newly added boundary, thereby avoiding misjudging temporary occlusion as registration failure.

[0043] When determining the source of a newly added boundary, the computer device can identify the occlusion region based on abrupt changes in brightness, missing textures, boundary occlusion relationships, instrument contour recognition results, and changes in visible area near the new boundary. If the texture on one side of the new boundary remains continuous before and after occlusion, and the new boundary moves in the same direction as the occluded region, the computer device can prioritize identifying the new boundary as an occlusion boundary. If both sides of the new boundary have clear textures, and the local motion features on both sides maintain different motion trends over multiple consecutive frames, the computer device can use the new boundary as a candidate boundary for registration breakage. This method improves the repeatability of determining the source of a new boundary.

[0044] When classifying registration fracture boundaries and occlusion boundaries, the computer device can also acquire the changes in the visible area of ​​the image regions on both sides of the newly added boundary in consecutive intraoperative image frames. The visible area can be determined based on the number of unoccluded pixels in the image region. When the rate of change of the visible area of ​​one image region within a preset number of frames is less than zero, and the cumulative decrease exceeds a preset area threshold, while the rate of change of the visible area of ​​the other image region within the same frame interval is greater than zero, and the cumulative increase exceeds a preset increment threshold, it indicates that one image region is gradually being occluded, and the other image region is gradually occupying the field of view. In this case, the computer device uses the newly added boundary as an occlusion boundary. When the visible areas of the image regions on both sides of the newly added boundary are both greater than a preset visible area threshold, and the motion correlation between the local motion features on both sides is lower than a preset synchronization threshold, it indicates that both image regions are observable, but there is a relative motion difference between them. In this case, the computer device uses the newly added boundary as a candidate boundary for registration fracture.

[0045] After prohibiting the setting of propagation blocking markers based on newly added boundaries, computer equipment can perform attribution restoration judgment on the corresponding image region after the occlusion area disappears. The computer equipment acquires the boundary features, texture features, and local motion features of the corresponding image region after the occlusion area disappears, and compares these features with the corresponding features of the image region before occlusion to obtain boundary similarity, texture similarity, and motion similarity. When the boundary similarity reaches a preset boundary restoration threshold, the texture similarity reaches a preset texture restoration threshold, and the motion similarity reaches a preset motion restoration threshold, it indicates that the corresponding image region after the occlusion area disappears has the same or continuous tissue attribution as the image region before occlusion. At this time, the computer equipment restores the image region attribution before occlusion and maintains the local registration processing results determined before the occlusion occurred. In this way, the original regional relationships can be restored after the occlusion disappears, avoiding unnecessary changes in tissue identification markers due to temporary occlusion.

[0046] When generating dynamic tissue identification data, the computer device can generate a region identifier for each image region and associate the region identifier with the local registration result, boundary change result, region motion result, propagation blockage identifier, occlusion status, and attribution restoration status of that image region. Dynamic tissue identification data can include tissue region category, motion state category, registration break status, occlusion status, confidence information, and time frame information. The computer device can update the region identifier based on the region matching result in consecutive intraoperative image frames, ensuring that the same tissue region maintains a consistent identification identifier across different frames; when an image region is determined to be a newly appearing image region, the computer device generates a new region identifier for it. In this way, intraoperative fused image data and dynamic tissue identification data can be correlated, facilitating the continuous differentiation of image regions with different motion states or different tissue attributions within intraoperative images.

[0047] During the attribution restoration determination, the computer device can also reverse-map the corresponding image region to the intraoperative image frame before occlusion based on the local motion characteristics of the corresponding image region after the occlusion disappears. The reverse mapping can be determined based on local displacement, motion direction, and velocity changes, allowing the image region after occlusion to be traced back to its position before occlusion in the time dimension. Subsequently, the computer device calculates the boundary overlap and texture consistency between the reverse-mapped region and the image region before occlusion. Boundary overlap characterizes the degree of positional overlap between the boundary of the reverse-mapped region and the boundary of the image region before occlusion, while texture consistency characterizes the similarity between the texture distribution of the reverse-mapped region and the texture distribution of the image region before occlusion. When both the boundary overlap and texture consistency reach a preset threshold, the computer device restores the attribution of the image region before occlusion. When either the boundary overlap or texture consistency does not reach the corresponding threshold, it indicates a lack of stable correspondence between the corresponding image region after occlusion disappears and the image region before occlusion, and the computer device marks the corresponding image region as a newly appearing image region.

[0048] Example 1:

[0049] In one specific implementation, the computer device acquires continuous intraoperative image frames output by the intraoperative image acquisition device. The resolution of the continuous intraoperative image frames is 1920×1080, and the frame rate is 30 frames / second. The computer device divides each frame of intraoperative image into multiple image regions according to a preset region division rule, and extracts the boundary features, texture features, and motion features of each image region between adjacent intraoperative image frames. Boundary features are used to characterize the positional changes of tissue contours or instrument edges, texture features are used to characterize the grayscale distribution, color distribution, and texture direction of the tissue surface, and motion features are used to characterize the displacement, direction, and velocity changes of the image region between adjacent frames.

[0050] The computer equipment can first perform denoising, brightness normalization, and invalid edge removal on the intraoperative image frames. Then, it can form multiple candidate regions based on tissue boundaries, gray-level abrupt changes, color differences, texture direction consistency, and pixel spatial connectivity. For candidate regions that are too small, lack sufficient texture information, or have differences from adjacent regions below the preset merging criteria, the computer equipment merges them with adjacent image regions. For candidate regions whose internal gray-level changes, color changes, or motion changes exceed the preset splitting criteria, the computer equipment further splits them to ensure that the image content within the same image region has high local consistency.

[0051] The computer device determines the effective pixel set within each image region, excluding overexposed pixels, shadowed pixels, blurred pixels, and pixels occluded by instruments. It determines boundary features based on edge position, contour direction, boundary continuity, and boundary curvature variations within the effective pixel set; texture features based on grayscale distribution, color distribution, texture direction, texture coarseness, and texture repeatability; and motion features based on feature point matching, block matching, or motion tracking results between adjacent frames. For image regions with low matching confidence, the computer device reduces the weight of that region in subsequent global motion component determination and registration breakage assessment.

[0052] The computer device selects image regions from multiple image regions that exhibit consistent motion direction, stable texture matching, and do not belong to instrument or occluded areas as global motion reference regions. It then determines global motion components based on the common displacement, rotation trend, and scale change of these global motion reference regions between adjacent frames. Based on these global motion components, the computer device corrects the motion characteristics of each image region, removing the common motion effects caused by lens movement, endoscope posture changes, or overall field-of-view drift, thus obtaining local motion characteristics that reflect tissue deformation, traction, or relative displacement.

[0053] The computer device further compares the differences in local motion features between adjacent image regions and determines the registration breakage status between adjacent image regions by combining boundary change information and texture continuity information. Since whether a registration breakage occurs between adjacent image regions is mainly affected by the differences in local motion on both sides, boundary misalignment changes, and texture residual changes, the computer device first obtains the differences in local motion features through velocity differences and motion direction differences, and then obtains the registration improvement evaluation value through the changes in boundary misalignment and texture residual under the continuous registration assumption and the separate registration assumption. Figure 2 The process of local motion feature difference increasing over time and exceeding motion difference threshold in consecutive frame pairs is shown. The gray-scale marked area after the third frame pair represents the continuous interval that meets the breakage judgment condition, which is used to intuitively reflect the changing trend of relative motion difference between image region A and image region B from weak to strong.

[0054] The difference in local motion characteristics can be determined as follows:

[0055]

[0056] in, This represents the difference in local motion features between adjacent image regions A and B. This represents the local displacement velocity of image region A after global motion correction. This represents the local displacement velocity of image region B after global motion correction. This indicates the local motion direction of image region A. This indicates the local motion direction of image region B. This represents the directional difference weight. Taking image region A as an example, with a local displacement velocity of 2.8 pixels / frame and a local motion direction of 18 degrees, and image region B as a local displacement velocity of 0.9 pixels / frame and a local motion direction of 41 degrees, with a directional difference weight of 0.05, the following calculation is performed: When the preset motion difference threshold is 2.5, the computer device determines that there is a significant local motion difference between image region A and image region B.

[0057] The computer device performs continuous registration and separate registration assumption processing on image region A and image region B, respectively. The continuous registration assumption assumes that adjacent image regions belong to the same continuous tissue region or have a continuously transferable motion relationship. Under this assumption, the computer device uses the same registration correction amount for image region A and image region B, obtaining the corresponding boundary misalignment and texture residual amounts. The separate registration assumption assumes that there are differences in tissue deformation, traction, or relative motion between adjacent image regions. Under this assumption, the computer device uses different registration correction amounts for image region A and image region B, obtaining the corresponding boundary misalignment and texture residual amounts.

[0058] The registration improvement evaluation value can be determined as follows:

[0059]

[0060] in, This indicates the registration improvement evaluation value. This represents the boundary misalignment under the assumption of continuous registration. This represents the boundary misalignment under the separation registration assumption. This represents the texture residual under the assumption of continuous registration. This represents the texture residual under the assumption of separate registration. Indicates the weight of the boundary misalignment. This represents the weight of the texture residual. 6.2 pixels 2.1 pixels 0.34 0.16 For 1, Taking 10 as an example, the calculation is as follows: When the preset improvement threshold is 5.0, it indicates that the separate registration method can significantly reduce boundary misalignment and texture residuals, and the computer equipment determines that there is a registration break between image region A and image region B. Figure 3Further comparisons were made between the boundary misalignment and texture residuals under the continuous registration assumption and the separate registration assumption. The texture residuals, after being magnified by the same factor, were displayed together with the boundary misalignment, which showed that both types of errors decreased after the separate registration process, and the overall improvement evaluation value exceeded the improvement threshold.

[0061] When the registration breakage condition meets the breakage judgment threshold, the computer device sets a propagation blocking marker between image region A and image region B. The computer device uses the boundary pixels of image region A and image region B as the initial pixel set to form a blocking pixel band. The position of the blocking pixel band corresponds to the boundary position between adjacent image regions. The width of the blocking pixel band is adjusted based on boundary change information, texture continuity information, and local motion feature differences within the blocking pixel band. When the boundary change amplitude increases, texture continuity decreases, or local motion feature differences increase, the computer device increases the width of the blocking pixel band to cover the registration breakage influence area; when the boundary change amplitude decreases, texture continuity increases, or local motion feature differences decrease, the computer device decreases the width of the blocking pixel band to match the actual breakage area.

[0062] When adjusting the width of the blocking pixel band, the computer device acquires the texture distribution of the image regions on both sides of the blocking pixel band and extends and matches these regions into the blocking pixel band, obtaining the corresponding texture continuity matching degree. The texture continuity matching degree characterizes the degree of matching between the texture of one image region extended into the blocking pixel band and the texture within the blocking pixel band. When the difference in texture continuity matching degrees between the two sides exceeds a preset difference threshold, it indicates that the blocking pixel band has shifted relative to the actual break position. The computer device then extends the blocking pixel band towards the side with the lower texture continuity matching degree according to a preset expansion step size, ensuring that the blocking pixel band covers the area with poor texture continuity.

[0063] During local registration processing, the computer device sets pixels within the blocked pixel band as isolated pixels. These isolated pixels participate in determining the boundary positions of adjacent image regions to improve the accuracy of break boundary localization. Simultaneously, isolated pixels are excluded from the calculation sample of registration correction amounts, ensuring that the estimation of registration correction amounts is not affected by pixels with mixed textures, boundary transitions, or break regions within the blocked pixel band. The computer device uses the blocked pixel band as the termination region for the transmission of registration correction amounts, preventing the transmission of registration correction amounts from one side of the blocked pixel band to the other, thereby preventing image regions in different motion states from interfering with each other.

[0064] The computer equipment selects effective registration samples from both sides of the blocked pixel band. Effective registration samples include pixels or feature points with stable textures, clear boundaries, motion tracking confidence levels meeting preset requirements, and no occlusion. The computer equipment determines the registration correction amount corresponding to image region A based on the effective registration samples from one side of image region A, and determines the registration correction amount corresponding to image region B based on the effective registration samples from the other side of image region B, ensuring that each registration correction amount is transmitted only within the same image region defined by the propagation blocking marker. If the number of effective registration samples on one side is insufficient, the computer equipment supplements it using stable features of adjacent image regions on that side, stable features of the same tissue region in adjacent frames, or locally registered results confirmed before occlusion.

[0065] In consecutive intraoperative image frames, the computer continuously verifies the same blocking pixel band. The computer tracks the width and position changes of the same blocking pixel band over eight consecutive frames and calculates the motion difference between the positional change of the blocking pixel band and the weighted local motion features of the image regions on either side of the blocking pixel band. When the width change of the blocking pixel band is less than 2 pixels and the motion difference is less than 1.5 pixels, it indicates that the blocking pixel band has a stable spatial position and motion relationship, and the computer retains the blocking pixel band and its corresponding propagation blocking marker; otherwise, it indicates that the blocking pixel band may be caused by reflection, temporary occlusion, or texture disturbance, and the computer cancels the corresponding propagation blocking marker. Figure 4 The diagram shows that during the verification process of 8 consecutive frames, the change in the width of the blocked pixel band and the motion difference remain below the corresponding threshold. This state indicates that the blocked pixel band is not generated by single-frame noise or brief disturbances, but has continuous stability, and can therefore be used as the transfer termination region in subsequent local registration processing.

[0066] After the above processing, the computer equipment generates intraoperative fused image data based on the local registration results of image region A and image region B, and generates dynamic tissue identification data based on the local registration results, boundary change results, regional motion results, propagation blocking markers, and continuous verification results. In this way, image regions with different motion states or tissue affiliations can be distinguished in intraoperative images, reducing the impact of overall field of view drift, local tissue traction, and temporary texture disturbances on the image fusion results.

[0067] Example 2:

[0068] In another specific implementation, the computer device acquires continuous intraoperative image frames output by the intraoperative image acquisition device. The resolution of the continuous intraoperative image frames is 1920×1080, and the frame rate is 30 frames / second. The computer device compares the boundaries of adjacent intraoperative image frames, detects newly formed tissue contours, instrument edges, occlusion edges, or boundaries formed by relative movement between tissues in the current frame, and uses these as new boundaries.

[0069] The computer device acquires local motion features of the image regions on both sides of the newly added boundary. These local motion features include local displacement velocity and local motion direction after global motion component correction. The computer device also tracks the positional change of the newly added boundary between adjacent frames, determines the moving speed and direction of the newly added boundary, and calculates the motion correlation between the newly added boundary and the image regions on both sides.

[0070] Since occlusion boundaries typically maintain similar movement speeds and directions to the occluded area, while true registration fracture boundaries usually have higher motion correlation with one side of the tissue region and lower motion correlation with the other side, computer equipment constructs motion correlation based on the speed and direction differences between the newly added boundary and the image region. Motion correlation can be determined as follows:

[0071]

[0072] in, Indicates the newly added boundary and the first Motion correlation of image regions Indicates the movement speed of the newly added boundary. Indicates the first Local displacement velocity of each image region Indicates the direction of movement of the newly added boundary. Indicates the first Local motion direction of each image region This represents the weight of directional differences. In this formula, the smaller the velocity difference, the smaller the directional difference, and the smaller the denominator, the higher the motion correlation, indicating that the motion relationship between the newly added boundary and the corresponding image region is closer.

[0073] Taking a newly added boundary e with a movement speed of 3.1 pixels / frame and a movement direction of 27 degrees, a local displacement speed of image region A with a local movement direction of 3.0 pixels / frame and a local movement direction of 29 degrees, and a local displacement speed of image region B with a local movement direction of 0.8 pixels / frame and a local movement direction of 56 degrees, with a direction difference weight of 0.04 as an example, the computer device obtains a motion correlation degree of 0.85 between the newly added boundary e and image region A, and a motion correlation degree of 0.22 between the newly added boundary e and image region B. When the first motion threshold is 0.75 and the second motion threshold is 0.45, the motion correlation degree between the newly added boundary e and image region A reaches the first motion threshold, and the motion correlation degree with image region B is lower than the second motion threshold. Therefore, the computer device uses the newly added boundary e as a candidate boundary for registration break. Figure 5The difference in motion correlation between the newly added boundary and the image regions on both sides is shown. The correlation of image region A is above the first motion threshold, while the correlation of image region B is below the second motion threshold. This indicates that the newly added boundary does not move simultaneously with the regions on both sides, but rather is closer to the motion state of one side of the tissue region.

[0074] The computer device further determines whether the newly added boundary moves with the occluded area. The occluded area can be determined based on abrupt changes in brightness near the newly added boundary, missing textures, boundary occlusion relationships, instrument contour recognition results, and changes in visible area. If the newly added boundary moves in the same direction as the occluded area and disappears synchronously after the occluded area disappears, the computer device determines that the newly added boundary is mainly caused by occlusion and prohibits the setting of propagation blocking markers based on the newly added boundary.

[0075] When classifying quasi-fracture boundaries and occlusion boundaries, the computer device tracks the visible area of ​​the image regions on both sides of the newly added boundary over six consecutive frames. The visible area is determined based on the number of effective pixels in the image region that are not covered by instruments, smoke, blood, or other obstructions. When the visible area of ​​one image region continuously decreases and the cumulative decrease exceeds 15%, while the visible area of ​​the other image region continuously increases and the cumulative increase exceeds 15%, it indicates that one image region is gradually being occluded, and the other image region is gradually occupying the field of view. In this case, the computer device treats the newly added boundary as the occlusion boundary. Figure 6 The diagram illustrates the reverse change in the visible area of ​​the image regions on both sides within six consecutive frames. The visible area on one side decreases from 100% to 83%, while the visible area on the other side increases from 100% to 118%, and crosses the decision line for decreasing by 15% and increasing by 15% respectively, to support the determination that the newly added boundary belongs to the occlusion boundary.

[0076] When the visible area of ​​the image regions on both sides of the newly added boundary is greater than 70% of the initial area of ​​the corresponding region in the current frame, and the motion correlation between the local motion features on both sides is lower than the preset synchronization threshold, it indicates that the image regions on both sides are observable, but there is a relative motion difference between them. The computer device will use the newly added boundary as a candidate boundary for registration break, and allow subsequent determination of whether to set a propagation blocking flag by combining boundary change information and texture continuity information.

[0077] After prohibiting the setting of propagation blocking markers based on newly added boundaries, the computer device performs attribution recovery judgment on the corresponding image region after the occluded area disappears. The computer device acquires the boundary features, texture features, and local motion features of the corresponding image region after the occlusion disappears, and compares these features with the corresponding features of the image region before occlusion to obtain boundary similarity, texture similarity, and motion similarity. Boundary similarity is used to represent the degree of consistency in boundary shape and boundary position, texture similarity is used to represent the degree of consistency in grayscale distribution, color distribution, and texture direction, and motion similarity is used to represent the continuity of local displacement velocity and motion direction.

[0078] Since determining whether an image region after occlusion removal belongs to the same tissue region as before occlusion requires consideration of boundary morphology, texture distribution, and motion continuity, computer devices can weight boundary similarity, texture similarity, and motion similarity to obtain a comprehensive restoration similarity. The comprehensive restoration similarity can be determined as follows:

[0079]

[0080] in, Indicates the overall recovery similarity. Indicates boundary similarity, Indicates texture similarity, Indicates motion similarity, Represents the boundary similarity weight. Indicates texture similarity weights. This represents the motion similarity weight. In this formula, the three similarities—boundary, texture, and motion—together reflect the consistency of organizational affiliation between the image region after the occlusion is removed and the image region before the occlusion.

[0081] Taking an example where the boundary similarity between the corresponding image region after occlusion disappears and the image region before occlusion is 0.86, the texture similarity is 0.82, and the motion similarity is 0.79, with boundary similarity weights of 0.35, texture similarity weights of 0.40, and motion similarity weights of 0.25, the computer device obtains a comprehensive restored similarity of 0.83. When the comprehensive restored similarity threshold is 0.80, the computer device determines that the corresponding image region after occlusion disappears and the image region before occlusion have a stable organizational affiliation, restores the affiliation of the image region before occlusion, and maintains the local registration processing result determined before occlusion occurred. Figure 7 The results of boundary similarity, texture similarity, motion similarity, and comprehensive recovery similarity are shown. Among them, the comprehensive recovery similarity reaches the recovery threshold, which is used to indicate that there is a recoverable tissue correspondence between the image region after the occlusion is removed and the image region before the occlusion.

[0082] When the boundary similarity, texture similarity, or motion similarity does not reach the corresponding recovery threshold, and the overall recovery similarity does not reach the preset recovery threshold, the computer device reverse maps the corresponding image region to the intraoperative image frame before the occlusion, based on the local motion characteristics of the corresponding image region after the occlusion disappears. The reverse mapping is determined based on local displacement, motion direction, and velocity changes, so that the image region after the occlusion disappears is traced back to the position before the occlusion occurred in the time dimension.

[0083] The computer device calculates the boundary coincidence and texture consistency between the inverted mapped region and the original image region. Boundary coincidence characterizes the degree of positional overlap between the boundary of the inverted mapped region and the boundary of the original image region, while texture consistency characterizes the similarity between the texture distribution of the inverted mapped region and the texture distribution of the original image region. When both boundary coincidence and texture consistency reach a preset threshold, the computer device restores the original image region's attribution; when either threshold is not met, the computer device marks the corresponding image region as a newly appearing image region.

[0084] In this way, the computer equipment can distinguish between real registration failure boundaries and temporary occlusion boundaries when new boundaries appear, avoiding false blockage caused by instrument occlusion, blood occlusion or short-term visual field obstruction, and restoring the original image region affiliation after the occlusion disappears, thus maintaining the stability of intraoperative fused image data and dynamic tissue identification data in continuous intraoperative image frames.

Claims

1. A method for intraoperative image fusion and dynamic tissue recognition, characterized in that the method is executed by a computer device and includes: Acquire consecutive intraoperative image frames, divide the consecutive intraoperative image frames into multiple image regions, and extract the boundary features, texture features and motion features of the image regions in adjacent intraoperative image frames; Global motion components are calculated based on the motion characteristics of multiple image regions. The motion characteristics of the image regions are corrected based on the global motion components to obtain local motion characteristics. The registration break state between adjacent image regions is determined based on the difference in local motion characteristics, boundary change information, and texture continuity information of adjacent image regions. When the registration breakage condition meets the breakage judgment threshold, a propagation blocking marker is set between the corresponding adjacent image regions. Based on the propagation blocking marker, the registration correction amount of the image regions on both sides of the registration breakage is determined and local registration processing is performed to generate intraoperative fused image data and dynamic tissue identification marker data.

2. The intraoperative image fusion and dynamic tissue recognition processing method according to claim 1, characterized in that, when determining the registration break state between adjacent image regions, continuous registration assumption processing and separate registration assumption processing are respectively performed on the adjacent image regions; The continuous registration assumption process uses the same registration correction amount, while the separate registration assumption process uses different registration correction amounts. When the reduction in the boundary misalignment amount obtained by the separation registration hypothesis processing relative to the boundary misalignment amount obtained by the continuous registration hypothesis processing reaches a preset boundary threshold, and the reduction in the texture residual amount reaches a preset texture threshold, it is determined that there is a registration break between the adjacent image regions.

3. The intraoperative image fusion and dynamic tissue recognition processing method according to claim 1, characterized in that, when setting propagation blocking markers between corresponding adjacent image regions, the boundary pixels of the adjacent image regions form a blocking pixel band, and the width of the blocking pixel band is adjusted according to the boundary change information, texture continuity information and local motion feature difference within the blocking pixel band; when performing local registration processing, the blocking pixel band is used as the transmission termination area of ​​the registration correction amount.

4. The intraoperative image fusion and dynamic tissue recognition processing method according to claim 1, characterized in that, before determining the registration break state between adjacent image regions, quasi-break boundaries and occlusion boundaries are assigned according to the motion correlation degree of newly added boundaries in the continuous intraoperative image frames; when the motion correlation degree between the newly added boundary and the local motion features of one side image region reaches a first motion threshold, and the motion correlation degree with the local motion features of the other side image region is lower than a second motion threshold, the newly added boundary is used as a candidate boundary for registration break; when the newly added boundary moves with the occlusion region and disappears after the occlusion region disappears, setting a propagation blocking marker based on the newly added boundary is prohibited.

5. The intraoperative image fusion and dynamic tissue recognition processing method according to claim 3, characterized in that, when adjusting the width of the blocking pixel band, the texture continuity matching degree of the image regions on both sides of the blocking pixel band after extending into the blocking pixel band is calculated respectively; when the difference between the texture continuity matching degrees on both sides exceeds a preset difference threshold, the blocking pixel band is extended to the side with the lower texture continuity matching degree value according to a preset expansion step size.

6. The intraoperative image fusion and dynamic tissue recognition processing method according to claim 3, characterized in that, when performing local registration processing, the pixels within the blocking pixel band are set as isolated pixels; the isolated pixels participate in the determination of the boundary positions of adjacent image regions and are excluded from the calculation samples of the registration correction amount, so as to prevent the registration correction amount from being transmitted through the blocking pixel band.

7. The intraoperative image fusion and dynamic tissue recognition processing method according to claim 3, characterized in that, the same blocking pixel band is continuously verified in consecutive intraoperative image frames; when the width change of the blocking pixel band within a preset number of frames is less than a preset width change threshold, and the motion difference between the position change of the blocking pixel band and the weighted local motion features of the image regions on both sides of the blocking pixel band is less than a preset position threshold, the blocking pixel band is retained; otherwise, the corresponding propagation blocking identifier is canceled.

8. The intraoperative image fusion and dynamic tissue recognition processing method according to claim 4, characterized in that, when distinguishing between registration fracture boundaries and occlusion boundaries, the visible area changes of the image regions on both sides of the newly added boundary in consecutive intraoperative image frames are obtained; when the visible area change rate of one image region within a preset number of frames is less than zero and the cumulative decrease exceeds a preset area threshold, and the visible area change rate of the other image region within the same frame interval is greater than zero and the cumulative increase exceeds a preset increment threshold, the newly added boundary is used as an occlusion boundary; when the visible areas of the image regions on both sides of the newly added boundary are both greater than a preset visible area threshold, and the motion correlation between local motion features is lower than a preset synchronization threshold, the newly added boundary is used as a registration fracture candidate boundary.

9. The intraoperative image fusion and dynamic tissue recognition processing method according to claim 4, characterized in that, after setting the propagation blocking marker based on the newly added boundary, the attribution recovery judgment is not performed on the corresponding image region after the occlusion area disappears; when the boundary similarity, texture similarity and motion similarity between the boundary features, texture features and local motion features of the corresponding image region and the image region before occlusion reach the preset boundary recovery threshold, preset texture recovery threshold and preset motion recovery threshold respectively, the attribution of the image region before occlusion is restored, and the local registration processing result determined before the occlusion occurs is maintained.

10. The intraoperative image fusion and dynamic tissue recognition processing method according to claim 9, characterized in that, when performing the attribution restoration judgment on the corresponding image region after the disappearance of the occluded region, the corresponding image region is reverse-mapped to the intraoperative image frame before occlusion based on the local motion characteristics of the corresponding image region, and the boundary overlap and texture consistency between the reverse-mapped region and the image region before occlusion are calculated; when the boundary overlap and texture consistency reach the preset boundary overlap threshold and preset texture consistency threshold respectively, the attribution of the image region before occlusion is restored; when the boundary overlap or texture consistency does not reach the corresponding threshold, the corresponding image region is marked as a newly appearing image region.

Images