An endoscopic image stabilization control method

Abstract

This invention provides an endoscopic image stabilization control method. It acquires the original shaking image sequence and inertial data of the endoscope, fuses the original shaking image sequence and inertial data to obtain an initial mixed motion field, performs surgical semantic segmentation on the original shaking image sequence, and obtains a collaborative attention map by combining physical motion mechanisms. A spatiotemporal adaptive filtering kernel is obtained through the collaborative attention map, and the initial mixed motion field is filtered according to the spatiotemporal adaptive filtering kernel to obtain a stable motion field. The stable motion field is then processed by image deformation and interpolation synthesis to obtain a stable image sequence. This integrated technical solution of surgical semantic segmentation, dynamic image stabilization optimization, and precise shaking suppression solves the problem of endoscopic shaking interfering with the surgical field of neurosurgery.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to an endoscopic image stabilization control method. Background Technology

[0002] With the rapid development of minimally invasive neurosurgery, endoscopes have become the core minimally invasive diagnostic and therapeutic instruments for delicate surgeries such as cortical vascular anastomosis. The surgical image sequences they acquire serve as the basis for doctors to observe lesions and operate instruments.

[0003] Existing image stabilization technologies mostly rely on a single data point for stabilization, failing to fully consider the weak texture characteristics of neurosurgical surgical scenes, and failing to distinguish the different motion differences between static background, instrument movement, and physiological movement in the surgical area. During the stabilization process, either excessive suppression of jitter leads to loss of visual field details and distortion of instrument and physiological movement, or insufficient stabilization results in pixel shifts and inter-frame jumps in the visual field, failing to effectively eliminate visual field interference caused by jitter.

[0004] Therefore, there is an urgent need for a method to solve the problem of endoscopic tremors interfering with the surgical field of neurosurgery. Summary of the Invention

[0005] In view of the aforementioned problems, and in conjunction with the first aspect of the present invention, embodiments of the present invention provide an endoscopic image stabilization control method, the method comprising: acquiring an original jitter image sequence and inertial data of an endoscope; fusing the original jitter image sequence and inertial data to obtain an initial mixed motion field; performing surgical semantic segmentation on the original jitter image sequence and obtaining a collaborative attention map in conjunction with a physical motion mechanism, wherein the collaborative attention map includes a collaborative attention value for each pixel; obtaining a spatiotemporal adaptive filtering kernel based on the collaborative attention map; filtering the initial mixed motion field based on the spatiotemporal adaptive filtering kernel to obtain a stable motion field; and performing image deformation and interpolation synthesis processing on the stable motion field to obtain a stable image sequence.

[0006] As a further aspect of the present invention, the original jitter image sequence and inertial data of the endoscope are acquired, and the original jitter image sequence and inertial data are fused to obtain an initial hybrid motion field. This includes: acquiring the original image sequence based on the endoscope; acquiring inertial data based on the endoscope's built-in IMU sensor; preprocessing the inertial data by denoising and zero-bias correction, and integrating the processed inertial data to obtain coarse motion estimation data; performing non-local texture enhancement on adjacent frames of the original jitter image; acquiring a dense motion field between adjacent frames of the original jitter image sequence based on a lightweight deep learning optical flow network; using the coarse motion estimation data as a motion prior and constructing a fusion filter in conjunction with kinematic and geometric constraints; and performing scale recovery, global motion compensation, and drift correction on the dense motion field based on the fusion filter to output the initial hybrid motion field.

[0007] As a further aspect of the present invention, surgical semantic segmentation is performed on the original jittery image sequence, and a collaborative attention map is obtained by combining the physical motion mechanism. This includes: performing instrument instance segmentation, physiological motion region detection, and static structure inference on the original jittery image sequence to generate an instrument mask, a physiological motion mask, and a static mask; generating a collaborative attention map based on the instrument mask, the physiological motion mask, and the static mask, combined with external ECG physiological signals and the physical motion mechanism; the generation rules of the collaborative attention map include: assigning high attention values ​​to the static region corresponding to the static mask, assigning attention values ​​to the physiological motion region corresponding to the physiological motion mask, and assigning low attention values ​​to the instrument region corresponding to the instrument mask; if there is region overlap, the attention values ​​of each region are fused, and the attention values ​​of the physiological motion region are periodically modulated by combining the external ECG physiological signals.

[0008] As a further aspect of the present invention, the method further includes: segmenting instrument instances based on a real-time segmentation network optimized for surgical instruments; detecting physiological motion regions based on temporal analysis and prior shape, wherein the temporal analysis includes inter-frame difference and phase change, the prior shape is tubular, and the physiological motion regions include, but are not limited to, vascular pulsation regions and tissue respiration fluctuation regions; and inferring static structures based on the spatiotemporal consistency of the surgical scene.

[0009] As a further embodiment of the present invention, the method further includes: applying rigid body kinematic constraints to the instrument region corresponding to the instrument mask, and setting a first deviation threshold and a second deviation threshold; comparing the motion vector with the historical motion trajectory to obtain a first deviation, and comparing the motion vector with the coarse motion estimation data corresponding to the inertial data to obtain a second deviation; if there is a situation where the first deviation is greater than the first deviation threshold or the second deviation is greater than the second deviation threshold, then the motion in the instrument region is determined to be non-instrument movement, and motion correction is performed simultaneously.

[0010] As a further aspect of the present invention, a spatiotemporal adaptive filtering kernel is obtained based on a collaborative attention map, and the initial mixed motion field is filtered based on the spatiotemporal adaptive filtering kernel to obtain a stable motion field. This includes: generating a spatiotemporal adaptive filtering kernel for each pixel and each temporal frame in the original jittery image sequence based on the collaborative attention map and the initial mixed motion field; performing convolutional filtering on the initial mixed motion field based on the spatiotemporal adaptive filtering kernel to obtain a stable motion field; wherein, the convolutional filtering is represented as an anisotropic diffusion process modulated by the collaborative attention map, and the diffusion coefficient is controlled by the collaborative attention map.

[0011] As a further aspect of the present invention, the spatiotemporal adaptive filtering kernel includes: the size, shape, and dimensions of the spatiotemporal adaptive filtering kernel are adaptively modulated according to the collaborative attention map and the initial mixed motion field; in the static region, a 6×6 spatiotemporal kernel is used for strong smoothing motion; in the instrument region, a 1×1 identity kernel or a 2×2 extremely small spatiotemporal kernel is used to maintain the original instrument motion; in the physiological motion region, the size and smoothing intensity of the spatiotemporal adaptive filtering kernel change linearly and continuously with the collaborative attention value; and the directional kernel shape of the spatiotemporal adaptive filtering kernel is modulated based on the local consistency of the initial mixed motion field.

[0012] As a further aspect of the present invention, image deformation and interpolation synthesis processing is performed on the stable motion field to obtain a stable image sequence, including: obtaining a deformation field from the original jittered frame to the stable frame based on the stable motion field, wherein the original jittered frame represents a single frame image in the original jittered image sequence, and the stable frame represents a single frame image corresponding to the single frame image in the original jittered image sequence; interpolating the deformation field based on a differentiated interpolation strategy; wherein an edge-preserving interpolation algorithm is used for the instrument region corresponding to the instrument mask, and a bilinear interpolation algorithm is used for the non-instrument region; and synthesizing the interpolated deformation field to obtain a stable image sequence. Attached Figure Description

[0013] Figure 1 This is a flowchart of the steps of an endoscopic image stabilization control method according to the present invention;

[0014] Figure 2 This is a flowchart of the steps for obtaining the initial mixed motion field in an endoscopic image stabilization control method of the present invention. Detailed Implementation

[0015] The present invention will now be described in detail with reference to the accompanying drawings. Figure 1 This is a schematic diagram of the execution flow of an endoscopic image stabilization control method according to an embodiment of the present invention. Figure 2 This is a flowchart of the steps for obtaining the initial mixed motion field in an endoscopic image stabilization control method of the present invention. The following is a detailed description of this endoscopic image stabilization control method.

[0016] Step S1: Obtain the original shaking image sequence and inertial data of the endoscope, fuse the original shaking image sequence and inertial data to obtain the initial mixed motion field.

[0017] In this embodiment, step S1 includes:

[0018] Step S1-1: Acquire the original image sequence based on the endoscope and acquire inertial data based on the IMU sensor built into the endoscope.

[0019] Specifically, the endoscope uses a high-definition endoscope specifically designed for neurosurgery. Its lens is aimed at the surgical area and continuously acquires images of the surgical scene according to a preset sampling frame rate to form a raw image sequence. Since the doctor inevitably shakes while holding the endoscope during the operation, this raw image sequence is the raw shake image sequence. The endoscope handle has a built-in IMU sensor, which includes a three-axis accelerometer and a three-axis gyroscope. It can acquire the acceleration and angular velocity data of the endoscope in space in real time and form inertial data.

[0020] It should be noted that the inertial data can acquire the instantaneous motion state of the endoscope, with a fast response speed, and can track the subtle tremors of the doctor's hand in real time. In contrast, other sensor data such as ultrasound and infrared have higher response delays and cannot meet the real-time image stabilization requirements of surgery. At the same time, there are instrument obstructions and physiological motion interference from brain tissue in the surgical scene, and visual data is prone to optical flow drift. Inertial data directly acquires the motion of the endoscope itself and is not affected by scene obstruction and physiological motion. Therefore, in this embodiment, inertial data is collected as the core analysis data.

[0021] In some possible embodiments, taking cortical vascular anastomosis as an example, the doctor holds an endoscope and operates on the anastomosis site of the middle cerebral artery. The endoscope continuously acquires images of the anastomosis site and surrounding brain tissue, blood vessels, and surgical instruments at a frame rate of 30 frames per second, forming the original jitter image sequence I(t). Due to the slight tremor of the doctor's hand, the outline of the anastomosis blood vessels in the image will be slightly blurred and the position will be slightly offset. The IMU sensor built into the endoscope handle synchronously acquires the endoscopic acceleration data and angular velocity data caused by the tremor of the doctor's hand, forming inertial data S(t). For example, at a certain frame t=10, the acquired acceleration data is (0.02, 0.015, 0.008) and the angular velocity data is (0.003, 0.002, 0.001), which are synchronously matched with the original jitter image I(10) corresponding to t=10, providing a reference for subsequent jitter analysis.

[0022] Step S1-2 involves preprocessing the inertial data by denoising and zero bias correction, and then integrating the processed inertial data to obtain coarse motion estimation data.

[0023] It should be noted that during the acquisition of inertial data, the IMU sensor is affected by factors such as current interference and instrument vibration in the surgical environment. The acquired raw inertial data often contains noise. At the same time, the sensor itself has an inherent zero bias error, which is a non-zero small value output by the sensor when it is stationary. If used directly, it will cause deviations in subsequent motion estimation. Therefore, the raw inertial data must be preprocessed by denoising and zero bias correction before use.

[0024] Specifically, a Kalman filter algorithm can be used to filter high-frequency noise and environmental interference, retaining effective inertial data reflecting endoscope jitter; a static calibration method can be used, by collecting the output data of the sensor in a static state, calculating the zero-bias error value, and then subtracting this zero-bias value from the original inertial data, thereby eliminating the inherent error of the sensor; after preprocessing, integration is performed on the clean inertial data, including a second integration of the acceleration data to obtain the displacement change of the endoscope; a first integration of the angular velocity data to obtain the angle change of the endoscope, which is then combined to form the coarse motion estimation data T(t).

[0025] In some possible embodiments, continuing with the example of cortical vascular anastomosis, the acquired raw inertial data S(t) often contains interference noise from the current of the surgical electrocoagulation instruments, causing abnormal peaks in the acceleration data of some frames. For example, at t=15, the acceleration in a certain direction abnormally reaches 0.1 m / s². In this case, a Kalman filter algorithm can be used to denoise S(t) first, filtering out the abnormal peaks to obtain clean inertial data. Subsequently, zero-bias correction is performed. Through calibration, the zero-bias value of acceleration of the IMU sensor is found to be (0.002, 0.001, 0.0005), and the zero-bias value of angular velocity is (0.0001, 0.0001, ...). 0.00005), subtract this zero bias value from the denoised inertial data to complete the preprocessing; integrate the preprocessed inertial data, such as the acceleration data from t=10 to t=11, to obtain the displacement change of the endoscope in this time period as (0.0001mm, 0.00008mm, 0.00005mm); integrate the angular velocity data in this time period to obtain the angle change as (0.00003rad, 0.00002rad, 0.00001rad), and combine the two to form the coarse motion estimation data T(11) at t=11, which initially reflects the jitter displacement and angle of the endoscope at this moment.

[0026] Steps S1-3: Perform nonlocal texture enhancement on adjacent elements of the original jittery image.

[0027] Specifically, in the original jittery image sequence, the key areas of the surgical scene are weakly textured regions, such as brain tissue sulci and small blood vessels. The pixel grayscale changes in these areas are gradual. If not processed, optical flow estimation drift and insufficient accuracy are likely to occur when calculating the motion field between adjacent frames, thus affecting the accuracy of jitter separation. Therefore, this step performs non-local texture enhancement processing on the images of adjacent frames in the original jittery image sequence. Specifically, by using the non-local similarity of the images and combining it with pixel blocks similar to the target pixel in the search image, the target pixel is weighted and enhanced, focusing on strengthening the pixel grayscale contrast and edge features of weakly textured regions, while preserving the details of strong textured regions, such as surgical instruments and blood vessel contours, to avoid over-enhancement that could lead to image distortion.

[0028] Furthermore, the nonlocal texture enhancement process includes setting the search window size to 7×7 and the similarity window size to 3×3. Centering on each target pixel in the fine sulci region of the cerebral cortex, within the 7×7 search window, iterates to find all similar pixel blocks whose grayscale distribution matches that of the target pixel in the 3×3 similarity window. The grayscale Euclidean distance between each similar pixel block and the target pixel block is calculated. The smaller the Euclidean distance, the higher the similarity, and the larger the assigned weight coefficient. The weight coefficient is normalized using a Gaussian function. In this embodiment, to ensure that the weight of highly similar pixel blocks is prominent, the Gaussian standard deviation is set to 1.2. The specific weight calculation formula is as follows: In this equation, A represents the weight, d represents the Euclidean distance, and S represents the Gaussian standard deviation. After normalization, the sum of the weights of all similar pixel blocks is 1. Finally, the pixel gray values ​​of all similar pixel blocks are weighted and summed with the corresponding weight coefficients. The calculation result is used as the new gray value of the target pixel to complete the weighted enhancement of the target pixel. This process is repeated for all pixels in the groove region to complete the overall weighted enhancement.

[0029] In some possible embodiments, continuing with the surgical example described above, in the original jittery image sequence I(t), the texture features of the fine sulci and gyri of the cerebral cortex are weak, and the pixel grayscale difference is small. In the adjacent frames t=10 and t=11, the outline of this region is almost indistinguishable. Therefore, nonlocal texture enhancement processing is performed on the adjacent frames t=10 and t=11. Similar pixel blocks of each sulcus and gyri region are searched and weighted. After nonlocal texture enhancement, it is found that the pixel grayscale contrast of the sulcus and gyri region is improved, and the metal outline details of the bipolar electrocoagulation forceps are not affected.

[0030] Steps S1-4: Obtain the dense motion field between adjacent frames of the original jittery image sequence based on a lightweight deep learning optical flow network.

[0031] Specifically, this step describes the motion vector of each pixel between adjacent frames in the original jittery image sequence by acquiring a dense motion field, thereby reflecting the positional changes of pixels between adjacent frames. The adjacent frame images after nonlocal texture enhancement are input into a lightweight deep learning optical flow network. This network outputs the motion vector of each pixel between adjacent frames through feature extraction, correlation calculation, iterative optimization, and other processes, and forms a dense motion field Flow(t). The dense motion field can capture all pixel positional changes between adjacent frames caused by endoscope jitter, instrument movement, and physiological movement.

[0032] It should be noted that the lightweight deep learning optical flow network is a lightweight optimization of the original deep learning optical flow network. Its main architecture includes: an input layer: receiving adjacent frame images after nonlocal texture enhancement, and normalizing the input image to normalize the pixel grayscale values ​​to the [0,1] range; a feature extraction layer: adopting a lightweight convolutional neural network structure, including 4 convolutional blocks, each consisting of a 3×3 convolutional layer, a BN layer, and a ReLU activation function, used to extract multi-scale visual features of the input adjacent frame images. The shallow convolutional blocks extract low-dimensional features of the image, such as pixel grayscale changes and simple edges, adapting to feature capture of weakly textured areas of brain tissue, while the deep convolutional blocks extract high-dimensional features of the image, such as instrument outlines and blood vessel textures. At the same time, the feature map size is compressed through a convolution operation with a stride of 2, thereby improving the extraction accuracy; and a correlation layer: used to calculate the pixel correlation between the feature maps of adjacent frames, constructing the correlation volume, and calculating the correlation volume of each pixel in the feature map of frame t. The similarity between the corresponding neighboring pixels in the feature map of frame t+1 is used to obtain the motion correlation between pixels, providing a foundation for subsequent motion vector prediction. The focus is on optimizing the correlation calculation in weak texture areas to reduce the risk of optical flow drift. The recurring update layer contains 3 recurring update units to iteratively optimize the motion vector. Combined with the correlation volume output by the correlation layer, the deviation of the initial motion vector is gradually corrected. Each recurring update unit introduces residual connections to avoid gradient vanishing. At the same time, the hidden layer dimension of the GRU unit is reduced to ensure the calculation speed. Finally, a preliminary dense motion vector is output. The refinement and adjustment layer contains 1 deconvolution block and 1×1 convolution layer to restore the size of the motion vector output by the recurring update layer. At the same time, the accuracy of the motion vector is refined, and the motion vector deviation in edge areas, such as instrument tips and blood vessel edges, is corrected to meet the motion capture requirements of fine areas in surgical scenes. The output layer outputs the motion vector of each pixel between adjacent frames and forms a dense motion field.

[0033] Understandably, lightweight networks significantly reduce computational complexity by simplifying the number of parameters and optimizing the network structure, thus better meeting the needs of real-time image stabilization and avoiding surgical field delays caused by computational latency. The lightweight design does not excessively sacrifice optical flow estimation accuracy; in particular, for motion vector capture in weakly textured regions in this embodiment, high-precision optical flow estimation can be achieved by optimizing the feature extraction module, reducing optical flow drift and outperforming traditional optical flow algorithms in weakly textured regions. Deep learning networks possess stronger anti-interference capabilities, effectively handling complex situations such as instrument occlusion, lighting changes, and physiological motion interference in surgical scenarios, outputting a stable dense motion field. Therefore, a lightweight deep learning optical flow network is selected in this embodiment, which is more suitable for the neurosurgical surgical scenario compared to traditional optical flow algorithms such as the Lucas-Kanade algorithm and non-lightweight deep learning optical flow networks such as the original RAFT algorithm.

[0034] In some possible embodiments, continuing with the surgical example, the texture-enhanced images of adjacent frames at t=10 and t=11 are input into a lightweight deep learning optical flow network. This network quickly extracts features from the two frames, calculates the correlation of each pixel, iteratively optimizes the motion vector, and finally outputs a dense motion field. In the output dense motion field Flow(10), each pixel has a corresponding motion vector. For example, the motion vector of the anastomotic vessel pixel mainly comes from the endoscope jitter and vascular pulsation, and the value is small, such as (0.3, 0.2) pixels / frame; the motion vector of the bipolar electrocoagulation forceps pixel comes from the doctor's active operation, and the value is relatively large, such as (1.5, 1.2) pixels / frame; the motion vector of the brain tissue sulcus pixel mainly comes from the endoscope jitter, and the value is small, such as (0.2, 0.1) pixels / frame.

[0035] Steps S1-5: Use coarse motion estimation data as motion priors and construct a fusion filter by combining kinematic and geometric constraints.

[0036] Specifically, the coarse motion estimation data T(t) can reflect the overall jitter trajectory of the endoscope, but its accuracy is low; the dense motion field Flow(t) can reflect the motion details of each pixel, but it is prone to drift errors in weak texture areas. Therefore, this step combines the advantages of the two types of data by constructing a fusion filter to make up for their respective shortcomings. The fusion filter uses the coarse motion estimation data T(t) as the motion prior, that is, the overall jitter trajectory of the endoscope as the reference benchmark, and incorporates kinematic constraints, such as the continuity of endoscope motion and rigid body motion characteristics, and geometric constraints, such as the spatial consistency of pixel motion, to finally construct a fusion filter that can accurately fuse inertial data and visual data. The kinematic constraints ensure that the fused motion trajectory is smooth and continuous, which conforms to the actual motion law of the endoscope; the geometric constraints ensure that the motion vectors of adjacent pixels are consistent, avoiding the situation of motion vector chaos.

[0037] It should be noted that the kinematic constraints are designed to conform to the actual motion characteristics of the endoscope, clearly defining and constraining its motion patterns to prevent abnormalities in the fused motion trajectory that contradict physical principles. In this embodiment, the endoscope, as a rigid instrument, strictly follows the characteristics of rigid body motion, meaning it only exhibits translation and rotation, without deformation, twisting, or other abnormal movements. Therefore, the kinematic constraints primarily define the endoscope's motion as rigid body motion involving translation and rotation, prohibiting endoscope deformation or sudden jumps reflected in pixel motion vectors, such as abrupt changes in the amplitude of endoscope motion between adjacent frames exceeding the normal range of the doctor's hand tremors. Simultaneously, the endoscope's tremors are subtle, continuous movements of the doctor's hand, with a continuous trajectory and no instantaneous pauses or reverse abrupt changes. Therefore, the kinematic constraints also include constraints on the continuous smoothness of the motion trajectory. By limiting the difference in endoscope motion vectors between adjacent frames (in this embodiment, the difference threshold is set to 0.1 pixels / frame), the kinematic constraints ensure that the fused motion trajectory conforms to the actual patterns of the doctor's hand tremors. The geometric constraints are designed to conform to the pixel space of the surgical image. The geometric constraints ensure the correlation of motion between adjacent pixels and correct optical flow drift errors in weak texture areas. In surgical images, pixels within the same local area belong to the same physical structure, and their motion states are highly consistent. There will be no situation where adjacent pixels move in opposite directions or have excessively different motion amplitudes. Therefore, the geometric constraints first clarify the consistency of local pixel motion and limit the difference in motion vectors between adjacent pixels within the same local area to no more than 0.1 pixels / frame. In this embodiment, a local window of 5×5 pixels is set to avoid the phenomenon of chaotic pixel motion vectors. At the same time, combined with the image characteristics of neurosurgical surgical scenes, the motion vectors of pixels in edge areas such as blood vessels and brain tissue contours should be adapted to the edge direction to avoid abnormal motion vectors perpendicular to the edge direction. Therefore, the geometric constraints also include edge pixel motion adaptation constraints. By identifying image edge features, the motion vectors of pixels in edge areas are guided, and the optical flow drift errors in weak texture areas are corrected to ensure that the fused motion vectors conform to both the pixel spatial distribution law and the actual physical structure motion characteristics of the surgical scene.

[0038] In some possible embodiments, continuing with the surgical example, the coarse motion estimation data T(10) at t=10, with displacements (0.0001mm, 0.00008mm) and angles (0.00003rad, 0.00002rad), reflects the overall jitter trajectory of the endoscope and serves as the motion prior input fusion filter. At the same time, kinematic constraints are incorporated to ensure that the fused motion trajectory conforms to the rigid body motion law of the endoscope, and geometric constraints are incorporated to ensure that the motion vector difference between adjacent pixels at the anastomosis does not exceed 0.1 pixels / frame, avoiding the situation where adjacent pixels move in opposite directions. Finally, the fusion filter is constructed.

[0039] Steps S1-6: Based on the fusion filter, scale recovery, global motion compensation and drift correction are performed on the dense motion field to output the initial mixed motion field.

[0040] Specifically, since the motion vector unit of the dense motion field is pixels, while the unit of the coarse motion estimation data is physical size, scale recovery is used to convert the pixel-level motion vector of the dense motion field into physical size motion vectors, ensuring that the scales of the two types of data are consistent. Furthermore, the overall jitter trajectory of the endoscope is used as a global motion reference to compensate for the global jitter component in the dense motion field, initially separating the endoscope jitter from the local motion. Finally, the kinematic and geometric constraints in the fusion filter are used to correct the motion vector drift error of the dense motion field in the weak texture region. After optimization, the initial mixed motion field M(t) of the preliminary fusion is output.

[0041] It should be noted that the scale recovery process includes obtaining the pixel-physical size mapping coefficient k in advance through an endoscope calibration experiment. The calibration process includes placing a standard scale plate in front of the endoscope lens, adjusting the lens focal length to the commonly used focal length in surgery, acquiring an image of the scale plate, measuring the number of pixels corresponding to the standard scale in the image, and calculating the mapping coefficient = actual size of the standard scale / number of corresponding pixels. In this embodiment, after calibration, k = 0.02 mm / pixel. Further, the motion vector of all pixels in the dense motion field is transformed using a linear mapping formula. The transformation formula is physical size motion vector = pixel-level motion vector × k. At the same time, the transformed physical size motion vector is normalized so that its numerical range is consistent with the numerical range of the coarse motion estimation data. In this embodiment, it is normalized to [0, 0.1 mm / frame].

[0042] In some possible embodiments, continuing with the surgical example, the dense motion field Flow(10) at t=10 is optimized, including converting the motion vectors of all pixels in Flow(10) into physical dimensions, for example, the motion vector of the bipolar electrocautery forceps pixel (1.5, 1.2) pixels / frame is converted to (0.03mm, 0.024mm) / frame; then global motion compensation is performed, using the overall endoscope jitter trajectory T(10) as a reference, to compensate for the global jitter components in Flow(10), such as brain tissue. The motion vector of the grooved pixel is (0.2, 0.1) pixels / frame. After compensation, the (0.1, 0.05) pixel / frame component caused by endoscope jitter is removed, and the remaining component caused by physiological motion is retained. Finally, drift correction is performed to correct the motion vector drift in the weak texture area in Flow(10). The original motion vector (0.5, 0.4) pixels / frame with large deviation is corrected to (0.3, 0.2) pixels / frame, so that it is consistent with the actual motion state. After optimization, the initial mixed motion field M(10) at t=10 is output.

[0043] Step S2: Perform surgical semantic segmentation on the original jittery image sequence and obtain a collaborative attention map by combining the physical motion mechanism. The collaborative attention map includes the collaborative attention value of each pixel.

[0044] In this embodiment, step S2 includes:

[0045] Step S2-1: Perform instrument instance segmentation, physiological motion region detection, and static structure inference on the original shaking image sequence to generate instrument mask, physiological motion mask, and static mask.

[0046] In this embodiment, step S2-1 includes:

[0047] Step S2-1-1: Perform instrument instance segmentation based on a real-time segmentation network optimized for surgical instruments.

[0048] Specifically, to accurately identify the position, outline, and category of all surgical instruments in the original jittery image sequence and distinguish between instrument and non-instrument regions, this step employs a real-time segmentation network optimized for neurosurgical instruments. This network, based on general instance segmentation networks such as Mask R-CNN, incorporates the shape characteristics of neurosurgical instruments for model optimization, such as the slender structure of bipolar electrocautery forceps and the tubular structure of suction devices, thereby improving the accuracy and real-time performance of instrument segmentation while avoiding misclassification of non-instrument regions such as blood vessels and brain tissue as instruments. After segmentation, an instrument mask is generated, where regions with a pixel value of 1 represent instrument regions and regions with a pixel value of 0 represent non-instrument regions.

[0049] It should be noted that the real-time segmentation network in this embodiment includes an input layer: receiving single-frame images from the original jittery image sequence, maintaining the original acquisition size, and simultaneously normalizing the input image to normalize the pixel grayscale values ​​to the [0,1] range; a feature extraction layer: using ResNet-18 with lightweight optimization as the feature extraction layer, containing four residual stages, each consisting of two residual blocks, each containing a 3×3 convolutional layer, a BN layer, a ReLU activation function, and residual connections. The shallow residual stages in stages 1-2 mainly extract low-dimensional image features, such as instrument edges and grayscale changes, thereby adapting to the initial capture of the slender edges of neurosurgical instruments; the deep residual stages in stages 3-4... The feature map size is compressed using a convolution operation with a stride of 2, extracting high-dimensional features from the image, such as the outline of the instrument and local details. Simultaneously, an SE attention module is introduced in the deep residual stage to strengthen the feature weights of the instrument region. The feature fusion layer employs a feature pyramid network structure to fuse multi-scale feature maps, i.e., the output feature maps from the four residual stages, addressing the problem of shallow features having high resolution but weak semantic information, and deep features having strong semantic information but low resolution. Feature fusion upsamples deep high-semantic features to the size of shallow feature maps and lateral connections concatenate the shallow high-resolution features with the upsampled deep features, generating fused feature maps at four scales: 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input image, ensuring accurate capture of the instrument's dimensions. It accurately identifies instrument categories and overall contours even with subtle edges; the region proposal layer generates region proposals based on the 1 / 16 scale feature map in the fused feature map, using a 3×3 sliding window to traverse the fused feature map. Each sliding window generates 9 anchor boxes. By judging whether the anchor box contains an instrument classification branch and correcting the anchor box coordinates through a regression branch, candidate regions with a confidence score greater than or equal to 0.7 are selected, and background candidate regions are eliminated to reduce the computational load of subsequent instance segmentation; the instance segmentation head layer includes a category classification branch, a bounding box regression branch, and a mask generation branch. The category classification branch uses a 1×1 convolutional layer and a Softmax activation function to output the category probability of each candidate region, ensuring accurate identification of commonly used surgical instruments; The bounding box regression branch uses a 1×1 convolutional layer to output the bounding box coordinate offset of each candidate region, i.e., the bounding box regression error, to further correct the candidate region coordinates so that the bounding box accurately fits the instrument contour. The mask generation branch uses a transposed convolutional layer to generate a binary mask of the instrument corresponding to the candidate region. In the mask, a pixel value of 1 represents an instrument region, and a pixel value of 0 represents a non-instrument region. At the same time, a detail correction module is introduced into the mask generation branch to correct the contours of fine areas such as the instrument tip and rod. The output layer integrates the output results of the instance segmentation head layer and filters candidate regions with a classification confidence greater than or equal to 0.7 and a bounding box regression error less than or equal to 2 pixels. The filtered bounding boxes are matched with the corresponding masks to generate an instrument mask.

[0050] In some possible embodiments, continuing the example of cortical vascular anastomosis, the main instruments used in the surgery are bipolar electrocoagulation forceps and a suction device. The original jitter image I(10) at t=10 is input into the optimized real-time segmentation network. The network quickly identifies the bipolar electrocoagulation forceps and the suction device in the image and accurately segments their contours to generate an instrument mask. In this instrument mask, the pixel value of the area corresponding to the bipolar electrocoagulation forceps and the suction device is 1, and the pixel value of non-instrument areas such as brain tissue and anastomotic vessels is 0.

[0051] Step S2-1-2: Detect physiological motion regions based on temporal analysis and prior shape. The temporal analysis includes inter-frame difference and phase change. The prior shape is tubular. The physiological motion regions include, but are not limited to, vascular pulsation areas and tissue respiration fluctuation areas.

[0052] Specifically, regions with periodic physiological movements in the original jittery image sequence are identified. These movements are normal physiological phenomena, such as vascular pulsation areas and brain tissue respiratory fluctuation areas. They do not require complete suppression but require moderate image stabilization and therefore need to be distinguished from static and instrument-related areas. This step uses temporal analysis combined with prior shape for detection. Temporal analysis includes inter-frame difference and phase change. Inter-frame difference captures regions with periodic movements by calculating the change in pixel grayscale between adjacent frames. Phase change identifies the frequency of periodic movements by analyzing the temporal phase characteristics of pixel grayscale, such as the frequency of vascular pulsation matching the heart rate, and the frequency of brain tissue respiratory fluctuation matching the respiratory rate. The prior shape is tubular, primarily targeting vascular pulsation areas, thereby improving the detection accuracy of vascular pulsation areas and avoiding misjudgment of non-tubular physiological movement areas. After detection, a physiological movement mask is generated. Regions with a pixel value of 1 in the mask represent physiological movement areas, and regions with a pixel value of 0 represent non-physiological movement areas.

[0053] In some possible embodiments, continuing with the surgical example described above, physiological motion region detection is performed on the original jittery image sequence from t=8 to t=12. Specifically, this process includes calculating the pixel grayscale change between adjacent frames using inter-frame difference, such as t=10 and t=11. It is found that the grayscale change in the middle cerebral artery region and the brain tissue surface region around the anastomosis is significantly greater than in other regions, initially identifying them as physiological motion regions. Through phase change analysis, it is determined that the motion frequency of this region is 1.2Hz, consistent with the patient's heart rate, confirming the vascular pulsation region. Simultaneously, it is found that the motion frequency of the brain tissue surface region is 0.3Hz, consistent with the patient's respiratory rate, confirming the brain tissue respiratory fluctuation region. Combined with the tubular prior shape, the tubular outline of the middle cerebral artery is accurately identified, further locking down the vascular pulsation region. Finally, a physiological motion mask is generated, in which the pixel value of the vascular pulsation region and the brain tissue respiratory fluctuation region is 1.

[0054] Step S2-1-3: Perform static structure inference based on the spatiotemporal consistency of the surgical scenario.

[0055] Specifically, the original jittery image sequence contains background tissue regions with almost no motion or minimal motion. The motion in these regions mainly comes from endoscopic jitter and needs to be completely suppressed and strongly stabilized. Therefore, they need to be distinguished from physiological motion regions and instrument regions. This step is based on the spatiotemporal consistency of the surgical scene. The spatiotemporal consistency includes spatial consistency, that is, in the same image, the motion states of adjacent pixels are correlated, and adjacent pixels in static regions have no obvious motion; and temporal consistency, that is, in different time frames, the position and contour of static regions change very little, and the motion amplitude is much smaller than the amplitude of endoscopic jitter. By analyzing the spatiotemporal consistency of the original jittery image sequence, after excluding instrument regions and physiological motion regions, the background tissue regions that are most likely to be static or quasi-static in the current operation stage are inferred, and a static mask is generated. In the mask, regions with a pixel value of 1 represent static regions, and regions with a pixel value of 0 represent non-static regions.

[0056] In some possible embodiments, continuing the example of cortical vascular anastomosis, a spatiotemporal consistency analysis was performed on the original jitter image sequence from t=8 to t=12. Spatially, in the skull base region at the image edge, the grayscale change of adjacent pixels was extremely small, almost zero, which is consistent with spatial consistency. Temporally, in the 5 frames of images from t=8 to t=12, the position and contour of this skull base region remained almost unchanged, with a motion amplitude of only 0.05 pixels / frame, far less than the amplitude of endoscopic jitter, which is consistent with temporal consistency. At the same time, the possibility that this region is an instrument region or a physiological motion region was ruled out, and it was inferred that it was a static region, in which the pixel value of the skull base and non-operational brain tissue was 1.

[0057] Step S2-2: Based on the instrument mask, physiological motion mask, and static mask, combined with external ECG physiological signals and physical motion mechanisms, a collaborative attention map is generated.

[0058] The rules for generating the collaborative attention map include assigning high attention values ​​to the static region corresponding to the static mask, attention values ​​to the physiological motion region corresponding to the physiological motion mask, and low attention values ​​to the instrument region corresponding to the instrument mask. If there is regional overlap, the attention values ​​of each region are fused, and the attention values ​​of the physiological motion region are periodically modulated in conjunction with the external ECG physiological signal.

[0059] Specifically, the collaborative attention map quantifies the image stabilization requirement of each pixel, i.e., the probability that the pixel's motion belongs to the background jitter that needs stabilization. The value range is [0,1], denoted as A(x,y,t), where x and y are pixel coordinates, and t is the temporal frame index. The generation process is based on instrument masks, physiological motion masks, and static masks, combined with physical motion mechanisms, i.e., the inherent motion characteristics of each region, and external ECG physiological signal modulation. According to the physical motion mechanism, static regions only have endoscopic jitter and require strong image stabilization, thus being assigned a high attention value; physiological motion regions have peripheral jitter. Periodic physiological movements require moderate image stabilization and are assigned a medium attention value; active manipulation movements in the instrument area do not require excessive image stabilization and are assigned a low attention value; if the three types of areas overlap, such as when the instrument and blood vessel partially overlap, the attention values ​​corresponding to each area are weighted and fused, and the fused value is taken as the attention value of the overlapping area; at the same time, the periodicity of vascular pulsation is reflected by the external ECG physiological signal, and the attention value of the physiological movement area is periodically modulated in combination with this signal, so that the attention value changes synchronously with the vascular pulsation frequency, thereby adapting to the periodic characteristics of physiological movement.

[0060] It should be noted that if there is regional overlap, the weight allocation principle for each type of overlapping region is determined based on the priority of the motion characteristics of each region. This includes prioritizing the retention of active instrument motion, which is represented by the lowest weight and the lowest corresponding attention value weight; moderate retention of physiological motion, which is represented by the medium weight and the medium corresponding attention value weight; and priority suppression of endoscopic tremor, which is represented by the highest weight of static regions and the highest corresponding attention value weight. Secondly, fixed base weights are set: 0.4 for instrument regions, 0.6 for physiological motion regions, and 0.7 for static regions. If two types of regions overlap, the base weights for the corresponding two types of regions are used; if three types of regions overlap, the base weights for all three types of regions are used. All weights involved in the fusion process must be normalized, and the sum of the normalized weights is 1. Finally, a weighted summation formula is used to calculate the fused attention value.

[0061] Furthermore, the modulation method includes: acquiring the patient's real-time electrocardiogram (ECG) signal using an external ECG sensor; preprocessing the ECG signal; extracting the R-wave peak value of the ECG signal; calculating the patient's real-time heart rate f; and obtaining the vascular pulsation cycle T, which is completely synchronized with the vascular pulsation cycle; secondly, setting the modulation amplitude, which is 0.05 in this embodiment, to control the fluctuation range of the attention value, ensuring that the fluctuation amplitude is moderate, adapting to the changes in the strength of vascular pulsation, and avoiding excessive fluctuations that would lead to unstable image stabilization; finally, periodically modulating the attention value of the physiological motion region using a sinusoidal modulation formula, the modulation formula being Amodulation(x,y,t) = Ainitial(x,y,t) + 0.05sin(2πt / T), where Ainitial(x,y,t) is the attention value of the physiological motion region before modulation, t is the time corresponding to the current time frame, and T is the vascular pulsation cycle; during the modulation process, only the pixels corresponding to the physiological motion mask are modulated, while pixels in the static region, instrument region, and overlapping region that are not dominated by physiological motion are not modulated.

[0062] In some possible embodiments, continuing the example of cortical vascular anastomosis, a collaborative attention map A(x,y,10) is generated based on the three types of masks at t=10, the physical motion mechanism, and the external ECG physiological signal. According to the generation rules, the static mask, corresponding to the skull base and non-operational brain tissue (i.e., static areas), requires strong image stabilization and is therefore assigned a high attention value of 0.9. The physiological motion mask, corresponding to the vascular pulsation area and the brain tissue respiratory fluctuation area (i.e., physiological motion areas), requires moderate image stabilization and is therefore assigned a medium attention value of 0.5. The instrument mask, corresponding to the bipolar electrocoagulation forceps and suction device (i.e., instrument areas), requires preservation of active motion and is therefore assigned a low attention value. A low attention value of 0.1 was assigned. The bipolar electrocoagulation forceps tip and the pulsating vascular area partially overlapped in the image. The attention values ​​of this overlapping area were weighted and fused, resulting in a final attention value of 0.1 × 0.4 + 0.5 × 0.6 = 0.34. Simultaneously, an external ECG physiological signal captured a patient heart rate of 60 beats / minute and a pulsating vascular frequency of 1 Hz. This signal was used to periodically modulate the attention value of the pulsating vascular area, causing the attention value to change synchronously with the heart rate between 0.45 and 0.55, adapting to the periodicity of pulsation. This ultimately generated a complete collaborative attention map, with each pixel having a corresponding collaborative attention value.

[0063] In step S2-2, the method further includes applying rigid body kinematic constraints to the instrument region corresponding to the instrument mask, and setting a first deviation threshold and a second deviation threshold.

[0064] The motion vector is compared with the historical motion trajectory to obtain the first deviation, and the motion vector is compared with the coarse motion estimation data corresponding to the inertial data to obtain the second deviation.

[0065] If either the first deviation is greater than the first deviation threshold or the second deviation is greater than the second deviation threshold, the movement in the instrument area is determined to be non-instrument movement, and movement correction is performed simultaneously.

[0066] Specifically, the motion correction in this step aims to avoid misjudging extreme endoscope jitter as active movement of surgical instruments, ensuring that the movement of the instrument area only reflects the surgeon's active manipulation. First, rigid body kinematic constraints are applied to the instrument area corresponding to the instrument mask. Because surgical instruments are rigid structures, their movement is translation and rotation, and they are rigid body movements without significant deformation. The movement trajectory is continuous and will not involve sudden, violent movements. Then, two independent deviation thresholds are set: the first deviation threshold λ1 is used to compare the motion vector with the historical motion trajectory; the second deviation threshold λ2 is used to compare the motion vector with coarse motion estimation data. The two deviations are calculated separately. If either deviation is greater than the corresponding threshold, the movement of the current instrument area is determined to be non-instrument movement, i.e., extreme endoscope jitter. A model-based prediction algorithm, such as the Kalman prediction algorithm, is then used for motion correction to correct the misjudged jitter motion vector to a vector that conforms to the active movement law of the instrument.

[0067] Furthermore, in this step, the first deviation threshold λ1 is set to 0.8 mm / frame, and the second deviation threshold λ2 is set to 0.5 mm / frame. The setting process includes, based on the actual operating speed characteristics of instruments in neurosurgical procedures, this embodiment targets cortical vascular anastomosis. Commonly used instruments in surgery, such as bipolar electrocoagulation forceps and suction devices, require precise control of force and speed during active operation by the surgeon to avoid damage to brain tissue and blood vessels. The active operating speed of these instruments typically does not exceed 0.6 mm / frame, and the maximum instantaneous operating speed does not exceed 0.7 mm / frame. Considering the slight operational fluctuations that may occur during surgery, such as those caused by the surgeon's hand... For subtle changes in force, a margin of error is reserved, and λ1 is set to 0.8 mm / frame. This effectively distinguishes between active instrument operation and extreme shaking, and avoids misjudgment. Based on the fluctuation range of the coarse motion estimation data and the actual amplitude of the endoscope shaking, after denoising and zero-bias correction in steps S1-2, the fluctuation range of the coarse motion estimation data is controlled within 0.3-0.4 mm / frame, meaning that the maximum amplitude of normal endoscope shaking does not exceed 0.4 mm / frame. At the same time, considering the slight fluctuations in inertial data that may be caused by the operation of electrocoagulation instruments and tissue contact during surgery, a margin of error is reserved, and λ2 is finally set to 0.5 mm / frame.

[0068] In some possible embodiments, continuing the above surgical example, rigid body kinematic constraints are applied to the bipolar electrocoagulation forceps region corresponding to the instrument mask at t=10 to ensure that its motion vector conforms to the rigid body motion law and avoids deformation-type motion vectors; at the same time, the first deviation threshold λ1 is set to 0.8 mm / frame, and the second deviation threshold λ2 is set to 0.5 mm / frame; the motion vector of the bipolar electrocoagulation forceps in the current frame is 0.9 mm / frame, and the average motion vector of the historical motion trajectory is 0.4 mm / frame. The historical motion trajectory is represented as frames from t=5 to t=9. The first deviation is |0.9-0.4|=0.5 mm / frame, which is less than λ1; the endoscopic jitter vector corresponding to the coarse motion estimation data T(10) in the current frame is 0.7 mm / frame, and the second deviation is |0.9-0.7|=0.2 mm / frame, which is less than λ2. Therefore, the motion is determined to be... If the movement is active, no correction is made. Assuming that at t=15, due to a sudden and severe tremor in the doctor's hand, the motion vector of the bipolar electrocautery forceps becomes 1.2 mm / frame, the average historical motion trajectory is 0.45 mm / frame, the first deviation is |1.2-0.45|=0.75 mm / frame, which is less than λ1, the tremor vector corresponding to the coarse motion estimation data T(15) is 1.1 mm / frame, the second deviation is |1.2-1.1|=0.1 mm / frame, which is less than λ2, and it is still judged as active movement. If at t=20, the motion vector becomes 1.5 mm / frame, the average historical trajectory is 0.5 mm / frame, the first deviation is 1.0 mm / frame, which is greater than λ1, then it is judged as non-instrument movement, the Kalman prediction algorithm is used to correct the motion vector to 0.6 mm / frame, which is in line with the active operation law of the instrument and avoids misjudgment of tremor.

[0069] Step S3: Obtain the spatiotemporal adaptive filtering kernel based on the collaborative attention map, and filter the initial mixed motion field based on the spatiotemporal adaptive filtering kernel to obtain the stable motion field.

[0070] In this embodiment, step S3 includes:

[0071] Step S3-1: Based on the collaborative attention map and the initial mixed motion field, generate a spatiotemporal adaptive filtering kernel for each pixel and each temporal frame in the original jittery image sequence.

[0072] The size, shape, and dimensions of the spatiotemporal adaptive filter kernel are adaptively modulated according to the collaborative attention map and the initial mixed motion field.

[0073] In this embodiment, step S3-1 includes:

[0074] Step S3-1-1: In the static region, a 6×6 spatiotemporal kernel is used to perform a powerful smooth motion.

[0075] Specifically, the image stabilization requirement for the static region is to strongly suppress endoscopic jitter. Since there is no physiological or instrumental motion in the static region, only endoscopic jitter exists. Therefore, a large spatiotemporal kernel is needed for powerful motion smoothing. In this embodiment, a 6×6 spatiotemporal kernel is used for the static region. This size kernel can cover more adjacent pixels. By powerfully smoothing the motion vector of this region in the initial mixed motion field, the high-frequency jitter and low-frequency drift of the endoscope are effectively filtered out, making the motion vector of the static background region tend to be stable. This ensures that the static background is clear and jitter-free in the subsequently generated stable image, providing the doctor with a stable surgical field reference. At the same time, the 6×6 size balances smoothing effect and real-time performance, avoiding increased processing delay due to excessive size.

[0076] In some possible embodiments, continuing the example of cortical vascular anastomosis, in the collaborative attention map at t=10, the attention value of the static region, the skull base region, is 0.9. A 6×6 spatiotemporal adaptive filtering kernel is generated for each pixel in this region. In the initial mixed motion field, the motion vectors in this region are mainly small vectors per pixel / frame caused by endoscopic jitter. These motion vectors are strongly smoothed by the 6×6 spatiotemporal kernel, correcting the scattered, small-amplitude fluctuating motion vectors into stable vectors. For example, the original motion vector of a certain pixel (x=100, y=200) in this region is (0.25, 0.22) pixels / frame. After smoothing with the 6×6 spatiotemporal kernel, the motion vector is corrected to (0.20, 0.20) pixels / frame, tending to be stable, ensuring that the stable frames generated in this region are jitter-free.

[0077] Step S3-1-2: In the instrument area, the original instrument movement is maintained using a 1×1 identity kernel or a 2×2 extremely small empty kernel.

[0078] Specifically, the requirement for the instrument area is to preserve the doctor's active manipulation movements and avoid excessive image stabilization. The movement in the instrument area mainly consists of the doctor's active manipulation of the instruments. The endoscopic jitter component has been preliminarily separated through initial fusion processing. Therefore, a very small spatiotemporal kernel is needed to preserve the original instrument motion vector to the maximum extent. In this embodiment, the instrument area uses a 1×1 identity kernel or a 2×2 extremely small spatiotemporal kernel: the 1×1 identity kernel does not perform any smoothing on the motion vector, completely preserving the original instrument motion vector, and is suitable for areas such as the instrument tip where precise preservation of motion details is required; the 2×2 extremely small spatiotemporal kernel slightly smooths the motion vector, filtering out only very weak residual jitter, and is suitable for areas such as the instrument shaft where the requirements for motion details are slightly lower. The choice of the two kernels is adaptively adjusted according to the specific location of the instrument area to ensure the authenticity and accuracy of the instrument movement.

[0079] In some possible embodiments, continuing with the surgical example above, in the collaborative attention map at t=10, the attention value of the bipolar electrocoagulation forceps and suction device in the instrument area is 0.1. A 1×1 identity kernel is used for the tip area of ​​the bipolar electrocoagulation forceps. The original motion vector of this area is (1.5, 1.2) pixels / frame. After processing with the identity kernel, the motion vector remains unchanged, completely preserving the precise trajectory of the doctor's manipulation of the instrument. A 2×2 extremely small spaced kernel is used for the bar area of ​​the bipolar electrocoagulation forceps, which has slightly lower detail requirements. The original motion vector of this area is (1.0, 0.8) pixels / frame, which includes 0.1 pixels / frame of residual endoscopic jitter. After slight smoothing with the 2×2 extremely small spaced kernel, the motion vector is corrected to (0.95, 0.75) pixels / frame. While filtering out residual jitter, the active motion trajectory of the instrument is completely preserved, ensuring that in subsequent stable images, the instrument movement is clear and precise, and completely synchronized with the doctor's operation.

[0080] In step S3-1-3, in the physiological motor region, the size and smoothing intensity of the spatiotemporal adaptive filter kernel change linearly and continuously with the collaborative attention value.

[0081] Specifically, the physiological motion region requires moderate image stabilization, which suppresses endoscopic jitter while preserving periodic physiological motion. The motion in the physiological motion region includes both endoscopic jitter and periodic physiological motion components. Therefore, the parameters of the filter kernel need to be adaptively adjusted according to the image stabilization requirements to achieve on-demand smoothing. In this embodiment, the size and smoothing intensity of the spatiotemporally adaptive filter kernel change linearly and continuously with the co-attention value. The closer the co-attention value is to 0.8, i.e., the closer to the static region, the larger the kernel size, not exceeding 5×5, and the stronger the smoothing intensity, which is used to suppress more endoscopic jitter. The closer the co-attention value is to 0.2, i.e., the closer to the instrument region, the smaller the kernel size, not less than 3×3, and the weaker the smoothing intensity, which is used to preserve more physiological motion. When the co-attention value is 0.5, the kernel size is 4×4, and the smoothing intensity is moderate, achieving a balance between jitter suppression and physiological motion preservation.

[0082] In some possible embodiments, continuing with the above surgical example, in the collaborative attention map at t=10, the attention values ​​of the vascular pulsation area and the brain tissue respiratory fluctuation area are between 0.34 and 0.55, and the collaborative attention value is between 0.2 and 0.8. The size of the filter kernel and the smoothing intensity change linearly and continuously with this attention value. For example, the collaborative attention value of a certain pixel (x=300, y=400) in the vascular pulsation area is 0.5, corresponding to a kernel size of 4×4 and moderate smoothing intensity. In the initial mixed motion field M(10), the motion vector of this pixel is (0.3, 0.2) pixels / frame. After processing by the 4×4 filter kernel, the motion vector is corrected to (0.18, 0.12) pixels / frame, effectively suppressing most of the endoscope jitter while preserving the physiological motion of vascular pulsation. The attention value of another pixel (x=310, y=410) in the same vascular pulsation area is 0.7, corresponding to... The kernel size is 5×5 with strong smoothing intensity. The original motion vector is (0.35, 0.25) pixels / frame. After filtering, the motion vector is corrected to (0.15, 0.10) pixels / frame, suppressing more jitter. On the other hand, the attention value of a certain pixel in the respiratory fluctuation area of ​​brain tissue is 0.3, corresponding to a kernel size of 3×3 with weak smoothing intensity. The original motion vector is (0.25, 0.15) pixels / frame. After filtering, the motion vector is corrected to (0.20, 0.12) pixels / frame, preserving more respiratory fluctuation motion and achieving on-demand image stabilization.

[0083] Step S3-1-4: Modulate the orientation kernel shape of the spatiotemporal adaptive filter kernel based on the local consistency of the initial mixed motion field.

[0084] Specifically, the local consistency of the initial mixed motion field is represented by pixel motion vectors within the same local area having similar directions and amplitudes. For example, the pixel motion direction in the pulsating blood vessel area is along the direction of the blood vessel, and the pixel motion direction in the static area is consistent. The direction and shape of the spatiotemporal adaptive filter kernel are modulated through the local consistency. This includes modulating the direction of the filter kernel to match the direction of the motion vector in a certain local area, such as the linear motion along the direction of the blood vessel in the pulsating blood vessel area, and modulating the shape to be elongated to avoid blurring the blood vessel outline during smoothing. If the pixel motion vector in a certain local area has no fixed direction, such as the random jitter in the static area, the filter kernel is modulated to be circular to achieve uniform smoothing in all directions and ensure jitter suppression effect. This step optimizes the direction and shape of the filter kernel so that the filter kernel can adapt to the local consistency of pixel motion in the initial mixed motion field, improve the filtering effect, and avoid motion blur or loss of detail.

[0085] In some possible embodiments, continuing with the surgical example above, in the initial mixed motion field at t=10, the pixel motion vector direction of the vascular pulsation area is along the direction of the middle cerebral artery, roughly horizontal, with strong local consistency. Based on this consistency, the direction of the filter kernel in this area is modulated to be horizontal, and the shape is modulated to be elongated, such as a 4×1 elongated kernel, to adapt to horizontal motion. After processing by this filter kernel, the jitter along the direction of the blood vessel is effectively suppressed, while the contour perpendicular to the direction of the blood vessel remains sharp, avoiding blurring of the blood vessel contour; the image of the static background area The pixel motion vectors have no fixed direction, and local consistency is characterized by similar amplitudes and chaotic directions. Modulating the filter kernel in this area into a circle, such as a 6×6 circular kernel, achieves all-round uniform smoothness, effectively filtering out random jitter and making the static background clearer. The pixel motion vectors in the brain tissue respiratory fluctuation area are roughly vertical, with strong local consistency. Modulating the filter kernel direction to vertical and the shape to a slender shape, such as a 1×4 slender kernel, not only suppresses vertical endoscopic jitter but also preserves the vertical motion details of brain tissue respiratory fluctuations, improving the filtering effect.

[0086] Step S3-2: Perform convolution filtering on the initial mixed motion field based on the spatiotemporal adaptive filtering kernel to obtain a stable motion field.

[0087] The convolutional filtering is represented as an anisotropic diffusion process modulated by a collaborative attention map, and the diffusion coefficient is controlled by the collaborative attention map.

[0088] Specifically, the convolutional filtering process is essentially an anisotropic diffusion process modulated by a collaborative attention map. This anisotropic diffusion means that the diffusion intensity adapts to the pixel's motion state and image stabilization requirements, rather than a uniform diffusion in all directions. This allows for smoothing jitter while preserving regional edges and details. The diffusion coefficient is controlled by the collaborative attention map. The higher the attention value, the larger the diffusion coefficient, the stronger the diffusion intensity, and the better the jitter suppression effect. The lower the attention value, the smaller the diffusion coefficient, the weaker the diffusion intensity, and the better the motion preservation effect. With a medium attention value, the diffusion coefficient changes linearly with the attention value, thus achieving a balance between jitter suppression and motion preservation. After convolutional filtering, a stable motion field is output, which contains only a small number of instrument and physiological motion components that need to be preserved, effectively suppressing endoscopic jitter components.

[0089] It should be noted that the convolutional filtering process includes: for each pixel in the initial mixed motion field, calling the spatiotemporal adaptive filtering kernel generated in step S3-1 corresponding to that pixel, and simultaneously reading the collaborative attention value corresponding to that pixel to determine the corresponding diffusion coefficient D(x,y,t). The diffusion coefficient is linearly positively correlated with the attention value, ensuring that the higher the attention value, the larger the diffusion coefficient. Specifically, the correspondence is D(x,y,t)=A(x,y,t); with the current pixel as the center, selecting the local neighbor pixel set of that pixel in the initial mixed motion field according to the size of the filtering kernel. The neighborhood range is perfectly matched to the filter kernel size. For example, a 6×6 filter kernel corresponds to 6×6 neighboring pixels, while a 1×1 identity kernel only selects the current pixel itself. The motion vector of the neighboring pixels is denoted as M(x+i,y+j,t), where i and j are the coordinate offsets of the neighboring pixels relative to the center pixel, and their values ​​are determined by the kernel size. The weight value of each kernel element in the filter kernel K(x+i,y+j,t) is multiplied by the motion vector of the corresponding neighboring pixel, and then all the weighted multiplication results are summed to obtain the preliminary motion vector M'(x,y,t) of the current pixel after convolution filtering. Specifically, M'(x,y,t) = Σ[K(x+i,y+j,t)×M(x+i,y+j,t)×D(x,y,t)] is used. The motion vector M'(x,y,t) after initial filtering is subjected to component separation. Combining the motion characteristics of the pixel in the initial mixed motion field, the endoscopic jitter component and the motion component to be retained are further separated. If it is a static region, M'(x,y,t) is directly retained; if it is an instrument region, only the extremely weak residual jitter in M'(x,y,t) is fine-tuned to retain the original instrument motion vector to the maximum extent; if it is a physiological movement... For each region, amplitude calibration is performed on M'(x,y,t) to ensure that the amplitude of the physiological motion component conforms to actual physiological laws after filtering jitter, such as matching the amplitude of vascular pulsation with the attention value after ECG modulation, avoiding excessive smoothing that leads to physiological motion distortion. After traversing all pixels in the initial chaotic motion field and completing the above convolutional filtering calculation, global consistency calibration is performed on the filtered motion vectors of all pixels to ensure that the motion vector transition between adjacent pixels is natural and to avoid abrupt changes in motion vectors at region boundaries, such as at the junction of static regions and physiological motion regions, further improving the reliability of the stable motion field.

[0090] In some possible embodiments, continuing the above surgical example, the initial mixed motion field at t=10 is subjected to convolutional filtering. The filtering kernel is the spatiotemporal adaptive filtering kernel generated in step S3-1, and the diffusion coefficient is controlled by the collaborative attention map. The diffusion coefficient of the static background region is the largest, which is set to 0.9, consistent with the collaborative attention value. After convolutional filtering, the motion vector of this region is strongly smoothed, the jitter component is almost completely suppressed, and the output stable motion vector tends to be stable. The diffusion coefficient of the instrument region is the smallest, which is set to 0.1, consistent with the collaborative attention value. After filtering, the original instrument motion vector is almost completely preserved, and only a very weak residual jitter is filtered out. The diffusion coefficient of the physiological motion region changes linearly with the attention value, that is, it changes linearly with the collaborative attention value of 0.34-0.55 by 0.34-0.55. After filtering, the endoscope jitter component is moderately suppressed, and the physiological motion component is preserved. For example, the motion vector of the vascular pulsation region is corrected from (0.3,0.2) pixels / frame to (0.18,0.12) pixels / frame, which both suppresses jitter and preserves vascular pulsation.

[0091] Step S4: Perform image deformation and interpolation synthesis on the stable motion field to obtain a stable image sequence.

[0092] In this embodiment, step S4 includes:

[0093] Step S4-1: Obtain the deformation field from the original jittered frame to the stable frame based on the stable motion field. The original jittered frame is represented as a single frame image in the original jittered image sequence, and the stable frame is represented as a single frame image corresponding to the single frame image in the original jittered image sequence.

[0094] Specifically, the core objective of this step is to generate a deformation field. This deformation field describes the pixel position mapping relationship from the original jittery frame to the stable frame, that is, where each pixel in the original jittery frame needs to be moved in the stable frame to correct the jitter deviation and achieve image stabilization. Here, the original jittery frame is a single frame image in the original jittery image sequence, containing pixel position offsets caused by endoscope jitter. The stable frame is the single frame stable image to be generated corresponding to the original jittery frame, which is the target image after jitter correction. The motion vector of each pixel in the stable motion field reflects the jitter offset that needs to be corrected for that pixel. The deformation field obtains the target coordinates of each pixel in the original jittery frame in the stable frame by integrating and mapping these motion vectors, forming a complete position mapping relationship, ensuring that the original jittery frame can be corrected into a stable frame through subsequent deformation processing.

[0095] Understandably, the process of generating the deformation field includes: firstly, preprocessing all pixel motion vectors in the stable motion field to remove abnormal motion vectors, such as vectors exceeding a reasonable range due to local filtering deviations. In this embodiment, the reasonable range is set to [-0.5, 0.5] pixels / frame. Vectors exceeding this range are replaced by the average value of neighboring pixel motion vectors to ensure the reliability of the motion vectors. Simultaneously, the preprocessed motion vectors are smoothed using a 3×3 extremely small space kernel. Then, inter-frame integration is performed on the preprocessed stable motion field to accumulate the motion vectors of adjacent frames. The motion vector deviation is used to obtain the total jitter offset Δ(x,y,t) of each pixel from the initial frame to the current frame. The integral calculation uses the summation formula Δ(x,y,t)=Δ(x,y,t-1)+Ms(x,y,t), where Δ(x,y,t-1) is the total jitter offset of the pixel in the previous frame, and Ms(x,y,t) is the stable motion vector of the pixel in the current frame. The total jitter offset of the initial frame t=1 is Δ(x,y,1)=Ms(x,y,1). The accumulated jitter offset Δ(x,y,t) is used to establish the original jitter frame pixel coordinates (x,y). The mapping relationship between the target coordinates (x', y') and the stable frame is given by the following formulas: x' = x - Δx(x, y, t), y' = y - Δy(x, y, t), where Δx(x, y, t) is the component of the cumulative jitter offset Δ(x, y, t) in the x-axis direction, and Δy(x, y, t) is the component of the cumulative jitter offset Δ(x, y, t) in the y-axis direction. The minus sign indicates reverse correction, that is, according to the jitter offset direction, the pixel is moved in the opposite direction by the corresponding distance to cancel the positional offset caused by jitter. After completing the coordinate mapping of all pixels, the preliminary deformation field is obtained. Since the mapped target coordinates (x', y') may exceed the image range of the stable frame (e.g., x' is less than 0, x' is greater than the image width, y' is less than 0, y' is greater than the image height), a boundary padding strategy can be adopted. This involves copying the mapping relationship of boundary pixels to fill in the coordinates that exceed the range, ensuring that the deformation field covers all pixel positions of the stable frame. At the same time, the deformation field is normalized by using a linear interpolation algorithm to correct non-integer deviations in the mapped coordinates, ensuring that the coordinate mapping of the deformation field is smooth and continuous, and avoiding problems such as jagged edges and breaks in subsequent image deformation.

[0096] In some possible embodiments, continuing the example of cortical vascular anastomosis, the original jitter frame at t=10 is shown. In this image, due to endoscopic jitter, the anastomotic vessel contour is slightly offset, and the actual position of a certain pixel (x=200, y=300) on the vessel contour is offset by 0.2 pixels / frame. The stable motion field at t=10 is Ms(10), and the stable motion vector corresponding to this pixel is (0.18, 0.12) pixels / frame, reflecting the jitter offset that needs to be corrected for this pixel. Based on Ms(10), a deformation field is generated. By integrating and mapping this motion vector, the target coordinates of pixel (200, 300) in the original jitter frame in the stable frame are obtained as (200-0.18, 300-0.12) = (199.82, 299.88). Similarly, for each pixel in the original jitter frame, the deformation field is mapped to the corresponding position in the stable frame to form a deformation field.

[0097] Step S4-2: Perform interpolation processing on the deformation field based on the differentiated interpolation strategy.

[0098] Specifically, edge-preserving interpolation is used for the instrument region corresponding to the instrument mask, while bilinear interpolation is used for the non-instrument region.

[0099] Specifically, since the target coordinates generated in the deformation field may be non-integer pixels, while the image pixel coordinates must be integers, interpolation processing is required for the deformation field to calculate the pixel grayscale values ​​corresponding to the non-integer coordinates, ensuring that the subsequently synthesized stable frames are clear, jagged, and blur-free. This step adopts a differentiated interpolation strategy, selecting an appropriate interpolation algorithm based on the region type to achieve on-demand interpolation: for the instrument region corresponding to the instrument mask, an edge-preserving interpolation algorithm, such as a bilateral interpolation algorithm, is used to preserve the edge contour and details of the instrument, avoiding blurring of the instrument contour and edge blurring caused by interpolation, ensuring that the instrument is clearly distinguishable in the stable frame; for non-instrument regions, a bilinear interpolation algorithm is used, which can quickly calculate the pixel grayscale values ​​of non-integer coordinates while ensuring the smoothness of the image, adapting to the real-time image stabilization requirements of surgery.

[0100] In some possible embodiments, continuing with the surgical example above, interpolation processing is performed on the deformation field at t=10. For the bipolar electrocoagulation forceps region corresponding to the instrument mask, a bilateral interpolation algorithm is used. The target coordinates of the deformation field in this region contain a large number of non-integer coordinates, such as the target coordinates of the instrument tip pixel (250.3, 350.7). When calculating the pixel grayscale value corresponding to these non-integer coordinates using the bilateral interpolation algorithm, not only are the grayscale values ​​of adjacent pixels considered, but also the edge features of the pixels are taken into account. This ensures that after interpolation, the metal edge contour of the bipolar electrocoagulation forceps is clear and unblurred, the tip details are intact, and it is consistent with the original instrument contour. For non-instrument regions, such as the static area of ​​the skull base... For the regions and physiological motion regions of blood vessel pulsation, a bilinear interpolation algorithm is used. For example, the target coordinates of the static region pixel (100, 200) are (99.7, 199.9). By using the bilinear interpolation algorithm, the gray value of the target coordinate is quickly calculated by referring to the gray values ​​of the four integer coordinate pixels around this coordinate. After interpolation, the image of this region is smooth and jagged-free, and jitter is effectively suppressed. The target coordinates of the blood vessel pulsation region pixel (300, 400) are (299.82, 399.88). After interpolation using the bilinear interpolation algorithm, the blood vessel contour is smooth, while the subtle changes of blood vessel pulsation are preserved. This ensures both image stabilization and the preservation of physiological motion details.

[0101] Step S4-3: Synthesize the interpolated deformation field to obtain a stable image sequence.

[0102] Specifically, for each temporal frame, the interpolated deformation field is applied to the original jittered frame. According to the pixel position mapping relationship in the deformation field, each pixel in the original jittered image sequence is moved to the corresponding target coordinate position to complete the image deformation and obtain a single-frame stable frame Is(t). This process is repeated for all temporal frames in the original jittered image sequence to obtain the corresponding stable frames Is(1), Is(2),..., Is(n). Finally, these stable frames are integrated in temporal order to output a semantically preserved, clear and stable image sequence Is(t).

[0103] For example, continuing the example of cortical vascular anastomosis, at t=10, the interpolated deformation field is applied to I(10), and each pixel in I(10) is moved to the target coordinate according to the pixel position mapping relationship to correct the offset caused by endoscopic jitter, resulting in a stable frame Is(10). In this frame, the skull base is clear and stable, the blood vessel pulsation is natural, and the bipolar electrocoagulation forceps move precisely. From t=11 to t=300, the above processing is repeated to obtain stable frames Is(11) to Is(300) respectively. The frames Is(1) to Is(300) are integrated in chronological order to output a stable image sequence Is(t).

[0104] It should be noted that by stabilizing the image sequence, the inherent endoscopic tremor is effectively suppressed, while the problems of image blurring and positional displacement are solved; the active movement of the bipolar electrocoagulation forceps and the operating trajectory of the suction device are precisely preserved; physiological movements such as vascular pulsation and brain tissue respiration are naturally presented, and the surgical field is clear and stable. Doctors can accurately observe the state of the anastomosis through the stable image sequence and successfully complete the operation of cortical vascular anastomosis, avoiding operational errors caused by visual field tremor.

[0105] The specific usage and function of this embodiment are explained below:

[0106] First, the original jitter image sequence and inertial data of the endoscope are acquired. These are then fused to obtain an initial mixed motion field. Next, surgical semantic segmentation is performed on the original jitter image sequence, and a collaborative attention map is obtained by combining the physical motion mechanism. Then, a spatiotemporal adaptive filtering kernel is obtained through the collaborative attention map. The initial mixed motion field is filtered using this kernel to obtain a stable motion field. Finally, image deformation and interpolation synthesis are performed on the stable motion field to obtain a stable image sequence. This invention optimizes the dense motion field by constructing a fusion filter, thus mitigating optical flow drift errors in weakly textured regions. The corrections improved the accuracy of motion vector estimation. Furthermore, by performing surgical semantic segmentation on the original jittery image sequence, instrument masks, physiological motion masks, and static masks were generated to distinguish the motion characteristics of different regions. Simultaneously, a spatiotemporal adaptive filtering kernel was used to adaptively adjust the size, shape, and smoothing intensity of the filtering kernel according to the image stabilization requirements of different regions, filtering the initial mixed motion field. This maximized the separation of endoscopic jitter components, ensuring that the static background area was jitter-free and drift-free, thus improving the stability of the surgical field. This integrated technical solution solved the problem of endoscopic jitter interfering with the surgical field of neurosurgery.

[0107] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. An endoscopic image stabilization control method, characterized in that, It includes the following steps: Acquire the original shaking image sequence and inertial data of the endoscope, fuse the original shaking image sequence and inertial data to obtain the initial mixed motion field; The process involves: acquiring original image sequences using an endoscope; acquiring inertial data using the IMU sensor built into the endoscope; preprocessing the inertial data by denoising and zero-bias correction, and integrating the processed inertial data to obtain coarse motion estimation data; performing nonlocal texture enhancement on adjacent frames of the original jittery images; acquiring dense motion fields between adjacent frames of the original jittery image sequence using a lightweight deep learning optical flow network; using the coarse motion estimation data as a motion prior and constructing a fusion filter based on kinematic and geometric constraints; and performing scale recovery, global motion compensation, and drift correction on the dense motion field based on the fusion filter to output an initial mixed motion field. Surgical semantic segmentation is performed on the original jittery image sequence, and a collaborative attention map is obtained by combining the physical motion mechanism. The collaborative attention map includes the collaborative attention value of each pixel. Specifically, the original jittery image sequence is segmented into instrument instances, physiological motion regions are detected, and static structures are inferred to generate instrument masks, physiological motion masks, and static masks. Based on the instrument masks, physiological motion masks, and static masks, combined with external ECG physiological signals and physical motion mechanisms, a collaborative attention map is generated. The generation rules of the collaborative attention map include assigning high attention values ​​to the static regions corresponding to the static masks, attention values ​​to the physiological motion regions corresponding to the physiological motion masks, and low attention values ​​to the instrument regions corresponding to the instrument masks. If there is regional overlap, the attention values ​​of each region are fused, and the attention values ​​of the physiological motion regions are periodically modulated in combination with external ECG physiological signals. A spatiotemporal adaptive filtering kernel is obtained based on the collaborative attention map, and the initial mixed motion field is filtered based on the spatiotemporal adaptive filtering kernel to obtain a stable motion field; Specifically, based on the collaborative attention map and the initial mixed motion field, a spatiotemporal adaptive filtering kernel is generated for each pixel and each temporal frame in the original jittery image sequence; the initial mixed motion field is then subjected to convolutional filtering based on the spatiotemporal adaptive filtering kernel to obtain a stable motion field; the convolutional filtering is represented as an anisotropic diffusion process modulated by the collaborative attention map, and the diffusion coefficient is controlled by the collaborative attention map. A stable image sequence is obtained by performing image deformation and interpolation synthesis on a stable motion field.

2. The endoscopic image stabilization control method according to claim 1, characterized in that, The method further includes: Instrument instance segmentation is performed using a real-time segmentation network optimized for surgical instruments; Physiological motion region detection is performed based on temporal analysis and prior shape. The temporal analysis includes inter-frame difference and phase change. The prior shape is tubular. The physiological motion region includes, but is not limited to, the pulsating vascular region and the tissue respiration fluctuation region. Static structure inference is performed based on the spatiotemporal consistency of the surgical scenario.

3. The endoscopic image stabilization control method according to claim 1, characterized in that, The method further includes: Apply rigid body kinematic constraints to the instrument region corresponding to the instrument mask, and set a first deviation threshold and a second deviation threshold. The motion vector is compared with the historical motion trajectory to obtain the first deviation, and the motion vector is compared with the coarse motion estimation data corresponding to the inertial data to obtain the second deviation. If either the first deviation is greater than the first deviation threshold or the second deviation is greater than the second deviation threshold, the movement in the instrument area is determined to be non-instrument movement, and movement correction is performed simultaneously.

4. The endoscopic image stabilization control method according to claim 1, characterized in that, Spatiotemporal adaptive filtering kernels include: The size, shape, and dimensions of the spatiotemporal adaptive filter kernel are adaptively modulated according to the collaborative attention map and the initial mixed motion field; In the static region, a 6×6 spacetime kernel is used for powerful smooth motion; In the instrument area, the original instrument motion is maintained by using a 1×1 identity kernel or a 2×2 extremely small empty kernel; In the physiological motor region, the size and smoothing intensity of the spatiotemporal adaptive filter kernel change linearly and continuously with the collaborative attention value; The orientation kernel shape of the spatiotemporal adaptive filter kernel is modulated based on the local consistency of the initial mixed motion field.

5. The endoscopic image stabilization control method according to claim 1, characterized in that, Image deformation and interpolation synthesis are performed on a stable motion field to obtain a stable image sequence, including: The deformation field from the original jittered frame to the stable frame is obtained based on the stable motion field. The original jittered frame is represented as a single frame image in the original jittered image sequence, and the stable frame is represented as a single frame image corresponding to the single frame image in the original jittered image sequence. The deformation field is interpolated based on a differentiated interpolation strategy; Among them, the edge-preserving interpolation algorithm is used for the instrument region corresponding to the instrument mask, and the bilinear interpolation algorithm is used for the non-instrument region; The interpolated deformation field is synthesized to obtain a stable image sequence.

Images