Intelligent Jitter Recognition and Compensation Method and System for Laparoscopic Surgery Images

Abstract

The present invention discloses an intelligent jitter recognition and compensation method and system for endoscopic surgery images. The method includes: acquiring original image data, performing feature point extraction and motion field construction to obtain an initial motion vector distribution; according to the initial motion vector distribution, performing dense motion estimation and interpolation filling, generating full-frame dense motion field data and performing six-degree-of-freedom decomposition to obtain a translation component; if the translation component is lower than a preset threshold, performing trend characterization extraction, obtaining a dynamic jitter component and performing high-pass filtering separation, obtaining an unexpected jitter trajectory and through six-degree-of-freedom inverse transformation, obtaining a dynamic compensation field and adjusting the image pixel positions, adaptively updating compensation parameters in combination with the endoscopic angle to generate a stable image sequence; according to the image sequence, performing micro jitter component compensation detection, iteratively optimizing parameters and updating the compensation field, and outputting a final stable image. This method can achieve jitter recognition and image dynamic compensation, effectively improving image stability.

Description

Technical Field

[0001] The present invention relates to the technical field of jitter recognition, and in particular to an intelligent jitter recognition and compensation method and system for endoscopic surgery images. Background Art

[0002] As the core technology of modern minimally invasive medicine, the stability of the imaging system in endoscopic surgery directly affects the accuracy and safety of surgical operations. With the development of high-definition imaging and intelligent instruments, doctors' demand for real-time image stability is increasing day by day. Especially in complex surgical scenarios, the problem of image jitter caused by factors such as instrument movement and tissue deformation needs to be solved urgently.

[0003] The existing technology mainly combines a hardware anti-shake mechanism with software compensation based on feature point matching. For example, motion data is collected by an inertial sensor for motion estimation and software compensation. However, when dealing with rapid instrument turning or tissue occlusion, the loss of feature points leads to incomplete reconstruction of the motion field, and there is a mechanical delay in hardware compensation, making it difficult to eliminate high-frequency jitter components, resulting in a contradiction between image stability and operation real-time performance.

[0004] In summary, the existing technology has incomplete reconstruction of the motion field and cannot compensate in real time, resulting in insufficient image stability. Summary of the Invention

[0005] The present invention provides an intelligent jitter recognition and compensation method and system for endoscopic surgery images to achieve jitter recognition and dynamic image compensation, effectively improving image stability.

[0006] In a first aspect, to solve the above technical problems, the present invention provides an intelligent jitter recognition and compensation method for endoscopic surgery images, including:

[0007] Obtain the original image data of a continuous frame sequence, perform feature point extraction and motion field construction to obtain an initial motion vector distribution;

[0008] According to the initial motion vector distribution, perform dense motion estimation and interpolation filling to generate full-frame dense motion field data;

[0009] According to the full-frame dense motion field data, perform a six-degree-of-freedom decomposition operation to obtain a translation component;

[0010] If the translation component is lower than a preset threshold, perform trend characterization extraction to obtain a dynamic jitter component;

[0011] According to the dynamic jitter component, perform a high-pass filtering separation operation to obtain an unexpected jitter trajectory;

[0012] Based on the unexpected jitter trajectory, perform a six-degree-of-freedom inverse transformation to obtain a dynamic compensation field;

[0013] Adjust the image pixel positions according to the dynamic compensation field, adaptively update the compensation parameters in combination with the endoscope angle, and generate a stable image sequence;

[0014] Perform micro-vibration component compensation detection according to the stable image sequence. If there is a micro-vibration component, iteratively optimize the parameters and update the compensation field, and output the final stable image.

[0015] As an alternative implementation, obtaining the original image data of the continuous frame sequence, performing feature point extraction and motion field construction to obtain the initial motion vector distribution includes:

[0016] Obtain the original image data of the continuous frame sequence and perform grayscale conversion processing to obtain grayscale image data;

[0017] According to the grayscale image data, use the SIFT algorithm to perform feature point extraction to obtain significant feature points;

[0018] According to the significant feature points, construct an initial motion field in combination with the Lucas-Kanade optical flow method;

[0019] According to the initial motion field, perform motion field interpolation and smoothing processing to obtain the initial motion vector distribution.

[0020] As an alternative implementation, performing dense motion estimation and interpolation filling according to the initial motion vector distribution to generate full-frame dense motion field data includes:

[0021] According to the initial motion vector distribution, perform per-pixel dense motion trajectory calculation to obtain the initial full-frame motion field data;

[0022] According to the initial full-frame motion field data, perform bilinear interpolation filling processing on the sparse area of feature points to generate a continuous motion field distribution covering the full frame;

[0023] According to the continuous motion field distribution, perform motion field smoothing processing to eliminate local motion noise and generate optimized motion field data;

[0024] According to the optimized motion field data, perform motion field continuity verification and iterative recalculation processing to obtain full-frame dense motion field data.

[0025] As an alternative implementation, performing six-degree-of-freedom decomposition operation according to the full-frame dense motion field data to obtain the translation component includes:

[0026] Obtain the depth values of each feature point in the motion field;

[0027] Based on the full-frame dense motion field data and the depth values, establish a rigid body motion hypothesis model through three-dimensional geometric constraint conditions to obtain a rigid body motion hypothesis model;

[0028] According to the rigid body motion hypothesis model, calculate a six-degree-of-freedom parameter set to obtain a six-degree-of-freedom parameter set;

[0029] According to the six-degree-of-freedom parameter set, perform motion component separation and optimization to obtain a translation component.

[0030] As an optional implementation manner, if the translation component is lower than a preset threshold, perform trend characterization extraction to obtain a dynamic jitter component, including:

[0031] According to the translation component, when the translation amount is lower than the preset threshold, perform spatio-temporal motion feature extraction to obtain spatio-temporal motion features;

[0032] According to the spatio-temporal motion features, perform an operation for dividing a dynamic jitter region to obtain a dynamic jitter region;

[0033] According to the dynamic jitter region, construct a non-steady motion vector set to generate a dynamic jitter component.

[0034] As an optional implementation manner, according to the dynamic jitter component, perform a high-pass filtering separation operation to obtain an unexpected jitter trajectory, including:

[0035] According to the dynamic jitter component, perform frequency domain decomposition processing to obtain a high-frequency jitter component;

[0036] According to the high-frequency jitter component, perform smoothing optimization of the jitter trajectory to obtain an optimized distribution of the jitter trajectory;

[0037] According to the optimized distribution of the jitter trajectory, perform division of an independent jitter region to obtain an independent jitter motion region;

[0038] According to the independent jitter motion region, construct a parabolic motion model of the unexpected jitter trajectory and perform trajectory reconstruction processing to obtain an unexpected jitter trajectory.

[0039] As an optional implementation manner, based on the unexpected jitter trajectory, obtain a dynamic compensation field through six-degree-of-freedom inverse transformation, including:

[0040] According to the unexpected jitter trajectory, calculate a correction parameter matrix to obtain a correction parameter matrix;

[0041] According to the correction parameter matrix, construct an inverse kinematics model and decompose a translation correction vector and a rotation correction component;

[0042] Based on the translation correction vector and the rotation correction component, construct and optimize the dynamic compensation vector field to generate a pixel-level dynamic compensation field.

[0043] As an alternative implementation, the method of adjusting the image pixel positions according to the dynamic compensation field, adaptively updating the compensation parameters in combination with the endoscope angle, and generating a stable image sequence includes:

[0044] Perform bilinear interpolation repositioning processing according to the dynamic compensation field to generate preliminary stable image data;

[0045] Perform adaptive parameter adjustment and update according to the angle change data collected by the endoscope attitude sensor to obtain the compensation intensity parameter;

[0046] Perform spatial domain dynamic adjustment processing according to the compensation intensity parameter and the dynamic compensation field to generate optimized compensation field data;

[0047] Perform motion artifact elimination processing according to the optimized compensation field data and the preliminary stable image data to generate a stable image sequence.

[0048] As an alternative implementation, the method of performing micro-jitter component compensation detection according to the stable image sequence, iteratively optimizing the parameters and updating the compensation field if there is a micro-jitter component, and outputting the final stable image includes:

[0049] Perform distribution construction through Lucas-Kanade optical flow analysis according to the stable image sequence to obtain the residual motion field distribution;

[0050] Perform amplitude statistics according to the residual motion field distribution to determine whether there is a micro-jitter component exceeding the set jitter threshold;

[0051] When a micro-jitter component is detected, perform optimization and update of the compensation parameters according to the residual motion field to obtain optimized compensation parameters;

[0052] Perform motion trajectory correction processing on the dynamic compensation field according to the optimized compensation parameters to generate optimized compensation field data;

[0053] Perform secondary motion compensation processing on the stable image sequence according to the optimized compensation field data and output the final stable image.

[0054] In a second aspect, the present invention provides an intelligent jitter recognition and compensation system for endoscopic surgery images, including:

[0055] A feature point extraction module, configured to obtain the original image data of a continuous frame sequence, perform feature point extraction and motion field construction to obtain an initial motion vector distribution;

[0056] A motion estimation module, which is used to perform dense motion estimation and interpolation filling according to the initial motion vector distribution to generate full-frame dense motion field data;

[0057] A degree-of-freedom decomposition module, which is used to perform six-degree-of-freedom decomposition operation according to the full-frame dense motion field data to obtain a translation component;

[0058] A trend characterization module, which is used to perform trend characterization extraction to obtain a dynamic jitter component if the translation component is lower than a preset threshold;

[0059] A filtering and separation module, which is used to perform high-pass filtering and separation operation according to the dynamic jitter component to obtain an unexpected jitter trajectory;

[0060] A degree-of-freedom transformation module, which is used to obtain a dynamic compensation field through six-degree-of-freedom inverse transformation based on the unexpected jitter trajectory;

[0061] An image generation module, which is used to adjust the image pixel positions according to the dynamic compensation field, adaptively update the compensation parameters in combination with the endoscope angle, and generate a stable image sequence;

[0062] An iterative optimization module, which is used to perform detection of compensation for tiny jitter components according to the stable image sequence, and if there are tiny jitter components, iterate to optimize the parameters and update the compensation field, and output the final stable image.

[0063] Compared with the prior art, the present invention has the following beneficial effects:

[0064] (1) Through the fusion technology of the optical flow analysis algorithm and bilinear interpolation, the present invention calculates the motion trajectories of each pixel, performs interpolation filling, smoothing optimization and iterative verification processing, expands the sparse motion field into continuous motion field data covering the full frame, and provides high-precision motion information support for subsequent jitter separation and compensation; through the accurate reconstruction of the full-frame continuous motion field, the problem of compensation failure caused by sparse feature points in the traditional method is effectively solved, and high-precision motion estimation can be maintained even in areas with poor tissue texture. In the endoscope surgery scenario, the sensitivity of jitter detection and the accuracy of compensation can be significantly improved, ensuring the real-time requirements of image stability;

[0065] (2) Through three-dimensional geometric constraints and iterative verification, the present invention greatly improves the accuracy of translation component estimation, provides a basis for effectively distinguishing normal operation movement and unexpected jitter subsequently, enhances the pertinence of the compensation algorithm, and ensures the stable compensation effect in the dynamic endoscope surgery scenario;

[0066] (3) Through spatio-temporal motion feature analysis, dynamic region division, and unsteady motion vector aggregation, the present invention separates high-frequency jitter components from the global motion field, enabling precise differentiation between normal movement and jitter interference, providing targeted input for subsequent compensation. Specifically, the extraction of dynamic jitter components is achieved based on the temporal characteristics and spatial distribution features of the translation components, combined with an adaptive threshold and a motion pattern classification algorithm, significantly enhancing the sensitivity and robustness of jitter detection.

[0067] (4) The present invention performs high-pass filtering separation operations based on the dynamic jitter components to obtain unexpected jitter trajectories. Specifically, the generation of unexpected jitter trajectories is achieved based on frequency domain analysis, kinematic modeling, and region segmentation techniques, combined with adaptive filtering and parabolic fitting, significantly enhancing the accuracy and interpretability of the jitter trajectories.

[0068] (5) The present invention adjusts the pixel positions of the image according to the dynamic compensation field, combined with the adaptive update of the endoscope angle to compensate parameters, generating a stable image sequence. Specifically, through pixel repositioning, parameter dynamic adjustment, compensation field optimization, and artifact elimination processing, real-time stable image output is achieved, capable of eliminating pixel misalignment and blurring caused by jitter, ensuring the clarity and continuity of the surgical field of view.

[0069] (6) Based on optical flow residual analysis and secondary compensation technology, the present invention performs micro jitter component compensation detection and iterative optimization according to the stable image sequence to eliminate residual micro jitter, achieving high-precision stable output of the final image. Among them, the iterative optimization is based on the spatio-temporal characteristics of the residual motion field, combined with adaptive parameter update and compensation field correction, significantly enhancing the fault tolerance and dynamic adaptability of the system. Finally, it effectively suppresses the micro residual jitter that is difficult to eliminate by traditional methods, improving the image stability during long-term surgeries. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 is a schematic flow chart of an intelligent jitter recognition and compensation method for endoscopic surgical images provided by an embodiment of the present invention;

[0071] Figure 2 is a schematic structural diagram of an intelligent jitter recognition and compensation system for endoscopic surgical images provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0072] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present invention.

[0073] Refer to Figure 1, the first embodiment of the present invention provides an intelligent jitter recognition and compensation method for endoscopic surgical images, including the following steps:

[0074] S11, Obtain the original image data of the continuous frame sequence, perform feature point extraction and motion field construction to obtain the initial motion vector distribution;

[0075] S12, According to the initial motion vector distribution, perform dense motion estimation and interpolation filling to generate full-frame dense motion field data;

[0076] S13, According to the full-frame dense motion field data, perform six-degree-of-freedom decomposition operation to obtain the translation component;

[0077] S14, If the translation component is lower than the preset threshold, perform trend characterization extraction to obtain the dynamic jitter component;

[0078] S15, According to the dynamic jitter component, perform high-pass filter separation operation to obtain the unexpected jitter trajectory;

[0079] S16, Based on the unexpected jitter trajectory, obtain the dynamic compensation field through six-degree-of-freedom inverse transformation;

[0080] S17, According to the dynamic compensation field, adjust the image pixel positions, adaptively update the compensation parameters in combination with the endoscopic angle, and generate a stable image sequence;

[0081] S18, According to the stable image sequence, perform micro jitter component compensation detection. If there is a micro jitter component, iterate and optimize the parameters and update the compensation field, and output the final stable image.

[0082] In step S11, the obtaining the original image data of the continuous frame sequence, performing feature point extraction and motion field construction to obtain the initial motion vector distribution includes:

[0083] Obtain the original image data of the continuous frame sequence, perform gray-scale conversion processing to obtain gray-scale image data;

[0084] According to the gray-scale image data, use the SIFT algorithm to perform feature point extraction to obtain significant feature points;

[0085] According to the significant feature points, combine the Lucas-Kanade optical flow method to construct the initial motion field;

[0086] According to the initial motion field, perform motion field interpolation and smoothing processing to obtain the initial motion vector distribution.

[0087] It should be noted that the original image data of the continuous frame sequence is obtained, feature points are extracted and a motion field is constructed to obtain the initial motion vector distribution, including the following operation processes: First, grayscale conversion processing is performed to convert the original RGB image into a grayscale image, and the grayscale value is calculated by the weighted average method: where the weight of the red channel is 0.299, the green channel is 0.587, and the blue channel is 0.114, 、 、 are the red, green, and blue channel values of the pixel point respectively, and is the converted grayscale value. This processing can reduce the data dimension and retain the key luminance information, providing an efficient input for subsequent feature extraction. Then, SIFT feature point extraction is performed, and the Scale-Invariant Feature Transform (SIFT) algorithm is used to detect the significant feature points in the grayscale image. The specific process includes: constructing a multi-scale space, generating a scale space by convolving with Gaussian kernels of different scales, and calculating the difference between adjacent scales. The specific calculation formula is:

[0088] ;

[0089] where is the image after Gaussian convolution, is the scale factor, so as to detect the local extreme points to locate the key positions; key point direction assignment, determining the main direction of the feature points based on the gradient direction histogram to ensure rotational invariance; finally, generating feature descriptors, extracting the local gradient distribution features centered on the key points and normalizing them to form a 128-dimensional description vector. After that, the Lucas-Kanade optical flow method is used to construct the initial motion field. Based on the assumption of luminance constancy between adjacent frames and the spatial consistency constraint, the optical flow equation is solved within the local window around the feature points to calculate the motion vectors of the feature points. The calculation formula is as follows:

[0090] ;

[0091] where , is the image gradient, which is calculated by applying a gradient operator to the grayscale image; is the time derivative, which is approximately obtained by calculating the pixel difference between the current frame and the next frame; , is the optical flow vector, which is obtained by solving the optical flow equation. The displacement estimation is optimized by the least squares method to ensure the local smoothness and accuracy of the motion vectors. Finally, motion field interpolation and smoothing processing are performed. For the area where feature points are missing, the bilinear interpolation method is used. The calculation formula is as follows: where and The motion vectors corresponding to adjacent feature points are calculated by the optical flow method; are the coordinates of known grid points, forming the four corner points of the interpolation region; are the coordinates of the target interpolation point; Use adjacent motion vectors to fill in the blanks to ensure the continuity of the full-frame motion field; Apply the Gaussian kernel function to globally smooth the motion field, and the calculation formula is as follows: , where, is the standard deviation, which controls the smoothing intensity of the Gaussian kernel and is set by the user according to the noise intensity and application scenario; is the normalization factor to ensure that the total weight of the Gaussian kernel is 1 and avoid the overall brightness shift of the filtered image; is the exponential term, which gives the highest weight to the center point and the weight decays with the distance.

[0092] In step S12, the generating of the full-frame dense motion field data by performing dense motion estimation and interpolation filling according to the initial motion vector distribution includes:

[0093] Calculating the dense motion trajectories at the pixel level according to the initial motion vector distribution to obtain the initial full-frame motion field data;

[0094] Performing bilinear interpolation filling on the sparse region of feature points according to the initial full-frame motion field data to generate a continuous motion field distribution covering the full frame;

[0095] Performing motion field smoothing on the continuous motion field distribution to eliminate local motion noise and generate optimized motion field data;

[0096] Performing motion field continuity verification and iterative recalculation on the optimized motion field data to obtain the full-frame dense motion field data.

[0097] It should be noted that the process of generating the full-frame dense motion field data by performing dense motion estimation and interpolation filling according to the initial motion vector distribution refers to the process of expanding the sparse motion field into continuous motion field data covering the full frame through pixel-by-pixel motion trajectory calculation, interpolation filling, smoothing optimization, and iterative verification. This operation can provide high-precision motion information support for subsequent jitter separation and compensation. In the embodiments of the present invention, the generation of the dense motion field is based on the initial motion vector distribution and is realized through steps such as pixel-by-pixel optical flow estimation, bilinear interpolation, Gaussian smoothing, and iterative optimization. In the endoscopic surgery scenario, using the full-frame dense motion field can significantly improve the sensitivity of jitter detection and the accuracy of compensation, ensuring the real-time requirements of image stability.

[0098] Among them, the dense motion field generation is based on the initial motion vector distribution. Based on the initial motion vector distribution, the motion trajectories of all pixels in each frame are calculated. Specifically, first, a Gaussian pyramid is constructed for the input frame sequence to generate multi-scale image layers to adapt to pixels with different motion speeds. Then, in each layer of the pyramid, the Farneback algorithm is used to model the gray-scale change of the local neighborhood, and the formula is: , where is the coordinate vector, is the quadratic coefficient matrix, which describes the curvature information of the local area of the image and is calculated by polynomial fitting; is the linear term vector, which represents the gradient direction of the image gray scale and is calculated by the gradient operator; is the constant term, corresponding to the gray scale mean of the local area, which is directly obtained by pixel brightness statistics. Finally, the displacement vector between adjacent frames ( ) is solved by the least squares method to generate the initial full-frame motion field data. Bilinear interpolation filling is used to handle the missing motion vectors in the sparse area of feature points, and interpolation is performed using adjacent known vectors. Specifically, first, the image is divided into regular grids, with the coordinates of adjacent feature points , as the boundaries of the interpolation area. Then, the interpolation weights are dynamically calculated according to the distance between the target point and the grid points, and the formula is: ;

[0099] where and correspond to the motion vectors of adjacent feature points respectively, which are calculated by the optical flow method; is the coordinate of the known grid point, forming the four corner points of the interpolation area; is the coordinate of the target interpolation point. The motion field smoothing process globally smooths the interpolated motion field to suppress noise and outliers. Specifically, first, a two-dimensional Gaussian function is applied to perform weighted averaging on the motion field, and the formula is: , where is the standard deviation, which controls the smoothing intensity (default set to 1.5) and is set by the user according to the noise level; is the position coordinate of a certain point in the Gaussian kernel relative to the center, and the weights are calculated by traversing all positions in the kernel. Then, convolution operations are used to eliminate abnormal motion vectors caused by sensor noise or instantaneous jitter, thereby enhancing the global consistency of the motion field, reducing the interference of high-frequency noise on subsequent jitter compensation, and improving the reliability of the compensation parameters. The continuity verification and iterative recalculation operation verify the spatio-temporal continuity of the motion field, detect and correct inconsistent areas. Specifically, first, the motion field residual is calculated to identify abnormal vectors that deviate from the global trend, and the calculation formula is as follows:

[0100] ;

[0101] Among them, represents the residual of the sports field at the coordinate ; represents the number of pixels within the analysis window corresponding to the current coordinate (taking a value of an 8×8 neighborhood, = 64); represents the th pixel's motion vector (from the initial full-frame sports field data); represents the median value of the motion vectors within the analysis window Ω; The Levenberg-Marquardt algorithm is used to update the motion estimation parameters to minimize the residual objective function: , where , are the components of the optical flow vector to be optimized, and are adjusted iteratively to minimize the luminance residual; is the pixel luminance value at time , which is directly obtained from the image data; Finally, the optical flow estimation and interpolation are re-executed for the abnormal region until the sports field meets the continuity threshold.

[0102] In step S13, the six-degree-of-freedom decomposition operation is performed according to the full-frame dense sports field data to obtain the translation component, including: obtaining the depth values of each feature point in the sports field;

[0103] According to the full-frame dense sports field data and the depth values, a rigid body motion hypothesis model is established through three-dimensional geometric constraint conditions to obtain a rigid body motion hypothesis model;

[0104] According to the rigid body motion hypothesis model, a six-degree-of-freedom parameter set is calculated to obtain a six-degree-of-freedom parameter set;

[0105] According to the six-degree-of-freedom parameter set, motion component separation and optimization are performed to obtain the translation component.

[0106] It should be noted that the process of performing the six-degree-of-freedom decomposition operation according to the full-frame dense sports field data to obtain the translation component refers to separating the translation component of the camera from the global sports field through rigid body motion modeling, six-degree-of-freedom parameter calculation, and motion component optimization processing. This operation can provide accurate translation motion parameters for jitter compensation. In the embodiments of the present invention, the six-degree-of-freedom decomposition is based on the full-frame dense sports field data and is implemented by combining three-dimensional geometric constraints and optimization algorithms. In the endoscopic surgery scenario, accurately extracting the translation component can effectively distinguish normal operation movements from unexpected jitters and improve the pertinence of the compensation algorithm.

[0107] Among them, the depth values are obtained by directly measuring the depth values of each feature pixel point in the sports field through the ToF depth sensor of the endoscope . The rigid body motion modeling operation is based on three-dimensional geometric constraints. Assuming that the movement of the endoscope is a rigid body motion (without deformation), the mapping relationship between the camera motion and the image displacement is established. Specifically, first, combining the two-dimensional optical flow vector and the depth value , the three-dimensional motion components are inversely deduced using the projection model: , where is the focal length, is the depth value, is the two-dimensional optical flow vector component, is the transformation rate of depth over time, is the three-dimensional motion component; then the translation vector and the rotation matrix of the camera are defined, and the motion equation of the three-dimensional point is: , where represents a 3×3 rotation matrix, describing the rotation of the camera around the three axes, represents a 3×1 translation vector, describing the displacement of the camera in three-dimensional space. The six-degree-of-freedom parameter calculation solves the translation and rotation parameters of the camera from the dense motion field. Specifically, first, using the matched feature point pairs, the essential matrix is estimated by the Random Sample Consensus (RANSAC) algorithm, satisfying: , where and are the normalized image coordinates; then the essential matrix is subjected to singular value decomposition , and the rotation matrix and the translation vector are extracted, satisfying: , , where is an anti-symmetric matrix, is the essential matrix, calculated from the matched point pairs, reflecting the epipolar geometric constraint of the camera motion, is the intermediate matrix of the SVD decomposition, used to separate the rotation and translation components. Finally, the motion component separation and optimization are performed. The translation component is extracted from the six-degree-of-freedom parameters, and its accuracy is optimized. Specifically, first, the translation vector is directly taken as the preliminary result; then the translation components of consecutive frames are weighted and averaged to suppress instantaneous noise, and the formula is: , where is the Gaussian weight, is the time window size, is the translation vector, directly obtained from the essential matrix decomposition.

[0108] In step S14, if the translation component is lower than the preset threshold, the trend characterization is extracted to obtain the dynamic jitter component, including:

[0109] According to the translation component, when the translation amount is lower than a preset threshold, spatio-temporal motion feature extraction is performed to obtain spatio-temporal motion features;

[0110] According to the spatio-temporal motion features, a dynamic jitter region division operation is performed to obtain a dynamic jitter region;

[0111] According to the dynamic jitter region, a non-steady motion vector set is constructed to generate a dynamic jitter component.

[0112] It should be noted that the process of performing trend characterization extraction to obtain a dynamic jitter component when the translation component is lower than a preset threshold refers to the process of separating high-frequency jitter components from the global motion field through spatio-temporal motion feature analysis, dynamic region division, and non-steady motion vector aggregation. This operation can accurately distinguish normal movement from jitter interference and provide targeted input for subsequent compensation. In the embodiments of the present invention, the extraction of the dynamic jitter component is realized based on the temporal characteristics and spatial distribution characteristics of the translation component, combined with an adaptive threshold and a motion mode classification algorithm, significantly improving the sensitivity and robustness of jitter detection.

[0113] It is worth noting that the spatio-temporal motion feature extraction operation extracts the spatio-temporal change features of the motion field when the translation component is lower than a preset threshold ( < 0.5 pixels / frame). Specifically, first, a fast Fourier transform (FFT) is performed on the translation component to extract high-frequency components, and the formula is: , where represents the translation component of the th frame (from the six-degree-of-freedom decomposition result), represents the frequency-domain amplitude, reflecting the periodic jitter intensity of the translation component; then, the local variance of the motion field is calculated to identify non-uniform motion regions, and the formula is: , where represents the local motion variance, measuring the consistency of motion within the region, represents the motion vector of the rd pixel within the local window (from the dense optical flow field), represents the mean motion vector within the window (8×8 pixel window). The dynamic jitter region division operation divides the image into a jitter region and a non-jitter region according to the spatio-temporal motion features. The specific implementation process is as follows: First, the K-means algorithm is used to cluster the local motion variance to divide the high-variance region (jitter) and the low-variance region (non-jitter); then, combined with the high-frequency energy distribution map (FFT amplitude), a binary mask is generated:

[0114] ;

[0115] where is the frequency domain energy threshold (set to 0.8). The set of non-steady motion vectors constructs the motion vectors of the aggregated jitter region to generate the dynamic jitter component. Specifically, first, the motion vectors of the masked marked region are extracted from the full-frame motion field; then, outliers are removed based on the Mahalanobis distance, and the formula is: , where is the mean vector, is the covariance matrix, is the motion vector of the masked marked region. Finally, the filtered vector set is used as the output.

[0116] In step S15, the high-pass filtering separation operation is performed according to the dynamic jitter component to obtain the unexpected jitter trajectory, including:

[0117] Perform frequency domain decomposition processing on the dynamic jitter component to obtain the high-frequency jitter component;

[0118] Perform jitter trajectory smoothing optimization on the high-frequency jitter component to obtain the optimized distribution of the jitter trajectory;

[0119] Perform independent jitter region division according to the optimized distribution of the jitter trajectory to obtain the independent jitter motion region;

[0120] According to the independent jitter motion region, construct a parabolic motion model of the unexpected jitter trajectory and perform trajectory reconstruction processing to obtain the unexpected jitter trajectory.

[0121] It should be noted that the process of performing the high-pass filtering separation operation according to the dynamic jitter component to obtain the unexpected jitter trajectory refers to extracting the physically meaningful unexpected jitter trajectory from the dynamic jitter component through frequency domain decomposition, trajectory optimization, region division, and model reconstruction. This operation can separate high-frequency noise from real jitter and provide accurate trajectory input for the compensation algorithm. In the embodiments of the present invention, the generation of the unexpected jitter trajectory is realized based on frequency domain analysis, kinematic modeling, and region segmentation technologies, combined with adaptive filtering and parabolic fitting, significantly improving the accuracy and interpretability of the jitter trajectory.

[0122] Among them, the frequency domain decomposition processing separates the high-frequency jitter signal from the dynamic jitter component through a high-pass filter. The specific implementation process is as follows: First, convert the time domain signal of the dynamic jitter component into a frequency domain signal , and the formula is: ; then apply the Butterworth high-pass filter to retain the high-frequency components, and the formula is: , where is the cut-off frequency (default 5Hz), which is determined by historical data statistics and is used to distinguish high-frequency jitter from low-frequency normal motion, is the filter order (default is 4), which controls the steepness of the transition band. The higher the order, the steeper the filtering. Through frequency-domain decomposition, it effectively separates high-frequency jitter signals (such as hand tremors) from low-frequency interferences (such as respiratory movements), enhancing the pertinence of subsequent trajectory analysis. For the smoothing and optimization of the jitter trajectory, the high-frequency jitter components are smoothed to suppress noise and extract the main motion trend. Specifically, first, for the time-domain signal perform mean filtering within a window, and the formula is: ; then further apply a Gaussian kernel function to enhance local continuity, and the formula is: , where N is the size of the sliding window, which controls the smoothing intensity. The larger the window, the more significant the smoothing effect; is the standard deviation of the Gaussian kernel, which is dynamically adjusted by the signal-to-noise ratio of the jitter signal.

[0123] Among them, the operation of dividing independent jitter regions divides independent jitter regions according to the distribution of the smoothed trajectory. The specific implementation process: First, set the amplitude threshold , which controls the sensitivity of region division and is adjusted by the user according to the scenario. The regions with amplitudes higher than the threshold are marked as candidate jitter regions. Then, use morphological closing operation to connect adjacent candidate regions to generate independent connected domains. The construction of the parabolic motion model and the trajectory reconstruction process fit a parabolic model to the independent jitter regions to reconstruct the unexpected jitter trajectory. The specific implementation: First, use the least squares method to fit the trajectory data points, and the model is: , where the coefficients respectively represent the quadratic coefficients of the parabola in the x, y, and z components, reflecting the acceleration characteristics of the jitter, respectively represent the linear coefficients in the x, y, and z components, describing the initial velocity of the jitter, respectively represent the constant terms in the x, y, and z components, corresponding to the initial position deviation, and are solved by minimizing the sum of the squares of the residuals. Then, generate a smoothed trajectory according to the fitting parameters and align it with the original data.

[0124] In step S16, based on the unexpected jitter trajectory, through six-degree-of-freedom inverse transformation, a dynamic compensation field is obtained, including:

[0125] According to the unexpected jitter trajectory, calculate the correction parameter matrix to obtain the correction parameter matrix;

[0126] According to the correction parameter matrix, construct an inverse kinematic model to decompose the translational correction vector and the rotational correction component;

[0127] According to the translational correction vector and the rotational correction component, construct and optimize the dynamic compensation vector field to generate a pixel-level dynamic compensation field.

[0128] It should be noted that the process of generating a dynamic compensation field through six - degree - of - freedom inverse transformation based on the unexpected jitter trajectory refers to converting the jitter trajectory into a pixel - level compensation vector field through parametric matrix calculation, inverse kinematic modeling, and compensation field optimization. This operation can provide accurate spatial correction parameters for real - time jitter compensation. In the embodiments of the present invention, the generation of the dynamic compensation field is based on the kinematic characteristics of the unexpected jitter trajectory, and is realized by combining six - degree - of - freedom inverse transformation and optimization algorithms, significantly improving the local adaptability and global consistency of the compensation.

[0129] Among them, the calculation operation of the correction parameter matrix calculates the six - degree - of - freedom parameter matrix required for compensation according to the parabola model parameters of the unexpected jitter trajectory. The specific implementation process: First, map the parabola model parameters to translational acceleration and rotational angular velocity . The formula is: , where is the translational acceleration, directly derived from the quadratic coefficient of the parabola, reflecting the acceleration characteristics of the jitter; the rotational angular velocity is calculated from the linear coefficient and the depth , describing the rotational trend of the jitter. is the estimated value of the scene depth, obtained through stereo matching; represents the camera calibration parameter (mm / pixel), that is, the physical length of each pixel, converting pixel displacement into actual physical displacement for calculating the rotational angular velocity . Then, construct a six - degree - of - freedom correction matrix , including translational and rotational components: . The operation of constructing the inverse kinematic model decomposes the correction matrix to obtain the translational correction vector and the rotational correction component . Specifically, first perform singular value decomposition (SVD) on to separate the rotational and translational components: , where the translational part of the translational correction vector , and the rotational correction matrix , where is an anti - symmetric matrix used to maintain the orthogonality of the rotation matrix.

[0130] It is worth noting that the construction and optimization of the dynamic compensation field map the translational and rotational correction amounts to the image plane to generate a pixel - level dynamic compensation field.

[0131] In step S17, adjusting the image pixel positions according to the dynamic compensation field, adaptively updating the compensation parameters in combination with the endoscope angle, and generating a stable image sequence includes:

[0132] Perform bilinear interpolation repositioning processing according to the dynamic compensation field to generate preliminary stable image data;

[0133] Perform adaptive parameter adjustment and update according to the angle change data collected by the endoscope attitude sensor to obtain the compensation intensity parameter;

[0134] Perform spatial domain dynamic adjustment processing according to the compensation intensity parameter and the dynamic compensation field to generate optimized compensation field data;

[0135] Perform motion artifact elimination processing according to the optimized compensation field data and the preliminary stable image data to generate a stable image sequence.

[0136] It should be noted that the process of adjusting the image pixel position according to the dynamic compensation field, adaptively updating the compensation parameters in combination with the endoscope angle, and generating a stable image sequence refers to realizing real-time image stable output through pixel repositioning, parameter dynamic adjustment, compensation field optimization, and artifact elimination processing. This operation can eliminate pixel misalignment and blurring caused by jitter, ensuring the clarity and continuity of the surgical field of view. In the embodiments of the present invention, the generation of the stable image sequence is based on the deep fusion of the dynamic compensation field and the endoscope attitude data, combined with adaptive parameter update and image restoration technology, significantly improving the robustness and real-time performance of the image quality.

[0137] Among them, the bilinear interpolation repositioning processing is to reposition the original image pixels according to the displacement vector of the dynamic compensation field. The specific implementation process is as follows: map the displacement vector of each pixel in the compensation field to the original image coordinates; then perform weighted interpolation on the four adjacent pixels around the target position, and the formula is: , where represents the brightness value of the adjacent pixel, which is directly obtained from the original image; represents the boundary coordinates of the interpolation area, which are determined by the compensation vector. The adaptive parameter adjustment and update operation is to dynamically adjust the compensation intensity parameter according to the real-time angle data collected by the endoscope attitude sensor. The specific implementation process is as follows: first establish the linear relationship between the endoscope pitch and the compensation intensity : , where is the reference intensity (default 0.8), is the slope, controlling the sensitivity of the angle change to the compensation intensity (default 0.05 / °), represents the endoscope pitch angle, which is obtained in real time by the attitude sensor. Then update the compensation intensity parameter for each frame to ensure enhanced compensation when the endoscope rotates rapidly and reduce the risk of overcompensation when it is stationary.

[0138] It should be noted that the spatial domain dynamic adjustment process combines the compensation intensity parameter with the dynamic compensation field to optimize the spatial distribution of the compensation vector. The specific implementation process is as follows: First, the compensation intensity and the compensation field vector are weighted: , where represents the original compensation field; then, a bilateral filter is used to smooth the compensation field while retaining edge details. The formula is: , where represents the standard deviation of the Gaussian kernel (default 1.5), which controls the spatial smoothing range; represents the brightness difference threshold (default 10), which protects the compensation vectors in the edge regions, represents the weighted compensation field. The motion artifact elimination process is to eliminate the remaining artifacts (such as ghosting or blurring) during the compensation process. The specific implementation is as follows: First, the optical flow residual between the stable image and the original image is calculated to detect the artifact regions. Then, a repair model based on U-Net is used to perform pixel-level repair on the artifact regions. The loss function is: , where is the pixel reconstruction loss, is the perceptual loss, is the loss weight (set to 0.7 and 0.3), which balances pixel accuracy and semantic consistency.

[0139] In step S18, the micro-jitter component compensation detection is performed according to the stable image sequence. If there is a micro-jitter component, the parameters are iteratively optimized and the compensation field is updated, and the final stable image is output, including:

[0140] According to the stable image sequence, a distribution is constructed through the Lucas-Kanade optical flow analysis method to obtain the residual motion field distribution;

[0141] According to the residual motion field distribution, amplitude statistics are performed to determine whether there is a micro-jitter component exceeding the set jitter threshold;

[0142] When a micro-jitter component is detected, according to the residual motion field, the compensation parameters are optimized and updated to obtain the optimized compensation parameters;

[0143] According to the optimized compensation parameters, motion trajectory correction processing is performed on the dynamic compensation field to generate optimized compensation field data;

[0144] According to the optimized compensation field data, secondary motion compensation processing is performed on the stable image sequence to output the final stable image.

[0145] It should be noted that the process of compensating for and detecting minute jitter components and iteratively optimizing based on a stable image sequence refers to eliminating residual minute jitter through residual motion field construction, threshold detection, parameter optimization, and secondary compensation processing to achieve high-precision stable output of the final image. This operation can further enhance the robustness of the compensation algorithm and ensure the clinical usability of the image quality. In the embodiments of the present invention, the iterative optimization is based on the spatio-temporal characteristics of the residual motion field, combined with adaptive parameter update and compensation field correction, significantly improving the fault tolerance and dynamic adaptability of the system.

[0146] Among them, the operation of constructing the residual motion field distribution analyzes the residual motion between the stable image sequence and the original image through the Lucas-Kanade optical flow method. The specific implementation process is as follows: Based on the assumption of constant brightness, calculate the residual optical flow vector between the stable image and the original image , and the formula is: , where is the image spatial gradient, calculated by the Sobel operator; is the time derivative, calculated through the frame difference between the stable image and the original image; thus generating a residual motion field distribution to quantify the residual minute jitter after compensation. The detection of minute jitter components is to statistically calculate the amplitude of the residual motion field and determine whether it exceeds a preset threshold. Specifically, first calculate the amplitude of the residual vector for each pixel: , where , is the residual optical flow vector at the pixel point ( ); then statistically calculate the proportion of the pixels in the entire frame whose amplitude exceeds the threshold . If , it is determined that there is significant minute jitter. The operation of optimizing and updating compensation parameters is to optimize the compensation parameters based on the residual motion field to improve the compensation accuracy. The specific implementation process is as follows: First, minimize the total energy of the residual motion field, and the formula is:

[0147] ;

[0148] where is the total energy of the residual motion field, representing the cumulative intensity of the residual jitter of all pixels, calculated by summing the squares of the magnitudes of the residual vectors of each pixel in the residual motion field, and used to quantify the overall level of the residual jitter after compensation; is the magnitude of the residual vector, that is, the residual motion amplitude at the pixel point ( ), calculated by the Lucas-Kanade optical flow method to calculate the residual optical flow vector between the stable image and the original image; then iteratively adjust the compensation field parameter , and the update formula is: , where It represents the field parameters to be compensated, which are the variables to be optimized in the dynamic compensation field; It is the step size coefficient of gradient descent, which controls the amplitude of parameter update and is set to 0.01 according to experimental calibration; It is the loss function The partial derivative of the parameter indicates the parameter adjustment direction. The dynamic compensation field correction process is to correct the compensation field according to the optimized parameters to generate optimized compensation field data. The specific implementation process is as follows: First, map the optimized parameters to the compensation field and update the displacement vector ; Then, apply the Kalman filter to smooth the compensation field in time series. The formula is: , where is the state transition and observation matrix, which is defined by the motion model; is the Kalman gain, which dynamically adjusts the weights of prediction and observation and is set by the user according to the actual scenario. Finally, perform the secondary motion compensation process, and apply the optimized compensation field to perform secondary compensation on the stable image sequence. Specifically, first, apply the optimized compensation field to the stable image using bilinear interpolation. The formula is the same as in step S17; Then, calculate the residual motion field after the secondary compensation. If the amplitude meets the standard ( ), output the final image; Otherwise, trigger a new round of optimization. Through closed-loop iteration, completely eliminate the residual jitter and ensure that the output image meets the clinical stability standard.

[0149] Referring to Figure 2 , the second embodiment of the present invention provides an intelligent jitter recognition and compensation system for endoscopic surgical images, including:

[0150] A feature point extraction module, which is used to obtain the original image data of the continuous frame sequence, perform feature point extraction and motion field construction, and obtain the initial motion vector distribution;

[0151] A motion estimation module, which is used to perform dense motion estimation and interpolation filling according to the initial motion vector distribution to generate full-frame dense motion field data;

[0152] A degree of freedom decomposition module, which is used to perform six-degree-of-freedom decomposition operation according to the full-frame dense motion field data to obtain the translation component;

[0153] A trend characterization module, which is used to perform trend characterization extraction to obtain the dynamic jitter component if the translation component is lower than the preset threshold;

[0154] A filtering and separation module, which is used to perform high-pass filtering and separation operation according to the dynamic jitter component to obtain the unexpected jitter trajectory;

[0155] A degree of freedom transformation module, which is used to obtain the dynamic compensation field through six-degree-of-freedom inverse transformation based on the unexpected jitter trajectory;

[0156] An image generation module, configured to adjust the positions of image pixels according to the dynamic compensation field, adaptively update compensation parameters in combination with the endoscope angle, and generate a stable image sequence;

[0157] An iterative optimization module, configured to perform detection of micro-vibration component compensation according to the stable image sequence, and if there is a micro-vibration component, iteratively optimize parameters and update the compensation field, and output a final stable image.

[0158] It should be noted that the intelligent jitter recognition and compensation system for endoscopic surgical images provided in the embodiments of the present invention is used to execute all the process steps of the intelligent jitter recognition and compensation method for endoscopic surgical images in the above embodiments. The working principles and beneficial effects of the two correspond one by one, and thus will not be elaborated herein.

[0159] The embodiments of the present invention further provide a terminal device. The terminal device includes: a processor, a memory, and a computer program stored in the memory and executable on the processor, such as an intelligent jitter recognition and compensation program for endoscopic surgical images. When the processor executes the computer program, the steps in the embodiments of the above-mentioned intelligent jitter recognition and compensation method for endoscopic surgical images are implemented, such as Figure 1 the step S11 shown. Alternatively, when the processor executes the computer program, the functions of each module / unit in the above system embodiments are implemented.

[0160] Exemplarily, the computer program may be divided into one or more modules / units. The one or more modules / units are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the computer program in the terminal device.

[0161] The terminal device may be a computing device such as a desktop computer, a notebook, a palm computer, and a smart tablet. The terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art can understand that the above components are only examples of the terminal device, and do not constitute a limitation on the terminal device. It may include more or fewer components than the above, or combine some components, or different components. For example, the terminal device may further include input / output devices, network access devices, a bus, etc.

[0162] The so-called processor may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The processor is the control center of the terminal device, connecting various parts of the entire terminal device through various interfaces and circuits.

[0163] The memory can be used to store the computer programs and / or modules. The processor realizes various functions of the terminal device by running or executing the computer programs and / or modules stored in the memory, and by calling the data stored in the memory. The memory may mainly include a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created according to the use of the mobile phone (such as audio data, phone book, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as a hard disk, memory, plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, at least one magnetic disk storage device, flash device, or other volatile solid-state storage devices.

[0164] Among them, if the modules / units integrated in the terminal device are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, to implement all or part of the processes in the above-described embodiment methods of the present invention, it can also be completed by a computer program instructing relevant hardware. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-described various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or system capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0165] It should be noted that the system embodiments described above are only illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, in the attached drawings of the system embodiments provided by the present invention, the connection relationship between the modules indicates that they have a communication connection, which can be specifically implemented as one or more communication buses or signal lines. Those of ordinary skill in the art can understand and implement it without creative effort.

[0166] The above-described specific embodiments have further elaborated on the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only the specific embodiments of the present invention and is not used to limit the protection scope of the present invention. It is particularly pointed out that for those skilled in the art, any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An intelligent jitter recognition and compensation method for laparoscopic surgery images, characterized in that, Including: Obtain the original image data of a continuous frame sequence, perform feature point extraction and motion field construction to obtain an initial motion vector distribution; According to the initial motion vector distribution, perform dense motion estimation and interpolation filling to generate full-frame dense motion field data; According to the full-frame dense motion field data, perform a six-degree-of-freedom decomposition operation to obtain a translation component; If the translation component is lower than a preset threshold, perform trend characterization extraction to obtain a dynamic jitter component; According to the dynamic jitter component, perform a high-pass filter separation operation to obtain an unexpected jitter trajectory; Based on the unexpected jitter trajectory, perform a six-degree-of-freedom inverse transformation to obtain a dynamic compensation field; According to the dynamic compensation field, adjust the image pixel positions, adaptively update the compensation parameters in combination with the endoscope angle, and generate a stable image sequence; According to the stable image sequence, perform a micro jitter component compensation detection. If there is a micro jitter component, iteratively optimize the parameters and update the compensation field, and output the final stable image; Among them, the step of adjusting the image pixel positions according to the dynamic compensation field, adaptively updating the compensation parameters in combination with the endoscope angle, and generating a stable image sequence includes: According to the dynamic compensation field, perform bilinear interpolation repositioning processing to generate preliminary stable image data; According to the angle change data collected in real time by the endoscope attitude sensor, perform adaptive parameter adjustment and update to obtain a compensation intensity parameter; According to the compensation intensity parameter and the dynamic compensation field, perform spatial domain dynamic adjustment processing to generate optimized compensation field data; According to the optimized compensation field data and the preliminary stable image data, perform motion artifact elimination processing to generate a stable image sequence; Among them, the step of performing a micro jitter component compensation detection according to the stable image sequence. If there is a micro jitter component, iteratively optimize the parameters and update the compensation field, and output the final stable image includes: According to the stable image sequence, perform distribution construction through the Lucas-Kanade optical flow analysis method to obtain a residual motion field distribution; According to the residual motion field distribution, perform amplitude statistics to determine whether there is a micro jitter component exceeding a set jitter threshold; When a micro jitter component is detected, optimize and update the compensation parameters according to the residual motion field to obtain optimized compensation parameters; According to the optimized compensation parameters, perform motion trajectory correction processing on the dynamic compensation field to generate optimized compensation field data; According to the optimized compensation field data, perform secondary motion compensation processing on the stable image sequence and output the final stable image.

2. The intelligent jitter recognition and compensation method for laparoscopic surgery images according to claim 1, characterized in that, The step of obtaining the original image data of a continuous frame sequence, performing feature point extraction and motion field construction to obtain an initial motion vector distribution includes: Obtain the original image data of a continuous frame sequence and perform gray-scale conversion processing to obtain gray-scale image data; According to the gray-scale image data, use the SIFT algorithm to perform feature point extraction to obtain significant feature points; According to the significant feature points, combine with the Lucas-Kanade optical flow method to construct an initial motion field; According to the initial motion field, perform motion field interpolation and smoothing processing to obtain an initial motion vector distribution.

3. The intelligent jitter recognition and compensation method for endoscopic surgical images according to claim 1, characterized in that, Performing dense motion estimation and interpolation filling according to the initial motion vector distribution to generate full-frame dense motion field data, including: Calculating dense motion trajectories at the pixel level according to the initial motion vector distribution to obtain initial full-frame motion field data; Performing bilinear interpolation filling on the sparse region of feature points according to the initial full-frame motion field data to generate a continuous motion field distribution covering the full frame; Performing motion field smoothing processing according to the continuous motion field distribution to eliminate local motion noise and generate optimized motion field data; Performing motion field continuity verification and iterative recalculation processing according to the optimized motion field data to obtain full-frame dense motion field data.

4. The intelligent jitter recognition and compensation method for endoscopic surgery images according to claim 1, characterized in that, Performing six-degree-of-freedom decomposition operation according to the full-frame dense motion field data to obtain a translation component, including: Obtaining the depth values of each feature point in the motion field; Establishing a rigid body motion hypothesis model through three-dimensional geometric constraint conditions according to the full-frame dense motion field data and the depth values to obtain a rigid body motion hypothesis model; Calculating a six-degree-of-freedom parameter set according to the rigid body motion hypothesis model to obtain a six-degree-of-freedom parameter set; Performing motion component separation and optimization according to the six-degree-of-freedom parameter set to obtain a translation component.

5. The intelligent jitter recognition and compensation method for laparoscopic surgery images according to claim 1, wherein If the translation component is lower than a preset threshold, performing trend characterization extraction to obtain a dynamic jitter component, including: Performing spatio-temporal motion feature extraction when the translation amount is lower than the preset threshold according to the translation component to obtain spatio-temporal motion features; Performing dynamic jitter region division operation according to the spatio-temporal motion features to obtain a dynamic jitter region; Constructing a non-steady motion vector set according to the dynamic jitter region to generate a dynamic jitter component.

6. The intelligent jitter recognition and compensation method for laparoscopic surgery images according to claim 1, wherein Performing high-pass filter separation operation according to the dynamic jitter component to obtain an unexpected jitter trajectory, including: Performing frequency domain decomposition processing according to the dynamic jitter component to obtain a high-frequency jitter component; Performing jitter trajectory smoothing and optimization according to the high-frequency jitter component to obtain an optimized distribution of jitter trajectories; Performing independent jitter region division according to the optimized distribution of jitter trajectories to obtain an independent jitter motion region; Constructing a parabolic motion model of the unexpected jitter trajectory and performing trajectory reconstruction processing according to the independent jitter motion region to obtain an unexpected jitter trajectory.

7. The intelligent jitter recognition and compensation method for endoscopic surgery images according to claim 1, characterized in that, Based on the unexpected jitter trajectory, performing six-degree-of-freedom inverse transformation to obtain a dynamic compensation field, including: Calculating a correction parameter matrix according to the unexpected jitter trajectory to obtain a correction parameter matrix; Constructing an inverse kinematic model according to the correction parameter matrix and decomposing a translation correction vector and a rotation correction component; Constructing and optimizing a dynamic compensation vector field according to the translation correction vector and the rotation correction component to generate a pixel-level dynamic compensation field.

8. An intelligent jitter recognition and compensation system for laparoscopic surgery images, characterized in that, Implementing the intelligent jitter recognition and compensation method for endoscopic surgery images as described in any one of claims 1 to 7, including: A feature point extraction module for obtaining the original image data of a continuous frame sequence, performing feature point extraction and motion field construction to obtain an initial motion vector distribution; A motion estimation module, configured to perform dense motion estimation and interpolation filling according to the initial motion vector distribution, and generate full-frame dense motion field data; A degree-of-freedom decomposition module, configured to perform six-degree-of-freedom decomposition operation according to the full-frame dense motion field data to obtain a translation component; A trend characterization module, configured to perform trend characterization extraction to obtain a dynamic jitter component if the translation component is lower than a preset threshold; A filtering and separation module, configured to perform high-pass filtering and separation operation according to the dynamic jitter component to obtain an unexpected jitter trajectory; A degree-of-freedom transformation module, configured to obtain a dynamic compensation field through six-degree-of-freedom inverse transformation based on the unexpected jitter trajectory; An image generation module, configured to adjust the pixel positions of an image according to the dynamic compensation field, adaptively update compensation parameters in combination with the endoscope angle, and generate a stable image sequence; An iterative optimization module, configured to perform detection of compensation for minute jitter components according to the stable image sequence, and if there are minute jitter components, iteratively optimize parameters and update the compensation field, and output a final stable image.

Images