Intelligent jitter identification and compensation method and system for endoscopic surgery image

Abstract

The invention discloses an intelligent jitter identification and compensation method and system for an endoscopic surgery image, and the method comprises the steps: obtaining original image data, carrying out the feature point extraction and motion field construction, and obtaining initial motion vector distribution; performing dense motion estimation and interpolation filling according to the initial motion vector distribution, generating full-frame dense motion field data, and performing six-degree-of-freedom decomposition to obtain translation components; if the translation component is lower than a preset threshold value, trend representation extraction is carried out, a dynamic jitter component is obtained, high-pass filtering separation is carried out, an unexpected jitter track is obtained, a dynamic compensation field is obtained through six-degree-of-freedom inverse transformation, an image pixel position is adjusted, compensation parameters are adaptively updated in combination with an endoscope angle, and a stable image sequence is generated; and according to the image sequence, performing tiny jitter component compensation detection iteration optimization parameters and updating a compensation field, and outputting a final stable image. According to the method, jitter identification and image dynamic compensation can be realized, and the image stability is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of jitter recognition, and particularly to an intelligent jitter recognition and compensation method and system for endoscopic surgical images. Background Art

[0002] As a core technology of modern minimally invasive medicine, the stability of the imaging system in endoscopic surgery directly affects the accuracy and safety of surgical operations. With the development of high-definition imaging and intelligent instruments, doctors' demand for real-time image stability is increasing day by day. Especially in complex surgical scenarios, the problem of image jitter caused by factors such as instrument movement and tissue deformation needs to be solved urgently.

[0003] The existing technology mainly adopts a combination of a hardware anti-shake mechanism and software compensation based on feature point matching. For example, motion data is collected by an inertial sensor for motion estimation and software compensation. However, when dealing with rapid instrument turning or tissue occlusion, the reconstruction of the motion field is incomplete due to the loss of feature points, and there is a mechanical delay in hardware compensation, making it difficult to eliminate high-frequency jitter components, resulting in a contradiction between image stability and operation real-time performance.

[0004] In summary, the existing technology has an incomplete reconstruction of the motion field and cannot perform real-time compensation, resulting in insufficient image stability. Summary of the Invention

[0005] The present invention provides an intelligent jitter recognition and compensation method and system for endoscopic surgical images to achieve jitter recognition and dynamic image compensation, effectively improving image stability.

[0006] In a first aspect, to solve the above technical problems, the present invention provides an intelligent jitter recognition and compensation method for endoscopic surgical images, including: Obtaining the original image data of a continuous frame sequence, performing feature point extraction and motion field construction to obtain an initial motion vector distribution; According to the initial motion vector distribution, performing dense motion estimation and interpolation filling to generate full-frame dense motion field data; According to the full-frame dense motion field data, performing six-degree-of-freedom decomposition operation to obtain a translation component; If the translation component is lower than a preset threshold, performing trend characterization extraction to obtain a dynamic jitter component; According to the dynamic jitter component, performing high-pass filtering separation operation to obtain an unexpected jitter trajectory; Based on the unexpected jitter trajectory, obtaining a dynamic compensation field through six-degree-of-freedom inverse transformation; According to the dynamic compensation field, adjusting the pixel positions of the image, adaptively updating the compensation parameters in combination with the endoscopic angle, and generating a stable image sequence; Based on the stable image sequence, perform micro-jitter component compensation detection. If there is a micro-jitter component, iteratively optimize the parameters and update the compensation field, and output the final stable image.

[0007] As an alternative implementation, the obtaining of the original image data of the continuous frame sequence, performing feature point extraction and motion field construction to obtain the initial motion vector distribution includes: Obtain the original image data of the continuous frame sequence and perform grayscale conversion processing to obtain grayscale image data; Based on the grayscale image data, use the SIFT algorithm to perform feature point extraction to obtain significant feature points; Based on the significant feature points, combine with the Lucas-Kanade optical flow method to construct an initial motion field; Based on the initial motion field, perform motion field interpolation and smoothing processing to obtain the initial motion vector distribution.

[0008] As an alternative implementation, the performing of dense motion estimation and interpolation filling based on the initial motion vector distribution to generate full-frame dense motion field data includes: Based on the initial motion vector distribution, perform per-pixel dense motion trajectory calculation to obtain the initial full-frame motion field data; Based on the initial full-frame motion field data, perform bilinear interpolation filling processing on the sparse feature point area to generate a continuous motion field distribution covering the full frame; Based on the continuous motion field distribution, perform motion field smoothing processing to eliminate local motion noise and generate optimized motion field data; Based on the optimized motion field data, perform motion field continuity verification and iterative recalculation processing to obtain full-frame dense motion field data.

[0009] As an alternative implementation, the performing of six-degree-of-freedom decomposition operation based on the full-frame dense motion field data to obtain the translation component includes: Obtain the depth values of each feature point in the motion field; Based on the full-frame dense motion field data and the depth values, establish a rigid body motion hypothesis model through three-dimensional geometric constraint conditions to obtain the rigid body motion hypothesis model; Based on the rigid body motion hypothesis model, perform six-degree-of-freedom parameter set calculation to obtain the six-degree-of-freedom parameter set; Based on the six-degree-of-freedom parameter set, perform motion component separation and optimization to obtain the translation component.

[0010] As an alternative implementation, if the translation component is lower than a preset threshold, perform trend characterization extraction to obtain the dynamic jitter component, including: According to the translation component, when the translation amount is lower than a preset threshold, spatio-temporal motion feature extraction is performed to obtain spatio-temporal motion features; According to the spatio-temporal motion features, a dynamic jitter region division operation is performed to obtain a dynamic jitter region; According to the dynamic jitter region, a non-steady-state motion vector set is constructed to generate a dynamic jitter component.

[0011] As an optional implementation manner, the performing a high-pass filtering separation operation according to the dynamic jitter component to obtain an unexpected jitter trajectory includes: Performing frequency-domain decomposition processing according to the dynamic jitter component to obtain a high-frequency jitter component; Performing jitter trajectory smoothing optimization according to the high-frequency jitter component to obtain an optimized distribution of the jitter trajectory; Performing independent jitter region division according to the optimized distribution of the jitter trajectory to obtain an independent jitter motion region; According to the independent jitter motion region, a parabolic motion model of the unexpected jitter trajectory is constructed and trajectory reconstruction processing is performed to obtain an unexpected jitter trajectory.

[0012] As an optional implementation manner, the obtaining a dynamic compensation field based on the unexpected jitter trajectory through a six-degree-of-freedom inverse transformation includes: Performing calculation of a correction parameter matrix according to the unexpected jitter trajectory to obtain a correction parameter matrix; According to the correction parameter matrix, a reverse kinematics model is constructed to decompose a translation correction vector and a rotation correction component; According to the translation correction vector and the rotation correction component, a dynamic compensation vector field is constructed and optimized to generate a pixel-level dynamic compensation field.

[0013] As an optional implementation manner, the adjusting the image pixel positions according to the dynamic compensation field, adaptively updating compensation parameters in combination with the endoscope angle, and generating a stable image sequence includes: Performing bilinear interpolation repositioning processing according to the dynamic compensation field to generate preliminary stable image data; Performing adaptive parameter adjustment and update according to the angle change data collected in real time by the endoscope attitude sensor to obtain a compensation intensity parameter; Performing spatial-domain dynamic adjustment processing according to the compensation intensity parameter and the dynamic compensation field to generate optimized compensation field data; Performing motion artifact elimination processing according to the optimized compensation field data and the preliminary stable image data to generate a stable image sequence.

[0014] As an alternative implementation, performing micro-jitter component compensation detection based on the stable image sequence, and if there is a micro-jitter component, iteratively optimizing the parameters and updating the compensation field, and outputting the final stable image, including: Based on the stable image sequence, construct a distribution through Lucas-Kanade optical flow analysis method to obtain the residual motion field distribution; Based on the residual motion field distribution, perform amplitude statistics to determine whether there is a micro-jitter component exceeding the set jitter threshold; When a micro-jitter component is detected, optimize and update the compensation parameters according to the residual motion field to obtain optimized compensation parameters; According to the optimized compensation parameters, perform motion trajectory correction processing on the dynamic compensation field to generate optimized compensation field data; Based on the optimized compensation field data, perform secondary motion compensation processing on the stable image sequence to output the final stable image.

[0015] In a second aspect, the present invention provides an intelligent jitter recognition and compensation system for endoscopic surgery images, including: A feature point extraction module, configured to obtain the original image data of a continuous frame sequence, perform feature point extraction and motion field construction to obtain an initial motion vector distribution; A motion estimation module, configured to perform dense motion estimation and interpolation filling based on the initial motion vector distribution to generate full-frame dense motion field data; A degree-of-freedom decomposition module, configured to perform six-degree-of-freedom decomposition operation based on the full-frame dense motion field data to obtain a translation component; A trend characterization module, configured to perform trend characterization extraction to obtain a dynamic jitter component if the translation component is lower than a preset threshold; A filtering and separation module, configured to perform high-pass filtering and separation operation based on the dynamic jitter component to obtain an unexpected jitter trajectory; A degree-of-freedom transformation module, configured to obtain a dynamic compensation field through six-degree-of-freedom inverse transformation based on the unexpected jitter trajectory; An image generation module, configured to adjust the pixel positions of the image according to the dynamic compensation field, adaptively update the compensation parameters in combination with the endoscopic angle, and generate a stable image sequence; An iterative optimization module, configured to perform micro-jitter component compensation detection based on the stable image sequence, and if there is a micro-jitter component, iteratively optimize the parameters and update the compensation field, and output the final stable image.

[0016] Compared with the prior art, the present invention has the following beneficial effects: (1) Through the fusion technology of the optical flow analysis algorithm and bilinear interpolation, the present invention calculates the motion trajectory pixel by pixel, performs interpolation filling, smoothing optimization and iterative verification processing, expands the sparse motion field into continuous motion field data covering the entire frame, and provides high-precision motion information support for subsequent jitter separation and compensation; through the accurate reconstruction of the full-frame continuous motion field, the problem of compensation failure caused by sparse feature points in traditional methods is effectively solved, and high-precision motion estimation can be maintained even in areas with poor tissue texture. In the endoscopic surgery scenario, the sensitivity of jitter detection and the accuracy of compensation can be significantly improved, ensuring the real-time requirements of image stability; (2) Through three-dimensional geometric constraints and iterative verification, the present invention greatly improves the accuracy of translational component estimation, provides a basis for effectively distinguishing normal operation movement and unexpected jitter subsequently, enhances the pertinence of the compensation algorithm, and ensures a stable compensation effect in the dynamic endoscopic surgery scenario; (3) Through spatio-temporal motion feature analysis, dynamic region division and non-steady motion vector aggregation, the present invention separates high-frequency jitter components from the global motion field, can accurately distinguish normal movement and jitter interference, and provides targeted input for subsequent compensation. Specifically, the extraction of dynamic jitter components is realized based on the temporal characteristics and spatial distribution characteristics of the translational component, combined with an adaptive threshold and a motion pattern classification algorithm, significantly enhancing the sensitivity and robustness of jitter detection; (4) The present invention performs high-pass filtering separation operation according to the dynamic jitter component to obtain an unexpected jitter trajectory. Specifically, the generation of the unexpected jitter trajectory is realized based on frequency domain analysis, kinematic modeling and region segmentation technology, combined with adaptive filtering and parabolic fitting, significantly enhancing the accuracy and interpretability of the jitter trajectory; (5) The present invention adjusts the image pixel position according to the dynamic compensation field, combines the adaptive update of the endoscopic angle to compensate the parameters, and generates a stable image sequence; specifically, through pixel repositioning, parameter dynamic adjustment, compensation field optimization and artifact elimination processing, real-time image stable output is realized, which can eliminate pixel misalignment and blurring caused by jitter, and ensure the clarity and continuity of the surgical field of view; (6) Based on optical flow residual analysis and secondary compensation technology, the present invention performs compensation detection and iterative optimization of minute jitter components according to the stable image sequence to eliminate residual minute jitter and achieve high-precision stable output of the final image; among them, the iterative optimization is based on the spatio-temporal characteristics of the residual motion field, combined with adaptive parameter update and compensation field correction, significantly enhancing the fault tolerance and dynamic adaptability of the system, and finally effectively suppressing minute residual jitter that is difficult to eliminate by traditional methods, and improving the image stability during long-term surgery. Description of the Drawings

[0017] Figure 1 is a schematic flow chart of an intelligent jitter recognition and compensation method for endoscopic surgery images provided by an embodiment of the present invention; Figure 2 It is a schematic structural diagram of an intelligent jitter recognition and compensation system for endoscopic surgical images provided by an embodiment of the present invention. Specific implementation manners

[0018] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0019] Referring to Figure 1 , a first embodiment of the present invention provides an intelligent jitter recognition and compensation method for endoscopic surgical images, including the following steps: S11. Obtain the original image data of a continuous frame sequence, perform feature point extraction and motion field construction to obtain an initial motion vector distribution; S12. According to the initial motion vector distribution, perform dense motion estimation and interpolation filling to generate full-frame dense motion field data; S13. According to the full-frame dense motion field data, perform a six-degree-of-freedom decomposition operation to obtain a translation component; S14. If the translation component is lower than a preset threshold, perform trend characterization extraction to obtain a dynamic jitter component; S15. According to the dynamic jitter component, perform a high-pass filtering separation operation to obtain an unexpected jitter trajectory; S16. Based on the unexpected jitter trajectory, perform an inverse six-degree-of-freedom transformation to obtain a dynamic compensation field; S17. According to the dynamic compensation field, adjust the image pixel positions, adaptively update the compensation parameters in combination with the endoscopic angle, and generate a stable image sequence; S18. According to the stable image sequence, perform a micro jitter component compensation detection. If there is a micro jitter component, iterate and optimize the parameters and update the compensation field, and output the final stable image.

[0020] In step S11, the obtaining the original image data of a continuous frame sequence, performing feature point extraction and motion field construction to obtain an initial motion vector distribution includes: Obtain the original image data of a continuous frame sequence, perform gray-scale conversion processing to obtain gray-scale image data; According to the gray-scale image data, use the SIFT algorithm to perform feature point extraction to obtain significant feature points; According to the significant feature points, combine the Lucas-Kanade optical flow method to construct an initial motion field; Based on the initial motion field, perform motion field interpolation and smoothing to obtain the initial motion vector distribution.

[0021] It should be noted that obtaining the original image data of the continuous frame sequence, performing feature point extraction and motion field construction to obtain the initial motion vector distribution includes the following operation processes: First, perform grayscale conversion processing, convert the original RGB image into a grayscale image, and calculate the grayscale value by the weighted average method: , where the weight of the red channel is 0.299, the green channel is 0.587, and the blue channel is 0.114, 、 、 are the red, green, and blue channel values of the pixel point respectively, is the converted grayscale value. This processing can reduce the data dimension and retain the key brightness information, providing an efficient input for subsequent feature extraction. Then perform SIFT feature point extraction, and use the Scale-Invariant Feature Transform (SIFT) algorithm to detect significant feature points in the grayscale image. The specific process includes: constructing a multi-scale space, generating a scale space by convolving with Gaussian kernels of different scales, and calculating the difference between adjacent scales. The specific calculation formula is: ; where, is the image after Gaussian convolution, is the scale factor, so as to detect local extreme points to locate the key positions; assign key point directions, determine the main direction of the feature points based on the gradient direction histogram to ensure rotational invariance; finally, generate feature descriptors, extract local gradient distribution features centered on the key points and normalize them to form a 128-dimensional description vector. After that, use the Lucas-Kanade optical flow method to construct the initial motion field. Based on the assumption of brightness constancy between adjacent frames and the spatial consistency constraint, solve the optical flow equation within the local window around the feature points to calculate the motion vectors of the feature points. The calculation formula is as follows: ;

[0022] where, , is the image gradient, which is calculated by applying a gradient operator to the grayscale image; is the time derivative, which is approximately obtained by calculating the pixel difference between the current frame and the next frame; , is the optical flow vector, which is obtained by solving the optical flow equation. Optimize the displacement estimation by the least squares method to ensure the local smoothness and accuracy of the motion vectors. Finally, perform motion field interpolation and smoothing. For the area where feature points are missing, use the bilinear interpolation method. The calculation formula is as follows: , where, and The motion vectors corresponding to adjacent feature points are calculated by the optical flow method; are the coordinates of known grid points, forming the four corner points of the interpolation region; are the coordinates of the target interpolation point; adjacent motion vectors are used to fill in the blanks to ensure the continuity of the full-frame motion field; a Gaussian kernel function is applied to globally smooth the motion field, and the calculation formula is as follows: , where, is the standard deviation, which controls the smoothing intensity of the Gaussian kernel and is set by the user according to the noise intensity and application scenario; is the normalization factor, which ensures that the total weight of the Gaussian kernel is 1 and avoids the overall brightness shift of the filtered image; is the exponential term, which gives the highest weight to the center point and the weight decays with the distance.

[0023] In step S12, the generating of the full-frame dense motion field data by performing dense motion estimation and interpolation filling according to the initial motion vector distribution includes: Performing pixel-by-pixel dense motion trajectory calculation according to the initial motion vector distribution to obtain initial full-frame motion field data; Performing bilinear interpolation filling processing on the sparse region of feature points according to the initial full-frame motion field data to generate a continuous motion field distribution covering the full frame; Performing motion field smoothing processing according to the continuous motion field distribution to eliminate local motion noise and generate optimized motion field data; Performing motion field continuity verification and iterative recalculation processing according to the optimized motion field data to obtain full-frame dense motion field data.

[0024] It should be noted that the process of generating full-frame dense motion field data by performing dense motion estimation and interpolation filling according to the initial motion vector distribution refers to the process of expanding the sparse motion field into continuous motion field data covering the full frame through pixel-by-pixel motion trajectory calculation, interpolation filling, smoothing optimization, and iterative verification processing. This operation can provide high-precision motion information support for subsequent jitter separation and compensation. In the embodiments of the present invention, the generation of the dense motion field is based on the initial motion vector distribution and is realized through steps such as pixel-by-pixel optical flow estimation, bilinear interpolation, Gaussian smoothing, and iterative optimization. In the endoscopic surgery scenario, using the full-frame dense motion field can significantly improve the sensitivity of jitter detection and the accuracy of compensation, ensuring the real-time requirements of image stability.

[0025] Among them, the dense motion field generation is based on the initial motion vector distribution. Based on the initial motion vector distribution, the motion trajectories of all pixels in each frame are calculated. Specifically, first, a Gaussian pyramid is constructed for the input frame sequence to generate multi-scale image layers to adapt to pixels with different motion speeds. Then, in each layer of the pyramid, the Farneback algorithm is used to model the gray-scale changes in the local neighborhood, and the formula is: , where, is the coordinate vector, is the quadratic coefficient matrix, which describes the curvature information of the local area of the image and is calculated by polynomial fitting; is the linear term vector, which represents the gradient direction of the image gray scale and is calculated by the gradient operator; is the constant term, corresponding to the gray-scale mean value of the local area, which is directly obtained by statistical pixel brightness. Finally, the displacement vector between adjacent frames ( ) is solved by the least squares method to generate the initial full-frame motion field data. The bilinear interpolation filling process aims at the missing motion vectors in the sparse area of feature points and uses the adjacent known vectors for interpolation filling. Specifically, first, the image is divided into regular grids, with the coordinates of adjacent feature points , as the boundaries of the interpolation area. Then, according to the distance between the target point and the grid points, the interpolation weights are calculated dynamically, and the formula is: ; where, and correspond to the motion vectors of adjacent feature points respectively, which are calculated by the optical flow method; is the coordinate of the known grid point, forming the four corner points of the interpolation area; is the coordinate of the target interpolation point. The motion field smoothing process globally smooths the interpolated motion field to suppress noise and outliers. Specifically, first, the two-dimensional Gaussian function is applied to perform weighted averaging on the motion field, and the formula is: , where, is the standard deviation, which controls the smoothing intensity (default set to 1.5) and is set by the user according to the noise level; is the position coordinate of a certain point in the Gaussian kernel relative to the center, and the weights are calculated by traversing all positions in the kernel. Then, through the convolution operation, the abnormal motion vectors caused by sensor noise or instantaneous jitter are eliminated, thereby enhancing the global consistency of the motion field, reducing the interference of high-frequency noise on subsequent jitter compensation, and improving the reliability of the compensation parameters. The continuity verification and iterative recalculation operation verify the spatio-temporal continuity of the motion field, detect and correct inconsistent areas. Specifically, first, the motion field residual is calculated to identify the abnormal vectors deviating from the global trend, and the calculation formula is as follows: ; where, represents the coordinate The residual of the motion field at ; Indicates the number of pixels in the analysis window corresponding to the current coordinate (the value is 8×8 neighborhood, =64); Indicates Motion vectors for pixels (from the initial full-frame motion field data); Represents the median of the motion vector within the analysis window Ω; the Levenberg-Marquardt algorithm is used to update the motion estimation parameters to minimize the residual objective function: ,in, , is the optical flow vector component to be optimized, which is iteratively adjusted to minimize the brightness residual; For the moment The pixel brightness value is directly obtained from the image data; finally, the optical flow estimation and interpolation are re-performed on the abnormal area until the motion field meets the continuity threshold.

[0026] In step S13, performing a six-degree-of-freedom decomposition operation based on the full-frame dense motion field data to obtain a translation component includes: obtaining a depth value of each feature point in the motion field; According to the full-frame dense motion field data and the depth value, a rigid body motion hypothesis model is established through three-dimensional geometric constraints to obtain a rigid body motion hypothesis model; According to the rigid body motion assumption model, a six-degree-of-freedom parameter set is calculated to obtain a six-degree-of-freedom parameter set; According to the six-degree-of-freedom parameter set, motion components are separated and optimized to obtain translation components.

[0027] It should be noted that the process of performing a six-degree-of-freedom decomposition operation based on the full-frame dense motion field data to obtain the translation component refers to separating the camera's translation component from the global motion field through rigid body motion modeling, six-degree-of-freedom parameter calculation and motion component optimization processing. This operation can provide accurate translation motion parameters for jitter compensation. In an embodiment of the present invention, the six-degree-of-freedom decomposition is based on full-frame dense motion field data, combined with three-dimensional geometric constraints and optimization algorithms. In laparoscopic surgery scenarios, accurate extraction of translation components can effectively distinguish normal operation movements from unexpected jitters, thereby improving the pertinence of the compensation algorithm.

[0028] The depth value is obtained by directly measuring the depth value of each characteristic pixel in the motion field through the ToF depth sensor of the cavity mirror. The rigid body motion modeling operation is based on three-dimensional geometric constraints. It is assumed that the motion of the laparoscope is a rigid body motion (without deformation) and the mapping relationship between the camera motion and the image displacement is established. Specifically, firstly, the two-dimensional optical flow vector With depth value , The three-dimensional motion components are inversely deduced using the projection model: , where is the focal length, is the depth value, is the two-dimensional optical flow vector component, is the rate of change of depth over time, is the three-dimensional motion component; then the translation vector of the camera and the rotation matrix are defined. The motion equation of the three-dimensional point is: , where represents a 3×3 rotation matrix, describing the rotation of the camera around the three axes, represents a 3×1 translation vector, describing the displacement of the camera in three-dimensional space. The six-degree-of-freedom parameter calculation solves the translation and rotation parameters of the camera from the dense motion field. Specifically, first, using the matched feature point pairs, the essential matrix is estimated through the Random Sample Consensus (RANSAC) algorithm, satisfying: , where and are the normalized image coordinates; then the singular value decomposition is performed on the essential matrix , and the rotation matrix and the translation vector are extracted, satisfying: , , where is an anti-symmetric matrix, is the essential matrix, calculated from the matched point pairs, reflecting the epipolar geometric constraint of the camera motion, is the intermediate matrix of the SVD decomposition, used to separate the rotation and translation components. Finally, the motion component separation and optimization are performed. The translation component is extracted from the six-degree-of-freedom parameters and its accuracy is optimized. Specifically, first, the translation vector is directly taken as the preliminary result; then the translation components of consecutive frames are weighted and averaged to suppress instantaneous noise. The formula is: , where is the Gaussian weight, is the time window size, is the translation vector, directly obtained from the decomposition of the essential matrix.

[0029] In step S14, if the translation component is lower than the preset threshold, trend characterization extraction is performed to obtain the dynamic jitter component, including: According to the translation component, when the translation amount is lower than the preset threshold, spatio-temporal motion feature extraction is performed to obtain spatio-temporal motion features; According to the spatio-temporal motion features, dynamic jitter region division operation is performed to obtain the dynamic jitter region; Construct an unsteady motion vector set according to the dynamic jitter region to generate a dynamic jitter component.

[0030] It should be noted that the process of extracting the trend characterization to obtain the dynamic jitter component when the translation component is lower than the preset threshold refers to the process of separating the high-frequency jitter component from the global motion field through spatio-temporal motion feature analysis, dynamic region division, and unsteady motion vector aggregation. This operation can accurately distinguish normal movement from jitter interference and provide targeted input for subsequent compensation. In the embodiment of the present invention, the extraction of the dynamic jitter component is realized based on the temporal characteristics and spatial distribution characteristics of the translation component, and combined with the adaptive threshold and the motion pattern classification algorithm, which significantly improves the sensitivity and robustness of jitter detection.

[0031] It is worth noting that the spatio-temporal motion feature extraction operation extracts the spatio-temporal change features of the motion field when the translation component is lower than the preset threshold ( < 0.5 pixels / frame). Specifically, first perform a fast Fourier transform (FFT) on the translation component to extract the high-frequency components, and the formula is: , where represents the translation component of the th frame (from the six-degree-of-freedom decomposition result), represents the frequency-domain amplitude, reflecting the periodic jitter intensity of the translation component; then calculate the local variance of the motion field to identify the non-uniform motion region, and the formula is: , where represents the local motion variance, measuring the consistency of the motion within the region, represents the motion vector of the th pixel within the local window (from the dense optical flow field), represents the mean value of the motion vectors within the window (8×8 pixel window). The dynamic jitter region division operation divides the image into a jitter region and a non-jitter region according to the spatio-temporal motion features. The specific implementation process is as follows: First, use the K-means algorithm to cluster the local motion variance to divide the high-variance region (jitter) and the low-variance region (non-jitter); then combine the high-frequency energy distribution map (FFT amplitude) to generate a binary mask: ; where is the frequency-domain energy threshold (set to 0.8). The construction of the unsteady motion vector set aggregates the motion vectors of the jitter region to generate a dynamic jitter component. Specifically, first extract the motion vectors of the mask-marked region from the full-frame motion field; then remove the outliers based on the Mahalanobis distance, and the formula is: , where is the mean vector, is the covariance matrix, is the motion vector of the mask marking area. Finally, the filtered vector set is used as the output.

[0032] In step S15, the high-pass filtering separation operation is performed according to the dynamic jitter component to obtain an unexpected jitter trajectory, including: Performing frequency-domain decomposition processing on the dynamic jitter component to obtain a high-frequency jitter component; Performing jitter trajectory smoothing optimization on the high-frequency jitter component to obtain an optimized distribution of the jitter trajectory; Performing independent jitter region division according to the optimized distribution of the jitter trajectory to obtain an independent jitter motion region; According to the independent jitter motion region, constructing a parabolic motion model of the unexpected jitter trajectory and performing trajectory reconstruction processing to obtain an unexpected jitter trajectory.

[0033] It should be noted that the process of performing high-pass filtering separation operation according to the dynamic jitter component to obtain an unexpected jitter trajectory refers to extracting the physically meaningful unexpected jitter trajectory from the dynamic jitter component through frequency-domain decomposition, trajectory optimization, region division, and model reconstruction. This operation can separate high-frequency noise from real jitter and provide accurate trajectory input for the compensation algorithm. In the embodiment of the present invention, the generation of the unexpected jitter trajectory is realized based on frequency-domain analysis, kinematic modeling, and region segmentation techniques, combined with adaptive filtering and parabolic fitting, significantly improving the accuracy and interpretability of the jitter trajectory.

[0034] Among them, the frequency-domain decomposition processing separates the high-frequency jitter signal from the dynamic jitter component through a high-pass filter. The specific implementation process is as follows: First, the time-domain signal of the dynamic jitter component is converted into a frequency-domain signal , and the formula is: ; then the Butterworth high-pass filter is applied to retain the high-frequency components, and the formula is: , where is the cut-off frequency (default 5Hz), which is determined by historical data statistics and is used to distinguish high-frequency jitter from low-frequency normal motion, is the filter order (default 4th order), which controls the steepness of the transition band. The higher the order, the steeper the filtering. Through frequency-domain decomposition, the high-frequency jitter signal (such as hand tremor) and low-frequency interference (such as respiratory movement) are effectively separated, improving the pertinence of subsequent trajectory analysis. The jitter trajectory smoothing optimization process smooths the high-frequency jitter component, suppresses noise, and extracts the main motion trend. Specifically, first, the time-domain signal is subjected to mean filtering within the window, and the formula is: ; then the Gaussian kernel function is further applied to enhance local continuity, and the formula is: , where N is the size of the sliding window, which controls the smoothing intensity. The larger the window, the more significant the smoothing effect. is the standard deviation of the Gaussian kernel, which is dynamically adjusted by the signal-to-noise ratio of the jitter signal.

[0035] Among them, the operation of dividing independent jitter regions divides independent jitter regions according to the distribution of the smoothed trajectory. The specific implementation process: First, set the amplitude threshold , which controls the sensitivity of region division and is adjusted by the user according to the scenario. The regions with amplitudes higher than the threshold are marked as candidate jitter regions. Then, morphological closing operation is used to connect adjacent candidate regions to generate independent connected domains. The construction of the parabolic motion model and the trajectory reconstruction process fit the parabolic model to the independent jitter regions to reconstruct the unexpected jitter trajectory. The specific implementation: First, use the least squares method to fit the trajectory data points, and the model is: , where the coefficients represent the quadratic coefficients of the parabola in the x, y, and z components respectively, reflecting the acceleration characteristics of the jitter. represent the linear coefficients in the x, y, and z components respectively, describing the initial velocity of the jitter. represent the constant terms in the x, y, and z components respectively, corresponding to the initial position deviation, and are solved by minimizing the sum of the squares of the residuals. Then, the smoothed trajectory is generated according to the fitting parameters and aligned with the original data.

[0036] In step S16, the dynamic compensation field obtained by six-degree-of-freedom inverse transformation based on the unexpected jitter trajectory includes: Calculate the correction parameter matrix according to the unexpected jitter trajectory to obtain the correction parameter matrix; Construct an inverse kinematic model according to the correction parameter matrix, and decompose the translation correction vector and the rotation correction component; Construct and optimize the dynamic compensation vector field according to the translation correction vector and the rotation correction component to generate a pixel-level dynamic compensation field.

[0037] It should be noted that the process of generating the dynamic compensation field by six-degree-of-freedom inverse transformation based on the unexpected jitter trajectory refers to converting the jitter trajectory into a pixel-level compensation vector field through parameter matrix calculation, inverse kinematic modeling, and compensation field optimization. This operation can provide accurate spatial correction parameters for real-time jitter compensation. In the embodiments of the present invention, the generation of the dynamic compensation field is based on the kinematic characteristics of the unexpected jitter trajectory, combined with six-degree-of-freedom inverse transformation and optimization algorithms, significantly improving the local adaptability and global consistency of the compensation.

[0038] Among them, the operation of calculating the correction parameter matrix calculates the six-degree-of-freedom parameter matrix required for compensation according to the parabolic model parameters of the unexpected jitter trajectory. The specific implementation process: First, the parabolic model parameters Mapped to translational acceleration and rotational angular velocity , the formula is: , where is the translational acceleration, directly derived from the quadratic coefficient of the parabola, reflecting the acceleration characteristics of jitter; the rotational angular velocity is calculated from the linear coefficient and the depth , describing the rotational trend of jitter. is the scene depth estimate value, obtained through stereo matching; represents the camera calibration parameter (mm / pixel), that is, the physical length of each pixel, converting pixel displacement into actual physical displacement, and is used to calculate the rotational angular velocity . Then, a six-degree-of-freedom correction matrix is constructed, including translational and rotational components: . For the inverse kinematics model construction operation, the correction matrix is decomposed to obtain the translational correction vector and the rotational correction component . Specifically, first, a singular value decomposition (SVD) is performed on to separate the rotational and translational components: , where the translational part of the translational correction vector , and the rotational correction matrix , where is an anti-symmetric matrix, used to maintain the orthogonality of the rotation matrix.

[0039] It should be noted that the construction and optimization of the dynamic compensation field map the translational and rotational correction amounts to the image plane to generate a pixel-level dynamic compensation field.

[0040] In step S17, adjusting the image pixel positions according to the dynamic compensation field, adaptively updating the compensation parameters in combination with the endoscope angle, and generating a stable image sequence includes: Performing bilinear interpolation repositioning processing according to the dynamic compensation field to generate preliminary stable image data; Performing adaptive parameter adjustment and update according to the angle change data collected in real time by the endoscope attitude sensor to obtain the compensation intensity parameter; Performing spatial domain dynamic adjustment processing according to the compensation intensity parameter and the dynamic compensation field to generate optimized compensation field data; Performing motion artifact elimination processing according to the optimized compensation field data and the preliminary stable image data to generate a stable image sequence.

[0041] It should be noted that the process of adjusting the image pixel positions according to the dynamic compensation field, adaptively updating the compensation parameters in combination with the endoscope angle, and generating a stable image sequence refers to achieving real-time stable image output through pixel repositioning, parameter dynamic adjustment, compensation field optimization, and artifact elimination processing. This operation can eliminate pixel misalignment and blurring caused by jitter, ensuring the clarity and continuity of the surgical field of view. In the embodiments of the present invention, the generation of the stable image sequence is based on the deep fusion of the dynamic compensation field and the endoscope pose data, combined with the adaptive parameter update and image restoration technology, significantly improving the robustness and real-time performance of the image quality.

[0042] Among them, the bilinear interpolation repositioning process is to reposition the original image pixels according to the displacement vectors of the dynamic compensation field. The specific implementation process is as follows: map the displacement vector of each pixel in the compensation field to the original image coordinates; then perform weighted interpolation on the four adjacent pixels around the target position, and the formula is: , where represents the brightness value of the adjacent pixel, which is directly obtained from the original image; represents the boundary coordinates of the interpolation area, which are determined by the compensation vector. The adaptive parameter adjustment and update operation is to dynamically adjust the compensation intensity parameter according to the real-time angle data collected by the endoscope pose sensor. The specific implementation process is as follows: first establish the linear relationship between the endoscope pitch and the compensation intensity : , where, is the reference intensity (default 0.8), is the slope, controlling the sensitivity of the angle change to the compensation intensity (default 0.05 / °), represents the endoscope pitch angle, which is obtained in real time by the pose sensor. Then update the compensation intensity parameter for each frame to ensure enhanced compensation when the endoscope rotates rapidly and reduce the risk of overcompensation when it is stationary.

[0043] It should be noted that the spatial domain dynamic adjustment process is to optimize the spatial distribution of the compensation vector by combining the compensation intensity parameter and the dynamic compensation field. The specific implementation process is as follows: first weight the compensation intensity and the compensation field vector : , where represents the original compensation field; then use a bilateral filter to smooth the compensation field and retain the edge details, and the formula is: , where, represents the standard deviation of the Gaussian kernel (default 1.5), controlling the spatial smoothing range; represents the brightness difference threshold (default 10), protecting the compensation vector in the edge area, Represents the compensated field after weighting. Motion artifact elimination processing is to eliminate the remaining artifacts (such as ghosting or blurring) during the compensation process. The specific implementation is as follows: First, calculate the optical flow residual between the stable image and the original image to detect the artifact area. Then, adopt a restoration model based on U-Net to perform pixel-level restoration on the artifact area. The loss function is: , where is the pixel reconstruction loss, is the perceptual loss, are the loss weights (set to 0.7 and 0.3) to balance pixel accuracy and semantic consistency.

[0044] In step S18, the detection of micro-jitter component compensation is performed according to the stable image sequence. If there is a micro-jitter component, the parameters are iteratively optimized and the compensation field is updated, and the final stable image is output, including: According to the stable image sequence, a distribution is constructed through Lucas-Kanade optical flow analysis to obtain the residual motion field distribution; According to the residual motion field distribution, amplitude statistics are performed to determine whether there is a micro-jitter component exceeding the set jitter threshold; When a micro-jitter component is detected, according to the residual motion field, the compensation parameters are optimized and updated to obtain the optimized compensation parameters; According to the optimized compensation parameters, the motion trajectory of the dynamic compensation field is corrected to generate optimized compensation field data; According to the optimized compensation field data, secondary motion compensation processing is performed on the stable image sequence to output the final stable image.

[0045] It should be noted that the process of detecting micro-jitter component compensation and iterative optimization according to the stable image sequence refers to eliminating the remaining micro-jitter through residual motion field construction, threshold detection, parameter optimization, and secondary compensation processing to achieve high-precision stable output of the final image. This operation can further improve the robustness of the compensation algorithm and ensure the clinical usability of the image quality. In the embodiments of the present invention, the iterative optimization is based on the spatio-temporal characteristics of the residual motion field, combined with adaptive parameter update and compensation field correction, significantly improving the fault tolerance and dynamic adaptability of the system.

[0046] Among them, the operation of constructing the residual motion field distribution analyzes the residual motion between the stable image sequence and the original image through the Lucas-Kanade optical flow method. The specific implementation process is as follows: Based on the assumption of brightness constancy, calculate the residual optical flow vector between the stable image and the original image , and the formula is: , where is the image spatial gradient, calculated by the Sobel operator; is the time derivative, calculated by the inter-frame difference between the stabilized image and the original image; thereby generating a residual motion field distribution to quantify the small jitters remaining after compensation. The small jitter component detection is to statistically calculate the amplitude of the residual motion field and determine whether it exceeds a preset threshold. Specifically, first calculate the amplitude of the residual vector for each pixel: , where , is the residual optical flow vector at the pixel point ( ); then statistically calculate the proportion of pixels in the entire frame whose amplitude exceeds the threshold . If , it is determined that significant small jitters exist. The compensation parameter optimization and update operation is to optimize the compensation parameters based on the residual motion field to improve the compensation accuracy. The specific implementation process is as follows: First, minimize the total energy of the residual motion field, and the formula is: ; where, is the total energy of the residual motion field, representing the cumulative intensity of the residual jitters of all pixels, calculated by summing the squares of the magnitudes of the residual vectors of each pixel in the residual motion field, and is used to quantify the overall level of the residual jitters after compensation; is the magnitude of the residual vector, that is, the residual motion amplitude at the pixel point ( ), obtained by calculating the residual optical flow vector between the stabilized image and the original image through the Lucas-Kanade optical flow method; then iteratively adjust the compensation field parameter , and the update formula is: , where, is the compensation field parameter, representing the variable to be optimized in the dynamic compensation field; is the step coefficient of gradient descent, controlling the amplitude of parameter update, set to 0.01 according to experimental calibration; is the loss function with respect to the parameter , indicating the parameter adjustment direction. The dynamic compensation field correction process is to correct the compensation field according to the optimized parameters to generate optimized compensation field data. The specific implementation process is as follows: First, map the optimized parameter to the compensation field and update the displacement vector ; then apply the Kalman filter to perform temporal smoothing on the compensation field, and the formula is: , where, is the state transition and observation matrix, defined by the motion model; is the Kalman gain, which dynamically adjusts the weights of prediction and observation and is set by the user according to the actual scenario. Finally, secondary motion compensation processing is performed, and the optimized compensation field is applied to the stable image sequence for secondary compensation. Specifically, first, the optimized compensation field is applied to the stable image by bilinear interpolation, and the formula is the same as in step S17; then, the residual motion field after secondary compensation is calculated. If the amplitude meets the standard ( ), the final image is output; otherwise, a new round of optimization is triggered. Through closed-loop iteration, the residual jitter is completely eliminated to ensure that the output image meets the clinical stability standard.

[0047] Referring to Figure 2 , the second embodiment of the present invention provides an intelligent jitter recognition and compensation system for endoscopic surgical images, including: A feature point extraction module, configured to obtain the original image data of a continuous frame sequence, perform feature point extraction and construction of a motion field, and obtain an initial motion vector distribution; A motion estimation module, configured to perform dense motion estimation and interpolation filling according to the initial motion vector distribution to generate full-frame dense motion field data; A degree-of-freedom decomposition module, configured to perform six-degree-of-freedom decomposition operations according to the full-frame dense motion field data to obtain a translation component; A trend characterization module, configured to perform trend characterization extraction to obtain a dynamic jitter component if the translation component is lower than a preset threshold; A filtering and separation module, configured to perform high-pass filtering and separation operations according to the dynamic jitter component to obtain an unexpected jitter trajectory; A degree-of-freedom transformation module, configured to obtain a dynamic compensation field through six-degree-of-freedom inverse transformation based on the unexpected jitter trajectory; An image generation module, configured to adjust the pixel positions of the image according to the dynamic compensation field, adaptively update the compensation parameters in combination with the endoscopic angle, and generate a stable image sequence; An iterative optimization module, configured to perform detection of compensation for minute jitter components according to the stable image sequence. If there are minute jitter components, the parameters are iteratively optimized and the compensation field is updated to output the final stable image.

[0048] It should be noted that the intelligent jitter recognition and compensation system for endoscopic surgical images provided in the embodiments of the present invention is used to execute all the process steps of the intelligent jitter recognition and compensation method for endoscopic surgical images in the above embodiments. The working principles and beneficial effects of the two correspond one by one, so they will not be elaborated here.

[0049] An embodiment of the present invention also provides a terminal device. The terminal device includes: a processor, a memory, and a computer program stored in the memory and executable on the processor, such as an intelligent jitter recognition and compensation program for endoscopic surgery images. When the processor executes the computer program, the steps in the embodiments of the above-mentioned intelligent jitter recognition and compensation method for endoscopic surgery images are implemented, such as Figure 1 the step S11 shown. Alternatively, when the processor executes the computer program, the functions of each module / unit in the above-mentioned system embodiments are implemented.

[0050] Exemplarily, the computer program can be divided into one or more modules / units. The one or more modules / units are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the computer program in the terminal device.

[0051] The terminal device can be a computing device such as a desktop computer, a notebook, a palm computer, and a smart tablet. The terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art can understand that the above components are only examples of the terminal device and do not constitute a limitation on the terminal device. It may include more or fewer components than the above, or combine certain components, or different components. For example, the terminal device may further include input / output devices, network access devices, a bus, etc.

[0052] The so-called processor may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The processor is the control center of the terminal device and connects various parts of the entire terminal device through various interfaces and lines.

[0053] The memory can be used to store the computer programs and / or modules. By running or executing the computer programs and / or modules stored in the memory, and invoking the data stored in the memory, the processor implements various functions of the terminal device. The memory mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created according to the use of the mobile phone (such as audio data, phone book, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as a hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage devices.

[0054] Among them, if the modules / units integrated in the terminal device are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, to implement all or part of the processes in the above embodiment methods of the present invention, it can also be completed by a computer program instructing relevant hardware. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or system that can carry the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0055] It should be noted that the system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, in the attached drawings of the system embodiments provided by the present invention, the connection relationship between modules indicates that they have a communication connection, which can be specifically implemented as one or more communication buses or signal lines. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0056] The specific embodiments described above further elaborate on the purpose, technical solutions, and beneficial effects of the present invention. It should be understood that the above description is only for the specific embodiments of the present invention and is not used to limit the protection scope of the present invention. In particular, it is pointed out that for those skilled in the art, any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An intelligent jitter recognition and compensation method for laparoscopic surgery images, characterized in that: include: Obtain the original image data of the continuous frame sequence, perform feature point extraction and motion field construction, and obtain the initial motion vector distribution; According to the initial motion vector distribution, dense motion estimation and interpolation filling are performed to generate full-frame dense motion field data; Performing a six-degree-of-freedom decomposition operation according to the full-frame dense motion field data to obtain a translation component; If the translation component is lower than a preset threshold, a trend characterization extraction is performed to obtain a dynamic jitter component; According to the dynamic jitter component, a high-pass filter separation operation is performed to obtain an unexpected jitter trajectory; Based on the unexpected jitter trajectory, a dynamic compensation field is obtained through a six-degree-of-freedom inverse transformation; According to the dynamic compensation field, the image pixel position is adjusted, and the compensation parameters are adaptively updated in combination with the cavity mirror angle to generate a stable image sequence; According to the stable image sequence, a small jitter component compensation detection is performed, and if a small jitter component exists, the parameters are iteratively optimized and the compensation field is updated to output a final stable image.

2. The intelligent jitter recognition and compensation method for laparoscopic surgery images according to claim 1 is characterized in that: The method of obtaining the original image data of the continuous frame sequence, extracting feature points and constructing the motion field to obtain the initial motion vector distribution includes: Acquire the original image data of the continuous frame sequence, perform grayscale conversion processing, and obtain grayscale image data; According to the grayscale image data, SIFT algorithm is used to extract feature points to obtain significant feature points; According to the significant feature points, an initial motion field is constructed by combining the Lucas-Kanade optical flow method; According to the initial motion field, motion field interpolation and smoothing processing are performed to obtain initial motion vector distribution.

3. The intelligent jitter recognition and compensation method for laparoscopic surgery images according to claim 1 is characterized in that: The method of performing dense motion estimation and interpolation filling according to the initial motion vector distribution to generate full-frame dense motion field data includes: According to the initial motion vector distribution, dense motion trajectory calculation is performed at a pixel level to obtain initial full-frame motion field data; According to the initial full-frame motion field data, a bilinear interpolation filling process is performed on a sparse feature point region to generate a continuous motion field distribution covering the full frame; According to the continuous motion field distribution, a motion field smoothing process is performed to eliminate local motion noise and generate optimized motion field data; According to the optimized motion field data, motion field continuity verification and iterative recalculation processing are performed to obtain full-frame dense motion field data.

4. The intelligent jitter recognition and compensation method for laparoscopic surgery images according to claim 1, characterized in that: The step of performing a six-degree-of-freedom decomposition operation according to the full-frame dense motion field data to obtain a translation component includes: Get the depth value of each feature point in the motion field; According to the full-frame dense motion field data and the depth value, a rigid body motion hypothesis model is established through three-dimensional geometric constraints to obtain a rigid body motion hypothesis model; According to the rigid body motion assumption model, a six-degree-of-freedom parameter set is calculated to obtain a six-degree-of-freedom parameter set; According to the six-degree-of-freedom parameter set, motion components are separated and optimized to obtain translation components.

5. The intelligent jitter recognition and compensation method for laparoscopic surgery images according to claim 1, characterized in that: If the translation component is lower than a preset threshold, performing trend characterization extraction to obtain a dynamic jitter component includes: According to the translation component, when the translation amount is lower than a preset threshold, spatiotemporal motion feature extraction is performed to obtain spatiotemporal motion features; According to the spatiotemporal motion characteristics, a dynamic jitter region division operation is performed to obtain a dynamic jitter region; According to the dynamic shaking area, a non-steady-state motion vector set is constructed to generate a dynamic shaking component.

6. The intelligent jitter recognition and compensation method for laparoscopic surgery images according to claim 1, characterized in that: The step of performing a high-pass filtering separation operation according to the dynamic jitter component to obtain an unexpected jitter trajectory includes: Performing frequency domain decomposition processing according to the dynamic jitter component to obtain a high-frequency jitter component; According to the high-frequency jitter component, a jitter trajectory smoothing optimization is performed to obtain a jitter trajectory optimized distribution; According to the jitter trajectory optimization distribution, the independent jitter area is divided to obtain the independent jitter motion area; According to the independent shaking motion area, a parabolic motion model of the unexpected shaking trajectory is constructed and trajectory reconstruction processing is performed to obtain the unexpected shaking trajectory.

7. The intelligent jitter recognition and compensation method for laparoscopic surgery images according to claim 1 is characterized in that: The method of obtaining a dynamic compensation field based on the unexpected jitter trajectory through a six-degree-of-freedom inverse transformation includes: Calculating a correction parameter matrix according to the unexpected jitter trajectory to obtain a correction parameter matrix; According to the correction parameter matrix, an inverse kinematics model is constructed to decompose a translation correction vector and a rotation correction component; According to the translation correction vector and the rotation correction component, a dynamic compensation vector field is constructed and optimized to generate a pixel-level dynamic compensation field.

8. The intelligent jitter recognition and compensation method for laparoscopic surgery images according to claim 1, characterized in that: The method of adjusting the image pixel position according to the dynamic compensation field and adaptively updating the compensation parameters in combination with the cavity mirror angle to generate a stable image sequence includes: According to the dynamic compensation field, a bilinear interpolation repositioning process is performed to generate preliminary stabilized image data; According to the angle change data collected in real time by the laparoscope attitude sensor, adaptive parameter adjustment and update are performed to obtain compensation intensity parameters; Performing spatial domain dynamic adjustment processing according to the compensation intensity parameter and the dynamic compensation field to generate optimized compensation field data; According to the optimized compensation field data and the preliminary stabilized image data, motion artifact elimination processing is performed to generate a stabilized image sequence.

9. The intelligent jitter recognition and compensation method for laparoscopic surgery images according to claim 1, characterized in that: The method of performing a small jitter component compensation detection according to the stable image sequence, iteratively optimizing parameters and updating the compensation field if a small jitter component exists, and outputting a final stable image includes: According to the stable image sequence, a distribution is constructed by Lucas-Kanade optical flow analysis method to obtain a residual motion field distribution; According to the residual motion field distribution, amplitude statistics are performed to determine whether there is a small jitter component exceeding a set jitter threshold; When a tiny jitter component is detected, the compensation parameters are optimized and updated according to the residual motion field to obtain the optimized compensation parameters; According to the optimized compensation parameters, the dynamic compensation field is subjected to motion trajectory correction processing to generate optimized compensation field data; According to the optimized compensation field data, secondary motion compensation processing is performed on the stabilized image sequence to output a final stabilized image.

10. An intelligent jitter recognition and compensation system for laparoscopic surgery images, characterized in that: include: The feature point extraction module is used to obtain the original image data of the continuous frame sequence, perform feature point extraction and motion field construction, and obtain the initial motion vector distribution; A motion estimation module, used to perform dense motion estimation and interpolation filling according to the initial motion vector distribution to generate full-frame dense motion field data; A degree of freedom decomposition module, used for performing a six-degree-of-freedom decomposition operation according to the full-frame dense motion field data to obtain a translation component; A trend characterization module, used for performing trend characterization extraction to obtain a dynamic jitter component if the translation component is lower than a preset threshold; A filtering and separation module, used for performing a high-pass filtering and separation operation according to the dynamic jitter component to obtain an unexpected jitter trajectory; A degree of freedom transformation module, used for obtaining a dynamic compensation field through a six-degree-of-freedom inverse transformation based on the unexpected jitter trajectory; An image generation module, used to adjust the image pixel position according to the dynamic compensation field, adaptively update the compensation parameters in combination with the cavity mirror angle, and generate a stable image sequence; The iterative optimization module is used to perform compensation detection of small jitter components according to the stable image sequence, iteratively optimize parameters and update the compensation field if small jitter components exist, and output a final stable image.

Images