An endoscope image anti-shake method based on motion vector compensation

Abstract

The present application belongs to the technical field of endoscope and image processing, and in particular to an endoscope image anti-shake method based on motion vector compensation, comprising the following steps: collecting a sequence of continuous video frames, performing stability screening on the initial feature point set, eliminating unstable feature points located in the edge region, high light reflection region and low contrast region of the image to form a reliable feature point set, performing robust estimation processing on the original motion vector set based on the matched feature points in the reliable feature point set to generate a stabilized output frame, wherein the remapping adopts a bilinear interpolation algorithm; by constructing a closed-loop anti-shake system taking local feature motion vectors as input, spatio-temporal domain joint filtering as core and sub-pixel remapping as output, high-precision separation and compensation of high-frequency micro-shaking in endoscope imaging are realized without introducing additional mechanical structures.

Description

Technical Field

[0001] This invention belongs to the field of endoscope and image processing technology, specifically a method for stabilizing endoscopic images based on motion vector compensation. Background Technology

[0002] Industrial endoscopes, as key tools for non-destructive testing and precision observation, are widely used in aerospace, energy and power, machinery manufacturing, and medical diagnostics. Their core value lies in transmitting the internal structural details and state information of the inspected object through high-fidelity images. In such applications, image stability is directly related to the accuracy of the test results and the reliability of the operator's judgment. However, due to the physical characteristics of the slender and flexible probe of the endoscope and its complex operation in confined spaces, it is easily affected by a variety of disturbances during the imaging process. These include high-frequency micro-shaking introduced by the operator's hand, mechanical vibration generated when the probe is pushed and rotated in the pipe or cavity, and instantaneous displacement of the lens tip due to contact with tissue or wall. Although these disturbances are small in amplitude, they are enough to cause significant image jumps under high magnification, seriously weakening the usability and continuity of the image.

[0003] To address these challenges, existing technologies mainly follow two paths: one is optical image stabilization based on mechanical structures, which integrates movable lenses or sensor platforms into the lens module and uses actuators to cancel external disturbances in real time; the other is electronic image stabilization based on image processing, which usually relies on a global motion model (such as affine transformation or translation model) to estimate and compensate for the overall displacement between consecutive frames. The former is relatively mature in consumer-grade camera devices, but its high requirements for space volume, power consumption and structural complexity make it difficult to adapt to industrial endoscope probes with diameters often less than 5 mm. Especially in the compact layout that needs to take into account illumination, imaging and working channels, it is almost not feasible in engineering. The latter, with the advantage of pure software implementation, has been widely used in embedded systems. Its typical process includes feature matching, motion estimation, trajectory smoothing and image remapping, which can theoretically improve image stability without changing the hardware. As industrial inspection scenarios place increasingly stringent demands on image quality and operational responsiveness, the inherent limitations of the aforementioned electronic image stabilization solutions at the principle level are gradually becoming apparent, exposing deep-seated technical contradictions. Specifically, traditional methods generally assume that scene motion has global rigidity, meaning that the entire image follows a uniform translation or rotation law. This assumption may hold approximately in open, textured environments, but under the unique imaging conditions of endoscopy, such as highly reflective areas on the inner walls of metal pipes, weakly textured areas on the surface of biological tissues, or complex cavities with local deformation and occlusion, feature point extraction is prone to failure or a large number of mismatches. More importantly, endoscopic operations are often accompanied by non-rigid movements such as relative sliding between the probe and the object being examined, elastic deformation of tissues, or fluid flow. These local dynamic behaviors and global jitter may overlap in the frequency domain, causing motion estimation algorithms to be unable to effectively distinguish between the true operational intent and unintentional disturbances. Based on this, if a global model is forcibly used for compensation, not only will additional distortion be introduced due to incorrect estimation, but the doctor's intentional operation of slowly advancing or finely rotating may also be misjudged as shaking and filtered out, causing the screen to become sticky or the response to be slow, which seriously affects the user experience and diagnostic efficiency. Furthermore, the fixed cropping strategy adopted to cover up the missing edges of the compensated image often excessively sacrifices the already limited field of view, further weakening the endoscope's observation capabilities in narrow spaces.

[0004] Ultimately, the core dilemma of existing electronic image stabilization technology lies in the irreconcilable structural conflict between the global rigid motion model it relies on and the local non-rigidity, low signal-to-noise ratio, and high dynamic interference that are common in actual endoscopic imaging. This conflict not only causes motion estimation to fail in key areas, but also makes the subsequent jitter separation and compensation logic based on unreliable inputs, thus leading to a series of secondary problems such as insufficient compensation, over-smoothing, or loss of field of view.

[0005] Therefore, a method for stabilizing endoscopic images based on motion vector compensation is proposed to address the above problems. Summary of the Invention

[0006] The purpose of this invention is to address the shortcomings of existing technologies by providing an endoscopic image stabilization method based on motion vector compensation, thereby solving the technical problems mentioned in the background art.

[0007] To address the above technical problems, the following technical solution is adopted: an endoscopic image stabilization method based on motion vector compensation, comprising the following deterministic steps: First, a continuous sequence of video frames is acquired using an endoscopic image sensor. Let the first frame be... Frame image is ( , ,in , These are pixel coordinates; subsequently, for the current frame... With the previous frame Perform feature point extraction operations separately, and use a corner detection algorithm to extract an initial set of feature points. = The corner detection algorithm is any one of the FAST algorithm, ORB algorithm, or Shi-Tomasi algorithm; The extracted feature points are subjected to stability screening, and unstable feature points located in image edge areas, high reflectivity areas and low contrast areas are removed to form a set of reliable feature points after screening.

[0008] After obtaining a reliable set of feature points, the feature point pairs that match successfully in the current frame and the previous frame are compared. , Calculate the local motion vector =( , ),in = - , = - , Indicates the first The two-dimensional vectors corresponding to the inter-frame displacements of each feature point; all local motion vectors constitute the original motion vector set. To eliminate the contamination of the overall estimation by mismatched vectors caused by occlusion, tissue deformation, or noise interference, robust estimation processing is performed on the original motion vector set. Specifically, this includes: filtering all vector components separately using a median filter, or fitting the optimal translation model using the RANSAC algorithm and removing outlier vectors; based on this, the global reference motion vector is calculated. It is defined as the median vector of all retained vectors, i.e. =median( ), or the global translation vector obtained by fitting using the least squares method.

[0009] Preferably, the global reference motion vector Considered as low-frequency real-world motion components With high-frequency jitter components It is formed by stacking, that is = + To separate the two, a time-domain low-pass filtering mechanism is introduced, and a smooth motion trajectory is generated through an exponentially weighted moving average. Its recursive relation is = · -1smooth+(1− )· ,in The smoothing coefficient is preset and its value is strictly limited to the open interval (0,1); this smoothing trajectory It characterizes the true motion trend at low frequencies after removing high-frequency disturbances.

[0010] Based on the difference between the smoothed trajectory and the original motion estimate of the current frame, the jitter cancellation vector for image compensation is calculated. Its expression is = - The vector This is the amount of compensation that needs to be applied to the current frame to counteract the effects of jitter; based on the compensation vector. =(cx,cy), for the current frame Perform image remapping operation to generate a stable output frame. The remapping process uses a bilinear interpolation algorithm to complete sub-pixel level image sampling, ensuring that the compensated image has no obvious jagged edges or blur.

[0011] To address the issue of missing pixels in boundary regions caused by image compensation, this invention employs a dynamic boundary processing strategy: a fixed-width safety cropping area is reserved around the original imaging area of ​​the image sensor. This area does not participate in the final display but is used to accommodate the maximum expected compensation displacement. After each compensation, only the central visible area is output, with a constant size that is smaller than the original image size. As an alternative, an edge content filling mechanism can be used to extrapolate and fill the missing boundaries using the gradient direction and intensity information of neighboring pixels, or a context-aware image expansion algorithm can be used to generate visually coherent boundary content.

[0012] The overall data flow of the method is uniformly scheduled by a central collaborative controller. The feature extraction module, motion vector calculation module, robust estimation module, trajectory smoothing module, compensation calculation module, and image remapping module are sequentially connected in series and intermediate data is passed through a shared memory buffer. Each module is deployed in an embedded image signal processor, and its timing control is triggered by the video frame synchronization signal to ensure that the processing delay of each frame does not exceed a single frame period. The upper limit of the number of feature points is rigidly limited to a preset threshold N. When the initial number of extracted points exceeds N, it is truncated to N points in descending order of response strength to ensure real-time performance.

[0013] Preferably, the corner detection algorithm uses the ORB algorithm, with its scale level fixed at a single scale, and direction calculation is disabled to reduce computational load; feature matching uses the Hamming distance nearest neighbor ratio criterion, with the ratio threshold set to 0.8; the RANSAC iteration count is fixed at 100 times, and the interior point determination distance threshold is set to 2 pixels; the smoothing coefficient... Dynamically adjusts based on the current frame rate; the higher the frame rate, the more... The closer it is to 1, the better, to maintain a balance between trajectory smoothness and response speed.

[0014] Preferably, the width of the safe cropping area is dynamically updated based on the historical maximum compensation amplitude. The update rule is as follows: if the compensation amplitude exceeds half the current cropping area width in K consecutive frames, the cropping area width is automatically expanded to 1.2 times the maximum amplitude, and an attenuation mechanism is set to gradually shrink it in subsequent cases where there is no over-limit, so as to avoid long-term excessive sacrifice of the field of view.

[0015] The method requires no hardware modifications to the endoscope probe structure. All processing logic runs on the back-end image processing unit, and it is compatible with different resolutions (including but not limited to 640×480, 1280×720, 1920×1080) and different frame rates (including but not limited to 30fps, 60fps) of input video streams. Its algorithm flow is completely determined, with no branch selection or conditional jump relying on runtime judgments other than external configuration. All parameters are fixed during the system initialization phase or set by receiving external instructions through a standard communication interface.

[0016] Preferably, a motion consistency verification unit is provided between the motion vector robust estimation module and the trajectory smoothing module. This unit continuously monitors multiple consecutive frames. If the rate of change is detected and the magnitude of the mutation exceeds a preset threshold, the system will be temporarily frozen. Updated to prevent trajectory distortion caused by sudden occlusion or strong interference; the mutation threshold is adaptively adjusted according to the complexity of the image content, which is estimated by the local gradient entropy.

[0017] Preferably, the image remapping module and the boundary processing module are integrated into the same hardware acceleration unit, and a pipelined architecture is used to achieve parallel processing of pixel address generation, interpolation weight calculation, and boundary filling; the address generator receives the compensation vector. Then, the source coordinates of each output pixel in the input frame are calculated in real time, and it is simultaneously determined whether the coordinates fall within the valid image area; if they fall outside the cropping area, the filling logic is triggered, otherwise bilinear interpolation is started.

[0018] The beneficial effects of this invention are: The characteristics of weak texture, strong reflection, narrow field of view and local non-rigid motion in the process of endoscopic imaging are addressed by using a motion vector-based image stabilization method to improve the stability and reliability of image stabilization in complex endoscopic environments. It can effectively separate jittery motion from real operation motion. By performing statistical analysis and time-domain filtering on inter-frame motion vectors, it can separate high-frequency, small-amplitude jittery components from low-frequency, continuous real scene motion, avoiding misjudging the doctor's intentional operation as jitter, thereby preventing the image from being overly smoothed. The image stabilization effect is stable and the image continuity is good. It constructs a smooth motion trajectory based on continuous frame motion vectors. The image stabilization compensation process is continuous and gradual, which effectively reduces the phenomenon of image jumps and abrupt changes, and significantly improves the visual stability of endoscopic video. This invention adopts a motion vector-based compensation method, which does not require complex 3D modeling or high-order optical flow calculations. It has a small computational load and is easy to implement in endoscope host, embedded processor or real-time video system, meeting the real-time display requirements. It has low hardware dependence, strong system adaptability, and does not require additional mechanical modifications to the endoscope lens or imaging structure. It can be directly deployed in the existing endoscope system through software and is suitable for endoscope devices of different models, resolutions and frame rates. With strong robustness and high anti-interference ability, this invention can effectively suppress erroneous motion estimation caused by reflection, occlusion, tissue deformation or noise by performing anomaly removal and robust estimation on motion vectors, thereby improving the reliability of the anti-shake algorithm in actual clinical environments. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] In the attached diagram: Figure 1 This is a schematic diagram of the overall process of the endoscopic image stabilization method based on motion vector compensation described in this invention; Figure 2 This is a schematic diagram of the module structure for local motion vector calculation and robust estimation processing in this invention; Figure 3 This is a schematic diagram of the image remapping and dynamic boundary processing mechanism in this invention. Detailed Implementation

[0021] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0022] Specific implementation examples are given below.

[0023] Example Please see Figures 1-3This invention provides an endoscopic image stabilization method based on motion vector compensation, comprising an input to the endoscopic image stabilization system being a sequence of digital video frames continuously output by an image sensor at the front end of an endoscope, wherein the input to the endoscopic image stabilization system is a sequence of digital video frames continuously output by an image sensor at the front end of an endoscope, and the input to the endoscopic image stabilization system is ... and the input to the endoscopic image stabilization system is a sequence of digital video frames continuously output by an image sensor at the front end of an endoscope, and the input to the endoscopic image stabilization system is a sequence of digital video frames continuously output by an image sensor at the front end of an endoscope, and the input to the endoscopic image stabilization system is a sequence of digital video frames continuously output by an image sensor at the front end of an endoscope, and the input to the endoscopic image stabilization system is a sequence of digital video frames continuously output by an image sensor at the front end of an endoscope, and the input to the endoscopic image stabilization system is a sequence of digital video frames continuously output by Frame image denoted as ( , ,in , These represent the horizontal and vertical pixel coordinates in the image plane, respectively. The frame number and The system processes the sequence frame by frame, with each frame's processing cycle strictly limited to a single frame time interval to ensure real-time performance. All processing logic is deployed in an embedded image signal processor (ISP), which integrates a dedicated hardware acceleration unit for performing computationally intensive tasks such as feature extraction, motion estimation, and image remapping.

[0024] After system startup, various operating parameters are initialized first. The upper limit of the number of feature points N is set to a fixed value, typically 500. The corner detection algorithm is selected as ORB algorithm, with its scale level fixed at a single scale, and the direction calculation function is disabled to reduce computational complexity. Feature matching adopts the Hamming distance nearest neighbor ratio criterion, with the ratio threshold preset to 0.8. The number of iterations of the RANSAC algorithm is fixed at 100, and the inlier detection distance threshold is set to 2 pixels. The initial value of the smoothing coefficient α is set according to the system's nominal frame rate, for example, α=0.9 at 30fps and α=0.95 at 60fps. The initial width of the safe cropping area is set to 5% of the length of the short side of the image. The initial value of the mutation threshold of the motion consistency verification unit is set to 3 pixels / frame, and will be dynamically adjusted according to the content complexity.

[0025] After parameter initialization is complete, the system enters the main processing loop, processing each input frame. First, feature point extraction is performed; Specifically, for the current frame With the previous frame Run the ORB corner detection algorithm independently to generate an initial set of feature points. = and = ; Each feature point From its pixel coordinates ( , Together with the corresponding descriptor vectors, they represent the characteristics; To control computational load, if the total number of feature points extracted in a frame exceeds a preset limit... Then, they are sorted in descending order of characteristic response intensity, and the first few are truncated and retained. One point.

[0026] Subsequently, and The feature points in the descriptor are matched. The matching process is based on the Hamming distance between descriptors, and the nearest neighbor ratio criterion is used to select reliable matching pairs. any feature point in Calculate its relationship with Find the minimum Hamming distance among all feature points in the dataset. and the second smallest distance ,like Then determine ( , A pair of valid matches is considered a preliminary matching set. {( ,pit)}.

[0027] In obtaining the matching set Then, feature point stability screening is performed. The screening rules include: removing feature points whose coordinates are located in the image edge region (defined as the region less than 10 pixels from the image boundary); removing feature points located in the highly reflective region, which is identified by the standard that the local brightness standard deviation exceeds twice the global mean; and removing feature points located in the low contrast region, which is identified by the standard that the local gradient magnitude is lower than a preset threshold (e.g., 10 gray levels). After screening, the retained matching pairs constitute a reliable feature point set. .

[0028] Based on reliable feature point sets Calculate the local motion vectors corresponding to each matching pair; For the Matching pairs ( , ), its local motion vector Defined as =( , ), where dxi = xit− = - , = - All local motion vectors constitute the original motion vector set. .

[0029] To eliminate anomalous vectors introduced by mismatches or non-rigid deformations, robust estimation processing is performed on the original set of motion vectors; This processing can be performed in one of two modes: the first mode is component median filtering, that is, filtering each component separately. Take the median as the global value Components, for all Take the median as the global value The second mode is RANSAC fitting, which involves randomly sampling the minimum point set (here, 1 point, because the model is a pure translation) from the original vector set, fitting the global translation vector, and counting the number of interior points. After repeating this process 100 times, the model with the most interior points is selected as the optimal estimate, and the vector corresponding to its interior points is retained. Regardless of the mode used, the final output is a purified subset of motion vectors. .

[0030] Based on this, the global reference motion vector is calculated. , Defined as a purified vector subset The median vector, i.e. , ( The vector represents the overall displacement estimate of the current frame relative to the previous frame.

[0031] Global reference motion vector Considered as low-frequency real-world motion components With high-frequency jitter components The superposition, that is = + To separate the two, the system introduces a time-domain low-pass filtering mechanism to generate a smooth motion trajectory. The trajectory is updated recursively using an exponentially weighted moving average, and its mathematical expression is: ; in It is a smoothing coefficient, and its value strictly lies within the open interval. ; It characterizes the true low-frequency motion trend after removing high-frequency disturbances; At the initial time t=0, Set as the zero vector; In calculation Meanwhile, the motion consistency verification unit continuously monitors the temporal changes of Vt; This verification unit calculates consecutive frame intervals. rate of change And estimate the local gradient entropy of the current frame image to characterize the content complexity; Gradient entropy Defined as: ; in This represents the normalized probability distribution of the image gradient magnitude histogram. like Exceeding the adaptive threshold Then temporarily freeze The update, that is ; Adaptive threshold With gradient entropy There is a negative correlation, specifically: ,in Use the baseline threshold (e.g., 3 pixels / frame). With an adjustment factor (e.g., 0.5), this mechanism can effectively prevent trajectory distortion caused by sudden occlusion, strong reflection, or rapid tissue deformation. Based on smooth trajectory Compared with the original motion estimation of the current frame Calculate the jitter cancellation vector Its expression is:

[0032] The vector That is, it needs to be applied to the current frame. To compensate for high-frequency jitter; Subsequently, based on the compensation vector For the current frame Perform image remapping operation to generate a stable output frame. The remapping relationship is defined as: because and Typically, the values ​​are non-integer values. The remapping process uses a bilinear interpolation algorithm to complete sub-pixel level sampling. For any pixel position (x, y) in the output image, its value in the input frame is... The source coordinates are (xs, ys) = (x + cx, y + cy). Let (xs, ys) be located within the cell bounded by four integer pixel points (i, j), (i+1, j), (i, j+1), and (i+1, j+1), where i = floor(xs) and j = floor(ys). Then the interpolation result is: ; in, , ; The image remapping operation is performed by a dedicated hardware acceleration unit. This unit adopts a pipelined architecture, simultaneously completing pixel address generation, interpolation weight calculation, and boundary processing. The address generator receives the compensation vector. Then, the source coordinates (xs, ys) of each output pixel are calculated in real time, and it is determined whether it falls within the effective image area. The effective image area is defined as the central rectangular area of ​​the original image minus the outer safe cropping area. The width of the safe cropping area Wcrop is initially set to a fixed value, for example, 54 pixels at a resolution of 1920×1080 (i.e., 27 pixels on each side and 27 pixels on each top and bottom).

[0033] If the source coordinates (xs, ys) fall within the cropping region (i.e., within the valid region), bilinear interpolation is performed normally. If (xs, ys) exceeds the cropping region but is still within the original image range, interpolation can still be performed, but this pixel will not be included in the final output. If (xs, ys) completely exceeds the boundary of the original image, boundary padding logic is triggered. Boundary filling employs one of two strategies. The first is a fixed cropping strategy: only the central visible area is output, with a constant size (e.g., 1866×954 for 1080p). All compensation operations are completed within the reserved cropping area without generating any content. The second is a dynamic filling strategy: when the source coordinates are detected to be out of bounds, an edge content filling mechanism is activated. This mechanism first analyzes the gradient direction field of the region adjacent to the out-of-bounds direction and extrapolates the pixel value along the dominant gradient direction; or it uses a context-aware image expansion algorithm to predict the content of the missing region using a convolutional neural network to ensure visual coherence. In this embodiment, the system defaults to using the fixed cropping strategy to ensure determinism and low latency.

[0034] The width of the safe cropping area supports dynamic updates. The system maintains a historical maximum compensation amplitude record Mmax, initially set to 0. After each frame is processed, the Euclidean norm |Ct| of the current compensation vector is calculated. If |Ct| > Wcrop / 2, then Mmax is updated to max(Mmax,| |); If | appears in K consecutive frames (e.g., K=10) If Wcrop / 2, then the cropping area width will be updated to Wcrop=1.2×Mmax; Meanwhile, the system has an attenuation mechanism: if no further over-limit situation occurs in the subsequent L frames (e.g., L=100), Wcrop will gradually shrink back to the initial value in an exponential manner. The attenuation formula is Wcrop←γWcrop+(1−γ)W0, where γ=0.99 and W0 is the initial width. This mechanism ensures the effectiveness of image stabilization while avoiding excessive sacrifice of the field of view over a long period of time.

[0035] The entire data flow is uniformly scheduled by the central collaborative controller; Each functional module includes a feature extraction module, a motion vector calculation module, a robust estimation module, a trajectory smoothing module, a compensation calculation module, and an image remapping module, which are connected in series and pass intermediate data through a shared memory buffer. Inter-module communication adopts a circular buffer queue to ensure non-blocking data transmission. The execution of all modules is triggered by the video frame synchronization signal (VSYNC), and the processing timing is strictly aligned with the frame boundaries.

[0036] In one specific embodiment, an endoscopic video sequence with a resolution of 1280×720 and a frame rate of 60fps is used as input; the system configuration is as follows: upper limit of feature points N=500; ORB algorithm, single scale, no direction; matching ratio threshold 0.8; RANSAC iteration 100 times, inlier threshold 2 pixels; smoothing coefficient α=0.95; initial width of safe cropping region is 36 pixels; motion consistency check is enabled, T0=3, β=0.5; fixed cropping strategy is used for boundary processing.

[0037] In the comparative example, the traditional global optical flow method is used to achieve electronic image stabilization. This method calculates dense optical flow for the entire frame image, takes the average value of all optical flow vectors as the global motion estimate, separates the jitter components through low-pass filtering, and finally performs global translation compensation. The remaining conditions (resolution, frame rate, cropping area, interpolation method) are consistent with the example.

[0038] The two schemes were tested on the same 10-second video containing high-frequency hand shaking, weak texture mucous membrane areas, and brief strong reflective interference. The evaluation metrics included: motion estimation error (based on manually annotated real shaking trajectories), output video PSNR (based on ideal stable frames), effective field of view retention rate (the ratio of output image area to original image area), and single-frame processing latency.

[0039] The test results are summarized in the table below:

[0040] Data shows that the present invention significantly outperforms traditional methods in motion estimation accuracy, especially in areas with weak texture and strong interference; the output image quality is higher and the field of view loss is smaller; the processing latency meets the 60fps real-time requirement, while the contrast ratio is close to the performance bottleneck.

[0041] Furthermore, in another set of extreme tests, the input video contained a rapid insertion operation (low-frequency large displacement) superimposed with high-frequency micro-jitter (amplitude < 2 pixels, frequency > 10 Hz); the example successfully separated and compensated for the high-frequency jitter while completely preserving the trajectory continuity of the insertion motion; while the comparative example, due to the global averaging effect, misjudged some jitter as real motion, resulting in insufficient compensation, and the output video still showed obvious jitter.

[0042] In terms of hardware resource consumption, the computational cost of the feature extraction and matching stages of this invention is only about 18% of that of the dense optical flow method due to the sparse feature point strategy adopted by this invention; the robust estimation and trajectory smoothing modules are both lightweight operations that can be executed efficiently on general-purpose DSP cores; although the image remapping module involves full-frame operations, the throughput reaches 120fps (1080p) due to the use of hardware acceleration pipeline, which far exceeds the system requirements.

[0043] In summary, this invention constructs a system-level image stabilization architecture based on local motion vectors and integrating spatiotemporal robust estimation and dynamic compensation mechanisms. This architecture achieves accurate identification and effective cancellation of high-frequency jitter in endoscopic images. The technical solution exhibits high determinism, real-time performance, and robustness in engineering implementation, making it suitable for the demanding imaging environments of industrial and medical endoscopes. In the description of this invention, it should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired result of the technical solution provided in this disclosure can be achieved, and no limitation is imposed herein.

[0044] The above description is merely a preferred embodiment of the present invention and does not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for stabilizing endoscopic images based on motion vector compensation, characterized in that, The endoscopic image stabilization method includes the following steps: Acquire a continuous video frame sequence, and extract feature points from the current frame and the previous frame to obtain an initial feature point set. = ; The initial set of feature points is subjected to stability screening to remove unstable feature points and form a reliable set of feature points. Based on the feature point pairs successfully matched in the reliable feature point set ( , Calculate the local motion vector =( , ); in = - , = - , Indicates the first The two-dimensional vector corresponding to the inter-frame displacement of each feature point. This refers to the i-th feature point in the previous frame that successfully matches the current frame. Let i be the i-th feature point in the current frame that successfully matches the current frame. Let be the displacement of the i-th feature point in the x-direction. Let be the displacement of the i-th feature point in the y-direction; All local motion vectors constitute the original set of motion vectors. ; Robust estimation processing is performed on the original set of motion vectors, and the global reference motion vector is calculated after removing outlier vectors; The global reference motion vector is smoothed by time-domain low-pass filtering to generate a smooth motion trajectory representing the low-frequency true motion trend. Calculate the jitter cancellation vector; Subpixel-level image remapping is performed on the current frame based on the jitter cancellation vector to generate a stable output frame. The subpixel-level image remapping uses a bilinear interpolation algorithm.

2. The endoscopic image stabilization method based on motion vector compensation according to claim 1, characterized in that: The feature point extraction uses the ORB algorithm, and direction calculation is disabled, with the scale level fixed at a single scale.

3. The endoscopic image stabilization method based on motion vector compensation according to claim 1, characterized in that: The robust estimation process employs the RANSAC algorithm to fit the global translation model; or it uses a median filter to apply the values ​​to all local motion vectors. Components and The median value of each component is taken to obtain the global reference motion vector. .

4. The endoscopic image stabilization method based on motion vector compensation according to claim 3, characterized in that: The global reference motion vector Considered as low-frequency real-world motion components With high-frequency jitter components It is formed by stacking, that is = + To separate the two, a time-domain low-pass filtering mechanism is introduced, and a smooth motion trajectory is generated through an exponentially weighted moving average. Its recursive relation is = · -1smooth+(1− )· ,in The preset smoothing coefficient has a value range strictly limited to the open interval (0,1).

5. The endoscopic image stabilization method based on motion vector compensation according to claim 1, characterized in that: In calculating smooth motion trajectories During the process, a motion consistency verification mechanism is introduced: continuously monitoring the global reference motion vector between consecutive frames. rate of change Δ = - If |Δ |Exceeded the adaptive threshold Then freeze Update, keep The adaptive threshold The local gradient entropy of the current frame image Dynamically determined, satisfying ,in As the baseline threshold, This is the adjustment coefficient.

6. The endoscopic image stabilization method based on motion vector compensation according to claim 5, characterized in that: The local gradient entropy Probability distribution through image gradient magnitude histogram The calculation is expressed as: .

7. The endoscopic image stabilization method based on motion vector compensation according to claim 1, characterized in that: The subpixel-level image remapping reserves a fixed-width safety cropping area around the original image, and only outputs the central visible area. The size of the central visible area is constant and smaller than the size of the original image. The safety cropping area does not participate in the final display and is used to accommodate the maximum expected compensation displacement.

8. The endoscopic image stabilization method based on motion vector compensation according to claim 7, characterized in that: The width of the safety cutting area is dynamically updated based on the compensation amplitude: if continuous Compensation vector appears in the frame Euclidean norm If the current cropping area exceeds half the current cropping width, the cropping area width will be updated to 1.2 times the compensation amplitude; and in subsequent L frames that do not exceed the limit, it will gradually shrink back to the initial width in an exponential decay manner.

9. The endoscopic image stabilization method based on motion vector compensation according to claim 1, characterized in that: The upper limit of the number of reliable feature points is rigidly limited to a preset threshold. When the total number of initially extracted feature points exceeds When the characteristic response intensity is in descending order, the response is truncated, retaining the first few. 1 feature point.

10. The endoscopic image stabilization method based on motion vector compensation according to claim 1, characterized in that: The endoscopic image stabilization method based on motion vector compensation is deployed in an embedded image signal processor. Each processing module is triggered by the video frame synchronization signal, and the processing delay of each frame does not exceed a single frame cycle. Image remapping and boundary processing are integrated into the same hardware acceleration unit, and pixel address generation, interpolation weight calculation and boundary filling judgment are executed in parallel using a pipeline architecture.

Images