Video image real-time electronic image stabilization method, system, device and computer medium

Abstract

The application provides a video image real-time electronic stabilization method, system, device and computer medium, and relates to the technical field of video image processing. The video image real-time electronic stabilization method comprises the following steps: acquiring a video image frame after input and preprocessing, extracting a main body region of a highlight celestial body target as a region of interest in the video image frame; generating a support mask for registration based on image gradient information in the region of interest, and processing a current frame and a reference frame according to the support mask to obtain a displacement vector and a correlation coefficient; performing hierarchical determination on the effectiveness of the displacement vector according to the correlation coefficient, the quality of the support mask and the boundary safety of the region of interest, and outputting a result according to the result of the hierarchical determination.

Description

Technical Field

[0001] This application relates to the field of video image processing technology, and more specifically, to a method, system, device, and computer medium for real-time electronic image stabilization of video images. Background Technology

[0002] In existing methods, those based on contours or bright spot centers are extremely sensitive to exposure fluctuations, cloud cover, local overexposure, shadows in dark areas, lunar phase changes, and disturbances at the edge of the sun. Once the main contour is incomplete, the estimated center or boundary points are prone to change. Feature point matching and conventional optical flow methods rely heavily on stable and repeatable local features, but solar or lunar images often exhibit large areas of weak texture, strong brightness gradients, periodic textures, or edge-dominated distributions, leading to a significant increase in the risk of mismatches. At the same time, single-scale registration is prone to convergence failure with large displacements and is easily affected by local noise with small displacements. Furthermore, full-image registration includes black borders, backgrounds, and occluded areas in the calculation, further reducing reliability. Offline image stabilization not only fails to improve the image display and recording quality in real time during observation but also cannot provide timely feedback for automatic satellite guidance, exposure adjustment, or frame quality selection. If the reference frame is fixed for a long time, it is difficult to adapt to gradual changes in image brightness and texture. If it is updated frame by frame, it may cause cumulative drift or incorrect updates on low-quality frames. In addition, existing real-time implementations generally lack a unified mechanism for confidence assessment, ROI boundary security protection, and failure cause determination, making it difficult to run stably in embedded devices or real-time observation software and also making troubleshooting difficult. Summary of the Invention

[0003] In view of the above, the purpose of this application is to provide a method, system, device and computer medium for real-time electronic image stabilization of video images, so as to improve the above-mentioned problems existing in the prior art.

[0004] In a first aspect, embodiments of this application provide a real-time electronic image stabilization method for video images. The method includes: acquiring an input and preprocessed video image frame; extracting the main region of a bright celestial target as a region of interest (ROI) within the video image frame; generating a support mask for registration within the ROI based on image gradient information; processing the current frame and a reference frame according to the support mask to obtain a displacement vector and a correlation coefficient; classifying the validity of the displacement vector based on the correlation coefficient, the quality of the support mask, and the boundary security of the ROI; and outputting a result based on the classification result.

[0005] In the above implementation process, the main region of the bright celestial target is extracted as the region of interest, and a support mask is generated using image gradient information for registration. Then, the displacement vector is graded and determined by combining the correlation coefficient, the quality of the support mask, and the boundary safety of the region of interest. This can reduce the image jitter caused by unreliable displacement estimation during electronic image stabilization and help maintain the stability of video output.

[0006] Optionally, the step of acquiring and preprocessing the input video image frame, and extracting the main region of the bright celestial target as the region of interest in the video image frame, includes: converting the video image into a grayscale image, and performing noise reduction and local contrast enhancement; wherein the intensity of the local contrast enhancement is adaptively adjusted or bypassed according to the target type, exposure state, and video image quality; adaptively segmenting the preprocessed video image and extracting the main contour; if no effective contour is extracted or the contour area does not meet the preset conditions, the video image is taken as the region of interest and marked as full-image fallback mode; in the full-image fallback mode, more stringent constraints are imposed on the generation of the support mask and / or the validity determination of the displacement vector compared to the non-fallback mode.

[0007] In the above implementation process, by converting the video image into a grayscale image and performing noise reduction and local contrast enhancement, the enhancement intensity can be adaptively adjusted or bypassed according to the target type, exposure state and video image quality. Combined with adaptive segmentation to extract the main body contour, and adopting a full image backoff mode when an effective contour cannot be extracted or the contour area does not meet the preset conditions, more stringent restrictions are imposed on mask generation and displacement vector validity determination. This helps to improve the reliability of region of interest extraction under complex observation conditions and reduce the risk of image stabilization failure caused by incorrect segmentation.

[0008] Optionally, generating a support mask for registration based on image gradient information within the region of interest includes: excluding low-confidence pixel regions within the subject mask corresponding to the region of interest to construct an effective pixel set; calculating the gradient intensity of the effective pixel set and adaptively selecting a gradient threshold according to the distribution of the gradient intensity to generate an initial support mask; adjusting the gradient threshold if the number of effective pixels in the initial support mask does not meet a preset condition; and degrading the use of the subject mask as the support mask if the number of effective pixels in the initial support mask still does not meet the preset condition after adjusting the gradient threshold.

[0009] In the above implementation process, when generating a support mask in the region of interest, low-confidence pixel regions in the main mask are first excluded. Then, the gradient threshold is adaptively selected according to the gradient intensity distribution of the effective pixels. If the number of effective pixels does not meet the conditions, the threshold is adjusted or the main mask is degraded. This helps to obtain a more reliable support mask when the texture intensity of the target region changes, and reduces the risk of registration deviation caused by the mask being too sparse or containing unstable regions.

[0010] Optionally, the step of processing the current frame and the reference frame according to the support mask to obtain the displacement vector and correlation coefficient includes: constructing a multi-scale image pyramid and a corresponding mask pyramid for the region of interest of the reference frame and caching them; constructing a current image pyramid for the region of interest corresponding to the current frame; starting from the low-resolution layer, performing translation registration iterations according to the support mask, and passing the estimation results of the low-resolution layer to the higher-resolution layer as initial values, until the displacement vector and correlation coefficient are obtained.

[0011] In the above implementation process, a multi-scale image pyramid and corresponding mask pyramid of the region of interest of the reference frame are constructed and cached. For the current frame, an image pyramid is constructed, and translation registration is iterated from the low-resolution layer according to the support mask. The estimation results of the low layer are passed to the higher layer as initial values. In the case of large inter-frame displacement or multi-scale changes in target texture, the displacement vector and correlation coefficient are gradually converged through a coarse-to-fine search method. At the same time, the cache is used to reduce the overhead of repeated construction of the reference frame pyramid, which helps to improve the registration stability and processing efficiency.

[0012] Optionally, the step of performing translation registration iterations based on the support mask and passing the estimation results of the low-resolution layer to the higher-resolution layer as initial values ​​until the displacement vector and correlation coefficient are obtained includes: determining the initial value of the iteration based on the real-time state; wherein, in the case of the initial state, the reference frame being switched, or the displacement result of the previous frame being invalid, zero displacement is used as the initial value; and in the case of continuous and stable tracking and the displacement result of the previous frame being valid, the displacement vector of the previous frame is used as the initial value.

[0013] In the above implementation process, during the translation registration iteration, the initial value of the iteration is determined according to the real-time state. Zero displacement is used as the initial value in the case of initial state, reference frame switching, or invalid displacement of the previous frame. In the case of continuous and stable tracking and valid displacement of the previous frame, the displacement vector of the previous frame is used as the initial value. This enables the registration search to take advantage of the continuity of motion between frames while avoiding the introduction of erroneous historical results into the estimation of the new stage, which helps to improve the convergence speed and the reliability of the displacement vector.

[0014] Optionally, the construction and updating of the cache includes: caching the reference frame region of interest, the multi-scale image pyramid, the normalized floating-point pyramid, and the mask pyramid; when the reference frame is updated, the aligned region of interest after displacement compensation is preferentially obtained from the current grayscale frame to reuse the cache; when the aligned region of interest fails to be obtained or the effective area is insufficient, translation compensation is performed on the entire grayscale image and the cache is reconstructed.

[0015] In the above implementation process, the reference frame region of interest, multi-scale image pyramid, normalized floating-point pyramid and mask pyramid are cached. When the reference frame is updated, the aligned region of interest after displacement compensation is first attempted to be obtained from the current grayscale frame to reuse the cache. Only when the aligned region is not obtained or the effective region is insufficient, translation compensation is performed based on the whole frame grayscale image and the cache is rebuilt. This can reduce the repetitive calculation overhead caused by frequent pyramid reconstruction and help maintain the stability of the processing frame rate during continuous image stabilization.

[0016] Optionally, the step of classifying the validity of the displacement vector according to the correlation coefficient, the quality of the support mask, and the boundary security of the region of interest, and outputting the result according to the classification result, includes: if the validity of the displacement vector is determined to be valid, performing image translation compensation on the current frame using the displacement vector, and updating the reference frame according to a preset rule, wherein the reference frame includes a first reference frame for short-term tracking and a second reference frame for drift suppression.

[0017] In the above implementation process, when the displacement vector is determined to be valid, the displacement vector is used to perform translation compensation on the current frame, and the first reference frame for short-term tracking and the second reference frame for suppressing drift are updated according to the preset rules. This can take into account both timely response to short-term target motion and constraint on long-term cumulative drift during continuous image stabilization, which helps to maintain the position reference stability of the output video under long-term observation.

[0018] Optionally, the step of classifying the validity of the displacement vector based on the correlation coefficient, the quality of the support mask, and the boundary security of the region of interest, and outputting the result based on the classification result, further includes: if the validity of the displacement vector is determined to be invalid, rejecting the current compensation and / or rejecting the reference frame update, and performing the corresponding exception handling operation.

[0019] In the above implementation process, if the displacement vector is determined to be invalid, the compensation and / or the reference frame update are rejected and the corresponding exception handling operation is executed. This can prevent unreliable displacement estimation from being applied to the image stabilization output, avoid sudden jumps in the image due to erroneous translation, and protect the reference frame reference from being contaminated by erroneous information. This helps to maintain the fault tolerance of image stabilization in cases of target loss or image quality deterioration.

[0020] Secondly, embodiments of this application provide a real-time electronic image stabilization system for video images. The real-time electronic image stabilization system includes: an image acquisition and preprocessing module, a region of interest (ROI) extraction module, a support mask generation module, a registration processing module, and a hierarchical judgment and output module. The image acquisition and preprocessing module is used to acquire and preprocess input video image frames. The ROI extraction module is used to extract the main region of a bright celestial object as a ROI from the video image frame. The support mask generation module is used to generate a support mask for registration within the ROI based on image gradient information. The registration processing module is used to process the current frame and a reference frame according to the support mask to obtain a displacement vector and a correlation coefficient. The hierarchical judgment and output module is used to hierarchically judge the validity of the displacement vector based on the correlation coefficient, the quality of the support mask, and the boundary security of the ROI, and output the result based on the hierarchical judgment result.

[0021] In the above implementation process, through the collaboration of the image acquisition and preprocessing module, the region of interest extraction module, the support mask generation module, the registration processing module, and the hierarchical judgment and output module, the video image frames are preprocessed, the main region is extracted, the gradient mask is generated, the registration is estimated, and the displacement validity is hierarchically judged in sequence. This enables the electronic image stabilization process to form a closed loop with quality screening from the input image to the final output, reducing the impact of unreliable displacement estimation on the image stabilization results and helping to maintain the stability of the video output when observing bright celestial targets.

[0022] Thirdly, embodiments of this application also provide an electronic device, which includes a memory and a processor. The memory stores program instructions, and when the processor reads and runs the program instructions, it executes the steps in any of the above implementation methods.

[0023] Fourthly, embodiments of this application also provide a computer medium storing computer program instructions, which are read and executed by a processor to perform the steps in any of the above implementations. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a first schematic diagram of a real-time electronic image stabilization method for video images provided in an embodiment of this application; Figure 2 This is a second schematic diagram of a real-time electronic image stabilization method for video images provided in an embodiment of this application; Figure 3 This is a third schematic diagram of the real-time electronic image stabilization method for video images provided in the embodiments of this application; Figure 4 This is a fourth schematic diagram of the real-time electronic image stabilization method for video images provided in the embodiments of this application; Figure 5 This is a fifth schematic diagram of the real-time electronic image stabilization method for video images provided in the embodiments of this application; Figure 6 This is a sixth schematic diagram of the real-time electronic image stabilization method for video images provided in the embodiments of this application; Figure 7 A schematic diagram of a real-time electronic image stabilization system for video images provided in an embodiment of this application; Figure 8 This is a block diagram of an electronic device provided in an embodiment of this application.

[0026] Icons: 001-Image acquisition and preprocessing module; 002-Region of interest extraction module; 003-Mask generation module; 004-Registration processing module; 005-Grading judgment and output module; 100-Electronic device; 111-Memory; 112-Memory controller; 113-Processor; 114-Peripheral interface; 115-Input / output unit; 116-Display unit. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of the embodiments of this application.

[0028] In a first aspect, embodiments of this application provide a real-time electronic image stabilization method for video images, applied to a server. The server can be an electronic device with logical computing functions, such as a personal computer (PC), tablet computer, smartphone, or personal digital assistant (PDA).

[0029] Please see Figure 1 , Figure 1 This is a first schematic diagram of a real-time electronic image stabilization method for video images provided in an embodiment of this application.

[0030] A real-time electronic image stabilization method for video images includes: acquiring and preprocessing input video image frames; extracting the main region of a bright celestial object as a region of interest (ROI) within the video image frame; generating a support mask for registration within the ROI based on image gradient information; processing the current frame and a reference frame using the support mask to obtain a displacement vector and a correlation coefficient; classifying the validity of the displacement vector based on the correlation coefficient, the quality of the support mask, and the boundary security of the ROI; and outputting the result based on the classification result.

[0031] In the above implementation process, after acquiring the preprocessed video image frames, the main body region of the bright celestial target is extracted as the region of interest (ROI). This avoids interference from the target's periphery or background regions on registration, allowing subsequent processing to focus more on the structurally stable target. Within the ROI, a support mask for registration is generated based on image gradient information. The mask covers areas with strong gradients and significant textures in the image, thus relying more on regions with high reliability during registration. Using this support mask to process the current frame and the reference frame yields inter-frame displacement vectors and correlation coefficients reflecting the degree of registration similarity. Based on this, the effectiveness of the displacement vectors is graded according to the correlation coefficient, the quality of the support mask itself, and the safety of the ROI's proximity to or contact with the image boundary, rather than simply accepting every calculated displacement vector. In this way, displacement vectors with weak correlation or unsafe boundaries are identified as less effective when the target part moves out of the field of view or when the support mask's quality deteriorates due to image blurring or noise. This avoids directly using these potentially erroneous displacements for image stabilization compensation. Finally, by selecting or processing the displacement vector based on the classification results to output the stabilized image, the risk of inter-frame jitter caused by misregistration can be reduced, and the stability of the video sequence can be improved when continuously observing bright celestial targets.

[0032] Optionally, please refer to, Figure 2 This is a second schematic diagram of a real-time electronic image stabilization method for video images provided in an embodiment of this application.

[0033] The process involves acquiring and preprocessing video image frames as input. Within these frames, the main body region of a bright celestial target is extracted as the region of interest (ROI). This includes: converting the video image to grayscale and performing noise reduction and local contrast enhancement; wherein the intensity of the local contrast enhancement is adaptively adjusted or bypassed based on the target type, exposure state, and video image quality; adaptively segmenting the preprocessed video image and extracting the main body contour; if no effective contour is extracted or the contour area does not meet preset conditions, the video image is designated as the ROI and marked as a full-image fallback mode; in the full-image fallback mode, stricter constraints are imposed on the generation of the support mask and / or the validity determination of the displacement vector compared to the non-fallback mode.

[0034] In the above implementation process, after acquiring video image frames, they are converted into grayscale images and denoised to help suppress the interference of sensor noise on subsequent segmentation and registration. The intensity of local contrast enhancement is adaptively adjusted or bypassed according to the target type, exposure state, and video image quality. This allows for the appropriate highlighting of subject features when the contrast between the target and the background is low, while the enhancement process can be weakened or even turned off when the image quality is good or enhancement may introduce artifacts, avoiding the artificial enhancement of false edges or textures. Adaptive segmentation and extraction of the subject contour of the preprocessed image can locate the main area of ​​the bright celestial target as the region of interest. If no effective contour is extracted or the contour area does not meet the preset conditions, unreliable local regions are not forced. Instead, the entire video frame is used as the region of interest and marked as full-image backoff mode. At the same time, the generation of the mask and the validity determination of the displacement vector are subject to stricter restrictions compared to the non-backoff mode. For example, higher gradient intensity is required to be included in the mask, or a more conservative threshold is set for the correlation coefficient.

[0035] In a specific embodiment of this application, the intensity of local contrast enhancement is adaptively adjusted or bypassed based on the target type, exposure state, and video image quality. Specifically, the target type can be distinguished by user-preset mode identifiers, such as normal lunar surface mode, H-alpha sun mode, white light sun mode, etc., with different CLAHE basic parameter sets set for different modes. The exposure state is judged based on the saturation ratio and mean brightness of the grayscale histogram of the current frame. When the saturated pixel ratio exceeds a preset threshold, it indicates significant overexposure. When the mean brightness is too low, it indicates possible underexposure. Both situations trigger a reduction in enhancement intensity or bypass enhancement processing. The video image quality is measured based on the variance estimation of residual noise after Gaussian filtering and the local contrast statistics of the effective pixel set. When the noise level is high or the local contrast is already sufficient, the enhancement intensity is reduced accordingly to suppress the introduction of false edges. When the noise level is low and the local contrast is insufficient, the enhancement intensity is moderately increased to improve texture registration. This adaptive adjustment, through a comprehensive judgment of target type, exposure status, and image quality, enables the enhancement processing to achieve a trade-off between improving registration conditions and avoiding artificial artifacts in different observation scenarios.

[0036] Regarding the specific implementation of the full-image fallback mode, when the extraction of the region of interest fails, the entire frame is not unconditionally used for registration and compensation. Instead, a controlled fault-tolerance mechanism is entered. While maintaining the continuity of the algorithm flow, it still retains support for mask construction, registration quality judgment, safe boundary judgment, and anomaly rejection mechanisms. After performing Otsu binarization on the preprocessed grayscale image and performing morphological opening operations using elliptical structuring elements, if no external contour is extracted, or the area of ​​the extracted maximum contour is less than a preset threshold (e.g., 1000 pixels), or the generated subject mask is deemed unreliable after evaluation, the extraction of the region of interest is determined to have failed. At this time, the region of interest is set as the entire image region of the current frame, and the full-image fallback flag is recorded, entering the full-image fallback mode.

[0037] In this full-image backoff mode, instead of directly using all pixels in the entire frame for enhanced correlation coefficient registration, the gradient magnitude is still calculated within the region of interest in the full image, and an attempt is made to generate a strong gradient mask. Specifically, after excluding saturated areas, near-black shadow areas, and low local contrast areas within the entire image, gradient statistics are performed on the effective pixel set to generate a support mask. If a sufficient number of effective strong gradient pixels can be obtained, this strong gradient mask is used for enhanced correlation coefficient registration; if there are insufficient strong gradient pixels, the gradient threshold is lowered according to a preset percentile backoff mechanism, and the attempt is repeated; if the number of effective pixels after backoff still does not meet the condition, the frame is determined to be a mask unusable or a low-quality frame, thus rejecting compensation and avoiding the excessive participation of black backgrounds, overexposed flat areas, cloud-covered areas, or sensor noise in the enhanced correlation coefficient calculation in full-image mode, which could lead to erroneous displacement.

[0038] To meet real-time requirements, a maximum processing size limit mechanism is introduced in the full-image fallback mode. When the image size of the full-image region of interest or the extended region of interest exceeds the preset maximum processing size, instead of directly performing enhanced correlation coefficient registration on the original-size full image, the grayscale region of interest for registration is first scaled down to within the maximum processing size. Then, an image pyramid is constructed on the scaled image, and enhanced correlation coefficients are iteratively solved. After the solution is completed, the displacements dx and dy estimated at low resolution are mapped back to the original image size according to the scaling ratio, and used to perform image translation compensation on the original resolution image. The maximum processing size can be set to the maximum processing side length, the maximum total number of pixels processed, or the pixel budget corresponding to the maximum processing time per frame, depending on the processor performance, camera resolution, and target frame rate. This effectively avoids the frame rate drop problem caused by excessive computation of enhanced correlation coefficients at high-resolution full images, ensuring that registration and compensation can still be completed in full-image fallback mode while maintaining real-time processing capabilities.

[0039] In standard region of interest (ROI) mode, the pixel distance between the ROI and the four boundaries of the image can be used to determine whether a target is within a safe zone. However, in full-image fallback mode, since the ROI itself is equivalent to the entire frame, the distance between the ROI boundaries alone no longer carries the same safety indication. Therefore, in full-image fallback mode, the safety determination is transformed into displacement safety determination and effective overlap region safety determination. For example, it checks whether the displacement estimated by the enhanced correlation coefficient exceeds the preset maximum allowable displacement, whether the proportion of the effective overlap region after compensation is lower than a preset threshold, whether the black border area generated at the edge of the image after compensation is too large, and whether the difference between the estimated displacement and the successful displacement of the previous frame exceeds a reasonable range.

[0040] If any of the above conditions are not met, compensation for the current frame is rejected and the reference frame is not updated. The full-image backtracking mode still uses a combination of factors, including the enhanced correlation coefficient solution status, the confidence level of the returned correlation coefficient, the number of effective mask pixels, the maximum displacement threshold, displacement continuity, the convergence status of each pyramid layer, and the reason for failure, for judgment. Simultaneously, because background regions, image boundaries, and occlusion interference are more easily introduced in full-image mode, a more conservative judgment threshold is adopted than in the normal region of interest mode. For example, a higher confidence lower limit is required, a smaller maximum allowable displacement is restricted, a higher requirement for the effective overlap ratio after compensation is set, and refreshing the global anchor points used to suppress drift is prohibited in full-image backtracking mode. Successful registration results generated in full-image backtracking mode can be used for temporary output compensation of the current frame, ensuring that the real-time display link is not interrupted. However, stricter conditions are applied to the updates of the reference frame and global anchor points to avoid low-quality frames contaminating the long-term reference state.

[0041] Optionally, please refer to Figure 3 , Figure 3 This is a third schematic diagram of the real-time electronic image stabilization method for video images provided in the embodiments of this application.

[0042] Within the region of interest, a support mask for registration is generated based on image gradient information, including: excluding low-confidence pixel regions within the subject mask corresponding to the region of interest to construct an effective pixel set; calculating the gradient intensity of the effective pixel set and adaptively selecting a gradient threshold based on the distribution of gradient intensity to generate an initial support mask; adjusting the gradient threshold if the number of effective pixels in the initial support mask does not meet a preset condition; and degrading the use of the subject mask as the support mask if the number of effective pixels in the initial support mask still does not meet the preset condition after adjusting the gradient threshold.

[0043] In the above implementation process, within the subject mask corresponding to the region of interest, low-confidence pixel regions are excluded to construct an effective pixel set. This pre-filters unreliable pixels in overexposed, underexposed, or noisy locations, ensuring that subsequent gradient calculations are based on relatively stable pixels. The gradient intensity of the effective pixel set is calculated, and a gradient threshold is adaptively selected based on the gradient intensity distribution to generate the initial support mask. This allows the threshold to dynamically change according to the texture strength of the current image, naturally retaining sufficient strong gradient pixels in areas with rich texture, while avoiding an overly sparse mask due to an excessively high threshold in areas with weak texture. If the number of effective pixels in the initial support mask does not meet the preset conditions, the gradient threshold is adjusted, which is equivalent to appropriately relaxing the restrictions to include more pixels with a certain gradient response, increasing the mask's support capability for registration calculations. If, after adjusting the gradient threshold, the number of effective pixels in the initial support mask still does not meet the preset conditions, the main mask is degraded and used as the support mask. This ensures that the registration process still has a usable mask to support it, even when the texture of the target region is extremely weak and it is difficult to extract sufficiently strong gradient pixels. This avoids displacement estimation failure or a severe decrease in reliability due to the mask area being too small or missing. Through this step-by-step degradation generation method, in scenarios with fluctuating image quality and changing target surface texture, the support mask can both prioritize strong gradient information and retain alternative region support when information is insufficient. This helps improve the stability of inter-frame registration and the adaptability of electronic image stabilization to observation conditions.

[0044] In one embodiment of this application, a support mask for registration is generated based on image gradient information within the region of interest. First, low-confidence pixel regions are excluded from the main mask corresponding to the region of interest; that is, unreliable pixels such as overexposed saturated areas, near-black shadow areas, sensor black edges, low local contrast areas, and areas with obvious cloud or fog obscuring are removed. The remaining pixels are then used to construct an effective pixel set. Subsequently, the gradient magnitude of each point within this effective pixel set is calculated. For example, the horizontal and vertical gradients are obtained using the Sobel operator and then synthesized. A gradient threshold is adaptively selected based on the gradient intensity distribution of the effective pixel set. For example, a preset percentile of the gradient magnitude is used as the initial threshold. Pixels with magnitudes not lower than this threshold are marked as the initial support mask. This ensures that the mask automatically concentrates on strong gradient regions when the texture is rich and is appropriately widened when the texture is weak to retain sufficient support points. If the number of effective pixels in the initial support mask does not meet the preset conditions (e.g., the absolute number is lower than the minimum pixel limit, the proportion of effective pixels to the effective pixel set is insufficient, the spatial grid coverage is insufficient, or the gradient direction distribution is too uniform), the gradient threshold is adjusted, such as by lowering the percentile to include more pixels with a certain gradient response, and the support mask is regenerated. If the number of effective pixels still does not meet the preset conditions after adjusting the gradient threshold, the main mask is degraded as the support mask. However, at this time, the frame is marked as a low-trust fallback state, and stricter conditions are applied to the validity of subsequent displacement vectors. For example, the minimum threshold of the correlation coefficient is increased, the maximum allowable displacement range is limited, and the difference between the displacement and the historical success value is required to be within a narrower range. At the same time, updating the global anchor point is prohibited in this state to avoid unreliable displacement estimation due to insufficient mask support when the target area texture is extremely scarce, and this estimation will be introduced into image stabilization compensation or long-term reference frames.

[0045] In a specific embodiment of this application, when performing Otsu binarization and morphological opening on the preprocessed grayscale image, the kernel size of the elliptical structuring element and the minimum contour area threshold are not fixed values, but are selected based on the image resolution, the typical proportion of the target in the image, and the type of the currently observed target. For high-resolution lunar or solar images, if the target subject occupies a large proportion of the image, a larger morphological kernel can be used to effectively remove diffuse halos, cloud patches, and background stray bright spots, while a higher minimum contour area is set to avoid misidentifying noise patches as subjects. For low-resolution preview images or scenes where the target subject occupies a small proportion, the morphological kernel size is reduced accordingly and the minimum contour area threshold is lowered to prevent small crescents, partial lunar surfaces, or small solar targets from being mistakenly filtered out as noise. When multiple contours exist and their area differences are not significant, shape features such as the roundness or rectangularity of the contours can be used to assist in the selection, prioritizing contours that are closer to circles or ellipses as subject candidates to reduce the probability of irregular noise patches being mistakenly selected.

[0046] Optionally, please refer to Figure 4 , Figure 4 This is a fourth schematic diagram of the real-time electronic image stabilization method for video images provided in the embodiments of this application.

[0047] Based on the support mask, the current frame and the reference frame are processed to obtain the displacement vector and correlation coefficient. This includes: constructing a multi-scale image pyramid and the corresponding mask pyramid for the region of interest in the reference frame and caching them; constructing the current image pyramid for the region of interest corresponding to the current frame; starting from the low-resolution layer, performing translation registration iterations based on the support mask, and passing the estimation results of the low-resolution layer to the higher-resolution layer as initial values, until the displacement vector and correlation coefficient are obtained.

[0048] In the above implementation, a multi-scale image pyramid and a corresponding mask pyramid are constructed and cached for the region of interest (ROI) of the reference frame. This allows the pyramid structure of the reference frame to be directly reused during multi-frame continuous processing, avoiding the computational overhead of repeatedly constructing it frame by frame. A current image pyramid of the same number of layers is constructed for the ROI corresponding to the current frame, forming the corresponding scale space. Starting from the low-resolution layer, translational registration iterations are performed based on the support mask. Each pixel in the low-resolution image corresponds to a larger receptive field in the original image, enabling faster capture of larger displacements between frames. Furthermore, the low-resolution layer is less sensitive to high-frequency noise and local texture details, helping to avoid noise-induced local extrema interference during the search process. The displacement estimation results obtained from the low-resolution layer are passed to the higher-resolution layer as initial values ​​for iteration. This allows the high-resolution layer to start the search from a position closer to the actual displacement, reducing the number of iterations required for the high-resolution layer and lowering the possibility of getting trapped in erroneous local optima in high-resolution detail images. Through this layer-by-layer transfer registration process, the displacement vector and correlation coefficient are finally obtained. When the inter-frame displacement range is large and the texture of the target region exhibits different characteristics under scale changes, the coarse-to-fine estimation method can provide more robust displacement output, helping to maintain the continuity of video image stabilization and response speed.

[0049] In one embodiment of this application, a multi-scale image pyramid is first constructed for the region of interest (ROI) of the reference frame. This involves generating multi-layered image representations with increasing resolution by downsampling layer by layer, starting from the ROI of the reference grayscale frame. Each layer is then converted into a normalized floating-point image to facilitate subsequent calculation of the correlation coefficient. Simultaneously, a mask pyramid with the same number of layers is generated at the corresponding resolution based on the support mask. The mask size of each layer in the mask pyramid is consistent with the image size of that layer, and the mask values ​​are scaled accordingly with image downsampling. The aforementioned ROI of the reference frame, multi-scale image pyramid, normalized floating-point pyramid, and mask pyramid are all cached, allowing these data structures to be directly reused during continuous multi-frame processing as long as the reference frame remains unchanged, avoiding the computational overhead of repeated frame-by-frame construction. For the current frame, the corresponding region in the current grayscale frame is extracted within the same ROI coordinate range, and an image pyramid for the current frame is constructed using the same downsampling method, with the same number of layers as the reference frame pyramid. After constructing the pyramid for the current frame, translation registration iterations begin from the lowest resolution layer. At this layer, the enhanced correlation coefficient is used as the registration criterion. Within the effective pixel region defined by the support mask, only the two-dimensional displacement parameters under the translation model are solved. Each iteration calculates the displacement update based on the gradient information of the current layer and the mask constraints, gradually converging. After registration at the low-resolution layer, the displacement estimate obtained from this layer is amplified according to the size ratio of the current layer to the previous layer and passed to the adjacent higher-resolution layer as the initial value for the registration iteration of that layer, allowing the search at higher layers to start from a position close to the true displacement. Translation registration iterations continue at the higher-resolution layer under the constraints of the support mask to correct and refine the displacement estimate. This process is repeated layer by layer from the lowest to the highest layer until the displacement vector of the current frame relative to the reference frame and the correlation coefficient reflecting the correlation between the region of interest in the current frame and the region of interest in the reference frame under the estimated translation relationship are obtained at the original resolution layer. This coarse-to-fine multi-scale registration method can quickly capture the approximate offset when there is a large displacement between frames, and can perform fine correction in higher-level iterations when there are multi-scale detail changes in the target texture. At the same time, the mask pyramid is scaled synchronously with the image resolution, so that the registration of each layer is limited to the reliable gradient region, which helps to improve the stability and convergence efficiency of displacement estimation under different observation conditions.

[0050] In a specific embodiment of this application, when excluding low-confidence pixel regions within the main mask to construct an effective pixel set, the specific exclusion rules for low-confidence pixel regions include: marking pixels with gray values ​​higher than a preset saturation threshold as saturated areas and excluding them to avoid unstable gradients from overexposed truncated areas participating in mask statistics; marking pixels with gray values ​​lower than a preset shadow threshold as near-black areas and excluding them to avoid readout noise from low signal-to-noise ratio areas being mistakenly identified as registrable textures; directly excluding pixels located outside the effective sensor area (such as black border filling areas); calculating the local contrast for the remaining pixels within the main mask, i.e., calculating the gray standard deviation or gradient mean within a local window of a certain size, and marking flat areas with local contrast lower than a preset threshold or low-contrast areas obscured by clouds or fog as low-stability pixels and excluding them. Through the above step-by-step exclusion rules, the effective pixel set retains only unsaturated, non-near-black target areas with local contrast and within the effective range of the sensor, providing a relatively stable pixel basis for subsequent gradient statistics and percentile threshold selection.

[0051] In a specific embodiment of this application, after generating an initial support mask by selecting a gradient threshold based on the gradient intensity distribution, the gradient direction distribution of pixels within the support mask is further checked to see if it satisfies the two-dimensional constraint conditions. Specifically, the gradient directions of each pixel within the support mask are statistically analyzed, and the directions are divided into multiple sectors according to a preset interval. The number of effective gradient pixels or the percentage of gradient energy in each sector is calculated. If the gradient energy is excessively concentrated in a certain direction (e.g., the ratio of horizontal gradient energy to vertical gradient energy exceeds a preset range, or the eigenvalue ratio of the structure tensor is lower than a preset threshold), it is determined that the gradient direction distribution of the current mask is too singular and can only provide effective constraints for translation registration in one direction. In this case, the system can introduce more pixels with different gradient directions by lowering the gradient percentile threshold, or appropriately expand the mask space to cover structures outside the target's internal texture and edges, regenerate the support mask, and check the direction distribution again until the direction distribution satisfies the two-dimensional constraint conditions or reaches the degradation threshold.

[0052] Optionally, please refer to Figure 5 , Figure 5 This is a fifth schematic diagram of the real-time electronic image stabilization method for video images provided in the embodiments of this application.

[0053] The translation registration is iterated based on the support mask, and the estimation results of the low-resolution layer are passed to the higher-resolution layer as initial values ​​until the displacement vector and correlation coefficient are obtained. This includes: determining the initial value of the iteration based on the real-time state; wherein, in the case of initial state, reference frame switching, or invalid displacement result of the previous frame, zero displacement is used as the initial value; in the case of continuous and stable tracking and valid displacement result of the previous frame, the displacement vector of the previous frame is used as the initial value.

[0054] In the above implementation process, the initial value for the translation registration iteration is determined based on the real-time state at the start, ensuring that the selection of the search starting point is adapted to the current tracking state. When the initial state, reference frame change, or the displacement result of the previous frame is deemed invalid, using zero displacement as the initial value provides an unbiased starting point for the registration search. This avoids directly incorporating erroneous positional deviations into the current estimate due to changes in the reference frame or unreliable historical results, helping to re-establish accurate displacement estimates under a new reference benchmark. When continuous and stable tracking is achieved and the displacement result of the previous frame is valid, using the displacement vector of the previous frame as the initial value leverages the short-term continuity of motion between frames, ensuring that the search starting position is already close to the true displacement of the current frame. This reduces the number of iterations, accelerates convergence, and lowers the possibility of getting trapped in local optima within a large search range. Through this initial value selection strategy based on state switching, the registration iteration avoids wasting computational resources on a full-range search when motion information is available, and also avoids deviations caused by erroneous initial values ​​when historical information is unreliable. This helps maintain the stability and accuracy of displacement estimation when the tracking state changes or the target motion mode switches.

[0055] In one embodiment of this application, an appropriate initial value strategy is selected before each translation registration iteration based on the current real-time state. When in an initial state (e.g., entering the first frame, state reset, or re-initialization after a resolution change), when the reference frame is switched (e.g., switching from a normal reference frame to a global anchor point, or changing the reference due to reference frame reconstruction), or when the previous frame's displacement result is deemed invalid (e.g., the enhancement correlation coefficient calculation for the previous frame failed, the confidence level is lower than a preset threshold, the region of interest is out of bounds or unsafe, the support mask quality is insufficient, or it is in a low-confidence state such as full image rollback or base mask rollback), zero displacement is used as the initial value for iteration. That is, the unit translation matrix is ​​used as the search starting point, and the registration iteration of the enhancement correlation coefficient begins from an unbiased initial position in the current layer. When continuous and stable tracking is achieved and the previous frame's displacement result is valid (i.e., the current image stabilization process is not interrupted, the reference frame is not switched, and the displacement vector of the previous frame has been successfully applied to image translation compensation and passed all validity judgment conditions), the displacement vector of the previous frame is used as the initial value for iteration. The previous frame displacement vector used here refers to the compensated displacement obtained when the current image in the previous frame was aligned to the reference frame used at that time. This displacement reflects the short-term continuity of motion between frames. Using it as the starting point for the current frame registration iteration ensures that the search starting position is close to the true displacement of the current frame, thereby reducing the number of iterations required for the current layer, accelerating the convergence speed of the enhanced correlation coefficient, and reducing the possibility of getting trapped in local optima due to local texture similarity within a large search range. When the reference frame type changes, such as switching from a normal reference frame path to a global anchor point path, the previous frame displacement under the normal reference frame path is no longer used as the initial value. Instead, it is restored to the zero displacement initial value or a prediction initial value independently maintained for the global anchor point path is adopted to avoid the initial value introducing deviations due to the different reference coordinate systems of the two reference frames.

[0056] Optionally, please refer to Figure 6 , Figure 6 This is a sixth schematic diagram of the real-time electronic image stabilization method for video images provided in the embodiments of this application.

[0057] The construction and updating of the cache includes: caching the region of interest of the reference frame, multi-scale image pyramid, normalized floating-point pyramid, and mask pyramid; when the reference frame is updated, the aligned region of interest after displacement compensation is preferentially obtained from the current grayscale frame to reuse the cache; if the aligned region of interest fails to be obtained or the effective area is insufficient, translation compensation is performed on the entire grayscale image and the cache is rebuilt.

[0058] In the above implementation process, the regions of interest (ROIs), multi-scale image pyramids, normalized floating-point pyramids, and mask pyramids of the reference frames are pre-cached. This allows these pre-built structures to be directly invoked during the multi-frame registration stage, eliminating the need for repeated calculations frame by frame. When the reference frame is updated, the aligned ROI, after displacement compensation, is preferentially obtained from the current grayscale frame to spatially align with the cached ROIs of the reference frame, thus directly reusing the original pyramid and mask caches. When the inter-frame motion amplitude is limited, the aligned ROI does not exceed the image boundary, and contains sufficient effective pixels, this reuse method avoids ROI segmentation and multi-layer pyramid reconstruction in the new reference frame, reducing the instantaneous computational load caused by reference frame switching and helping to maintain the frame rate stability of real-time image stabilization. If the ROI alignment fails to be obtained or the effective area is insufficient, translation compensation is performed on the entire grayscale image, the ROI is re-extracted, and all cached content is reconstructed, ensuring that the updated reference frame pyramid and mask accurately match the current actual position and shape of the target. By prioritizing reuse and rebuilding the cache update method when failure occurs, unnecessary repetitive calculations can be reduced during periods of smooth motion under different inter-frame displacement conditions and reference frame switching scenarios. At the same time, the registration reference can be updated in a timely manner when the motion is large or the target boundary changes. Overall, the image stabilization process is improved in terms of adaptability to changes in motion amplitude and the efficiency of computing resource utilization.

[0059] In one embodiment of this application, a multi-scale image pyramid, a normalized floating-point pyramid, and a corresponding mask pyramid are pre-constructed for the region of interest (ROI) of the reference frame. These data, along with the ROI itself, are cached so that they can be directly accessed during the registration process of subsequent consecutive frames, eliminating the need to repeatedly perform pyramid downsampling and mask scaling operations frame by frame. When the reference frame needs updating, for example, when the current frame's image stabilization compensation is successful and the reference frame update conditions are met, the aligned ROI after displacement compensation is preferentially retrieved from the current grayscale frame to reuse the aforementioned cache.

[0060] Specifically, based on the displacement vector obtained by the current frame relative to the reference frame, a context region is extracted from the current original grayscale frame that covers the region of interest of the original reference frame and includes the displacement redundancy margin. The redundancy margin at least covers the components of the displacement in the horizontal and vertical directions and the extra pixels required for the interpolation kernel radius. Then, a local affine inverse mapping is performed on the context region to generate an aligned region of interest grayscale image that is completely consistent with the size of the region of interest of the reference frame. The aligned grayscale image directly replaces the region of interest content in the reference buffer, and the data of each layer of the normalized floating-point pyramid and the mask pyramid are updated accordingly. This avoids performing a complete translation compensation and re-extracting the region of interest for the entire grayscale image, effectively reducing the instantaneous computational load when switching reference frames. In cases where aligning the region of interest (ROI) fails or the effective region is insufficient—for example, if the context region cannot completely cover all source pixel positions required for inverse mapping, the effective overlap ratio within the ROI after displacement compensation is lower than a preset threshold, or the ROI touches the current frame boundary causing excessive black border filling—the process switches to performing the same translation transformation as the output compensation on the entire grayscale image. The ROI is then re-extracted from the compensated grayscale image, and a multi-scale image pyramid, normalized floating-point pyramid, and mask pyramid are reconstructed based on this ROI. All reconstructed data is then updated in the cache. This cache update method, prioritizing reuse and rebuilding upon failure, significantly reduces redundant computation to maintain the real-time processing frame rate during stable periods when inter-frame motion is limited and the ROI does not touch the image boundary. However, in cases of large motion or insufficient ROI security, full-frame reconstruction ensures the accuracy and completeness of the cached reference data, thus maintaining the reliability of the registration reference under various motion conditions.

[0061] In a specific embodiment of this application, when constructing a multi-scale image pyramid of the region of interest (ROI) of a reference frame, the specific dimensions of each layer are determined as follows: the original resolution layer is the width and height of the ROI itself; each lower resolution layer is obtained by scaling down the width and height of the previous layer by a preset scaling factor, typically 0.5, meaning each layer is half the size of the previous layer. The total number of pyramid layers is determined by the initial size of the ROI and the preset minimum layer size. Whenever half the current layer size is still not less than the minimum layer size, another layer is added until either the long or short side is lower than the minimum layer size. For example, when the initial ROI size is large and the minimum layer size is set to 64 pixels, the pyramid may have 3 or more layers; when the initial ROI size is small or the input is a low-resolution preview frame, the actual number of pyramid layers may be less than 3. The dimensions of each layer of the mask pyramid are determined synchronously with each layer of the image pyramid. The mask values ​​maintain spatial consistency with the corresponding layer image through downsampling using nearest neighbor or region scaling, avoiding false mask boundaries caused by interpolation in low-resolution layers.

[0062] Optionally, the validity of the displacement vector is graded based on the correlation coefficient, the quality of the support mask, and the boundary security of the region of interest. The result is output based on the graded determination, including: if the validity of the displacement vector is determined to be valid, performing image translation compensation on the current frame using the displacement vector, and updating the reference frame according to preset rules. The reference frame includes a first reference frame for short-term tracking and a second reference frame for suppressing drift.

[0063] In the above implementation process, after classifying the validity of the displacement vector based on the correlation coefficient, support mask quality, and region of interest boundary security, if the determination result is valid, the displacement vector is applied to the image translation compensation of the current frame, so that the stabilized output can offset the shaking shift between frames. Simultaneously, reference frames are updated according to preset rules. These reference frames are divided into a first reference frame for short-term tracking and a second reference frame for drift suppression. The first reference frame updates at a faster pace when tracking conditions are stable, reflecting the target's current viewing position and shape changes in a timely manner. In the event of short-term swaying or movement of the target, the stabilization compensation can quickly follow the target's movement, avoiding response lag. The second reference frame updates under more conservative conditions or at a lower frequency, providing a longer-term reference for the entire stabilization process. During continuous observation of the target, it can limit the slow shift in image position caused by the gradual accumulation of small errors frame by frame. Through the combined action of the two reference frames, in long-term continuous stabilization scenarios, both adaptability to short-term target dynamics and the risk of accumulated drift under large-span observation are maintained, making the positioning reference of the output video more stable in the time dimension.

[0064] In one embodiment of this application, the displacement vector is graded based on a combination of the confidence score returned by the enhanced correlation coefficient solution, the number of effective pixels in the current frame's support mask, the gradient direction distribution, and whether the region of interest (ROI) has sufficient safety margin in the image. The displacement vector is deemed valid when the confidence score is not lower than a preset compensation threshold, the support mask meets the effective pixel and direction coverage conditions, and the ROI is not too close to the image boundary, resulting in insufficient effective overlap after compensation. In this case, a two-dimensional affine matrix containing only translation components is constructed using this displacement vector. Inverse mapping is then performed on the current color frame to achieve image translation compensation. Instead of independently outputting the stabilized image to a display or storage buffer, the compensation result is directly copied back to the original input image buffer to achieve in-situ update by overwriting the original unstabilized frame. Since both the preview and recording links read image data from the original input buffer, after adopting the in-situ update mechanism, all possible downstream processing modules—including real-time preview rendering, video encoding compression, and file storage writing—can directly obtain image frames that have undergone image stabilization without any interface or pipeline modifications. This enables transparent and non-intrusive image stabilization output within the existing observation software and hardware architecture.

[0065] Simultaneously, two types of reference frames are updated according to preset rules. The first reference frame, used for short-term tracking, is updated after each successful image stabilization using the displacement-compensated region of interest to quickly follow short-term changes in target brightness, texture blackness, and local morphology. The second reference frame, used to suppress drift, participates in registration and updates at preset intervals. For example, the second reference frame is used to replace the first reference frame for registration correction after several consecutive frames, and the current aligned frame is refreshed as the new second reference frame when consecutive successes are achieved and the frame quality reaches a higher level. This limits long-term coordinate drift caused by the accumulation of small, frame-by-frame updates. The update rhythm of the two types of reference frames can be adaptively adjusted based on real-time conditions such as cumulative drift, number of consecutive successful frames, frame quality score, and displacement continuity, ensuring timely response to short-term target motion while maintaining a stable position reference for long-term observation.

[0066] In a specific embodiment of this application, the hierarchical determination of the validity of the displacement vector by the correlation coefficient, support mask quality, and region of interest boundary security is not based on a uniform accept or reject threshold, but rather the determination results are applied in layers to the current frame compensation output, the first reference frame update, and the second reference frame update.

[0067] Specifically, the compensation gating conditions are relatively lenient: when the confidence level is not lower than the compensation threshold, the supporting mask meets the basic effective pixel requirements, and the effective overlapping area after compensation is sufficient, the displacement vector is allowed to be used for image translation compensation in the current frame and output. The first reference frame update gating conditions are more stringent: in addition to meeting the compensation gating conditions, it also requires that the confidence level is not lower than the reference update threshold, the supporting mask is not in the base mask rollback state, the gradient direction distribution meets the two-dimensional constraints, and the region of interest has sufficient safety margin from the image boundary. The first reference frame is only allowed to be updated when all of the above conditions are met. The second reference frame update gating conditions are the most stringent: based on the first reference frame update gating conditions, it also requires that the current frame is in non-rollback mode, the confidence level is not lower than the anchor point update threshold, the number of consecutive successful frames reaches a preset lower limit, the displacement is continuous with the historical displacement, and the overall frame quality score is higher than the anchor point update threshold. Through this hierarchical gating mechanism, while ensuring the continuity of real-time display, the permission for frames of different quality levels to participate in short-term reference and long-term anchor point updates can be determined separately, avoiding the use of frames suitable for temporary display but lacking long-term reference value to pollute the reference benchmark.

[0068] In a specific embodiment of this application, the updating and switching of the first reference frame and the second reference frame are performed as follows: When the first reference frame is successfully stabilized in each frame and the reference update gating condition is met, it is updated with the region of interest (ROI) aligned after displacement compensation in the current frame, ensuring that the first reference frame always closely follows the short-term motion and morphological changes of the target. The second reference frame does not participate in frame-by-frame updates, but is temporarily switched as the registration reference object at preset frame intervals (e.g., every few frames). In this frame, the system uses the second reference frame as the reference frame to perform a complete ROI extraction, mask generation, multi-scale enhanced correlation coefficient registration, and displacement validity determination process. If the displacement determination is valid and the second reference frame update gating condition is met, the aligned ROI is refreshed as the new second reference frame, thus completing one drift correction. If the frame fails the update gating, the original second reference frame is retained, and attempts are continued at intervals in subsequent frames. The estimated relative displacement relationship can also be maintained between the first reference frame and the second reference frame. When the system continuously uses the first reference frame for image stabilization, the displacement of each frame can be accumulated to obtain the cumulative offset relative to the second reference frame. When the cumulative offset exceeds the preset threshold, the second reference frame correction is triggered in advance to enhance the ability to suppress long-term drift.

[0069] Optionally, the validity of the displacement vector is graded based on the correlation coefficient, the quality of the supporting mask, and the boundary security of the region of interest. The result is output based on the graded determination. The method also includes: if the validity of the displacement vector is determined to be invalid, rejecting the current compensation and / or rejecting the reference frame update, and performing the corresponding exception handling operation.

[0070] In the above implementation process, after classifying the validity of the displacement vector based on the correlation coefficient, support mask quality, and the safety of the region of interest boundary, if the determination result is invalid, the displacement vector is rejected for use in the image translation compensation of the current frame. This ensures that even if the registration process outputs a displacement vector with significant errors due to an overly sparse support mask, low correlation, or the target subject touching the image boundary, this vector will not be injected into the image stabilization compensation stage, reducing the risk of abnormal image shift caused by erroneous translation. Simultaneously, updating the first reference frame used for short-term tracking and the second reference frame used for drift suppression is refused, keeping the reference frames in their previously confirmed valid state and preventing them from being replaced by unreliable content or incorrect alignment results of the current frame, thus protecting the baseline quality relied upon for subsequent frame registration. Corresponding anomaly handling operations are performed, which may include continuing to use the compensation result of the previous valid frame, outputting the original uncompensated frame, or switching to a more conservative image stabilization strategy. In situations such as the target briefly moving out of the field of view, the lens being subjected to instantaneous disturbances, or severe local blurring of the image, the image stabilization process can avoid introducing new jitter during periods when the displacement estimation is unreliable, and continue to work from the original stable reference benchmark after the observation conditions are restored. This helps to maintain the overall stability and recovery capability of the output video under intermittent abnormal conditions.

[0071] In one embodiment of this application, the displacement vector is deemed invalid when the following situations occur: failure to solve the enhanced correlation coefficient, a returned confidence value lower than a preset compensation threshold, the number of effective pixels supporting the mask or the gradient direction distribution not meeting the minimum requirements, the region of interest touching or being too close to the image boundary resulting in insufficient proportion of effective overlapping area after compensation, or the current state is in full-image rollback mode or base mask rollback state and cannot pass more stringent additional judgment conditions. In this case, the displacement vector is not applied to the image translation compensation of the current frame; that is, the reverse mapping operation is not performed, and the uncompensated current frame is directly used as the output to avoid abnormal jumping or incorrect translation of the image due to unreliable displacement estimation. At the same time, the first reference frame used for short-term tracking and the second reference frame used for drift suppression are not updated, so that both types of reference frames remain in the state of being confirmed as valid in the previous frame, preventing unreliable content or incorrect alignment results in the current frame from replacing the existing stable reference benchmark. The corresponding anomaly handling operations include: recording the reason for this rejection (such as insufficient confidence, mask quality not meeting the conditions, boundary insecurity, or backoff mode limitations), updating the consecutive failure count, and retaining the successful displacement and reference state of the previous frame unchanged. If the consecutive failure count reaches a preset number, a reset operation is performed, clearing the reference frame buffer, global anchor point buffer, historical displacement records, and various count states, and re-initializing the image stabilization process starting from the subsequently entering high-quality frame. Through this rejection and reset mechanism, during periods when the target briefly moves out of the field of view, the image is severely blurred locally, exposure changes abruptly, or cloud cover causes unreliable registration information, error compensation and reference contamination can be avoided. After the observation conditions recover, the system continues to work from the original stable state or the re-initialized clean state, maintaining the overall stability and recovery capability of the output video under intermittent abnormal conditions.

[0072] In one specific embodiment of this application, when the displacement vector is determined to be invalid and an exception handling operation is performed, the system processes the exception according to a graded exit process, rather than simply resetting all states immediately upon a single frame failure. The specific process is as follows: a single or occasional invalid displacement frame only triggers mild anomaly handling of "refusing compensation and refusing to update the reference frame". The current frame is directly output as the original image, the reference frame and global anchor point remain unchanged, the system records the reason for failure and accumulates the consecutive failure count, but does not clear the historical state; when the consecutive failure count reaches the first preset number (e.g., 3 consecutive frames), the system determines that the current reference frame may no longer be reliable. At this time, it only refuses to compensate and refuses to update the reference frame, and further refuses to use the first reference frame as the registration reference for subsequent frames, waiting for the next frame to re-establish effective registration through the global anchor point path; when the consecutive failure count reaches the second preset number (e.g., 10 consecutive frames), the system determines that the target has been unreliable for a long time, performs a complete reset operation, clears the first reference frame buffer, the second reference frame buffer, the multi-scale pyramid buffer, historical displacement records and all count states, and uses the next frame as the new initial frame to re-establish the image stabilization process. In this way, the strength of anomaly handling is gradually increased as the severity of the fault deepens, taking into account both the fault tolerance capability for short-term interference and the safe exit requirement for long-term anomalies.

[0073] Secondly, embodiments of this application provide a real-time electronic image stabilization system for video images. Please refer to [link to relevant documentation]. Figure 7 , Figure 7 A schematic diagram of a real-time electronic image stabilization system for video images provided in an embodiment of this application.

[0074] A real-time electronic image stabilization system for video images includes: an image acquisition and preprocessing module 001, a region of interest (ROI) extraction module 002, a support mask generation module 003, a registration processing module 004, and a hierarchical judgment and output module 005. The image acquisition and preprocessing module 001 acquires and preprocesses input video image frames. The ROI extraction module 002 extracts the main region of a bright celestial object from the video image frame as the ROI. The support mask generation module 003 generates a support mask for registration based on image gradient information within the ROI. The registration processing module 004 processes the current frame and a reference frame according to the support mask to obtain a displacement vector and a correlation coefficient. The hierarchical judgment and output module 005 performs a hierarchical judgment on the validity of the displacement vector based on the correlation coefficient, the quality of the support mask, and the boundary security of the ROI, and outputs the result based on the hierarchical judgment.

[0075] In the above implementation process, the image acquisition and preprocessing module 001 acquires the input video image frames and performs preprocessing, providing a high-quality image foundation for subsequent steps. The region of interest extraction module 002 extracts the main body region of the bright celestial target in the video image frame as the region of interest, focusing the processing range on the relatively stable main body of the target structure and avoiding diffuse halos or background interference. The support mask generation module 003 generates a support mask for registration based on image gradient information within the region of interest, limiting the pixels that registration depends on to positions with strong gradients and high texture reliability, reducing the misleading effect of flat or noisy regions on registration. The registration processing module 004 performs registration processing on the current frame and the reference frame according to the support mask, obtaining the displacement vector and correlation coefficient, providing motion estimation basis for image stabilization compensation. The hierarchical judgment and output module 005 receives the output of the registration processing module and, considering the correlation coefficient, the quality of the support mask, and the safety of whether the region of interest touches or approaches the image boundary, performs hierarchical judgment on the effectiveness of the displacement vector. When the effectiveness is high, the displacement vector is used to compensate the output; when the effectiveness is insufficient, the corresponding displacement vector is restricted from use, thus avoiding the introduction of results with large registration errors into the image stabilization compensation stage. The information transmission between modules constitutes a continuous processing link from image acquisition to result output. The hierarchical judgment stage provides a feedback and filtering mechanism for the estimation quality of the entire link. Under conditions such as changes in target brightness, fluctuations in image quality, and targets approaching the edge of the field of view, the system can limit the propagation of unreliable compensation, enhancing the stability and fault tolerance of electronic image stabilization in continuous observation.

[0076] It should be noted that, in a specific embodiment of this application, the above processing procedure specifically includes: converting the input video image into a grayscale image and using a 7×7 Gaussian kernel for filtering and noise reduction to reduce the impact of random noise and seeing disturbances on subsequent gradient calculations; subsequently, performing CLAHE local contrast enhancement, with a default value of clipLimit of 5.5 and a default value of tileGridSize of 18×18. The enhancement intensity is adaptively adjusted according to the target type, exposure state, and video image quality. For example, in cases of severe overexposure or low signal-to-noise ratio, clipLimit is lowered or bypassed, and the enhancement only applies to the grayscale image used for registration and mask generation. Otsu adaptive binary segmentation is performed on the preprocessed grayscale image, using 15×15 elliptical structuring elements for morphological opening to remove small noise patches. After extracting the outer contour, the contour with the largest area is selected as the main candidate; if the contour area is less than 1000 pixels or other preset conditions are met, it is determined that no valid contour has been extracted. If a valid contour is extracted, the region of interest is obtained by expanding the rectangle surrounding the subject by 50 pixels. If no valid contour is extracted or the contour area does not meet the conditions, the entire frame is taken as the region of interest and marked as full-image fallback mode. In this mode, stricter restrictions are imposed on mask generation and displacement vector validity determination.

[0077] Within the subject mask corresponding to the region of interest, low-confidence pixels such as saturated areas, near-black shadow areas, and low local contrast areas are excluded to construct an effective pixel set. The gradient magnitude of each point in the effective pixel set is calculated, and the 88th percentile of the gradient magnitude is taken as the initial gradient threshold to generate an initial support mask. If the number of effective pixels in the initial support mask is less than 80, or if other preset conditions are met, the gradient percentile is lowered to 80% as the backoff threshold, and a new support mask is generated. If the number of effective pixels still does not meet the preset conditions after adjusting the gradient threshold, the subject mask is degraded as the support mask, and the frame is marked as a low-confidence backoff state.

[0078] A multi-scale image pyramid is constructed for the region of interest (ROI) of the reference frame. The maximum number of pyramid layers is preset to 3, and the minimum layer size is preset to 64 pixels. Downsampling stops when the long or short side of a layer is less than 64 pixels. A corresponding mask pyramid is constructed simultaneously, and the ROI of the reference frame, the image pyramid, the normalized floating-point pyramid, and the mask pyramid are cached together. For the corresponding ROI of the current frame, an image pyramid of the same number of layers is constructed. Starting from the lowest resolution layer, translation and registration iterations are performed based on the support mask. The maximum number of iterations to enhance the correlation coefficient in each layer is set to 20, and the convergence accuracy is set to 1e-4. The estimation results of the low-resolution layer are amplified according to the inter-layer ratio and passed to higher layers as initial values ​​until the displacement vector and correlation coefficient are obtained at the original resolution layer.

[0079] In the initial state, if the reference frame changes, or if the displacement result of the previous frame is invalid, zero displacement is used as the initial value. If continuous and stable tracking is achieved and the displacement result of the previous frame is valid, the displacement vector of the previous frame is used as the initial value. The previous frame displacement result is considered invalid in cases such as the enhanced correlation coefficient solution failing to achieve convergence accuracy 1e-4 within the maximum 20 iterations, or the returned confidence level being lower than the threshold of 0.60. The initial value is only used to assist in the convergence of the enhanced correlation coefficient iteration; the final displacement is still based on the registration solution result of the current frame and the reference frame.

[0080] The system pre-caches the region of interest (ROI) of the reference frame, a multi-scale image pyramid (maximum 3 layers, minimum layer size 64 pixels), a normalized floating-point pyramid, and a mask pyramid. When the reference frame needs to be updated, the aligned ROI after displacement compensation is preferentially obtained from the current grayscale frame to reuse the cache. If the aligned ROI cannot be obtained or the effective area is insufficient, translation compensation is performed on the entire grayscale image, the ROI is re-extracted, and all the above cached contents are reconstructed.

[0081] If the correlation coefficient is not lower than the compensation threshold of 0.60, the mask quality meets the requirements, and the boundary of the region of interest is safe, the displacement vector is deemed valid. Image translation compensation is then performed on the current frame using this displacement vector, and the reference frame is updated according to preset rules. The reference frame includes a first reference frame for short-term tracking and a second reference frame (i.e., a global anchor point) for drift suppression. The second reference frame is preset to participate in registration correction every 5 frames, and its update timing is preset to refresh after every 3 successful reference updates, in order to achieve a balance between short-term tracking continuity and long-term drift resistance.

[0082] In cases where the correlation coefficient calculation fails, the confidence level is below the threshold of 0.60, the number of effective pixels supporting the mask is less than 80, the gradient direction distribution does not meet the conditions, or the boundary of the region of interest is unsafe, the displacement vector is deemed invalid, the compensation is rejected, and the first and second reference frames are not updated. Anomaly handling includes recording the reason for failure and counting consecutive failures. After a preset number of consecutive failures, the reference frame buffer, global anchor point buffer, and historical displacement states are cleared. The stabilization process is reinitialized after subsequent high-quality frames arrive.

[0083] It should be noted that the parameters listed in the foregoing embodiments, such as Gaussian kernel size, CLAHE's clipLimit and tileGridSize, morphological kernel size, region of interest expansion, gradient percentile threshold, minimum effective pixel count, pyramid layer count and minimum layer size, enhancement correlation coefficient iteration count and convergence accuracy, confidence threshold, and global anchor usage interval and refresh interval, are all preferred default values ​​or empirical initial values ​​for typical real-time solar and lunar observation scenarios, and are not limitations on the scope of protection. In actual implementation, these parameters can be adjusted or adaptively adjusted according to factors such as the bit depth, resolution, target type, exposure status, frame rate, and processor real-time computing power of the input image. Their selection is mainly based on the following considerations.

[0084] Regarding preprocessing parameters, the selection of the Gaussian filter kernel needs to strike a balance between suppressing random noise and preserving the fine edges of the target: if the filter kernel is too small, random noise, seeing disturbances, or camera noise may directly affect the generation of the gradient mask and the convergence of the enhancement correlation coefficient; if the filter kernel is too large, features such as the solar edge, lunar surface texture, solar surface details, or sunspot boundaries may be excessively smoothed, weakening the ability to estimate fine displacement. CLAHE's clipLimit and tileGridSize also need to be balanced: if the enhancement intensity is insufficient, the lunar surface texture, solar surface details, or edge gradients are not significant enough, reducing the effective information participating in the registration of the enhancement correlation coefficient; if the enhancement intensity is too high, the local gray-level relationship may change nonlinearly, and may amplify noise, cloud textures, compression artifacts, or interference at overexposed boundaries. This makes the enhancement correlation coefficient, while having some robustness to global linear brightness changes, quite sensitive to strong nonlinear local enhancement, local occlusion, overexposure saturation, and local contrast reversal caused by clouds. Therefore, in the preferred embodiment, CLAHE is only used to generate grayscale images and gradient masks for registration, without changing the final output image itself; and clipLimit can be reduced and tileGridSize can be increased according to image quality, or CLAHE can be bypassed or switched to a weaker enhancement in overexposed, foggy, or low signal-to-noise ratio scenes.

[0085] In terms of region of interest (ROI) extraction and mask parameters, the morphological kernel size directly affects the extraction quality of the subject contour: a kernel size that is too small is prone to retaining noise, cloud patches, or stray bright spots in the background, leading to incorrect identification of the ROI; a kernel size that is too large may destroy the crescent moon, local structures at the edge of the sun, or small targets. When the minimum contour area threshold is set too low, noise or local bright spots are easily misidentified as the target subject; when the threshold is set too high, the correct subject may be misidentified as the ROI in small targets, local lunar surfaces, or occluded scenes, resulting in failed ROI extraction. The selection of the ROI extension amount needs to balance target jitter redundancy and the introduction of invalid regions: if the extension amount is too small, effective edges or textures may be clipped when the target jitter amplitude is large; if the extension amount is too large, more black background, cloud occlusion, black borders, and invalid regions will be introduced, increasing the computational load and reducing the reliability of enhanced correlation coefficient registration. The gradient percentile setting aims to provide a sufficient number of strong gradient pixels with high reliability for enhanced correlation coefficient registration. If the percentile value is too high, too few effective pixels will participate in the registration, potentially leading to unstable convergence. If the percentile value is too low, overexposed flat areas, low-texture areas, or background noise may enter the mask, weakening the strong gradient mask's ability to suppress interference. Therefore, this application establishes a mechanism for percentile backoff and degradation of the main mask when strong gradient pixels are insufficient, to avoid the failure of a single fixed threshold under different celestial types and exposure conditions.

[0086] Regarding enhanced correlation coefficient registration and state management parameters, too few pyramid layers reduce the convergence probability when there are large displacement jitters between frames; too many layers increase computational load, and low-resolution layers may lose the ability to express small-scale textures. Too few maximum iterations or too wide a convergence precision for enhanced correlation coefficients may lead to insufficient displacement estimation; too many iterations or too strict convergence precision increase real-time computational overhead and may cause overfitting to local anomalies in low-quality frames, fog-occluded frames, or overexposed frames. The confidence threshold setting needs to balance compensation continuity and rejection of erroneous displacements: too high a threshold may lead to frequent rejection of valid frames, resulting in intermittent non-compensation; too low a threshold may accept erroneous displacements, causing image jumps and introducing contamination into the reference frame. The frequency of global anchor point usage also needs to be moderate: too frequent use may reduce the system's adaptability to short-term brightness and texture changes; too infrequent use may lead to gradual accumulation of drift during frame-by-frame updates of ordinary reference frames. Therefore, the usage interval and refresh interval of global anchor points can be adjusted according to the frame rate, target motion stability, and scene change speed.

[0087] Regarding the adaptability of the input image, the method provided in this application is applicable to sun and moon video inputs capable of generating grayscale images for registration. The input image can be monochrome or color; the bit depth can be 8 bits, 10 bits, 12 bits, or 16 bits, where 10-bit and 12-bit images can be stored in a 16-bit data container in a common manner. For color input, it can first be converted to grayscale or luminance channels for region of interest extraction, mask generation, and enhanced correlation coefficient calculation, and then the same translation amount is applied to each channel of the original color image; for monochrome input, a single-channel image is directly used to generate a grayscale image for registration. For Bayer RAW input, de-mosaic processing can be performed to obtain a color image or luminance image, or a suitable single-channel or luminance approximation image can be selected as the registration input without affecting the main structure. To avoid the impact of bit depth differences on the validity of parameters, it is preferable to normalize input images with different bit depths to a uniform floating-point grayscale range, or map them to a uniform 8-bit or 16-bit processing domain, before performing Otsu binarization, CLAHE enhancement, gradient statistics, and enhanced correlation coefficient registration.

[0088] Thirdly, embodiments of this application also provide an electronic device, which includes a memory and a processor. The memory stores program instructions, and when the processor reads and runs the program instructions, it executes the steps in any of the above implementation methods.

[0089] Optionally, please refer to Figure 8 , Figure 8 This is a block diagram illustrating an electronic device according to an embodiment of this application. The electronic device 100 may include a memory 111, a memory controller 112, a processor 113, a peripheral interface 114, an input / output unit 115, and a display unit 116. Those skilled in the art will understand that... Figure 8 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device 100. For example, the electronic device 100 may also include components that are more... Figure 8 The more or fewer components shown, or having the same Figure 8 The different configurations shown.

[0090] The aforementioned memory 111, memory controller 112, processor 113, peripheral interface 114, input / output unit 115, and display unit 116 are electrically connected directly or indirectly to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The aforementioned processor 113 is used to execute executable modules stored in the memory.

[0091] The memory 111 can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory 111 stores programs. After receiving execution instructions, the processor 113 executes the programs. The methods executed by the electronic device 100 as defined in any embodiment of this application can be applied to the processor 113, or implemented by the processor 113.

[0092] The aforementioned processor 113 may be an integrated circuit chip with signal processing capabilities. The processor 113 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a digital signal processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor.

[0093] The peripheral interface 114 described above couples various input / output devices to the processor 113 and the memory 111. In some embodiments, the peripheral interface 114, the processor 113, and the memory controller 112 can be implemented on a single chip. In other instances, they can be implemented on separate chips.

[0094] The input / output unit 115 described above is used to provide user input data. The input / output unit 115 can be, but is not limited to, a mouse and a keyboard.

[0095] The aforementioned display unit 116 provides an interactive interface (e.g., a user interface) between the electronic device 100 and the user, or displays image data for the user's reference. In this embodiment, the display unit can be a liquid crystal display (LCD) or a touch display. If it is a touch display, it can be a capacitive touchscreen or a resistive touchscreen that supports single-point and multi-point touch operations. Supporting single-point and multi-point touch operations means that the touch display can sense touch operations generated simultaneously from one or more locations on the touch display and pass the sensed touch operations to the processor for calculation and processing.

[0096] Fourthly, embodiments of this application also provide a computer medium storing computer program instructions, which are read and executed by a processor to perform the steps in any of the above implementations.

[0097] In the several embodiments provided in this application, it should be understood that the disclosed device can also be implemented in other ways. The device embodiments described above are merely illustrative; for example, the block diagrams in the accompanying drawings illustrate the possible architecture, functions, and operations of the device according to various embodiments of this application. In this regard, each block in the block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram, and combinations of block diagrams, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0098] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0099] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0100] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0101] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

[0102] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A real-time electronic image stabilization method for video images, characterized in that, The real-time electronic image stabilization method for video images includes: After acquiring and preprocessing the input video image frames, the main region of the bright celestial object is extracted as the region of interest in the video image frames. Within the region of interest, a support mask for registration is generated based on image gradient information. The current frame and the reference frame are then processed according to the support mask to obtain the displacement vector and correlation coefficient. The validity of the displacement vector is graded based on the correlation coefficient, the quality of the support mask, and the boundary security of the region of interest, and the result is output based on the graded determination.

2. The real-time electronic image stabilization method for video images according to claim 1, characterized in that, The process of acquiring and preprocessing the input video image frame includes extracting the main region of a bright celestial object as the region of interest within the video image frame, including: The video image is converted to grayscale and then subjected to noise reduction and local contrast enhancement; wherein the intensity of the local contrast enhancement is adaptively adjusted or bypassed according to the target type, exposure state and video image quality. The preprocessed video image is adaptively segmented and the main outline is extracted; if no effective outline is extracted or the outline area does not meet the preset conditions, the video image is taken as the region of interest and marked as full image rollback mode. In the full-map rollback mode, stricter constraints are imposed on the generation of the support mask and / or the determination of the validity of the displacement vector compared to the non-rollback mode.

3. The real-time electronic image stabilization method for video images according to claim 1, characterized in that, The step of generating a support mask for registration based on image gradient information within the region of interest includes: Within the subject mask corresponding to the region of interest, low-confidence pixel regions are excluded to construct an effective pixel set; Calculate the gradient intensity of the effective pixel set, and adaptively select a gradient threshold based on the distribution of the gradient intensity to generate an initial support mask; If the number of effective pixels in the initial support mask does not meet the preset condition, adjust the gradient threshold; If the number of effective pixels of the initial support mask still does not meet the preset condition after adjusting the gradient threshold, the main mask is degraded to be used as the support mask.

4. The real-time electronic image stabilization method for video images according to claim 1, characterized in that, The step of processing the current frame and the reference frame according to the support mask to obtain the displacement vector and correlation coefficient includes: Construct and cache the multi-scale image pyramid and corresponding mask pyramid of the region of interest in the reference frame; Construct the current image pyramid for the region of interest corresponding to the current frame; Starting from the low-resolution layer, translation registration is iterated based on the support mask, and the estimation results of the low-resolution layer are passed to the higher-resolution layer as initial values ​​until the displacement vector and correlation coefficient are obtained.

5. The real-time electronic image stabilization method for video images according to claim 4, characterized in that, The step of performing translation registration iterations based on the support mask, and passing the estimation results of the low-resolution layer to the higher-resolution layer as initial values, until the displacement vector and correlation coefficient are obtained, includes: Determine the initial value of the iteration based on the real-time status; Specifically, in the initial state, when the reference frame is switched, or when the displacement result of the previous frame is invalid, zero displacement is used as the initial value; when continuous and stable tracking is achieved and the displacement result of the previous frame is valid, the displacement vector of the previous frame is used as the initial value.

6. The real-time electronic image stabilization method for video images according to claim 4, characterized in that, The construction and updating of the cache includes: The reference frame region of interest, the multi-scale image pyramid, the normalized floating-point pyramid, and the mask pyramid are cached. When the reference frame is updated, the aligned region of interest after displacement compensation is obtained first from the current grayscale frame to reuse the cache; If the alignment of the region of interest fails to be obtained or the effective region is insufficient, translation compensation is performed on the entire grayscale image and the buffer is rebuilt.

7. The real-time electronic image stabilization method for video images according to claim 1, characterized in that, The validity of the displacement vector is graded based on the correlation coefficient, the quality of the support mask, and the boundary security of the region of interest. The result is then output based on the graded determination, including: If the displacement vector is deemed valid, image translation compensation is performed on the current frame using the displacement vector, and the reference frame is updated according to a preset rule. The reference frame includes a first reference frame for short-term tracking and a second reference frame for suppressing drift.

8. The real-time electronic image stabilization method for video images according to claim 7, characterized in that, The step of classifying the validity of the displacement vector based on the correlation coefficient, the quality of the support mask, and the boundary security of the region of interest, and outputting the result based on the classification result, further includes: If the validity of the displacement vector is determined to be invalid, the current compensation and / or reference frame update are rejected, and the corresponding exception handling operation is performed.

9. A real-time electronic image stabilization system for video images, characterized in that, The real-time electronic image stabilization system for video images includes: an image acquisition and preprocessing module, a region of interest extraction module, a mask generation support module, a registration processing module, and a hierarchical judgment and output module. The image acquisition and preprocessing module is used to acquire input and preprocess video image frames; The region of interest extraction module is used to extract the main region of a bright celestial object as the region of interest in the video image frame. The support mask generation module is used to generate a support mask for registration based on image gradient information within the region of interest. The registration processing module is used to process the current frame and the reference frame according to the support mask to obtain the displacement vector and correlation coefficient; The hierarchical determination and output module is used to hierarchically determine the validity of the displacement vector based on the correlation coefficient, the quality of the support mask, and the boundary security of the region of interest, and output the result based on the hierarchical determination result.

10. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing program instructions, and the processor executing the steps of the method according to any one of claims 1-8 when running the program instructions.

11. A computer medium, characterized in that, The computer medium stores computer program instructions, which, when executed by a processor, perform the steps of the method according to any one of claims 1-8.

Images