Key frame extraction method, device and system for water depth estimation scene

Abstract

The invention discloses a water depth estimation scene-oriented key frame extraction method, device and system. The method comprises the steps of S1, acquiring historical flood event scene video data; s2, performing foreground extraction on the video data through Gaussian mixture modeling and Otsu adaptive threshold processing to generate a foreground mask sequence; s3, carrying out image perceptual hash coding based on the grayscale image, dividing video clips according to inter-frame structural similarity, and adaptively estimating a frame sampling interval based on a hash difference value of adjacent frames; s4, performing inter-frame difference on the foreground mask sequence, and selecting a frame with significant motion mutation as a candidate key frame based on a statistical threshold of global motion amplitude; and S5, according to the candidate key frames, carrying out water body and texture feature combined screening. By adopting the technical scheme of the invention, the recognition efficiency and the analysis precision of the typical scene in the flood short video are improved.

Description

Technical Field

[0001] This invention belongs to the field of information processing technology, and in particular relates to a method, apparatus and system for extracting keyframes for water depth estimation scenarios. Background Technology

[0002] In flood disasters, water depth estimation based on video images has significant application potential. However, actual flood videos are often accompanied by interference such as camera shake, repeated images, water surface reflections, and debris obstruction, making it difficult for traditional frame extraction methods to accurately identify key frames with analytical value. Therefore, there is an urgent need for a targeted key frame extraction method. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a keyframe extraction method, apparatus, and system for water depth estimation scenarios.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] A keyframe extraction method for water depth estimation scenarios includes:

[0006] Step S1: Obtain video data of historical flood events;

[0007] Step S2: Extract the foreground from the video data and generate a foreground mask sequence by using Gaussian mixture modeling and Otsu adaptive thresholding.

[0008] Step S3: Perform image-aware hash coding based on grayscale images, divide video segments according to inter-frame structural similarity, and adaptively estimate the frame sampling interval based on the hash difference between adjacent frames.

[0009] Step S4: Perform inter-frame difference on the foreground mask sequence, and select frames with significant motion abrupt changes as candidate keyframes based on the statistical threshold of global motion amplitude.

[0010] Step S5: Perform joint screening of water body and texture features based on candidate keyframes.

[0011] Preferably, in step S3, the inter-frame structural similarity is calculated based on image-aware hashing, the video is segmented, and the frame sampling interval is adaptively estimated based on the statistical characteristics of the hash differences between adjacent frames.

[0012] Preferably, in step S5, the Laplacian texture perturbation features and water index (SBDI) of the lower half region are extracted from the candidate frames, and the adaptive threshold is estimated by the Mean Shift clustering method. Only when both are higher than their respective thresholds is it determined to be the final key frame.

[0013] The present invention also provides a keyframe extraction device for water depth estimation scenarios, comprising:

[0014] The first processing module is used to acquire video data of historical flood events.

[0015] The second processing module is used to extract the foreground mask sequence of video frames through Gaussian mixture modeling and Otsu adaptive thresholding.

[0016] The third processing module performs image-aware hash coding on the video grayscale frames, segments them according to the structural similarity between frames, and adaptively estimates the frame sampling interval based on the statistical characteristics of hash differences.

[0017] The fourth processing module is used to perform inter-frame difference on the foreground mask sequence and select frames with significant motion abrupt changes as candidate keyframes based on the statistical threshold of global motion amplitude.

[0018] The fifth processing module is used to perform joint screening of water and texture features based on candidate keyframes.

[0019] Preferably, the third processing module calculates the inter-frame structural similarity based on image-aware hashing, divides the video into segments, and adaptively estimates the frame sampling interval based on the statistical characteristics of the hash differences between adjacent frames.

[0020] As a preferred approach, the fifth processing module extracts the Laplacian texture perturbation features and water index (SBDI) of the lower half of the candidate frame, and uniformly uses the Mean Shift clustering method to estimate their adaptive thresholds. Only when both are higher than their respective thresholds is the frame determined to be the final keyframe.

[0021] The present invention also provides a keyframe extraction system for water depth estimation scenarios, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes a keyframe extraction method for water depth estimation scenarios when executed by the processor.

[0022] This invention integrates multi-feature keyframe extraction using image hashing clustering, foreground frame difference, and water texture disturbance perception. First, it utilizes image hashing perception for content-driven video segmentation and adaptive sampling. Then, it combines texture disturbance intensity and saturated blue difference index for dual-channel discrimination, improving the robustness and semantic representativeness of keyframe extraction in complex flood scenarios. The keyframes extracted by this invention have a more balanced distribution and more complete semantic coverage, effectively identifying core dynamic events such as water disturbance and wading behavior. This invention is applicable to video processing tasks in complex disaster scenarios such as floods, providing a reliable data foundation and technical support for subsequent water depth estimation and emergency response. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0024] Figure 1 This is a flowchart of the keyframe extraction method for water depth estimation scenarios according to an embodiment of the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0027] Example 1:

[0028] like Figure 1 As shown, this embodiment of the invention provides a keyframe extraction method for water depth estimation scenarios, including:

[0029] Step S1: Obtain video data of historical flood events;

[0030] Step S2: Generate a foreground mask sequence by using Gaussian mixture modeling and Otsu adaptive thresholding.

[0031] Step S3: Perform image-aware hash coding based on grayscale images, divide video segments according to inter-frame structural similarity, and adaptively estimate frame sampling interval based on hash difference between adjacent frames to improve processing efficiency;

[0032] Step S4, Inter-frame Differentiation and Candidate Frame Selection: Perform inter-frame differential on the foreground mask sequence, and select frames with significant motion abrupt changes as candidate keyframes based on the statistical threshold of global motion amplitude.

[0033] Step S5: Joint screening of water and texture features: Extract the Laplacian texture perturbation features and water index (SBDI) of the lower half of the candidate frame, and use the Mean Shift clustering method to estimate their adaptive thresholds. Only when both are higher than their respective thresholds are they determined to be the final keyframes, so as to ensure the dual significance of texture complexity and water semantics.

[0034] In one embodiment of the present invention, in step S2, to accurately capture significant regions related to dynamic changes in water bodies in flood videos, the present invention performs foreground modeling and image preprocessing on the video sequence before keyframe screening. First, the input video is read frame by frame and converted to grayscale images to reduce computational complexity and unify the subsequent processing format. Then, a Gaussian mixture model is used for background modeling to extract foreground regions (including moving objects and water disturbance areas), providing a basis for subsequent inter-frame dynamic analysis. MOG2 can adapt to changes in illumination, periodic disturbances, and slight camera shake, making it suitable for target detection tasks with complex backgrounds in flood scenes.

[0035] Given the frequent brightness differences and visual noise in various video scenes, this invention introduces the Otsu adaptive thresholding method to binarize the MOG2 output to improve the adaptability and stability of the foreground mask. This method automatically sets the segmentation threshold by maximizing the inter-class variance, making it more robust than traditional fixed threshold methods under conditions such as uneven lighting and complex structures.

[0036] Ultimately, the system synchronously generates three types of image sequences for each frame, including the original frame, grayscale image, and foreground mask image, as data input for subsequent image structure modeling and inter-frame dynamic analysis.

[0037] In one embodiment of the present invention, in step S3, to reduce the interference of redundant frames in the video on the key frame extraction effect, the present invention introduces an image content-driven segmentation mechanism before dynamic analysis. Unlike traditional strategies that rely on fixed intervals or uniform sampling, this mechanism divides the video into structurally consistent segments based on inter-frame visual differences, and adaptively estimates the inter-frame sampling interval by the degree of image hash change, thereby improving processing efficiency and semantic representativeness of frame selection, to better adapt to the characteristics of large illumination fluctuations and uneven dynamic intensity in flood videos.

[0038] First, each frame of grayscale image is encoded using the Average Hash algorithm. This method generates a 64-bit hash code by scaling the image to 8×8 and comparing it with the average brightness. The structural difference between adjacent frames is measured using Hamming distance, defined as:

[0039] d H (i,j)=Hamming(h i ,h j (1)

[0040] Where h i and h jThis represents the average hash code of frame i and frame j. To complete frame clustering and segmentation, the system uses a sequential scanning method for frame classification: if the hash distance between the current frame and any frame in the current segment is less than or equal to a set threshold, they are considered to belong to the same segment; otherwise, they are considered the starting frame of a new segment. In this embodiment of the invention, the Hamming distance threshold is set to 20.0. This value is based on the empirical range set by image hashing algorithms in similar image recognition and video content analysis. It is suggested that in structural image clustering, a Hamming distance greater than 15–20 can be considered as a significant semantic difference. This approach achieves a good balance between frame segmentation granularity and subsequent event recognition, and has advantages in both dynamic event detection coverage and redundancy suppression capabilities.

[0041] After segmentation, this embodiment of the invention further adaptively estimates the recommended frame sampling interval based on the mean of inter-frame hash differences. Let Δ be the sequence of adjacent hash differences in a segment. H ={d1,d2,...}, then based on its mean Dynamically adjusting frame sampling frequency within an empirical threshold range (Yao & Zhao, 2024):

[0042]

[0043] To improve the adaptability and efficiency of keyframe extraction, this invention employs a five-level frame interval adjustment mechanism based on the mean of inter-frame hash differences. This mechanism adaptively adjusts the frame sampling density according to the intensity of video structural changes: when the mean hash difference is large (e.g., greater than 15), it indicates drastic inter-frame changes, and the system recommends high-density sampling (frame interval of 1) to capture key events; conversely, when the difference is small (e.g., less than 3), it tends to use low-density frame extraction (frame interval of 8) to compress redundant information. By setting reasonable segmentation intervals, this mechanism can retain sufficient frames to cover the core dynamics in drastically changing segments and effectively compress static or redundant segments, significantly improving the efficiency and semantic representativeness of keyframe extraction. The relevant interval thresholds are set based on statistical analysis results of multiple flood video segments, effectively reflecting the actual relationship between image structural changes and event frequency.

[0044] In summary, the segmentation and interval estimation module effectively compresses redundancy while ensuring image structural similarity, and constructs a frame sampling system that adapts to dynamic and complex scenes. It is one of the key links in the entire keyframe extraction method.

[0045] As one embodiment of the present invention, in step S4, based on image structure segmentation, the present invention further introduces an analysis method based on inter-frame motion changes to identify representative keyframes in the segments. Specifically, inter-frame difference calculation is performed on the foreground mask image sequence within each hash segment. Let the foreground binary image sequence be {F1, F2, ..., F...} n}, then the degree of pixel change between adjacent frames can be defined as:

[0046]

[0047] Where P is the total number of foreground pixels in a unit frame; D t This represents the normalized difference intensity between frame t and frame t+1. This index remains robust to unstable factors such as lighting fluctuations and water surface disturbances, and can reflect significant events such as viewpoint changes or the entry of new objects relatively well.

[0048] To avoid variations in the adaptability of a fixed threshold across different segments, this embodiment of the invention introduces an adaptive dynamic thresholding strategy based on the mean and percentiles. First, let the frame difference sequence in a certain segment be {D1, D2, ..., D...}. n-1} Calculate the mean D of all inter-frame differences in the current segment and the p-th quantile D. p Then, the modulation factor α is constructed, and the dynamic determination threshold is set as follows:

[0049]

[0050] Where ε is a small constant term to avoid the denominator being zero. When the differential intensity of a certain frame satisfies D t When the difference is greater than or equal to τ, the frame is considered to contain significant dynamic changes and is included in the candidate keyframe set. This mechanism can enhance noise suppression in static segments with low overall frame difference levels, and improve responsiveness to sudden events in dynamic segments with widening difference distributions.

[0051] In this embodiment of the invention, the 80th percentile value is selected by default as the dynamic threshold standard for frame difference discrimination, mainly considering the asymmetry and long-tail characteristics of frame difference distribution in flood videos. Compared with a fixed threshold, the high percentile strategy can enhance the system's response capability to strong dynamic events such as sudden changes in water flow, while effectively suppressing false triggers caused by small camera shake or local noise.

[0052] This strategy not only avoids the generalization error caused by setting a fixed threshold under different video conditions, but also enhances the system's ability to respond to local dynamic emergencies (such as pedestrians passing by, water surges, etc.).

[0053] As one embodiment of the present invention, in step S5, although inter-frame difference can effectively identify frames with structural changes, in flood videos, some key semantic events (such as increased water surface ripples and changes in turbidity) are often difficult to capture directly through motion information. Therefore, this embodiment of the present invention introduces a joint detection module for two types of features—texture perturbation and saturated blue difference index—on top of the candidate frames to further improve the semantic sensitivity and discrimination accuracy of key frame selection.

[0054] Considering that water disturbances in flood scenes are typically concentrated in the lower region of the image, this embodiment of the invention performs a Laplacian operation on the lower half of each frame to enhance edge and texture response. Let the grayscale image frame be I. t Its texture perturbation intensity T t The calculation is as follows:

[0055]

[0056] in, Let L(·) represent the lower half of the region in frame t, where L(·) is the Laplacian operation and Var(·) is the variance operation. This metric is widely used in image quality assessment and video blur detection, and can effectively reflect micro-texture differences such as changes in water surface complexity and reflection disturbances.

[0057] To further enhance the judgment of water color characteristics, this embodiment of the invention employs the Normalized Difference Water Index (NDWI) and, specifically for the characteristics of near-surface RGB video, constructs a Saturation-Blue Difference Index (SBDI). This index characterizes water areas by analyzing the relative difference between the saturation channel (S) and the blue channel (B) in the image. First, the video frames are converted from the RGB color space to the HSV color space. The saturation channel (S) is extracted from the HSV space, while the blue channel (B) retains the blue component from the original RGB space. The mean saturation value S is calculated for the lower half of the image. t and the mean of the blue component B t :

[0058]

[0059] Where N is the total number of pixels in the lower half of the region.

[0060] The saturated blue difference index is defined as follows:

[0061]

[0062] Where ∈ is a minimal constant to avoid division by zero. In typical urban flooding scenarios, water bodies typically exhibit high blue light reflectance and low saturation (compared to surrounding roads, vegetation, etc.), therefore SBDI... t The value is typically positive, and a higher value indicates a greater likelihood that the area possesses the color characteristics of water. This index is designed to capture the unique color reflection patterns of standing water from a ground perspective.

[0063] To achieve adaptive discrimination, the embodiments of the present invention uniformly employ Mean Shift clustering to evaluate the texture perturbation intensity T. t Water Body Index (SBDI) t Threshold estimation is performed. Within each hash segment, the feature value sequence of all candidate frames {T1,T2,...} or {SBDI1,SBDI2,...} is input; local density maxima in the feature space are found through Gaussian kernel density estimation, and the feature value corresponding to the maximum density peak is used as the threshold τ; threshold τ is applied only when the candidate frame simultaneously satisfies T... t >τ T With SBDI t >τ S A frame is considered a keyframe only if two conditions are met. This strategy ensures that the selected keyframes are salient in both structural texture and semantic water, improving extraction accuracy and representativeness.

[0064] The present invention has the following advantages:

[0065] 1. A content-driven segmentation and frame sampling strategy based on image hashing is proposed, which can dynamically adjust the sampling interval according to the intensity of changes in video content, effectively improving the method's adaptability to variable flood scenarios;

[0066] 2. A multi-stage screening mechanism integrating inter-frame foreground difference and texture-water dual-channel features was constructed, which enabled accurate extraction and redundancy suppression of key semantic frames;

[0067] 3. A multi-level adaptive threshold estimation method based on Mean Shift clustering was designed to improve the response capability to sudden event frames and the robustness across video parameters.

[0068] Example 2:

[0069] This invention also provides a keyframe extraction device for water depth estimation scenarios, comprising:

[0070] The first processing module is used to acquire video data of historical flood events.

[0071] The second processing module is used to extract the foreground mask sequence of video frames through Gaussian mixture modeling and Otsu adaptive thresholding.

[0072] The third processing module performs image-aware hash coding on the video grayscale frames, segments them according to the structural similarity between frames, and adaptively estimates the frame sampling interval based on the statistical characteristics of hash differences.

[0073] The fourth processing module is used to perform inter-frame difference on the foreground mask sequence and select frames with significant motion abrupt changes as candidate keyframes based on the statistical threshold of global motion amplitude.

[0074] The fifth processing module is used to perform joint screening of water and texture features based on candidate keyframes.

[0075] As one embodiment of the present invention, the third processing module calculates the inter-frame structural similarity based on image-aware hashing, divides the video into segments, and adaptively estimates the frame sampling interval based on the statistical characteristics of the hash differences between adjacent frames.

[0076] In one embodiment of the present invention, the fifth processing module extracts the Laplacian texture perturbation features and the Body Water Index (SBDI) of the lower half of the candidate frame, and uniformly uses the Mean Shift clustering method to estimate their adaptive thresholds. Only when both are higher than their respective thresholds is it determined to be the final keyframe.

[0077] Example 3:

[0078] This invention also provides a keyframe extraction system for water depth estimation scenarios, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes a keyframe extraction method for water depth estimation scenarios when run by the processor.

[0079] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A keyframe extraction method for water depth estimation scenarios, characterized in that, include: Step S1: Obtain video data of historical flood events; Step S2: Extract the foreground from the video data and generate a foreground mask sequence by using Gaussian mixture modeling and Otsu adaptive thresholding. Step S3: Perform image-aware hash coding based on grayscale images, divide video segments according to inter-frame structural similarity, and adaptively estimate the frame sampling interval based on the hash difference between adjacent frames. Step S4: Perform inter-frame difference on the foreground mask sequence, and select frames with significant motion abrupt changes as candidate keyframes based on the statistical threshold of global motion amplitude. Step S5: Perform joint screening of water body and texture features based on candidate keyframes.

2. The keyframe extraction method for water depth estimation scenarios as described in claim 1, characterized in that, In step S3, the inter-frame structural similarity is calculated based on image-aware hashing, the video is segmented, and the frame sampling interval is adaptively estimated based on the statistical characteristics of the hash differences between adjacent frames.

3. The keyframe extraction method for water depth estimation scenarios as described in claim 2, characterized in that, In step S5, the Laplacian texture perturbation features and water index (SBDI) of the lower half region are extracted from the candidate frames, and the adaptive thresholds are estimated using the MeanShift clustering method. Only when both are higher than their respective thresholds is the frame determined as the final keyframe.

4. A keyframe extraction device for water depth estimation scenarios, characterized in that, include: The first processing module is used to acquire video data of historical flood events. The second processing module is used to extract the foreground mask sequence of video frames through Gaussian mixture modeling and Otsu adaptive thresholding. The third processing module performs image-aware hash encoding on the video grayscale frames, segments them according to the structural similarity between frames, and adaptively estimates the frame sampling interval based on the statistical characteristics of hash differences. The fourth processing module is used to perform inter-frame difference on the foreground mask sequence and select frames with significant motion abrupt changes as candidate keyframes based on the statistical threshold of global motion amplitude. The fifth processing module is used to perform joint screening of water and texture features based on candidate keyframes.

5. The keyframe extraction device for water depth estimation scenarios as described in claim 4, characterized in that, The third processing module calculates the inter-frame structural similarity based on image-aware hashing, divides the video into segments, and adaptively estimates the frame sampling interval based on the statistical characteristics of the hash differences between adjacent frames.

6. The keyframe extraction device for water depth estimation scenarios as described in claim 5, characterized in that, The fifth processing module extracts the Laplacian texture perturbation features and water index (SBDI) of the lower half of the candidate frame, and uses the MeanShift clustering method to estimate their adaptive thresholds. Only when both are higher than their respective thresholds is it determined to be the final keyframe.

7. A keyframe extraction system for water depth estimation scenarios, characterized in that, include: A memory and a processor, wherein the memory stores a computer program executed by the processor, the computer program, when executed by the processor, performs the keyframe extraction method for a water depth estimation scenario as described in any one of claims 1-3.

Images