Water area video target segmentation method and system based on progressive representation extension

Abstract

This invention relates to a method and system for water video target segmentation based on progressive representation expansion, belonging to the fields of image processing and video target detection technology. It utilizes a small number of initialized feature prototypes to generate target segmentation results through an attention network; the reconstruction error is calculated, and if the error exceeds a threshold, the feature prototype with the largest reconstruction error is copied and subjected to random perturbation to expand the number of feature prototypes; this process is repeated until the error converges; 3D structural similarity loss is used as a measure of reconstruction error to maintain clear spatial boundaries and temporal continuity. This invention solves the over-segmentation problem in video target segmentation through a progressive representation expansion mechanism, achieving adaptive expansion of representation capacity. It can be applied to scenarios such as floating object detection in water areas, foreign object detection in smart water meters, and inspection of water conservancy facilities, achieving high-precision, adaptive target segmentation.

Description

Technical Field

[0001] This invention relates to a method and system for water video target segmentation based on progressive representation expansion, which is applicable to scenarios such as water floating object detection and intelligent monitoring, and belongs to the fields of image processing and video target detection technology. Background Technology

[0002] Video object segmentation is an important research direction in the field of computer vision, aiming to separate moving objects from the background in a video sequence. In applications such as smart water conservancy and aquatic environment monitoring, accurate segmentation of floating objects (such as plastic bottles, foam blocks, and dead branches) is of great significance for garbage cleanup and water quality monitoring.

[0003] Existing video object segmentation methods mainly suffer from the following technical problems: 1. Oversegmentation problem: Existing segmentation models often incorrectly segment a single object into multiple parts when decomposing a video into object representations. For example, a plastic bottle floating on the water surface may be split into two independent targets, the "bottle body" and the "cap," leading to inaccurate detection results; 2. Fixed Representation Capacity: Traditional methods fix the number of object representations at the start of training, making it impossible to dynamically adjust according to scene complexity. For simple scenes (such as a single floating object), a fixed number of representations can lead to redundancy; for complex scenes (such as multiple floating objects clustered together), a fixed number of representations may be insufficient to represent all targets. 3. Blurred boundary problem: Floating objects on the water surface are highly similar to the water background in color and texture, making it difficult for traditional methods to accurately segment the target boundary, resulting in coarse segmentation results and inaccurate positioning; 4. Poor temporal consistency: Existing methods perform poorly in detecting the first few frames of video sequences and lack effective utilization of global temporal information, resulting in inconsistent representation of the target in the temporal dimension.

[0004] Therefore, a video target segmentation method that can adaptively expand representation capacity, avoid over-segmentation, and maintain clear boundaries and temporal consistency is needed to solve the technical challenges in practical applications. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a water video target segmentation method based on progressive representation expansion. The progressive representation expansion mechanism solves the over-segmentation problem in video target segmentation and realizes adaptive expansion of representation capacity.

[0006] The technical solution adopted in this invention is as follows: The water area video target segmentation method based on progressive representation expansion includes the following steps: S1. Collect video sequence data in the water scene and obtain multiple consecutive frames of images; S2. A small set of feature prototypes is obtained through feature prototype initialization; S3. Input the feature prototype set and consecutive multi-frame images into the attention network, calculate the correlation between each feature prototype and image features through the attention mechanism, obtain the association probability between each feature prototype and each pixel position, and generate the target segmentation result under the current number of feature prototypes. S4. Progressively expand the feature prototypes, calculate the reconstruction error of the current segmentation result, and if the reconstruction error is greater than the preset threshold, copy the feature prototype with the largest reconstruction error and apply random perturbation to expand the number of feature prototypes. S5. Repeat steps S3 and S4 with the expanded feature prototype set until the reconstruction error converges or the preset maximum number of feature prototypes is reached, and output the final target segmentation result. S6. The total structural perception loss is calculated by weighting the 3D structural similarity loss and the detection loss. The loss is minimized through backpropagation, and all parameters of the feature prototype vector and the attention network are optimized to obtain the final segmentation model.

[0007] The above method also includes a step of optimizing the result using a bidirectional cyclic inference mechanism after step S6: inputting the video sequence into the attention network in a forward order to obtain a forward inference result; inputting the video sequence into the attention network in a reverse order to obtain a backward inference result; and fusing the forward inference result and the backward inference result to output the optimized segmentation result.

[0008] In the above method, each feature prototype in step S2 is defined as a learnable vector with a dimension of 256, represented as follows: , Where K is the total number of current feature prototypes, initially K=2, S1 corresponds to the foreground target prototype, and S2 corresponds to the background prototype.

[0009] In step S3, the attention network comprises three sub-modules: an encoder, an attention computation unit, and an iterative update unit. (1) The encoder uses a CNN backbone network to extract high-dimensional feature representations from the original input image and outputs a feature map F. Before inputting the attention calculation unit, the feature map F is enhanced with positional encoding to preserve spatial location information: , Encode sinusoidal positions or learnable position embeddings to ensure that the attention mechanism can perceive the spatial relationships between different positions; (2) The attention calculation unit uses the feature prototype as a query, image features Using key and value as keys, the correspondence between each feature prototype and image features is calculated through a cross-attention mechanism to obtain the aggregated feature U: , Where Q is the projection of the feature prototype, and K and V are the projections of the image features; (3) The aggregated feature U is iteratively updated using a multilayer perceptron to gradually refine the representational power of the feature prototype, as expressed by the formula: , Where α is the learning rate scaling factor, used to control the update step size and avoid training instability; After each iteration, the attention weight matrix obtained in the attention computation unit represents the association probability between each feature prototype and each pixel location. , Softmax normalization is performed along the feature prototype dimension, ensuring that the sum of the probabilities of all prototypes at each pixel location is 1. The attention weights are then rearranged along the feature prototype dimension to obtain a pixel-level segmentation probability map. This probability map represents the target segmentation result for the current number of prototypes and can be directly used to calculate the cross-entropy loss and the progressive representation extension module to calculate the reconstruction error.

[0010] The progressive representation expansion process in step S4 is as follows: (1) First, calculate the reconstruction error between the current segmentation result and the real mask. As a basis for judging whether expansion is needed, the weighted sum of cross-entropy loss and structural similarity loss is adopted; (2) Expansion trigger condition judgment, when the reconstruction error Greater than the preset threshold And the current number of feature prototypes is less than the preset maximum value. When the time is right, trigger the expansion; otherwise, continue the current training without expansion. (3) Calculate the pixel-level reconstruction error of the region corresponding to each feature prototype, select the prototype with the largest error, calculate the reconstruction error of each pixel i, which is the difference between the true class and the predicted probability of the pixel, aggregate the error according to the feature prototype, and select the feature prototype with the largest cumulative reconstruction error for expansion. (4) Copy the selected feature prototype and apply perturbation to generate a new feature prototype. (5) Add the new feature prototype to the existing set to form the expanded feature prototype set, and update the number of feature prototypes at the same time. The expanded feature prototype set continues to participate in the subsequent training iterations.

[0011] In step S6, 3D structural similarity loss is used, and its calculation formula is as follows: , in, , The mean of the local region. , Standard deviation, For covariance, , As a stability constant, the loss function is calculated simultaneously in both spatial and temporal dimensions and then averaged to ensure that the segmentation results maintain clear boundaries in space and remain continuous and consistent in time. The total loss function is: , in, To detect the loss, This is the balance coefficient.

[0012] Another objective of this invention is to provide a water video target segmentation system based on progressive representation expansion, comprising a video acquisition module, a feature prototype initialization module, an attention-based segmentation module, a reconstruction error calculation and comparison judgment module, a progressive representation expansion module, a structure-aware total loss calculation module, and a network update and optimization module. The video acquisition module collects video sequence data in aquatic scenes and obtains multiple consecutive frames of images; The feature prototype initialization module obtains a small set of feature prototypes through feature prototype initialization. The attention-based segmentation module takes a set of feature prototypes and multiple consecutive frames of images as input. It calculates the correlation between each feature prototype and image features through an attention mechanism, obtains the association probability between each feature prototype and each pixel position, and generates the target segmentation result for the current number of feature prototypes. The reconstruction error calculation and comparison module calculates the reconstruction error of the current segmentation result and judges the reconstruction error. Is it greater than the preset threshold? And the current number of feature prototypes is less than the preset maximum value. Determine whether the number of feature prototypes is less than a preset threshold or reaches a preset maximum number of feature prototypes. The progressive characterization extension module replicates the feature prototype with the largest reconstruction error and applies random perturbations to expand the number of feature prototypes; The structure-aware total loss calculation module uses a weighted average of 3D structure similarity loss and detection loss to calculate the total structure-aware loss; The network update and optimization module optimizes all parameters of the feature prototype vector and the attention network simultaneously through backpropagation with loss minimization.

[0013] The water video target segmentation system based on progressive representation expansion also includes a bidirectional cyclic inference optimization module. The module inputs the video sequence into the attention network in a forward order to obtain the forward inference result; it inputs the video sequence into the attention network in a reverse order to obtain the backward inference result; and it fuses the forward inference result and the backward inference result to output the optimized segmentation result.

[0014] The beneficial effects of this invention are: (1) Adaptive representation capacity: Through a progressive expansion mechanism, the number of feature prototypes can be dynamically increased according to the complexity of the scene. Simple scenes (such as a single floating object) will not generate redundancy, and complex scenes (such as multiple floating objects clustered together) can also obtain sufficient representation capacity, avoiding the shortcomings or waste caused by the fixed capacity of traditional methods.

[0015] (2) Effectively avoid over-segmentation: When a single object region is covered by multiple prototype errors, the expansion mechanism will merge or subdivide the prototypes, so that each object corresponds to a complete prototype, which solves the problem of existing methods splitting a single object (such as a plastic bottle) into multiple independent targets.

[0016] (3) Enhance temporal consistency: The 3D SSIM loss is calculated simultaneously in the time dimension, constraining the segmentation results between adjacent frames to maintain structural continuity; the bidirectional cyclic inference mechanism utilizes forward and reverse temporal information fusion to further eliminate the problems of poor performance in the first few frames of the video and inconsistent target representation in time.

[0017] (4) Wide range of applications: It is particularly suitable for practical engineering scenarios such as floating object detection in water areas (drones / monitoring), foreign object detection in smart water meters, and inspection of water conservancy facilities. It can output the location, type, and outline of floating objects. Attached Figure Description

[0018] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a flowchart of the progressive characterization extension process in the method of the present invention. Detailed Implementation

[0019] The present invention will be further described below with reference to embodiments.

[0020] Example 1: A water area video target segmentation method based on progressive representation expansion, comprising the following steps: S1. Acquire video sequence data in aquatic scenes to obtain multiple consecutive frames of images: Video acquisition equipment was used to obtain video sequence data under different scenarios. In the water monitoring scenario, a drone equipped with a 4K visible light camera flew along the river route to collect video data of floating objects on the water surface. The types of floating objects (plastic bottles, foam blocks, dead branches) and their contour masks were labeled to construct a water area subset dataset. In the general validation scenario, publicly available video datasets were used for model validation and comparative testing. The water monitoring scenario used drones or fixed monitoring cameras to collect video images of rivers, lakes, and nearshore waters, covering samples with different lighting conditions (daytime and nighttime), different water qualities (clear and turbid), and different types of floating objects (plastic bottles, foam blocks, dead branches, algae), providing data support for subsequent segmentation.

[0021] S2. A small set of feature prototypes is obtained through feature prototype initialization.

[0022] Feature prototypes are a set of learnable parameter vectors that dynamically expand during training. Each feature prototype represents a high-level semantic concept of an object category or background (e.g., "plastic bottle," "foam block," "water surface background"), independent of the specific input image, but learned from data through training. The initial number of feature prototypes, K0, is set to 2, used to represent the foreground object (floating object) and the background region (water surface), respectively. Each feature prototype is defined as a learnable vector with 256 dimensions, represented as follows: , Where K is the total number of current feature prototypes, initially K=2. Corresponding to the prototype of the foreground target. Corresponding background prototypes. Alignment with image features is achieved through learnable linear projection, with the initial two feature prototypes serving as the starting point for subsequent progressive expansion.

[0023] S3. Input the feature prototype set and consecutive multi-frame images into the attention network, calculate the correlation between each feature prototype and image features through the attention mechanism, obtain the association probability between each feature prototype and each pixel position, and generate the target segmentation result under the current number of feature prototypes.

[0024] The attention network consists of three sub-modules: an encoder, an attention computation unit, and an iterative update unit. The encoder uses a CNN backbone network to extract high-dimensional feature representations from the original input image, which has a size of 640×640×3. The output feature map is F. Before inputting into the attention computation unit, the feature map F undergoes positional encoding enhancement to preserve spatial location information: , in, Encode sinusoidal positions or learnable position embeddings to ensure that the attention mechanism can perceive the spatial relationships between different positions.

[0025] The attention computation unit uses the feature prototype as the query and the image features as the key and value. Through a cross-attention mechanism, it calculates the correspondence between each feature prototype and the image features, resulting in the aggregated feature U. , Where Q is the projection of the feature prototype, and K and V are the projections of the image features.

[0026] The aggregated features U are iteratively updated using a multilayer perceptron (MLP) to gradually refine the representational power of the feature prototypes. The formula is as follows: , Here, α is the learning rate scaling factor, used to control the update step size and avoid training instability.

[0027] After each iteration, the attention weight matrix obtained in the attention computation unit represents the association probability between each feature prototype and each pixel location. , Softmax normalization is performed along the feature prototype dimension, ensuring that the sum of the probabilities of all prototypes at each pixel location is 1. Let the number of current feature prototypes be K, and the spatial size of the image features be H×W, then... ,in This represents the probability that position (h, w) belongs to the k-th feature prototype. Rearranging the attention weights along the feature prototype dimension yields a pixel-level segmentation probability map. This probability map represents the target segmentation result with the current number of prototypes, and can be directly used to calculate the cross-entropy loss and the progressive representation extension module to calculate the reconstruction error.

[0028] S4. Perform progressive representation expansion on the feature prototypes, calculate the reconstruction error of the current segmentation result, and if the reconstruction error is greater than a preset threshold, copy the feature prototype with the largest reconstruction error and apply random perturbation to expand the number of feature prototypes.

[0029] (1) Calculate the reconstruction error L between the current segmentation result and the true mask. rec As a criterion for determining whether expansion is needed, a weighted sum of cross-entropy loss and structural similarity loss is used. Specifically: for each pixel location, the cross-entropy loss between the model-predicted object category and the true category is calculated: , Where N is the total number of pixels, and K is the number of current feature prototypes. Let i be the true label of pixel i belonging to the k-th feature prototype. The probability that pixel i, predicted by the model, belongs to the k-th feature prototype.

[0030] The 3D structural similarity loss is used to measure the structural consistency between the segmentation mask and the real mask: , in, The segmentation mask is the one to be predicted (obtained by aggregating attention weights). For a true mask, SSIM3D calculates simultaneously in both spatial and temporal dimensions, including comparisons of three components: brightness, contrast, and structure.

[0031] The reconstruction error is a weighted sum of the two types of losses mentioned above: , The balance coefficient is set to , This combination ensures pixel-level classification accuracy while maintaining the clarity of segmentation boundaries and temporal consistency.

[0032] Reconstruction error serves as the trigger condition for expansion; it does not participate in gradient updates but is merely a decision metric. Reconstruction error requires fixed weights to ensure consistency in expansion decisions across different stages and scenarios.

[0033] (2) Extended trigger condition judgment. When reconstruction error Greater than the preset threshold And the current number of feature prototypes is less than the preset maximum value. When this occurs, the expansion mechanism is triggered. Among these, the threshold... Experiments have shown that when the reconstruction error is below a threshold, it indicates that the current feature prototype set can represent the video content well; when it exceeds the threshold, it indicates that there is a large reconstruction residual and the representation capacity needs to be increased. K is the preset maximum number of feature prototypes. This upper limit is set according to the complexity of the target scene. For floating object detection in water, the number of floating objects in a single frame image is usually no more than 10, so K is set to... _max =11 (including background prototype) can meet the needs of most scenarios.

[0034] If both conditions are met, perform the following expansion operation; otherwise, continue the current training without expansion.

[0035] (3) Identify the feature prototypes to be expanded. To determine which feature prototypes most need to be expanded, calculate the pixel-level reconstruction error of the region corresponding to each feature prototype, select the prototype with the largest sum of errors, and identify the feature prototype that contributes the most to the reconstruction error, i.e., the feature prototype responsible for the least accurate region. Specifically: for each pixel i, calculate its reconstruction error, which is the difference between the true class and the predicted probability of that pixel: , in, For the true category index of pixel i, This represents the model's predicted probability for the correct category. A larger error value indicates a less accurate classification of the pixel.

[0036] Error aggregation by feature prototype. Pixel-level errors are aggregated according to the current feature prototype allocation to obtain the cumulative error for the region responsible for each feature prototype: , in, The model predicts the set of pixels belonging to the k-th feature prototype. This indicates the degree of inaccuracy in the overall reconstruction of the region covered by the k-th feature prototype.

[0037] Select the feature prototype with the largest error. Determine the feature prototype index to be expanded: , This involves selecting the feature prototype with the largest accumulated reconstruction error for expansion. This feature prototype is typically the region with the weakest current representation capability, the most complex region it is responsible for, or a region containing multiple mixed objects.

[0038] (4) Generate new feature prototypes. The selected feature prototypes are copied and perturbed to generate new feature prototypes: First, the selected feature prototypes are copied: , in, This represents the feature prototype vector to be expanded. The copying operation preserves the learned semantic information of the original feature prototype, allowing the new prototype to continue to be optimized based on existing knowledge.

[0039] Gaussian noise perturbation is applied to the replicated feature prototype to generate differential features: , in, The noise standard deviation is set to 0.2, and I is the identity matrix. The noise perturbation causes the new feature prototype to shift near the original semantics, enabling the exploration of feature subspaces not covered by the parent prototype, thus representing different sub-objects in the parent prototype region more precisely.

[0040] (5) Expanded feature prototype set and iterative optimization. Add the new feature prototypes to the existing set to form the expanded feature prototype set: , Simultaneously, the number of feature prototypes is updated: the expanded set of feature prototypes continues to participate in subsequent training iterations. In subsequent attention calculations, the newly added feature prototypes will compete with other prototypes for pixel allocation, gradually learning to represent the underrepresented sub-parts in the parent prototype region.

[0041] After the expansion operation is completed, the system continues with subsequent training iterations. Since the new feature prototype is generated by adding small perturbations to the parent prototype, its initial representation is similar to the parent prototype. Therefore, in the next round of attention calculation, the new prototype automatically attracts pixels with features similar to the parent prototype, gradually achieving region subdivision. This process requires no additional initialization or retraining, achieving smooth representation expansion.

[0042] S5. Repeat steps S3 and S4 for the expanded feature prototype set until the reconstruction error converges or the preset maximum number of feature prototypes is reached, and output the final target segmentation result.

[0043] The 3D structural similarity loss is used instead of the traditional mean squared error loss, and its calculation formula is as follows: , in, , The mean of the local region. , Standard deviation, For covariance, , This is a stability constant. The loss function is calculated simultaneously in both spatial and temporal dimensions to ensure that the segmentation results maintain clear boundaries in space and remain continuous and consistent in time.

[0044] The total loss function is: , in, To detect loss (such as cross-entropy loss). This is the balance coefficient.

[0045] Total loss: Minimized through backpropagation as the model training objective. Simultaneously, it optimizes all parameters of the feature prototype vector and the attention network, directly driving model learning. The total loss requires adjustable weights, which can be increased depending on the specific task (e.g., increasing the weights if boundary clarity is more important). Balancing the impact of classification accuracy and structural similarity.

[0046] S6. The total structural perception loss is calculated by weighting the 3D structural similarity loss and the detection loss. The loss is minimized through backpropagation, and all parameters of the feature prototype vector and the attention network are optimized to obtain the final segmentation model.

[0047] S7. Construct a circular inference module. Design a bidirectional inference mechanism to address the temporal characteristics of video sequences: Forward inference involves inputting the video sequence into the network in chronological order. The model processes the video from the first frame to the last, accumulating global contextual information. Backward inference involves inputting the video sequence into the same network in reverse chronological order. The model processes the video from the last frame back to the first frame, using future information to optimize the representation of past frames. Finally, the results of forward and backward inference are weighted and averaged to output the final segmentation result. The recurrent inference module is used only during the inference phase, adding no training complexity and only a small amount of computational overhead.

[0048] Application scenario expansion and implementation: Floating Object Detection in Water Areas: This invention can be deployed on drones or fixed surveillance cameras to detect floating objects on the water surface in real time, outputting the location, type, and outline information of the floating objects. Through a progressive characterization expansion mechanism, it can accurately distinguish between single floating objects and clusters of multiple floating objects, avoiding over-segmentation. The detection results, combined with geographic coordinates, generate a heat map of pollution points, enabling sanitation vessels to determine the optimal collection routes and improving the efficiency of waterway waste cleanup.

[0049] Foreign Object Detection in Smart Water Meters: This invention can be adapted to a water meter camera to detect foreign objects (such as leaves or garbage) inside the pipes, thus providing early warning of abnormal water quality.

[0050] Water conservancy facility inspection: This invention can be deployed on underwater robots to detect obstacles in culverts and pipelines, providing target location for dredging operations.

[0051] Example 2: A water area video target segmentation system based on progressive representation expansion, including a video acquisition module, a feature prototype initialization module, an attention-based segmentation module, a reconstruction error calculation and comparison judgment module, a progressive representation expansion module, a structure-aware total loss calculation module, and a network update and optimization module. The video acquisition module collects video sequence data in aquatic scenes and obtains multiple consecutive frames of images; The feature prototype initialization module obtains a small set of feature prototypes through feature prototype initialization. The attention-based segmentation module takes a set of feature prototypes and multiple consecutive frames of images as input. It calculates the correlation between each feature prototype and image features through an attention mechanism, obtains the association probability between each feature prototype and each pixel position, and generates the target segmentation result for the current number of feature prototypes. The reconstruction error calculation and comparison module calculates the reconstruction error of the current segmentation result and judges the reconstruction error. Is it greater than the preset threshold? And the current number of feature prototypes is less than the preset maximum value. Determine whether the number of feature prototypes is less than a preset threshold or reaches a preset maximum number of feature prototypes. The progressive characterization extension module replicates the feature prototype with the largest reconstruction error and applies random perturbations to expand the number of feature prototypes; The structure-aware total loss calculation module uses a weighted average of 3D structure similarity loss and detection loss to calculate the total structure-aware loss; The network update and optimization module optimizes all parameters of the feature prototype vector and the attention network simultaneously through backpropagation with loss minimization.

[0052] The water video target segmentation system based on progressive representation expansion also includes a bidirectional cyclic inference optimization module. The module inputs the video sequence into the attention network in a forward order to obtain the forward inference result; it inputs the video sequence into the attention network in a reverse order to obtain the backward inference result; and it fuses the forward inference result and the backward inference result to output the optimized segmentation result.

[0053] The above is a further description of the present invention in conjunction with specific embodiments, and the scope of protection of the present invention is not limited thereto.

Claims

1. A water area video target segmentation method based on progressive representation expansion, characterized by: The steps include the following: S1. Collect video sequence data in the water scene and obtain multiple consecutive frames of images; S2. A small set of feature prototypes is obtained through feature prototype initialization; S3. Input the feature prototype set and consecutive multi-frame images into the attention network, calculate the correlation between each feature prototype and image features through the attention mechanism, obtain the association probability between each feature prototype and each pixel position, and generate the target segmentation result under the current number of feature prototypes. S4. Progressively expand the feature prototypes, calculate the reconstruction error of the current segmentation result, and if the reconstruction error is greater than the preset threshold, copy the feature prototype with the largest reconstruction error and apply random perturbation to expand the number of feature prototypes. S5. Repeat steps S3 and S4 with the expanded feature prototype set until the reconstruction error converges or the preset maximum number of feature prototypes is reached, and output the final target segmentation result. S6. The total structural perception loss is calculated by weighting the 3D structural similarity loss and the detection loss. The loss is minimized through backpropagation, and all parameters of the feature prototype vector and the attention network are optimized to obtain the final segmentation model.

2. The water area video target segmentation method based on progressive representation expansion according to claim 1, characterized in that, It also includes a step after step S6 to optimize the result using a bidirectional cyclic inference mechanism, in which the video sequence is input into the attention network in a forward order to obtain the forward inference result; the video sequence is input into the attention network in a reverse order to obtain the reverse inference result; and the forward inference result and the reverse inference result are fused to output the optimized segmentation result.

3. The water area video target segmentation method based on progressive representation expansion according to claim 1, characterized in that, In step S2, each feature prototype is defined as a learnable vector with a dimension of 256, represented as follows: ， Where K is the total number of current feature prototypes, initially K=2, S1 corresponds to the foreground target prototype, and S2 corresponds to the background prototype.

4. The water area video target segmentation method based on progressive representation expansion according to claim 1, characterized in that, In step S3, the attention network comprises three sub-modules: an encoder, an attention computation unit, and an iterative update unit. (1) The encoder uses a CNN backbone network to extract high-dimensional feature representations from the original input image and outputs a feature map F. Before inputting the attention calculation unit, the feature map F is enhanced with positional encoding to preserve spatial location information: , For sinusoidal position encoding or learnable position embedding; (2) The attention calculation unit uses the feature prototype as the query, image features Using key and value pairs, the correspondence between each feature prototype and image feature is calculated through a cross-attention mechanism to obtain the aggregated feature U: ， Where Q is the projection of the feature prototype, and K and V are the projections of the image features; (3) The aggregated feature U is iteratively updated using a multilayer perceptron to gradually refine the representational power of the feature prototype, as expressed by the formula: ， Where α is the learning rate scaling factor, used to control the update step size and avoid training instability; After each iteration, the attention weight matrix obtained in the attention computation unit represents the association probability between each feature prototype and each pixel location. ， Softmax normalization is performed along the feature prototype dimension, ensuring that the sum of the probabilities of all prototypes at each pixel location is 1. The attention weights are then rearranged along the feature prototype dimension to obtain a pixel-level segmentation probability map, which is the target segmentation result for the current number of prototypes.

5. The water area video target segmentation method based on progressive representation expansion according to claim 1, characterized in that, The progressive representation expansion process in step S4 is as follows: (1) First, calculate the reconstruction error L between the current segmentation result and the real mask. rec As a basis for judging whether expansion is needed, the weighted sum of cross-entropy loss and structural similarity loss is adopted; (2) Expansion trigger condition judgment, when the reconstruction error Greater than the preset threshold And the current number of feature prototypes is less than the preset maximum value. When the time is right, trigger the expansion; otherwise, continue the current training without expansion. (3) Calculate the pixel-level reconstruction error of the region corresponding to each feature prototype, select the prototype with the largest error, calculate the reconstruction error of each pixel i, which is the difference between the true class and the predicted probability of the pixel, aggregate the error according to the feature prototype, and select the feature prototype with the largest cumulative reconstruction error for expansion. (4) Copy the selected feature prototype and apply perturbation to generate a new feature prototype. (5) Add the new feature prototype to the existing set to form the expanded feature prototype set, and update the number of feature prototypes at the same time. The expanded feature prototype set continues to participate in subsequent training iterations.

6. The water area video target segmentation method based on progressive representation expansion according to claim 1, characterized in that, In step S6, 3D structural similarity loss is used, and its calculation formula is as follows: ， in, , The mean of the local region. , Standard deviation, For covariance, , As a stability constant, the loss function is calculated simultaneously in both spatial and temporal dimensions and then averaged to ensure that the segmentation results maintain clear boundaries in space and remain continuous and consistent in time. The total loss function is: ， in, To detect the loss, This is the balance coefficient.

7. A water area video target segmentation system based on progressive representation expansion, characterized in that, It includes a video acquisition module, a feature prototype initialization module, an attention-based segmentation module, a reconstruction error calculation and comparison module, a progressive representation expansion module, a structure-aware total loss calculation module, and a network update and optimization module. The video acquisition module collects video sequence data in aquatic scenes and obtains multiple consecutive frames of images; The feature prototype initialization module obtains a small set of feature prototypes through feature prototype initialization. The attention-based segmentation module takes a set of feature prototypes and multiple consecutive frames of images as input. It calculates the correlation between each feature prototype and image features through an attention mechanism, obtains the association probability between each feature prototype and each pixel position, and generates the target segmentation result for the current number of feature prototypes. The reconstruction error calculation and comparison module calculates the reconstruction error of the current segmentation result and judges the reconstruction error. Is it greater than the preset threshold? And the current number of feature prototypes is less than the preset maximum value. ; Determine whether the number of feature prototypes is less than a preset threshold or reaches a preset maximum number of feature prototypes; The progressive characterization extension module replicates the feature prototype with the largest reconstruction error and applies random perturbations to expand the number of feature prototypes; The structure-aware total loss calculation module uses a weighted average of 3D structure similarity loss and detection loss to calculate the total structure-aware loss; The network update and optimization module optimizes all parameters of the feature prototype vector and the attention network simultaneously through backpropagation with loss minimization.

8. The water video target segmentation system based on progressive representation expansion according to claim 7, characterized in that, It also includes a bidirectional cyclic inference optimization module, which inputs the video sequence into the attention network in a forward order to obtain the forward inference result; inputs the video sequence into the attention network in a reverse order to obtain the reverse inference result; and fuses the forward inference result and the reverse inference result to output the optimized segmentation result.

Images