Directive video segmentation method and device based on mask screening and reinforcement learning, and medium
By combining large language models and reinforcement learning methods, the problem of accurate target segmentation in complex video segmentation scenarios is solved, achieving low-cost and highly robust video target segmentation and improving the accuracy and stability of the segmentation results.
Patent Information
- Application Number
- CN202511757983.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-27
- Publication Date
- 2026-02-24
AI Technical Summary
Existing video segmentation methods are difficult to meet the requirements in open domain scenarios, resulting in mask drift or target loss. Furthermore, end-to-end deep models are heavily dependent on large-scale labeled data and computing resources, and lack natural language interaction capabilities.
By combining the semantic parsing capabilities of a large language model with reinforcement learning, and through a candidate mask generation module, a video feature extraction module, a semantic parsing module, a fusion matching module, and a reinforcement learning optimization module, an adaptive optimization using a high-score set coverage-sequential reward function is employed to achieve accurate segmentation of video targets.
It achieves highly robust and temporally consistent video target segmentation under low-cost conditions, improves the accuracy and stability of segmentation results, and has good semantic adaptability and multimodal generalization capabilities.
Smart Images
Figure CN121564618A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of language-oriented segmentation technology, and in particular to a method, device and medium for directional video segmentation based on mask filtering and reinforcement learning. Background Technology
[0002] Language-oriented video segmentation, a crucial task in computer vision, aims to extract semantically relevant target regions from video sequences. Existing basic video segmentation methods typically rely on predefined category sets, which struggle to meet the demands of open-domain scenarios; others depend on mask propagation for frame-by-frame target tracking, prone to mask drift or target loss. While some end-to-end deep learning models improve segmentation accuracy, their heavy reliance on large-scale labeled data and computational resources limits their practical application. Meanwhile, with the development of multimodal large language models, these models have demonstrated strong semantic understanding and reasoning capabilities, enabling them to understand complex videos based on natural language. However, the lack of pixel-level prediction mechanisms hinders their direct application in segmentation tasks. In contrast, while basic segmentation models can generate high-quality candidate masks, they cannot accurately select from numerous masks based on semantic information. Summary of the Invention
[0003] The purpose of this invention is to provide a directional video segmentation method, device, and medium based on mask filtering and reinforcement learning, aiming to solve or improve at least one of the above-mentioned technical problems.
[0004] To achieve the above objectives, the present invention provides the following solution: A directional video segmentation method based on mask filtering and reinforcement learning includes: Acquire the target video; A directional video segmentation model is constructed; the directional video segmentation model includes a candidate mask generation module, a video feature extraction module, a semantic parsing module, a fusion matching module, and a reinforcement learning optimization module; wherein, the reinforcement learning optimization module adopts a high-score set coverage-sequential reward function for adaptive optimization; The target video is input into the trained directional video segmentation model to calculate the segmentation result.
[0005] Optionally, the candidate mask generation module employs a segmentation model; the video feature extraction module includes an image encoder and a temporal modeling unit; and the semantic parsing module employs a large language model.
[0006] Optionally, the training process of the directional video segmentation model includes: Acquire training videos and build a proxy model; The training video is input into the proxy model. Multiple candidate masks are obtained through the candidate mask generation module. Mask features are obtained through the video feature extraction module. Language features for the segmentation task are obtained through the semantic parsing module. The preliminary segmentation result is determined through the fusion matching module. Finally, the proxy model is trained using the high-score set coverage-sequential reward function and reward function feedback mechanism through the reinforcement learning optimization module. The trained proxy model is then determined as the directional video segmentation model.
[0007] Optionally, the high-score set coverage-order reward function is constructed using a softening mechanism, specifically as follows: in, This indicates the set hit softening calculation value. Indicates the quantity to be indicated. This represents the first-ranked softening calculation value. This indicates that the calculated values are softened in the same order.
[0008] Optionally, the step of inputting the target video into the trained directional video segmentation model to calculate the segmentation result specifically includes: The target video is input into the trained directional video segmentation model. In the candidate mask generation module, multiple candidate masks are generated for each frame of the video using the existing segmentation model. In the video feature extraction module, global spatiotemporal features are extracted, and mask features corresponding to the candidate masks are obtained through mask pooling. In the semantic parsing module, the task language description input by the user is semantically encoded using a large language model to obtain language features. In the fusion matching module, the mask features and language features are interactively modeled, and the candidate mask that best meets the semantic requirements is determined through relevance matching calculation. Then, weighted fusion is performed to obtain the segmentation result.
[0009] The present invention also provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the above-described directional video segmentation method based on mask filtering and reinforcement learning.
[0010] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the directional video segmentation method based on mask filtering and reinforcement learning as described above.
[0011] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects: This invention discloses a directional video segmentation method, device, and medium based on mask selection and reinforcement learning. The method includes acquiring a target video; constructing a directional video segmentation model; the directional video segmentation model includes a candidate mask generation module, a video feature extraction module, a semantic parsing module, a fusion matching module, and a reinforcement learning optimization module; wherein, the reinforcement learning optimization module uses a high-score set coverage-sequential reward function for adaptive optimization; and inputting the target video into the trained directional video segmentation model to calculate the segmentation result. This invention introduces a large language model for semantic parsing and selection from the candidate masks generated by the segmentation model, and performs reinforcement learning optimization, thereby achieving accurate segmentation of video targets under natural language description. This fully leverages the advantages of the segmentation model in mask generation and utilizes the semantic reasoning ability of the large language model to improve the accuracy of selection and matching, achieving low-cost, highly robust, and temporally consistent segmentation results. Attached Figure Description
[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0013] Figure 1 This is a schematic diagram of the model training process in this embodiment; Figure 2 This is a schematic diagram of the model reinforcement learning process in this embodiment. Detailed Implementation
[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0015] The purpose of this invention is to provide a directional video segmentation method, device, and medium based on mask filtering and reinforcement learning, aiming to solve or improve at least one of the above-mentioned technical problems.
[0016] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0017] like Figures 1-2As shown, this invention provides a directional video segmentation method based on mask selection and reinforcement learning. Specifically, it designs an optimized method for applying a large language model to video data to accurately semantically match and fuse multiple masks output by the segmentation model. Basic segmentation models lack the ability to understand long and complex text prompts. In complex scenarios, they cannot accurately segment based on user-input questions or descriptions. The model often generates multiple candidate masks, with the region most semantically relevant being mixed among several candidate masks, thus limiting its application in interactions with more complex human commands. Meanwhile, with the development of large language models, some multimodal large language models now possess good video understanding capabilities, but segmentation is not their primary task, thus complementing traditional segmentation models. Therefore, combining the two has become a trend. This invention proposes utilizing the understanding capabilities of large language models to select more accurate results from the numerous masks output by the segmentation model for fusion and comprehensive output. Reinforcement learning is used to improve the matching accuracy of the model in both single-objective and multi-objective segmentation tasks. This strategy not only effectively combines the advantages of single-modal segmentation models and large language models but also integrates traditional segmentation and classification tasks.
[0018] The core objective of this invention is to achieve accurate target segmentation of video data under complex natural language descriptions. It constructs a low-training-cost, large-language-model-based, mask-selection, and reinforcement learning-based directional video segmentation framework, and uses reinforcement learning to improve the model's ability to select high-quality masks. This addresses the problems of traditional video segmentation methods struggling to select accurate target masks in complex scenes and lacking natural language interaction capabilities. The proposed method introduces the semantic parsing capabilities of a large-language model to perform semantic matching and fusion of candidate masks generated by the segmentation model, thereby achieving accurate video target segmentation driven by natural language and possessing good semantic adaptability and multimodal generalization capabilities.
[0019] In terms of technical solution, this invention mainly consists of five parts: a candidate mask generation module, a video feature extraction module, a semantic parsing module, a fusion matching module, and a reinforcement learning optimization module. The candidate mask generation module generates multiple candidate masks for each frame of the video using existing segmentation models. The video feature extraction module extracts global spatiotemporal features through an image encoder and a temporal modeling unit, and simultaneously obtains the feature representation of the candidate masks through mask pooling. The semantic parsing module uses a large language model to semantically encode the natural language description input by the user, obtaining task-related semantic vectors. The fusion matching module interactively models the mask features and language features, and determines the candidate mask that best meets the semantic requirements through relevance matching calculation, then performs weighted fusion to obtain preliminary segmentation results. The reinforcement learning optimization module uses the Top-K Set Coverage-Order Reward (TK-SCOR) function proposed in this invention to further adaptively optimize the mask selection process. It employs a reward function feedback mechanism to enable the model to prioritize the identification of high-quality masks with high semantic matching, thereby improving the accuracy and stability of language-oriented video segmentation results.
[0020] Based on this, the training process is divided into two stages: In the first stage (supervised training stage), the system mainly relies on labeled data to train the semantic parsing and fusion matching modules, enabling the model to learn the correlation between semantic vectors and candidate masks and to possess preliminary semantic-driven segmentation capabilities; In the second stage (reinforcement learning optimization stage), the system further introduces the high-score set coverage-order reward function (TK-SCOR) proposed in this patent to adaptively optimize the candidate mask selection strategy. The optimization focus of this stage is to guide the model to prioritize the identification of high-quality masks that highly match semantics, rather than focusing excessively on the scores or rankings of low-quality masks, thereby improving the model's sensitivity and filtering ability to high-score masks, ultimately ensuring that the output results better meet semantic requirements, and significantly enhancing the accuracy and stability of language-oriented video segmentation.
[0021] As one specific implementation method, the specific processing steps of the present invention include: Step 100: Obtain the target video.
[0022] Step 200: Construct a directional video segmentation model; the directional video segmentation model includes a candidate mask generation module, a video feature extraction module, a semantic parsing module, a fusion matching module, and a reinforcement learning optimization module; wherein, the reinforcement learning optimization module adopts a high-score set coverage-sequential reward function for adaptive optimization.
[0023] Step 300: Input the target video into the trained directional video segmentation model to calculate the segmentation result.
[0024] The training process of the directional video segmentation model includes: Acquire training videos and construct a proxy model; input the training videos into the proxy model, process them through a candidate mask generation module to obtain multiple candidate masks, obtain mask features through a video feature extraction module, obtain linguistic features for the segmentation task through a semantic parsing module, determine the preliminary segmentation result through a fusion matching module, and finally train the proxy model through a reinforcement learning optimization module using a high-score set coverage-sequential reward function and a reward function feedback mechanism, and determine the trained proxy model as the directional video segmentation model.
[0025] For the high-score set coverage – sequential reward function mentioned above: First, let the true intersection-union vector be: Next, let the predicted score be: When K takes any positive integer, the following sets and sequences are defined: :according to The first K indexes obtained from largest to smallest; :according to The first K indexes obtained from largest to smallest; and : These represent the first index in the actual and predicted results, respectively; , : Represent the top K ordered lists of the true and predicted values, respectively. Based on this, construct the following indicators: It also provides a hierarchical discrete reward: That is, only when the sets are consistent When the sets are consistent and the first name is consistent When the set is consistent with the total order .
[0026] Then, a softening mechanism is used to process the high-score set coverage-order reward function: Collection hit softening: Its value represents the overlap ratio between the predicted set and the true set, and ranges from [0,1].
[0027] Consistent softening sequence: in: This metric measures the consistency between the predicted order and the actual order, with a value range of [0,1].
[0028] First place: Softened The above formula is used to indicate whether the true best candidate appears in the predicted top K.
[0029] Softened reward function: The High-Score Set Coverage – Sequential Reward Function (TK-SCOR) provides positive feedback to the model even when the set is partially correct or the sequence is partially correct, thus preventing gradient vanishing during training and ensuring a smoother and more stable optimization process. During training, this reward is used as a return, combined with advantage estimation and policy gradients, to update the parameters of the fusion matching and scoring selection modules. This allows the model to maintain high sensitivity to high-quality masks while improving the reasonable allocation and comprehensive coverage of multi-object segmentation.
[0030] Through the above design, the framework proposed in this invention transforms language-oriented video segmentation from generative prediction to a semantically driven mask matching and selection task. During training, only the semantic matching and selection parts need to be optimized, significantly reducing computational power and data dependence. Introducing a large language model in the semantic parsing stage enables the system to understand complex user language descriptions, automatically filter candidate masks, and fuse the output, thereby greatly improving the semantic accuracy and multimodal adaptability of the segmentation results.
[0031] Based on the above technical solution, taking a camera-based visual perception system (an environmental perception module carried by a service robot) as an example, the following embodiment is provided.
[0032] This system acquires images or video frame sequences of the indoor environment via an onboard RGB or RGB-D camera, such as continuous image frames with a resolution of 640×480 pixels. Users issue task commands via voice, which are then transcribed into natural language text by a speech recognition module. A pre-segmentation module pre-segments the continuous image frames, generating a series of pixel-level candidate masks. The control unit inputs the current environment image and the corresponding language command into the language-guided mask selection module of this invention. The selection module selects the target mask that precisely matches the semantics of the command from the series of candidate masks. The mask size is consistent with the original image, used to accurately represent the target's position and contour range in the image. During training and testing, a dataset consisting of video frame sequences of the indoor scene, natural language commands, and corresponding pixel-level segmentation annotations can be constructed for training and performance verification of the method of this invention. During runtime, the visual perception system drives the subsequent control module to perform tasks such as target grasping, obstacle avoidance, or interaction based on the obtained segmentation mask, thereby achieving accurate perception and manipulation of task-related targets.
[0033] In summary, this invention transforms the traditional discriminative segmentation task into a selection-matching task. By leveraging the semantic parsing capabilities of a large language model, it addresses the challenge of traditional segmentation methods failing to select targets based on natural language tasks, achieving natural human-computer interactive language-oriented video segmentation. Furthermore, it transforms the traditional generative segmentation task into an understanding-matching-driven segmentation task. Among multiple candidate masks output by the segmentation model, it automatically selects and merges regions that best match the task's semantics, improving the accuracy and robustness of the segmentation results. Moreover, by introducing a reinforcement learning mechanism, this invention proposes a high-score set coverage-sequential reward function (TK-SCOR) to provide feedback on the mask selection results. This ensures that the model prioritizes candidate masks that highly match the semantics, making the model more sensitive and capable of filtering high-quality masks, thereby significantly improving the accuracy and stability of language-oriented video segmentation results. This invention also combines video visual features and language semantic features, fully utilizing the complementarity of the two types of models to enhance target recognition and segmentation capabilities in complex scenes.
[0034] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0035] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. A directional video segmentation method based on mask selection and reinforcement learning, characterized in that, include: Acquire the target video; A directional video segmentation model is constructed; the directional video segmentation model includes a candidate mask generation module, a video feature extraction module, a semantic parsing module, a fusion matching module, and a reinforcement learning optimization module; wherein, the reinforcement learning optimization module adopts a high-score set coverage-sequential reward function for adaptive optimization; The target video is input into the trained directional video segmentation model to calculate the segmentation result.
2. The directional video segmentation method based on mask filtering and reinforcement learning according to claim 1, characterized in that, The candidate mask generation module uses a segmentation model; the video feature extraction module includes an image encoder and a temporal modeling unit; and the semantic parsing module uses a large language model.
3. The directional video segmentation method based on mask filtering and reinforcement learning according to claim 1, characterized in that, The training process of the directional video segmentation model includes: Acquire training videos and build a proxy model; The training video is input into the proxy model. Multiple candidate masks are obtained through the candidate mask generation module. Mask features are obtained through the video feature extraction module. Language features for the segmentation task are obtained through the semantic parsing module. The preliminary segmentation result is determined through the fusion matching module. Finally, the proxy model is trained using the high-score set coverage-sequential reward function and reward function feedback mechanism through the reinforcement learning optimization module. The trained proxy model is then determined as the directional video segmentation model.
4. The directional video segmentation method based on mask filtering and reinforcement learning according to claim 1, characterized in that, The high-score set coverage-order reward function is constructed using a softening mechanism, specifically as follows: in, This indicates the set hit softening calculation value. Indicates the quantity to be indicated. This represents the first-ranked softening calculation value. This indicates that the calculated values are softened in the same order.
5. The directional video segmentation method based on mask filtering and reinforcement learning according to claim 1, characterized in that, The step of inputting the target video into the trained directional video segmentation model to calculate the segmentation result specifically includes: The target video is input into the trained directional video segmentation model. In the candidate mask generation module, multiple candidate masks are generated for each frame of the video using the existing segmentation model. In the video feature extraction module, global spatiotemporal features are extracted, and mask features corresponding to the candidate masks are obtained through mask pooling. In the semantic parsing module, the task language description input by the user is semantically encoded using a large language model to obtain language features. In the fusion matching module, the mask features and language features are interactively modeled, and the candidate mask that best meets the semantic requirements is determined through relevance matching calculation. Then, weighted fusion is performed to obtain the segmentation result.
6. An electronic device, characterized in that, The device includes a memory and a processor, the memory being used to store a computer program, and the processor running the computer program to cause the electronic device to perform the directional video segmentation method based on mask filtering and reinforcement learning according to any one of claims 1-5.
7. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the directional video segmentation method based on mask filtering and reinforcement learning as described in any one of claims 1-5.