A method and system for video image matting
By compressing the video and extracting and fusing multi-scale features, the problems of cumbersome operation and low accuracy of existing video matting methods are solved, and efficient video image matting is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GUANGDONG UNIV OF TECH
- Filing Date
- 2023-09-05
- Publication Date
- 2026-07-07
AI Technical Summary
Existing video cutout methods are cumbersome to operate and have low accuracy, especially when processing high frame rate videos, which requires a large amount of computation, demands high user expertise, and is inefficient.
After compressing the video, keyframes and non-keyframes are extracted. Multi-scale feature extraction and feature fusion models are used, combined with motion vector graphics for feature fusion and decoding, to generate a mask image to obtain the foreground image. Finally, the images are integrated to obtain the cutout image.
It reduces computational costs, improves image matting efficiency, prevents accuracy loss due to reduced resolution, and achieves fast and effective video image matting.
Smart Images
Figure CN117173207B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and in particular to a method for extracting images from video. Background Technology
[0002] Image matting aims to accurately extract foreground objects from images. Image matting methods have evolved from traditional pixel-pair optimization and propagation-based methods to deep learning-based methods. Since videos are composed of many video frames, video matting is essentially a process of matting video frame images. As video matting methods have evolved along with image matting methods, a large number of deep learning-based video matting methods have been proposed in recent years.
[0003] Current video matting methods can be divided into interactive and non-interactive approaches. Interactive matting methods require users to depict the foreground, background, and blurred edges of the foreground in the image, and the network then performs matting based on this. This method requires a high level of user expertise and is cumbersome to operate. Non-interactive matting methods do not require user input, but the accuracy of matting is not high. Moreover, current video matting methods, whether interactive or non-interactive, are relatively inefficient because they require frame-by-frame prediction of high frame rate videos, which involves a large amount of computation. Most methods to improve efficiency focus on improving the network structure, but the improvement is limited.
[0004] Most videos are compressed videos. Video frames can be divided into keyframes and non-keyframes. Keyframes are independent frames in a video sequence. They contain complete image information and do not depend on any other frames. They are usually used as the starting point of a video sequence. Non-keyframes are frames relative to keyframes. They depend on previous keyframes or other frames. In the video encoding process, non-keyframes predict the content of the current frame by referencing the data and motion vectors of previous frames. Their data volume is relatively small.
[0005] Existing technology provides a video matting method and apparatus. Based on a preset human contour model, it determines the contour region to be detected in the current frame of the video stream. Based on the pixel values of each pixel in this region and the brightness difference between each pixel and the corresponding pixel in the previous frame, an energy equation is constructed. A graph segmentation algorithm is then used to solve this equation for its minimum solution, thereby obtaining a binary label for each node in the graph structure corresponding to the algorithm. Based on the binary labels of each node, the contour region is segmented, and the human figure region in the current frame is extracted. This invention requires frame-by-frame prediction for high frame rate videos, which involves a large amount of computation. It requires the user to depict the foreground, background, and blurred edges of the foreground in the image before the network performs matting. This method requires a high level of user expertise and is cumbersome to operate. Summary of the Invention
[0006] To overcome the shortcomings of the prior art, such as cumbersome operation and low precision in image matting, the present invention provides a video image matting method and system.
[0007] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:
[0008] This invention provides a method for video image matting, the method comprising:
[0009] S1: Obtain the original video;
[0010] S2: Compress the original video to obtain a compressed video;
[0011] S3: Extract frames from the compressed video to obtain several original images and corresponding motion vector graphics;
[0012] S4: Perform multi-scale high-resolution feature extraction and multi-scale low-resolution feature extraction on any original image to obtain multi-scale high-resolution features and multi-scale low-resolution features.
[0013] S5: Feature fusion is performed on the motion vector map, the high-resolution features of the corresponding scale, and the low-resolution features to obtain multi-scale aggregated features and edge detail features;
[0014] S6: Decode the multi-scale aggregate features and edge detail features to obtain the corresponding mask image;
[0015] S7: Based on the mask image and the original image, obtain the corresponding foreground image;
[0016] S8: Repeat steps S4-S7 to obtain the foreground images corresponding to all original images;
[0017] S9: Integrate all foreground images to obtain the cut-out image.
[0018] Preferably, in step S2, the original video is compressed based on the HEVC / H.265 compression standard to obtain a compressed video.
[0019] Preferably, in step S3, the original image includes keyframes and non-keyframes.
[0020] Preferably, in step S4, a multi-scale feature extraction model is constructed to perform multi-scale high-resolution feature extraction and multi-scale low-resolution feature extraction on key frames and non-key frames in the original image, respectively. The multi-scale feature extraction model includes a first coding unit, a second coding unit, a third coding unit, a fourth coding unit, and an ASPP sub-module connected in sequence.
[0021] The first coding unit outputs high-resolution features and low-resolution features at the first scale; the second coding unit outputs high-resolution features at the second scale and low-resolution features at the second scale; the third coding unit outputs high-resolution features and low-resolution features at the third scale; the fourth coding unit outputs high-resolution features and low-resolution features at the fourth scale; and the ASPP submodule outputs edge detail features.
[0022] Preferably, in step S5, the feature fusion model constructed by inputting the motion vector image, high-resolution features of the corresponding scale, and low-resolution features is used for feature fusion.
[0023] The feature fusion model includes four sub-models arranged in parallel: a first feature fusion sub-model, a second feature fusion sub-model, a third feature fusion sub-model, and a fourth feature fusion sub-model. The output of each first encoding unit is connected to the input of the first feature fusion sub-model, the output of each second encoding unit is connected to the input of the second feature fusion sub-model, the output of each third encoding unit is connected to the input of the third feature fusion sub-model, and the output of each fourth encoding unit is connected to the input of the fourth feature fusion sub-model. The four feature fusion sub-models have the same structure, each including a feature warping layer, a first convolutional unit, a second convolutional unit, a third convolutional unit, a local attention unit, and a fusion layer.
[0024] The output of the feature distortion layer is connected to the input of both the first and second convolutional units. The outputs of the first, second, and third convolutional units are all connected to the local attention unit. The output of the local attention unit is connected to the input of the fusion layer.
[0025] Preferably, the specific method for fusing features into a feature fusion model constructed by inputting motion vector graphics, high-resolution features of the corresponding scale, and low-resolution features to obtain multi-scale aggregated features is as follows: high-resolution features, low-resolution features, and the corresponding motion vector graphics are all input into the corresponding feature fusion sub-models. The high-resolution features pass through a feature distortion layer, and the distorted features are input into the first convolutional unit and the second convolutional unit respectively to obtain the Key matrix and Value matrix. The low-resolution features pass through a third convolutional unit to obtain the Query matrix. The Key matrix, Value matrix, and Query matrix are input into the local attention unit. The feature fusion model aligns the high-resolution features with non-keyframes, and then performs feature fusion through local attention to obtain aggregated features of the corresponding scale. That is, the first feature fusion sub-model outputs the first-scale aggregated features, the second feature fusion sub-model outputs the second-scale aggregated features, the third feature fusion sub-model outputs the third-scale aggregated features, and the fourth feature fusion sub-model outputs the fourth-scale aggregated features.
[0026] Preferably, in step S6, a decoding model is constructed to decode multi-scale aggregated features; the decoding model includes a fourth decoding unit, a third decoding unit, a second decoding unit, and a first decoding unit connected in sequence; the output of the first feature fusion sub-model is connected to the input of the first decoding unit, the output of the second feature fusion sub-model is connected to the input of the second decoding unit, the output of the third feature fusion sub-model is connected to the input of the third decoding unit, and the outputs of the fourth feature fusion sub-model and the ASPP sub-module are connected to the input of the fourth decoding unit.
[0027] Preferably, in step S7, the formula for obtaining the corresponding foreground image by performing a product operation based on the mask image and the original image is:
[0028] F=αI
[0029] Where I is the original image, F is the foreground image, and α is the mask image.
[0030] Preferably, in step S9, all foreground images are integrated using FFmpeg to obtain a cutout image.
[0031] The present invention also provides a video image matting system for implementing the above-described method, the system comprising:
[0032] The video acquisition module acquires the original video.
[0033] The video compression module compresses the original video to obtain a compressed video.
[0034] The frame extraction module extracts frames from the compressed video to obtain several original images and corresponding motion vector graphics.
[0035] The feature extraction module performs multi-scale high-resolution feature extraction and multi-scale low-resolution feature extraction on any original image to obtain multi-scale high-resolution features and multi-scale low-resolution features.
[0036] The feature fusion module fuses motion vector graphics, high-resolution features at the corresponding scale, and low-resolution features to obtain multi-scale aggregated features and edge detail features.
[0037] The decoding module decodes multi-scale aggregate features and edge detail features to obtain the corresponding mask image;
[0038] The foreground image acquisition module obtains the corresponding foreground image based on the mask image and the original image;
[0039] The update module is used to update the original image and return it to the feature extraction module to obtain the foreground images corresponding to all the original images.
[0040] The integration module integrates all foreground images to obtain the cut-out image.
[0041] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:
[0042] This invention improves the efficiency of image matting by reducing video resolution and extracting features from the original images in the compressed video, thereby reducing computational costs and improving efficiency. To prevent the reduction in resolution from causing a decrease in accuracy, a feature fusion model is used to fuse high-resolution features into low-resolution features, thereby compensating for the loss of detailed features due to the reduced resolution, and ultimately achieving fast and effective image matting of video images. Attached Figure Description
[0043] Figure 1 This is a flowchart of the video image matting method described in Example 1.
[0044] Figure 2 This is a flowchart for frame extraction from the original video described in Example 2.
[0045] Figure 3 This is a flowchart of the video image matting method described in Example 2.
[0046] Figure 4 This is a schematic diagram of the feature extraction model described in Example 2.
[0047] Figure 5 This is a schematic diagram of the video image matting system described in Example 3. Detailed Implementation
[0048] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent.
[0049] To better illustrate this embodiment, some parts in the accompanying drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions;
[0050] It will be understood by those skilled in the art that certain well-known structures and their descriptions may be omitted in the accompanying drawings.
[0051] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.
[0052] Example 1
[0053] This embodiment provides a video image matting method, such as... Figure 1 As shown, the method includes:
[0054] S1: Obtain the original video;
[0055] S2: Compress the original video to obtain a compressed video;
[0056] S3: Extract frames from the compressed video to obtain several original images and corresponding motion vector graphics;
[0057] S4: Perform multi-scale high-resolution feature extraction and multi-scale low-resolution feature extraction on any original image to obtain multi-scale high-resolution features and multi-scale low-resolution features.
[0058] S5: The motion vector image, high-resolution features at the corresponding scale, and low-resolution features are fused to obtain multi-scale aggregated features.
[0059] S6: Decode the multi-scale aggregated feature transfer to obtain the corresponding mask image;
[0060] S7: Based on the mask image and the original image, obtain the corresponding foreground image;
[0061] S8: Repeat steps S4-S7 to obtain the foreground images corresponding to all original images;
[0062] S9: Integrate all foreground images to obtain the cut-out image.
[0063] The process involves acquiring the original video and compressing it to obtain a compressed video. Next, frames are extracted from the compressed video to obtain several original images and corresponding motion vector graphics. For any given original image, multi-scale high-resolution feature extraction and multi-scale low-resolution feature extraction are performed to obtain multi-scale high-resolution features and multi-scale low-resolution features. Then, the motion vector graphics, corresponding high-resolution features, and low-resolution features are fused to obtain multi-scale aggregated features. Next, the multi-scale aggregated features are decoded to obtain the corresponding mask images. Using these mask images and the original images, the corresponding foreground images are obtained. Steps S4-S7 are repeated to obtain the foreground images corresponding to all original images. Finally, all foreground images are integrated to obtain the final cutout image.
[0064] Example 2
[0065] This embodiment provides a video image matting method, the method including:
[0066] S1: Obtain the original video;
[0067] S2: Compress the original video based on the HEVC / H.265 compression standard to obtain a compressed video;
[0068] S3: Extract frames from the compressed video to obtain several key frames and non-key frames and their corresponding motion vector graphics;
[0069] S4: Construct a multi-scale feature extraction model to perform multi-scale high-resolution feature extraction and multi-scale low-resolution feature extraction on key frames and non-key frames in the original image, respectively. The multi-scale feature extraction model includes a first coding unit, a second coding unit, a third coding unit, a fourth coding unit, and an ASPP sub-module connected in sequence.
[0070] The first coding unit outputs high-resolution features and low-resolution features at the first scale; the second coding unit outputs high-resolution features at the second scale and low-resolution features at the second scale; the third coding unit outputs high-resolution features at the third scale and low-resolution features at the third scale; the fourth coding unit outputs high-resolution features at the fourth scale and low-resolution features at the fourth scale; and the ASPP submodule outputs edge detail features.
[0071] S5: Feature fusion is performed on a feature fusion model constructed by inputting motion vector graphics, high-resolution features of the corresponding scale, and low-resolution features;
[0072] The feature fusion model includes four sub-models arranged in parallel: a first feature fusion sub-model, a second feature fusion sub-model, a third feature fusion sub-model, and a fourth feature fusion sub-model. The outputs of the first encoding units are all connected to the inputs of the first feature fusion sub-model, the outputs of the second encoding units are all connected to the inputs of the second feature fusion sub-model, the outputs of the third encoding units are all connected to the inputs of the third feature fusion sub-model, and the outputs of the fourth encoding units are all connected to the inputs of the fourth feature fusion sub-model. The four feature fusion sub-models have the same structure, each including a feature distortion layer, a first convolutional unit, a second convolutional unit, a third convolutional unit, a local attention unit, and a fusion layer.
[0073] The output of the feature distortion layer is connected to the input of both the first and second convolutional units. The outputs of the first, second, and third convolutional units are all connected to the local attention unit. The output of the local attention unit is connected to the input of the fusion layer.
[0074] S6: Construct a decoding model to decode multi-scale aggregated features; the decoding model includes a fourth decoding unit, a third decoding unit, a second decoding unit, and a first decoding unit connected in sequence; the output of the first feature fusion sub-model is connected to the input of the first decoding unit, the output of the second feature fusion sub-model is connected to the input of the second decoding unit, the output of the third feature fusion sub-model is connected to the input of the third decoding unit, and the outputs of the fourth feature fusion sub-model and the ASPP sub-module are connected to the input of the fourth decoding unit; obtain the corresponding mask image;
[0075] S7: Based on the mask image and the original image, perform a product operation to obtain the corresponding foreground image using the following formula:
[0076] F=αI
[0077] Where I is the original image, F is the foreground image, and α is the mask image;
[0078] S8: Repeat steps S4-S7 to obtain the foreground images corresponding to all original images;
[0079] S9: Use FFmpeg to integrate all foreground images to obtain the cutout image.
[0080] like Figure 2 As shown, the original video is compressed using a unified video compression standard, such as HEVC / H.265. Then, keyframes, non-keyframes, and motion vector graphics are extracted from the compressed video.
[0081] Non-keyframes contain less information, while keyframes contain complete information. Video frames are essentially time-dependent, and the missing details in non-keyframes can be retrieved from the corresponding areas in the keyframes based on motion cues. Motion vector graphics in compressed videos provide such motion cues. Therefore, it is necessary to uniformly encode and compress the original video, and then extract keyframes, non-keyframes, and motion vector graphics from it.
[0082] like Figure 3 As shown, the multi-scale feature extraction model consists of four coding units, with an ASPP submodule connected after the last coding unit to better capture contextual information at different scales. Non-keyframes are input to the low-resolution feature extraction branch, where efficiency is improved by reducing the resolution of the input frames, and feature extraction is performed sequentially through four coding units. Keyframes are input to the high-resolution feature extraction branch to extract high-precision detail features, which are extracted through four coding units respectively. These high-precision detail features can be used to guide low-resolution matting and supplement missing detail features in non-keyframes.
[0083] Non-keyframes are extracted at low resolution and scale. While reducing resolution improves efficiency, it also sacrifices accuracy. Keyframes, on the other hand, contain complete information. Video frames are inherently time-dependent, and missing details in non-keyframes can be retrieved from corresponding regions within the keyframes using motion cues. Motion vector graphics in compressed video provide such motion cues, such as... Figure 3 As shown, this embodiment proposes to fuse the output features of each coding unit in the low-resolution feature extraction branch with the output features of each coding unit in the high-resolution feature extraction branch using corresponding motion vector graphics. Specifically, the structure diagram of each feature fusion submodule is shown below. Figure 4As shown, the feature fusion module takes multi-scale high-resolution features, multi-scale low-resolution features, and corresponding motion vector maps as input to generate multi-scale aggregated features.
[0084] The motion vector-based feature warping operation first warps high-resolution features to the spatial layout of low-resolution frames. Considering that warped features often contain noise, this embodiment uses local attention to effectively fuse features based on their pixel similarity. The specific method for fusing features into a feature fusion model, using motion vectors, corresponding high-resolution features, and low-resolution features as inputs, to obtain multi-scale aggregated features is as follows: High-resolution features, low-resolution features, and corresponding motion vectors are all input into their respective feature fusion sub-models. High-resolution features pass through a feature warping layer, and the warped features are input into the first and second convolutional units to obtain the Key and Value matrices, respectively. Low-resolution features pass through a third convolutional unit to obtain the Query matrix. The Key, Value, and Query matrices are then input into the local attention unit. The feature fusion model aligns the high-resolution features with non-key frames and then performs feature fusion through local attention to obtain aggregated features at the corresponding scales. Specifically, the first feature fusion sub-model outputs first-scale aggregated features, the second feature fusion sub-model outputs second-scale aggregated features, the third feature fusion sub-model outputs third-scale aggregated features, and the fourth feature fusion sub-model outputs fourth-scale aggregated features.
[0085] Feature extraction often results in the loss of a lot of important information through downsampling and upsampling, such as... Figure 3 As shown, the features output by each encoding unit are passed to the corresponding decoding unit through the feature fusion submodule, which contains more detailed information. This process preserves as much detail as possible to address the problem of insufficient detail in low-resolution features, thereby providing more detailed features to obtain a more refined matting result.
[0086] The foreground image can be represented by the following mathematical formula: I = αF + (1-α)B, α∈[0,1], where I is the original image, F is the foreground image, B is the background image, and α is the mask image. The foreground image can be obtained by combining the original image and the mask image. Since the mask image is a single-channel image, while the original image is an RGB three-channel image, to obtain the foreground image, the transparency mask needs to be expanded to the same three channels as the original image, with each channel having the same value. This is equivalent to copying the transparency mask three times and combining them into a three-channel image. Then, the transparency mask is multiplied by the corresponding channel pixel value of the original image to obtain the foreground image, i.e., F = αI. In this embodiment, the transparency mask output by the network is processed in the above way with the original image (keyframe and non-keyframe) corresponding to the input network to obtain the foreground image of the compressed video. Then, FFmpeg is used to integrate the foreground image of the compressed video into a foreground video to obtain the final cutout image.
[0087] Example 3
[0088] This embodiment also provides a video image matting system, such as Figure 5 As shown, the system for implementing the method of embodiment 1 or 2 includes:
[0089] The video acquisition module acquires the original video.
[0090] The video compression module compresses the original video to obtain a compressed video.
[0091] The frame extraction module extracts frames from the compressed video to obtain several original images and corresponding motion vector graphics.
[0092] The feature extraction module performs multi-scale high-resolution feature extraction and multi-scale low-resolution feature extraction on any original image to obtain multi-scale high-resolution features and multi-scale low-resolution features.
[0093] The feature fusion module fuses motion vector graphics, high-resolution features at the corresponding scale, and low-resolution features to obtain multi-scale aggregated features and edge detail features.
[0094] The decoding module decodes multi-scale aggregate features and edge detail features to obtain the corresponding mask image;
[0095] The foreground image acquisition module obtains the corresponding foreground image based on the mask image and the original image;
[0096] The update module is used to update the original image and return it to the feature extraction module to obtain the foreground images corresponding to all the original images.
[0097] The integration module integrates all foreground images to obtain the cut-out image.
[0098] The same or similar labels correspond to the same or similar parts;
[0099] The terms used to describe positional relationships in the accompanying drawings are for illustrative purposes only and should not be construed as limiting this patent.
[0100] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.
Claims
1. A method for video image matting, characterized in that, The method includes: S1: Obtain the original video; S2: Compress the original video to obtain a compressed video; S3: Extract frames from the compressed video to obtain several original images and corresponding motion vector graphics; S4: Perform multi-scale high-resolution feature extraction and multi-scale low-resolution feature extraction on any original image to obtain multi-scale high-resolution features and multi-scale low-resolution features. S5: Feature fusion is performed on the motion vector map, the high-resolution features of the corresponding scale, and the low-resolution features to obtain multi-scale aggregated features and edge detail features; S6: Decode the multi-scale aggregate features and edge detail features to obtain the corresponding mask image; S7: Based on the mask image and the original image, obtain the corresponding foreground image; S8: Repeat steps S4-S7 to obtain the foreground images corresponding to all original images; S9: Integrate all foreground images to obtain the cut-out image; In S3, the original image includes keyframes and non-keyframes; In S4, a multi-scale feature extraction model is constructed to perform multi-scale high-resolution feature extraction and multi-scale low-resolution feature extraction on key frames and non-key frames in the original image, respectively. The multi-scale feature extraction model includes a first coding unit, a second coding unit, a third coding unit, a fourth coding unit, and an ASPP sub-module connected in sequence. The first coding unit outputs high-resolution features and low-resolution features at the first scale; the second coding unit outputs high-resolution features at the second scale and low-resolution features at the second scale; the third coding unit outputs high-resolution features at the third scale and low-resolution features at the third scale; the fourth coding unit outputs high-resolution features at the fourth scale and low-resolution features at the fourth scale; and the ASPP submodule outputs edge detail features. In step S5, the feature fusion model constructed by inputting the motion vector image, high-resolution features of the corresponding scale, and low-resolution features is used for feature fusion. The feature fusion model includes four sub-models arranged in parallel: a first feature fusion sub-model, a second feature fusion sub-model, a third feature fusion sub-model, and a fourth feature fusion sub-model. The output of each first encoding unit is connected to the input of the first feature fusion sub-model, the output of each second encoding unit is connected to the input of the second feature fusion sub-model, the output of each third encoding unit is connected to the input of the third feature fusion sub-model, and the output of each fourth encoding unit is connected to the input of the fourth feature fusion sub-model. The four feature fusion sub-models have the same structure, each including a feature warping layer, a first convolutional unit, a second convolutional unit, a third convolutional unit, a local attention unit, and a fusion layer. The output of the feature distortion layer is connected to the input of both the first and second convolutional units. The outputs of the first, second, and third convolutional units are all connected to the local attention unit. The output of the local attention unit is connected to the input of the fusion layer. The specific method for fusing features into a feature fusion model constructed by inputting motion vector graphics, high-resolution features at the corresponding scale, and low-resolution features to obtain multi-scale aggregated features is as follows: High-resolution features, low-resolution features, and the corresponding motion vector graphics are all input into the corresponding feature fusion sub-models. The high-resolution features are passed through a feature warping layer, and the warped features are input into the first and second convolutional units respectively to obtain the Key matrix and Value matrix. The low-resolution features are passed through a third convolutional unit to obtain the Query matrix. The Key matrix, Value matrix, and Query matrix are input into the local attention unit. The feature fusion model aligns the high-resolution features with non-keyframes and then performs feature fusion through local attention to obtain aggregated features at the corresponding scale. That is, the first feature fusion sub-model outputs the first-scale aggregated features, the second feature fusion sub-model outputs the second-scale aggregated features, the third feature fusion sub-model outputs the third-scale aggregated features, and the fourth feature fusion sub-model outputs the fourth-scale aggregated features.
2. The video image matting method according to claim 1, characterized in that, In step S2, the original video is compressed based on the HEVC / H.265 compression standard to obtain a compressed video.
3. The video image matting method according to claim 1, characterized in that, In step S6, a decoding model is constructed to decode multi-scale aggregated features. The decoding model includes a fourth decoding unit, a third decoding unit, a second decoding unit, and a first decoding unit connected in sequence. The output of the first feature fusion sub-model is connected to the input of the first decoding unit, the output of the second feature fusion sub-model is connected to the input of the second decoding unit, the output of the third feature fusion sub-model is connected to the input of the third decoding unit, and the outputs of the fourth feature fusion sub-model and the ASPP sub-module are connected to the input of the fourth decoding unit.
4. The video image matting method according to claim 1, characterized in that, In step S7, the formula for obtaining the corresponding foreground image by performing a product operation based on the mask image and the original image is as follows: in, This is the original image. Foreground image, This is a mask image.
5. The video image matting method according to claim 1, characterized in that, In S9 All foreground images are integrated using FFmpeg to obtain the cut-out image.
6. A video image matting system, used to implement the method according to any one of claims 1-5, characterized in that, The system includes: The video acquisition module acquires the original video. The video compression module compresses the original video to obtain a compressed video. The frame extraction module extracts frames from the compressed video to obtain several original images and corresponding motion vector graphics. The feature extraction module performs multi-scale high-resolution feature extraction and multi-scale low-resolution feature extraction on any original image to obtain multi-scale high-resolution features and multi-scale low-resolution features. The feature fusion module fuses motion vector graphics, high-resolution features at the corresponding scale, and low-resolution features to obtain multi-scale aggregated features and edge detail features. The decoding module decodes multi-scale aggregate features and edge detail features to obtain the corresponding mask image; The foreground image acquisition module obtains the corresponding foreground image based on the mask image and the original image; The update module is used to update the original image and return it to the feature extraction module to obtain the foreground images corresponding to all the original images. The integration module integrates all foreground images to obtain the cut-out image.