Optical flow-assisted hybrid video segmentation for moving objects

Abstract

The method and system include: applying gamma adjustment to frames of an input video with specific parameter values ​​to generate at least one set of gamma-adjusted frames; applying segmentation techniques to at least one set of gamma-adjusted frames to generate a segmentation mask; applying optical flow techniques to at least one set of gamma-adjusted frames to generate an optical flow map; and combining the segmentation mask and the optical flow map to generate a hybrid segmentation mask.

Description

Technical Field

[0001] This disclosure relates to video segmentation, and more specifically, to generating improved segmentation masks. Background Technology

[0002] Conventional machine learning (ML)-based segmentation techniques using masks can generate sufficiently good results for non-professional media content, such as low-resolution videos on social media. However, the quality of the segmentation masks may not be high enough to meet the requirements of professional image / video processing tasks. For example, edge sharpness varies from frame to frame, which may lead to incorrectly inferred subpixels in the masked areas. Therefore, in some scenarios, ML-based segmentation techniques may fail to produce reliable and / or consistent segmentation masks. These scenarios may include: high-resolution images (e.g., HD, 4K); dynamic scenes, especially those with fast-moving objects; color-grading content (e.g., low brightness, texture-like); dark scenes; and multiple target objects in a scene to be segmented independently. Summary of the Invention

[0003] This disclosure provides for generating improved segmentation masks for segmenting objects in motion.

[0004] In one implementation, a method is disclosed. The method includes: applying gamma adjustment to frames of an input video with specific parameter values ​​to generate at least one set of gamma-adjusted frames; applying a segmentation technique to the at least one set of gamma-adjusted frames to generate a segmentation mask; applying an optical flow technique to the at least one set of gamma-adjusted frames to generate an optical flow map; and combining the segmentation mask and the optical flow map to generate a hybrid segmentation mask.

[0005] In one implementation, combining the segmentation mask and optical flow map involves multiplying the pixel-level code values ​​of the red, green, and blue components of the sub-pixels of the object to which the segmentation technique has been applied. In one implementation, the segmentation technique includes: classifying each frame in at least one set of gamma-adjusted frames into a class; detecting objects within each frame and drawing boundaries around the objects; and identifying portions of each frame and mapping each portion of the frame to an object. In one implementation, each flow graph in the optical flow maps is a graph of the motion of objects between consecutive frames in at least one set of gamma-adjusted frames. In one implementation, the at least one set of gamma-adjusted frames includes a first set of gamma-adjusted frames and a second set of gamma-adjusted frames. In one implementation, a first set of gamma-adjusted frames is generated for the segmentation technique. In one implementation, a second set of gamma-adjusted frames is generated for the optical flow technique. In one implementation, the method further includes: applying gamma adjustments to frames of the input video with different parameter values ​​to generate different sets of gamma-adjusted frames when there is a misalignment between the segmentation mask and the optical flow map, and repeating the application and combination of the segmentation technique and the optical flow technique.

[0006] In another implementation, a system is disclosed. The system includes: a gamma function applicator for applying gamma adjustment to frames of an input video using a set of parameter values ​​to generate at least one set of gamma-adjusted frames; a segmentation mask generator for applying segmentation techniques to at least one set of gamma-adjusted frames to generate a segmentation mask; an optical flow graph generator for applying optical flow techniques to at least one set of gamma-adjusted frames to generate an optical flow graph; and a combiner for combining the segmentation mask and the optical flow graph to generate a hybrid segmentation mask.

[0007] In one implementation, the system further includes a processor for receiving a hybrid segmentation mask and determining whether the combiner produces an acceptable alignment between the segmentation mask and the optical flow graph. If the alignment is unacceptable, the processor instructs a gamma function applicator to use another set of parameter values ​​to improve the alignment between the segmentation mask and the optical flow graph and repeats the process performed by the segmentation mask generator and the optical flow graph generator. In one implementation, the segmentation mask generator includes: a classifier for classifying each frame in at least one set of gamma-adjusted frames into classes; a detector for detecting objects within each frame and drawing boundaries around the objects; and a recognizer for identifying portions of each frame and mapping each portion of the frame to an object. In one implementation, each flow graph in each optical flow graph is a graph of the motion of objects between consecutive frames in at least one set of gamma-adjusted frames. In one implementation, the at least one set of gamma-adjusted frames includes a first set of gamma-adjusted frames and a second set of gamma-adjusted frames. In one implementation, a first set of gamma-adjusted frames is generated for the segmentation technique. In one implementation, a second set of gamma-adjusted frames is generated for the optical flow technique.

[0008] In a further implementation, a non-transient computer-readable storage medium is disclosed storing a computer program for generating a segmentation mask. The computer program includes executable instructions that cause a computer to perform the following processes: apply gamma adjustment to frames of an input video with specific parameter values ​​to generate at least one set of gamma-adjusted frames; apply segmentation techniques to at least one set of gamma-adjusted frames to generate a segmentation mask; apply optical flow techniques to at least one set of gamma-adjusted frames to generate an optical flow map; and combine the segmentation mask and the optical flow map to generate a hybrid segmentation mask.

[0009] In one implementation, the segmentation technique includes executable instructions that cause a computer to perform the following processes: classifying each frame in at least one set of gamma-adjusted frames into a class; detecting objects within each frame and drawing boundaries around the objects; and identifying portions of each frame and corresponding each portion of the frame to an object. In one implementation, each flow graph in the optical flow graphs is a graph of the motion of objects between consecutive frames in at least one set of gamma-adjusted frames. In one implementation, the at least one set of gamma-adjusted frames includes a first set of gamma-adjusted frames and a second set of gamma-adjusted frames, wherein the first set of gamma-adjusted frames is generated for the segmentation technique, and wherein the second set of gamma-adjusted frames is generated for the optical flow technique. In one implementation, the computer program also includes executable instructions that cause a computer to perform the following processes: applying gamma adjustments to frames of the input video with different parameter values ​​to generate different sets of gamma-adjusted frames when there is a misalignment between the segmentation mask and the optical flow graph, and repeating the application and combination of the segmentation technique and the optical flow technique.

[0010] As will become clear from other features and advantages of this specification, various aspects of this disclosure are illustrated by example. Attached Figure Description

[0011] Details of this disclosure regarding both its structure and operation can be partially gathered through the study of the accompanying drawings, in which similar reference numerals denote similar parts, wherein,

[0012] Figure 1 This is a flowchart of an implementation of a method for generating more consistent and accurate segmentation masks according to this disclosure;

[0013] Figure 2 Represents the original frames of the input video and the gamma-adjusted frames of the second video;

[0014] Figure 3 This represents the gamma-adjusted frame and the generated segmentation mask;

[0015] Figure 4A An implementation representing the gamma-adjusted frame and the generated optical flow map;

[0016] Figure 4B This represents the optical flow map generated from a gamma-adjusted frame when the frame contains multiple moving objects;

[0017] Figure 5 This represents the segmentation mask, optical flow map, and the generated hybrid segmentation mask;

[0018] Figure 6 This is a block diagram of a segmentation mask generation system according to an implementation of this disclosure;

[0019] Figure 7AThis refers to the computer system and user representation according to embodiments of this disclosure; and

[0020] Figure 7B This is a functional block diagram illustrating a computer system for a managed video application according to an embodiment of the present disclosure. Detailed Implementation

[0021] As mentioned above, conventional ML-based segmentation techniques may not produce reliable and / or consistent segmentation masks in scenarios involving high-resolution images, dynamic scenes including fast-moving objects, color-grading content, dark scenes, and / or scenarios with multiple target objects in the scene.

[0022] Some implementations of this disclosure provide methods and systems for generating more consistent and accurate segmentation masks using hybrid segmentation and optical flow techniques. Furthermore, image preprocessing techniques, including gamma correction, are used to ensure the effectiveness of the techniques. Although the optimal input videos for the segmentation mask generation process and the optical flow map generation process may differ, each input video can be adjusted (e.g., using gamma correction) to achieve improved combined performance of the stacked layer segmentation results.

[0023] After reading the following description, it will become clear how this disclosure can be implemented in various implementations and applications. Although various implementations of this disclosure will be described herein, it should be understood that these implementations are presented as examples only and not as limitations. Therefore, the detailed descriptions of the various implementations should not be construed as limiting the scope or breadth of this disclosure.

[0024] In one implementation, to improve the consistency and accuracy of the segmentation mask, the following steps can be taken: (a) applying a gamma function to the original frame to fine-tune brightness and / or contrast; (b) applying a segmentation technique to the gamma-adjusted frame to generate a segmentation mask; (c) applying an optical flow technique to the gamma-adjusted frame to generate an optical flow map; and (d) stacking (or combining) the segmentation mask and the optical flow map to generate a hybrid segmentation mask. If the stacking process in step (d) produces unacceptable alignment, the image preprocessing in step (a) (e.g., brightness and contrast control) is adjusted to improve the alignment. The term "unacceptable alignment" refers to a misalignment of the segmentation mask and the optical flow map along the boundary by a predetermined number of pixels, which also results in visual misalignment. In one implementation, the predefined number of pixels is 5.

[0025] Figure 1 This is a flowchart of a method 100 for generating more consistent and accurate segmentation masks according to one implementation of this disclosure. In one implementation, a hybrid ML-based segmentation and optical flow technique is used to generate the segmentation mask.

[0026] exist Figure 1In the illustrated implementation, at step 110, an input video is received and gamma adjustment is applied to frames of the input video using specific parameter values ​​(including brightness and contrast) to generate at least one set of gamma-adjusted frames. The application of the gamma function generates a second video with gamma-adjusted frames. In one implementation, the application of the gamma function or gamma adjustment refers to a non-linear adjustment of brightness and contrast to enhance midtones while reducing the impact on extreme dark and bright areas. Therefore, by applying gamma adjustment, brightness and contrast are enhanced, making the output image appear brighter and more natural. Figure 2 This represents the original frame 200 of the input video and the gamma-adjusted frame 210 of the second video.

[0027] exist Figure 1 In the illustrated implementation, at step 120, the segmentation technique is applied to the gamma-adjusted frame to generate a segmentation mask. In one implementation, the segmentation technique may include: classifying the image into classes (e.g., person, bicycle, mountain, etc.); detecting objects within the image and drawing boundaries around the objects; and identifying portions (or “subpixels”) of the image and mapping them to objects. Figure 3 This represents the gamma-adjusted frame 300 and the generated segmentation mask 310.

[0028] exist Figure 1 In the illustrated implementation, at step 130, optical flow techniques are applied to the gamma-adjusted frame to generate an optical flow map. In one implementation, the optical flow map is a graph of the motion of an object across consecutive frames of video caused by movement between the camera and the object. For example, in one implementation, the optical flow map technique may include: assigning gray levels to each vector based on the orientation of each vector; and changing the intensity of the gray levels based on the norm of the gray levels. Figure 4A This represents one implementation of the gamma-adjusted frame 400 and the generated optical flow map 410. In another implementation, Figure 4B This represents the optical flow map 430 generated from the gamma-adjusted frame 420 when the frame includes multiple moving objects. Figure 4B The optical flow diagram 430 in the image represents objects and / or people 432, 434 moving in different directions in frames indicated by different gray levels.

[0029] exist Figure 1 In the illustrated implementation, the segmentation mask and optical flow map are stacked or combined at step 140 to generate a hybrid segmentation mask. In one implementation, the stacking process is a pixel-level code value multiplication for the red, green, and blue components, respectively, where the multiplication applies only to the segmented sub-pixels of a certain class of objects in the segmentation map. As described above, the two source images used for the stacking process include a segmentation mask and an optical flow map. For segmentation purposes, moving objects in the frame are of interest. Figure 5 This represents the segmentation mask 510, the optical flow map 520, and the generated hybrid segmentation mask 530.

[0030] As mentioned above, for the segmentation mask generation process (such as...) Figure 3 (as shown) and the optical flow map generation process (as shown) Figure 4A and Figure 4B The optimal video input (as shown) may differ. Therefore, each video should be adjusted (using appropriate but different gamma correction) to achieve improved performance of the stacked layer segmentation results.

[0031] exist Figure 1 In the illustrated implementation, if the stacking process in step 140 (such as...) Figure 5 If an unacceptable alignment is generated at step 150, then frame preprocessing (e.g., brightness and contrast control) is adjusted at step 160 to improve the alignment. In one implementation, frame preprocessing after step 140 may include another gamma correction, but with a different amount of correction. In another implementation, a different preprocessing method utilizing a 3D lookup table (LUT) is used. The goal of this step is to normalize the appearance of the color-graded image so that existing segmentation and / or optical flow algorithms can produce accurate results. Therefore, step 160 includes: applying gamma adjustment to frames of the input video with different parameter values ​​to generate different sets of gamma-adjusted frames when there is misalignment between the segmentation mask and the optical flow map, and repeating the application and combination of segmentation and optical flow techniques. In one implementation, parameters include brightness and / or contrast. In another implementation, parameters include brightness and / or hue. As mentioned above, the term "unacceptable alignment" refers to a misalignment between the segmentation mask and the optical flow map along the boundary by a predetermined number of pixels (e.g., 5), which also results in visual misalignment.

[0032] Method 100 then moves to repeat steps 120 through 140. Otherwise, if the stacking process in step 140 produces an acceptable alignment, then at step 150, method 100 terminates with a generated hybrid segmentation mask 530 for video segmentation.

[0033] Figure 6 This is a block diagram of a segmentation mask generation system 600 according to an implementation of this disclosure. Figure 6 In the illustrated implementation, the segmentation mask generation system 600 includes a gamma function applicator 610, a segmentation mask generator 620, an optical flow graph generator 630, a combiner 640, and a processor 650. In one implementation, the segmentation mask generation system 600 is a fully hardware-configured system, which includes one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable arrays (FPGAs), or other equivalent integrated or discrete logic circuits. In another implementation, the segmentation mask generation system 600 is configured by a combination of hardware and software.

[0034] exist Figure 6 In this implementation, a gamma function applicator 610 receives input video 602 and applies a gamma function to frames of input video 602 to fine-tune the brightness and / or contrast of input video 602 (especially moving objects). The gamma function applicator 610 generates a second video 612 with gamma-adjusted frames. In one implementation, a segmentation mask generator 620 applies segmentation techniques to the gamma-adjusted frames to generate a segmentation mask. In one implementation, the segmentation mask generator 620 includes: a classifier 622 for classifying each frame in at least one set of gamma-adjusted frames into a class; a detector 624 for detecting objects within each frame and drawing boundaries around the objects; and a recognizer 626 for identifying portions of each frame and corresponding them to objects. In one implementation, an optical flow map generator 630 applies optical flow techniques to the gamma-adjusted frames to generate an optical flow map. A combiner 640 stacks or combines the segmentation mask and the optical flow map to generate a hybrid segmentation mask 604.

[0035] As described above, the optimal gamma-adjusted frame for the second video 612 used in the segmentation mask generator 620 may differ from the optimal gamma-adjusted frame for the second video 612 used in the optical flow map generator 630. Therefore, the gamma-adjusted frames for the second video 612 used in each of the generators 620 and 630 should be adjusted (with appropriate but different gamma correction) to achieve improved performance of the stacked layer segmentation results.

[0036] exist Figure 6 In this process, the hybrid segmentation mask 642 generated by combiner 640 is received by processor 650, which checks to determine whether the combining process has produced an acceptable alignment. If not, processor 650 instructs gamma function applicator 610 to adjust frame preprocessing (e.g., brightness and contrast control) to improve alignment and repeats the process performed by segmentation mask generator 620 and optical flow map generator 630. Otherwise, if processor 650 determines that combiner 640 has produced an acceptable alignment, the hybrid segmentation mask 604 generated by combiner 640 is output for video segmentation.

[0037] Figure 7A This describes the behavior of a computer system 700 and a user 702 according to an embodiment of this disclosure. User 702 uses the computer system 700 to implement [the functionality] according to [the present disclosure]. Figure 1 Method 100 and Figure 6 The system 600 generates video segmentation masks for application 790.

[0038] Computer System 700 Store and Execute Figure 7BThe video application 790. Additionally, the computer system 700 can communicate with the software program 704. The software program 704 may include software code for the video application 790. As explained further later, the software program 704 can be loaded onto external media such as a CD, DVD, or memory drive.

[0039] Furthermore, computer system 700 can connect to network 780. Network 780 can be connected in various different architectures (e.g., client-server architecture, peer-to-peer network architecture, or other types of architecture). For example, network 780 can communicate with server 785, which coordinates the engine and data used within video application 790. Moreover, the network can be of different types. For example, network 780 can be the Internet, a local area network (LAN) or any variant of a LAN, a wide area network (WAN), a metropolitan area network (MAN), an intranet or extranet, or a wireless network.

[0040] Figure 7B This is a functional block diagram illustrating a computer system 700 hosting a video application 790 according to an implementation of the present disclosure. A controller 710 is a programmable processor that controls the operation of the computer system 700 and its components. The controller 710 loads instructions (e.g., in the form of a computer program) from a memory 720 or an embedded controller memory (not shown) and executes these instructions to control the system. In its execution, the controller 710 provides the video application 790 with a software system to achieve, for example, the generation of more consistent and accurate segmentation masks. Alternatively, this service can be implemented as a separate hardware component within the controller 710 or the computer system 700.

[0041] The memory 720 temporarily stores data for use by other components of the computer system 700.

[0042] In one implementation, memory 720 is implemented as RAM. In another implementation, memory 720 also includes long-term or permanent memory, such as flash memory and / or ROM.

[0043] Storage device 730 stores data for use by other components of computer system 700, either temporarily or for a long period of time. For example, storage device 730 stores data used by video application 790.

[0044] In one implementation, storage device 730 is a hard disk drive.

[0045] Media device 740 receives removable media and reads data from and / or writes data to the inserted media. In one example, media device 740 is an optical disc drive.

[0046] User interface 750 includes components for accepting user input from a user of computer system 700 and presenting information to user 702. In one implementation, user interface 750 includes a keyboard, mouse, audio speakers, and a display. Controller 710 uses input from user 702 to adjust the operation of computer system 700.

[0047] I / O interface 760 includes one or more I / O ports for connecting to corresponding I / O devices such as external storage devices or supplementary devices (e.g., printers or PDAs). In one implementation, the ports of I / O interface 760 include ports such as USB ports, PCMCIA ports, serial ports, and / or parallel ports. In another implementation, I / O interface 760 includes a wireless interface for wireless communication with external devices.

[0048] Network interface 770 includes wired and / or wireless network connections, such as an RJ-45 or “Wi-Fi” interface (including but not limited to 802.11) that supports Ethernet connectivity.

[0049] Computer system 700 includes typical additional hardware and software of a computer system (e.g., power supply, cooling, operating system), but for simplicity, these components are not listed here. Figure 7B This is illustrated in detail. In other implementations, different configurations of the computer system may be used (e.g., different bus or storage configurations or multiprocessor configurations).

[0050] This disclosure describes a hybrid approach that utilizes segmentation and optical flow techniques to generate more consistent and accurate segmentation masks. Furthermore, image preprocessing techniques such as gamma correction are used to enhance the effectiveness of the approach. The advantages of the hybrid approach include: (a) efficiency in applying gamma correction to semantic understanding of scene structure to handle source images that are inherently difficult to analyze (e.g., dark scenes); efficiency of the segmentation network when the scene is static; accuracy of the optical flow network when depicting motion vectors in dynamic scenes; the ability of optical flow to identify multiple moving objects as independent objects; and independent optimization of gamma correction for the segmentation engine and the optical flow engine to maximize the performance of each engine.

[0051] The description of the disclosed implementations is provided to enable those skilled in the art to make or use this disclosure. Many modifications to these implementations will be apparent to those skilled in the art, and the guidelines defined herein may be applied to other implementations without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not intended to be limited to the implementations shown herein, but is to be accorded the broadest scope consistent with the principal and novel features disclosed herein.

[0052] Therefore, other variations and implementations are also possible. For example, one implementation leverages multiple video segmentation methods that excel in handling different types of source images to provide consistent and stable video segmentation results across various video scenes. Furthermore, input image preprocessing is optimized for each video segmentation method to maximize the overall performance of the combined video segmentation process. One part of the video segmentation method is based on an optical flow model, which is advantageous in scenes with motion and the independent recognition of multiple objects.

[0053] A higher-fidelity video segmentation engine trained using the above-described approach can be used to automate the actor and object masking process during the visual effects (VFX) phase of content creation. This masking is currently done manually, but this time-consuming process can be automated using machine learning-based tools.

[0054] Further variations and implementations include: eliminating the need for a green screen in content creation; image segmentation and shape recognition for robotics, autonomous driving, and factory automation; enhanced object recognition performance by focusing on target objects; and enhanced image matching performance by leveraging object segmentation.

[0055] Those skilled in the art will understand that the various illustrative modules and method steps described herein can be implemented as electronic hardware, software, firmware, or a combination thereof. To clearly illustrate this interchangeability between hardware and software, the various illustrative modules and method steps are described generally according to their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and design constraints imposed on the system as a whole. Those skilled in the art may implement the described functionality in different ways for each specific application, but such implementation decisions should not be construed as causing a departure from the scope of this disclosure. Furthermore, the grouping of functions within a module or step is for ease of description. Specific functions may be moved from one module or step to another without departing from this disclosure.

[0056] In certain implementations of this disclosure, not all features of each of the examples above are necessarily required. Furthermore, it should be understood that the descriptions and figures given herein represent subject matter broadly contemplated by this disclosure. It should also be understood that the scope of this disclosure fully covers other implementations that may become apparent to those skilled in the art, and therefore, the scope of this disclosure is limited only by the appended claims.

Claims

1. A method comprising: Apply gamma adjustment to the frames of the input video using a set of parameter values ​​to generate at least one set of gamma-adjusted frames; The segmentation technique is applied to the at least one set of gamma-adjusted frames to generate a segmentation mask; Optical flow technology is applied to the at least one set of gamma-adjusted frames to generate an optical flow map; as well as Combine the segmentation mask and optical flow map to generate a hybrid segmentation mask. The combination of segmentation mask and optical flow map includes multiplying the pixel-level code values ​​of the red, green, and blue components of the sub-pixels of the object to which the segmentation technique has been applied.

2. The method according to claim 1, wherein, The segmentation technology includes: Each frame in the at least one set of gamma-adjusted frames is classified into a class; Detect objects within each frame and draw the boundaries around the objects; and Identify the parts of each frame and map each part of the frame to an object.

3. The method according to claim 2, wherein, Each flow graph in the optical flow graphs is a graph of the motion of objects between consecutive frames in the at least one set of gamma-adjusted frames.

4. The method according to claim 1, wherein, The at least one set of gamma-adjusted frames includes a first set of gamma-adjusted frames and a second set of gamma-adjusted frames.

5. The method according to claim 4, wherein, The first set of gamma-adjusted frames is generated for the segmentation technique.

6. The method according to claim 4, wherein, Generate a second set of gamma-adjusted frames for optical flow technology.

7. The method according to claim 1, further comprising: When there is a misalignment between the segmentation mask and the optical flow map: Apply gamma adjustment to the frames of the input video using another set of parameter values ​​to generate at least one additional set of gamma-adjusted frames. The segmentation technique is applied to the additional at least one set of gamma-adjusted frames to generate a segmentation mask; Optical flow technology is applied to at least one additional set of gamma-adjusted frames to generate an optical flow map; and Combine the segmentation mask and optical flow map to generate a hybrid segmentation mask.

8. A system comprising: A gamma function applicator for applying gamma adjustment to frames of an input video using a set of parameter values ​​to generate at least one set of gamma-adjusted frames. A segmentation mask generator for applying segmentation techniques to the at least one set of gamma-adjusted frames to generate a segmentation mask; An optical flow graph generator for applying optical flow technology to the at least one set of gamma-adjusted frames to generate an optical flow graph; and A combiner used to combine segmentation masks and optical flow maps to generate hybrid segmentation masks. The combination of segmentation mask and optical flow map includes multiplying the pixel-level code values ​​of the red, green, and blue components of the sub-pixels of the object to which the segmentation technique has been applied.

9. The system according to claim 8, further comprising: A processor for receiving a hybrid segmentation mask and determining whether the combiner produces an acceptable alignment between the segmentation mask and the optical flow map; if the alignment is unacceptable, the processor instructs, in order to improve the alignment between the segmentation mask and the optical flow map: The gamma function applicator applies gamma adjustment to the frames of the input video using another set of parameter values ​​to generate at least one additional set of gamma-adjusted frames. The segmentation mask generator applies segmentation techniques to at least one additional set of gamma-adjusted frames to generate a segmentation mask; The optical flow map generator applies optical flow technology to at least one additional set of gamma-adjusted frames to generate an optical flow map; and The combiner combines the segmentation mask and the optical flow map to generate a hybrid segmentation mask.

10. The system according to claim 8, wherein, The segmentation mask generator includes: A classifier used to classify each frame in at least one set of gamma-adjusted frames into a class; A detector used to detect objects within each frame and draw the boundaries around the objects; and A recognizer used to identify portions of each frame and map each portion of a frame to an object.

11. The system according to claim 10, wherein, Each flow graph in the optical flow graphs is a graph of the motion of objects between consecutive frames in the at least one set of gamma-adjusted frames.

12. The system according to claim 8, wherein, The at least one set of gamma-adjusted frames includes a first set of gamma-adjusted frames and a second set of gamma-adjusted frames.

13. The system according to claim 12, wherein, The first set of gamma-adjusted frames is generated for the segmentation technique.

14. The system according to claim 12, wherein, Generate a second set of gamma-adjusted frames for optical flow technology.

15. A non-transient computer-readable storage medium storing a computer program for generating a segmentation mask, said computer program comprising executable instructions that cause a computer to perform the following processes: Apply gamma adjustment to the frames of the input video using a set of parameter values ​​to generate at least one set of gamma-adjusted frames; The segmentation technique is applied to the at least one set of gamma-adjusted frames to generate a segmentation mask; Optical flow technology is applied to the at least one set of gamma-adjusted frames to generate an optical flow map; as well as Combine the segmentation mask and optical flow map to generate a hybrid segmentation mask. The combination of segmentation mask and optical flow map includes multiplying the pixel-level code values ​​of the red, green, and blue components of the sub-pixels of the object to which the segmentation technique has been applied.

16. The computer-readable storage medium according to claim 15, wherein, Segmentation techniques include executable instructions that cause a computer to perform the following processes: Each frame in the at least one set of gamma-adjusted frames is classified into a class; Detect objects within each frame and draw the boundaries around the objects; as well as Identify the parts of each frame and map each part of the frame to an object.

17. The computer-readable storage medium according to claim 16, wherein, Each flow graph in the optical flow graphs is a graph of the motion of objects between consecutive frames in the at least one set of gamma-adjusted frames.

18. The computer-readable storage medium according to claim 15, wherein, The at least one set of gamma-adjusted frames includes a first set of gamma-adjusted frames and a second set of gamma-adjusted frames. This involves generating the first set of gamma-adjusted frames using segmentation techniques, and The second set of gamma-adjusted frames is generated using optical flow technology.

19. The computer-readable storage medium of claim 15, further comprising executable instructions that cause a computer to perform the following processes: When there is a misalignment between the segmentation mask and the optical flow map: Apply gamma adjustment to the frames of the input video using another set of parameter values ​​to generate at least one additional set of gamma-adjusted frames. The segmentation technique is applied to the additional at least one set of gamma-adjusted frames to generate a segmentation mask; Optical flow technology is applied to at least one additional set of gamma-adjusted frames to generate an optical flow map; and Combine the segmentation mask and optical flow map to generate a hybrid segmentation mask.

Images