Video target segmentation method

Abstract

The invention discloses a video target segmentation method, and relates to the technical field of target segmentation. Calculating a gray mean value and a gray mean square error value of pixel coordinate points in the first n frames of images; selecting a target pixel coordinate point of which the error absolute value between the gray value and the gray mean value is lower than a gray error standard value; constructing a background image based on the target pixel coordinate points; carrying out binarization processing on an absolute gray difference image between adjacent frames of images; carrying out image AND operation on the binary frame difference images of the kth and (k + 1) th frame images and the binary frame difference images of the (k + 1) th and (k + 2) th frame images to obtain a symmetric difference binary image of the (k + 1) th frame image; performing connected region analysis on the symmetric difference binary image of each frame of image; if the connected region does not exist, updating the connected region as a background image; and otherwise, marking a pixel region corresponding to the connected region in the image as a segmentation target. The video target segmentation method disclosed by the invention can improve the segmentation precision of the moving target.

Description

Technical Field

[0001] The present invention belongs to the technical field of target segmentation, and in particular relates to a video target segmentation method. Background Art

[0002] Video target segmentation is one of the core technologies in the field of computer vision. Its goal is to accurately extract specific moving targets (such as pedestrians, vehicles, animals, etc.) from continuous video frames. It is widely used in various scenarios such as target detection, target tracking, and video surveillance.

[0003] Currently, a common method for segmenting moving objects in videos is to extract them using the inter-frame difference method. This method locates the moving area by directly comparing the pixel differences between consecutive video frames. This method makes it very simple and convenient to segment and extract moving objects. However, when using the inter-frame difference method to extract moving objects, if the moving object's surface lacks texture, it is prone to problems such as holes and missing edges, resulting in inaccurate segmentation of the moving object, thus affecting the segmentation quality.

[0004] Therefore, how to provide an effective solution to accurately segment moving objects from videos has become a difficult problem to be solved in the prior art. Summary of the Invention

[0005] The purpose of the present invention is to provide a video object segmentation method to solve the above problems existing in the prior art.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions: In a first aspect, the present invention provides a video object segmentation method, comprising: Perform frame processing on the video stream to be segmented to obtain an image frame sequence; Calculating the grayscale mean and grayscale mean square error corresponding to each pixel coordinate point in the target image of the first n frames of the image frame sequence; Determine a grayscale error standard value based on the grayscale mean square error value, and select all target pixel coordinate points from each target image whose absolute value of the error between the corresponding grayscale value and the grayscale mean is lower than the grayscale error standard value; The image background is constructed based on the grayscale values ​​corresponding to all the determined target pixel coordinates to obtain a background image; Calculate the absolute grayscale difference image between all adjacent frame images in the image frame sequence; Binarizing the absolute grayscale difference images between adjacent frame images to obtain binary frame difference images corresponding to the adjacent frame images; Perform an image AND operation on the binary frame difference image corresponding to the k-th frame image and the k+1-th frame image, and the binary frame difference image corresponding to the k+1-th frame image and the k+2-th frame image, to obtain a symmetric differential binary image corresponding to the k+1-th frame image, where k represents a positive integer; Performing connected region analysis on the symmetric difference binary image corresponding to each frame image in time sequence, and eliminating connected regions with an area smaller than a preset area, to obtain a connected region analysis image corresponding to each frame image; If there is no connected region in the connected region analysis image corresponding to a certain frame image, updating the certain frame image as the background image; If a connected area exists in the connected area analysis image corresponding to a certain frame image, the pixel area corresponding to the connected area in the certain frame image is marked as a segmentation target, and the background of the remaining areas outside the pixel area corresponding to the connected area in the certain frame image is updated.

[0007] Based on the above disclosed content, the present invention obtains an image frame sequence by performing frame processing on the segmented video stream; calculates the grayscale mean and grayscale mean square error value corresponding to each pixel coordinate point in the target image of the first n frames of the image frame sequence; determines the grayscale error standard value based on the grayscale mean square error value, and selects all target pixel coordinate points from each target image whose absolute value of error between the corresponding grayscale value and the grayscale mean is lower than the grayscale error standard value; constructs an image background based on the grayscale values ​​corresponding to all the determined target pixel coordinate points to obtain a background image; calculates the absolute grayscale difference image between all adjacent frame images in the image frame sequence; binarizes the absolute grayscale difference image between each adjacent frame image to obtain a binarized frame difference image corresponding to each adjacent frame image; and compares the binary values ​​corresponding to the kth frame image and the k+1th frame image. A frame difference image is obtained, and an image and operation is performed on the binary frame difference image corresponding to the k+1th frame image and the k+2th frame image to obtain a symmetric differential binary image corresponding to the k+1th frame image, wherein k represents a positive integer; a connected region analysis is performed on the symmetric differential binary image corresponding to each frame image in time sequence, and connected regions with an area smaller than a preset area are eliminated to obtain a connected region analysis image corresponding to each frame image; if a connected region does not exist in the connected region analysis image corresponding to a certain frame image, the certain frame image is updated as a background image; if a connected region exists in the connected region analysis image corresponding to a certain frame image, the pixel region corresponding to the connected region in the certain frame image is marked as a segmentation target, and the background update is performed on the remaining regions other than the pixel region corresponding to the connected region in the certain frame image. In this way, the background subtraction algorithm (frame difference image image and operation) can be used to fill the holes caused by the inter-frame difference calculation. When segmenting the video target, the edge contour provided by the inter-frame difference method and the target internal pixel information provided by the background subtraction algorithm jointly participate in the moving target segmentation, avoiding inaccurate target segmentation due to the occurrence of holes, improving the moving target segmentation accuracy, and enhancing the target segmentation quality.

[0008] In one possible design, image background construction is performed based on the grayscale values ​​corresponding to all the determined target pixel coordinates, including: Based on the grayscale values ​​corresponding to all the determined target pixel coordinates, the image background is constructed according to the following formula: ; Where m represents the total number of target pixel coordinate points with pixel coordinates (x, y). Indicates the grayscale value corresponding to the i-th target pixel coordinate point with pixel coordinates (x, y).

[0009] In a possible design, the absolute grayscale difference image between adjacent frame images is ,in Indicates the grayscale value corresponding to the pixel coordinate point with pixel coordinates (x, y) in the k-th frame image. Indicates the grayscale value corresponding to the pixel coordinate point with pixel coordinates (x, y) in the k-1th frame image.

[0010] In one possible design, after calculating the absolute grayscale difference images between all adjacent frame images in the image frame sequence, the method further includes: Performing filtering on the absolute grayscale difference images between adjacent frame images; The binarization process is performed on the absolute grayscale difference image between each adjacent frame image to obtain the binarized frame difference image corresponding to each adjacent frame image, including: The absolute grayscale difference image between each adjacent frame image after filtering is binarized to obtain a binarized frame difference image corresponding to each adjacent frame image.

[0011] In one possible design, after performing an image AND operation on a binary frame difference image corresponding to the kth frame image and the k+1th frame image, and a binary frame difference image corresponding to the +1th frame image and the k+2th frame image, to obtain a symmetric difference binary image corresponding to the k+1th frame image, the method further includes: The symmetric difference binary image corresponding to each frame image is corroded and expanded.

[0012] In one possible design, an image AND operation is performed on a binary frame difference image corresponding to the k-th frame image and the k+1-th frame image, and a binary frame difference image corresponding to the k+1-th frame image and the k+2-th frame image, to obtain a symmetric differential binary image corresponding to the k+1-th frame image, including: According to the following formula, perform an image AND operation on the binary frame difference image corresponding to the k-th frame image and the k+1-th frame image, and the binary frame difference image corresponding to the k+1-th frame image and the k+2-th frame image to obtain a symmetric differential binary image corresponding to the k+1-th frame image; ; in, Represents the binary value corresponding to the pixel coordinate point (x, y) in the symmetric difference binary image corresponding to the k+1 frame image. Indicates the binary value corresponding to the pixel coordinate point (x, y) in the binary frame difference image corresponding to the k-th frame image and the k+1-th frame image. Indicates the binarized value corresponding to the pixel coordinate point with pixel coordinate (x, y) in the binarized frame difference image corresponding to the k+1th frame image and the k+2th frame image.

[0013] In a second aspect, the present invention provides a video object segmentation device, comprising: A framing unit, used for performing frame processing on the video stream to be segmented to obtain an image frame sequence; The first calculation unit is used to calculate the grayscale mean and grayscale mean square error corresponding to each pixel coordinate point in the target image of the first n frames of the image frame sequence; a determination and selection unit, configured to determine a grayscale error standard value based on the grayscale mean square error value, and select from each target image all target pixel coordinate points whose absolute value of the error between the corresponding grayscale value and the grayscale mean is lower than the grayscale error standard value; A background construction unit is used to construct an image background based on the grayscale values ​​corresponding to all the determined target pixel coordinate points to obtain a background image; A second calculation unit is used to calculate an absolute grayscale difference image between all adjacent frame images in the image frame sequence; A binarization unit is used to perform binarization processing on the absolute grayscale difference image between each adjacent frame image to obtain a binarized frame difference image corresponding to each adjacent frame image; an image AND operation unit, configured to perform an image AND operation on a binary frame difference image corresponding to the k-th frame image and the k+1-th frame image, and a binary frame difference image corresponding to the k+1-th frame image and the k+2-th frame image, to obtain a symmetric differential binary image corresponding to the k+1-th frame image, wherein k represents a positive integer; A connected region analysis unit is used to perform connected region analysis on the symmetric difference binary image corresponding to each frame image in time sequence, and eliminate connected regions with an area smaller than a preset area to obtain a connected region analysis image corresponding to each frame image; a marking and background updating unit, configured to update a frame image as a background image if no connected region exists in the connected region analysis image corresponding to the frame image; and If a connected area exists in the connected area analysis image corresponding to a certain frame image, the pixel area corresponding to the connected area in the certain frame image is marked as a segmentation target, and the background of the remaining areas outside the pixel area corresponding to the connected area in the certain frame image is updated.

[0014] In a third aspect, the present invention provides an electronic device comprising a memory, a processor and a transceiver that are communicatively connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program and execute the video target segmentation method as described in the first aspect or any possible design of the first aspect.

[0015] In a fourth aspect, the present invention provides a computer-readable storage medium having instructions stored thereon. When the instructions are run on a computer, the video target segmentation method described in the first aspect or any possible design of the first aspect is executed.

[0016] In a fifth aspect, the present invention provides a computer program product comprising instructions, which, when executed on a computer, causes the computer to execute the video object segmentation method as described in the first aspect or any possible design of the first aspect.

[0017] Beneficial effects: The present invention can fill the holes caused by inter-frame difference calculation through the background subtraction algorithm (image and operation on frame difference images). When segmenting video targets, the edge contour provided by the inter-frame difference method and the target internal pixel information provided by the background subtraction algorithm jointly participate in the moving target segmentation, avoiding inaccurate target segmentation due to the occurrence of holes, improving the moving target segmentation accuracy, enhancing the target segmentation quality, and facilitating practical application and promotion. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Flowchart of the video object segmentation method provided in an embodiment of the present application; Figure 2 A schematic block diagram of a video object segmentation device provided in an embodiment of the present application; Figure 3 A schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the present invention will be briefly introduced below in conjunction with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the structure of the drawings is only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

[0020] It should be understood that although the terms "first," "second," etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first element can be referred to as a second element, and similarly, a second element can be referred to as a first element without departing from the scope of the exemplary embodiments of the present invention.

[0021] It should be understood that the term "and / or" that may appear in this document is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B may indicate three situations: A exists alone, B exists alone, and A and B exist at the same time. The term " / and" that may appear in this document describes another type of association object relationship, indicating that two relationships may exist. For example, A / and B may indicate two situations: A exists alone, and A and B exist alone. In addition, the character " / " that may appear in this document generally indicates that the previous and subsequent associated objects are in an "or" relationship.

[0022] In order to achieve moving target segmentation, an embodiment of the present application provides a video target segmentation method, which can avoid inaccurate target segmentation due to the occurrence of holes and improve the accuracy of moving target segmentation.

[0023] The video object segmentation method provided in the embodiment of the present application can be applied to a camera or a terminal device connected to a camera. It is understood that the execution subject does not constitute a limitation on the embodiment of the present application.

[0024] The video object segmentation method provided in the embodiment of the present application will be described in detail below.

[0025] like Figure 1 , which is a flowchart of a video target segmentation method provided in the first aspect of an embodiment of the present application. The video target segmentation method may include, but is not limited to, the following steps S101 - S110 .

[0026] Step S101: Perform frame processing on the video stream to be segmented to obtain an image frame sequence.

[0027] Step S102: Calculate the grayscale mean and grayscale mean square error corresponding to each pixel coordinate point in the target image of the first n frames of the image frame sequence.

[0028] The value of n should not be too small to ensure that the background image can be collimated and constructed later. It is understandable that in some other embodiments, n consecutive frames of images in the middle of the image frame sequence can also be taken, and the grayscale mean value and grayscale mean square error value corresponding to each pixel coordinate point in the n consecutive frames of images can be calculated.

[0029] It can be understood that before calculating the grayscale mean and grayscale mean square error corresponding to each pixel coordinate point in the target image of the first n frames of the image frame sequence, each frame image in the image frame sequence needs to be converted into a grayscale image.

[0030] Step S103: Determine a grayscale error standard value based on the grayscale mean square error value, and select all target pixel coordinate points from each target image whose absolute value of the error between the corresponding grayscale value and the grayscale mean is lower than the grayscale error standard value.

[0031] Among them, the grayscale error standard value can be , where α is a preset proportional coefficient, which can generally be between 0.5 and 1. Represents the grayscale mean square error value.

[0032] Step S104: constructing an image background based on the determined grayscale values ​​corresponding to all target pixel coordinate points to obtain a background image.

[0033] Specifically, the image background may be constructed according to the following formula based on the grayscale values ​​corresponding to all the determined target pixel coordinates.

[0034] ; Where m represents the total number of target pixel coordinate points with pixel coordinates (x, y). Represents the grayscale value corresponding to the i-th target pixel coordinate point at pixel coordinates (x, y). That is, for any pixel coordinate (x, y), the grayscale value of any pixel coordinate (x, y) in the image background can be determined by calculating the grayscale mean of all target pixel coordinates at (x, y).

[0035] Step S105: Calculate the absolute grayscale difference image between all adjacent frame images in the image frame sequence.

[0036] The absolute grayscale difference image between adjacent frame images can be expressed as ,in Indicates the grayscale value corresponding to the pixel coordinate point with pixel coordinates (x, y) in the k-th frame image. Represents the grayscale value corresponding to the pixel coordinate point with pixel coordinates (x, y) in the k-1th frame image, where k is a positive integer.

[0037] Step S106 : Binarize the absolute grayscale difference image between each adjacent frame image to obtain a binary frame difference image corresponding to each adjacent frame image.

[0038] In one or more embodiments, after calculating the absolute grayscale difference images between all adjacent frame images in the image frame sequence, the absolute grayscale difference images between the adjacent frame images may be filtered. During the binarization process, the filtered absolute grayscale difference images between the adjacent frame images may be binarized to obtain binarized frame difference images corresponding to the adjacent frame images. Filtering can eliminate noise data in each absolute grayscale difference image.

[0039] Step S107. Perform an image AND operation on the binary frame difference image corresponding to the k-th frame image and the k+1-th frame image, and the binary frame difference image corresponding to the k+1-th frame image and the k+2-th frame image to obtain a symmetric differential binary image corresponding to the k+1-th frame image, where k represents a positive integer.

[0040] Image AND operation, that is, only when the corresponding two binary values ​​are both 1 (or 255), the binary value corresponding to the pixel point after the image AND operation is 1 (or 255), otherwise it is 0.

[0041] Specifically, the binary frame difference image corresponding to the k-th frame image and the k+1-th frame image, and the binary frame difference image corresponding to the k+1-th frame image and the k+2-th frame image may be subjected to an image AND operation according to the following formula to obtain a symmetric differential binary image corresponding to the k+1-th frame image; ; in, Represents the binary value corresponding to the pixel coordinate point (x, y) in the symmetric difference binary image corresponding to the k+1 frame image. Indicates the binary value corresponding to the pixel coordinate point (x, y) in the binary frame difference image corresponding to the k-th frame image and the k+1-th frame image. Indicates the binarized value corresponding to the pixel coordinate point with pixel coordinate (x, y) in the binarized frame difference image corresponding to the k+1th frame image and the k+2th frame image.

[0042] In one or more embodiments, after performing an image AND operation on the binary frame difference image corresponding to the kth frame image and the k+1th frame image, and the binary frame difference image corresponding to the +1th frame image and the k+2th frame image to obtain a symmetric difference binary image corresponding to the k+1th frame image, erosion and dilation processing may be performed on the symmetric difference binary images corresponding to each frame image. The erosion and dilation processing may eliminate some smaller noise points in the symmetric difference binary image.

[0043] Step S108 . Performing connected region analysis on the symmetric difference binary image corresponding to each frame image in time sequence, and eliminating connected regions with an area smaller than a preset area to obtain a connected region analysis image corresponding to each frame image.

[0044] Specifically, when performing the analysis, the connected region analysis is first performed on the symmetric difference binary image corresponding to the second frame image, and then the connected region analysis is performed on the symmetric difference binary image corresponding to the third frame image, and then the connected region analysis is performed on the symmetric difference binary image corresponding to the fourth frame image, and so on.

[0045] Step S109: If there is no connected region in the connected region analysis image corresponding to a certain frame image, the certain frame image is updated as the background image.

[0046] If there is no connected region in the connected region analysis image corresponding to a certain frame image, the certain frame image is directly updated as the current latest background image.

[0047] Step S110. If a connected area exists in the connected area analysis image corresponding to a certain frame image, the pixel area corresponding to the connected area in the certain frame image is marked as a segmentation target, and the background of the remaining areas other than the pixel area corresponding to the connected area in the certain frame image is updated.

[0048] If a connected region exists in the connected region analysis image corresponding to a particular frame, the pixel region corresponding to the connected region in that frame is marked as a segmentation target and annotated with a circumscribed rectangular frame. Simultaneously, the background of the remaining pixel regions outside the connected region in that frame is updated. This allows for real-time segmentation and extraction of moving objects in a video, as well as real-time background updates.

[0049] In summary, the video target segmentation method provided by the present invention obtains an image frame sequence by performing frame processing on the video stream to be segmented; calculates the grayscale mean and grayscale mean square error value corresponding to each pixel coordinate point in the target image of the first n frames of the image frame sequence; determines the grayscale error standard value based on the grayscale mean square error value, and selects all target pixel coordinate points from each target image whose absolute value of error between the corresponding grayscale value and the grayscale mean is lower than the grayscale error standard value; constructs an image background based on the grayscale values ​​corresponding to all the determined target pixel coordinate points to obtain a background image; calculates the absolute grayscale difference image between all adjacent frame images in the image frame sequence; binarizes the absolute grayscale difference image between each adjacent frame image to obtain a binarized frame difference image corresponding to each adjacent frame image; compares the image corresponding to the kth frame and the image corresponding to the k+1th frame. The binarized frame difference image is subjected to an image AND operation with the binarized frame difference images corresponding to the k+1th frame image and the k+2th frame image to obtain a symmetric differential binary image corresponding to the k+1th frame image, wherein k represents a positive integer; a connected region analysis is performed on the symmetric differential binary images corresponding to each frame image in time sequence, and connected regions with an area smaller than a preset area are eliminated to obtain a connected region analysis image corresponding to each frame image; if a connected region does not exist in the connected region analysis image corresponding to a certain frame image, the certain frame image is updated as a background image; if a connected region exists in the connected region analysis image corresponding to a certain frame image, the pixel region corresponding to the connected region in the certain frame image is marked as a segmentation target, and the background update is performed on the remaining regions other than the pixel region corresponding to the connected region in the certain frame image. In this way, the background subtraction algorithm (frame difference image image and operation) can be used to fill the holes caused by the inter-frame difference calculation. When segmenting video targets, the edge contour provided by the inter-frame difference method and the target internal pixel information provided by the background subtraction algorithm jointly participate in the moving target segmentation, avoiding inaccurate target segmentation due to the occurrence of holes, improving the moving target segmentation accuracy, improving the target segmentation quality, and facilitating practical application and promotion.

[0050] See also Figure 2 A second aspect of an embodiment of the present application provides a video object segmentation device, comprising: A framing unit, used for performing frame processing on the video stream to be segmented to obtain an image frame sequence; The first calculation unit is used to calculate the grayscale mean and grayscale mean square error corresponding to each pixel coordinate point in the target image of the first n frames of the image frame sequence; a determination and selection unit, configured to determine a grayscale error standard value based on the grayscale mean square error value, and select from each target image all target pixel coordinate points whose absolute value of the error between the corresponding grayscale value and the grayscale mean is lower than the grayscale error standard value; A background construction unit is used to construct an image background based on the grayscale values ​​corresponding to all the determined target pixel coordinate points to obtain a background image; A second calculation unit is used to calculate an absolute grayscale difference image between all adjacent frame images in the image frame sequence; A binarization unit is used to perform binarization processing on the absolute grayscale difference image between each adjacent frame image to obtain a binarized frame difference image corresponding to each adjacent frame image; an image AND operation unit, configured to perform an image AND operation on a binary frame difference image corresponding to the k-th frame image and the k+1-th frame image, and a binary frame difference image corresponding to the k+1-th frame image and the k+2-th frame image, to obtain a symmetric differential binary image corresponding to the k+1-th frame image, wherein k represents a positive integer; A connected region analysis unit is used to perform connected region analysis on the symmetric difference binary image corresponding to each frame image in time sequence, and eliminate connected regions with an area smaller than a preset area to obtain a connected region analysis image corresponding to each frame image; a marking and background updating unit, configured to update a frame image as a background image if no connected region exists in the connected region analysis image corresponding to the frame image; and If a connected area exists in the connected area analysis image corresponding to a certain frame image, the pixel area corresponding to the connected area in the certain frame image is marked as a segmentation target, and the background of the remaining areas outside the pixel area corresponding to the connected area in the certain frame image is updated.

[0051] The working process, working details and technical effects of the video object segmentation device provided in the second aspect of this embodiment can be found in the first aspect of the embodiment and will not be described in detail here.

[0052] like Figure 3 As shown, the third aspect of an embodiment of the present application provides an electronic device, comprising a memory, a processor and a transceiver that are communicatively connected in sequence, wherein the memory is used to store computer programs, the transceiver is used to send and receive messages, and the processor is used to read the computer program and execute the video target segmentation method as described in the first aspect of the embodiment.

[0053] For example, the memory may include, but is not limited to, random access memory (RAM), read-only memory (ROM), flash memory, first-in-first-out memory (FIFO) and / or first-in-last-out memory (FILO), etc.; the processor may include, but is not limited to, a microprocessor of the STM32F105 series, an ARM (Advanced RISC Machines), an X86 or other architecture processor, or a processor with an integrated NPU (neural-network processing units); the transceiver may include, but is not limited to, a WiFi (Wireless Fidelity) wireless transceiver, a Bluetooth wireless transceiver, a General Packet Radio Service (GPRS) wireless transceiver, a ZigBee protocol (a low-power local area network protocol based on the IEEE802.15.4 standard, ZigBee) wireless transceiver, a 3G transceiver, a 4G transceiver and / or a 5G transceiver, etc.

[0054] A fourth aspect of this embodiment provides a computer-readable storage medium storing instructions containing the video object segmentation method described in the first aspect of the embodiment. Specifically, the computer-readable storage medium stores instructions that, when executed on a computer, execute the video object segmentation method described in the first aspect. The computer-readable storage medium refers to a data storage medium and may include, but is not limited to, a floppy disk, a CD, a hard disk, a flash memory, a USB flash drive, and / or a memory stick. The computer may be a general-purpose computer, a dedicated computer, a computer network, or other programmable device.

[0055] A fifth aspect of this embodiment provides a computer program product comprising instructions, which, when executed on a computer, causes the computer to execute the video target segmentation method as described in the first aspect of the embodiment, wherein the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0056] It should be understood that certain details are provided in the following description to facilitate a thorough understanding of the example embodiments. However, one of ordinary skill in the art will appreciate that the example embodiments can be practiced without these specific details. For example, a system may be shown in block diagrams to avoid obscuring the example with unnecessary detail. In other instances, well-known processes, structures, and techniques may be shown without unnecessary detail to avoid obscuring the example embodiments.

[0057] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the scope of protection of the present invention.

Claims

1. A video object segmentation method, characterized in that: include: Perform frame processing on the video stream to be segmented to obtain an image frame sequence; Calculating the grayscale mean and grayscale mean square error corresponding to each pixel coordinate point in the target image of the first n frames of the image frame sequence; Determine a grayscale error standard value based on the grayscale mean square error value, and select all target pixel coordinate points from each target image whose absolute value of the error between the corresponding grayscale value and the grayscale mean is lower than the grayscale error standard value; The image background is constructed based on the grayscale values ​​corresponding to all the determined target pixel coordinates to obtain a background image; Calculate the absolute grayscale difference image between all adjacent frame images in the image frame sequence; Binarizing the absolute grayscale difference images between adjacent frame images to obtain binary frame difference images corresponding to the adjacent frame images; Perform an image AND operation on the binary frame difference image corresponding to the k-th frame image and the k+1-th frame image, and the binary frame difference image corresponding to the k+1-th frame image and the k+2-th frame image, to obtain a symmetric differential binary image corresponding to the k+1-th frame image, where k represents a positive integer; Performing connected region analysis on the symmetric difference binary image corresponding to each frame image in time sequence, and eliminating connected regions with an area smaller than a preset area, to obtain a connected region analysis image corresponding to each frame image; If there is no connected region in the connected region analysis image corresponding to a certain frame image, updating the certain frame image as the background image; If a connected area exists in the connected area analysis image corresponding to a certain frame image, the pixel area corresponding to the connected area in the certain frame image is marked as a segmentation target, and the background of the remaining areas outside the pixel area corresponding to the connected area in the certain frame image is updated.

2. The video target segmentation method according to claim 1, characterized in that: The image background is constructed based on the grayscale values ​​corresponding to all the determined target pixel coordinates, including: Based on the grayscale values ​​corresponding to all the determined target pixel coordinates, the image background is constructed according to the following formula: ； Where m represents the total number of target pixel coordinate points with pixel coordinates (x, y). Indicates the grayscale value corresponding to the i-th target pixel coordinate point with pixel coordinates (x, y).

3. The video target segmentation method according to claim 1, characterized in that: The absolute grayscale difference image between adjacent frame images is ,in Indicates the grayscale value corresponding to the pixel coordinate point with pixel coordinates (x, y) in the k-th frame image. Indicates the grayscale value corresponding to the pixel coordinate point with pixel coordinates (x, y) in the k-1th frame image.

4. The video object segmentation method according to claim 1, characterized in that: After calculating the absolute grayscale difference images between all adjacent frame images in the image frame sequence, the method further includes: Performing filtering on the absolute grayscale difference images between adjacent frame images; The binarization process is performed on the absolute grayscale difference image between each adjacent frame image to obtain the binarized frame difference image corresponding to each adjacent frame image, including: The absolute grayscale difference image between each adjacent frame image after filtering is binarized to obtain a binarized frame difference image corresponding to each adjacent frame image.

5. The video object segmentation method according to claim 1, characterized in that: After performing an image AND operation on the binary frame difference image corresponding to the kth frame image and the k+1th frame image, and the binary frame difference image corresponding to the +1th frame image and the k+2th frame image, to obtain a symmetric differential binary image corresponding to the k+1th frame image, the method further includes: The symmetric difference binary image corresponding to each frame image is corroded and expanded.

6. The video object segmentation method according to claim 1, characterized in that: Performing an image AND operation on the binary frame difference image corresponding to the k-th frame image and the k+1-th frame image, and the binary frame difference image corresponding to the k+1-th frame image and the k+2-th frame image, to obtain a symmetric differential binary image corresponding to the k+1-th frame image, including: According to the following formula, perform an image AND operation on the binary frame difference image corresponding to the k-th frame image and the k+1-th frame image, and the binary frame difference image corresponding to the k+1-th frame image and the k+2-th frame image to obtain a symmetric differential binary image corresponding to the k+1-th frame image; ； in, Represents the binary value corresponding to the pixel coordinate point (x, y) in the symmetric difference binary image corresponding to the k+1 frame image. Indicates the binary value corresponding to the pixel coordinate point (x, y) in the binary frame difference image corresponding to the k-th frame image and the k+1-th frame image. Indicates the binarized value corresponding to the pixel coordinate point with pixel coordinate (x, y) in the binarized frame difference image corresponding to the k+1th frame image and the k+2th frame image.

7. A video object segmentation device, characterized in that: include: A framing unit, used for performing frame processing on the video stream to be segmented to obtain an image frame sequence; The first calculation unit is used to calculate the grayscale mean and grayscale mean square error corresponding to each pixel coordinate point in the target image of the first n frames of the image frame sequence; a determination and selection unit, configured to determine a grayscale error standard value based on the grayscale mean square error value, and select from each target image all target pixel coordinate points whose absolute value of the error between the corresponding grayscale value and the grayscale mean is lower than the grayscale error standard value; A background construction unit is used to construct an image background based on the grayscale values ​​corresponding to all the determined target pixel coordinate points to obtain a background image; A second calculation unit is used to calculate an absolute grayscale difference image between all adjacent frame images in the image frame sequence; A binarization unit is used to perform binarization processing on the absolute grayscale difference image between each adjacent frame image to obtain a binarized frame difference image corresponding to each adjacent frame image; an image AND operation unit, configured to perform an image AND operation on a binary frame difference image corresponding to the k-th frame image and the k+1-th frame image, and a binary frame difference image corresponding to the k+1-th frame image and the k+2-th frame image, to obtain a symmetric differential binary image corresponding to the k+1-th frame image, wherein k represents a positive integer; A connected region analysis unit is used to perform connected region analysis on the symmetric difference binary image corresponding to each frame image in time sequence, and eliminate connected regions with an area smaller than a preset area to obtain a connected region analysis image corresponding to each frame image; a marking and background updating unit, configured to update a frame image as a background image if no connected region exists in the connected region analysis image corresponding to the frame image; as well as If a connected area exists in the connected area analysis image corresponding to a certain frame image, the pixel area corresponding to the connected area in the certain frame image is marked as a segmentation target, and the background of the remaining areas outside the pixel area corresponding to the connected area in the certain frame image is updated.

8. An electronic device, characterized in that: The method comprises a memory, a processor and a transceiver which are communicatively connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program and execute the video target segmentation method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the video object segmentation method according to any one of claims 1 to 6 is executed.

10. A computer program product comprising a computer program or instructions, characterized in that When the computer program or the instruction is executed by a computer, the video object segmentation method according to any one of claims 1 to 6 is implemented.

Images