Online Video Foreground Extraction Method and Device

Abstract

The present application relates to an online video foreground extraction method and apparatus, including: processing the current video frame of the online video by using the method of low-rank matrix decomposition to determine the noise of the current video frame; for each pixel point, constructing a hidden Markov model based on the noise of the pixel point and the hidden variable corresponding to the noise of the pixel point; solving the hidden Markov model based on the input parameters to obtain output parameters, where the output parameters include the variances of the mixture Gaussian distributions corresponding to the noises of each pixel point of the current video frame; determining whether each pixel point of the current video frame is a foreground pixel point according to the variances of the mixture Gaussian distributions corresponding to the noises of each pixel point of the current video frame. The online video foreground extraction method of the present application expresses the whole process as a probability model under the assumption of dependent errors, constructs a corresponding algorithm for iterative solution, which can not only meet the requirements of real-time processing, but also can extract the foreground better.

Description

Technical Field

[0001] This application relates to the field of computer vision technology. Specifically, it relates to an online video foreground extraction method and device. Background Art

[0002] Extracting foreground objects from videos is affected by complex background scenes. For example, in the case of changes in conditions such as weather and lighting, the background in surveillance videos changes dynamically, making foreground extraction and background modeling very difficult.

[0003] In existing online methods for video surveillance foreground extraction, background subtraction, as a basic method for extracting foregrounds in video processing, is a prerequisite and foundation for target detection and recognition. Background subtraction generally can extract the background image only when there are moving objects. This requirement can make the background update adapt to changes in different situations of the scene, such as changes in environmental light, movement of fixed objects, and the influence of shadows. However, the disadvantage of such methods is that they ignore useful video structure knowledge, such as the temporal similarity of the background scene and the spatial continuity of foreground objects. Therefore, in practical applications, good performance cannot always be guaranteed. Additionally, the low-rank matrix factorization method of the mixture Gaussian noise model can better simulate the noise distribution, but there is an unreasonable assumption of independence between adjacent frames. Summary of the Invention

[0004] To overcome at least one deficiency in the prior art, embodiments of this application provide an online video foreground extraction method and device.

[0005] In a first aspect, an online video foreground extraction method is provided, including:

[0006] Processing the current video frame of the online video using the low-rank matrix factorization method to determine the noise of the current video frame; the noise of the current video frame includes the noise of each pixel point;

[0007] For each pixel point, constructing a hidden Markov model based on the noise of the pixel point and the hidden variable corresponding to the noise of the pixel point; the noise of the pixel point follows a mixture Gaussian distribution, and the input parameters of the hidden Markov model include the variance of the mixture Gaussian distribution corresponding to the noise of the pixel point in the previous video frame, the mixing coefficient of the mixture Gaussian distribution, the coefficient matrix vector of the background subspace, the basis matrix vector of the background subspace, the transition probability, and two intermediate parameters for generating the basis matrix vector of the background subspace;

[0008] Solving the hidden Markov model based on the input parameters to obtain output parameters, where the output parameters include the variance of the mixture Gaussian distribution corresponding to the noise of each pixel point in the current video frame;

[0009] Determine whether each pixel of the current video frame is a foreground pixel according to the variance of the Gaussian mixture distribution corresponding to the noise of each pixel of the current video frame;

[0010] Obtain all foreground pixels to form the foreground of the current video frame.

[0011] In one embodiment, the output parameters further include: the mixing coefficients of the Gaussian mixture distribution corresponding to the noise of each pixel of the current video frame, the coefficient matrix vector of the background subspace, the basis matrix vector of the background subspace, the transition probability, and two intermediate parameters for generating the basis matrix vector of the background subspace;

[0012] The output parameters serve as the input parameters of the hidden Markov model corresponding to the pixels of the next video frame.

[0013] In one embodiment, solve the hidden Markov model based on the input parameters to obtain output parameters, including:

[0014] Solve the hidden Markov model using the E-M method to obtain output parameters.

[0015] In one embodiment, the two intermediate parameters for generating the basis matrix vector of the background subspace are calculated using the following formula:

[0016]

[0017]

[0018] Where, is the first intermediate parameter of the basis matrix vector of the background subspace corresponding to the noise of the i-th pixel of the current video frame t, is the first intermediate parameter of the basis matrix vector of the background subspace corresponding to the noise of the i-th pixel of the previous video frame t-1, is the second intermediate parameter of the basis matrix vector of the background subspace corresponding to the noise of the i-th pixel of the current video frame t, is the second intermediate parameter of the basis matrix vector of the background subspace corresponding to the noise of the i-th pixel of the previous video frame t-1, ρ is the parameter coefficient, is the Gaussian distribution weight value of the noise of the i-th pixel of the current video frame t, is the coefficient matrix vector of the background subspace corresponding to the noise of the i-th pixel of the current video frame t, is the noise value of the i-th pixel of the current video frame t.

[0019] In one embodiment, determining whether each pixel of the current video frame is a foreground pixel according to the variance of the Gaussian mixture distribution corresponding to the noise of each pixel of the current video frame includes:

[0020] The Gaussian mixture distribution is a third-order Gaussian mixture distribution, including Gaussian distributions in three states. The variances of the Gaussian mixture distribution include the variances of the Gaussian distributions in three states. The Gaussian distributions in the three states correspond to foreground, camera noise, and shadow respectively.

[0021] For each pixel, determine the maximum value of the variances of the Gaussian distributions in three states corresponding to the noise of the pixel.

[0022] If the Gaussian distribution corresponding to the maximum variance is the Gaussian distribution corresponding to the foreground, then the pixel is a foreground pixel; otherwise, the pixel is not a foreground pixel.

[0023] In one embodiment, if the current frame is the first frame, the input parameters are initial parameters, which include: the initial variance of the Gaussian mixture distribution, the initial mixing coefficients of the Gaussian mixture distribution, the initial coefficient matrix vector of the background subspace, the initial basis matrix vector of the background subspace, the initial transition probability, and two initial intermediate parameters for generating the basis matrix vector of the background subspace.

[0024] In one embodiment, obtain multiple video frames, and use the principal component analysis method for the multiple video frames to determine the initial parameters.

[0025] In a second aspect, there is provided an online video foreground extraction device, including:

[0026] A noise determination module, configured to process the current video frame of the online video by using the method of low-rank matrix decomposition to determine the noise of the current video frame; the noise of the current video frame includes the noise of each pixel.

[0027] A model construction module, configured to construct a hidden Markov model for each pixel based on the noise of the pixel and the hidden variable corresponding to the noise of the pixel; the input parameters of the hidden Markov model include the variance of the Gaussian mixture distribution corresponding to the noise of the pixel in the previous video frame, the mixing coefficients of the Gaussian mixture distribution, the coefficient matrix vector of the background subspace, the basis matrix vector of the background subspace, the transition probability, and two intermediate parameters for generating the basis matrix vector of the background subspace.

[0028] A model solving module, configured to solve the hidden Markov model based on the input parameters to obtain output parameters, where the output parameters include the variance of the Gaussian mixture distribution corresponding to the noise of each pixel in the current video frame.

[0029] A foreground pixel determination module, configured to determine whether each pixel in the current video frame is a foreground pixel according to the variance of the Gaussian mixture distribution corresponding to the noise of each pixel in the current video frame.

[0030] A foreground extraction module, configured to obtain all foreground pixels to form the foreground of the current video frame.

[0031] In a third aspect, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the above-mentioned online video foreground extraction method is implemented.

[0032] In a fourth aspect, a computer program product is provided, including a computer program / instructions. When the computer program / instructions are executed by a processor, the above-mentioned online video foreground extraction method is implemented.

[0033] Compared with the prior art, the present application has the following beneficial effects: After descriptive analysis of actual video data, combined with the characteristics of the hidden Markov model, an online algorithm for foreground extraction based on the hidden Markov model is constructed. It can not only meet the requirements of real-time processing, but also extract the foreground better, preparing for the next step of target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The present application can be better understood by referring to the description given below in conjunction with the accompanying drawings. The drawings, together with the following detailed description, are included in this specification and form a part of this specification. In the drawings:

[0035] Figure 1 A flowchart showing the online video foreground extraction method according to an embodiment of the present application is shown;

[0036] Figure 2 A schematic diagram showing the hidden Markov model according to an embodiment of the present application is shown;

[0037] Figure 3 A block diagram showing the structure of the online video foreground extraction device according to an embodiment of the present application is shown;

[0038] Figure 4 A foreground extraction result diagram obtained by using the online video foreground extraction method according to an embodiment of the present application is shown

[0039] Figure 5 A comparison diagram of foreground extraction results between the online video foreground extraction method according to an embodiment of the present application and the prior method is shown. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0040] Hereinafter, exemplary embodiments of the present application will be described in conjunction with the accompanying drawings. For clarity and conciseness, not all features of the actual embodiments are described in the specification. However, it should be understood that many specific decisions specific to the embodiments can be made during the development of any such actual embodiment to achieve the specific goals of the developer, and these decisions may vary with different embodiments.

[0041] Here, it should also be noted that in order to avoid obscuring the present application due to unnecessary details, only the device structures closely related to the solution according to the present application are shown in the accompanying drawings, while other details less relevant to the present application are omitted.

[0042] It should be understood that the present application is not limited to the described embodiments only due to the following description with reference to the accompanying drawings. In this article, where feasible, embodiments can be combined with each other, features can be replaced or borrowed between different embodiments, and one or more features can be omitted in one embodiment.

[0043] The present application relates to an online video foreground extraction method and device, which uses a dependent mixture of Gaussians to model the foreground of each frame of the video, determines the relationship between frames as a dependent structure through descriptive data analysis, and uses a hidden Markov process to model the entire video sequence to complete the iterative update of the foreground. Different from the previous assumption that the errors of adjacent frames are independent background low-rank structures, under the assumption of dependent errors, the entire process is expressed as a probability model, and the corresponding algorithm is constructed for iterative solution.

[0044] Figure 1 The flowchart of the online video foreground extraction method according to an embodiment of the present application is shown, and the method includes:

[0045] Step S110, process the current video frame t of the online video using the method of low-rank matrix decomposition to obtain the noise of the current video frame t; the noise of the current video frame t includes the noise of each pixel point; in this step, low-rank matrix decomposition is one of the most commonly used subspace learning methods in background subtraction, and its main idea is to extract the low-rank approximation of the data matrix from the product of two smaller matrices, the basis matrix and the coefficient matrix; by performing low-rank matrix decomposition on the pixel values X of the current video frame, the basis matrix U and the coefficient matrix V of the background subspace are obtained, and the noise can be obtained using X - UV T The i-th row vector of the basis matrix U of the background subspace corresponds to the i-th pixel point, denoted as the basis matrix vector u of the background subspace i , and the i-th pixel point of the coefficient matrix V of the background subspace, denoted as the coefficient matrix vector v of the background subspace i .

[0046] Step S120, for each pixel point i, i = 1, 2,..., d, where d is the number of pixel points in the video frame, construct a hidden Markov model based on the noise of pixel point i and the hidden variable corresponding to the noise of pixel point i; the noise of the pixel point follows a mixture of Gaussians distribution, and the input parameters of the hidden Markov model include the variance of the mixture of Gaussians distribution corresponding to the noise of pixel point i in the previous video frame t - 1 The mixing coefficient of the mixture of Gaussians distribution The basis matrix vector of the background subspace Coefficient matrix vector of the background subspace Transition probability And the basis matrix vector for generating the background subspace Two intermediate parameters And where t ≥ 2 Figure 2 FIG. shows a schematic diagram of a hidden Markov model according to an embodiment of the present application. In the figure, x 1 , x 2 , …, x T respectively represent the noises of the same pixel point in each of the T video frames, which is an explicit sequence. z 1 , z 2 , …, z T respectively represent the hidden variable sequences corresponding to x 1 , x 2 , …, x T ; In this step, the noise of each pixel point follows a third-order mixture Gaussian distribution. The value of the hidden variable corresponding to the pixel point determines which Gaussian distribution the noise of the pixel point specifically follows. There is a dependency relationship between adjacent video frames, that is, there is a transition probability from the hidden variable of the pixel point in the previous video frame to the hidden variable of the pixel point in the next video frame.

[0047] Step S130, solving the hidden Markov model based on the input parameters to obtain output parameters. The output parameters include the variance of the mixture Gaussian distribution corresponding to the noise of each pixel point i in the current video frame t In this step, the noise of each pixel point follows a third-order mixture Gaussian distribution. The variance obtained here includes the variances of the Gaussian distributions in three states.

[0048] Step S140, determining whether each pixel point i in the current video frame t is a foreground pixel point according to the variance of the mixture Gaussian distribution corresponding to the noise of each pixel point i in the current video frame t ;

[0049] Step S150, obtaining all foreground pixel points to form the foreground of the current video frame.

[0050] In this embodiment, if the current frame is the first frame, the input parameters are initial parameters, and the initial parameters include: the initial variance of the mixture Gaussian distribution The initial mixing coefficient of the mixture Gaussian distribution The initial coefficient matrix vector of the background subspace The initial basis matrix vector of the background subspace The initial transition probability And two initial intermediate parameters of the basis matrix vector for generating the background subspace And Here, 30 video frames can be selected, and the principal component analysis method is used for the 30 video frames to determine the above initial parameters.

[0051] In one embodiment, the output parameters further include: the mixing coefficients of the Gaussian mixture distribution corresponding to the noise of each pixel point i in the current video frame t The basis matrix vector of the background subspace The coefficient matrix vector of the background subspace Transition probability And two intermediate parameters for generating the basis matrix vector of the background subspace And The output parameters are used as the input parameters of the hidden Markov model corresponding to the pixel point i in the next video frame t + 1.

[0052] In one embodiment, the E-M method can be used to solve the hidden Markov model to obtain the output parameters. Specifically, the two intermediate parameters for generating the basis matrix vector of the background subspace can be calculated using the following formula:

[0053]

[0054]

[0055] Among them, is the first intermediate parameter of the basis matrix vector of the background subspace corresponding to the noise of the i-th pixel point in the current video frame t, is the first intermediate parameter of the basis matrix vector of the background subspace corresponding to the noise of the i-th pixel point in the previous video frame t - 1, is the second intermediate parameter of the basis matrix vector of the background subspace corresponding to the noise of the i-th pixel point in the current video frame t, is the second intermediate parameter of the basis matrix vector of the background subspace corresponding to the noise of the i-th pixel point in the previous video frame t - 1, ρ is a parameter coefficient, taking an empirical value, represents the Gaussian distribution weight value of the noise of the i-th pixel point in the current video frame t, is the natural conjugate prior value of the k-th Gaussian distribution, K is the number of Gaussian distributions, is the estimated value of the posterior probability expectation corresponding to the k-th Gaussian distribution corresponding to the noise of the i-th pixel point in the current video frame t, that is is the hidden variable corresponding to the k-th Gaussian distribution corresponding to the noise of the i-th pixel point in the current video frame t, is the coefficient matrix vector of the background subspace corresponding to the noise of the i-th pixel point in the current video frame t, is the noise value of the i-th pixel point in the current video frame t.

[0056] In one embodiment, determining whether each pixel point of the current video frame is a foreground pixel point according to the variance of the Gaussian mixture distribution corresponding to the noise of each pixel point of the current video frame may include:

[0057] The Gaussian mixture distribution is a third-order Gaussian mixture distribution, including Gaussian distributions in three states. The variance of the Gaussian mixture distribution includes the variances of the Gaussian distributions in three states. The Gaussian distributions in the three states respectively correspond to foreground, camera noise, and shadow. Among them, it can be set that the Gaussian distribution corresponding to k = 3 corresponds to the foreground.

[0058] For each pixel point, determine the maximum value of the variances of the Gaussian distributions in the three states corresponding to the noise of the pixel point;

[0059] If the Gaussian distribution corresponding to the maximum value of the variance is the Gaussian distribution corresponding to the foreground, then the pixel point is a foreground pixel point; otherwise, the pixel point is not a foreground pixel point.

[0060] Based on the same inventive concept as the online video foreground extraction method provided in the embodiments of the present application, Figure 3 An online video foreground extraction device provided in the embodiments of the present application is shown. The device includes:

[0061] A noise determination module 310, configured to process the current video frame of the online video by using a low-rank matrix decomposition method to determine the noise of the current video frame. The noise of the current video frame includes the noise of each pixel point;

[0062] A model construction module 320, configured to construct a hidden Markov model for each pixel point based on the noise of the pixel point and the hidden variable corresponding to the noise of the pixel point. The input parameters of the hidden Markov model include the variance of the Gaussian mixture distribution corresponding to the noise of the pixel point in the previous video frame, the mixing coefficient of the Gaussian mixture distribution, the coefficient matrix vector of the background subspace, the basis matrix vector of the background subspace, the transition probability, and two intermediate parameters for generating the basis matrix vector of the background subspace;

[0063] A model solution module 330, configured to solve the hidden Markov model based on the input parameters to obtain output parameters. The output parameters include the variance of the Gaussian mixture distribution corresponding to the noise of each pixel point of the current video frame;

[0064] A foreground pixel point determination module 340, configured to determine whether each pixel point of the current video frame is a foreground pixel point according to the variance of the Gaussian mixture distribution corresponding to the noise of each pixel point of the current video frame;

[0065] A foreground extraction module 350, configured to obtain all foreground pixel points to form the foreground of the current video frame.

[0066] In the embodiments of this application, a dependent mixture of Gaussians is used to model the foreground of each frame of the video. By descriptive data analysis, the relationship between frames is determined to be a dependent structure, and a hidden Markov process is used to model the entire video sequence to complete the iterative update of the foreground. Different from the previous assumption that the errors of adjacent frames are independent background low-rank structures, under the assumption of dependent errors, the entire process is expressed as a probability model, and a corresponding algorithm is constructed for iterative solution.

[0067] The embodiments of this application provide a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above online video foreground extraction method is implemented.

[0068] The embodiments of this application provide a computer program product, including a computer program / instructions. When the computer program / instructions are executed by a processor, the above online video foreground extraction method is implemented.

[0069] The following conducts an experimental analysis on the technical effects of the online video foreground extraction method of this application. By comparing with the low-rank matrix factorization algorithm based on Gaussian mixture distribution (MOG-LRMF), the classic Gaussian mixture algorithm (MOG), the particle swarm optimization algorithm based on Gaussian mixture (MOG-PSO), the subspace learning algorithm based on principal component analysis (SL-PCA), and the subspace learning algorithm based on independent component analysis (SL-ICA) on a public dataset, it can be found that the method of this application can effectively and quickly perform online extraction of the video foreground. For static videos or dynamic videos with slight shaking, based on the dependent low-rank structure of the video sequence, an iterative algorithm can be used to quickly perform online processing and effectively separate the foreground and background of the video.

[0070] Apply the method of this application to the office scene dataset, and select the foreground extraction image output by the 607th frame experiment. Figure 4 The foreground extraction result graph obtained by using the online video foreground extraction method of the embodiments of this application is shown. See Figure 4 , from left to right, from top to bottom are the original image, background, noise, shadow, camera noise, and foreground. The output results show that the method of this application can preferably achieve online separation of the video foreground and background, with clear separation of the background and good separation effect of the foreground.

[0071] For the 607th frame of the office scene in the CDnet2014 dataset, compare the method of this application with the independent Gaussian mixture distribution (GMM) method. As can be seen from Table 1, the method of this application has a higher F value (F-measure) and better foreground extraction effect for the video.

[0072] Table 1

[0073]

[0074] Experiments were conducted on the surveillance videos of restaurants in the Wallflower dataset. The video consists of 299 frames of images, and each image has a size of 160×120. Many typical foreground-background separation methods were compared with the method of this application. The typical methods include: the low-rank matrix factorization algorithm based on Gaussian mixture distribution (MOG-LRMF), the classical Gaussian mixture algorithm (MOG), the particle swarm optimization algorithm based on Gaussian mixture (MOG-PSO), the subspace learning algorithm based on principal component analysis (SL-PCA), and the subspace learning algorithm based on independent component analysis (SL-ICA).

[0075] The experimental results of different algorithms were compared with the foreground ground truth of the 299th frame of the dataset. Figure 5 The figure shows the comparison chart of the foreground extraction results between the online video foreground extraction method of the embodiment of this application and the existing methods. Compared with the real foreground, the boundary segmented by the method of this application (LR-HMM) is clearer and does not extract too much information. The MOG-LRMF method and the MOG-PSO method have the problem of incomplete extraction of human edge information. The method with the worst foreground separation effect is the SL-PCA algorithm, where there are a large number of background pixels in the foreground pixel area, extracting too much information, while the SL-ICA method is not sensitive to foreground information, resulting in too little image information being segmented.

[0076] Table 2 gives the metrics FN (False Negative, predicting the real background as background), FP (False Positive, predicting the real background as foreground), TN (True Negative, predicting the real foreground as background), TP (True Positive, predicting the real foreground as foreground), as well as the calculated values of precision P, recall R, and F value, enabling numerical comparison between algorithms. The FN value of the SL-PCA algorithm is the smallest, but the FP value is significantly larger than other methods, which means it predicts too much actual background information as foreground information. Compared with the MOG-LRMF algorithm, the method of this application has a higher TN value and a lower FP value, and the precision is 74.8%, indicating that the proportion of real foreground in the results predicted as foreground is much higher than that of MOG-LRMF, and the actual effect of foreground extraction is better.

[0077] By comparing the F values of different algorithms, the metric results of the method of this application are not much different numerically from those of classical methods such as MOG-LRMF and MOG, and it performs better in terms of metrics FP, precision, and TN, indicating that the method of this application can effectively segment the foreground and background.

[0078] Table 2

[0079]

[0080] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are only illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of devices, methods, and computer program products according to multiple embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of the code, and the module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0081] In addition, in each embodiment of the present application, the functional modules can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part.

[0082] If the above functions are implemented in the form of software functional modules and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical disks, etc., which can store program codes.

[0083] The above is only various embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed in the present application can easily think of changes or substitutions, which should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for extracting foreground from online videos, characterized in that: include: A low-rank matrix decomposition method is used to process the current video frame of the online video to determine the noise of the current video frame; The noise of the current video frame includes the noise of each pixel; For each pixel, a hidden Markov model is constructed based on the noise of the pixel and the hidden variable corresponding to the noise of the pixel; the noise of the pixel obeys a mixed Gaussian distribution, and the input parameters of the hidden Markov model include the variance of the mixed Gaussian distribution corresponding to the noise of the pixel in the previous video frame, the mixing coefficient of the mixed Gaussian distribution, the coefficient matrix vector of the background subspace, the basis matrix vector of the background subspace, the transition probability, and two intermediate parameters for generating the basis matrix vector of the background subspace; Solving the hidden Markov model based on the input parameters to obtain output parameters, wherein the output parameters include the variance of the mixed Gaussian distribution corresponding to the noise of each pixel point of the current video frame; determining whether each pixel point of the current video frame is a foreground pixel point according to the variance of the mixed Gaussian distribution corresponding to the noise of each pixel point of the current video frame; All the foreground pixels are obtained to form the foreground of the current video frame.

2. The online video foreground extraction method according to claim 1, wherein: The output parameters also include: a mixing coefficient of a mixed Gaussian distribution corresponding to the noise of each pixel point of the current video frame, a coefficient matrix vector of the background subspace, a base matrix vector of the background subspace, a transition probability, and two intermediate parameters for generating a base matrix vector of the background subspace; The output parameters are used as input parameters of the hidden Markov model corresponding to the pixel points of the next video frame.

3. The method according to claim 1, wherein Solving the hidden Markov model based on the input parameters to obtain output parameters includes: The hidden Markov model is solved using the EM method to obtain the output parameters.

4. The method according to claim 2, wherein The two intermediate parameters of the basis matrix vector for generating the background subspace are calculated using the following formula: in, is the first intermediate parameter of the basis matrix vector of the background subspace corresponding to the noise of the i-th pixel in the current video frame t, is the first intermediate parameter of the basis matrix vector of the background subspace corresponding to the noise of the i-th pixel in the previous video frame t-1, is the second intermediate parameter of the basis matrix vector of the background subspace corresponding to the noise of the i-th pixel of the current video frame t, is the second intermediate parameter of the basis matrix vector of the background subspace corresponding to the noise of the i-th pixel in the previous video frame t-1, ρ is the parameter coefficient, is the Gaussian distribution weight value of the noise of the i-th pixel in the current video frame t, is the coefficient matrix vector of the background subspace corresponding to the noise of the i-th pixel in the current video frame t, is the noise value of the i-th pixel in the current video frame t.

5. The method according to claim 1, wherein The determining, based on the variance of the mixed Gaussian distribution corresponding to the noise of each pixel of the current video frame, whether each pixel of the current video frame is a foreground pixel, includes: The mixed Gaussian distribution is a 3rd-order mixed Gaussian distribution, including Gaussian distributions of 3 states, and the variance of the mixed Gaussian distribution includes the variances of the Gaussian distributions of the 3 states; the Gaussian distributions of the 3 states correspond to foreground, camera noise, and shadow, respectively; For each pixel, determine the maximum value of the variance of the Gaussian distribution of the three states corresponding to the noise of the pixel; If the Gaussian distribution corresponding to the maximum value of the variance is the Gaussian distribution corresponding to the foreground, then the pixel point is a foreground pixel point; otherwise, the pixel point is not a foreground pixel point.

6. The method according to claim 1, wherein If the current frame is the first frame, the input parameters are initial parameters, which include: the initial variance of the mixed Gaussian distribution, the initial mixing coefficient of the mixed Gaussian distribution, the initial coefficient matrix vector of the background subspace, the initial basis matrix vector of the background subspace, the initial transition probability and two initial intermediate parameters for generating the basis matrix vector of the background subspace.

7. The method according to claim 6, wherein A plurality of video frames are acquired, and a principal component analysis method is applied to the plurality of video frames to determine the initial parameters.

8. An online video foreground extraction device, characterized in that: include: A noise determination module is used to process a current video frame of an online video using a low-rank matrix decomposition method to determine the noise of the current video frame; The noise of the current video frame includes the noise of each pixel; A model construction module is used to construct a hidden Markov model for each pixel based on the noise of the pixel and the hidden variable corresponding to the noise of the pixel; the input parameters of the hidden Markov model include the variance of the mixed Gaussian distribution corresponding to the noise of the pixel of the previous video frame, the mixing coefficient of the mixed Gaussian distribution, the coefficient matrix vector of the background subspace, the basis matrix vector of the background subspace, the transition probability and two intermediate parameters for generating the basis matrix vector of the background subspace; A model solving module, configured to solve the hidden Markov model based on the input parameters to obtain output parameters, wherein the output parameters include the variance of the mixed Gaussian distribution corresponding to the noise of each pixel point in the current video frame; a foreground pixel determination module, configured to determine whether each pixel of the current video frame is a foreground pixel based on a variance of a mixed Gaussian distribution corresponding to the noise of each pixel of the current video frame; The foreground extraction module is used to obtain all the foreground pixels to form the foreground of the current video frame.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the online video foreground extraction method according to any one of claims 1 to 7 is implemented.

10. A computer program product, characterized in that The method comprises a computer program / instruction, which, when executed by a processor, implements the online video foreground extraction method according to any one of claims 1 to 7.

Images