A moving target detection method based on fast independent component analysis combined with spatiotemporal differences

Abstract

The present invention belongs to the field of image and video processing technology, and specifically relates to a method for detecting moving targets based on fast independent component analysis combined with spatiotemporal differences, comprising: combining FastICA with a frame difference method, and obtaining a contour F by performing FastICA processing on the difference in the time domain between the current frame and the adjacent frames. R1 ; Send the video sequence to the Gaussian mixture model to generate the background image and the current frame to perform FastICA processing on the spatial domain to obtain the contour F R2 ; Through the fusion strategy, the contour F of the target in the time domain is transformed R1 And the target profile F in the spatial domain R2 The fused contours are morphologically processed to obtain accurate moving targets. This invention combines the temporal and spatial differences in the detection scene through fast independent component analysis, effectively making up for the shortcomings of the two methods in various fields, effectively avoiding these problems, and effectively improving the accuracy and effectiveness of the method.

Description

Technical Field

[0001] The present invention belongs to the technical field of image and video processing, and in particular relates to a moving target detection method based on fast independent component analysis combined with spatiotemporal differences. Background Art

[0002] Moving object detection involves using algorithms to separate moving foreground objects from relatively stationary background objects in video sequences. It is a fundamental information extraction process in many computer vision applications, including human pose tracking, traffic monitoring, and video semantic annotation. Its primary task is to extract moving objects from video sequences and separate the foreground from the noisy background.

[0003] The inter-frame difference method performs a difference operation on two consecutive frames in a video sequence to obtain the outline of the moving object. This method recognizes the presence of a moving object by considering the difference between two consecutive frames.

[0004] Background subtraction is considered one of the most reliable methods for detecting moving objects. It works by initializing a background model. For each pixel in the current frame, the difference between the pixel value and the assumed background model is calculated. If the difference is greater than a threshold, the pixel is considered foreground; otherwise, it is considered background.

[0005] Independent component analysis (ICA) is a computational method used in signal processing to separate a multivariate signal into additive subcomponents. This method assumes that the subcomponents are non-Gaussian and statistically independent. In the ICA model, each source signal is composed of several independent signals. Furthermore, each source signal has a non-Gaussian distribution.

[0006] The ICA model finds the unmixing matrix W in an unsupervised learning process, which is used to separate the independent signals from the mixed signal X observed by the transformation.

[0007] Fast Independent Component Analysis (FastICA) is an improvement to the classic ICA algorithm. Based on a fixed-point recursive algorithm, FastICA is applicable to any data type. This approach allows ICA to analyze high-dimensional data. FastICA is a fast iterative optimization algorithm that uses a batch processing approach, with each iteration involving a large number of sample data. The algorithm uses a fixed-point iterative optimization strategy, resulting in fast convergence and robustness. FastICA can perform signal separation based on fourth-order cumulants, maximum likelihood, and maximum negentropy.

[0008] Gaussian mixture models (GMMs) are used to model video backgrounds by representing the pixel value of each pixel in a video frame as a superposition of K Gaussian distributions. GMMs are able to separate foreground and background based on the fact that in long-term observations, the background is present most of the time, so more data supports the background distribution.

[0009] Currently, common target detection methods usually only detect moving targets based on temporal differences or spatial differences. For example, the inter-frame difference method and the optical flow method distinguish the foreground and background by the difference between two frames at different times. The background difference method uses background modeling to compare the current frame to be detected with the background model, and detects the moving target by their spatial differences. Simply comparing spatial differences or temporal differences may result in inaccurate or even invalid detection of the target. For example, when the target is moving slowly, the temporal difference between two frames of the video sequence is too small, which may cause the inter-frame difference method to fail. For example, when the moving target is small relative to the background, the lack of a sufficiently significant difference between it and the background may cause the background difference classification method to fail. In order to solve the above problems, our method detects moving targets simultaneously from both temporal and spatial dimensions, making the detection of moving targets more accurate.

[0010] The detection of moving targets usually results in unsatisfactory results due to factors such as noise and lighting changes in the environment. In objective reality, the impact of these external factors on the algorithm is almost inevitable. Therefore, in order to deal with these problems, we need to use some image processing techniques. Image processing techniques mainly include grayscale, noise removal, binarization, and morphological processing. Grayscale includes the representation of grayscale images, the conversion between true color bitmaps and grayscale images; noise removal methods include Gaussian filtering and median filtering; binarization is to use a certain threshold to divide the image into foreground targets and background parts. Morphological processing is to eliminate isolated noise from the target image, fill edge holes, etc., and includes processing methods such as corrosion and dilation.

[0011] To sum up, the existing technical problems are: currently common target detection methods usually only detect moving targets based on temporal differences or spatial differences. Simply comparing spatial differences or temporal differences may cause the final detected targets to be inaccurate or even invalid; the detection of moving targets usually leads to unsatisfactory results due to factors such as noise and lighting changes in the environment, and in objective reality, the impact of these external factors on the algorithm is almost inevitable. Summary of the Invention

[0012] To solve the above technical problems, the present invention proposes a moving target detection method based on fast independent component analysis combined with spatiotemporal differences, comprising the following steps:

[0013] S1: Select the current frame that needs to be detected from the video sequence, and perform target detection on the current frame I n Its adjacent frame sequence (I n’-1 , I n’ , I n’+1 ) in the valid adjacent frame I n’-1 The frame difference method is used to detect the difference between the foreground and the contour F of the target in the time domain is obtained by combining FastICA. R1 ;

[0014] S2: Use Gaussian mixture model to model the background of the video sequence, and then use the background image g n With the current frame I n The frame difference method is used to detect the difference between the foreground and background and FastICA processing is performed to obtain the contour F of the target in the spatial domain. R2 ;

[0015] S3: The contour F of the target in the time domain is transformed through the fusion strategy R1 And the target profile F in the spatial domain R2 Perform fusion to obtain a fused contour map;

[0016] S4: Perform morphological processing on the fused contour image to obtain the final moving target image.

[0017] Beneficial effects of the present invention:

[0018] 1) This invention combines traditional moving object detection methods, inter-frame difference method and Gaussian mixture model, through fast independent component analysis. This effectively overcomes the shortcomings of both methods in various fields. For example, Gaussian mixture model requires the input of video sequence to model the background. While inter-frame difference method has the advantages of speed and portability, it often produces undesirable results such as holes and ghosting due to the excessively fast or slow movement of moving objects. By combining these two methods, we effectively avoid these problems and significantly improve the accuracy and effectiveness of the method.

[0019] 2) The present invention combines temporal and spatial differences in the scene to detect moving targets, detecting moving targets from two dimensions. This allows for effective detection in scenes that were difficult to detect with previous methods, such as when the difference between foreground and background is small or when foreground motion is not obvious. This effectively addresses the problem of previous methods being ineffective with small targets or slow-moving targets, while achieving higher precision and accuracy. Furthermore, the present invention is applicable to a wider range of applications and possesses greater robustness and practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a simplified flow chart of the moving target detection algorithm of the present invention;

[0021] Figure 2 This is the specific process of the fast independent component analysis combined with the frame difference method algorithm of the present invention;

[0022] Figure 3 This is the specific process of the present invention's fast independent component analysis combined with Gaussian mixture model;

[0023] Figure 4 Schematic diagram of Gaussian mixture model background modeling of the present invention;

[0024] Figure 5 2 is a schematic diagram of the moving target detection result of the present invention. DETAILED DESCRIPTION

[0025] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0026] A moving target detection method based on fast independent component analysis combined with spatiotemporal differences, such as Figure 1 As shown, the following steps are included:

[0027] S1: Cut the video image sequence into n frames to be detected, select the current frame that needs to be detected from the n frames to be detected, and perform target detection on the current frame I n Its adjacent frame sequence (I n’-1 ,I n’ ,I n’+1 ) in the valid adjacent frame I n’-1 The frame difference method is used to detect the difference between the foreground and the contour F of the target in the time domain is obtained by combining FastICA. R1 ;

[0028] S2: Use Gaussian mixture model to model the background of the video sequence, and then use the background image g n With the current frame I n The frame difference method is used to detect the difference between the foreground and background and FastICA processing is performed to obtain the contour F of the target in the spatial domain. R2 ;

[0029] S3: The contour F of the target in the time domain is transformed through the fusion strategy R1 And the target profile F in the spatial domain R2 Perform fusion to obtain a fused contour map;

[0030] S4: Perform morphological processing on the fused contour image to obtain the final moving target image.

[0031] like Figure 2 As shown, the current frame I where the moving target needs to be detected is obtained in the video sequence. n and adjacent frames I n’-1 Finally, the target profile F in the time domain is obtained by combining FastICA R1 ,include:

[0032] S11: The current frame I n Image and valid adjacent frames I n’-1 The image is transformed into one-dimensional signals R1, R2, and the one-dimensional signals R1, R2 are combined to obtain a composite matrix X c ;

[0033] S12: Random matrix T with values ​​between 0 and 1 r With the composite matrix X c Multiply to get the mixing matrix, and center the mixing matrix to get the zero-mean matrix M s ;

[0034] S13: Calculate the zero mean matrix M s Correlation matrix Eigenvector matrix E and eigendiagonal matrix D;

[0035] S14: According to the correlation matrix The whitening matrix is ​​calculated by calculating the eigenvector matrix E and the eigendiagonal matrix D

[0036] S15: Initialize the 2×2 separation weight matrix W, the value of W is evenly distributed between 0 and 1, adjust the weight matrix W through iterative optimization, and obtain a one-dimensional separation signal through the separation weight matrix W Separate the signal in one dimension Expand into a two-dimensional image and get the final detection result F R1 .

[0037] Calculate the zero-mean matrix M s Correlation matrix And obtain the eigenvector matrix E and the eigendiagonal matrix D, including:

[0038] Calculate the covariance matrix C:

[0039] C(x,y)=cov(M s T )

[0040] Calculate the correlation matrix include:

[0041]

[0042] in, Represents the correlation matrix The value at position (x,y), C(x,y) represents the zero-mean matrix M s The value of the covariance matrix C at position (x, y), cov() represents the covariance operation, T represents the transpose operation, and σ(x, y) is the zero mean matrix M s The standard deviation at position (x,y);

[0043] Obtain the eigenvalues ​​E and eigenvector matrix D of the covariance matrix C, including:

[0044] D,E=eigen(C)

[0045] Among them, cov() represents the covariance operation, eigen() represents the eigenvalue operation, and T represents the transpose operation.

[0046] According to the correlation matrix The whitening matrix is ​​calculated by calculating the eigenvector matrix E and the eigendiagonal matrix D include:

[0047]

[0048] in, represents the whitening matrix, E represents the eigenvector matrix, D represents the characteristic diagonal matrix, T represents the transpose operation, represents the correlation matrix.

[0049] Iterative optimization adjusts the weight matrix W, including:

[0050]

[0051]

[0052] Among them, W represents the weight matrix, E represents the eigenvector matrix, represents the whitening matrix, g represents the function, g=tanh(a1m)(1≤a1≤2), a1 represents the natural coefficient, m represents the input parameter of the function g, || || represents the sum of absolute values, g' represents the derivative of the function g, and T represents the transpose operation.

[0053] Separate the weight matrix W to obtain a one-dimensional separation signal include:

[0054]

[0055] in, represents the one-dimensional separation signal, M srepresents the zero-mean matrix, and W represents the weight matrix.

[0056] The combination of FastICA and Gaussian mixture model, see the specific process Figure 3 :First, the video sequence is input into the Gaussian mixture model to model the background and obtain the background image g n , background image such as Figure 4 As shown;

[0057] Then, whether the background modeling is stable is determined by whether the number of frames input to the background modeling is greater than the threshold K. When the number of input frames is greater than the threshold K, the current frame I n With background image g n Perform FastICA processing to obtain the moving target contour F R2 .

[0058] Otherwise, the detection of moving targets only depends on the method in the second step.

[0059] The contour F of the target in the time domain is transformed by the fusion strategy R1 And the target profile F in the spatial domain R2 Integration, including:

[0060] S31: Calculate the target profile F in the time domain R1 The contour F of the target in the current frame and adjacent frames and spatial domain R2 The MSE value of the current frame and the background image in ;

[0061] S32: Then pass the current frame I n With adjacent frame I n’-1 and background image g n The MSE value of the binarization threshold R1, R2 is given, where R1 is the contour F R1 The threshold value, R2 is the contour F R2 The two contour images are binarized by using the binarization thresholds R1 and R2;

[0062] S33: Fusing the two binarized contour images to obtain a fused contour image.

[0063] Optimization of the fusion strategy of the two contour results: By comparing the current frame I n With adjacent frame I n’-1 and background image g n The difference between foreground and foreground, and foreground and background is determined as follows:

[0064] First, calculate the MSE value between the current frame and the adjacent frames, and between the current frame and the background image. The calculation formula is as follows:

[0065]

[0066] Among them, MSE represents mean square error, h represents the height of the input image, w represents the width of the input image, and I1(i,j) represents the contour of the target in the time domain F R1 The grayscale value at the position (i, j) of the current frame and the adjacent frame in the image, I2(i, j) represents the contour of the target in the spatial domain F R2 The grayscale value at position (i, j) of the current frame and the background image in .

[0067] Then pass the current frame I n Valid adjacent frame I n’-1 and background image g n The MSE value of the binarization threshold T1, T2 is given, where T1 is the contour F R1 The threshold value, T2 is the contour F R2 The two contour images are binarized by using the binarization thresholds T1 and T2. The binarization formula is as follows:

[0068]

[0069] Where f(x,y) is the grayscale value of the image at the (x,y) position, T1 is the threshold value obtained based on the difference between the foreground and background, and T2 is the threshold value set based on the difference between the foreground and background.

[0070] Perform morphological processing on the fused contour map:

[0071] First, the fused image is corroded to eliminate isolated noise;

[0072] Then perform dilation operation on the fused image to fill the holes;

[0073] Finally, the final detected moving target is obtained, and the final result is as follows Figure 5 shown.

[0074] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A moving target detection method based on fast independent component analysis combined with spatiotemporal differences, characterized in that: include: S1: Select the current frame that needs to be detected from the video sequence, and perform target detection on the current frame I n Its adjacent frame sequence (I n’-1 , I n’ , I n’+1 ) in the valid adjacent frame I n’-1 Combined with FastICA, the target profile F in the time domain is obtained R1 ; S11: The current frame I n Image and valid adjacent frames I n’-1 The image is transformed into one-dimensional signals R1, R2, and the one-dimensional signals R1, R2 are combined to obtain a composite matrix X c ; S12: Random matrix T with values ​​between 0 and 1 r With the composite matrix X c Multiply to get the mixing matrix, and center the mixing matrix to get the zero-mean matrix M s ; S13: Calculate the zero mean matrix M s Correlation matrix And obtain the eigenvector matrix E and the eigendiagonal matrix D; S14: According to the correlation matrix The whitening matrix is ​​calculated by calculating the eigenvector matrix E and the eigendiagonal matrix D S15: Initialize the 2×2 separation weight matrix W, the value of W is evenly distributed between 0 and 1, adjust the weight matrix W through iterative optimization, and obtain a one-dimensional separation signal through the separation weight matrix W Separate the signal in one dimension Expand into a two-dimensional image and get the final detection result F R1 ; S2: Use Gaussian mixture model to model the background of the video sequence, and then use the background image g n With the current frame I n The frame difference method is used to detect the difference between the foreground and background and FastICA processing is performed to obtain the contour F of the target in the spatial domain. R2 ; S3: The contour F of the target in the time domain is transformed through the fusion strategy R1 And the target profile F in the spatial domain R2 Perform fusion to obtain a fused contour map; The contour F of the target in the time domain is transformed by the fusion strategy R1 And the target profile F in the spatial domain R2 Integration, including: S31: Calculate the target profile F in the time domain R1 The contour F of the target in the current frame and adjacent frames and spatial domain R2 The MSE value of the current frame and the background image in ; The calculation of the MSE value includes: Among them, MSE represents mean square error, h represents the height of the input image, w represents the width of the input image, and I1(i,j) represents the contour of the target in the time domain F R1 The grayscale value at the position (i, j) of the current frame and the adjacent frame in the image, I2(i, j) represents the contour of the target in the spatial domain F R2 The grayscale value at position (i, j) of the current frame and the background image in ; S32: Then pass the current frame I n With adjacent frame I n’-1 and background image g n The MSE value of the binarization threshold R1, R2 is given, where R1 is the contour F R1 The threshold value, R2 is the contour F R2 The two contour images are binarized by using the binarization thresholds R1 and R2; Binarize the contour image, including: Among them, f(x,y) represents the pixel value of the image at position (x,y), R represents the threshold of the binarization operation, and R1 represents the contour F R1 The threshold value, R2 represents the contour F R2 threshold value; S33: Fusing the two binarized contour images to obtain a fused contour image; S4: Perform morphological processing on the fused contour image to obtain the final moving target image.

2. The method for detecting moving targets based on fast independent component analysis combined with spatiotemporal differences according to claim 1, characterized in that: Calculate the zero-mean matrix M s Correlation matrix And obtain the eigenvector matrix E and the eigendiagonal matrix D, including: Calculate the correlation matrix include: in, Represents the correlation matrix The value at position (x,y), C(x,y) represents the matrix M s The value of the covariance matrix C at the position (x, y), C(x, y) = cov(M s T ), cov() represents the covariance operation, T represents the transpose operation, σ(x,y) is the zero-mean matrix M s The standard deviation at position (x,y); Obtain the eigenvalues ​​E and eigenvector matrix D of the covariance matrix C, including: D,E=eigen(C) Among them, cov() represents the covariance operation, eigen() represents the eigenvalue operation, T Represents a transpose operation.

3. The method for detecting moving targets based on fast independent component analysis combined with spatiotemporal differences according to claim 1, characterized in that: According to the correlation matrix The whitening matrix is ​​calculated by calculating the eigenvector matrix E and the eigendiagonal matrix D include: in, represents the whitening matrix, E represents the eigenvector matrix, D represents the characteristic diagonal matrix, T represents the transpose operation, represents the correlation matrix.

4. The method for detecting moving targets based on fast independent component analysis combined with spatiotemporal differences according to claim 1, characterized in that: Iterative optimization adjusts the weight matrix W, including: Among them, W represents the weight matrix, E represents the eigenvector matrix, represents the whitening matrix, g represents the function, g=tanh(a1m)(1≤a1≤2), a1 represents the natural coefficient, m represents the input parameter of the function g, || || represents the sum of absolute values, g' represents the derivative of the function g, and T represents the transpose operation.

5. The method for detecting moving targets based on fast independent component analysis combined with spatiotemporal differences according to claim 1, characterized in that: Separate the weight matrix W to obtain a one-dimensional separation signal include: in, represents the one-dimensional separation signal, M s represents the zero-mean matrix, and W represents the weight matrix.

6. The method for detecting moving targets based on fast independent component analysis combined with spatiotemporal differences according to claim 1, characterized in that: The background image g obtained by modeling n With the current frame I n Perform FastICA processing to obtain the target profile F in the spatial domain R2 ,include: Whether the background modeling is stable is determined by whether the number of input frames is greater than the threshold K. When the number of input frames is greater than the threshold K, the current frame and the background image are processed by FastICA to obtain the moving target contour F. R2 ; The threshold K is the number of frames of background modeling at which the background modeling of the Gaussian mixture model tends to be stable.

7. The method for detecting moving targets based on fast independent component analysis combined with spatiotemporal differences according to claim 1, characterized in that: Perform morphological processing on the fused contour map, including: The fused image is corroded to obtain a contour image after eliminating isolated noise, and the contour image after eliminating isolated noise is expanded to obtain the final moving target image after filling holes.

Images