A method for detecting a moving object

Abstract

The application discloses a moving target detection method, which comprises the following steps: dividing an image into blocks to establish a background model, using a sparse optical flow method to calculate the movement of the background, and using the movement to compensate the background model; using the detection result of a previous frame and the movement speed of the foreground as a background model update mask to accelerate the convergence speed of the background model; and using a saliency segmentation method to segment the target at the corresponding position of the detection result to obtain a complete target. The application can quickly and completely detect the moving target in a camera movement scene.

Description

Technical Field

[0001] This invention belongs to the field of computer vision moving target detection, and specifically relates to a moving target detection method. Background Technology

[0002] Commonly used moving target detection methods include optical flow, frame difference, and background subtraction. Optical flow is suitable when the camera is also moving, but dense optical flow has too much computational cost, making it difficult to meet real-time computing requirements on embedded devices. Frame difference and background subtraction cannot be applied to situations where the camera is moving, and they also suffer from problems such as holes in the detection results and incomplete target detection. Summary of the Invention

[0003] The purpose of this invention is to provide a moving target detection method that can quickly and accurately detect moving targets in scenes with camera movement.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] To detect moving targets even in scenes with camera movement, background motion needs to be calculated. To reduce computational overhead, a sparse optical flow method is used. To ensure accurate estimation of background motion from the optical flow results, the image is divided into a uniform grid, and feature points are selected within each grid to calculate inter-frame optical flow. This ensures the final optical flow is uniformly distributed, rather than concentrated on the target foreground. To accelerate the convergence of the background model and increase the stability of the detection results, the previous frame's foreground result and its motion velocity are used as an update mask when updating the background model. Specifically, the target location from the previous frame is not updated, preserving the previous model data. To ensure the completeness of the detected targets, saliency segmentation is used to segment the targets at the initially selected locations, ultimately yielding complete target data.

[0006] Specifically, the moving target detection method is implemented through the following steps:

[0007] Step 1: Convert the input image to a grayscale image and perform median filtering;

[0008] Step two: If it is the first frame image, initialize the background model using the first frame image; otherwise, proceed to step three.

[0009] (1) Divide the image into blocks, with each block being ΔB×ΔB in size;

[0010] (2) Calculate the grayscale mean M and variance V of each image patch. The specific calculation method is as follows:

[0011]

[0012] Where M i V represents the average gray level of the i-th image patch. i Let I represent the variance of the i-th image patch, I represent the grayscale value of the image, and ΔB represent the patch size. i Let j represent the set of pixels within the i-th image block, and let j represent image block B. i Pixel number within, V min This represents the minimum value of the set variance (a value greater than 0);

[0013] (3) Initialize a size of An array T consisting entirely of 1s f As the target distribution of the previous frame, where width and height represent the width and height of the image, respectively;

[0014] Step 3: Calculate the displacement of the background and foreground in the current frame and the previous frame, and obtain the affine transformation matrix of the background between the two frames:

[0015] (1) Divide the two frames into grids according to the length and width interval g, and select the corner point with the most obvious feature in each grid of the previous frame as the set of points P to be determined for the inter-frame displacement. pre ;

[0016] (2) Use KLT to find P in the previous step in the current frame image. pre The corresponding point P of each point cur ;

[0017] (3) Use RANSAC to calculate the affine transformation matrix H of the two corresponding point sets;

[0018] (4) The parameters related to the horizontal and vertical displacements in the affine transformation matrix obtained in step (3) are respectively taken as the displacements of the inter-frame background in the horizontal and vertical directions, and denoted as follows: Then, combining the feature point correspondence between the two frames obtained in step (2), the displacement of the foreground in each image block is calculated. And the velocity distribution map S of the foreground is obtained. f The specific calculation process is as follows:

[0019]

[0020] Step four: Use the affine transformation matrix obtained in step three to perform motion compensation on the existing background model:

[0021] (1) Calculate the coordinates of the current frame background model in the previous frame background model using the affine transformation matrix. The specific calculation formula is as follows:

[0022] X = H -1 X

[0023] X pre =H-1X cur

[0024] Where X pre and X cur These represent the coordinates in the previous frame and the current background model, respectively, and H represents the affine transformation matrix;

[0025] (2) Based on the obtained coordinate correspondence, the motion-compensated background model is obtained using bilinear interpolation. Points outside the range of the previous frame's background model are filled with 0. The calculation process for values ​​within the valid range is as follows:

[0026] Where M warp and V warp Representing the mean and variance after motion compensation, respectively, (x cur ,y cur (x) represents the coordinates of the current frame model. pre ,y pre ) represents the coordinates of the model in the previous frame. When the coordinates obtained in step four (1) satisfy The value is calculated according to the formula given above; otherwise, the value is 0. Here, width and height represent the width and height of the image, respectively.

[0027] Step 5: Update the background model:

[0028] (1) Calculate the block mean M of the current frame image according to the methods in steps (1) and (2) of step one. cur Sum of variance V cur ;

[0029] (2) Combining the target distribution T of the previous frame f and foreground velocity distribution S f The background model update scaling factor, rate, is calculated as follows:

[0030]

[0031] Where S f ∩T f This represents the region where neither the target distribution nor the foreground velocity distribution has a value of 0.

[0032] (3) Put M cur V cur And M obtained in step four warp V warp The specific calculation process is as follows: Summing is performed separately according to the obtained proportionality coefficients.

[0033]

[0034]

[0035] Where M update and V update These represent the updated model mean and variance, respectively; i represents the i-th image patch; and rate represents the set scaling factor.

[0036] Step Six, Target Area Division:

[0037] (1) Based on the grayscale value of the current frame image and the updated background model, calculate the foreground response surface F. map The specific calculation process is as follows:

[0038]

[0039] Where j represents the j-th pixel in the image, I represents the gray value, and i represents the image block number corresponding to pixel j;

[0040] (2) The response surface F map The portion with a median value greater than the threshold is considered foreground. The specific steps are as follows:

[0041]

[0042] Where t represents the set threshold;

[0043] Step 7: Filter and segment the target area:

[0044] (1) Perform connected component analysis on the foreground graph F obtained in step six to remove regions that are too small or do not conform to the target characteristics of the application scenario;

[0045] (2) In the current frame image, at the position corresponding to the centroid of the target region obtained in the previous step, the final position and size of the target are obtained by saliency segmentation;

[0046] (3) Obtain the target distribution T from the final target map using the method of calculating the mean of the background model. f .

[0047] This invention divides the target area and template area and utilizes the changes in the correlation coefficients of each part to quickly and effectively detect partial and complete occlusion, and can effectively suppress false alarms. Attached Figure Description

[0048] Figure 1 Here is a flowchart for moving target detection;

[0049] Figure 2 Background model;

[0050] Figure 3 To obtain the target prospect results;

[0051] Figure 4 This is the final result of the moving target detection. Detailed Implementation

[0052] The following will be combined with the appendix Figures 1-4 This invention will be described in detail, and the technical solutions in the embodiments of this invention will be clearly and completely described. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0053] A method for detecting moving targets includes the following steps:

[0054] Step 1: Convert the input image to a grayscale image and perform median filtering;

[0055] Step two: If it is the first frame image, initialize the background model using the first frame image; otherwise, proceed to step three.

[0056] (1) Divide the image into blocks, with each block being ΔB×ΔB in size;

[0057] (2) Calculate the grayscale mean M and variance V of each image patch. The specific calculation method is as follows:

[0058]

[0059] Where M i V represents the average gray level of the i-th image patch. i Let I represent the variance of the i-th image patch, I represent the grayscale value of the image, and ΔB represent the patch size. i Let j represent the set of pixels within the i-th image block, and let j represent image block B. i Pixel number within, V min This represents the minimum value of the set variance (a value greater than 0);

[0060] (3) Initialize a size of An array T consisting entirely of 1s f As the target distribution of the previous frame, where width and height represent the width and height of the image, respectively;

[0061] Step 3: Calculate the displacement of the background and foreground in the current frame and the previous frame, and obtain the affine transformation matrix of the background between the two frames:

[0062] (1) Divide the two frames into grids according to the length and width interval g, and select the corner point with the most obvious feature in each grid of the previous frame as the set of points P to be determined for the inter-frame displacement. pre ;

[0063] (2) Use KLT to find P in the previous step in the current frame image. pre The corresponding point P of each point cur ;

[0064] (3) Use RANSAC to calculate the affine transformation matrix H of the two corresponding point sets;

[0065] (4) The parameters related to the horizontal and vertical displacements in the affine transformation matrix obtained in step (3) are respectively taken as the displacements of the inter-frame background in the horizontal and vertical directions, and denoted as follows:

[0066] Then, combining the feature point correspondence between the two frames obtained in step (2), the displacement of the foreground in each image block is calculated. And the velocity distribution map S of the foreground is obtained. f The specific calculation process is as follows:

[0067]

[0068] Step four: Use the affine transformation matrix obtained in step three to perform motion compensation on the existing background model:

[0069] (1) Calculate the coordinates of the current frame background model in the previous frame background model using the affine transformation matrix. The specific calculation formula is as follows:

[0070] X pre =H -1 X cur

[0071] Where X pre and X cur These represent the coordinates in the previous frame and the current background model, respectively, and H represents the affine transformation matrix;

[0072] (2) Based on the obtained coordinate correspondence, the motion-compensated background model is obtained using bilinear interpolation. Points outside the range of the previous frame's background model are filled with 0. The calculation process for values ​​within the valid range is as follows:

[0073]

[0074] Where M warp and V warp Representing the mean and variance after motion compensation, respectively, (x cur ,ycur (x) represents the coordinates of the current frame model. pre ,y pre ) represents the coordinates of the model in the previous frame. When the coordinates obtained in step four (1) satisfy The value is calculated according to the formula given above; otherwise, the value is 0. Here, width and height represent the width and height of the image, respectively.

[0075] Step 5: Update the background model:

[0076] (1) Calculate the block mean M of the current frame image according to the methods in steps (1) and (2) of step one. cur Sum of variance V cur ;

[0077] (2) Combining the target distribution T of the previous frame f and foreground velocity distribution S f The background model update scaling factor, rate, is calculated as follows:

[0078]

[0079] Where S f ∩T f This represents the region where neither the target distribution nor the foreground velocity distribution has a value of 0.

[0080] (3) Put M cur V cur And M obtained in step four warp V warp The specific calculation process is as follows: Summing is performed separately according to the obtained proportionality coefficients.

[0081]

[0082]

[0083] Where M update and V update These represent the updated model mean and variance, respectively; i represents the i-th image patch; and rate represents the set scaling factor.

[0084] Step Six, Target Area Division:

[0085] (1) Based on the grayscale value of the current frame image and the updated background model, calculate the foreground response surface F. map The specific calculation process is as follows:

[0086]

[0087] Where j represents the j-th pixel in the image, I represents the gray value, and i represents the image block number corresponding to pixel j;

[0088] (2) The response surface F map The portion with a median value greater than the threshold is considered foreground. The specific steps are as follows:

[0089]

[0090] Where t represents the set threshold;

[0091] Step 7: Filter and segment the target area:

[0092] (1) Perform connected component analysis on the foreground graph F obtained in step six to remove regions that are too small or do not conform to the target characteristics of the application scenario;

[0093] (2) In the current frame image, at the position corresponding to the centroid of the target region obtained in the previous step, the final position and size of the target are obtained by saliency segmentation;

[0094] (3) Obtain the target distribution T from the final target map using the method of calculating the mean of the background model. f .

[0095] The values ​​in this embodiment are as follows:

[0096] ΔB = 4, V min =144, g=10.

[0097] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting moving targets, characterized in that, Includes the following steps, S1. Convert the input image to a grayscale image and perform median filtering; S2. If it is the first frame image, initialize the background model using the first frame image; if it is not the first frame, jump to step S3. S3. Calculate the displacement of the background and foreground in the current frame and the previous frame, and obtain the affine transformation matrix of the background between the two frames; the specific method for step S3 is as follows: S31. Divide the two frames into a grid with a length and width interval g, and select the corner point with the most obvious feature in each grid of the previous frame as the set of points to be determined for the inter-frame displacement. ; S32, find the corresponding points of the respective points in the current frame image using KLT ;​ S33. Calculate the affine transformation matrix of two corresponding point sets using RANSAC. H ; S34. The parameters related to the horizontal and vertical displacements in the affine transformation matrix obtained in step S33 are respectively used as the displacements of the inter-frame background in the horizontal and vertical directions, and are denoted as follows: , Then, combining the feature point correspondence between the two frames obtained in step S32, the displacement of the foreground in each image block is calculated. , And obtain the velocity distribution map of the foreground. The specific calculation process is as follows: ； S4. Use the affine transformation matrix obtained in step S3 to perform motion compensation on the existing background model. S5. Update the background model, including: S51. Calculate the block average of the current frame image. and variance ; S52. Calculate the background model update scaling factor by combining the target distribution and foreground velocity distribution of the previous frame; S53, put , Mean after exercise compensation and variance Sum the results according to the calculated proportionality coefficients; S6. Target area division, including: S61. Calculate the foreground response surface based on the grayscale values ​​of the current frame image and the updated background model. , S62, Response Surface The portion with a median value greater than the threshold is considered the foreground. S7. Filter and segment the target area, including... S71. The foreground image obtained in step S6 F Perform connected component analysis to remove regions that are too small or do not meet the target characteristics of the application scenario; S72. In the current frame image, at the position corresponding to the centroid of the target region obtained in the previous step, use saliency segmentation to obtain the final position and size of the target. S73. Obtain the target distribution from the final target map using the method of calculating the mean of the background model. .

2. The moving target detection method according to claim 1, characterized in that, The specific method for step S2 is as follows: S21. Divide the image into blocks, with each block being the size of a given block. △B × △B ; S22. Calculate the average gray value of each image patch. M and variance V The specific calculation method is as follows: ， in M i Representing the i The average gray level of each image patch. V i Representing the i The variance of each image patch, △B Represents the size of the block. B i Representing the i The set of pixels within an image patch j Represents image blocks B i Pixel number within, V min This represents the minimum value of the set variance. V min >0; I j represent j The grayscale value of a pixel image; S23. Initialize a space of size... An array of all 1s As the target distribution of the previous frame, width and height These represent the width and height of the image, respectively.

3. The moving target detection method according to claim 2, characterized in that, The specific method for step S4 is as follows: S41. Calculate the coordinates of the current frame's background model in the previous frame's background model using the affine transformation matrix. The specific calculation formula is as follows: ； in and These represent the coordinates in the previous frame and the current background model, respectively. H Represents the affine transformation matrix; S42. Based on the obtained coordinate correspondence, use bilinear interpolation to obtain the motion-compensated background model. Fill the values ​​of points that exceed the range of the previous frame's background model with 0.

4. The moving target detection method according to claim 3, characterized in that, In step S51, the block average of the current frame image is calculated using the methods in steps S21 and S22. M cur and variance V cur ; In step S52, the target distribution from the previous frame is combined. and foreground velocity distribution Calculate the background model update ratio coefficient rate The specific calculation process is as follows: in This represents the region where neither the target distribution nor the foreground velocity distribution has a value of 0. Λ represents the values ​​of the target distribution and the foreground velocity distribution. V max This represents the maximum value of the set variance; In step S53, M cur , V cur and the result obtained in step S4 M warp , V warp The specific calculation process is as follows: Summing is performed separately according to the obtained proportionality coefficients. ; ; in M update and V update These represent the updated model mean and variance, respectively. i Indicates the first i Image blocks, rate This represents the set proportional coefficient.

5. The moving target detection method according to claim 4, characterized in that, The foreground response surface is calculated in step S61. F map The specific calculation process is as follows: ， in j Indicates the first in the image j 1 pixel I Represents grayscale value, i Represents pixels j The corresponding image block number; In step S62, the response surface F map The specific steps for determining the portion of the median value greater than the threshold as foreground are as follows: in t This represents the set threshold.

Images