An improved ViBe indoor real-time foreground detection method and storage medium

Abstract

The present invention provides an improved ViBe indoor real-time foreground detection method and storage medium. On the basis of the ViBe algorithm, frame-to-frame change rate calculation and a background update strategy based on the frame-to-frame change rate to adjust the random factor are added to suppress the "ghost" and hole problems caused by the spatiotemporal propagation mechanism, and illumination detection based on the frame-to-frame change rate and the perceptual hash algorithm is used to determine whether there is a sudden illumination change. If a sudden illumination change occurs, the background model is rebuilt, which effectively overcomes various problems of the ViBe algorithm. Beneficial effects of the present invention: This method effectively suppresses the "ghost" and hole phenomena without sacrificing the real-time performance of the original ViBe algorithm for foreground detection, improves the detection capability of the algorithm to a large extent, and enhances the robustness to illumination changes. Compared with the ViBe algorithm, it is more suitable for foreground detection in indoor environments.

Description

Technical Field

[0001] The invention relates to the field of mobile target detection, and in particular to an improved ViBe indoor real-time foreground detection method and storage medium. Background Art

[0002] In recent years, video surveillance technology has been increasingly used in important fields such as national and social public security, aerospace, and many civilian fields. Installing video surveillance systems in unmanned areas can effectively eliminate safety hazards, provide timely warnings, and avoid accidents. Currently, the most commonly used intelligent video surveillance algorithms are based on foreground detection (moving target detection, foreground extraction) algorithms. Foreground detection is a technology that extracts foreground targets from image sequences.

[0003] The ViBe algorithm is a foreground detection algorithm that builds a background model at the pixel level and performs background difference with the current frame. The algorithm uses the characteristic that pixels in the same area have similar pixel values ​​and only selects the first frame of the video for background initialization. The ViBe algorithm includes three steps: background modeling, foreground detection, and background model update. There are also some challenges in the specific application process: (1) There are some common problems in the actual detection process, such as "ghost images" and holes caused by the spatiotemporal propagation mechanism used when updating the background model; (2) Due to the particularity of indoor space, manual operations such as turning on and off lights and windows, and drawing curtains will cause a large change in the global light intensity of the room. Therefore, the algorithm must have good robustness to light changes and requires a more comprehensive solution. Summary of the invention

[0004] The main technical problem to be solved by the present invention is to solve the "ghost" and hole problems caused by the traditional ViBe algorithm and the problem of poor robustness to illumination changes. In order to solve the above problems, the present invention provides an improved ViBe indoor real-time foreground detection method. Without sacrificing the real-time performance of the original ViBe algorithm for foreground detection, a random factor adjustment strategy based on the inter-frame change rate is used to effectively suppress the "ghost" and hole problems caused by the spatiotemporal propagation mechanism in background updating. In addition, the algorithm's robustness to illumination mutations is enhanced by an illumination mutation detection method based on the inter-frame change rate and a perceptual hash algorithm. Compared with the ViBe algorithm, the method is more suitable for foreground detection in indoor environments.

[0005] According to one aspect of the present invention, the present invention provides an improved ViBe indoor real-time foreground detection method, comprising the following steps:

[0006] S1: Receive the video stream and decode and preprocess it to obtain the grayscale image of each frame of the video image;

[0007] S2: Monitor the frame number of the current frame video image to determine whether it is the first frame of the video. If so, establish the background model of the foreground detection algorithm and enter S1. Otherwise, enter S3.

[0008] S3: Calculate the inter-frame change rate by using the current frame grayscale image and the previous frame grayscale image;

[0009] S4: Determine whether a sudden change in illumination occurs. If so, re-establish the background model in S2 and proceed to step S1. Otherwise, proceed to S5.

[0010] S5: Obtaining the foreground detection result of the current frame video image;

[0011] S6: Based on the inter-frame change rate obtained in S3, the background model is updated according to the inter-frame change rate adjustment strategy;

[0012] S7: Determine whether there is a next frame of video image. If so, go to S1, otherwise, the detection process ends.

[0013] Preferably, in S1, the steps of receiving, decoding and preprocessing the video stream include:

[0014] S11: receiving and decoding a video stream to obtain a frame-by-frame video image queue;

[0015] S12: performing grayscale conversion on a single-frame video image in the video image queue to obtain a grayscale image of the single-frame video image;

[0016] S13: denoising the grayscale image using Gaussian and median filtering to obtain a denoised grayscale image.

[0017] Preferably, in S2, the step of establishing the background model includes:

[0018] S21: Obtain the first grayscale image frame of the video, and construct a sample set with a maximum capacity of N for each pixel;

[0019] S22: Each pixel randomly selects the grayscale values ​​of the pixels in its eight neighborhoods and adds them to the sample set until the sample set capacity reaches the upper limit;

[0020] The expression of the sample set is as follows:

[0021] Sample(x,y)={Vi|i=1,2...N}

[0022] Where Sample(x,y) is the sample set of pixels, (x,y) is the Cartesian coordinate value of any pixel, Vi is the sample, and N is the number of samples.

[0023] Preferably, in S3, the step of calculating the frame-to-frame change rate includes:

[0024] S31: Select the current frame grayscale image and the previous frame grayscale image, perform difference operation on the pixel points at the same position in the Cartesian coordinate system and take their absolute values ​​to obtain a differential grayscale image. The formula is as follows:

[0025] D(x,y)=|fk(x,y)-fk-1(x,y)|

[0026] Where D(x, y) is the obtained differential grayscale image, fk(x, y) is the current frame grayscale image, and fk-1(x, y) is the previous frame grayscale image;

[0027] S32: Determine whether the absolute value of the grayscale value of each pixel in the differential grayscale image is greater than a threshold value T, if so, modify its grayscale value to 255, otherwise, set it to 0;

[0028] S33: The number of pixels with a grayscale value of 255 in the statistical difference grayscale image is num1, the total number of pixels in the image is num2, and the frame-to-frame change rate P=num1 / num2×100% is calculated.

[0029] Preferably, in S4, the method for determining whether a sudden change in illumination occurs includes:

[0030] S41: Determine whether the value of the inter-frame change rate P calculated by S33 is greater than the threshold value Tlight. If so, proceed to S42. Otherwise, determine that no sudden change in illumination has occurred.

[0031] S42: Use the perceptual hash algorithm to compare the current frame grayscale image with the differential grayscale image obtained in S31 to determine whether the Hamming distance between the two images is greater than the threshold Tenter. If so, it is determined that no illumination mutation has occurred; otherwise, it is determined that an illumination mutation has occurred.

[0032] Preferably, in S5, the step of obtaining the foreground detection result includes:

[0033] S51: for the grayscale image of the current frame, compare the grayscale value of each pixel point x with the N grayscale values ​​in its corresponding sample set Sample(x,y), and determine whether there are #min or more grayscale values ​​in the sample set Sample(x,y) whose two-dimensional Euclidean distance with the grayscale value of the pixel point x is less than the distance threshold R. If so, set the grayscale value of the pixel to 255, otherwise, set it to 0, #min represents the set foreground detection threshold, and N represents the number of samples;

[0034] S52: Perform morphological processing on the grayscale image processed by S51 to obtain a grayscale image after morphological processing.

[0035] Preferably, in S52, the step of morphological processing includes:

[0036] S521: Perform connectivity analysis on pixels with a gray value of 255 and calculate the size of the connected area;

[0037] S522: For pixel regions whose connected area is less than 9, the grayscale value of each pixel is set to 0;

[0038] S523: Perform erosion and dilation operations on the grayscale image to obtain a grayscale image after morphological processing.

[0039] Preferably, in S6, based on the inter-frame change rate obtained in S3, the step of updating the background model according to the inter-frame change rate adjustment strategy includes:

[0040] S61: Obtain a random factor θ according to the inter-frame change rate obtained in S3;

[0041] S62: Each pixel updates the background model according to the random factor θ.

[0042] Preferably, S62 includes:

[0043] All pixels whose grayscale values ​​are set to 0 have a probability of 1 / θ to update the sample set corresponding to themselves and their eight neighboring pixels. The specific update method is:

[0044] The grayscale value of the current pixel is randomly replaced with a grayscale value in the sample set corresponding to the current pixel;

[0045] The grayscale value of the current pixel is randomly replaced by a grayscale value in the sample set corresponding to the eight neighboring pixels of the current pixel.

[0046] According to another aspect of the present invention, a storage medium is provided, which is a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed, the steps of the improved ViBe indoor real-time foreground detection method are implemented.

[0047] The technical solution provided by the present invention has the following beneficial effects:

[0048] Without losing the real-time performance of the original ViBe algorithm, the concept of inter-frame change rate is introduced. The random factor adjustment strategy based on the inter-frame change rate can effectively suppress the generation of "ghosts" and holes caused by the spatiotemporal propagation mechanism, and enhance the detection ability of the algorithm to a large extent. In addition, the illumination detection method based on the inter-frame change rate and the perceptual hash algorithm enhances the robustness of the algorithm to illumination changes to a certain extent, and is more suitable for foreground detection in indoor environments than the original ViBe algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0050] Figure 1 It is a flow chart of an improved ViBe indoor real-time foreground detection method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, specific embodiments of the present invention are now described in detail with reference to the accompanying drawings.

[0052] Please refer to Figure 1 , an embodiment of the present invention provides an improved ViBe indoor real-time foreground detection method, comprising the following steps:

[0053] S1: Receive the video stream and decode and preprocess it to obtain the grayscale image of each frame of the video image.

[0054] S1 specifically includes:

[0055] S11: This embodiment uses the VideoCapture function of the OpenCV library to decode the input video and obtain each frame of the video image in time sequence;

[0056] S12: Convert the acquired video image into a grayscale image. The specific formula is as follows:

[0057] Gray=0.299*R+0.587*G+0.114*B

[0058] Wherein R, G, and B are the three primary colors of color images. This embodiment uses the cvtColor function of the OpenCV library to realize the conversion of color images to grayscale images.

[0059] S13: Use Gaussian filtering and median filtering in order to denoise the grayscale image. In this embodiment, Gaussian filtering uses a Gaussian kernel of size 5*5, and median filtering uses a filter template of size 5. The specific implementation method is to use GaussianBlur and medianBlur functions of the OpenCV library.

[0060] S2: Monitor the frame number of the current frame video image to determine whether it is the first frame of the video. If so, establish the background model of the foreground detection algorithm and enter S1. Otherwise, enter S3.

[0061] S2 specifically includes:

[0062] S21: Obtain the first grayscale image frame of the video, and construct a sample set with a maximum capacity of N for each pixel;

[0063] S22: Each pixel randomly selects the grayscale values ​​of the pixels in its eight neighborhoods and adds them to the sample set until the sample set capacity reaches the upper limit;

[0064] The expression of the sample set is as follows:

[0065] Sample(x,y)={Vi|i=1,2...N}

[0066] In the formula, Sample(x, y) is a sample set of pixels, (x, y) is the Cartesian coordinate value of any pixel, Vi is a sample, N is the number of samples, and the upper limit number N set in this embodiment is 20.

[0067] S3: Calculate the inter-frame change rate by using the current frame grayscale image and the previous frame grayscale image;

[0068] S3 specifically includes:

[0069] S31: Select the grayscale images of the current frame and the previous frame, and perform a differential operation on any pixel point according to the following formula to obtain a differential grayscale image:

[0070] D(x,y)=|f k (x,y)-f k-1 (x,y)|

[0071] Where D(x,y) is the obtained differential grayscale image, f k (x, y) is the grayscale image of the current frame, f k-1 (x, y) is the grayscale image of the previous frame;

[0072] S32: Determine whether the absolute value of the grayscale value of each pixel in the differential grayscale image is greater than a threshold value T. If yes, modify its grayscale value to 255; if no, set it to 0. In this embodiment, the threshold value T is set to 5, which can be set according to actual conditions.

[0073] S33: The number of pixels with a grayscale value of 255 in the statistical difference grayscale image is num1, the total number of pixels in the image is num2, and the frame-to-frame change rate P=num1 / num2×100% is calculated.

[0074] S4: Determine whether a sudden change in illumination occurs. If so, re-establish the background model in S2 and proceed to step S1. Otherwise, proceed to S5.

[0075] S4 specifically includes:

[0076] S41: Determine whether the value of the inter-frame change rate P calculated in S33 is greater than the threshold value T light If so, the process proceeds to S42, otherwise, it is determined that no sudden change in illumination has occurred. In this embodiment, the threshold T light Set it to 30%, which can be set according to actual situation;

[0077] S42: Use the perceptual hash algorithm to compare the grayscale image of the current frame with the differential grayscale image obtained in S31, and calculate the Hamming distance between the two images. In this embodiment, the threshold value T of the Hamming distance is enter Set to 10, which can be set according to actual conditions. If it is greater than the threshold, it is judged that no sudden change in illumination has occurred. Otherwise, it is judged that a sudden change in illumination has occurred.

[0078] S5: Obtaining the foreground detection result of the current frame video image;

[0079] S5 specifically includes:

[0080] S51: for the grayscale image of the current frame, the grayscale value of each pixel point x is compared with the N grayscale values ​​in its corresponding sample set Sample(x), and it is determined whether there are two or more grayscale values ​​in the sample set Sample(x, y) whose two-dimensional Euclidean distance with the grayscale value of pixel x is less than a threshold value of 20. If yes, the grayscale value of the pixel is set to 255, otherwise, it is set to 0. The threshold here can be set according to the actual situation;

[0081] S52: performing morphological processing on the grayscale image processed by S51;

[0082] In this embodiment, the findContours function of the OpenCV library is used to find all connected domains, and all connected domains are traversed. The contourArea function of the OpenCV library is used to calculate the area of ​​the connected domain. For all connected domains whose area is less than a threshold value of 9 (which can be set according to actual conditions), they are regarded as noise, and the gray value of each pixel is set to 0. After that, the grayscale image is eroded first and then expanded, specifically: the dilate and erode functions of the OpenCV library are used, and the convolution kernel size of the erosion and erode operations is set to 3*3.

[0083] S6: Based on the inter-frame change rate obtained in S3, the background model is updated according to the inter-frame change rate adjustment strategy;

[0084] S6 specifically includes:

[0085] S61: Obtain a random factor θ according to the inter-frame change rate obtained in S33, as shown in Table 1:

[0086] Table 1: Correspondence between frame rate of change and random factor

[0087]

[0088] The data in Table 1 are obtained through experiments and are applicable to most scenarios.

[0089] S62: All pixels with grayscale values ​​of 0 have a probability of 1 / θ to update their corresponding sample sets. The specific updating method is: randomly replace a grayscale value in the sample set corresponding to the current pixel with the grayscale value of the current pixel.

[0090] S63: All pixels with grayscale values ​​of 0 have a probability of 1 / θ to update the sample set corresponding to their neighboring pixels. The specific updating method is: randomly replace the grayscale value of the current pixel with a grayscale value in the sample set corresponding to the eight neighboring pixels of the current pixel.

[0091] S7: Determine whether there is a next frame of video image, if yes, go to S1, otherwise, the detection process ends. In this embodiment, the isOpened function of the OpenCV library is used to determine whether there is a next frame of video image.

[0092] In order to verify the beneficial effects of the present invention, this embodiment selects seven indoor subsets in the ChangeDetection2012 dataset and conducts comparative experiments with five commonly used foreground detection methods (mixed Gaussian model GMM, three-frame difference method TFD, nearest neighbor algorithm KNN, codebook algorithm codebook, visual background extractor algorithm ViBe), and the evaluation indicators are the percentage of wrong classifications PWC (Percentage of Wrong Classifications) and F-measure. The specific experimental results are shown in Tables 2 and 3:

[0093] Table 2 PWC comparison between the method of the present invention and five commonly used foreground detection methods

[0094]

[0095] Table 3 Comparison of F values ​​between the proposed method and five commonly used foreground detection methods

[0096]

[0097] In some embodiments, a storage medium is also provided, which is a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is executed, it is used to implement the steps of the improved ViBe indoor real-time foreground detection method.

[0098] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0099] The serial numbers of the embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In a unit claim that lists several means, several of these means may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order and these words may be interpreted as identifiers.

[0100] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An improved ViBe indoor real-time foreground detection method, characterized in that: The following steps are involved: S1: Receive the video stream and decode and preprocess it to obtain the grayscale image of each frame of the video image; S2: Monitor the frame number of the current frame video image to determine whether it is the first frame of the video. If so, establish the background model of the foreground detection algorithm and enter S1. Otherwise, enter S3. S3: Calculate the inter-frame change rate by using the current frame grayscale image and the previous frame grayscale image; S4: Determine whether a sudden change in illumination occurs. If so, re-establish the background model in S2 and proceed to step S1. Otherwise, proceed to S5. S5: Obtaining the foreground detection result of the current frame video image; S6: Based on the inter-frame change rate obtained in S3, the background model is updated according to the inter-frame change rate adjustment strategy; S61: Obtain a random factor θ according to the frame-to-frame change rate obtained in S3; S62: Each pixel updates the background model according to the random factor θ; all pixels whose grayscale values ​​are set to 0 have a probability of 1 / θ to update the sample set corresponding to itself and its eight neighboring pixels. The specific updating method is: The grayscale value of the current pixel is randomly replaced with a grayscale value in the sample set corresponding to the current pixel; the grayscale value of the current pixel is randomly replaced with a grayscale value in the sample set corresponding to the eight neighboring pixels of the current pixel; S7: Determine whether there is a next frame of video image. If so, go to S1, otherwise, the detection process ends.

2. The improved ViBe indoor real-time foreground detection method as claimed in claim 1, characterized in that: In S1, the steps of receiving, decoding and preprocessing the video stream include: S11: receiving and decoding the video stream to obtain a frame-by-frame video image queue; S12: performing grayscale conversion on a single-frame video image in the video image queue to obtain a grayscale image of the single-frame video image; S13: denoising the grayscale image using Gaussian and median filtering to obtain a denoised grayscale image.

3. The improved ViBe indoor real-time foreground detection method as claimed in claim 1, characterized in that: In S2, the steps of establishing the background model include: S21: Obtain the first grayscale image frame of the video, and construct a sample set with a maximum capacity of N for each pixel; S22: Each pixel randomly selects the grayscale values ​​of the pixels in its eight neighborhoods and adds them to the sample set until the sample set capacity reaches the upper limit; The expression of the sample set is as follows: Sample(x,y)={V i |i=1,2...N} In the formula, Sample(x,y) is the sample set of pixels, (x,y) is the Cartesian coordinate value of any pixel, V i is the sample, and N is the number of samples.

4. The improved ViBe indoor real-time foreground detection method as claimed in claim 1, characterized in that: In S3, the step of calculating the frame-to-frame change rate includes: S31: Select the current frame grayscale image and the previous frame grayscale image, perform difference operation on the pixel points at the same position in the Cartesian coordinate system and take their absolute values ​​to obtain a differential grayscale image. The formula is as follows: D(x,y)=|f k (x,y)-f k-1 (x,y)| In the formula, D(x,y) is the obtained differential grayscale image, f k (x, y) is the grayscale image of the current frame, f k-1 (x, y) is the grayscale image of the previous frame; S32: Determine whether the absolute value of the grayscale value of each pixel in the differential grayscale image is greater than a threshold value T, if so, modify its grayscale value to 255, otherwise, set it to 0; S33: The number of pixels with a grayscale value of 255 in the statistical difference grayscale image is num1, the total number of pixels in the image is num2, and the inter-frame change rate P=num1 / num2×100% is calculated.

5. The improved ViBe indoor real-time foreground detection method as claimed in claim 4, characterized in that: In S4, the method for determining whether a sudden illumination change occurs includes: S41: Determine whether the value of the inter-frame change rate P calculated in S33 is greater than the threshold value T light , if so, proceed to S42, otherwise, determine that no sudden change in illumination occurs; S42: Use the perceptual hash algorithm to compare the grayscale image of the current frame with the differential grayscale image obtained in S31 to determine whether the Hamming distance between the two images is greater than the threshold T enter If yes, it is judged that no sudden change in illumination has occurred; otherwise, it is judged that a sudden change in illumination has occurred.

6. The improved ViBe indoor real-time foreground detection method as claimed in claim 1, characterized in that: In S5, the step of obtaining the foreground detection result includes: S51: for the grayscale image of the current frame, compare the grayscale value of each pixel point x with the N grayscale values ​​in its corresponding sample set Sample(x,y), and determine whether there are #min or more grayscale values ​​in the sample set Sample(x,y) whose two-dimensional Euclidean distance with the grayscale value of the pixel point x is less than the distance threshold R. If so, set the grayscale value of the pixel to 255, otherwise, set it to 0, #min represents the set foreground detection threshold, and N represents the number of samples; S52: Perform morphological processing on the grayscale image processed by S51 to obtain a grayscale image after morphological processing.

7. The improved ViBe indoor real-time foreground detection method according to claim 6, characterized in that: In S52, the morphological processing step includes: S521: Perform connectivity analysis on pixels with a gray value of 255 and calculate the size of the connected area; S522: For pixel regions whose connected area is less than 9, the gray value of each pixel is set to 0; S523: Perform erosion and dilation operations on the grayscale image to obtain a grayscale image after morphological processing.

8. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, on which a computer program is stored. When the computer program is executed, the computer program is used to implement the steps of the improved ViBe indoor real-time foreground detection method as described in any one of claims 1 to 7.

Images