A background detection method for simultaneous modeling of foreground and background

Abstract

The present invention discloses a background detection method, including S1: receiving a background model for visual background extraction (ViBe) and a current frame from which high-frequency information including noise points, background changes, etc. is filtered out through integer DCT compression; S2: using an LBSP operator and classifying the pixel positions of the current frame into background point classes and foreground point classes according to a pixel classification threshold; S3: establishing and maintaining a foreground codebook for recording the features of a detected foreground object, and adjusting the pixel classification threshold and the background model update rate through the learning of foreground features. The method utilizes the background model in the original ViBe background detection method and introduces the LBSP operator into the pixel classification stage, establishes and maintains a foreground codebook for recording the features of a detected foreground object, and then adjusts the pixel classification threshold and the background model update rate through the learning process of foreground features, thereby obtaining a more accurate detection result.

Description

Technical Field

[0001] This patent relates to a background detection method, and more specifically to a new background detection method that establishes and maintains a "foreground codebook" that records the characteristics of foreground objects, optimizes the detection results through learning heuristics of foreground features, and specifically adjusts the classification threshold of each pixel and the background model update rate. Background Art

[0002] Background detection method is a general term for target detection methods that only reflect the detected target of interest in the detection results. As computer vision applications gradually develop and mature, high-level applications such as target tracking and target recognition have become the most basic requirements in certain application scenarios. Target detection is the premise and foundation of these high-level computer vision applications. Therefore, it is necessary to explore background detection methods with more accurate detection results, better real-time performance, and stronger robustness to meet the needs of high-level computer vision applications.

[0003] In the prior art, for example, the paper "ViBe: A universal background subtraction algorithm for video sequences, IEEE Trans. Image Process., vol. 20, no. 6, Jun. 2011 (author: O. Barnich and M. Van Droogenbroeck)" proposes a visual background extraction (ViBe) background detection method, which uses pixel values ​​as the only basis for building a model. In the background model update process, not only the pixel positions that need to update the background model are randomly selected, but also the new values ​​used to update the background model are obtained from its eight neighborhoods in a randomly selected manner. Therefore, the detection results of this background detection method are not accurate enough. The paper "SuBSENSE: AUniversal Change Detection Method With Local Adaptive Sensitivity, IEEE Transactions on Image Processing, Vol. 24, No. 1, January 2015 (Author: Pierre-Luc St-Charles, GABilodeau, R. Bergevin)" adds the LBSP operator to the background model and uses the eight-neighborhood spatial consistency in the model update phase, but only considers the spatial consistency between pixels in the pixel classification phase, and the scope of consideration of spatial consistency is limited to the eight-neighborhood space, which is not enough to obtain better detection results. More importantly, although this method takes into account the dynamic adjustment of the threshold and background model update rate when classifying a single pixel, it does not consider that other pixel positions with spatial similarity, color similarity, and texture similarity should also adjust the pixel classification threshold and model update rate. The paper "Background modeling and subtraction by codebook construction, in Proc. IEEE Int. Conf. Image Process., Singapore, Oct. 2004, vol. 5 (authors: K. Kim, T. Chalidabhongse, D. Harwood, and L. Davis)" constructs a "codebook" for all pixel positions based on their background feature values ​​and other information through a fairly long training video sequence. However, the background detection method using this method not only consumes a lot of memory but also cannot cope well with sudden changes in the scene because it cannot create new "codebook values".

[0004] In summary, although the background detection method in the prior art can provide detection results with certain effects, there is still much room for improvement. Summary of the invention

[0005] Based on the above technical problems, the present disclosure hopes to propose a background detection method with better detection effect, which can improve the insufficient accuracy of detection results of existing background detection methods, thereby obtaining better detection effect.

[0006] According to an exemplary aspect of the present disclosure, an improved background detection method includes:

[0007] S1: Receive the background model for visual background extraction (ViBe) and the current frame after integer DCT compression to filter out high-frequency information including noise, background changes, etc.;

[0008] S2: using pixel color and texture feature values ​​(local binary similarity (LBSP) operator), classifying the pixel position of the current frame into a background point class and a foreground point class according to each pixel classification threshold (each pixel has an independent classification threshold);

[0009] S3: Establish and maintain a "foreground codebook" to record the features of the detected foreground objects, and adjust the pixel classification threshold and background model update rate through learning the foreground features.

[0010] In one embodiment of the present disclosure, step S2: using pixel color and texture feature values ​​(local binary similarity (LBSP) operator) and classifying the pixels of the current frame into background point classes and foreground point classes according to the pixel classification threshold of each pixel further includes:

[0011] S21: When the pixel of the current frame satisfies the first predetermined condition and the second predetermined condition, the pixel of the current frame is classified as a background point class.

[0012] In one embodiment of the present disclosure, the step S2: using a local binary similarity pattern (LBSP) operator and classifying the pixels of the current frame into background point classes and foreground point classes according to a pixel classification threshold further includes:

[0013] S22: When the pixel of the current frame does not satisfy the first predetermined condition or does not satisfy the second predetermined condition, the pixel of the current frame is classified into a foreground point class.

[0014] In one embodiment of the present disclosure, the step S3: establishing a "foreground codebook" for recording the features of the detected foreground objects further comprises:

[0015] S31: Location features including historical center point location and volume radius, color features based on HSV color space, texture features obtained using the local binary similarity pattern (LBSP) operator, occurrence frequency, and the last occurrence time expressed in specific video frames.

[0016] In one embodiment of the present disclosure, the step S3: maintaining a "foreground codebook" for recording the features of the detected foreground objects further comprises:

[0017] S32: Update the "foreground codebook" using the latest detection results. The detection results of each frame will be compared with the records in the "foreground codebook". If the color and texture of the foreground object are similar to a record in the "foreground codebook" (determined by the pixel classification threshold of the area), the foreground object will be used to update the record in the "foreground codebook". If the color and texture of the foreground object are different from the records in the "foreground codebook", the features of the foreground object will be added to the "foreground codebook" as a new record.

[0018] In one implementation of the present disclosure, the background detection method step S3 further includes:

[0019] S33: Targetedly adjusting the pixel classification threshold and background model update rate by learning the foreground features, wherein:

[0020] S331: For an area where foreground objects often appear, the pixel classification threshold of the area should be reduced to a second threshold based on the first threshold, and the background model update rate of the area should be reduced to a second update rate based on the first update rate;

[0021] S332: Searching for a region around the detected foreground object that is similar to the foreground object in color and texture;

[0022] S333: For an area that is similar in color and texture to a detected foreground object, the pixel classification threshold of the area should be reduced based on the first threshold, and the background model update rate of the area should be reduced based on the first update rate.

[0023] In one embodiment of the present disclosure, the step S332 of specifically searching for an area around the detected foreground object that is similar to the foreground object in color and texture further includes:

[0024] S3321: After obtaining the preliminary detection result of the current frame, in an area with a radius of ζ around the determined foreground object, a third threshold value smaller than the first threshold value is used to specifically search for an area with a color and texture similar to the foreground object to reduce the missed detection rate.

[0025] In one embodiment of the present disclosure, the step S333: for an area similar to the detected foreground object in color and texture, the pixel classification threshold of the area should be reduced based on the first threshold, and the background model update rate of the area should be reduced based on the first update rate further includes:

[0026] S3331: For a region having a first behavior similarity in color and texture with a detected foreground object, the pixel classification threshold of the region should be reduced to a fourth threshold based on the first threshold, and the background model update rate of the region should be reduced to a third update rate based on the first update rate;

[0027] S3332: For an area that has a second behavioral similarity with a detected foreground object in color and texture, the pixel classification threshold of the area should be reduced to a fifth threshold on the basis of the first threshold, and the background model update rate of the area should be reduced to a fourth update rate on the basis of the first update rate, wherein the first behavioral similarity is higher than the second behavioral similarity and the fourth threshold and the third update rate are respectively smaller than the fifth threshold and the fourth update rate.

[0028] The background detection method according to the present disclosure utilizes the current frame that has been filtered out of high-frequency information including noise, background changes, etc. through integer DCT compression, and establishes and maintains a "foreground codebook" that records the characteristics of the detected foreground objects. Based on this, the pixel classification threshold and the background model update rate are adjusted in a targeted manner through learning the foreground features, so that the background detection method of the present disclosure can obtain more accurate detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A flowchart 100 showing an improved background detection method according to the present disclosure;

[0030] Figure 2 Shows the basis Figure 1 A block diagram of a background detection method 200;

[0031] Figure 3A block diagram 300 is shown for classifying each pixel position;

[0032] Figure 4 A block diagram 400 is shown of a dynamic process of establishing and maintaining a "foreground codebook" for recording the bit features of detected foreground objects, and adjusting the pixel classification threshold and background model update rate in a targeted manner through learning the foreground features. DETAILED DESCRIPTION

[0033] The background detection method proposed in the present disclosure establishes and maintains a "foreground codebook" that records the features of the detected foreground objects, and adjusts the pixel classification threshold and background model update rate of each pixel in a targeted manner through learning the foreground features. Specifically, during the detection of a new frame, the decision threshold and background model update rate of each pixel will be dynamically adjusted according to the records in the "foreground codebook". When adjusting, it is necessary to increase the pixel classification threshold Th for areas where foreground objects often appear. i，j (including color threshold Th i，j (color), texture threshold Th i，j (text) and position threshold Th i，j (dis), where:

[0034]

[0035]

[0036]

[0037] In the above formulas, δ color ,δ text and δ dis is the similarity factor representing the similarity, and: Dmin i，j =(1-β)·Dmin i，j +β·min{dis|FG(k)|},

[0038]

[0039] ) and the background model update rate, and at the same time, it specifically searches for areas with similar color and texture around the detected foreground object and increases the pixel classification threshold and background model update rate in the area.

[0040] By specifically searching for areas with similar color and texture around the detected foreground object, for some areas that meet the judgment criteria, these areas are similar to the detected foreground object in color and texture, but the similarity may be different. Some areas have a high similarity to the detected foreground object, while other areas have a low similarity to the detected foreground object. Therefore, dynamically adjusting the pixel classification threshold and background model update rate of the area that meets the judgment criteria around the detected foreground object is related to the similarity of the detected foreground object in color and texture.

[0041] As mentioned above, for all areas around a detected foreground object that meet the judgment conditions, the areas with a higher degree of similarity in color and texture with the detected foreground object will have their pixel classification threshold and background model update rate increased by a larger amount; conversely, the areas with a lower degree of similarity in color and texture with the detected foreground object will have their pixel classification threshold and background model update rate increased by a smaller amount.

[0042] In abstract terms, the background detection method according to the present disclosure learns the foreground target position, color and texture features, in order to use more accurate and targeted learning results to adjust the pixel classification threshold of each pixel and the background model update rate to obtain better detection results.

[0043] The background detection method according to the present disclosure will be described in further detail below in conjunction with the accompanying drawings. Figure 1 A flowchart 100 of an improved background detection method according to the present disclosure is shown. As can be seen from the figure, the improved background detection method 100 according to the present disclosure first receives a background model for visual background extraction (ViBe) and a current frame from which high-frequency information including noise, background changes, etc. is filtered out through integer DCT compression in method step 110; then, in the following method step 120, a local binary similarity pattern (LBSP) operator is used to classify the pixel positions of the current frame into background point classes and foreground point classes according to a pixel classification threshold; then, in method step 130, a "foreground codebook" is established and maintained for the detected foreground object to record its features, and the pixel classification threshold and the background model update rate are adjusted in a targeted manner through the learning of the foreground features. The background detection method according to the present disclosure establishes and maintains a "foreground codebook" for the detected foreground object to record its features, and the pixel classification threshold and the background model update rate are adjusted in a targeted manner through the learning of the foreground features, thereby obtaining a more accurate detection result.

[0044] The following will be combined Figure 2Further description based on Figure 1 Background detection method, Figure 2 Shows the basis Figure 1 A block diagram 200 of a background detection method is shown. As can be seen from the figure, the improved background detection method according to the present disclosure receives a background model for visual background extraction (ViBe) and a current frame from which high-frequency information including noise, background changes, etc. is filtered out through integer DCT compression, and then a pixel classification process is performed for each pixel position in the current frame. Finally, a "codebook" is established and maintained for recording the features of the detected foreground objects, and the pixel classification threshold and the background model update rate are adjusted in a targeted manner through learning the foreground features.

[0045] Next, Figure 3 A schematic diagram 300 of a block diagram for classifying each pixel position is shown. As can be seen from the figure, for the received background model and current frame, conditional judgment is performed on the pixel values ​​at the corresponding positions of the background model and the current frame. Specifically, when the pixel of the current frame meets the first predetermined condition and meets the second predetermined condition, the pixel of the current frame is classified as a background point class; when the pixel of the current frame does not meet the first predetermined condition or does not meet the second predetermined condition, the pixel of the current frame is classified as a foreground point class. Here, it can be seen from the figure that the first predetermined condition here is |PB k | <Th color , and the second predetermined condition is H(LBSP f , LBSP M )≤Th LBSP .

[0046] Figure 4 A block diagram 400 is shown of a process of establishing and maintaining a "foreground codebook" for recording the features of a detected foreground object, and adjusting the pixel classification threshold and the background model update rate by learning the foreground features. As can be seen from the figure, by learning the "foreground codebook" constructed by the features of the foreground object, and dynamically adjusting the color threshold Th based on the learning results, i，j (color), texture threshold Th i , j (text) and position threshold Th i，j ( dis ) and other pixel classification thresholds Th i，j (in:

[0047]

[0048]

[0049]

[0050] In the above formulas, δ color ,δ text and δ dis is the similarity factor representing the similarity, and:

[0051] Dmin i，j =(1-β)·Dmin i，j +β·min{dis|FG(k)|},

[0052]

[0053] ) and background model update rate, wherein the pixel classification threshold of the area where the foreground object often appears is reduced to the second threshold based on the first threshold, and the background model update rate of the area is reduced to the second update rate based on the first update rate. After obtaining the detection result of the current frame, the area with a radius of z around the detected foreground object is updated with the threshold value. ( d is the distance from each pixel to the nearest pixel of the foreground object) is used as the judgment condition to specifically look for areas with similar color and texture to the foreground object. For areas with a first behavioral similarity in color and texture to the detected foreground object, the pixel classification threshold of the area should be reduced to a fourth threshold on the basis of the first threshold, and the background model update rate of the area should be reduced to a third update rate on the basis of the first update rate; for areas with a second behavioral similarity in color and texture to the detected foreground object, the pixel classification threshold of the area should be reduced to a fifth threshold on the basis of the first threshold, and the background model update rate of the area should be reduced to a fourth update rate on the basis of the first update rate, wherein the first behavioral similarity is higher than the second behavioral similarity and the fourth threshold and the third update rate are respectively smaller than the fifth threshold and the fourth update rate.

[0054] The background detection method according to the content of the present disclosure combines the advantages of the two background detection methods mentioned in the background technology and overcomes their shortcomings. It utilizes the background model in the original ViBe background detection method and introduces the LBSP operator into the pixel classification stage. It establishes and maintains a "foreground codebook" (foreground codebook) that records the characteristics of the detected foreground objects, and then learns the foreground features and uses the learning results to dynamically adjust the pixel classification threshold and the update rate of the background model, so as to obtain more accurate detection results.

[0055] Although certain typical embodiments and details have been shown for the purpose of illustrating the present disclosure, it will be apparent to those skilled in the art that various changes may be made in the methods and apparatus disclosed herein without departing from the scope of the present disclosure.

Claims

1. A background detection method for simultaneously modeling foreground and background, the method comprising: S1: Receive the background model for visual background extraction and the current frame after integer DCT compression to filter out high-frequency information including noise and background changes; S2: using a local binary similarity operator and according to a pixel classification threshold, classifying the pixel positions of the current frame into a background point class and a foreground point class; S3: Establish and maintain a foreground codebook that records the features of the detected foreground objects, and adjust the pixel classification threshold and background model update rate in a targeted manner by learning the foreground features; The step S3: establishing a foreground codebook for recording the features of the detected foreground object further comprises: S31: location features including the historical center point location and volume radius, color features based on the HSV color space, texture features obtained using the local binary similarity pattern operator, occurrence frequency, and the last occurrence time feature represented by the number of video frames; S32: Update the foreground codebook using the latest detection results; the foreground object detected in each frame will be compared with the record in the foreground codebook. If the position, color and texture of the foreground object are the same as a record in the "foreground codebook", the record in the foreground codebook will be updated using the foreground object features. If the color and texture of the foreground object are different from the records in the foreground codebook, the features of the foreground object will be added to the foreground codebook as a new record.

2. The background detection method for simultaneous modeling of foreground and background according to claim 1, wherein: The step S2: using a local binary similarity pattern operator and classifying the pixels of the current frame into background point classes and foreground point classes according to a pixel classification threshold further comprises: S21: When the pixel of the current frame satisfies the first predetermined condition and the second predetermined condition, the pixel of the current frame is classified as a background point class.

3. The background detection method for simultaneous modeling of foreground and background according to claim 1 or 2, wherein: The step S2: using a local binary similarity pattern operator and classifying the pixels of the current frame into background point classes and foreground point classes according to a pixel classification threshold further comprises: S22: When the pixel of the current frame does not satisfy the first predetermined condition or does not satisfy the second predetermined condition, the pixel of the current frame is classified into a foreground point class.

Images