A moving target detection method based on visual brain network background modeling

Abstract

This invention discloses a moving target detection method based on visual brain network background modeling. First, it utilizes the visual brain network mechanism to acquire optical flow information from the visual cortex, and then divides video frames into motion candidate regions and background regions based on this information. A background model is then constructed using the background regions. Next, the frame to be detected is subtracted from the background model to obtain a binary foreground. Finally, a mask is created using the binary foreground to extract moving targets and update the background model. Compared with traditional background modeling algorithms, this invention's moving target detection method effectively overcomes problems such as ghosting and fragmentation, and the detected targets are more complete. Compared with deep learning-based target detection algorithms, this invention's method does not require data for training.

Description

Technical Field

[0001] This invention relates to the field of moving target detection technology, and in particular to a method for detecting moving targets in a fixed scene. Background Technology

[0002] Moving target detection algorithms are the most fundamental and crucial part of intelligent video analysis. They primarily refer to techniques that automatically detect continuously moving targets from the background in a video frame sequence by defining corresponding mathematical models and segmentation algorithms. Existing traditional methods are mainly divided into three categories: inter-frame differencing, optical flow, and background modeling. Inter-frame differencing obtains the contour of a moving target by performing difference operations on adjacent frames; however, it cannot extract the complete area of ​​the object, only its boundaries. Optical flow detects moving targets by estimating the optical flow field of the video image, but it is computationally intensive, time-consuming, and sensitive to lighting conditions. Background modeling compares the current frame in the image sequence with a background reference model to detect moving objects; it features fast detection speed and relatively complete target detection. Background modeling methods mainly include codebook methods, Gaussian mixture models, and pixel classification, but existing background modeling algorithms suffer from fragmentation and ghosting issues.

[0003] Deep learning-based methods utilize massive amounts of training data to obtain nonlinear mappings of complex backgrounds and foregrounds in video frames, enabling the detection of moving targets. However, training deep learning algorithms requires substantial data and computational resources, while data on moving targets in natural scenes is typically scarce and difficult to obtain. This presents a data shortage challenge for the research and development of deep learning-based motion target perception and detection algorithms. Furthermore, extensive parameter tuning is necessary during training to prevent overfitting or underfitting. Secondly, motion target detection needs to be performed under various lighting and pose conditions, requiring optimization and adjustment for different scenarios and conditions. Currently, most motion target detection algorithms are optimized for specific scenarios and conditions, thus requiring specialized adjustments and optimizations for other scenarios and conditions. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a moving target detection method based on visual brain network background modeling, so as to solve the fragmentation and ghosting phenomena existing in the existing background modeling algorithms.

[0005] The moving target detection method based on visual brain network background modeling of this invention includes the following steps:

[0006] (i) Utilizing visual brain network mechanisms to acquire optical flow information from the primary visual cortex, including:

[0007] 1) Using a three-dimensional Gaussian filter function f r(x,y,t) simulates the response of a simple V1 cell in the primary visual cortex to an input scale stimulus I. r Filtering operations for (x,y,t); I r The description of (x,y,t) is as follows:

[0008]

[0009] In the formula: * represents convolution operation; I r (x,y,t) represents the input video sequence, that is, the stimulus at the spatial location (x,y) at time t. The subscript r represents the spatiotemporal scale, r = 0, 1, 2, and r = 0 represents the original input.

[0010] The filtering operation is described as follows:

[0011]

[0012] In the formula: σ v1simple Let be the receptive field radius of a simple cell in the primary visual cortex V1; then the linear response of a simple neuron in the primary visual cortex V1 at spatial location (x, y), scale r, and spatiotemporal direction k can be described as follows:

[0013]

[0014] In the formula: ! represents factorial operation, T = 0, 1, 2, 3, Y = 0, 1, ..., 3-T, X = 3-YT, α v1lin Scale factor; unit vector To describe the spatiotemporal direction corresponding to the filter, where k = 1, 2, 3, ..., 28;

[0015] Constructing the output response S of simple cells in the primary visual cortex V1 kr :

[0016]

[0017] In the above formula, α filt→rate,r It maps the dimensionless filter response value to a more meaningful discharge frequency; α v1semi α is the half-saturation constant. v1rect σ is the scaling factor. v1norm The radius is Gaussian.

[0018] The average firing frequency C of V1 neurons in the primary visual cortex kr The calculation is as follows:

[0019]

[0020] In the above formula, α v1comp σ is the scaling factor. v1comp The radius is Gaussian.

[0021] 2) The neuronal characteristics of the MT layer are described using the IZH neuron model. The MT layer consists of CDS cells and PDS cells. Within its receptive field, the MT layer receives pulse information from the primary visual cortex V1 and utilizes the direction selectivity of MT layer cells to acquire optical flow motion information. The connection weights from the complex cells of the primary visual cortex V1 to the MT layer cells are set as follows:

[0022]

[0023] In the above formula, b r M represents the differential component of simple cells V1 in the primary visual cortex. Partially corresponding 28×28 matrices; Value product A set of interpolation weights can be used Representation, where the matrix The kth element The connection strength from the k-th V1 complex cell to the CDS cell is defined; a connection is only established when two cells are located at the same pixel position (x, y); the connection strength is defined for the motion direction θ. CDS Selective CDS cells (x CDS ,y CDS ) and the direction of motion θ PDS Selective PDS cells (x PDS ,y PDS The connection weights are designed as follows:

[0024]

[0025] In the above formula, Δθ=θ PDS -θ CDS Δx=x PDS -x CDS Δy=y PDS -y CDS ,σ PDS,pool α is the radius of the Gaussian neighborhood; CDS→PDS Δθ is the scaling factor; if |Δθ|>π / 2, the resulting connection weights will be negative, so the connections are passed to a population of inhibitory interneurons. Tuning normalization is achieved by passing inhibitory projections to the pool of inhibitory interneurons.

[0026]

[0027] In the above formula, σ PDS,tuned,dir <45°, σ PDS,tuned,loc =2;

[0028] Optical flow motion information of the image at pixel (x, y) The calculation is as follows:

[0029]

[0030] Where, r θ (x,y) represents the selective firing frequency of the MT PDS neuron at spatial location (x,y) in response to the direction of motion θ. θ It is a unit vector pointing in the direction of θ;

[0031] (ii) Obtaining the background sequence and constructing the background model:

[0032] Using the optical flow motion information obtained in step one), the image is divided into a background region and a candidate region where moving targets exist using the following visual cortex motion perception model;

[0033]

[0034] Where ||·|| denotes the Euclidean second norm, F threshold For the set discharge threshold, I t (x, y) represents the current frame image in the video sequence at time t. This refers to the optical flow motion information at the (x, y) pixel in the current frame image; when the pulse frequency of the MT layer cells reaches the set discharge threshold F... threshold When this happens, the current pixel is considered to be part A of the dynamic region. o Conversely, it is considered to be static background part A. bg ;

[0035] Continuous video frames are input into the visual cortex motion perception model to obtain N frames of background images without moving targets. The mean method is used to integrate the background regions to obtain a complete initial background model B.

[0036] (iii) Obtaining binary foreground and target extraction:

[0037] By comparing the frame to be detected with the already constructed background, a binary foreground can be obtained.

[0038]

[0039] Let represent the pixel value of the video frame image to be detected at position (x,y) at time t, and let B(x,y) represent the pixel value of point (x,y) in the background model image; T represents the threshold, and D... t (x,y) represents the foreground image; using binary foreground D t (x,y) and the current frame I t Perform an AND operation on (x,y) to extract the moving target.

[0040] Furthermore, the moving target detection method based on visual brain network background modeling also includes updating the background model using the following formula:

[0041]

[0042] In the formula, α and β are both real numbers between 0 and 1.

[0043] Furthermore, in step three), an adaptive threshold selection strategy that integrates the maximum inter-class difference and the minimum intra-class dispersion is adopted to select the threshold T.

[0044] The beneficial effects of this invention are:

[0045] (1) The present invention is a motion target detection method based on visual brain network background modeling. It utilizes the visual information processing mechanism of the primary visual cortex V1 and the temporal lobe MT in the brain, and the constructed model has biological rationality.

[0046] (2) This invention provides a moving target detection method based on visual brain network background modeling. It utilizes optical flow information from the visual cortex to dynamically segment video images, distinguishing dynamic parts (such as moving objects) from static parts (such as the background), and identifying the motion and background regions. The background regions are then integrated to generate an initial background model. Combined with background differencing, this method accurately extracts moving targets. Compared to traditional background modeling algorithms, this invention effectively overcomes problems such as ghosting and fragmentation, and detects relatively complete targets. Compared to deep learning-based target detection algorithms, this invention does not require data for training.

[0047] (3) The moving target detection method based on visual brain network background modeling of the present invention adopts an adaptive threshold selection strategy of maximum inter-class difference and minimum intra-class dispersion to eliminate noise and false targets in the foreground image; effectively overcomes the situation that when the threshold is fixed, if the threshold is too high, some foreground objects will be incorrectly identified as background, and if the threshold is too low, a large number of noise points will appear. Attached Figure Description

[0048] Figure 1 Flowchart of a moving target detection method based on visual brain network background modeling;

[0049] Figure 2 This is a diagram showing the firing status of MT neurons;

[0050] Figure 3 For visual cortical optical flow information;

[0051] Figure 4 Region segmentation based on visual cortical optical flow information;

[0052] Figure 5Background model;

[0053] Figure 6 The frame to be detected and the binary foreground;

[0054] Figure 7 For the extracted moving target. Detailed Implementation

[0055] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0056] The moving target detection method based on visual brain network background modeling in this embodiment includes the following steps:

[0057] (i) Utilizing visual brain network mechanisms to acquire optical flow information from the primary visual cortex, including:

[0058] 1) Using a three-dimensional Gaussian filter function f r (x,y,t) simulates the response of a simple V1 cell in the primary visual cortex to an input scale stimulus I. r Filtering operations for (x,y,t); I r The description of (x,y,t) is as follows:

[0059]

[0060] In the formula: * represents convolution operation; I r (x,y,t) represents the input video sequence, i.e. the stimulus at spatial location (x,y) at time t. The subscript r represents the spatiotemporal scale, r = 0, 1, 2, and r = 0 represents the original input.

[0061] The filtering operation is described as follows:

[0062]

[0063] In the formula: σ v1simple σ represents the receptive field radius of a simple cell in the primary visual cortex V1. v1simple = 1.25 pixels; then the linear response of a simple neuron in the primary visual cortex V1 at spatial location (x,y), scale r, and spatiotemporal direction k can be described as follows:

[0064]

[0065] In the formula: ! represents factorial operation, T = 0, 1, 2, 3, Y = 0, 1, ..., 3-T, X = 3-YT, α v1lin Scale factor; unit vector The formula describes the spatiotemporal direction corresponding to the filter, where k = 1, 2, 3, ..., 28.

[0066] When the position (x, y) is close to the edge of the image, the response of the simple cells V1 in the primary visual cortex is L. kr It can become very large; therefore, when the spatial location is close to the image boundary, a scaling factor is needed to suppress L. kr Edge effect, through L kr The output response S of simple cells in the primary visual cortex V1 was constructed by using the linear response within the half-square sum of squares and the normalized large Gaussian envelope. kr :

[0067]

[0068] In the above formula, α filt→rate,r This maps the dimensionless filter response value to a more meaningful discharge frequency, set to 15Hz; α v1semi α is the half-saturation constant. v1semi =0.1, α v1rect α is the scaling factor. v1rect =1.9263, σ v1norm Let σ be the Gaussian radius. v1norm = 1 pixel.

[0069] The average firing frequency C of V1 neurons in the primary visual cortex kr The calculation is as follows:

[0070]

[0071] In the above formula, α v1comp α is the scaling factor. v1comp =0.1, σ v1comp Let σ be the Gaussian radius. v1comp =1.6.

[0072] 2) The neuronal characteristics of the MT layer are described using the IZH neuron model. The MT layer consists of CDS cells and PDS cells. Within its receptive field, the MT layer receives pulse information from the primary visual cortex V1 and utilizes the direction selectivity of MT layer cells to acquire optical flow motion information. The connection weights from the complex cells of the primary visual cortex V1 to the MT layer cells are set as follows:

[0073]

[0074] In the above formula, b r M represents the differential component of simple cells V1 in the primary visual cortex. Partially corresponding 28×28 matrices; Value product A set of interpolation weights can be used Representation, where the matrix The kth element The connection strength from the k-th V1 complex cell to the CDS cell is defined; a connection is only established when two cells are located at the same pixel position (x, y); the connection strength is defined for the motion direction θ. CDS Selective CDS cells (x CDS ,y CDS ) and the direction of motion θ PDS Selective PDS cells (x PDS ,y PDS The connection weights are designed as follows:

[0075]

[0076] In the above formula, Δθ=θ PDS -θ CDS Δx=x PDS -x CDS Δy=y PDS -y CDS ,σ PDS,pool Let σ be the radius of the Gaussian neighborhood. PDS,pool = 3 pixels; α CDS→PDS Δθ is the scaling factor; if Δθ > π², the resulting connection weights will be negative, so the connections are passed to a population of inhibitory interneurons. Tuning normalization is achieved by passing inhibitory projections to the pool of inhibitory interneurons.

[0077]

[0078] In the above formula, σ PDS,tuned,dir <45°, so only one of the eight subgroups is activated; σ PDS,tuned,loc =2-pixel inhibitory populations will be linked back to PDS cells one-to-one.

[0079] Optical flow motion information of the image at pixel (x, y) The calculation is as follows:

[0080]

[0081] Where, r θ (x,y) represents the selective firing frequency of the MT PDS neuron at spatial location (x,y) in response to the direction of motion θ. θ It is a unit vector pointing in the θ direction; here, the angle of the vector represents the direction of motion, and the length of the vector represents the confidence level of the direction judgment, rather than the velocity commonly found in optical flow fields.

[0082] Neuronal responses in the MT layer, such as Figure 2 As shown, in this embodiment, 8 neurons are assigned to each pixel, and the number of pulses in each direction represents the confidence level of motion in that direction. The optical flow information integrated by the MT neurons is as follows: Figure 3 As shown, the optical flow information extracted from the pulse response of the MT is consistent with the actual motion information.

[0083] (ii) Obtaining the background sequence and constructing the background model:

[0084] Using the optical flow motion information obtained in step one), the image is divided into a background region and a candidate region where moving targets exist using the following visual cortex motion perception model;

[0085]

[0086] Where · represents the Euclidean second norm, F threshold For the set discharge threshold, I t (x, y) represents the current frame image in the video sequence at time t. This refers to the optical flow motion information at the (x, y) pixel in the current frame image; when the pulse frequency of the MT layer cells reaches the set discharge threshold F... threshold When this happens, the current pixel is considered to be part A of the dynamic region. o Conversely, it is considered to be static background part A. bg .

[0087] Continuous video frames are input into the visual cortex motion perception model to obtain N frames of background images without moving targets. The background regions are then integrated using the mean method to obtain a complete initial background model B(x,y).

[0088] In this embodiment, the first 30 frames of the video sequence are divided into background regions and candidate regions where moving targets exist, using optical flow motion information from the visual cortex. Figure 4 As shown, the candidate regions from 30 frames are integrated using the mean method to construct a "clean" background model without moving targets, as shown. Figure 5 As shown.

[0089] (iii) Obtaining binary foreground and target extraction:

[0090] By comparing the frame to be detected with the already constructed background, a binary foreground can be obtained.

[0091]

[0092] Let represent the pixel value of the video frame image to be detected at position (x,y) at time t, and let B(x,y) represent the pixel value of point (x,y) in the background image; T represents the threshold, and D... t (x,y) represents the foreground image; the threshold T is selected using an adaptive threshold selection strategy that fuses the maximum inter-class difference and the minimum intra-class dispersion to eliminate noise and false targets in the foreground image; the binary foreground D is used. t(x,y) and the current frame I t Perform a bitwise AND operation on (x, y) to extract moving targets. The frame to be detected and the extracted binary foreground are shown below. Figure 6 As shown in the figure. The extracted binary foreground is then used to create a mask for extracting moving targets, with the effect shown in the figure. Figure 7 As shown.

[0093] As an improvement to the above embodiments, the moving target detection method based on visual brain network background modeling further includes updating the background model using the following formula:

[0094]

[0095] In the formula, α and β are real numbers between 0 and 1. The learning rates α and β determine the fusion ratio of the foreground target and the update speed of the background, respectively. Together, they determine the influence of the current frame image on the background model and the update speed of the background model. When updating the background model, the static parts of the background are updated quickly to adapt to dynamic changes in the environment such as lighting in the scene. At the same time, the background image of pixels classified as foreground is updated slowly to prevent foreground pixels from eroding the background model, effectively solving the problem of targets being blended into the background due to prolonged inactivity. Because the background in the scene changes due to lighting, environment, etc., background updates ensure the accuracy and robustness of the background model.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A moving target detection method based on visual brain network background modeling, characterized in that: Including the following steps: (i) Utilizing visual brain network mechanisms to acquire optical flow information from the primary visual cortex, including: 1) Using a three-dimensional Gaussian filter function f r (x,y,t) simulates the response of a simple V1 cell in the primary visual cortex to an input scale stimulus I. r Filtering operations for (x,y,t); I r The description of (x,y,t) is as follows: In the formula: * represents convolution operation; I r (x,y,t) represents the input video sequence, that is, the stimulus at the spatial location (x,y) at time t. The subscript r represents the spatiotemporal scale, r = 0, 1, 2, and r = 0 represents the original input. The filtering operation is described as follows: In the formula: σ v1simple Let be the receptive field radius of a simple cell in the primary visual cortex V1; then the linear response of a simple neuron in the primary visual cortex V1 at spatial location (x, y), scale r, and spatiotemporal direction k can be described as follows: In the formula: ! represents factorial operation, T = 0, 1, 2, 3, Y = 0, 1, ..., 3-T, X = 3-YT, α v1lin Scale factor; unit vector To describe the spatiotemporal direction corresponding to the filter, where k = 1, 2, 3, ..., 28; Constructing the output response S of simple cells in the primary visual cortex V1 kr : In the above formula, α filt→rate,r It maps the dimensionless filter response value to a more meaningful discharge frequency; α v1semi α is the half-saturation constant. v1rect σ is the scaling factor. v1norm The radius is Gaussian. The average firing frequency C of V1 neurons in the primary visual cortex kr The calculation is as follows: In the above formula, α v1comp σ is the scaling factor. v1comp The radius is Gaussian. 2) The neuronal characteristics of the MT layer are described using the IZH neuron model. The MT layer consists of CDS cells and PDS cells. Within its receptive field, the MT layer receives pulse information from the primary visual cortex V1 and utilizes the direction selectivity of MT layer cells to acquire optical flow motion information. The connection weights from the complex cells of the primary visual cortex V1 to the MT layer cells are set as follows: In the above formula, b r M represents the differential component of simple cells V1 in the primary visual cortex. Partially corresponding 28×28 matrices; Value product A set of interpolation weights can be used Representation, where the matrix The kth element The connection strength from the k-th V1 complex cell to the CDS cell is defined; a connection is only established when two cells are located at the same pixel position (x, y); the connection strength is defined for the motion direction θ. CDS Selective CDS cells (x CDS ,y CDS ) and the direction of motion θ PDS Selective PDS cells (x PDS ,y PDS The connection weights are designed as follows: In the above formula, Δθ=θ PDS -θ CDS Δx=x PDS -x CDS Δy=y PDS -y CDS ,σ PDS,pool α is the radius of the Gaussian neighborhood; CDS→PDS Δθ is the scaling factor; if |Δθ|>π / 2, the resulting connection weights will be negative, so the connections are passed to a population of inhibitory interneurons. Tuning normalization is achieved by passing inhibitory projections to the pool of inhibitory interneurons. In the above formula, σ PDS,tuned,dir <45°, σ PDS,tuned,loc =2; Optical flow motion information of the image at pixel (x, y) The calculation is as follows: Where, r θ (x,y) represents the selective firing frequency of the MT PDS neuron at spatial location (x,y) in response to the direction of motion θ. θ It is a unit vector pointing in the direction of θ; (ii) Obtaining the background sequence and constructing the background model: Using the optical flow motion information obtained in step one), the image is divided into a background region and a candidate region where moving targets exist using the following visual cortex motion perception model; Where ||·|| denotes the Euclidean second norm, F thrshold For the set discharge threshold, I t (x, y) represents the current frame image in the video sequence at time t. This refers to the optical flow motion information at the (x, y) pixel in the current frame image; when the pulse frequency of the MT layer cells reaches the set discharge threshold F... thrshold When this happens, the current pixel is considered to be part A of the dynamic region. o Conversely, it is considered to be static background part A. bg ; Continuous video frames are input into the visual cortex motion perception model to obtain N frames of background images without moving targets. The mean method is used to integrate the background regions to obtain a complete initial background model B. (iii) Obtaining binary foreground and target extraction: By comparing the frame to be detected with the already constructed background, a binary foreground can be obtained. Let represent the pixel value of the video frame image to be detected at position (x,y) at time t, and let B(x,y) represent the pixel value of point (x,y) in the background model image; T represents the threshold, and D... t (x,y) represents the foreground image; using binary foreground D t (x,y) and the current frame I t Perform an AND operation on (x,y) to extract the moving target.

2. The moving target detection method based on visual brain network background modeling according to claim 1, characterized in that: This also includes updating the background model using the following formula: In the formula, α and β are both real numbers between 0 and 1.

3. The moving target detection method based on visual brain network background modeling according to claim 1, characterized in that: In step three), an adaptive threshold selection strategy that integrates the maximum inter-class difference and the minimum intra-class dispersion is used to select the threshold T.

Images