A night small animal detection method based on trajectory search

Abstract

The present application provides a kind of night small animal detection method based on trajectory search, comprising the following steps: background modeling;Foreground calculation;Merging extraction pipeline;Create speed matching model;Speed mode matching;Moving direction mode matching;Reclassification.The present application can greatly improve the detection rate without relying on the appearance characteristics of the target, and can greatly reduce the false detection caused by light and the like by using the combination of pattern matching and deep learning classifier.The method also has the advantages of fast running speed, good real-time performance and the like, and only needs 500 milliseconds to detect a 15-second video, with an accuracy of more than 95% and a detection rate of more than 95%.From the comprehensive evaluation of running speed and accuracy, the accuracy and running efficiency far exceed the use of deep learning such as yolo, ssd, faster-rcnn and the like framework.

Description

TECHNICAL FIELD

[0001] The present application relates to a night small animal presence detection method based on trajectory search. BACKGROUND

[0002] The hygiene condition of catering industry has been concerned by people, and from many exposed examples of kitchen mouse presence, the actual hygiene condition is difficult to satisfy consumers. A large part of the reason for this phenomenon is that the small animal presence in the catering kitchen mostly occurs at night when no one is present, and the monitoring range of this time period is relatively long and the night visibility is very low, and if manual screening is used, a lot of energy is consumed, which can be said to be difficult to achieve.

[0003] With the development and progress of computer vision technology and machine learning, it becomes more and more possible to automatically identify events through monitoring, such as pedestrian detection, face recognition, etc. These detection technologies can be divided into two categories:

[0004] 1. Traditional moving target detection technology, which is represented by frame difference method, background difference method, and optical flow method. The characteristics are simple calculation and strong real-time performance, but it cannot deal with too complex scenes.

[0005] 2. Target detection technology based on deep learning, which is represented by Faster-RCNN, YOLO, SSD, etc. The characteristics are strong adaptability to complex scenes, but the target to be detected requires obvious distinguishable features from the background.

[0006] Current target detection technology is difficult to use in mouse detection in night vision scenes, and there are two difficulties:

[0007] 1. The target to be detected in the night scene is blurred, and the distinguishable features are not obvious, especially the boundary profile of the mouse is easily integrated with the background, which makes the target detection technology based on deep learning have very low detection rate and low feasibility.

[0008] 2. Various disturbances may occur in the night scene, such as moths, light changes, wind, etc., which makes the traditional moving target detection technology cause a lot of false detections. SUMMARY

[0009] The present application aims to propose a technology based on Gaussian mixture background modeling, which measures the matching degree of target movement mode and preset movement mode based on KL divergence distance, and finally uses a classification model to detect the mouse trajectory. This technology has a very high detection rate for blurred targets under night infrared, and can significantly reduce the false detections caused by various disturbances.

[0010] The present application aims to overcome the shortcomings of the prior art and provide a target detection technology based on mixed Gaussian background modeling and classifier for features that are not obvious.

[0011] The specific technical solutions of the present application are as follows:

[0012] A night small animal appearing and disappearing detection method based on trajectory search comprises the following steps:

[0013] Step one: background modeling: using the mixed Gaussian model method to do background modeling. For a video, first use the first frame picture T1 to initialize the background, when there is a new picture frame T i , use the mixed Gaussian model method to update the background, at this time the background is B i .

[0014] Step two: foreground calculation: for T i frame image, use the mixed Gaussian model method to calculate the matching relationship between the pixel and the current background B i , if matched, the pixel is background, otherwise the pixel belongs to foreground, the set of all T i frame foreground pixels is the foreground F i . Then use the three steps of expansion, corrosion and expansion to remove noise points.

[0015] Step three: merging and extracting pipeline: for the foreground F i of the i-th frame picture, first use connected region analysis to find different connected regions, and according to the different positions on the picture, record them as a set {Q}. For any region Q i , if the position of the region overlaps with the position of a region Q j on the i-1-th frame image, it is added to the pipeline T j where Q i is located; otherwise, a new pipeline T is built.

[0016] Step four: creating a speed matching model: for any pipeline T n , calculate the moving distance of the target between each frame, record it as a set D={d i}, and regard the set as a uniform probability distribution Q s . Calculate the mean and variance of D, and record them as M D and G D respectively. According to M D and G D , a normal distribution model P G is established.

[0017] Step five: speed mode matching: use KL divergence to calculate the distance between Q s and P G , the calculation formula is The value of H(p,q) does not exceed the preset value H yThe movement of the living target has continuity, and the movement conforms to all or part of acceleration, uniform speed and deceleration. Therefore, the speed of the target should be close to the Gaussian distribution of its own speed, that is, the movement mode of the target should be a whole or part of Gaussian distribution, and the speed between frames should not change dramatically.

[0018] Step six: movement direction mode matching: for any pipe T n , the direction change C = {C i} of the target in each frame during movement is calculated, the mean of C is not more than a preset value M c , and the variance of C is not more than a preset value G c . The premise assumption is that the movement direction of the target will not change dramatically between adjacent frames.

[0019] Step seven: reclassification: if the speed mode and the movement direction mode can be successfully matched, it is judged that small animal movement is detected. For the pipe T n which cannot be determined by movement mode matching, all foreground images M = {M i} in the pipe are extracted, and the foreground images M = {M i} are classified using a classification model, which includes N layers of convolution, m layers of full connection and a classification layer, as shown in Figure 2 . Assuming that the classification preset threshold is T, if there are n pictures in M = {M i} greater than the preset threshold T, the pipe is determined as a small animal movement pipe.

[0020] For the detection of small targets such as rats at night, there is a key problem, that is, the mouse target is small, the characteristics are not obvious at night, and there are many interferences. This makes it difficult for both target detection based on deep learning and simple background modeling to be well applied. The present application proposes a night small animal detection technology combining background modeling technology and pattern matching, and finally combining a deep learning classifier for screening. This method does not rely on the appearance characteristics of the target, can greatly improve the detection rate, and can greatly reduce the false detection caused by light and the like by combining pattern matching and deep learning classifier. In addition to this, this method also has the advantages of fast running speed and good real-time performance, and only 500 milliseconds are needed to detect a 15-second video, with an accuracy of more than 95% and a detection rate of more than 95%. From the comprehensive evaluation of running speed and accuracy, the accuracy and running efficiency far exceed the use of deep learning frameworks such as yolo, ssd, faster-rcnn, etc. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 is a schematic diagram of a night small animal detection method based on trajectory search.

[0022] Figure 2is a schematic diagram of a step seven classification model of the present application. DETAILED DESCRIPTION

[0023] The present application will be further described in conjunction with the accompanying drawings and specific embodiments.

[0024] A trajectory search-based kitchen night mouse detection method, as shown in the figure, has the following steps: Figure 1

[0025] Step one: background modeling: use a mixed Gaussian model to model the background, the principle is that each pixel of the background image is modeled by a mixed Gaussian model composed of K Gaussian distributions, that is:

[0026]

[0027] x i represents the value of pixel j at time t, represents the estimated value of the weight coefficient of the i-th Gaussian distribution in the mixed Gaussian model at time t, and respectively represent the mean vector and the covariance matrix of the i-th Gaussian distribution in the mixed Gaussian model at time t; η represents the Gaussian distribution probability density function.

[0028] Parameter initialization: initialize the first Gaussian distribution corresponding to each pixel in the first frame image, assign the mean value as the current pixel value, and assign the weight value as 1. The mean and weight values of the Gaussian distribution functions other than the first one are initialized to zero.

[0029] Parameter update: at time t, match each pixel x t of the image frame with its corresponding Gaussian model.

[0030] The matching rule is: if the pixel value x t is less than 2.5 times the standard deviation of the mean of the i-th Gaussian distribution G i in the mixed Gaussian model, then define the Gaussian distribution G i as matching the pixel value x t .

[0031] If it is found that at least one Gaussian distribution in the mixed Gaussian model of the pixel matches the pixel value x t , then the parameter update rule of the mixed Gaussian model is:

[0032] 1) For the Gaussian distributions that do not match, their mean μ and covariance matrix ∑ remain unchanged;

[0033] 2) The mean μ and covariance matrix ∑ of the matching Gaussian distribution G i are updated according to the following formula:

[0034] ​μ i，t = (1 - p) - μ it-1 + p - X t ,

[0035] ∑ it = (1 - p) - ∑ it-1 + p - diag[(X t - μ it ) T (X t - μ it )],

[0036]

[0037] α is the learning rate of parameter estimation.

[0038] If there is no Gaussian distribution in the mixture Gaussian model corresponding to the pixel that matches the pixel value x t , the Gaussian distribution G j least likely to represent the background process is re-assigned.

[0039] Step two foreground calculation: for the newly incoming i-th frame image at time t, the matching relationship between each pixel value x t and the first B Gaussian distributions obtained is calculated. If the pixel value x t matches one of the first B Gaussian distributions, the pixel is a background point, otherwise the pixel is classified as foreground. In this way, the task of separating the target moving object from the background is completed.

[0040] Then the three-step method of dilation, erosion and dilation is used to remove noise points. The kernel size of dilation and erosion is in the range of 2-10.

[0041] Step three merging and extracting the pipeline: for the foreground F i of the i-th frame image, first use connected region analysis to find different connected regions, and according to the different positions on the image, record them as a set {Q}. For any region Q i , if the position of the region overlaps with the position of a region Q j on the i-1-th frame image, it is added to the pipeline T j where the i-1-th frame region Q i is located; otherwise, a new pipeline T is created.

[0042] There are two definitions of connected region: 4-adjacency and 8-adjacency. There are two basic methods of connected region analysis: 1) Two-Pass, which finds and labels all connected regions in the image by scanning the image twice. 2) Seed-Filling, which is based on region growing method, that is, recursive traversal. In this embodiment, we experiment with 4-adjacency and 8-adjacency, and two connected region analysis methods, and the results show that the influence on the accuracy and speed of detection can be ignored, that is, any connected region analysis method can realize the method.

[0043] Step four: create a speed matching model: for any pipe T n , calculate the distance of the target moving between each frame, denoted as a set D = {d i}, and take the set as a uniform probability distribution Q s . Calculate the mean and variance of D, denoted as M D and G D , respectively. Establish a normal distribution model P D according to M D and G G . Our assumption is that the movement of the living target has continuity, and the movement conforms to all or part of the acceleration to uniform speed to deceleration, so the speed of the target should be close to the Gaussian distribution of its own speed, that is, the movement mode of the target should be the whole or part of the Gaussian distribution, and the speed should not change dramatically between frames.

[0044] Step five: speed pattern matching: calculate the distance between Q s and P G using KL divergence, and the calculation formula is The value of H(p, q) does not exceed a preset value H y . The value of H y affects the detection rate and accuracy of the method, and in this embodiment, the value of H y ranges from 0 to 1.

[0045] Step six: moving direction pattern matching: for any pipe T n , calculate the direction change C = {C i} of the target moving in each frame, and the mean of C does not exceed a preset value M c , and the variance of C does not exceed a preset value G c . In this embodiment, the value of M c ranges from 0.1 to 0.8, and the value of G c ranges from 0.1 to 0.5.

[0046] Step seven: reclassification: if both the speed pattern and the moving direction pattern can be successfully matched, it is judged that a small animal is moving. For the pipe T n, extract all foreground images M = {M i} in the pipeline, and classify the foreground images M = {M i} using a classification model, which contains N layers of convolution and m layers of full connection, and 1 classification layer, as shown in Figure 2 . The classification threshold is T, and as long as one image M i} in M = {M i} is greater than the threshold, it is determined that the pipeline is a small animal movement pipeline. In this embodiment, the value of T is in the range of 0.5-0.99. N is greater than or equal to 1, and m is greater than or equal to 0.

[0047] The classifier in the present application can have various forms, as long as it is multi-layer convolution or multi-layer full connection + 2 classification, and the effect of the present method can be achieved.

Claims

1. A nocturnal small animal presence detection method based on trajectory search, characterized by, comprising the steps of: Step one background modeling: using the method of mixed Gaussian model to do background modeling; for a video, first use the first frame picture T1 to initialize the background, when there is a new picture frame T i When incoming, use the method of mixed Gaussian model to update the background, at this time the background is B i ; Step two foreground calculation: for T i Each pixel on the frame image, using the method of Gaussian mixture model to calculate the matching relationship between the pixel and the current background B i , if matching, the pixel is background, otherwise the pixel belongs to foreground, all T i Frame foreground pixel set is foreground map F i ; Then use the three-step method of expansion, corrosion and expansion again to remove noise points; Step three: merge the extraction pipeline: for the foreground F of the i-th frame picture i , first find out the different connected regions using connected region analysis, and record them as a set {Q}; for any region Q i , if the position of the region overlaps with the position of some region Q j on the (i-1)-th frame picture, then join it into the pipeline T j where Q i is located; otherwise, a new pipeline T is created; Step four creates a speed matching model: for any pipe T n , calculate the distance between each frame that the target moves, denoted as a set D = {d i} , take this set as a uniform probability distribution Q s ; calculate the mean and variance of D, denoted as M D and G D respectively; establish a normal distribution model P D according to M D and G G ; Step five velocity pattern matching: Q is calculated using KL divergence s The distance from P G , the formula is The value of H(p, q) does not exceed the preset value H y ; The movement of the living target has continuity, and when moving, it conforms to all or part of the acceleration to uniform speed to deceleration, so the speed of the target should be close to the Gaussian distribution of its own speed, and the movement pattern of the target should be the whole or part of the Gaussian distribution, and the speed does not change dramatically between frames. Step six moving direction pattern matching: for any pipe T n , calculate the direction change C = {C i} of each frame when the target moves, the mean of C is not more than a preset value M c , and the variance of C is not more than a preset value Gc; the premise is that the moving direction of the target will not suddenly change drastically between adjacent frames; Step seven reclassification: if both speed mode and moving direction mode can be matched successfully, it is judged that small animal movement is detected; for the pipe T n , extract all foreground maps M = {M i} in the pipe, classify the foreground maps M = {M i} using a classification model, which contains N layers of convolution and m layers of full connection and 1 classification layer; provided that The preset threshold value is T, if there are n pictures greater than the preset threshold value T in M={M i}, the pipe is determined as a small animal moving pipe.

Images