Target detection and tracking method for dynamic scene

Abstract

The invention discloses a target detection and tracking method for a dynamic scene, which comprises the following steps of: (1) dynamic scene target detection: (1) taking first t frames of images without targets in a monitoring video as an initial background model, namely an initial sampling set; (2) starting to detect a target from the input (t + 1) th frame of image: constructing a target image by introducing a nine-parameter motion model into a monitoring video; (3) removing possible extrinsic points by adopting an RANSAC (Random Sample Consensus) method to obtain correct matched feature point pairs, and solving parameters of a global motion model; and (4) performing motion compensation on the image, and finally obtaining a moving target by using a frame difference method. According to the method, the 9-parameter motion model is introduced to construct the target image, the RANSAC method is matched to remove possible outer points, and then the motion compensation and frame difference method is used for frame-by-frame processing, so that the definition of the target image is greatly improved, and the target detection is more real and accurate.

Description

Technical Field

[0001] The present invention relates to the field of intelligent visual monitoring, and in particular to a target detection and tracking method for dynamic scenes. Background Art

[0002] The visual processing system uses image sensors to observe moving targets (such as pedestrians, vehicles, etc.) in the monitoring scene in real time. By detecting, identifying and tracking moving targets in image sequences, the activities of targets in the scene are monitored, and the individual and mutual behaviors of targets can be understood and described. Visual monitoring technology appeared in the 1960s. The development of video-based monitoring systems started with analog monitoring (CCTV), and the robustness and automation of monitoring systems were low. With the development of technologies and equipment required for visual monitoring such as sensors, computer hardware and software, signal processing and communication, the widespread application and rapid development of visual monitoring have a solid material foundation. It is possible to use high-performance computers to acquire and process images in monitoring systems. At present, the tracking of moving targets under complex backgrounds mainly uses a single camera to monitor the scene. The detection and recognition of moving targets is the research focus. At present, developed countries such as the United States, Europe and Japan have developed automated visual monitoring technology and are rapidly developing towards fully digital intelligent visual monitoring technology.

[0003] The existing methods for target detection and tracking in dynamic scenes generally have the following problems: 1. The authenticity and stability of target detection are not strong, and frame-by-frame analysis and processing cannot be achieved, resulting in the inability to track the target subsequently; 2. It is more obviously affected by various types of light in different environments, and it is easy to lose the subsequent tracking of the target in certain special time periods. Summary of the invention

[0004] The technical problem solved by the present invention is that the prior art cannot perform frame-by-frame analysis and processing, and is significantly affected by various light sources in different environments.

[0005] In order to solve the above technical problems, the present invention provides a target detection and tracking method for dynamic scenes, comprising the following steps:

[0006] (1) Dynamic scene target detection:

[0007] ①, taking the first t frames of images in the surveillance video where no target appears as the initial background model, i.e. the initial sampling set;

[0008] ②. Detect the target from the input frame t+1: Construct the target image by introducing a nine-parameter motion model into the surveillance video. Specifically, add encoding to the video encoder. The encoding formula is as follows:

[0009]

[0010] Where x, y represent the pixel coordinates in the current frame, and xr, yr represent the corresponding pixel coordinates in the reference frame;

[0011] ③. Use the RANSAC method to remove possible external points, obtain the correct matching feature point pairs, and solve the parameters of the global motion model;

[0012] ④, then perform motion compensation on the image, and finally use the frame difference method to obtain the moving target;

[0013] (2) Dynamic scene target tracking:

[0014] ⑤. For the target area detected in the current frame image, establish a target model: color model, motion model and shape model;

[0015] ⑥ Use the improved SIFT algorithm to match the feature points of the detected target;

[0016] ⑦. Analyze the target motion status in the current frame according to the matching situation;

[0017] ⑧. Use the local ternary number (LTN) local saliency texture operator and combine it with the chromaticity information to obtain the complete target motion;

[0018] ⑨. Repeat the above steps ⑤-⑧ to track the moving target.

[0019] Preferably, in step ③, the specific operation method of the RANSAC method is as follows:

[0020] a) Determine a model with a minimum sampling set power of n, where n is the minimum number of samples required to initialize the model parameters, and a sample set P, where the number of samples in set P #(P)>n, and randomly select a subset S of P containing n samples from P to initialize the model M;

[0021] b) The residual set SC = P\S, the sample set whose error with the model M is less than a certain set threshold t and S constitute S*, S* is considered to be the internal point set, and they constitute the consensus set of S;

[0022] c) If #(S*) ≥ N, the correct model parameters are considered to be obtained, and the set S* (inliers) is used to recalculate the new model M* using the least squares method, and a new S is randomly drawn again, and the above process is repeated;

[0023] d) After a certain number of samplings, if no consistent set is found, the algorithm fails; otherwise, the largest consistent set obtained after sampling is selected to determine the internal and external points, and the algorithm ends.

[0024] Preferably, in step ④, the specific method of performing motion compensation on the image is:

[0025] a) Motion estimation: By comparing the difference between frame t and frame t+1, the best displacement vector is calculated to describe the motion information;

[0026] b) Motion compensation: Based on the estimated displacement vector, the image is translated and compensated during decoding to reconstruct the original image.

[0027] Preferably, in step ④, the specific steps of the frame difference method are:

[0028] a) Grayscale processing: reduce the size of each frame and increase computer processing speed;

[0029] b) Frame difference processing: finding the difference between frames;

[0030] c) Binarization: Set the grayscale value of the pixel on the image to 0 or 255, so that the entire image presents an obvious black and white effect, highlighting the outline of the target;

[0031] d) Erosion treatment: remove the "burrs" on the edge of the target to make the target object clearer and more distinct;

[0032] e) Expansion processing: thicken the target to make it more obvious;

[0033] f) Mark dynamic objects: save key points for marking;

[0034] g) Extract key points: extract key points for marking;

[0035] h) Finally, drawing is performed to form a clear image of the dynamic target.

[0036] Preferably, in step ⑥, the specific steps of the improved SIFT algorithm are as follows:

[0037] 1) Generation of scale space:

[0038] The dimension space of a two-dimensional image is defined as: L (x,y,σ) =G (x,y,σ) *I (x,y)

[0039] , where (x, y) is the spatial coordinate, σ is the scale coordinate, G (x,y,σ) It is a scale-variable Gaussian function. The size of σ determines the smoothness of the image. A large scale corresponds to the general features of the image, and a small scale corresponds to the detailed features of the image. A large σ value corresponds to a coarse scale, and vice versa, to a fine scale.

[0040] 2) Detecting extreme points in scale space:

[0041] Find the extreme points in the scale space. Compare the size of each sampling point with all its neighboring points in the image domain and the scale domain. Continue to use Gaussian blur to generate 3 images on the top layer of each group of images. The Gaussian pyramid has S+3 layers of images per group, and the DOG pyramid has S+2 layers of images per group.

[0042] 3) Accurately locate extreme points:

[0043] By fitting a three-dimensional quadratic function, the position and scale of key points are accurately determined, and low-contrast key points and unstable edge response points are removed to enhance matching stability and improve noise resistance.

[0044] 4) Specify direction parameters for each key point:

[0045] The gradient direction distribution characteristics of the pixels in the key point area are used to specify the direction parameters for each key point, so that the operator has rotation invariance, and a SIFT feature area is determined by the position, scale and direction of each key point;

[0046] 5) Generation of key point descriptors:

[0047] The coordinate axis is rotated to the direction of the key point to ensure rotation invariance. An 8*8 area is taken as the sampling window with the feature point as the center. The relative directions of the sampling points and the feature points are weighted by Gaussian and classified into 8 direction histograms to obtain a 2*2*8 32-dimensional feature descriptor.

[0048] Preferably, the Gaussian difference scale space formula constructed in the generation of the scale space is as follows:

[0049] D (x,y,σ) =(G (x,y,kσ) -G (x,y,σ) )*I (x,y) =L (x,y,kσ) -L (x,y,σ) .

[0050] Preferably, the function for accurately locating the spatial scale of the extreme point is: Derivative the spatial scale function and set it to 0 to obtain the exact position

[0051] Compared with the related art, the present invention has the following beneficial effects:

[0052] 1. The present invention introduces a 9-parameter motion model to construct the target image, and uses the RANSAC method to remove possible external points. After frame-by-frame processing using motion compensation and frame difference method, the clarity of the target image is greatly improved, making the detection of the target more realistic and accurate.

[0053] 2. The present invention adopts the local ternary number (LTN) local significant texture operator and combines it with chromaticity information, which can effectively avoid light interference in various environments and ensure that tracking can be carried out continuously.

[0054] In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0056] Figure 1 Flowchart of target detection and tracking of the present invention. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] In this embodiment, a method for detecting and tracking a target in a dynamic scene includes the following steps: (1) Detecting a target in a dynamic scene: ①, taking the first t frames of images in the surveillance video where the target does not appear as an initial background model, i.e., an initial sampling set; ②, detecting the target from the input t+1th frame image: constructing a target image by introducing a nine-parameter motion model into the surveillance video, specifically, adding encoding to the video encoder, and the encoding formula is as follows:

[0059]

[0060] Where x and y represent the pixel coordinates in the current frame, and xr and yr represent the corresponding pixel coordinates in the reference frame; ③, use the RANSAC method to remove possible external points, obtain the correct matching feature point pairs, and solve the parameters of the global motion model. The specific operation method of the RANSAC method is as follows: a) Determine a model with a minimum sampling set potential of n, where n is the minimum number of samples required to initialize the model parameters, and a sample set P, where the number of samples in the set P is #(P)>n, and randomly select a subset S of P containing n samples from P to initialize the model M; b) The sample set in the remainder SC=P\S whose error with the model M is less than a certain set threshold t and S constitute S*, which is considered to be the internal point set, and they constitute the consistent set (Consensus Set); c) If #(S*) ≥ N, it is considered that the correct model parameters are obtained, and the set S* (inliers) is used to recalculate the new model M* by using the least squares method, and a new S is randomly drawn again, and the above process is repeated; d) After completing a certain number of sampling times, if no consistent set is found, the algorithm fails, otherwise the largest consistent set obtained after sampling is selected to determine the inliers and inliers, and the algorithm ends. ; ④, then perform motion compensation on the image, and finally use the frame difference method to obtain the moving target. The specific method of motion compensation for the image is: a) Motion estimation: by comparing the difference between frame t and frame t+1, calculate the best displacement vector to describe the motion information; b) Motion compensation: according to the estimated displacement vector, translate and compensate the image during decoding to reconstruct the original image. The specific steps of the frame difference method are: a) Grayscale processing: reduce the image size of each frame and increase the computer processing speed; b) Frame difference processing: find the difference between frames; c) Binarization: set the grayscale value of the pixel on the image to 0 or 255, so that the entire image presents an obvious black and white effect, highlighting the outline of the target ; d) Erosion processing: remove the "burrs" on the edge of the target to make the target object clearer and the outline obvious; e) Dilation processing: thicken the target and make it more obvious; f) Mark dynamic objects: save key points for marking; g) Extract key points: extract key points for marking; h) Finally, draw to form a clear image of the dynamic target; (2) Dynamic scene target tracking: ⑤. For the target area detected in the current frame image, establish a target model: color model, motion model and shape model; ⑥. Use the improved SIFT algorithm to match the feature points of the detected target. The specific steps of the improved SIFT algorithm are as follows: 1) Generation of scale space: The size space of a two-dimensional image is defined as: L (x,y,σ) =G (x,y,σ) *I (x,y) , where (x, y) is the spatial coordinate, σ is the scale coordinate, G (x,y,σ)is a scale-variable Gaussian function. The size of σ determines the smoothness of the image. The large scale corresponds to the general features of the image, and the small scale corresponds to the detailed features of the image. The large value corresponds to the coarse scale, and vice versa, the Gaussian difference scale space formula constructed in the generation of the scale space is as follows: D (x,y,σ) =(G (x,y,kσ) -G (x,y,σ) )*I (x,y) =L (x,y,kσ) -L (x,y,σ) ; 2) Detect extreme points in scale space: Find extreme points in scale space, compare the size of each sampling point and all its adjacent points in the image domain and the scale domain, and continue to use Gaussian blur to generate 3 images at the top layer of each group of images. The Gaussian pyramid has S+3 layers of images per group, and the DOG pyramid has S+2 layers of images per group; 3) Accurately locate extreme points: Accurately determine the position and scale of key points by fitting a three-dimensional quadratic function, while removing low-contrast key points and unstable edge response points, enhancing matching stability and improving noise resistance. The function of the spatial scale in the accurate positioning of extreme points is: Derivative the spatial scale function and set it to 0 to obtain the exact position 4) Assign directional parameters to each key point: Assign directional parameters to each key point using the gradient directional distribution characteristics of the pixels in the key point area, so that the operator has rotation invariance, and determine a SIFT feature area by the position, scale and direction of each key point; 5) Generation of key point descriptors: Rotate the coordinate axis to the direction of the key point to ensure rotation invariance, take an 8*8 area centered on the feature point as the sampling window, and classify the relative directions of the sampling points and feature points into 8 directional histograms after Gaussian weighting to obtain a 2*2*8 32-dimensional feature descriptor; ⑦, Analyze the motion state of the target in the current frame according to the matching situation; ⑧, Use the local ternary number (LTN) local saliency texture operator and combine it with the chromaticity information to obtain the complete target motion; ⑨, Repeat the above operations ⑤-⑧ to achieve tracking of the moving target.

[0061] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A target detection and tracking method for dynamic scenes, characterized in that: The steps include: (1) Dynamic scene target detection: ①, taking the first t frames of images in the surveillance video where no target appears as the initial background model, i.e. the initial sampling set; ②. Detect the target from the input frame t+1: Construct the target image by introducing a nine-parameter motion model into the surveillance video. Specifically, add encoding to the video encoder. The encoding formula is as follows: Where x, y represent the pixel coordinates in the current frame, and xr, yr represent the corresponding pixel coordinates in the reference frame; ③. Use the RANSAC method to remove possible external points, obtain the correct matching feature point pairs, and solve the parameters of the global motion model; ④, then perform motion compensation on the image, and finally use the frame difference method to obtain the moving target; (2) Dynamic scene target tracking: ⑤. For the target area detected in the current frame image, establish a target model: color model, motion model and shape model; ⑥ Use the improved SIFT algorithm to match the feature points of the detected target; ⑦. Analyze the target motion status in the current frame according to the matching situation; ⑧. Use the local ternary number (LTN) local saliency texture operator and combine it with the chromaticity information to obtain the complete target motion; ⑨. Repeat the above steps ⑤-⑧ to track the moving target.

2. The target detection and tracking method for dynamic scenes according to claim 1, characterized in that: In step ③, the specific operation method of the RANSAC method is as follows: a) Determine a model with a minimum sampling set power of n, where n is the minimum number of samples required to initialize the model parameters, and a sample set P, where the number of samples in set P #(P)>n, and randomly select a subset S of P containing n samples from P to initialize the model M; b) The residual set SC = P\S, the sample set whose error with the model M is less than a certain set threshold t and S constitute S*, S* is considered to be the internal point set, and they constitute the consensus set of S; c) If #(S*) ≥ N, the correct model parameters are considered to be obtained, and the set S* (inliers) is used to recalculate the new model M* using the least squares method, and a new S is randomly drawn again, and the above process is repeated; d) After a certain number of samplings, if no consistent set is found, the algorithm fails; otherwise, the largest consistent set obtained after sampling is selected to determine the internal and external points, and the algorithm ends.

3. The target detection and tracking method for dynamic scenes according to claim 1, characterized in that: In step ④, the specific method of performing motion compensation on the image is: a) Motion estimation: By comparing the difference between frame t and frame t+1, the best displacement vector is calculated to describe the motion information; b) Motion compensation: Based on the estimated displacement vector, the image is translated and compensated during decoding to reconstruct the original image.

4. The target detection and tracking method for dynamic scenes according to claim 1, characterized in that: In step ④, the specific steps of the frame difference method are: a) Grayscale processing: reduce the size of each frame and increase computer processing speed; b) Frame difference processing: finding the difference between frames; c) Binarization: Set the grayscale value of the pixel on the image to 0 or 255, so that the entire image presents an obvious black and white effect, highlighting the outline of the target; d) Corrosion treatment: remove the "burrs" on the edge of the target to make the target object clearer and more distinct; e) Expansion processing: thicken the target to make it more obvious; f) Mark dynamic objects: save key points for marking; g) Extract key points: extract key points for marking; h) Finally, drawing is performed to form a clear image of the dynamic target.

5. The target detection and tracking method for dynamic scenes according to claim 1, characterized in that: In step ⑥, the specific steps of the improved SIFT algorithm are as follows: 1) Generation of scale space: The dimension space of a two-dimensional image is defined as: L (x,y,σ) =G (x,y,σ) *I (x,y) , where (x, y) is the spatial coordinate, σ is the scale coordinate, G (x,y,σ) It is a scale-variable Gaussian function. The size of σ determines the smoothness of the image. A large scale corresponds to the general features of the image, and a small scale corresponds to the detailed features of the image. A large σ value corresponds to a coarse scale, and vice versa, to a fine scale. 2) Detecting extreme points in scale space: Find the extreme points in the scale space. Compare the size of each sampling point with all its neighboring points in the image domain and the scale domain. Continue to use Gaussian blur to generate 3 images on the top layer of each group of images. The Gaussian pyramid has S+3 layers of images per group, and the DOG pyramid has S+2 layers of images per group. 3) Accurately locate extreme points: By fitting a three-dimensional quadratic function, the position and scale of key points are accurately determined, and low-contrast key points and unstable edge response points are removed to enhance matching stability and improve noise resistance. 4) Specify direction parameters for each key point: The gradient direction distribution characteristics of the pixels in the key point area are used to specify the direction parameters for each key point, so that the operator has rotation invariance, and a SIFT feature area is determined by the position, scale and direction of each key point; 5) Generation of key point descriptors: The coordinate axis is rotated to the direction of the key point to ensure rotation invariance. An 8*8 area is taken as the sampling window with the feature point as the center. The relative directions of the sampling points and the feature points are weighted by Gaussian and classified into 8 direction histograms to obtain a 2*2*8 32-dimensional feature descriptor.

6. The target detection and tracking method for dynamic scenes according to claim 5, characterized in that: The Gaussian difference scale space formula constructed in the generation of scale space is as follows: D (x,y,σ) =(G (x,y,kσ) -G (x,y,σ) )*I (x,y) =L (x,y,kσ) -L (x,y,σ) 。 7. The target detection and tracking method for dynamic scenes according to claim 5, characterized in that: The function of the spatial scale in accurately locating the extreme points is: Derivative the spatial scale function and set it to 0 to obtain the exact position

Images