An object feature tracking method and apparatus

Abstract

The embodiment of the application discloses a kind of object feature tracking method and device, the method comprises: after receiving behavior analysis alarm picture, obtain monitoring video before alarm;The behavior analysis alarm picture contains target object;Extract the part feature of the body part area of target object in the behavior analysis alarm picture, and store in target object feature library;From the last frame picture of the monitoring video, the part feature in the target object feature library is compared with the part feature of the body part area of target object in the picture contained by the monitoring video in turn;According to each comparison result, it is determined whether to update the target object feature library.Through the embodiment scheme, the variability of adaptive analysis scene is realized, to expand the function of video monitoring, realize stable, accurate target object tracking provides technical foundation.

Description

Technical Field

[0001] This application relates to behavior analysis technology, and more particularly to an object feature tracking method and apparatus. Background Technology

[0002] With socio-economic development, public demand for security and prevention has increased. In intelligent video surveillance, behavior analysis technology plays a crucial role in ensuring social security by promptly detecting and addressing abnormal behavior. Traditional intelligent video surveillance systems include functions such as intelligent facial recognition, human behavior analysis, passenger flow statistics analysis, intelligent vehicle management, intelligent object analysis, and intelligent video detection. However, these functions cannot work in tandem; behavior analysis can only detect abnormal behavior, and facial recognition can only confirm identity information, resulting in current video surveillance systems being functionally limited and operationally complex.

[0003] Furthermore, in behavioral analysis video surveillance scenarios, the diverse changes in people's positions and postures can lead to video tracking failures. Therefore, existing video surveillance technologies based on the variability of human body features such as color, contour, and texture are ill-suited to the undefined scenarios of behavioral analysis. While this problem can be addressed using deep learning-based human tracking algorithms, traditional deep learning-based human tracking algorithms require pre-labeling of features and training of the algorithm model, necessitating significant upfront work and making them unsuitable for unlabeled applications. Summary of the Invention

[0004] This application provides an object feature tracking method and apparatus that can adapt to the variability of analysis scenarios, providing a technical foundation for expanding the functions of video surveillance and achieving stable and accurate target object tracking.

[0005] This application provides an object feature tracking method, which may include:

[0006] Upon receiving a behavior analysis alarm image, the monitoring video prior to the alarm is retrieved; the behavior analysis alarm image contains the target object.

[0007] Extract the body part features of the target object in the behavior analysis alarm image and store them in a preset target object feature library;

[0008] Starting from the last frame of the surveillance video, the body part features in the target object feature library are compared sequentially with the body part features of the target object in the images contained in the surveillance video.

[0009] Whether to update the target object feature library is determined based on the results of each comparison.

[0010] In an exemplary embodiment of this application, extracting the part features of the target object's body part region from each frame of the behavior analysis alarm image and the surveillance video may include:

[0011] Obtain the body part region of the target object from the image from which the part features are to be extracted; the image from which the part features are to be extracted is either the behavior analysis alarm image or any frame of the surveillance video;

[0012] The body part features are extracted using a preset feature extraction algorithm.

[0013] In an exemplary embodiment of this application, the step of obtaining the body part region of the target object in the image from which the part features are to be extracted may include:

[0014] When the image from which the feature of the part to be extracted is the behavior analysis alarm image, a preset edge detection algorithm is used to extract the object outline of the target object in the target object box of the behavior analysis alarm image; the body part region is segmented according to the proportion of the object outline;

[0015] When the image from which the feature of the part to be extracted is each frame of the surveillance video, the edge detection algorithm is used to extract the object outline of the target object in each frame of the image based on the target object bounding box in the behavior analysis alarm image; the body part region is segmented according to the proportion of the object outline.

[0016] In an exemplary embodiment of this application, segmenting the body part region according to a proportional relationship of the object outline may include:

[0017] The object outline is divided into multiple regions according to a preset algorithm;

[0018] Calculate the first proportional relationship between the plurality of regions;

[0019] Obtain the pre-calculated second proportional relationship between different body part regions;

[0020] By comparing the first proportional relationship with the second proportional relationship, the body part region corresponding to each of the multiple regions is determined.

[0021] In an exemplary embodiment of this application, the feature extraction algorithm includes a first feature extraction algorithm and a second feature extraction algorithm; the step of extracting the part features of the body part region using a preset feature extraction algorithm may include:

[0022] The first feature extraction algorithm is used to extract corner points of the body part region, and one or more feature blocks in the image are selected based on the corner points to form a first feature point set; the second feature extraction algorithm is used to extract feature points of the body part region to form a second feature point set.

[0023] In an exemplary embodiment of this application, the step of sequentially comparing the part features in the target object feature library with the part features of the body parts of the target object in the images contained in the surveillance video may include: comparing the part features of the behavior analysis alarm image with the part features of any frame in the surveillance video according to the following method:

[0024] The first feature point set and the second feature point set corresponding to the behavior analysis alarm image are merged to obtain a first initial feature point set; the first feature point set and the second feature point set corresponding to any frame image in the monitoring video are merged to obtain a second initial feature point set.

[0025] Using the KD-tree algorithm and Euclidean distance, calculate whether the feature points in the first initial set of feature points and the second initial set of feature points match.

[0026] In an exemplary embodiment of this application, the step of using the KD-tree algorithm and Euclidean distance to calculate whether the feature points in the first initial set of feature points and the second initial set of feature points match may include:

[0027] Construct a balanced dimension search tree (KD tree) for the multidimensional feature points of the first initial feature point set, denoted as the first KD tree, and a balanced dimension search tree (KD tree) for the multidimensional feature points of the second initial feature point set, denoted as the second KD tree.

[0028] Starting from the root node of the first KD tree and the second KD tree, recursively visit the first KD tree and the second KD tree to perform the nearest search, and obtain the nearest and second nearest feature points in the second KD tree relative to each feature point in the first KD tree;

[0029] Calculate the ratio of the Euclidean distance between the nearest and second nearest feature points for each feature point;

[0030] When the ratio is less than or equal to a preset threshold, it is confirmed that the feature point matches the nearest feature point of the feature point on the second KD tree; when the ratio is greater than the preset threshold, it is confirmed that the feature point does not match the nearest feature point of the feature point on the second KD tree.

[0031] The number of matching feature points determines whether the feature points in the first initial feature point set and the second initial feature point set match.

[0032] In an exemplary embodiment of this application, determining whether the feature points in the first initial set of feature points and the second initial set of feature points match based on the number of matching feature points may include:

[0033] When the number of matching feature points is greater than or equal to a preset threshold, it is determined that the feature points in the first initial feature point set and the second initial feature point set are matched.

[0034] When the number of matching feature points is less than the preset threshold, it is determined that the feature points in the first initial feature point set and the second initial feature point set do not match.

[0035] In an exemplary embodiment of this application, before determining whether the feature points in the first initial set of feature points and the second initial set of feature points match based on the number of matching feature points, the method may further include:

[0036] The Random Sampling Consensus (RANSAC) algorithm is used to remove erroneous feature points from confirmed matching feature points.

[0037] In an exemplary embodiment of this application, the step of using the Random Sampling Consensus (RANSAC) algorithm to remove error feature points from confirmed matching feature points may include:

[0038] A. Use the feature point pairs formed by matching feature points as interest points;

[0039] B. Using the Random Sampling Consensus (RANSAC) algorithm, a preset number of interest points are extracted from the interest points to calculate the perspective transformation model of the behavior analysis alarm image;

[0040] C. Calculate the distance between the feature points corresponding to the unextracted interest points and the perspective transformation model, and obtain the maximum value among all calculated distances, storing it in a preset first set; detect whether the current iteration number has reached the preset iteration number; when the current iteration number has reached the preset iteration number, proceed to step D; when the current iteration number has not reached the preset iteration number, proceed to step B;

[0041] D. Compare the values ​​contained in the first set, and take the perspective transformation model corresponding to the largest value in the first set as the preferred feature point set.

[0042] In an exemplary embodiment of this application, determining whether to update the target object feature library based on each comparison result may include:

[0043] When the feature points in the first initial feature point set and the second initial feature point set match, the target object feature library remains unchanged;

[0044] When the feature points in the first initial feature point set and the second initial feature point set do not match, the target object feature library is updated using the part features in the image compared with the behavior analysis alarm image.

[0045] In an exemplary embodiment of this application, after comparing the part features in the target object feature library with the part features of the body part region of the target object in any frame of the surveillance video, before continuing to obtain the part features of the body part region in the previous frame, the method may further include:

[0046] Based on the target object feature library, the sparse optical flow method is used to predict the body position of the target object in the previous frame image; wherein, the previous frame image refers to the frame image preceding any frame image compared with the current behavior analysis alarm image.

[0047] In an exemplary embodiment of this application, predicting the body position of the target object in the previous frame image using sparse optical flow based on the target object feature library may include:

[0048] Extract pixels within the feature point neighborhood of the target object feature library;

[0049] Optical flow estimation is performed on the pixels to calculate the body position of the target object in the previous frame image.

[0050] In an exemplary embodiment of this application, the method may further include:

[0051] Each time it is predicted that the body position of the target object in the previous frame image is not within the preset tracking area, the part features of the body part region in the previous frame image are obtained based on the predicted body position of the target object in the previous frame image; and the target object feature library is updated using the extracted part features of the body part region in the previous frame image.

[0052] Each time the body position of the target object in the previous frame image is predicted to be within the preset tracking area, the target object feature library is kept unchanged.

[0053] This application also provides an object feature tracking device, which may include a processor and a computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when executed by the processor, implement the object feature tracking method.

[0054] Compared with related technologies, the embodiments of this application may include: after receiving a behavior analysis alarm image, acquiring the monitoring video before the alarm; the behavior analysis alarm image contains a target object; extracting the part features of the body parts of the target object in the behavior analysis alarm image and storing them in a target object feature library; starting from the last frame of the monitoring video, sequentially comparing the part features in the target object feature library with the part features of the body parts of the target object in the images contained in the monitoring video; and determining whether to update the target object feature library based on the comparison results of each comparison. This embodiment achieves adaptability to the variability of analysis scenarios, providing a technical foundation for expanding the functions of video surveillance and achieving stable and accurate target object tracking.

[0055] Other features and advantages of this application will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the application. Other advantages of this application can be realized and obtained by means of the solutions described in the description and the accompanying drawings. Attached Figure Description

[0056] The accompanying drawings are used to provide an understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.

[0057] Figure 1 This is a flowchart of the object feature tracking method according to an embodiment of this application;

[0058] Figure 2 This is a flowchart illustrating a method for extracting part features of a body part region of a target object according to an embodiment of this application.

[0059] Figure 3 This is a schematic outline diagram of the target object in an embodiment of this application;

[0060] Figure 4 This is a flowchart illustrating a method for comparing the location features of a behavior analysis alarm image with the location features of any frame in a surveillance video, according to an embodiment of this application.

[0061] Figure 5 This is a flowchart illustrating a method for calculating whether feature points in a first initial set of feature points and a second initial set of feature points match using the KD-tree algorithm and Euclidean distance, according to an embodiment of this application.

[0062] Figure 6 This is a block diagram of the object feature tracking device according to an embodiment of this application. Detailed Implementation

[0063] This application describes several embodiments, but these descriptions are exemplary and not restrictive, and it will be apparent to those skilled in the art that many more embodiments and implementations are possible within the scope of the embodiments described herein. Although many possible combinations of features are shown in the drawings and discussed in the detailed description, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature or element of any embodiment may be used in combination with, or may replace, any feature or element of any other embodiment.

[0064] This application includes and contemplates combinations of features and elements known to those skilled in the art. The embodiments, features, and elements disclosed in this application may also be combined with any conventional features or elements to form a unique inventive scheme as defined by the claims. Any feature or element of any embodiment may also be combined with features or elements from other inventive schemes to form another unique inventive scheme as defined by the claims. Therefore, it should be understood that any feature shown and / or discussed in this application may be implemented individually or in any suitable combination. Therefore, the embodiments are not limited except by the limitations imposed by the appended claims and their equivalents. Furthermore, various modifications and changes may be made within the scope of the appended claims.

[0065] Furthermore, in describing representative embodiments, the specification may have presented methods and / or processes as a specific sequence of steps. However, the method or process should not be limited to the specific order of steps described herein, to the extent that it does not depend on such a specific order. As will be understood by those skilled in the art, other sequences of steps are also possible. Therefore, the specific order of steps set forth in the specification should not be construed as a limitation of the claims. Moreover, the claims concerning the method and / or process should not be limited to the steps performed in the written order, and those skilled in the art will readily understand that these orders can be varied and still remain within the spirit and scope of the embodiments of this application.

[0066] This application provides an object feature tracking method, such as... Figure 1 As shown, the method may include steps S101-S104:

[0067] S101. After receiving the behavior analysis alarm image, obtain the monitoring video before the alarm; the behavior analysis alarm image contains the target object;

[0068] S102. Extract the part features of the body parts of the target object in the behavior analysis alarm image and store them in the target object feature library;

[0069] S103. Starting from the last frame of the surveillance video, the part features in the target object feature library are compared sequentially with the part features of the body parts of the target object in the images contained in the surveillance video.

[0070] S104. Determine whether to update the target object feature library based on the comparison results of each comparison.

[0071] In an exemplary embodiment of this application, the target object can be a person or an animal.

[0072] In the exemplary embodiments of this application, due to the complexity and variability of current intelligent video scenarios, object tracking is also complex and varied. Moreover, in cases where objects have diverse poses, similar outlines, body occlusion, or crowding, target object tracking is easily lost. To address this issue, the embodiments of this application propose an online learning object feature tracking method based on a preset feature extraction algorithm, such as the Harris-SURF (Harris Corner Detection-Accelerated Robust Features) algorithm. This method can perform feature comparison on each frame of the monitoring video before the time the behavior analysis alarm image is generated, based on the behavior analysis alarm image, thereby obtaining accurate and reliable target object body features from the monitoring video, providing a technical foundation for obtaining more information about the target object. For example, if the back view information of a target person is obtained through the current behavior analysis alarm image, according to the feature tracking scheme of this application embodiment, the feature of the target person's body parts when moving in the surveillance video can be automatically extracted based on the feature comparison of each frame image. The target object feature library obtained in advance through the behavior analysis alarm image is updated based on the extracted features to improve the accuracy of the target object features. This makes it easier to find information about the target person other than the back view information from the previous frames of the surveillance video more accurately. For example, the facial information of the target object may be obtained, thus providing an important basis for tracking the identity of the target object.

[0073] In exemplary embodiments of this application, as Figure 2 As shown, extracting the body part features of the target object in each frame of the behavior analysis alarm image and the surveillance video may include steps S201-S202:

[0074] S201. Obtain the body part region of the target object in the image from which the part features are to be extracted; the image from which the part features are to be extracted is either the behavior analysis alarm image or any frame of the surveillance video.

[0075] In an exemplary embodiment of this application, the step of obtaining the body part region of the target object in the image from which the part features are to be extracted may include:

[0076] When the image from which the feature of the part to be extracted is the behavior analysis alarm image, a preset edge detection algorithm (e.g., Sobel operator) is used to extract the object outline of the target object in the target object box of the behavior analysis alarm image; the body part region is segmented according to the proportion of the object outline;

[0077] When the image from which the feature of the part to be extracted is each frame of the surveillance video, the edge detection algorithm is used to extract the object outline of the target object in each frame of the image based on the target object bounding box in the behavior analysis alarm image; the body part region is segmented according to the proportion of the object outline.

[0078] In an exemplary embodiment of this application, when the image from which the feature of the part to be extracted is each frame of the surveillance video, the corresponding coordinates of the target object can first be obtained in each frame of the surveillance video based on the coordinates of the target object bounding box in the behavior analysis alarm image, and then the object outline of the target object can be extracted in the frame image using an edge detection algorithm based on the determined coordinates of the target object.

[0079] In an exemplary embodiment of this application, the edge detection algorithm may include, but is not limited to, differential edge detection algorithm, Reborts operator, Sobel operator, Prewitt operator, Kirsch operator algorithm, LOG (Laplacian of Gaussian) operator algorithm, Canny operator algorithm, and Laplace operator, etc., where Sobel operator can be selected.

[0080] In an exemplary embodiment of this application, segmenting the body part region according to a proportional relationship of the object outline may include:

[0081] The object outline is divided into multiple regions according to a preset algorithm;

[0082] Calculate the first proportional relationship between the plurality of regions;

[0083] Obtain the pre-calculated second proportional relationship between different body part regions;

[0084] By comparing the first proportional relationship with the second proportional relationship, the body part region corresponding to each of the multiple regions is determined.

[0085] In an exemplary embodiment of this application, for example, if the human body is pre-divided with the length of the head as one unit L, the height of the torso approximately 3L, the width 2L, the height of the neck 0.4L, the length of the upper arm 1.2L, and the length of the forearm 1L, the dimensional ratio of each body part can be pre-obtained: head: torso height: torso width: neck height: upper arm length: forearm length = L:3L:2L:0.4L:1.2L:L = 1:3:2:0.4:1.2:1 (i.e., the aforementioned second proportional relationship); after dividing the object outline in the current frame image into multiple regions, the dimensional ratio between each region can be calculated (i.e., the first proportional relationship), and each item in the second proportional relationship can be compared with each item in the first proportional relationship to determine which item in the first proportional relationship corresponds to each item. After determining the correspondence, it can be determined which body part corresponds to each item in the first proportional relationship, such as the head, torso, neck, upper arm, forearm, etc. After determining which body part each item in the first proportional relationship corresponds to, the region in the object outline corresponding to the first proportional relationship is obtained. Then, each region can be matched with the corresponding body part, thereby determining each body part of the target object in any frame of the surveillance video.

[0086] In exemplary embodiments of this application, as Figure 3 As shown, when the target object is a person, the outline of the target object can be divided into eight key parts: head, neck, shoulders, upper arms (left and right), forearms (left and right), and torso. Furthermore, eight feature subsets can be established for each part in subsequent content.

[0087] S202. Use a preset feature extraction algorithm to extract the part features of the body part region.

[0088] In an exemplary embodiment of this application, the preset feature extraction algorithm includes, but is not limited to: Harris Corner Detection-Accelerated Robust Feature Harris-SURF algorithm.

[0089] In an exemplary embodiment of this application, the Harris-SURF algorithm can be used to extract the location features of the body part region by combining the rotation invariance of Harris corner points and the scale invariance of SURF.

[0090] In an exemplary embodiment of this application, the feature extraction algorithm may include: a first feature extraction algorithm and a second feature extraction algorithm; wherein, the first feature extraction algorithm may include, but is not limited to: Harris (Harris Corner Detection) algorithm; and the second feature extraction algorithm may include, but is not limited to: SURF (Speeded Robust Features) algorithm.

[0091] In an exemplary embodiment of this application, the step of extracting the part features of the body part region using a preset feature extraction algorithm may include:

[0092] The first feature extraction algorithm, such as the Harris corner detection algorithm, is used to extract the corner points of the body part region. One or more feature blocks in the image are selected based on the corner points to form a first feature point set. The second feature extraction algorithm, such as the SURF (Speed-Up Robust Features) algorithm, is used to extract the feature points of the body part region to form a second feature point set.

[0093] In an exemplary embodiment of this application, the embodiment can be used to extract part features from each frame of the behavior analysis alarm image and the surveillance video.

[0094] In an exemplary embodiment of this application, extracting corner points of the body part region using the Harris algorithm may include:

[0095] Pixel I(x, y) is translated (Δ) at (x, y). x Δ y The self-similarity after ) is expressed as:

[0096] c(x, y) = ∑ W [I(x i y i )-I(x i +Δx, y i +Δy)] 2 (1)

[0097] Where (x, y) are the pixels within window W, (Δ x Δ y ) is the given displacement; i is the pixel index, where i is a positive integer; c(x, y) is the autocorrelation function, I(x) = 1 / 2 * ... i y i ) represents the grayscale value of the image, I(x) i +Δx, y i +Δy) represents the grayscale value of the translated image;

[0098] Taylor expansion of the above relation (1):

[0099]

[0100] Among them I x I y These are the gradients in the x-direction and the y-direction, respectively.

[0101] The spatial gradient value of the image is substituted into equation (1) to obtain:

[0102]

[0103] in,

[0104] The M matrix is ​​a partial derivative matrix (M T If = M), then there must exist a set of eigenvalues ​​(λ1, λ2) such that the metric function for detecting corner points is:

[0105]

[0106] in, Generally, an empirical value (0.04-0.15) is used to calculate P for each pixel in the image. c Measure, P c Pixels that exceed a certain threshold are measured as Harris corners;

[0107] Centered on Harris corner points, select image blocks of a preset size (e.g., 10*10) as feature blocks, and form the first feature point set from the obtained feature blocks.

[0108] In an exemplary embodiment of this application, extracting feature points of the body part region using the SURF algorithm may include four steps: scale-space pole detection, feature point localization, determining the principal direction of the feature points, and generating feature point descriptors.

[0109] In an exemplary embodiment of this application, scale-space extremum detection is performed by filtering the image to be feature extracted using box filters of different scales to obtain an image pyramid; extremum points are detected using a Hessian matrix for each layer of the image pyramid; the purpose of constructing the Hessian matrix is ​​to generate abrupt change points of stable image points, in preparation for feature extraction.

[0110] In an exemplary embodiment of this application, the step of scale-space pole detection includes:

[0111] For any pixel I(x, y) in an image, the Hessian matrix of pixel I(x, y) consists of the partial derivatives of the function:

[0112]

[0113] in, Let be the second-order partial derivatives of the image I(x, y) in each direction;

[0114] The discriminant of the H matrix is:

[0115]

[0116] Before constructing the Hessian matrix, the image is subjected to a second-order Gaussian filter, and then obtained through convolution with specific kernels:

[0117]

[0118] Where σ is the scale, then L xx (x, y, σ) is the result of a two-dimensional convolution of the second-order partial derivatives of the Gaussian function and the image, L xy (x, y, σ), L yy The same applies to (x, y, σ).

[0119] If sampling is performed at different scales σ, and the sampling point is the maximum or minimum value, then the sampling point is an extreme point of the image at that scale σ.

[0120] In an exemplary embodiment of this application, feature point localization involves locating feature points for each body part region.

[0121] In an exemplary embodiment of this application, the position of the feature point can be determined using a Δ(H) that approximates Det(H):

[0122] Δ(H)=D xx D yy -(0.9D xy ) 2 (8)

[0123] Among them, D xx D yy and D xy Δ(H) is the value of the image after convolution under σ-scale filtering. If Δ(H) is positive, then the pixel is a feature point.

[0124] In an exemplary embodiment of this application, the main direction of the feature point is determined by: using Haar wavelets, with the feature point as the center, δ as the scale value where the feature point is located, 6δ as the radius of the circle, and a range of 60 degrees as a region, calculating the Haar wavelet response of length 4δ in 6 regions, and adding them together to form a new vector, the direction of the vector with the largest modulus among the 6 vectors is the main direction of the feature point.

[0125] In an exemplary embodiment of this application, a feature point descriptor is generated by describing the feature points according to the determined main direction of the feature points.

[0126] In an exemplary embodiment of this application, a 20δ*20δ window can be constructed using the main direction of the feature point, with 5δ*5δ blocks as small blocks. The responses of the Haar wavelets within the small blocks relative to the main direction, horizontal direction, and vertical direction are calculated. Then, the responses of the Haar wavelets in each direction are accumulated to form a 64-dimensional vector within each small block. The 64-dimensional vector is normalized to obtain the descriptor of the feature point.

[0127] In an exemplary embodiment of this application, when comparing the part features in the target object feature library with the part features of the target object's body parts in the images contained in the surveillance video, the part features in the target object feature library can be compared with the part features of the target object's body parts in each frame of the surveillance video in chronological order. Alternatively, depending on the detection accuracy requirements (e.g., if the detection accuracy requirements are not high) or the different shooting scenes of the surveillance video (in some shooting scenes, the target object's movement may change less frequently), the part features in the target object feature library can be compared with the part features of the target object's body parts in every other frame or multiple frames of the surveillance video. Here, there are no restrictions on the detailed comparison frequency and comparison accuracy.

[0128] In exemplary embodiments of this application, as Figure 4 As shown, the step of sequentially comparing the part features in the target object feature library with the part features of the body parts of the target object in the images contained in the surveillance video may include: comparing the part features of the behavior analysis alarm image with the part features of any frame in the surveillance video according to the following steps S301-S302:

[0129] S301. Merge the first feature point set and the second feature point set corresponding to the behavior analysis alarm image to obtain a first initial feature point set; merge the first feature point set and the second feature point set corresponding to any frame image in the monitoring video to obtain a second initial feature point set.

[0130] In an exemplary embodiment of this application, for each image, the corner points extracted by the Harris algorithm can be merged with the feature points extracted by the SURF algorithm to form a preliminary feature point set.

[0131] S302. Using the KD-tree algorithm and Euclidean distance, calculate whether the feature points in the first initial feature point set and the second initial feature point set match.

[0132] In exemplary embodiments of this application, as Figure 5As shown, the step of using the KD-tree algorithm and Euclidean distance to calculate whether the feature points in the first initial feature point set and the second initial feature point set match may include steps S401-S405:

[0133] S401. Construct a balanced dimension search tree (KD tree) for the multidimensional feature points of the first initial feature point set, denoted as the first KD tree, and a balanced dimension search tree (KD tree) for the multidimensional feature points of the second initial feature point set, denoted as the second KD tree.

[0134] S402. Starting from the root nodes of the first KD tree and the second KD tree respectively, recursively visit the first KD tree and the second KD tree to perform the nearest search, and obtain the nearest feature point and the second nearest feature point on the second KD tree relative to each feature point on the first KD tree.

[0135] S403. Calculate the ratio of the Euclidean distance between the nearest feature point and the second nearest feature point for each feature point;

[0136] S404. When the ratio is less than or equal to a preset threshold, it is confirmed that the feature point matches the nearest feature point of the feature point on the second KD tree; when the ratio is greater than the preset threshold, it is confirmed that the feature point does not match the nearest feature point of the feature point on the second KD tree.

[0137] S405. Determine whether the feature points in the first initial feature point set and the second initial feature point set match based on the number of matching feature points.

[0138] In an exemplary embodiment of this application, the KD-tree algorithm can be used to quickly find points of interest (matched feature pairs), and combined with Euclidean distance, the initial matching of feature points can be completed.

[0139] In an exemplary embodiment of this application, for the initial set of feature points of the two frames of images to be compared, a balanced KD tree of 64-dimensional feature points can be constructed respectively, and then the nearest search is performed using the KD tree, starting from the root node of the KD tree and recursively visiting the KD tree downwards.

[0140] In an exemplary embodiment of this application, the descriptor of the feature points in the behavior analysis alarm image can be X. i =(x i1 x i2 , ..., x i64 The descriptor of feature points in any frame of a surveillance video can be Y. i =(y i1 y i2 , ..., y i64 ).

[0141] In an exemplary embodiment of this application, the initial matching of feature points can use the ratio of Euclidean distance as a measure of feature similarity (which can be used to measure whether feature points match): assuming any two points in n-dimensional space are X... i Y i , where X i =(x i1 x i2 , ..., x in ) T ,Y i =(y i1 y i2 , ..., y in ) T The Euclidean distance between x and y is:

[0142]

[0143] When the feature similarity reaches the preset similarity threshold, the two feature points can be determined to match; when the feature similarity does not reach the preset similarity threshold, the two feature points can be determined to not match.

[0144] In an exemplary embodiment of this application, a search KD tree combined with Euclidean distance is used to search for the Euclidean distance d1 between each feature point Q and its nearest feature point Q1 and the Euclidean distance d2 between each feature point Q and its second nearest feature point Q2.

[0145]

[0146] Where d1 is the Euclidean distance of feature point Q to the nearest feature point Q1, and d2 is the Euclidean distance of feature point Q to the second nearest feature point Q2; τ is the feature point similarity, and the value of feature similarity can be in the range of [0, 1]. The smaller the value, the smaller the feature similarity, and vice versa.

[0147] In an exemplary embodiment of this application, if τ is less than a pre-defined threshold (which can be selected from the range of feature similarity values, for example, 0.5), then Q and Q1 can be considered a matching feature point pair; conversely, if τ is greater than or equal to the pre-defined threshold, then Q and Q1 can be considered a mismatch.

[0148] In an exemplary embodiment of this application, determining whether the feature points in the first initial set of feature points and the second initial set of feature points match based on the number of matching feature points may include:

[0149] When the number of matching feature points is greater than or equal to a preset threshold, it is determined that the feature points in the first initial feature point set and the second initial feature point set are matched.

[0150] When the number of matching feature points is less than the preset threshold, it is determined that the feature points in the first initial feature point set and the second initial feature point set do not match.

[0151] In an exemplary embodiment of this application, before determining whether the feature points in the first initial set of feature points and the second initial set of feature points match based on the number of matching feature points, the method may further include:

[0152] The Random Sampling Consensus (RANSAC) algorithm is used to remove erroneous feature points from confirmed matching feature points.

[0153] In the exemplary embodiments of this application, the scheme of this embodiment can eliminate feature pairs that are mistakenly identified as matching during feature point matching, thereby ensuring the accuracy of the matching results and ensuring that the features are included.

[0154] In an exemplary embodiment of this application, the step of using the Random Sampling Consensus (RANSAC) algorithm to remove error feature points from confirmed matching feature points may include:

[0155] A. Use the feature point pairs formed by matching feature points as interest points;

[0156] B. Using the RANSAC (Random Sample Consensus) algorithm, a preset number of interest points are extracted from the interest points to calculate the perspective transformation model of the behavior analysis alarm image;

[0157] C. Calculate the distance between the feature points corresponding to the unextracted interest points and the perspective transformation model, and obtain the maximum value among all calculated distances, storing it in a preset first set; detect whether the current iteration number has reached the preset iteration number; when the current iteration number has reached the preset iteration number, proceed to step D; when the current iteration number has not reached the preset iteration number, proceed to step B;

[0158] D. Compare the values ​​contained in the first set, and take the perspective transformation model corresponding to the largest value in the first set as the preferred feature point set.

[0159] In an exemplary embodiment of this application, the RANSAC algorithm can be used to extract at least 8 pairs of interest points from all matched interest points multiple times to calculate the perspective transformation model M of the image, and to calculate the distance m between the remaining feature points and the perspective transformation model M. After multiple iterations, the perspective transformation model M with the largest m is selected as the preferred set of feature points.

[0160] In an exemplary embodiment of this application, the matching of feature points in the first initial feature point set and the second initial feature point set can be determined based on the preferred feature point set and the number of matching feature points.

[0161] In an exemplary embodiment of this application, determining whether to update the target object feature library based on each comparison result may include:

[0162] When the feature points in the first initial feature point set and the second initial feature point set match, the target object feature library remains unchanged;

[0163] When the feature points in the first initial feature point set and the second initial feature point set do not match, the target object feature library is updated using the part features in the image compared with the behavior analysis alarm image.

[0164] In an exemplary embodiment of this application, after comparing the part features in the target object feature library with the part features of the body part region of the target object in any frame of the surveillance video, before continuing to obtain the part features of the body part region in the previous frame, the method may further include:

[0165] Based on the target object feature library, the sparse optical flow method is used to predict the body position of the target object in the previous frame image; wherein, the previous frame image refers to the frame image preceding any frame image compared with the current behavior analysis alarm image.

[0166] In an exemplary embodiment of this application, predicting the body position of the target object in the previous frame image using sparse optical flow based on the target object feature library may include:

[0167] Extract pixels within the feature point neighborhood of the target object feature library;

[0168] Optical flow estimation is performed on the pixels to calculate the body position of the target object in the previous frame image.

[0169] In an exemplary embodiment of this application, in order to reduce the computational load when extracting body feature regions, the body position of the target object in the previous frame image can be predicted in advance.

[0170] In an exemplary embodiment of this application, the Harris-SURF algorithm is used to extract and compare the features of the monitored video frame by frame, and a weighted direction histogram of the optical flow field is established to predict the body position of the target object in the previous frame.

[0171] In an exemplary embodiment of this application, pixels in the neighborhood of Harris-SURF feature points can be extracted to perform optical flow estimation, thereby calculating the limb position of the target object in the previous frame image.

[0172] In an exemplary embodiment of this application, the body position of the target object in the surveillance video typically changes only slightly between two adjacent frames. Therefore, pixels in the neighborhood of Harris-SURF feature points can be extracted, and the optical flow equation is established as follows:

[0173] I(x,y,t)=I(x+dx,y+dy,t+dt); (11)

[0174] Where I(x, y, t) is the gray value of pixel (x, y) at time t;

[0175] Pixels p1, p2, ..., p n If the pixel is a neighborhood of the Harris-SURF feature point (x, y), then performing a Taylor expansion of equation (11) and dividing by dt yields:

[0176]

[0177] in, These are the components of the optical flow of that pixel along the x and y directions, respectively.

[0178] These are the partial derivatives of the grayscale value I with respect to x, y, and t, respectively.

[0179] In an exemplary embodiment of this application, an approximate solution can be obtained by the least squares method. The pixels in the neighborhood of the Harris-SURF feature point are non-singular and satisfy the conditions of the optical flow equation.

[0180] In an exemplary embodiment of this application, the direction of the optical flow vector is calculated, the value range is divided into 12 parts according to the principle of equal division, the direction histogram of the optical flow vector is calculated, the direction with the largest displacement expectation is selected as the moving direction of the target object, and the amplitude is the displacement expectation of the moving direction.

[0181] In an exemplary embodiment of this application, the method may further include:

[0182] Each time it is predicted that the body position of the target object in the previous frame image is not within the preset tracking area, the part features of the body part region in the previous frame image are obtained based on the predicted body position of the target object in the previous frame image; and the target object feature library is updated using the extracted part features of the body part region in the previous frame image.

[0183] Each time the body position of the target object in the previous frame image is predicted to be within the preset tracking area, the target object feature library is kept unchanged.

[0184] In an exemplary embodiment of this application, if the body position predicted based on feature points is not within a preset tracking area, it indicates that the behavior of the target object in the previous frame image is significantly different from the behavior of the target object in the target object feature library. In this case, the target object feature library can be updated. Conversely, if the predicted body position is within the preset tracking area, it indicates that the behavior of the target object in the previous frame image is not significantly different from the behavior of the target object in the target object feature library. Therefore, the target object feature library does not need to be updated, and the current target object feature library remains unchanged.

[0185] This application embodiment also provides an object feature tracking device 1, such as Figure 6 As shown, the device may include a processor 11 and a computer-readable storage medium 12, wherein the computer-readable storage medium 12 stores instructions that, when executed by the processor 11, implement the object feature tracking method.

[0186] In the exemplary embodiments of this application, any of the embodiments in the foregoing object feature tracking method embodiments are applicable to this device embodiment, and will not be described in detail here.

[0187] It will be understood by those skilled in the art that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

Claims

1. A method for tracking object features, characterized in that, The method includes: Upon receiving a behavior analysis alarm image, the monitoring video prior to the alarm is retrieved; the behavior analysis alarm image contains the target object. Extract the body part features of the target object in the behavior analysis alarm image and store them in a preset target object feature library; Starting from the last frame of the surveillance video, the body part features in the target object feature library are compared sequentially with the body part features of the target object in the images contained in the surveillance video. Determine whether to update the target object feature library based on the results of each comparison; Extracting the body part features of the target object in each frame of the behavior analysis alarm image and the surveillance video, including: Obtain the body part region of the target object in the image from which the features of the part to be extracted; The body part region is extracted using a preset feature extraction algorithm; The feature extraction algorithm includes a first feature extraction algorithm and a second feature extraction algorithm; the step of extracting the part features of the body part region using a preset feature extraction algorithm includes: The first feature extraction algorithm is used to extract corner points of the body part region, and one or more feature blocks in the image are selected based on the corner points to form a first feature point set; the second feature extraction algorithm is used to extract feature points of the body part region to form a second feature point set. The step of sequentially comparing the part features in the target object feature library with the part features of the body parts of the target object in the images contained in the surveillance video includes: comparing the part features of the behavior analysis alarm image with the part features of any frame in the surveillance video according to the following method: The first feature point set and the second feature point set corresponding to the behavior analysis alarm image are merged to obtain a first initial feature point set; the first feature point set and the second feature point set corresponding to any frame image in the monitoring video are merged to obtain a second initial feature point set. Using the KD-tree algorithm and Euclidean distance, calculate whether the feature points in the first initial set of feature points and the second initial set of feature points match.

2. The object feature tracking method according to claim 1, characterized in that, The step of obtaining the body part region of the target object in the image from which the feature of the part to be extracted includes: When the image from which the feature of the part to be extracted is the behavior analysis alarm image, a preset edge detection algorithm is used to extract the object outline of the target object in the target object box of the behavior analysis alarm image; the body part region is segmented according to the proportion of the object outline; When the image from which the feature of the part to be extracted is each frame of the surveillance video, the edge detection algorithm is used to extract the object outline of the target object in each frame of the image based on the target object box in the behavior analysis alarm image; the body part region is segmented according to the object outline in proportion.

3. The object feature tracking method according to claim 1, characterized in that, The step of using the KD-tree algorithm and Euclidean distance to calculate whether the feature points in the first initial set of feature points and the second initial set of feature points match includes: Construct a balanced dimension search tree (KD tree) for the multidimensional feature points of the first initial feature point set, denoted as the first KD tree, and a balanced dimension search tree (KD tree) for the multidimensional feature points of the second initial feature point set, denoted as the second KD tree. Starting from the root node of the first KD tree and the second KD tree, recursively visit the first KD tree and the second KD tree to perform the nearest search, and obtain the nearest and second nearest feature points in the second KD tree relative to each feature point in the first KD tree; Calculate the ratio of the Euclidean distance between the nearest and second nearest feature points for each feature point; When the ratio is less than or equal to a preset threshold, it is confirmed that the feature point matches the nearest feature point of the feature point on the second KD tree; when the ratio is greater than the preset threshold, it is confirmed that the feature point does not match the nearest feature point of the feature point on the second KD tree. The number of matching feature points determines whether the feature points in the first initial feature point set and the second initial feature point set match.

4. The object feature tracking method according to claim 3, characterized in that, Before determining whether the feature points in the first initial set of feature points and the second initial set of feature points match based on the number of matching feature points, the method further includes: The Random Sampling Consensus (RANSAC) algorithm is used to remove erroneous feature points from confirmed matching feature points. The step of using the Random Sampling Consensus (RANSAC) algorithm to remove erroneous feature points from confirmed matching feature points includes: A. Use the feature point pairs formed by the loss of mutually matching feature points as interest points; B. Using the Random Sampling Consensus (RANSAC) algorithm, a preset number of interest points are extracted from the interest points to calculate the perspective transformation model of the behavior analysis alarm image; C. Calculate the distance between the feature points corresponding to the unextracted interest points and the perspective transformation model, and obtain the maximum value among all calculated distances, storing it in a preset first set; detect whether the current iteration number has reached the preset iteration number; when the current iteration number has reached the preset iteration number, proceed to step D; when the current iteration number has not reached the preset iteration number, proceed to step B; D. Compare the values ​​contained in the first set, and take the perspective transformation model corresponding to the largest value in the first set as the preferred feature point set.

5. The object feature tracking method according to claim 1, characterized in that, The step of determining whether to update the target object feature library based on each comparison result includes: When the feature points in the first initial feature point set and the second initial feature point set match, the target object feature library remains unchanged; When the feature points in the first initial feature point set and the second initial feature point set do not match, the target object feature library is updated using the part features in the image compared with the behavior analysis alarm image.

6. The object feature tracking method according to claim 1, characterized in that, After comparing the part features in the target object feature library with the part features of the body part region of the target object in any frame of the surveillance video, before obtaining the part features of the body part region in the previous frame, the method further includes: Based on the target object feature library, the sparse optical flow method is used to predict the body position of the target object in the previous frame image; wherein, the previous frame image refers to the frame image preceding any frame image compared with the current behavior analysis alarm image.

7. The object feature tracking method according to claim 6, characterized in that, The step of predicting the body position of the target object in the previous frame image using the sparse optical flow method based on the target object feature library includes: Extract pixels within the feature point neighborhood of the target object feature library; Optical flow estimation is performed on the pixels to calculate the body position of the target object in the previous frame image.

8. The object feature tracking method according to claim 7, characterized in that, The method further includes: Each time it is predicted that the body position of the target object in the previous frame image is not within the preset tracking area, the part features of the body part region in the previous frame image are obtained based on the predicted body position of the target object in the previous frame image; and the target object feature library is updated using the extracted part features of the body part region in the previous frame image. Each time the body position of the target object in the previous frame image is predicted to be within the preset tracking area, the target object feature library is kept unchanged.

9. An object feature tracking device, comprising a processor and a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, characterized in that, When the instruction is executed by the processor, the object feature tracking method as described in any one of claims 1-8 is implemented.

Images