Video tracking trajectory correction method and system based on short-time behavior consistency

Abstract

The invention discloses a video tracking trajectory correction method and system based on short-time behavior consistency. The method comprises the following steps: carrying out abnormal value detection and low-pass filtering processing on a corrected video tracking trajectory to obtain a video trajectory sequence; constructing a short-time energy sequence based on the short-time energy characteristic value of each track point in the sequence; extracting a non-steering track fragment and a steering track fragment from the video track sequence based on the short-time energy sequence; correcting a plurality of track points in each non-steering track segment and a plurality of track points in each steering track segment based on the short-time consistency speed and the short-time consistency direction angle; and sequentially splicing the plurality of non-steering track correction segments and the plurality of steering track correction segments so as to realize video tracking track correction. According to the method, multi-target tracking trajectory correction is carried out by taking the short-time window as a unit based on the short-time behavior consistency rule of human walking, and the trajectory precision is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and particularly relates to a video tracking trajectory correction method and system based on short-term behavior consistency. Background Art

[0002] Video pedestrian trajectory data is mainly extracted by a multi-object tracking system (MOT) and is widely used in video anomaly detection such as covert following detection, trajectory anomaly detection, video crowd detection, trajectory similarity calculation, and video human behavior understanding research such as human intention / target / plan detection. However, statistics show that there are large detection errors in the trajectory coordinates generated by the multi-object detection system, and the current MOT system has proposed some post-processing methods for problems such as ID drift, trajectory smoothing, and outliers.

[0003] However, the existing technology still has deficiencies: (1) The current MOT post-processing methods only focus on ID drift, trajectory smoothing, and outlier processing, etc., and the improvement of trajectory accuracy is very limited, which hinders the correctness of human behavior understanding research. (2) The standard deviation of the direction change of the multi-object tracking trajectory within 1 s is as high as 25°, which is contrary to the normal walking law of humans. Therefore, the current MOT post-processing methods ignore the influence of the essential walking law of humans and cannot fundamentally improve the trajectory accuracy.

[0004] The above content is only used to assist in understanding the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main object of the present invention is to provide a video tracking trajectory correction method and system based on short-term behavior consistency, aiming to solve the technical problem of how to improve the trajectory accuracy.

[0006] To achieve the above object, the present invention provides a video tracking trajectory correction method based on short-term behavior consistency, and the video tracking trajectory correction method based on short-term behavior consistency includes:

[0007] Perform outlier detection and low-pass filtering on the video tracking trajectory to be corrected to obtain a video trajectory sequence;

[0008] Determine the short-term energy eigenvalue of each trajectory point in the video trajectory sequence, and construct a short-term energy sequence based on the short-term energy eigenvalues of each trajectory point;

[0009] Extract a plurality of non-turning trajectory segments and a plurality of turning trajectory segments from the video trajectory sequence based on the short-term energy sequence to generate a trajectory segment set sequence, and the non-turning trajectory segments and the turning trajectory segments are adjacent to each other before and after;

[0010] Based on the short - term consistency speed and the short - term consistency direction angle respectively, correct multiple trajectory points within each un - turned trajectory segment and multiple trajectory points within each turned trajectory segment to obtain multiple corrected un - turned trajectory segments and multiple corrected turned trajectory segments;

[0011] Sequentially splice the multiple corrected un - turned trajectory segments and the multiple corrected turned trajectory segments to achieve video tracking trajectory correction.

[0012] Optionally, the step of performing outlier detection and low - pass filtering on the video tracking trajectory to be corrected to obtain a video trajectory sequence includes:

[0013] Perform outlier detection on the video tracking trajectory to be corrected through the DBSCAN clustering function, and perform median interpolation on the outliers to obtain an interpolated trajectory sequence;

[0014] Determine the target filter coefficient through the hyperopt hyperparameter optimization algorithm, and perform low - pass filtering on the interpolated trajectory sequence based on the target filter coefficient to obtain a video trajectory sequence, where the video trajectory sequence includes multiple trajectory points.

[0015] Optionally, the step of determining the short - term energy eigenvalue of each trajectory point in the video trajectory sequence includes:

[0016] Determine the target step size through the hyperopt hyperparameter optimization algorithm;

[0017] Based on the target step size, determine the pre - vector and post - vector of each trajectory point in the video trajectory sequence;

[0018] Calculate the short - term energy eigenvalue of each trajectory point according to the pre - vector and the post - vector.

[0019] Optionally, the step of extracting multiple un - turned trajectory segments and multiple turned trajectory segments from the video trajectory sequence based on the short - term energy sequence includes:

[0020] Determine the target energy threshold through the hyperopt hyperparameter optimization algorithm;

[0021] Traverse the short - term energy eigenvalues of each trajectory point in the short - term energy sequence;

[0022] Regard the trajectory points with short - term energy eigenvalues greater than the target energy threshold as turning points;

[0023] Determine the start and end coordinate indices of each turning stage according to the coordinate index sequence of the turning points;

[0024] Extract multiple un - turned trajectory segments and multiple turned trajectory segments from the video trajectory sequence according to the start and end coordinate indices of each turning stage.

[0025] Optionally, before the step of correcting multiple trajectory points in each unturned trajectory segment based on the short-term consistency speed and the short-term consistency direction angle respectively, the method further includes:

[0026] Determining a target window length, a target global speed ratio, and a target global direction angle ratio through the hyperopt hyperparameter optimization algorithm;

[0027] Respectively determining a short-term window trajectory sequence corresponding to each unturned trajectory segment, where the short-term window trajectory sequence includes multiple short-term window trajectories;

[0028] Calculating the global speed, the global U-axis direction angle, and the global V-axis direction angle corresponding to each unturned trajectory segment, and calculating the local speed, the local U-axis direction angle, and the local V-axis direction angle corresponding to each short-term window trajectory;

[0029] Calculating the short-term consistency speed according to the target global speed ratio, the global speed, and the local speed;

[0030] Calculating the short-term consistency direction angle according to the target global direction angle ratio, the local direction angle ratio, the global U-axis direction angle, the global V-axis direction angle, the local U-axis direction angle, and the local V-axis direction angle.

[0031] Optionally, the step of calculating the global speed, the global U-axis direction angle, and the global V-axis direction angle corresponding to each unturned trajectory segment includes:

[0032] Calculating the global speed corresponding to each unturned trajectory segment according to the U-axis unit vector and the V-axis unit vector;

[0033] Respectively calculating the global U-axis direction angle between each unturned trajectory segment and the U-axis unit vector, and respectively calculating the global V-axis direction angle between each unturned trajectory segment and the V-axis unit vector.

[0034] Optionally, the step of calculating the local speed, the local U-axis direction angle, and the local V-axis direction angle corresponding to each short-term window trajectory includes:

[0035] Calculating the local speed corresponding to each short-term window trajectory according to the U-axis unit vector and the V-axis unit vector;

[0036] Respectively calculating the local U-axis direction angle between each short-term window trajectory and the U-axis unit vector, and respectively calculating the local V-axis direction angle between each short-term window trajectory and the V-axis unit vector.

[0037] Optionally, before the step of correcting multiple trajectory points in each turned trajectory segment based on the short-term consistency speed and the short-term consistency direction angle respectively, the method further includes:

[0038] Perform short-time window partitioning on each turning trajectory segment respectively to obtain a first turning short-time window and a second turning short-time window corresponding to each turning trajectory segment;

[0039] Calculate the local direction angle of the U-axis, the local direction angle of the V-axis and the local velocity corresponding to the first turning short-time window and the second turning short-time window respectively;

[0040] Calculate the short-time consistency velocity and the short-time consistency direction angle according to the local direction angle of the U-axis, the local direction angle of the V-axis and the local velocity.

[0041] In addition, to achieve the above object, the present invention also provides a video tracking trajectory correction system based on short-time behavior consistency, and the video tracking trajectory correction system based on short-time behavior consistency includes:

[0042] A processing module, configured to perform outlier detection and low-pass filtering processing on the video tracking trajectory to be corrected to obtain a video trajectory sequence;

[0043] A calculation module, configured to determine the short-time energy eigenvalue of each trajectory point in the video trajectory sequence, and construct a short-time energy sequence based on the short-time energy eigenvalues of each trajectory point;

[0044] A judgment module, configured to extract a plurality of non-turning trajectory segments and a plurality of turning trajectory segments from the video trajectory sequence based on the short-time energy sequence to generate a trajectory segment set sequence, and the non-turning trajectory segments and the turning trajectory segments are adjacent before and after;

[0045] A correction module, configured to correct a plurality of trajectory points in each non-turning trajectory segment and a plurality of trajectory points in each turning trajectory segment respectively based on the short-time consistency velocity and the short-time consistency direction angle to obtain a plurality of non-turning trajectory correction segments and a plurality of turning trajectory correction segments;

[0046] A splicing module, configured to perform sequential splicing on a plurality of non-turning trajectory correction segments and a plurality of turning trajectory correction segments to achieve video tracking trajectory correction.

[0047] In addition, to achieve the above object, the present invention also provides a video tracking trajectory correction device based on short-time behavior consistency, and the device includes: a memory, a processor, and a video tracking trajectory correction program based on short-time behavior consistency stored on the memory and executable on the processor, and the video tracking trajectory correction program based on short-time behavior consistency is configured to implement the steps of the video tracking trajectory correction method based on short-time behavior consistency as described above.

[0048] In addition, to achieve the above object, the present invention further provides a storage medium, on which a video tracking trajectory correction program based on short-term behavior consistency is stored. When the video tracking trajectory correction program based on short-term behavior consistency is executed by a processor, the steps of the video tracking trajectory correction method based on short-term behavior consistency as described above are implemented.

[0049] The present invention first performs outlier detection and low-pass filtering on the video tracking trajectory to be corrected to obtain a video trajectory sequence, then determines the short-term energy eigenvalue of each trajectory point in the video trajectory sequence, constructs a short-term energy sequence based on the short-term energy eigenvalues of each trajectory point, and then extracts a plurality of non-turning trajectory segments and a plurality of turning trajectory segments from the video trajectory sequence based on the short-term energy sequence to generate a trajectory segment set sequence, where the non-turning trajectory segments and the turning trajectory segments are adjacent to each other before and after. Finally, the short-term consistency speed and the short-term consistency direction angle are respectively used to correct a plurality of trajectory points in each non-turning trajectory segment and a plurality of trajectory points in each turning trajectory segment to obtain a plurality of non-turning trajectory correction segments and a plurality of turning trajectory correction segments, and the plurality of non-turning trajectory correction segments and the plurality of turning trajectory correction segments are sequentially spliced to achieve video tracking trajectory correction. Based on the short-term behavior consistency law of human walking, the present invention performs multi-target tracking trajectory correction in units of short-term windows, which not only improves the trajectory accuracy but also makes the corrected trajectory more conform to the walking law of humans. Description of the Drawings

[0050] Figure 1 is a schematic structural diagram of a video tracking trajectory correction device based on short-term behavior consistency in the hardware operating environment involved in the embodiment of the present invention;

[0051] Figure 2 is a schematic flowchart of the first embodiment of the video tracking trajectory correction method based on short-term behavior consistency of the present invention;

[0052] Figure 3 is a schematic diagram of turning judgment in the first embodiment of the video tracking trajectory correction method based on short-term behavior consistency of the present invention;

[0053] Figure 4 is a schematic diagram of non-turning segment correction in the first embodiment of the video tracking trajectory correction method based on short-term behavior consistency of the present invention;

[0054] Figure 5 is a schematic diagram of turning segment correction in the first embodiment of the video tracking trajectory correction method based on short-term behavior consistency of the present invention;

[0055] Figure 6 is a schematic block diagram of the first embodiment of the video tracking trajectory correction system based on short-term behavior consistency of the present invention.

[0056] The realization, functional features and advantages of the present invention will be further described with reference to the embodiments and the accompanying drawings. Specific Embodiments

[0057] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0058] Refer to Figure 1 , Figure 1 which is a schematic structural diagram of a video tracking trajectory correction device based on short-term behavior consistency for the hardware operating environment involved in the embodiment solution of the present invention.

[0059] As Figure 1 shown, the video tracking trajectory correction device based on short-term behavior consistency may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display) and an input unit such as a keyboard (Keyboard). Optionally, the user interface 1003 may further include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless-fidelity (Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM), or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. Optionally, the memory 1005 may also be a storage system independent of the aforementioned processor 1001.

[0060] Those skilled in the art can understand that Figure 1 the structure shown in

[0061] As Figure 1 shown, the memory 1005, as a storage medium, may include an operating system, a network communication module, a user interface module, and a video tracking trajectory correction program based on short-term behavior consistency.

[0062] In Figure 1In the video tracking trajectory correction device based on short-term behavior consistency shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the video tracking trajectory correction device based on short-term behavior consistency of the present invention can be arranged in the video tracking trajectory correction device based on short-term behavior consistency. The video tracking trajectory correction device based on short-term behavior consistency calls the video tracking trajectory correction program stored in the memory 1005 through the processor 1001 and executes the video tracking trajectory correction method provided by the embodiments of the present invention.

[0063] Embodiments of the present invention provide a video tracking trajectory correction method based on short-term behavior consistency. Refer to Figure 2 , Figure 2 which is a schematic flowchart of the first embodiment of the video tracking trajectory correction method based on short-term behavior consistency of the present invention.

[0064] In this embodiment, the video tracking trajectory correction method based on short-term behavior consistency includes the following steps:

[0065] Step S10: Perform outlier detection and low-pass filtering on the video tracking trajectory to be corrected to obtain a video trajectory sequence.

[0066] It is easy to understand that the execution subject of this embodiment can be a video tracking trajectory correction system based on short-term behavior consistency with functions such as data processing, network communication, and program running, or other computer devices with similar functions. This embodiment does not impose any restrictions.

[0067] In specific implementation, it is necessary to perform hyperopt hyperparameter optimization training on the filtering coefficient τ, step size s, energy threshold thres, window length l, global speed ratio σ v and the global speed ratio σ a to obtain the target filtering coefficient τ, target step size s, target energy threshold thres, target window length l, target global speed ratio σ v and the target global speed ratio σ a , and then perform trajectory processing through the target filtering coefficient τ, target step size s, target energy threshold thres, target window length l, target global speed ratio σ v and the target global speed ratio σ a to obtain the complete trajectory sequence of the corrected target.

[0068] It is defined as the following parameter space: And set appropriate selection ranges for these parameters in the form of [starting value, ending value, step size]: τ: [0.05, 0.1, 0.01], s: [30, 40, 1], thres: [0.3, 0.4, 0.1], l: [15, 24, 1], σ v : [0, 1, 0.1], σ a : [0, 1, 0.1]; and select appropriate parameters from these set selection ranges as the target filtering coefficient τ, target step size s, target energy threshold thres, target window length l, target global speed ratio σ v and target global speed ratio σ a .

[0069] Define the objective function: The objective function is defined as the trajectory correction scheme f VTC (·), the function input is the six hyperparameters in the parameter space inside, and the marked trajectory is denoted as The output of the objective function is the root mean square error (RMSE) e between the corrected trajectory and the marked trajectory:

[0070]

[0071] Furthermore, the marked trajectory is a manually marked high-precision true coordinate trajectory, serving as the training label;

[0072] Hyperparameter optimization training: First, given an appropriate number of iterations. In each iteration, the hyperopt optimization algorithm first randomly selects a set from the set parameter space and inputs it into the objective function, outputting the root mean square error e. Then, as the iteration progresses, minimize the root mean square error e. When e reaches the minimum, the output corrected trajectory is also the optimal complete trajectory after correction. The parameter space corresponding to the minimum root mean square error e is the optimal parameter combination (i.e., the optimal filtering coefficient τ, optimal step size s, optimal energy threshold thres, optimal window length l, optimal global speed ratio σ v and optimal global speed ratio σ a ):

[0073] best_faram = f min (f VTC (Q)) = f min (e)

[0074] It should also be noted that through hyperopt hyperparameter optimization training, the present invention can automatically search for the optimal parameter combination, thus avoiding cumbersome manual parameter tuning and improving research efficiency.

[0075] Further, the outlier detection is performed on the video tracking trajectory to be corrected through the DBSCAN clustering function, and the median interpolation is performed on the outliers to obtain the interpolated trajectory sequence; the target filtering coefficient is determined through the hyperopt hyperparameter optimization algorithm, and the low-pass filtering process is performed on the interpolated trajectory sequence based on the target filtering coefficient to obtain the video trajectory sequence, where the video trajectory sequence includes multiple trajectory points.

[0076] In a specific implementation, the video tracking trajectory to be corrected (i.e., the video tracking trajectory to be corrected) is input into the DBSCAN clustering function in the sklearn.cluster package, the outlier index is detected and output, and the median interpolation is performed on the outliers. The output interpolated trajectory sequence is denoted as

[0077] It should be noted that the detection radius parameter eps of the DBSCAN clustering function is set to 20, and the parameter min_samples of the minimum number of points in the circular neighborhood is set to 8.

[0078] It should be understood that through the target filtering coefficient, the trajectory sequence after outlier processing (i.e., the interpolated trajectory sequence) is input into the low-pass filter for trajectory filtering and smoothing processing. The filtering formula for the i-th point is as follows:

[0079] p i = τ × p0 i + (1 - τ) × p0 i-1

[0080] The preprocessed video trajectory sequence is output p i = (u i , v i ) is the i-th coordinate point (i.e., the trajectory point) in i , u i and v are the position coordinates in the UV coordinate system.

[0081] Step S20: Determine the short-time energy eigenvalue of each trajectory point in the video trajectory sequence, and construct a short-time energy sequence based on the short-time energy eigenvalues of each trajectory point.

[0082] Further, the target step size is determined through the hyperopt hyperparameter optimization algorithm; the pre-vector and post-vector of each trajectory point in the video trajectory sequence are determined based on the target step size; the short-time energy eigenvalue of each trajectory point is calculated according to the pre-vector and post-vector; and a short-time energy sequence is constructed based on the short-time energy eigenvalues of each trajectory point.

[0083] In this embodiment, the target step size is determined, and the preprocessed trajectory sequence output in step S10 is traversed For each trajectory point p in the sequence i , take the current point p i as the center, and obtain the forward vector p i -p i-s with a length of s forward and backward respectively, and the backward vector p i+s -p i , and calculate the included angle angle i between these two vectors:

[0084]

[0085] The energy value E i of the vector included angle angle i is expressed as the square value of angle i :

[0086] E i = |angle i | 2

[0087] Take this energy value as the short-time energy feature of the current point. Finally, output the short-time energy feature sequence E = {E 1 , …, E i , …, E f} of all trajectory points.

[0088] Furthermore, the direction of the forward vector p i -p i-s is from the previous point to the current point, and the direction of the backward vector p i+s -p i is from the current point to the next point.

[0089] Step S30: Extract multiple unturned trajectory segments and multiple turned trajectory segments from the video trajectory sequence based on the short-time energy sequence to generate a sequence of trajectory segment sets, where the unturned trajectory segments and the turned trajectory segments are adjacent to each other before and after.

[0090] Furthermore, determine the target energy threshold through the hyperopt hyperparameter optimization algorithm; traverse the short-time energy feature values of each trajectory point in the short-time energy sequence; take the trajectory points with short-time energy feature values greater than the target energy threshold as turning points; determine the start and end coordinate indices of each turning stage according to the coordinate index sequence of the turning points; extract multiple unturned trajectory segments and multiple turned trajectory segments from the video trajectory sequence according to the start and end coordinate indices of each turning stage.

[0091] In specific implementation, refer to Figure 3 , Figure 3Schematic diagram of steering judgment for the first embodiment of the video tracking trajectory correction method based on short-term behavior consistency of the present invention. Determine the target energy threshold thres, traverse the short-term energy feature sequence E output in step S20, determine the points exceeding the target energy threshold as steering points, and then extract the coordinate index sequence IS = {index 1 ,…,index i ,…}. Traverse IS. Since the steering phase is continuous, given a suitable threshold a, if the difference between the index index i+1 of the subsequent point and the current index value index i is less than <a, it is considered that the subsequent point and the current point belong to the same steering phase, otherwise the subsequent point belongs to the next steering phase. Finally, according to the start and end coordinate indices of each steering phase, output all non-steering sequences and steering sequence sets (i.e., the trajectory segment set sequence) are two adjacent non-steering segments and steering segments before and after, and m is the number of steering occurrences;

[0092] Further, for each steering segment, by backtracking the energy values corresponding to each steering point, the trajectory coordinates closest to the true steering point within the steering segment can be output.

[0093] Step S40: Correct multiple trajectory points within each non-steering trajectory segment and multiple trajectory points within each steering trajectory segment based on the short-term consistency speed and short-term consistency direction angle respectively to obtain multiple non-steering trajectory correction segments and multiple steering trajectory correction segments.

[0094] It should also be noted that before the step of correcting multiple trajectory points within each non-steering trajectory segment based on the short-term consistency speed and short-term consistency direction angle respectively, determine the target window length, target global speed ratio, and target global direction angle ratio through the hyperopt hyperparameter optimization algorithm; respectively determine the short-term window trajectory sequences corresponding to each non-steering trajectory segment, and the short-term window trajectory sequences include multiple short-term window trajectories; calculate the global speed, U-axis global direction angle, and V-axis global direction angle corresponding to each non-steering trajectory segment, and calculate the local speed, U-axis local direction angle, and V-axis local direction angle corresponding to each short-term window trajectory; calculate the short-term consistency speed according to the target global speed ratio, global speed, and local speed; calculate the short-term consistency direction angle according to the target global direction angle ratio, local direction angle ratio, U-axis global direction angle, V-axis global direction angle, U-axis local direction angle, and V-axis local direction angle.

[0095] The processing method for calculating the global velocity, the global U-axis direction angle, and the global V-axis direction angle corresponding to each unturned trajectory segment is to calculate the global velocity corresponding to each unturned trajectory segment according to the U-axis unit vector and the V-axis unit vector; calculate the global U-axis direction angle between each unturned trajectory segment and the U-axis unit vector respectively, and calculate the global V-axis direction angle between each unturned trajectory segment and the V-axis unit vector respectively.

[0096] The processing method for calculating the local velocity, the local U-axis direction angle, and the local V-axis direction angle corresponding to each short-time window trajectory is to calculate the local velocity corresponding to each short-time window trajectory according to the U-axis unit vector and the V-axis unit vector; calculate the local U-axis direction angle between each short-time window trajectory and the U-axis unit vector respectively, and calculate the local V-axis direction angle between each short-time window trajectory and the V-axis unit vector respectively.

[0097] In this embodiment, referring to Figure 4 , Figure 4 is the schematic diagram of the correction of the unturned segment in the first embodiment of the video tracking trajectory correction method based on short-time behavior consistency of the present invention. The detailed steps of the correction of the unturned segment are as follows:

[0098] 1. Short-time window division: Starting from the first unturned segment , with the optimal window length l, divide the unturned segment (i.e., the unturned trajectory segment) into several consecutive short-time window trajectory sequences is the i-th short-time window trajectory within . Among them, k is the number of trajectory points (number of frames, i.e., time) included in , z is the number of short-time windows divided for this segment , 1 ≤ i ≤ z, l < k is the number of trajectory points included in SPiN;

[0099] 2. Calculate the global UV direction angle and the global velocity: Given that the U-axis unit vector is defined as The V-axis unit vector is defined as Calculate the global U-axis direction angle between the entire unturned segment and the U-axis unit vector

[0100] Δu 1k = u k - u 1 , Δv 1k = v k - v 1

[0101]

[0102] Among them, k is the entire un-turned segment containing the number of trajectory points (number of frames, i.e., time), Δu 1k , Δv 1k respectively represent the differences in the U coordinate and V coordinate between the starting and ending points of this segment. ∥Δu 1k , Δv 1k )∥ represents the pixel distance of the un-turned trajectory segment .

[0103] The global speed is the average speed of the pixel distance between the starting and ending points of this un-turned segment .

[0104]

[0105] 3. Calculate the local UV direction angle and local speed as follows: Similar to calculating the global direction angle and global speed. Calculate the short-time window trajectory SP i N and the local U-axis direction angle with the unit vector of the U-axis and the local V-axis direction angle with the unit vector of the V-axis The local speed is the average speed V(SP i N ) of the distance between the starting and ending points of this short-time window trajectory SP i N ):

[0106]

[0107] Among them, l is the number of trajectory points (number of frames, i.e., time) included in this short-time window.

[0108] 4. Calculate the short-time consistency speed as follows: Determine the target global speed ratio σ v , then the local speed ratio is 1 - σ v . Then the short-time consistency speed V i N of the short-time window trajectory SP ′ (SP i N ) is expressed as the weighted cooperation of the global speed and the local speed:

[0109]

[0110] 5. Calculate the short-time consistency direction angle reflected in the U and V directions: Similar to calculating the short-time consistency speed. Determine the target global direction angle ratio σ a , then the local direction angle ratio is 1 - σ a . Then the short-time window trajectory SP i NThe short-term consistent UV direction angle is expressed as the weighted collaboration of the global UV direction angle and the local UV direction angle:

[0111]

[0112] In the correction stage, the U and V coordinates are corrected frame by frame in the U and V directions respectively. The first point of the unturned segment is used as the correction starting point. For the short-term window SP i N , given the short-term consistent speed V ′ (SP i N ), the U component is expressed as the product of the cosine value of the short-term consistent direction angle with the U axis: Similarly, the V component of the speed is expressed as: Then the pixel distance in the U direction at the future j-th frame can be expressed as the product of the U component of the short-term consistent speed V ′ (SP i N ) and the number of frames. Similarly, the pixel distance in the V direction can be expressed as the product of the V ′ (SP i N ) component and the number of frames. If the correction coordinates of the j-th point are then the U and V coordinates of the (j + 1)-th point are expressed as the sum of the U and V coordinates of the j-th point and the pixel distance within one frame. Refer to Figure 4 , Figure 4 is the schematic diagram of the correction of the unturned segment in the first embodiment of the video tracking trajectory correction method based on short-term behavior consistency of the present invention. Figure 4 The (b) and (c) in

[0113]

[0114] show the comparison between the correction schematic diagram of the (j + 1)-th point and the correction trajectory of this short-term window.

[0115] Furthermore, the correction starting point is set as: the correction starting point of the entire trajectory sequence is the first point, and for subsequent unturned segments or turned segments, the correction starting point of each short-term window is the last point of the previously corrected short-term window. Furthermore, after all short-term windows of the unturned segment

[0116] Further, before the step of correcting multiple trajectory points in each steering trajectory segment based on the short-term consistency speed and the short-term consistency direction angle respectively, perform short-term window division on each steering trajectory segment to obtain a first steering short-term window and a second steering short-term window corresponding to each steering trajectory segment; calculate the local direction angle of the U-axis, the local direction angle of the V-axis, and the local speed corresponding to the first steering short-term window and the second steering short-term window respectively; calculate the short-term consistency speed and the short-term consistency direction angle according to the local direction angle of the U-axis, the local direction angle of the V-axis, and the local speed.

[0117] In this embodiment, the detailed steps of steering segment correction are as follows:

[0118] 1. Short-term window division: For the steering segment Taking the window steering point obtained in step S20 as the center, divide it into two short-term windows: the first steering short-term window and the second steering short-term window where w is the number of trajectory points (number of frames, i.e., time) included in the entire steering segment and M < w;

[0119] 2. Calculate the local UV direction angle and the local speed: For the steering short-term window 1 ≤ i ≤ 2, output its local direction angle of the U-axis local direction angle of the V-axis and the average speed (i.e., the local speed)

[0120]

[0121] where, when i = 1, d = M, indicating the number of trajectory points (number of frames, i.e., time) included in the short-term window When i = 2, d = w - M, indicating the number of trajectory points (number of frames, i.e., time) included in the short-term window

[0122] 3. Calculate the steering short-term consistency speed: The short-term consistency speed of the short-term window is expressed as the mean of the current local speed and the short-term consistency speed of the previous short-term window. The short-term consistency direction angle is the local UV direction angle;

[0123] Further, the previous short-term window refers to the previous short-term window on the overall time axis, and this window may belong to an unsteered segment or the current steering segment.

[0124] 4. Steering segment correction: Correct all short-term windows of the unsteered segment in sequence: Take the last point of the previous unturned segment as the calibration starting point For the short - time window Given the short - time consistency speed The U - component is expressed as the product of the cosine value of the short - time consistency direction angle of the U - axis: Similarly, the V - component of the speed is expressed as: Then the pixel distance in the U - direction of the j - th future frame can be expressed as the product of the U - component of the short - time consistency speed of this short - time window and the number of frames. Similarly, the pixel distance in the V - direction can be expressed as the product of the V - component and the number of frames. If the calibration coordinates of the j - th point are then the U and V coordinates of the (j + 1)-th point are expressed as the sum of the U and V coordinates of the j - th point and the pixel distance within one frame.

[0125]

[0126] Further, denote the calibrated turned segment (i.e., the turned trajectory calibration segment) as: Refer to Figure 5 ,[[]]END]] Figure 5 which is the schematic diagram of the turned segment calibration in the first embodiment of the video tracking trajectory calibration method based on short - time behavior consistency of the present invention.

[0127] Step S50: Sequentially splice multiple unturned trajectory calibration segments and multiple turned trajectory calibration segments to achieve video tracking trajectory calibration.

[0128] Further, splice each calibrated trajectory segment in sequence and output the calibrated complete trajectory sequence, denoted as to achieve video tracking trajectory calibration.

[0129] In this embodiment, a steering judgment algorithm based on short-time energy is first established, and the trajectory point closest to the true steering point is obtained. Secondly, the unsteered segment and the steered segment are divided into several consecutive short-time windows. A short-time window is often the trajectory of one step. Therefore, the walking speed and direction within this window should satisfy short-time behavior consistency. For a certain short-time window of the unsteered segment, the global speed and UV direction angle of the entire segment, as well as the local speed and UV direction angle of this short-time window, are weighted and coordinated to obtain the short-time consistent UV direction angle and speed of this window. Then, the UV coordinates of the unsteered trajectory are corrected frame by frame using the speed components of the consistent speed on the U-axis and V-axis. For the steered trajectory, the steered segment is divided into two short-time windows using the steering point, and only the local direction angle and local speed are used to represent short-time consistency. Finally, based on the short-time behavior consistency law of human walking, multi-target tracking trajectory correction is performed in units of short-time windows, greatly improving the trajectory accuracy.

[0130] Referring to Figure 6 , Figure 6 FIG. is a structural block diagram of the first embodiment of the video tracking trajectory correction system based on short-time behavior consistency of the present invention.

[0131] As Figure 6 shown, the video tracking trajectory correction system based on short-time behavior consistency proposed in the embodiment of the present invention includes:

[0132] A processing module 6001, configured to perform outlier detection and low-pass filtering processing on the video tracking trajectory to be corrected, and obtain a video trajectory sequence;

[0133] A calculation module 6002, configured to determine the short-time energy eigenvalue of each trajectory point in the video trajectory sequence, and construct a short-time energy sequence based on the short-time energy eigenvalue of each trajectory point;

[0134] A judgment module 6003, configured to extract a plurality of unsteered trajectory segments and a plurality of steered trajectory segments from the video trajectory sequence based on the short-time energy sequence, so as to generate a trajectory segment set sequence, where the unsteered trajectory segments and the steered trajectory segments are adjacent to each other before and after;

[0135] A correction module 6004, configured to correct a plurality of trajectory points in each unsteered trajectory segment and a plurality of trajectory points in each steered trajectory segment based on the short-time consistent speed and the short-time consistent direction angle, respectively, to obtain a plurality of unsteered trajectory correction segments and a plurality of steered trajectory correction segments;

[0136] A splicing module 6005, configured to sequentially splice a plurality of unsteered trajectory correction segments and a plurality of steered trajectory correction segments to implement video tracking trajectory correction.

[0137] Other embodiments or specific implementations of the video tracking trajectory correction system based on short-term behavior consistency of the present invention can be referred to the above method embodiments, which will not be elaborated here.

[0138] It should be noted that, in this document, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or system including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or system. Without further limitation, an element defined by the statement "including one..." does not exclude the presence of another identical element in the process, method, article or system including that element.

[0139] The serial numbers of the above embodiments of the present invention are only for description and do not represent the superiority or inferiority of the embodiments.

[0140] Through the description of the above embodiments, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present invention, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk), and includes several instructions for causing a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0141] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent structure or equivalent process transformation made by using the specification and drawings of the present invention, or directly or indirectly applied to other related technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. A video tracking trajectory correction method based on short-term behavior consistency, characterized in that: The video tracking trajectory correction method based on short-term behavior consistency comprises the following steps: Perform outlier detection and low-pass filtering on the video tracking trajectory to be corrected to obtain a video trajectory sequence; Determine the short-time energy characteristic value of each trajectory point in the video trajectory sequence, and construct a short-time energy sequence based on the short-time energy characteristic value of each trajectory point; Extracting a plurality of unsteered trajectory segments and a plurality of steered trajectory segments from the video trajectory sequence based on the short-time energy sequence to generate a trajectory segment set sequence, wherein the unsteered trajectory segments are adjacent to the steered trajectory segments in front and behind; Based on the short-term consistent speed and the short-term consistent direction angle, a plurality of trajectory points in each unturned trajectory segment and a plurality of trajectory points in each turned trajectory segment are corrected respectively to obtain a plurality of unturned trajectory correction segments and a plurality of turned trajectory correction segments; A plurality of unsteered trajectory correction segments and a plurality of steered trajectory correction segments are sequentially spliced ​​to achieve video tracking trajectory correction.

2. The method according to claim 1, characterized in that The step of performing outlier detection and low-pass filtering on the video tracking trajectory to be corrected to obtain a video trajectory sequence comprises: Perform outlier detection on the video tracking trajectory to be corrected by using the DBSCAN clustering function, and perform median interpolation on the outliers to obtain an interpolated trajectory sequence; The target filter coefficient is determined by a hyperopt hyperparameter optimization algorithm, and the interpolation trajectory sequence is subjected to low-pass filtering based on the target filter coefficient to obtain a video trajectory sequence, wherein the video trajectory sequence includes a plurality of trajectory points.

3. The method according to claim 2, characterized in that The step of determining the short-time energy characteristic value of each track point in the video track sequence comprises: Determining the target step size by the hyperopt hyperparameter optimization algorithm; Determining a pre-vector and a post-vector of each track point in the video track sequence based on the target step length; The short-time energy characteristic value of each trajectory point is calculated according to the preceding vector and the following vector.

4. The method according to claim 3, characterized in that The step of extracting a plurality of non-turning trajectory segments and a plurality of turning trajectory segments from the video trajectory sequence based on the short-time energy sequence comprises: Determining a target energy threshold by the hyperopt hyperparameter optimization algorithm; Traversing the short-time energy characteristic value of each trajectory point in the short-time energy sequence; Taking the trajectory point where the short-time energy characteristic value is greater than the target energy threshold as a turning point; Determine the start and end coordinate indexes of each turning stage according to the coordinate index sequence of the turning point; A plurality of non-turning trajectory segments and a plurality of turning trajectory segments are extracted from the video trajectory sequence according to the start and end coordinate indexes of each turning stage.

5. The method according to any one of claims 1 to 4, characterized in that: Before the step of respectively correcting a plurality of track points in each unturned track segment based on the short-time consistent speed and the short-time consistent direction angle, the method further includes: Determining a target window length, a target global speed ratio, and a target global direction angle ratio by the hyperopt hyperparameter optimization algorithm; Determine the short-time window trajectory sequence corresponding to each unturned trajectory segment respectively, wherein the short-time window trajectory sequence includes a plurality of short-time window trajectories; Calculate the global speed, U-axis global direction angle and V-axis global direction angle corresponding to each unturned trajectory segment, and calculate the local speed, U-axis local direction angle and V-axis local direction angle corresponding to each short-time window trajectory; calculating a short-term consistency speed according to the target global speed ratio, the global speed and the local speed; The short-time consistency direction angle is calculated according to the target global direction angle ratio, the local direction angle ratio, the U-axis global direction angle, the V-axis global direction angle, the U-axis local direction angle and the V-axis local direction angle.

6. The method according to claim 5, characterized in that The step of calculating the global speed, the U-axis global direction angle, and the V-axis global direction angle corresponding to each unturned trajectory segment includes: Calculate the global velocity corresponding to each unturned trajectory segment according to the U-axis unit vector and the V-axis unit vector; The U-axis global direction angle between each unsteered trajectory segment and the U-axis unit vector is calculated respectively, and the V-axis global direction angle between each unsteered trajectory segment and the V-axis unit vector is calculated respectively.

7. The method according to claim 6, characterized in that The step of calculating the local speed, the U-axis local direction angle and the V-axis local direction angle corresponding to each short-time window trajectory includes: Calculate the local velocity corresponding to each short-time window trajectory according to the U-axis unit vector and the V-axis unit vector; The U-axis local direction angle between each short-time window trajectory and the U-axis unit vector is calculated respectively, and the V-axis local direction angle between each short-time window trajectory and the V-axis unit vector is calculated respectively.

8. The method according to any one of claims 1 to 4, characterized in that: Before the step of respectively correcting a plurality of track points in each turning track segment based on the short-time consistent speed and the short-time consistent direction angle, the method further includes: Divide each turning trajectory segment into a short-time window respectively, and obtain a first turning short-time window and a second turning short-time window corresponding to each turning trajectory segment; Calculate the U-axis local direction angle, V-axis local direction angle and local speed corresponding to the first steering short-time window and the second steering short-time window respectively; A short-term consistent speed and a short-term consistent direction angle are calculated according to the U-axis local direction angle, the V-axis local direction angle and the local speed.

9. A video tracking trajectory correction system based on short-term behavior consistency, characterized in that: The video tracking trajectory correction system based on short-term behavior consistency includes: A processing module is used to perform outlier detection and low-pass filtering on the video tracking trajectory to be corrected to obtain a video trajectory sequence; A calculation module, used to determine the short-time energy characteristic value of each trajectory point in the video trajectory sequence, and construct a short-time energy sequence based on the short-time energy characteristic value of each trajectory point; A judgment module, configured to extract a plurality of unsteered trajectory segments and a plurality of steered trajectory segments from the video trajectory sequence based on the short-time energy sequence to generate a trajectory segment set sequence, wherein the unsteered trajectory segment is adjacent to the steered trajectory segment in front and behind; a correction module, configured to correct a plurality of trajectory points in each unturned trajectory segment and a plurality of trajectory points in each turned trajectory segment based on the short-term consistent speed and the short-term consistent direction angle, so as to obtain a plurality of unturned trajectory correction segments and a plurality of turned trajectory correction segments; The splicing module is used to sequentially splice multiple un-steering trajectory correction segments and multiple steering trajectory correction segments to achieve video tracking trajectory correction.

Images