Short trajectory matching methods, devices, electronic equipment and storage media

Abstract

This invention discloses a short trajectory matching method, apparatus, electronic device, and storage medium. The method includes: determining any two short trajectories as short trajectories to be matched, wherein the short trajectories include a first feature sequence based on human body features and a second feature sequence based on human head features; determining a first similarity between the short trajectories to be matched based on the first feature sequence, and determining a second similarity between the short trajectories to be matched based on the second feature sequence; and determining a matching result between the short trajectories to be matched based on the first similarity and the second similarity. By using the second feature sequence based on human head features as an auxiliary sequence, the problem of low image tracking accuracy when the human body is occluded is addressed. Simultaneously, since the short trajectories pre-incorporate some highly correlated human body features, the computational scale of frame-by-frame image tracking is reduced, thereby improving the processing speed of image tracking.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and specifically to a short trajectory matching method, apparatus, electronic device, and storage medium. Background Technology

[0002] Multiple short trajectory matching is an important computer vision task. A popular approach is to first detect the target and then associate the detection boxes and features between different frames. However, when tracking human bodies, this method is very prone to incorrect associations due to pose changes of the same person or mutual occlusion between different people. This leads to inaccurate trajectory matching in densely populated scenes, where occlusion between different people causes the trajectory matching to be inaccurate, resulting in a decrease in the accuracy of image tracking. Summary of the Invention

[0003] This invention provides a short trajectory matching method to address the problem of inaccurate trajectory matching and decreased image tracking accuracy in densely populated scenes due to occlusion between different people. The method uses a first similarity obtained from a first feature sequence based on human body features and a second similarity obtained from a second feature sequence based on human head features to match the short trajectory. The second feature sequence based on human head features is used as an auxiliary sequence to handle the low image tracking accuracy when human bodies are occluded. Furthermore, because the short trajectory pre-incorporates some highly correlated human body features, the computational scale of frame-by-frame image tracking is reduced, improving the processing speed of image tracking.

[0004] Firstly, the main objective of this invention is to provide a short trajectory matching method, comprising:

[0005] Determine any two short trajectories as short trajectories to be matched, wherein the short trajectories include a first feature sequence based on human body features and a second feature sequence based on human head features;

[0006] Based on the first feature sequence, a first similarity between the short trajectories to be matched is determined, and based on the second feature, a second similarity between the short trajectories to be matched is determined;

[0007] The matching result between the short trajectories to be matched is determined based on the first similarity and the second similarity.

[0008] Optionally, before determining any two short trajectories as short trajectories to be matched, the method further includes:

[0009] Image tracking processing is performed on the video to be tracked to obtain short trajectories based on human body features and head tracking trajectories based on human head features;

[0010] The human features in each frame of the short trajectory are matched with the head features in the head tracking trajectory to obtain the head feature identifier corresponding to each frame of human features.

[0011] A first feature sequence is determined based on the short trajectory, and a second feature sequence corresponding to the short trajectory is determined based on the human head feature identifier corresponding to the human body feature in each frame.

[0012] Optionally, the image tracking processing of the video to be tracked to obtain a short trajectory based on human features includes:

[0013] Human features are extracted from each frame of the video to be tracked to obtain the human features corresponding to each frame.

[0014] The video to be tracked is processed by image tracking based on the human body characteristics to obtain the original tracking trajectory based on human body characteristics;

[0015] Based on the original tracking trajectory, a trajectory node graph of the video to be tracked is determined, wherein each segment of the original tracking trajectory in the trajectory node graph corresponds to a node;

[0016] The trajectory node graph is segmented according to a preset segmentation rule to obtain a node subgraph;

[0017] Based on the node subgraph, a short trajectory for tracking based on human features is determined.

[0018] Optionally, the human body features include a human body detection box, and the head features include a head detection box and a tracking identifier. The step of matching the human body features of each frame in the short trajectory with the head features in the head tracking trajectory to obtain the head feature identifier corresponding to each frame of human body features includes:

[0019] For each frame of human body features in the short trajectory, predict the human head to obtain the human head prediction box corresponding to each frame of human body features.

[0020] The predicted head bounding box of each frame is matched with the head detection box in the corresponding frame. If the match is successful, the head feature identifier corresponding to the human body feature of each frame is determined according to the tracking identifier.

[0021] Optionally, the first feature sequence includes a first feature subsequence and a second feature subsequence, the second feature sequence includes a third feature subsequence and a fourth feature subsequence, and the step of determining the first feature sequence based on the short trajectory, and determining the second feature sequence corresponding to the short trajectory based on the head feature identifier corresponding to the human body features in each frame, includes:

[0022] A first feature subsequence is determined based on the human body features of the first k frames in the short trajectory and a preset first dynamic time coefficient, and a second feature subsequence is determined based on the human body features of the last k frames in the short trajectory and a preset second dynamic time coefficient, wherein the first dynamic time coefficient decreases with time and the second dynamic time coefficient increases with time.

[0023] Based on the head feature identifier corresponding to each frame of human body features in the short trajectory, a head feature identifier sequence is determined; based on the head feature identifier corresponding to the first k frames of human body features in the short trajectory, a third feature sub-sequence is determined; and based on the head feature identifier corresponding to the last k frames of human body features in the short trajectory, a fourth feature sub-sequence is determined.

[0024] Optionally, the short trajectories to be matched include a first short trajectory to be matched with an earlier time and a second short trajectory to be matched with a later time. The step of determining a first similarity between the short trajectories to be matched based on the first feature sequence, and determining a second similarity between the short trajectories to be matched based on the second feature, includes:

[0025] Based on the second feature subsequence of the first short trajectory to be matched and the first feature subsequence of the second short trajectory to be matched, a first similarity between the first short trajectory to be matched and the second short trajectory to be matched is determined.

[0026] Based on the fourth feature subsequence of the first short trajectory to be matched and the third feature subsequence of the second short trajectory to be matched, a second similarity between the first short trajectory to be matched and the second short trajectory to be matched is determined.

[0027] Optionally, determining the second similarity between the first short trajectory to be matched and the second short trajectory to be matched based on the fourth feature subsequence of the first short trajectory to be matched and the third feature subsequence of the second short trajectory to be matched includes:

[0028] The second similarity between the first short trajectory to be matched and the second short trajectory to be matched is determined based on the degree of repetition of head feature identifiers between the fourth feature subsequence of the first short trajectory to be matched and the third feature subsequence of the second short trajectory to be matched.

[0029] Optionally, determining the matching result between the short trajectories to be matched based on the first similarity and the second similarity includes:

[0030] If the second similarity is greater than the first threshold, then the matching result between the short trajectories to be matched is determined to be a successful match;

[0031] If the second similarity is less than or equal to the first threshold and greater than the second threshold, then the matching result between the short trajectories to be matched is determined based on the average similarity between the first similarity and the second similarity.

[0032] If the second similarity is less than or equal to the second threshold, then the matching result between the short trajectories to be matched is determined based on the second similarity.

[0033] Secondly, embodiments of the present invention provide a short trajectory matching device, comprising:

[0034] The first determining module is used to determine any two short trajectories as short trajectories to be matched, wherein the short trajectories include a first feature sequence based on human body features and a second feature sequence based on human head features;

[0035] The second determining module is used to determine a first similarity between the short trajectories to be matched based on the first feature sequence, and to determine a second similarity between the short trajectories to be matched based on the second feature;

[0036] The third determining module is used to determine the matching result between the short trajectories to be matched based on the first similarity and the second similarity.

[0037] Thirdly, embodiments of the present invention provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the short trajectory matching method as described above.

[0038] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the short trajectory matching method described above.

[0039] In this embodiment of the invention, any two short trajectories are determined as short trajectories to be matched. Each short trajectory includes a first feature sequence based on human body features and a second feature sequence based on human head features. A first similarity is determined between the short trajectories to be matched based on the first feature sequence, and a second similarity is determined based on the second feature sequence. A matching result between the short trajectories to be matched is determined based on the first similarity and the second similarity. The short trajectories to be matched are matched using the first similarity obtained from the first feature sequence based on human body features and the second similarity obtained from the second feature sequence based on human head features. The second feature sequence based on human head features is used as an auxiliary sequence to address the problem of low image tracking accuracy when the human body is occluded. Simultaneously, since some highly correlated human body features are pre-incorporated into the short trajectories, the computational scale of frame-by-frame image tracking is reduced, improving the processing speed of image tracking. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0041] Figure 1 This is a schematic diagram of the overall process of the short trajectory matching method provided in the embodiments of the present invention;

[0042] Figure 2 This is a schematic diagram illustrating the matching of a head detection box and a head prediction box according to an embodiment of the present invention;

[0043] Figure 3 This is a schematic diagram of a first similarity calculation provided in an embodiment of the present invention;

[0044] Figure 4 This is a schematic diagram of a second similarity calculation provided in an embodiment of the present invention;

[0045] Figure 5 This is a structural block diagram of the short trajectory matching device provided in an embodiment of the present invention;

[0046] Figure 6 This is a structural block diagram of an electronic device provided in an embodiment of the present invention.

[0047] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0049] The terms "first," "second," and "third," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects and not to describe a specific order. Furthermore, the term "comprising," and any variations thereof, is intended to cover a non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0050] First, let's take a look at the relevant accompanying drawings to illustrate the solution of the embodiments of this application.

[0051] like Figure 1 As shown, a specific embodiment of the present invention provides a short trajectory matching method, including:

[0052] 101. Determine any two short trajectories as the short trajectories to be matched.

[0053] In this embodiment of the invention, the aforementioned short trajectory to be matched includes two short trajectories, each comprising a first feature sequence based on human body features and a second feature sequence based on head features. These short trajectories can be obtained through preliminary image tracking of the video to be tracked. For a single target person, the video to be tracked may include one or more short trajectories. After preliminary image tracking processing of the video to be tracked, multiple short trajectories of the target person can be obtained. The aforementioned short trajectory matching refers to matching multiple short trajectories of the same target person, and then stitching the successfully matched short trajectories together according to time to obtain the complete trajectory of that target person in the video to be tracked.

[0054] The first feature sequence mentioned above can be understood as a feature sequence obtained based on human body features in a short trajectory, and the second feature sequence mentioned above can be understood as a feature sequence obtained based on human head features in a short trajectory. Furthermore, the second feature sequence mentioned above can be understood as a feature sequence obtained based on the matching results of human head features and human body features.

[0055] The aforementioned head features can be the features corresponding to the head tracking results obtained by image tracking of the heads in the video to be tracked during the initial step of image tracking. In this embodiment of the invention, a short trajectory includes a first feature sequence corresponding to human body features and a second feature sequence corresponding to head features. After obtaining the short trajectory of the video to be tracked, any two short trajectories can be read into memory as short trajectories to be matched for matching calculation.

[0056] 102. Determine the first similarity between the short trajectories to be matched based on the first feature sequence, and determine the second similarity between the short trajectories to be matched based on the second feature.

[0057] In this embodiment of the invention, a first feature sequence similarity can be calculated between the short trajectories to be matched, resulting in a first similarity representing the human body similarity between the short trajectories. A larger first similarity indicates more similar human body features in the short trajectories, while a smaller first similarity indicates less similar human body features. Simultaneously, a second feature sequence similarity is calculated between the short trajectories to be matched, resulting in a second similarity representing the head similarity between the short trajectories. A larger second similarity indicates more similar head features in the short trajectories, while a smaller second similarity indicates less similar head features.

[0058] 103. Based on the first similarity and the second similarity, determine the matching results between the short trajectories to be matched.

[0059] In this embodiment of the invention, a first similarity score represents the human body similarity between the short trajectories to be matched, and a second similarity score represents the head similarity between the short trajectories to be matched. When the human body similarity and head similarity score meet preset conditions, the matching result between the short trajectories to be matched can be determined as a successful match, indicating that the short trajectories to be matched belong to the same target person. When the human body similarity and head similarity score do not meet preset conditions, the matching result between the short trajectories to be matched can be determined as a failed match, indicating that the short trajectories to be matched do not belong to the same target person.

[0060] After obtaining the matching results, if the match is successful, the two corresponding short trajectories are added to the short trajectory set of the same target person. When all short trajectories have been matched, the short trajectory set of each target person can be obtained. The short trajectories in the short trajectory set can be spliced ​​together in chronological order to obtain the complete trajectory of the target person.

[0061] In this embodiment of the invention, any two short trajectories are determined as short trajectories to be matched. Each short trajectory includes a first feature sequence based on human body features and a second feature sequence based on human head features. A first similarity is determined between the short trajectories to be matched based on the first feature sequence, and a second similarity is determined based on the second feature sequence. A matching result between the short trajectories to be matched is determined based on the first similarity and the second similarity. The short trajectories to be matched are matched using the first similarity obtained from the first feature sequence based on human body features and the second similarity obtained from the second feature sequence based on human head features. The second feature sequence based on human head features is used as an auxiliary sequence to address the problem of low image tracking accuracy when the human body is occluded. Simultaneously, since some highly correlated human body features are pre-incorporated into the short trajectories, the computational scale of frame-by-frame image tracking is reduced, improving the processing speed of image tracking.

[0062] Optionally, before determining any two short trajectories as short trajectories to be matched, image tracking processing can be performed on the video to be tracked to obtain short trajectories based on human features and head tracking trajectories based on head features; the human features of each frame in the short trajectory are matched with the head features in the head tracking trajectory to obtain the head feature identifier corresponding to each frame of human features; a first feature sequence is determined based on the short trajectory, and a second feature sequence corresponding to the short trajectory is determined based on the head feature identifier corresponding to each frame of human features.

[0063] In this invention, human features and head features can be extracted from each frame of the video to be tracked. The similarity of human features between adjacent frames is calculated. Two human features with a similarity greater than a threshold are associated with the same human tracking identifier until no related human features appear in adjacent frames, resulting in a short tracking trajectory based on human features. The length of the short trajectory is related to the number of frames in which the related human features appear; specifically, the length of the short trajectory is the same as the number of frames in which the related human features appear. One short trajectory corresponds to only one target person, but one target person can have multiple short trajectories. The human features in the short trajectory are sorted frame by frame to obtain the human feature sequence of the short trajectory, which can be determined as the first feature sequence of the short trajectory.

[0064] The similarity of head features in two frames is calculated. Two head features with a similarity greater than a threshold are associated with the same head tracking identifier until there are no associated head features in two adjacent frames, thus obtaining a head tracking trajectory based on head features.

[0065] For a single frame of human features in a short trajectory, this frame of human features can be matched with all corresponding head features in that frame. If a head feature successfully matches the human features in that frame, the head tracking identifier can be used as the head feature identifier for the corresponding frame of human features. If no head feature successfully matches the human features in that frame, a preset identifier can be used as the head feature identifier for the corresponding frame of human features. The preset identifier can be -1 or 0. Each frame of human features in the short trajectory is associated with a corresponding head feature identifier, resulting in a corresponding head feature identifier sequence. This head feature identifier sequence can be determined as the second feature sequence corresponding to the short trajectory.

[0066] Optionally, in the step of performing image tracking processing on the video to be tracked to obtain a short trajectory based on human features, human features can be extracted from each frame of the video to be tracked to obtain the human features corresponding to each frame; image tracking processing can be performed on the video to be tracked based on the human features to obtain the original tracking trajectory based on the human features; the trajectory node graph of the video to be tracked can be determined based on the original tracking trajectory, wherein each segment of the original tracking trajectory corresponds to a node in the trajectory node graph; the trajectory node graph can be segmented into nodes according to a preset segmentation rule to obtain a node subgraph; and the short trajectory based on human features can be determined based on the node subgraph.

[0067] In this embodiment of the invention, existing image tracking algorithms can be used to process the video to be tracked, obtaining the original tracking trajectory based on human features. Each segment of the original tracking trajectory corresponds to a target person. By segmenting the original tracking trajectory according to a preset segmentation strategy, shorter trajectories of the target person can be obtained.

[0068] Specifically, each segment of the original tracking trajectory is treated as a node. These nodes are sorted according to their corresponding start times to obtain a trajectory node graph. During the segmentation of the trajectory node graph, a maximum number of nodes, k, is set for each subgraph. The segmentation time is determined based on the start time of the original tracking trajectory, ensuring that the number of nodes in each subgraph does not exceed k. Then, the trajectory node graph is segmented into several subgraphs based on the determined segmentation points. If a node crosses a segmentation time point, it is split into two nodes at that point. The first segment is solved in the preceding subgraph, and the second segment in the following subgraph, while recording that these two nodes originated from the same node. After obtaining the subgraphs, the human feature similarity between nodes is calculated for each subgraph. This similarity is compared to a preset value. If the similarity is greater than or equal to the preset value, the two nodes are concatenated to obtain the short trajectory based on human features for tracking corresponding to the subgraph. The duration of the short trajectory is less than or equal to the duration of the subgraph.

[0069] It should be noted that for global tracking, the number of nodes in the graph increases with the video duration. However, for tracking solutions, since the solved single chains are non-overlapping, adding nodes later will not change the solution for any optimal single chain representing different people. Instead, it will introduce additional edges and constraints, complicating the problem and reducing computational speed. Therefore, the original tracking trajectory is divided into shorter trajectories, which pre-incorporate some highly correlated human features, reducing the computational scale of frame-by-frame image tracking and improving the processing speed of image tracking.

[0070] Optionally, the human body features include a human body detection box, and the head features include a head detection box and a tracking identifier. In the step of matching the human body features of each frame in the short trajectory with the head features in the head tracking trajectory to obtain the head feature identifier corresponding to each frame of human body features, the head can be predicted for each frame of human body features in the short trajectory to obtain the head prediction box corresponding to each frame of human body features; the head prediction box of each frame is matched with the head detection box in the corresponding frame. If the match is successful, the head feature identifier corresponding to each frame of human body features is determined according to the tracking identifier.

[0071] In this embodiment of the invention, the human body detection box can be extracted simultaneously during human body feature extraction, and the human body detection box can be represented by (x1, y1, w1, h1, u1). The head detection box can be extracted simultaneously during head feature extraction, and the head detection box can be represented by (x2, y2, w2, h2, u2).

[0072] Specifically, in a short trajectory, for a frame of human features, a human detection box can be used to predict the head in the corresponding human image, resulting in a predicted head box for that frame. Further, a trained head prediction model can be used to predict the image corresponding to the human detection box. This head prediction model can be based on a deep convolutional neural network. During training, the human and head boxes in the complete human image are used as labels. The head is masked in the complete human image to obtain an image with the head obscured. The images with and without obscured heads are used as positive and negative samples to construct a dataset. The head prediction model is iteratively trained using this dataset. When the model iterates a preset number of times or converges, a trained head prediction model is obtained. This model can output the predicted head box corresponding to the human image.

[0073] Each frame of human features corresponds to a predicted head bounding box. The distance and overlap between the predicted head bounding box and each head detection bounding box in that frame are calculated. The smaller the distance and the greater the overlap, the higher the probability of a successful match. Specifically, when the distance between a predicted head bounding box and a head detection bounding box is greater than or equal to a preset distance, and the overlap is greater than or equal to a preset overlap, it can be determined that the head detection bounding box matches the human feature corresponding to the predicted head bounding box. The head detection bounding box is then identified as the head feature of the corresponding human feature. Simultaneously, the head tracking identifier corresponding to the head detection bounding box is determined as the head feature identifier corresponding to the human features in that frame. The aforementioned overlap can be the Intersection over Union (IOU) between the predicted head bounding box and the head detection bounding box, and the aforementioned head tracking identifier can be the head tracking ID.

[0074] Please refer to Figure 2 , Figure 2 This is a schematic diagram illustrating the matching of a head detection box and a head prediction box according to an embodiment of the present invention, as shown below. Figure 2 As shown, the single-frame head feature is the predicted head bounding box corresponding to the human body detection box. The head tracking trajectory includes three head tracking trajectories based on head features, with head tracking identifiers of 1, 2, and 3. During the matching process, the predicted head bounding box corresponding to the human body detection box is matched with the predicted head bounding box corresponding to the human body detection box in the same frame of the three head tracking trajectories. If a match is successfully made with the head detection box in the head tracking trajectory with head tracking identifier 1, the head tracking identifier is used as the head feature identifier and associated with the human body feature of the corresponding frame.

[0075] Optionally, the first feature sequence includes a first feature subsequence and a second feature subsequence, and the second feature sequence includes a third feature subsequence and a fourth feature subsequence. In the steps of determining the first feature sequence based on the short trajectory and determining the second feature sequence corresponding to the short trajectory based on the head feature identifier corresponding to each frame of human features, the first feature subsequence can be determined based on the human features of the first k frames in the short trajectory and a preset first dynamic time coefficient, and the second feature subsequence can be determined based on the human features of the last k frames in the short trajectory and a preset second dynamic time coefficient, wherein the first dynamic time coefficient decreases with time and the second dynamic time coefficient increases with time; a head feature identifier sequence is determined based on the head feature identifier corresponding to each frame of human features in the short trajectory, the third feature subsequence is determined based on the head feature identifier corresponding to the human features of the first k frames in the short trajectory, and the fourth feature subsequence is determined based on the head feature identifier corresponding to the human features of the last k frames in the short trajectory.

[0076] In this embodiment of the invention, a short trajectory includes four sequences: a first feature subsequence, a second feature subsequence, a third feature subsequence, and a fourth feature subsequence. The first and second feature subsequences are used to represent human body feature sequences, and the third and fourth feature subsequences are used to represent human head feature identification sequences.

[0077] The short trajectory has n frames, and the human features of the first k frames can be represented as A1 = a1, a2, ..., a k-1 a k In a short trajectory, the human body features in the last k frames can be represented as A2 = a n-k+1 a n-k+2 , ..., a n-1 a n If k is less than or equal to n, the first dynamic time coefficient can be understood as a weight sequence that decreases over time, and the second dynamic time coefficient can be understood as a weight sequence that increases over time. The first dynamic time coefficient can be represented by R1 = r1, r2, ..., r k-1 r k The second dynamic time coefficient can be represented as R² = r k r k-1 Let r1, ..., r2, r1 represent the coefficients r1, ..., r2. For any two adjacent coefficients r1 and r2, ... i-1 and r i Both have r i-1 Less than r i In one specific embodiment, the above-mentioned incremental relationship can be represented by the following formula:

[0078]

[0079] Specifically, the first feature subsequence B1 = r k ·a1,r k-1 ·a2,…,r2·a k-1 ,r1·a k The second characteristic subsequence B2 = r1·a n-k+1 ,r2·a n-k+2 , ..., r k-1 ·a n-1 r k ·a n .

[0080] Based on the human head feature identifiers corresponding to the human features in each frame of the short trajectory, determine the human head feature identifier sequence C0 = c1, c2, ..., c n-1 c n The third feature subsequence C1 = c1, c2, ..., c is determined based on the head feature identifiers corresponding to the human body features in the first k frames of the short trajectory. k-1 c k And based on the human head feature identifiers corresponding to the human body features in the last k frames of the short trajectory, the fourth feature subsequence C2 = c is determined. n-k+1 c n-k+2 c n-1 c n .

[0081] Optionally, the short trajectories to be matched include a first short trajectory to be matched that occurs earlier in time and a second short trajectory to be matched that occurs later in time. In the steps of determining the first similarity between the short trajectories to be matched based on the first feature sequence and determining the second similarity between the short trajectories to be matched based on the second feature subsequence of the first short trajectory to be matched and the first feature subsequence of the second short trajectory to be matched, the first similarity between the first short trajectory to be matched and the second short trajectory to be matched can be determined based on the fourth feature subsequence of the first short trajectory to be matched and the third feature subsequence of the second short trajectory to be matched.

[0082] In this embodiment of the invention, the first feature subsequence can be understood as the pre-feature sequence of the short trajectory, and the second feature subsequence can be understood as the post-feature sequence of the short trajectory. After the first feature subsequence is weighted by the first dynamic time coefficient, the first feature subsequence focuses more on the pre-feature features of the short trajectory. After the second feature subsequence is weighted by the second dynamic time coefficient, the second feature subsequence focuses more on the post-feature features of the short trajectory.

[0083] It should be noted that since the splicing of two short trajectories involves joining the rear end of the previous short trajectory with the front end of the next short trajectory, when matching two short trajectories, more attention can be paid to the rear features of the previous short trajectory and the front features of the next short trajectory, thereby improving the matching accuracy of short trajectories.

[0084] Specifically, the short trajectories to be matched include the first short trajectory with a earlier time and the second short trajectory with a later time. The cosine similarity between the second feature subsequence of the first short trajectory and the first feature subsequence of the second short trajectory can be calculated. That is, the cosine similarity is calculated between the later features of the first short trajectory and the earlier features of the second short trajectory, and the resulting cosine similarity is used as the first similarity. This first similarity is the similarity of human features. For example, please refer to... Figure 3 , Figure 3 This is a schematic diagram of a first similarity calculation provided by an embodiment of the present invention, as shown below. Figure 3 As shown, short trajectory i occurs before short trajectory j. The human features of short trajectory i are enhanced at the rear using a second dynamic time coefficient, resulting in the second feature subsequence of short trajectory i. Conversely, the human features of short trajectory j are enhanced at the front using a first dynamic time coefficient, resulting in the first feature subsequence of short trajectory j. The first similarity between short trajectory i and short trajectory j is calculated using the second feature subsequence of short trajectory i and the first feature subsequence of short trajectory j. Specifically, the formula for calculating the first similarity is as follows:

[0085] appearance_sim(i,j)=cos(appearance_feat_i,appearance_feat_j)

[0086] Where appearance_sim(i,j) represents the first similarity between short trajectory i and short trajectory j, appearance_feat_i represents the second feature subsequence of short trajectory i, and appearance_feat_j represents the first feature subsequence of short trajectory j. head_sim(i,j) represents the second similarity, max(head_feat_i^head_feat_j) represents the number of target head features, and len(head_feat_i+head_feat_j) represents the total number of head features between the first and second head trajectory vectors.

[0087] The aforementioned second similarity can be the cosine similarity or Euclidean distance similarity between the fourth feature subsequence of the first short trajectory to be matched and the second feature subsequence of the second short trajectory to be matched. For details, please refer to... Figure 4 , Figure 4 This is a schematic diagram of a second similarity calculation provided by an embodiment of the present invention, such as... Figure 4 As shown, short trajectory i occurs before short trajectory j. The fourth feature subsequence of short trajectory i is ..., 1, 2, 2, 3, 3, 3, and the third feature subsequence of short trajectory j is 3, 3, 3, 2, 2, 1, ... The second similarity between short trajectory i and short trajectory j is calculated using the fourth feature subsequence of short trajectory i and the third feature subsequence of short trajectory j.

[0088] Optionally, in the step of determining the second similarity between the first short trajectory to be matched and the second short trajectory to be matched based on the fourth feature subsequence of the first short trajectory to be matched and the third feature subsequence of the second short trajectory to be matched, the second similarity between the first short trajectory to be matched and the second short trajectory to be matched can be determined according to the repetition of head feature identifiers between the fourth feature subsequence of the first short trajectory to be matched and the third feature subsequence of the second short trajectory to be matched.

[0089] In this embodiment of the invention, the second similarity is the degree of repetition between the fourth feature subsequence of the first short trajectory to be matched and the second feature subsequence of the second short trajectory to be matched. The repetition can be understood as the number of the most repeated human head feature identifiers in the two short trajectories divided by the total number of human head feature identifiers in the two short trajectories. Assuming that the number of the most repeated human head feature identifiers is e and the total number of human head feature identifiers is n, then the repetition is e / n.

[0090] Specifically, the formula for calculating the second similarity is shown below:

[0091]

[0092] Where head_sim(i,j) represents the second similarity between short trajectory i and short trajectory j, max(head_feat_i∧head_feat_j) represents the number of identical head feature identifiers in short trajectory i and short trajectory j, and len(head_feat_i+head_feat_j) represents the total number of head feature identifiers in short trajectory i and short trajectory j.

[0093] Optionally, in the step of determining the matching result between the short trajectories to be matched based on the first similarity and the second similarity, if the second similarity is greater than the first threshold, then the matching result between the short trajectories to be matched is determined to be a successful match; if the second similarity is less than or equal to the first threshold and greater than the second threshold, then the matching result between the short trajectories to be matched is determined based on the average similarity between the first similarity and the second similarity; if the second similarity is less than or equal to the second threshold, then the matching result between the short trajectories to be matched is determined based on the second similarity.

[0094] In this embodiment of the invention, the trajectory similarity between the short trajectories to be matched is determined by the first similarity and the second similarity, and the matching result between the short trajectories to be matched is determined based on the trajectory similarity between the short trajectories to be matched.

[0095] Specifically, the trajectory similarity can be expressed as follows:

[0096]

[0097] Here, `tracklet_sim` represents the trajectory similarity, `p` represents the first threshold, and `q` represents the second threshold, where the first threshold is greater than the second threshold. The second similarity is determined based on the repetition of head feature identifiers. If the second similarity is greater than the first threshold, it indicates that the head is less obscured in the human features and represents the same target person's head features; in this case, the trajectory similarity can be set to 1, indicating a successful match between the short trajectories to be matched. If the second similarity is less than or equal to the first threshold but greater than the second threshold, it indicates that the head is more obscured in the human features. In this case, the average similarity between the first and second similarities can be calculated, and the trajectory similarity can be set as this average similarity. When the average similarity is greater than or equal to the preset trajectory similarity, the matching result between the short trajectories to be matched can be determined as a successful match; when the average similarity is less than or equal to the preset trajectory similarity, the matching result between the short trajectories to be matched can be determined as a failed match. If the second similarity is less than the second threshold, it indicates that the head is more obscured in the human features; in this case, the second similarity can be set as the trajectory similarity.

[0098] like Figure 5 As shown, an embodiment of the present invention provides a short trajectory matching device, comprising:

[0099] The first determining module 501 is used to determine any two short trajectories as short trajectories to be matched, wherein the short trajectories include a first feature sequence based on human body features and a second feature sequence based on human head features.

[0100] The second determining module 502 is used to determine a first similarity between the short trajectories to be matched based on the first feature sequence, and to determine a second similarity between the short trajectories to be matched based on the second feature;

[0101] The third determining module 503 is used to determine the matching result between the short trajectories to be matched based on the first similarity and the second similarity.

[0102] Optionally, the device further includes:

[0103] The processing module is used to perform image tracking processing on the video to be tracked, and to obtain short trajectories based on human body features and head tracking trajectories based on human head features.

[0104] The matching module is used to match the human body features of each frame in the short trajectory with the human head features in the human head tracking trajectory to obtain the human head feature identifier corresponding to each frame of human body features.

[0105] The fourth determining module is used to determine a first feature sequence based on the short trajectory, and to determine a second feature sequence corresponding to the short trajectory based on the human head feature identifier corresponding to each frame of human features.

[0106] Optionally, the processing module includes:

[0107] The extraction submodule is used to extract human features from each frame of the video to be tracked, and obtain the human features corresponding to each frame.

[0108] The processing submodule is used to perform image tracking processing on the video to be tracked based on the human body characteristics to obtain the original tracking trajectory based on the human body characteristics;

[0109] The first determining submodule is used to determine the trajectory node map of the video to be tracked based on the original tracking trajectory, wherein each segment of the original tracking trajectory in the trajectory node map corresponds to a node;

[0110] The segmentation module segments the trajectory node graph according to a preset segmentation rule to obtain a node subgraph;

[0111] The second determining submodule is used to determine a short trajectory for tracking based on human features according to the node subgraph.

[0112] Optionally, the matching module includes:

[0113] The prediction submodule is used to predict the human head for each frame of human features in the short trajectory, and obtain the human head prediction box corresponding to each frame of human features.

[0114] The matching submodule is used to match the predicted head bounding box of each frame with the head detection bounding box in the corresponding frame. If the match is successful, the head feature identifier corresponding to the human body feature of each frame is determined according to the tracking identifier.

[0115] Optionally, the fourth determining module includes:

[0116] The third determining submodule is used to determine a first feature subsequence based on the human body features of the first k frames in the short trajectory and a preset first dynamic time coefficient, and to determine a second feature subsequence based on the human body features of the last k frames in the short trajectory and a preset second dynamic time coefficient, wherein the first dynamic time coefficient decreases with time and the second dynamic time coefficient increases with time.

[0117] The fourth determining submodule is used to determine a head feature identifier sequence based on the head feature identifier corresponding to the human body features in each frame of the short trajectory, to determine a third feature subsequence based on the head feature identifier corresponding to the human body features in the first k frames of the short trajectory, and to determine a fourth feature subsequence based on the head feature identifier corresponding to the human body features in the last k frames of the short trajectory.

[0118] Optionally, the second determining module 502 includes:

[0119] The fifth determining submodule is used to determine the first similarity between the first short trajectory to be matched and the second short trajectory to be matched based on the second feature subsequence of the first short trajectory to be matched and the first feature subsequence of the second short trajectory to be matched.

[0120] The sixth determining submodule is used to determine the second similarity between the first short trajectory to be matched and the second short trajectory to be matched based on the fourth feature subsequence of the first short trajectory to be matched and the third feature subsequence of the second short trajectory to be matched.

[0121] Optionally, the sixth determining submodule includes:

[0122] The determining unit is configured to determine a second similarity between the first short trajectory to be matched and the second short trajectory to be matched based on the degree of repetition of head feature identifiers between the fourth feature subsequence of the first short trajectory to be matched and the third feature subsequence of the second short trajectory to be matched.

[0123] Optionally, the third determining module 503 includes:

[0124] The seventh determination submodule is used to determine that the matching result between the short trajectories to be matched is a successful match if the second similarity is greater than the first threshold.

[0125] The eighth determining submodule is used to determine the matching result between the short trajectories to be matched based on the average similarity between the first similarity and the second similarity if the second similarity is less than or equal to the first threshold and greater than the second threshold;

[0126] The ninth determining submodule is used to determine the matching result between the short trajectories to be matched based on the second similarity if the second similarity is less than or equal to the second threshold.

[0127] It should be noted that the short trajectory matching device provided in this embodiment of the invention can be applied to devices such as smart cameras, smartphones, computers, and servers that can perform short trajectory matching methods.

[0128] The short trajectory matching device provided in this embodiment of the invention can realize all the processes implemented by the gear injection molding control method in the above-described method embodiments, and can achieve the same beneficial effects. To avoid repetition, it will not be described again here.

[0129] like Figure 6 As shown, a specific embodiment of the present invention also provides an electronic device, including a memory 602, a processor 601, and a computer program stored in the memory 602 and executable on the processor 601. When the processor 601 executes the computer program, it implements the steps of the short trajectory matching method described above.

[0130] Specifically, processor 601 calls the computer program stored in memory 602 and performs the following steps:

[0131] Determine any two short trajectories as short trajectories to be matched, wherein the short trajectories include a first feature sequence based on human body features and a second feature sequence based on human head features;

[0132] A first similarity between the short trajectories to be matched is determined based on the first feature sequence, and a second similarity between the short trajectories to be matched is determined based on the second feature;

[0133] The matching result between the short trajectories to be matched is determined based on the first similarity and the second similarity.

[0134] Optionally, before determining any two short trajectories as short trajectories to be matched, the method executed by the processor 601 further includes:

[0135] Image tracking processing is performed on the video to be tracked to obtain short trajectories based on human body features and head tracking trajectories based on human head features;

[0136] The human features in each frame of the short trajectory are matched with the head features in the head tracking trajectory to obtain the head feature identifier corresponding to each frame of human features.

[0137] A first feature sequence is determined based on the short trajectory, and a second feature sequence corresponding to the short trajectory is determined based on the human head feature identifier corresponding to the human body feature in each frame.

[0138] Optionally, the image tracking processing of the video to be tracked performed by the processor 601 to obtain a short trajectory based on human features includes:

[0139] Human features are extracted from each frame of the video to be tracked to obtain the human features corresponding to each frame.

[0140] The video to be tracked is processed by image tracking based on the human body characteristics to obtain the original tracking trajectory based on human body characteristics;

[0141] Based on the original tracking trajectory, a trajectory node graph of the video to be tracked is determined, wherein each segment of the original tracking trajectory in the trajectory node graph corresponds to a node;

[0142] The trajectory node graph is segmented according to a preset segmentation rule to obtain a node subgraph;

[0143] Based on the node subgraph, a short trajectory for tracking based on human features is determined.

[0144] Optionally, the human body feature includes a human body detection box, and the head feature includes a head detection box and a tracking identifier. The step of matching the human body feature of each frame in the short trajectory with the head feature in the head tracking trajectory to obtain the head feature identifier corresponding to each frame of human body feature includes:

[0145] For each frame of human body features in the short trajectory, predict the human head to obtain the human head prediction box corresponding to each frame of human body features.

[0146] The predicted head bounding box of each frame is matched with the head detection box in the corresponding frame. If the match is successful, the head feature identifier corresponding to the human body feature of each frame is determined according to the tracking identifier.

[0147] Optionally, the first feature sequence includes a first feature subsequence and a second feature subsequence, the second feature sequence includes a third feature subsequence and a fourth feature subsequence, and the processor 601 executes the process of determining the first feature sequence based on the short trajectory and determining the second feature sequence corresponding to the short trajectory based on the human head feature identifier corresponding to each frame of human features, including:

[0148] A first feature subsequence is determined based on the human body features of the first k frames in the short trajectory and a preset first dynamic time coefficient, and a second feature subsequence is determined based on the human body features of the last k frames in the short trajectory and a preset second dynamic time coefficient, wherein the first dynamic time coefficient decreases with time and the second dynamic time coefficient increases with time.

[0149] Based on the head feature identifier corresponding to each frame of human body features in the short trajectory, a head feature identifier sequence is determined; based on the head feature identifier corresponding to the first k frames of human body features in the short trajectory, a third feature sub-sequence is determined; and based on the head feature identifier corresponding to the last k frames of human body features in the short trajectory, a fourth feature sub-sequence is determined.

[0150] Optionally, the short trajectories to be matched include a first short trajectory to be matched with a earlier time and a second short trajectory to be matched with a later time. The steps executed by the processor 601 to determine the first similarity between the short trajectories to be matched based on the first feature sequence and to determine the second similarity between the short trajectories to be matched based on the second feature include:

[0151] Based on the second feature subsequence of the first short trajectory to be matched and the first feature subsequence of the second short trajectory to be matched, a first similarity between the first short trajectory to be matched and the second short trajectory to be matched is determined.

[0152] Based on the fourth feature subsequence of the first short trajectory to be matched and the third feature subsequence of the second short trajectory to be matched, a second similarity between the first short trajectory to be matched and the second short trajectory to be matched is determined.

[0153] Optionally, the processor 601 executes the determination of a second similarity between the first short trajectory to be matched and the second short trajectory to be matched based on the fourth feature subsequence of the first short trajectory to be matched and the third feature subsequence of the second short trajectory to be matched, including:

[0154] The second similarity between the first short trajectory to be matched and the second short trajectory to be matched is determined based on the degree of repetition of head feature identifiers between the fourth feature subsequence of the first short trajectory to be matched and the third feature subsequence of the second short trajectory to be matched.

[0155] Optionally, the processor 601 performs the process of determining the matching result between the short trajectories to be matched based on the first similarity and the second similarity, including:

[0156] If the second similarity is greater than the first threshold, then the matching result between the short trajectories to be matched is determined to be a successful match;

[0157] If the second similarity is less than or equal to the first threshold and greater than the second threshold, then the matching result between the short trajectories to be matched is determined based on the average similarity between the first similarity and the second similarity.

[0158] If the second similarity is less than or equal to the second threshold, then the matching result between the short trajectories to be matched is determined based on the second similarity.

[0159] That is, in a specific embodiment of the present invention, when the processor 601 of the electronic device executes the computer program, it implements the steps of the above-mentioned short trajectory matching method. As a result, the accuracy of tracking in densely populated scenes can be improved, the correlation is stronger, and the short trajectory matching effect is better during the short trajectory matching process.

[0160] It should be noted that since the processor 601 of the electronic device implements the steps of the above-described short trajectory matching method when executing the computer program, all embodiments of the above-described short trajectory matching method are applicable to the electronic device and can achieve the same or similar beneficial effects.

[0161] The computer-readable storage medium provided in this embodiment of the invention stores a computer program. When the computer program is executed by a processor, it implements the various processes of the short trajectory matching method or the application-side short trajectory matching method provided in this embodiment of the invention, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0162] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0163] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0164] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. All equivalent structural transformations made under the concept of the present invention using the contents of the present invention specification and drawings, or direct / indirect applications in other related technical fields, are included within the patent protection scope of the present invention.

Claims

1. A short track matching method, characterized by, include: Image tracking processing is performed on the video to be tracked to obtain short trajectories based on human body features and head tracking trajectories based on human head features; The human features of each frame in the short trajectory are matched with the head features in the head tracking trajectory to obtain the head feature identifier corresponding to each frame of human features; a first feature subsequence is determined based on the human features of the first k frames in the short trajectory and a preset first dynamic time coefficient, and a second feature subsequence is determined based on the human features of the last k frames in the short trajectory and a preset second dynamic time coefficient, wherein the first dynamic time coefficient decreases with time and the second dynamic time coefficient increases with time; a head feature identifier sequence is determined based on the head feature identifier corresponding to each frame of human features in the short trajectory; a third feature subsequence is determined based on the head feature identifier corresponding to the human features of the first k frames in the short trajectory, and a fourth feature subsequence is determined based on the head feature identifier corresponding to the human features of the last k frames in the short trajectory; any two short trajectories are determined as short trajectories to be matched, wherein the short trajectory includes a first feature sequence based on human features and a second feature sequence based on head features, the first feature sequence includes a first feature subsequence and a second feature subsequence, and the second feature sequence includes a third feature subsequence and a fourth feature subsequence; A first similarity between the short trajectories to be matched is determined based on the first feature sequence, and a second similarity between the short trajectories to be matched is determined based on the second feature; The matching result between the short trajectories to be matched is determined based on the first similarity and the second similarity.

2. The short trajectory matching method according to claim 1, characterized in that, The image tracking processing of the video to be tracked to obtain a short trajectory based on human features includes: Human features are extracted from each frame of the video to be tracked to obtain the human features corresponding to each frame. The video to be tracked is processed by image tracking based on the human body characteristics to obtain the original tracking trajectory based on human body characteristics; Based on the original tracking trajectory, a trajectory node graph of the video to be tracked is determined, wherein each segment of the original tracking trajectory in the trajectory node graph corresponds to a node; The trajectory node graph is segmented according to a preset segmentation rule to obtain a node subgraph; Based on the node subgraph, a short trajectory for tracking based on human features is determined.

3. The short trajectory matching method according to claim 2, characterized in that, The human body features include a human body detection box, and the head features include a head detection box and a tracking marker. The step of matching the human body features of each frame in the short trajectory with the head features in the head tracking trajectory to obtain the head feature marker corresponding to each frame of human body features includes: For each frame of human body features in the short trajectory, predict the human head to obtain the human head prediction box corresponding to each frame of human body features. The predicted head bounding box of each frame is matched with the head detection box in the corresponding frame. If the match is successful, the head feature identifier corresponding to the human body feature of each frame is determined according to the tracking identifier.

4. The short trajectory matching method according to claim 1, characterized by, The short trajectories to be matched include a first short trajectories with an earlier time and a second short trajectories with a later time. The step of determining a first similarity between the short trajectories based on the first feature sequence and a second similarity between the short trajectories based on the second feature sequence includes: Based on the second feature subsequence of the first short trajectory to be matched and the first feature subsequence of the second short trajectory to be matched, a first similarity between the first short trajectory to be matched and the second short trajectory to be matched is determined. Based on the fourth feature subsequence of the first short trajectory to be matched and the third feature subsequence of the second short trajectory to be matched, a second similarity between the first short trajectory to be matched and the second short trajectory to be matched is determined.

5. The short trajectory matching method according to claim 4, characterized in that, The determination of the second similarity between the first short trajectory to be matched and the second short trajectory to be matched based on the fourth feature subsequence of the first short trajectory to be matched and the third feature subsequence of the second short trajectory to be matched includes: The second similarity between the first short trajectory to be matched and the second short trajectory to be matched is determined based on the degree of repetition of head feature identifiers between the fourth feature subsequence of the first short trajectory to be matched and the third feature subsequence of the second short trajectory to be matched.

6. The short trajectory matching method according to any one of claims 1 to 5, characterized in that, Based on the first similarity and the second similarity, the matching result between the short trajectories to be matched is determined, including: If the second similarity is greater than the first threshold, then the matching result between the short trajectories to be matched is determined to be a successful match; If the second similarity is less than or equal to the first threshold and greater than the second threshold, then the matching result between the short trajectories to be matched is determined based on the average similarity between the first similarity and the second similarity. If the second similarity is less than or equal to the second threshold, then the matching result between the short trajectories to be matched is determined based on the second similarity.

7. A short track matching apparatus characterized by comprising: include: The first determining module is used to perform image tracking processing on the video to be tracked, obtaining a short trajectory based on human features and a head tracking trajectory based on head features; matching the human features of each frame in the short trajectory with the head features in the head tracking trajectory to obtain a head feature identifier corresponding to each frame of human features; determining a first feature subsequence based on the human features of the first k frames in the short trajectory and a preset first dynamic time coefficient, and determining a second feature subsequence based on the human features of the last k frames in the short trajectory and a preset second dynamic time coefficient, wherein the first dynamic time coefficient decreases with time, and the second dynamic time coefficient is rooted in the first k frames. According to the time increment; based on the head feature identifier corresponding to the human body features in each frame of the short trajectory, a head feature identifier sequence is determined; based on the head feature identifier corresponding to the human body features in the first k frames of the short trajectory, a third feature sub-sequence is determined; and based on the head feature identifier corresponding to the human body features in the last k frames of the short trajectory, a fourth feature sub-sequence is determined; any two short trajectories are determined as short trajectories to be matched, wherein the short trajectory includes a first feature sequence based on human body features and a second feature sequence based on head features, the first feature sequence includes a first feature sub-sequence and a second feature sub-sequence, and the second feature sequence includes a third feature sub-sequence and a fourth feature sub-sequence; The second determining module is used to determine a first similarity between the short trajectories to be matched based on the first feature sequence, and to determine a second similarity between the short trajectories to be matched based on the second feature; The third determining module is used to determine the matching result between the short trajectories to be matched based on the first similarity and the second similarity.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the short trajectory matching method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. When the computer program is executed by a processor, it implements the steps of the short trajectory matching method as described in any one of claims 1 to 6.

Images