Tree pruning based high-speed moving target tracking method and device

Abstract

The application provides a high-speed moving target tracking method and device based on tree pruning, the method comprising: updating a tracking tree corresponding to a video stream by adding a layer and inserting a node in the newly-built layer according to a target detection result of a current frame of the video stream, the video stream containing at least one high-speed moving object, the tracking tree being used to represent a target tracking trajectory; pruning the tracking tree based on a pruning algorithm corresponding to a high-speed motion feature of the object, and performing structure reduction maintenance processing on the tracking tree when the pruned tracking tree meets a preset layer number control rule. The application can track one or multiple targets at the same time, the tracking process is not prone to interruption, the robustness and anti-interference capability of the high-speed moving target tracking process can be improved, the high-speed moving target tracking process has moderate time and space complexity, and thus the accuracy, efficiency and convenience of obtaining the target tracking trajectory can be effectively improved, and the error rate can be reduced.

Description

Technical Field

[0001] This application relates to the field of target tracking technology, and in particular to a method and apparatus for tracking high-speed moving targets based on tree pruning. Background Technology

[0002] In computer vision, object detection is a relatively mature technology. Based on object detection results, accurately locating the target position in each frame of a continuous video stream can be understood as object tracking. The core problem of object tracking can be summarized as follows: detecting and identifying the same target in consecutive frames of a video stream. It has wide applications in computer vision and artificial intelligence, and research on tracking high-speed moving targets is gradually becoming a key focus.

[0003] Currently, existing methods for tracking high-speed moving targets typically involve: using powerful target detection methods such as machine learning algorithms, supplemented by target position estimation to improve detection efficiency and accuracy, without any judgment step, directly selecting the target with the highest confidence within the estimated image range as the tracking result; or modeling and tracking the target, i.e. updating the target's template after detecting the target, etc.

[0004] However, regardless of which method is used, there are problems such as difficulty in multi-target tracking, easy interruption of the tracking process, and poor algorithm robustness. Summary of the Invention

[0005] In view of this, embodiments of this application provide a method and apparatus for tracking high-speed moving targets based on tree pruning, so as to eliminate or improve one or more defects existing in the prior art.

[0006] One aspect of this application provides a high-speed moving target tracking method based on tree pruning, comprising:

[0007] Based on the target detection results of the current frame of the video stream, the tracking tree corresponding to the video stream is updated by adding layers and inserting nodes in the newly created layers. The video stream contains at least one high-speed moving object, and the tracking tree is used to represent the target tracking trajectory. Each layer of the tracking tree corresponds one-to-one with the video frame, and each node in the tracking tree corresponds one-to-one with each trajectory through backtracking.

[0008] The tracking tree is pruned based on the high-speed motion characteristics of the object, and when the pruned tracking tree meets the preset layer control rules, the tracking tree is structurally reduced and maintained.

[0009] In some embodiments of the present application, the tracking tree comprises a plurality of longitudinally connected layers starting from a root node, and each layer is composed of at least one node, each node being connected to a parent node in the previous layer of the layer where the node is located;

[0010] The nodes comprise a first node and a second node, wherein the first node is used to represent the frame identifier corresponding to the layer where the first node is located and the detected target identifier, and the second node is used to realize the relationship backtracking between the parent node of the previous layer connected to the second node and the child node of the next layer connected to the second node.

[0011] In some embodiments of the present application, the pruning algorithm corresponding to the high-speed motion feature of the object is used to prune the tracking tree, comprising:

[0012] In the tracking tree, nodes not meeting the high-speed motion feature of the object are pruned to complete the preliminary pruning of the tracking tree.

[0013] In some embodiments of the present application, the pruning algorithm corresponding to the high-speed motion feature of the object is used to prune the tracking tree, comprising:

[0014] The nodes of the last layer in the tracking tree are completely backtracked to obtain a backtracking result containing the parameters of each target and the relationship data between the targets;

[0015] According to the high-speed motion feature of the object and the backtracking result, nodes belonging to non-trajectories and nodes meeting the similar suppression rule in the tracking tree are pruned to complete the backtracking pruning of the tracking tree.

[0016] In some embodiments of the present application, when the pruned tracking tree meets the preset layer number control rule, the structure of the tracking tree is reduced and maintained, comprising:

[0017] The current depth of the tracking tree and the depth of the longest trajectory displayed in the tracking tree are obtained respectively;

[0018] If the depth of the longest trajectory is less than the depth of the tracking tree, the root node of the tracking tree is transferred downward, and all nodes before the transferred root node are deleted to realize the structure reduction of the tracking tree.

[0019] In some embodiments of the present application, before the tracking tree corresponding to the video stream is updated by adding layers and inserting nodes in the newly added layers according to the target detection result of the current frame of the video stream, the method further comprises:

[0020] A video stream containing at least one high-speed moving object is received;

[0021] A tracking tree corresponding to the video stream is established.

[0022] During the input of the video stream, each frame of the video stream is sequentially detected based on a preset target detection mode, and after obtaining the target detection result of each frame, the target detection result is taken as the target detection result of the current frame to start the process of updating the tracking tree.

[0023] In some embodiments of the present application, further comprising:

[0024] If the current frame of the video stream is the last frame of the video stream, target tracking trajectory data corresponding to the video stream is generated according to the structure of the current tracking tree.

[0025] Another aspect of the present application provides a high-speed moving target tracking device based on tree pruning, comprising:

[0026] A new target adding module is configured to update the tracking tree corresponding to the video stream by adding layers and inserting nodes in the newly added layers according to the target detection result of the current frame of the video stream, wherein the video stream contains at least one high-speed moving object, the tracking tree is used to represent the target tracking trajectory, each layer of the tracking tree corresponds to a video frame, and each node in the tracking tree corresponds to each trajectory through backtracking;

[0027] A pruning and layer number control module is configured to prune the tracking tree based on a pruning algorithm corresponding to the high-speed motion characteristics of the object, and perform structural reduction maintenance processing on the tracking tree when the pruned tracking tree meets a preset layer number control rule.

[0028] Another aspect of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the high-speed moving target tracking method based on tree pruning.

[0029] Another aspect of the present application provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the high-speed moving target tracking method based on tree pruning.

[0030] The application provides a high-speed moving target tracking method based on tree pruning, which comprises the following steps: according to a target detection result of a current frame of a video stream, a tracking tree corresponding to the video stream is updated by adding a layer and inserting a node into the new layer, wherein the video stream comprises at least one high-speed moving object, the tracking tree is used for representing a target tracking trajectory, each layer of the tracking tree corresponds to a video frame one by one, and each node in the tracking tree corresponds to a trajectory one by one through backtracking; a pruning algorithm corresponding to a high-speed motion feature of the object is used to prune the tracking tree, and when the tracking tree after pruning meets a preset layer number control rule, a structure reduction maintenance processing is performed on the tracking tree; the application uses a tree structure to describe and distinguish possible target trajectories, analyzes the possible trajectories by traversing the nodes after pruning, can realize tracking of one or multiple targets, and due to the hierarchical connection and backtracking characteristics of the tree structure, the tracking process is not easy to be interrupted due to temporary loss of the target, can focus on information in a longer time domain range based on the tracking tree, make an accurate judgment, and reduce the false detection rate; the pruning based on the pruning algorithm corresponding to the high-speed motion feature of the object can effectively improve the robustness and anti-interference ability of the high-speed moving target tracking process; the layer number control of the tracking tree makes the high-speed moving target tracking process have moderate time and space complexity, can effectively release the memory space, reduce the depth of the tracking tree, and further improve the accuracy, efficiency and convenience of obtaining the target tracking trajectory.

[0031] Additional advantages, objects, and features of the application will be set forth in part by the description that follows, and will become apparent to those skilled in the art upon examination of the following detailed description and drawings in which

[0032] Those skilled in the art will appreciate that the objects and advantages of the application can be implemented in a manner different from that specifically set forth herein without departing from the spirit and scope of the application. The application resides in the claims hereinafter appended. BRIEF DESCRIPTION OF DRAWINGS

[0033] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description, serve to explain the principles of the application. The components in the drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the application. For purposes of clarity and understanding, it is also to be understood that certain portions of the drawings can be exaggerated and others omitted in order to more clearly disclose the application.

[0034] Figure 1 A tracking data schematic diagram in an existing high-speed moving object tracking method.

[0035] Figure 2 An example diagram of a three-frame corresponding unpruned tree structure.

[0036] Figure 3 An example diagram of a total flow of a tree pruning based high-speed moving target tracking method in an embodiment of the present application.

[0037] Figure 4 An example diagram of a specific flow of a tree pruning based high-speed moving target tracking method in an embodiment of the present application.

[0038] Figure 5 An example diagram of a structure of a tree pruning based high-speed moving target tracking device in another embodiment of the present application.

[0039] Figure 6 An example diagram of a total flow of a tree pruning based high-speed moving object tracking algorithm provided in an application example of the present application.

[0040] Figure 7 An example diagram of a pruning specific operation provided in an application example of the present application.

[0041] Figure 8 An example diagram of a tree structure before reducing a tree scale provided in an application example of the present application. DETAILED DESCRIPTION

[0042] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be described in further detail below with reference to the embodiments and drawings. Herein, the illustrative embodiments of the present application and their descriptions are used to explain the present application, but are not intended to limit the present application.

[0043] It should be noted that, in order to avoid the present application being obscured by unnecessary details, only the structures and / or processing steps closely related to the solutions according to the present application are shown in the drawings, and other details not closely related to the present application are omitted.

[0044] It should be emphasized that the term "comprises / comprising" as used herein is intended to mean that features, elements, steps or components are present, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0045] It should be noted that, if not specifically stated, the term "connected" as used herein can not only mean direct connection, but also indirect connection with an intermediate object.

[0046] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0047] In one or more embodiments of the present application, the high-speed moving target can specifically refer to objects moving at high speed, such as table tennis, golf, etc. Taking table tennis as an example, if the tracking is mainly aimed at the table tennis moving fast, the technical difficulties can be determined as follows: the table tennis target is small, the detection cost and the false detection probability are high; and the table tennis moves fast, the target shape difference between adjacent frames is large, but the motion characteristics are obvious.

[0048] In one or more embodiments of the present application, the high-speed target refers to that in the image plane corresponding to the video stream, the position change of the target between frames is greater than or close to its own scale (or the position change is not much smaller than its own scale), and at the same time, its own motion characteristics do not change greatly in the time unit of the frame interval (or the frame rate is much smaller than the change frequency of the displacement size and direction).

[0049] In one or more embodiments of the present application, the root node is used to represent the starting point of the tracking process and control the tree depth. Except for the root node, each unpruned node in the tree is used to represent a track. Each layer of the tree corresponds to a video frame in the video stream, that is, the nodes in the same layer correspond to the same frame of the video stream. The nodes can be divided into two categories, the first node: the node itself points to a target detected in the video frame of this layer; the second node: once known as a null node, the node itself does not point to a target detected from the video, and is only responsible for connecting two nodes belonging to the previous frame and the next frame.

[0050] The main way of tracking the target can be divided into two parts, one is target detection, and the other is target tracking, which are specifically explained as follows:

[0051] Target detection part:

[0052] (1) Perform difference processing on the front, middle and rear three frames of video images, and perform threshold screening to obtain a binary image.

[0053] (2) Perform connected component analysis to screen out the target (multiple) similar to the table tennis from the binary image.

[0054] Tracking part:

[0055] (3) If the current is in the continuous tracking state, go to step (5), otherwise go to step (4).

[0056] (4) Check all the targets tracked in the continuous three frames to find whether there is a target combination that makes the positional relationship between the targets meet the preset motion rule (usually approximately located on the same straight line and the distance between each other is similar); if so, take the target combination (3 targets) that best meets the motion rule, and enter the continuous tracking state at the same time; return to (3).

[0057] (5) In all the new image of the target selected between the two previous detected target position relationship with the most consistent motion rules as a new frame of the detection results; If unable to find the target consistent with the motion rules, then exit the continuous tracking state, return to (3), while the previous continuous tracking results as a complete trajectory, to be output.

[0058] Referring to Figure 1 , briefly explain the above algorithm, such as starting tracking stage need in frame 1, 2, 3 in all 8 path (from any one frame 1 target, to any one frame 3 target) to select the most consistent motion constraints of frame 1 target 2 to frame 2 target 1 arrow -> frame 2 target 1 to frame 3 target 2 arrow path; Into the continuous tracking stage, need in frame 3 -> frame 4 in three path selection with frame 2 target 1 to frame 3 target 2 arrow combination, the most consistent motion constraints of frame 3 target 2 to frame 4 target 2 path, as the result of trajectory output; If the most consistent motion constraints of the path found still can not meet the minimum threshold requirements, then do not enter the continuous tracking state or end tracking.

[0059] On the basis of the above way, one of the existing high-speed moving target tracking way is: a kind of efficient golf ball detection and tracking based on convolutional neural networks and Kalman filter (Efficient Golf Ball Detection and Tracking Based on Convolutional Neural Networks and Kalman Filter), using a more powerful target detection method, supplemented by the position estimation of target to improve the detection efficiency and accuracy, and no judgment step, directly in the estimated image range selects the target with the maximum confidence as the tracking result.

[0060] The second existing high-speed moving target tracking way is: a method of modeling and propagating CNNs in a tree structure for visual tracking (Modeling and Propagating CNNs in a Tree Structure for Visual Tracking), mainly using CNN convolutional neural network to model and track the target, that is, updating the template of the target after detecting the target, which uses a tree structure to describe the similarity between different templates of the same detected target, so as to obtain a template with lower distortion and less interference in a wider time domain, preventing the tracking result from being greatly reduced due to one-time erroneous detection.

[0061] However, no matter which way is described above, there are the following shortcomings:

[0062] 1. It is not easy to track multiple targets, and only single target tracking can be performed;

[0063] 2. Tracking process is easy to interrupt, a complete trajectory is often interrupted by tracking restart;

[0064] 3. Algorithm robustness is poor, motion constraint judgment is relatively simple, only combined with the nearest information.

[0065] In view of the existing relatively simple target detection method in the presence of a large amount of interference and noise; the present application considers that the tree structure and the pruning algorithm based on high-speed motion characteristics are applied to realize relatively accurate target tracking.

[0066] Referring to Figure 2 An unpruned tree structure is shown, a total of 3 frames, in which two possible targets are detected in each frame: target 1 and target 2. Specifically, the tree structure is explained as follows:

[0067] 1. There are two types of nodes in the tree: the second node Null (which can be written as N node) and the first node, wherein the second node represents that the frame may not detect the target, and the Root node of the tree is a special root node. The use of the second node is: when a single frame does not detect the target, the last frame and the next frame of the single frame can still be connected by the second node when backtracking; the first node is used to record a target detected by the tree in the corresponding frame. Different nodes in the same layer may correspond to the same target, but must not be the same track or tracking result.

[0068] 2. Each layer of the tree represents the detection result of the same frame, and the target of the new frame is placed in the lowest layer.

[0069] 3. Each node of the tree corresponds to a possible trajectory before the corresponding frame of the node layer.

[0070] 4. The parent node of all nodes is the first node detected in the last frame, or the second node of the last frame; the child node of all nodes is the first node detected in the next frame, or the second node of the next frame; the parent node of all nodes is unique.

[0071] Based on the above tree structure, the main operation of the tree structure provided in the present application can include the following contents:

[0072] 1. Add new frame possible target: for each upper layer node that is not pruned, insert all targets detected in the new frame below the node;

[0073] 2. Pruning: determine to prune the node after judging by the pruning algorithm; set the pointer to the parent node to null value (i.e. the node cannot be backtracked); the pruned node cannot exist as the parent node of any node;

[0074] 3. Backtracking: Backtracking to the root node can obtain the information of each tracked target in the trajectory, and the trajectory is judged, combined with the algorithm and function (usually given according to the motion law and target detection result) to judge whether to watch, output, or prune.

[0075] With the above tree structure and corresponding operation support, all possible trajectory options can be traversed and judged by traversing the nodes at the lowest layer one by one. In actual code implementation, the nodes at the lowest layer form a linked list, and the entire maintenance process of the tree can be completed by directly traversing the linked list.

[0076] Based on this, the embodiment of the application provides a high-speed moving target tracking method based on tree pruning, referring to Figure 3 , the high-speed moving target tracking method based on tree pruning specifically includes the following contents:

[0077] Step 100: updating a tracking tree corresponding to a video stream by adding a layer and inserting a node in the newly added layer according to a target detection result of a current frame of the video stream, wherein the video stream includes at least one high-speed moving object, the tracking tree is used to represent a target tracking trajectory, each layer of the tracking tree corresponds to a video frame one by one, and each node in the tracking tree corresponds to each trajectory one by one through backtracking.

[0078] In step 100, in the actual process, in order to facilitate the searching of parent nodes and the pruning of similar trajectories, the child nodes with the same parent node in the tracking tree are placed in adjacent positions in the linked list (corresponding to which is that the child nodes with the same target are placed in adjacent positions). The video frame refers to a video frame corresponding to the video stream.

[0079] Step 200: performing pruning processing on the tracking tree based on a pruning algorithm corresponding to the high-speed motion characteristics of the object, and performing structure reduction maintenance processing on the tracking tree when the pruned tracking tree meets a preset layer number control rule.

[0080] It can be understood that the target tracking trajectory or trajectory refers to the result of tracking, and a target chain composed of single-frame targets.

[0081] As can be seen from the above description, the high-speed moving target tracking method based on tree pruning provided in the embodiments of the present application uses a tree structure to describe and distinguish possible target trajectories, analyzes the possible trajectories by traversing the pruned nodes, can realize tracking of one or multiple targets at the same time, and due to the hierarchical connection and backtracking features of the tree structure, the tracking process is not prone to interruption, can focus on information in a longer time domain range based on the tracking tree, make accurate judgments, and reduce the false detection rate; by pruning based on the pruning algorithm corresponding to the high-speed motion feature of the object, the robustness and anti-interference ability of the high-speed moving target tracking process can be effectively improved; by controlling the number of layers of the tracking tree, the high-speed moving target tracking process has moderate time and space complexity, can effectively release memory space, reduce the depth of the tracking tree, and thus can effectively improve the accuracy, efficiency and convenience of obtaining the target tracking trajectory.

[0082] In order to effectively handle the situation of temporary loss of the target in the frame, in the high-speed moving target tracking method based on tree pruning provided in the embodiments of the present application, the tracking tree in the high-speed moving target tracking method based on tree pruning includes multiple longitudinally connected layers starting from one node, and each layer is composed of at least one node, and each node is connected to a parent node in the previous layer of the layer where the node is located.

[0083] The node includes a first node and a second node, wherein the first node is used to represent the frame identifier and the detected target identifier corresponding to the layer where the first node is located, and the second node is used to realize relationship backtracking between the parent node in the previous layer connected to the second node and the child node in the next layer connected to the second node.

[0084] As can be seen from the above description, the high-speed moving target tracking method based on tree pruning provided in the embodiments of the present application can effectively handle the situation of temporary loss of the target in the frame by using the second node for realizing relationship backtracking between the parent node in the previous layer connected to the second node and the child node in the next layer connected to the second node in the tree structure, can further ensure that the tracking process will not be interrupted due to temporary loss of the target, and thus can further improve the accuracy and reliability of obtaining the target tracking trajectory.

[0085] In order to improve the reliability and effectiveness of the pruning process of the tracking tree, in the high-speed moving target tracking method based on tree pruning provided in the embodiments of the present application, referring to Figure 4 , the step 200 in the high-speed moving target tracking method based on tree pruning specifically contains the following content:

[0086] Step 210: pruning the nodes in the tracking tree that do not conform to the high-speed motion feature of the object to complete the preliminary pruning of the tracking tree.

[0087] It can be understood that if the trajectory corresponding to a node obviously does not conform to the motion feature (usually only 2-3 frames of data can be combined to determine), the node will be pruned to control the size of the tree.

[0088] For the convenience of understanding, part of the trajectory analysis logic (including pruning and judgment) is given here as an example, which mainly aims to process the high-speed motion feature and relationship. However, it can be understood that the specific parameters and forms may change when facing targets with different motion features. On the basis of the specific operation of pruning in combination with the high-speed motion feature, the specific pruning and judgment logic can be set according to the actual application situation, and these specific pruning and judgment logic are within the protection scope.

[0089] From the above description, it can be seen that the high-speed moving target tracking method based on tree pruning provided by the embodiments of the application can effectively improve the reliability and effectiveness of pruning processing of the tracking tree by preliminarily pruning the tracking tree and pruning the nodes whose trajectories obviously do not conform to the motion feature, thereby further improving the efficiency and reliability of obtaining the target tracking trajectory.

[0090] In order to further improve the reliability and effectiveness of pruning processing of the tracking tree, in a high-speed moving target tracking method based on tree pruning provided by an embodiment of the application, referring to Figure 4 , step 200 in the high-speed moving target tracking method based on tree pruning further specifically includes the following content:

[0091] Step 220: performing complete backtracking on the nodes in the last layer of the tracking tree to obtain a backtracking result containing parameters of each target and relationship data between the targets.

[0092] Step 230: pruning the nodes belonging to non-trajectories and the nodes conforming to the similar suppression rule in the tracking tree according to the high-speed motion feature of the object and the backtracking result, to complete the backtracking pruning of the tracking tree.

[0093] It can be understood that the above step 220 can be executed with step 210 or simultaneously. In a preferred scheme of the application, in order to further improve the reliability of pruning, referring to Figure 4 , step 210 and step 220 can be executed in sequence.

[0094] Among them, the last layer of the nodes in the node is completely backtracked to find the specific parameters of each target, the positional relationship between the targets, and the judgment is made; for the following three cases, there are corresponding subsequent operations:

[0095] (1) Determine that this is a non-track without observation by motion characteristics, the proportion of N nodes in the track, etc. Directly prune.

[0096] (2) Cannot determine whether this is a track: do not do anything.

[0097] (3) Determine a track output: prune the same layer nodes pointing to the same first node and having the same parent node.

[0098] From the above description, the high-speed moving target tracking method based on tree pruning provided by the embodiments of the application can further improve the reliability and effectiveness of pruning processing of the tracking tree by backtracking pruning of the tracking tree, backtracking the nodes of the tracking tree, and cutting the nodes belonging to non-tracks and the nodes meeting the similar suppression rule in the tracking tree. In turn, the efficiency and reliability of obtaining the target tracking trajectory can be further improved.

[0099] In order to make the high-speed moving target tracking process have moderate time and space complexity, in the high-speed moving target tracking method based on tree pruning provided by the embodiments of the application, step 240 can be further included after step 210 or step 230, for example, see Figure 4 The step 200 of the high-speed moving target tracking method based on tree pruning further specifically includes the following contents:

[0100] Step 240: respectively acquire the current depth of the tracking tree and the depth of the longest trajectory displayed therein.

[0101] Step 250: If the depth of the current longest trajectory is less than the depth of the tracking tree, the root node of the tracking tree is transferred downward and all nodes before the transferred root node are deleted to realize the structure reduction of the tracking tree.

[0102] It can be understood that after multi-frame and long-time tracking, there are more pruned nodes in the tree, and the depth is also deeper, but in fact the length of the current trajectory is much smaller than the tree depth, so the root node can be transferred, all previous nodes can be completely discarded, and the memory space can be released to ensure that the algorithm can run normally.

[0103] From the above description, the high-speed moving target tracking method based on tree pruning provided by the embodiments of the application can make the high-speed moving target tracking process have moderate time and space complexity by transferring the root node of the tracking tree downward and deleting all nodes before the transferred root node. The memory space can be effectively released, and the depth of the tracking tree can be reduced.

[0104] In order to further improve the accuracy and reliability of the high-speed moving target tracking result, in the high-speed moving target tracking method based on tree pruning provided in the embodiment of the present application, referring to Figure 4 , the step 100 in the high-speed moving target tracking method based on tree pruning further specifically comprises the following content before the step 100:

[0105] Step 010: receiving a video stream containing at least one high-speed moving object.

[0106] Step 020: establishing a corresponding tracking tree for the video stream.

[0107] Step 030: during the continuous input of the video stream, sequentially performing target detection on each frame of the video stream based on a preset target detection method, and after obtaining the target detection result of each frame, taking the target detection result as the target detection result of the current frame to start the process of updating the tracking tree.

[0108] In step 030, the target detection method can select a target detection algorithm based on video difference. The algorithm can also be replaced by other target detection methods, and a trajectory discrimination algorithm designed in combination with the target detection method can be used to obtain better results without affecting the framework of the present application.

[0109] It can be understood that after obtaining the target detection result of each frame in step 030, the target detection result is taken as the target detection result of the current frame to start step 100.

[0110] As can be seen from the above description, the high-speed moving target tracking method based on tree pruning provided in the embodiment of the present application can effectively improve the accuracy and reliability of the high-speed moving target tracking result by sequentially performing target detection on each frame of the video stream based on a preset target detection method.

[0111] In order to improve the efficiency and reliability of the target tracking trajectory result output, in the high-speed moving target tracking method based on tree pruning provided in the embodiment of the present application, referring to Figure 4 , the step 200 in the high-speed moving target tracking method based on tree pruning further specifically comprises the following content after the step 200:

[0112] Step 300: if the current frame of the video stream is the last frame of the video stream, generating target tracking trajectory data corresponding to the video stream according to the structure of the current tracking tree.

[0113] In actual tracking, if a track is determined to end (a node is pruned), the track can be generated and output. Meanwhile, the video can be rendered in near real time during tracking, and it is not necessary to wait until the end of the video to output. The current output and file storage are completed uniformly after the video ends. As described above, the high-speed moving target tracking method based on tree pruning provided in the embodiments of the present application can generate and output the target tracking trajectory data corresponding to the video stream according to the structure of the current tracking tree, thereby effectively improving the efficiency and reliability of the target tracking trajectory result output, and further effectively improving the efficiency and convenience of the user obtaining the target tracking trajectory result.

[0114] From the software level, the present application also provides a high-speed moving target tracking device based on tree pruning for executing all or part of the high-speed moving target tracking method based on tree pruning. Referring to Figure 5 , the high-speed moving target tracking device based on tree pruning specifically includes the following contents:

[0115] A new target adding module 10 is configured to update the tracking tree corresponding to the video stream by adding layers and inserting nodes in the newly created layers according to the target detection result of the current frame of the video stream, wherein the video stream contains at least one high-speed moving object, the tracking tree is used to represent the target tracking trajectory, each layer of the tracking tree corresponds to a video frame, and each node in the tracking tree corresponds to each trajectory by backtracking.

[0116] A pruning and layer number control module 20 is configured to perform pruning processing on the tracking tree based on a pruning algorithm corresponding to the high-speed motion characteristics of the object, and perform structure reduction maintenance processing on the tracking tree when the pruned tracking tree meets the preset layer number control rule.

[0117] The embodiments of the high-speed moving target tracking device based on tree pruning provided in the present application can be specifically used to execute the processing flow of the embodiments of the high-speed moving target tracking method based on tree pruning described above, and the functions thereof will not be repeated here. For details, refer to the detailed description of the embodiments of the high-speed moving target tracking method based on tree pruning.

[0118] The part of the high-speed moving target tracking device based on tree pruning for performing high-speed moving target tracking can be executed in a server, and in another actual application scenario, all operations can be completed in a client device. Specifically, the selection can be made according to the processing capability of the client device and the limitation of the user usage scenario. The present application does not make any limitation thereon. If all operations are completed in the client device, the client device can further include a processor for performing the specific processing of the high-speed moving target tracking based on tree pruning.

[0119] The client device described above can have a communication module (i.e., a communication unit) that can be communicatively connected with a remote server to achieve data transmission with the server. The server can include a server of a task scheduling center side, and in other implementation scenarios, can also include a server of an intermediate platform, such as a server of a third-party server platform that is communicatively linked with the server of the task scheduling center. The server can include a single computer device, or can include a server cluster composed of multiple servers, or a server structure of a distributed device.

[0120] The server and the client device can communicate with each other using any suitable network protocol, including a network protocol that has not been developed as of the filing date of the present application. The network protocol can include, for example, a TCP / IP protocol, a UDP / IP protocol, an HTTP protocol, an HTTPS protocol, and the like. Of course, the network protocol can also include, for example, a RPC protocol (Remote Procedure Call Protocol) and a REST protocol (Representational State Transfer) used on top of the above-mentioned protocols.

[0121] As can be seen from the above description, the high-speed moving target tracking device based on tree pruning provided by the embodiments of the present application uses a tree structure to describe and distinguish possible target trajectories, analyzes the possible trajectories by traversing the pruned nodes, can realize tracking of one or multiple targets at the same time, and due to the hierarchical connection and backtracking characteristics of the tree structure, the tracking process is not prone to interruption, can focus on information in a longer time domain based on the tracking tree, make accurate judgments, and reduce the false detection rate. By pruning based on the pruning algorithm corresponding to the high-speed motion characteristics of the object, the robustness and anti-interference ability of the high-speed moving target tracking process can be effectively improved. By controlling the number of layers of the tracking tree, the high-speed moving target tracking process has moderate time and space complexity, can effectively release memory space, reduce the depth of the tracking tree, and thus can effectively improve the accuracy, efficiency and convenience of obtaining the target tracking trajectory.

[0122] To further illustrate the present scheme, the present application also provides a specific application example of a high-speed moving target tracking method based on tree pruning, which is described with reference to Figure 6 After the tracking tree is established, the tree is maintained through four main steps (B, C, D, and E), which mainly prevent the disorderly expansion of the tree size through a pruning algorithm, control the size of the tree within a reasonable range, and output the tracking result through backtracking judgment. The high-speed moving target tracking method based on tree pruning specifically includes the following contents.

[0123] A. The steps are merely a target detection method, which can be replaced with other target detection methods and combined with a trajectory discrimination algorithm designed in conjunction with the target detection method to achieve better results without affecting the framework of this application.

[0124] B. Tree-based operations: In practice, to facilitate finding parent nodes and pruning similar trajectories, child nodes with the same parent node are placed in adjacent positions in the linked list (correspondingly, child nodes with the same goal are placed in adjacent positions).

[0125] C. See also Figure 7 If the trajectory corresponding to a node clearly does not conform to the motion characteristics (usually this can be determined by combining only 2-3 frames of data), then that node will be pruned to control the size of the tree (e.g., Figure 7 In the second layer (left frame 1, left frame 2, target 2, and left frame 5, target 2), step C may not backtrack to the root node when performing node backtracking.

[0126] D. such as Figure 7 The third layer involves a complete backtracking of the nodes in the last layer to find the specific parameters of each target, the positional relationships between targets, and to make a judgment. For the following three cases, there are corresponding follow-up operations:

[0127] (1) Based on factors such as motion characteristics and the proportion of N nodes in the trajectory, it is determined that this is a non-trajectory that does not require observation; direct pruning is then performed; (e.g.) Figure 7 The N nodes in the third layer from the left, the ninth from the left, and the first from the right all have trajectories that exceed two N nodes, so they can be pruned directly.

[0128] (2) If it cannot be determined whether this is a trajectory: do not perform any operation;

[0129] (3) Determine a trajectory for output ( Figure 7 The leftmost frame 3 target 1 and the leftmost frame 3 target 2 in the third layer: for the same first node in the same layer ( Figure 7 The nodes at the third level (left 2, left 4, left 7, left 8, left 10, right 2, right 3, right 5, right 6, right 8) and nodes at the same level that share the same grandparent node ( Figure 7 Prune the branches in the left sixth, right fourth, and right seventh positions.

[0130] E. After multi-frame and long-term tracking, the tree will have many pruned nodes and be quite deep. However, the length of the current trajectory is much smaller than the tree depth. Therefore, the root node can be moved, all previous nodes can be discarded, memory space can be freed up, and the algorithm can continue to run normally. Figure 8 Root node transfer refers to moving the root node to the rightmost node N in the second level, which allows us to obtain the sum without affecting any other judgments.Figure 7 The same tree structure, reducing the depth of the tree.

[0131] As Figure 7 shown, two or more targets can be tracked in the tree structure at the same time, one of which has no detection result in the second frame, but will not cause tracking interruption, and the final tracking result output allows such a situation.

[0132] In the process of trying specific solutions, the direct application of tree structure brings the benefits of easy traversal and allows the processing of extremely complex situations (a very complete and universal architecture). The main problem is that the tree size expands rapidly over time and video frame input. Without a good pruning algorithm, the rapid expansion of the tree structure will make the algorithm unusable, so a lot of work and operations appear in the maintenance of the tree structure, such as pruning and judgment.

[0133] For ease of understanding, part of the trajectory analysis logic (including pruning and judgment) is given as an example here, which mainly aims to process high-speed motion characteristics and relationships. However, it can be understood that when facing targets with different motion characteristics, the specific parameters and forms may change. Based on the specific operation of pruning combined with high-speed motion characteristics, the specific pruning and judgment logic can be set according to the actual application situation. These specific pruning and judgment logic are within the scope of protection. Take the target as a table tennis ball as an example to explain the pruning and judgment logic:

[0134] 1. Compare the length and angle of the vector between the two targets tracked continuously 3 times; for the case of table tennis ball, the requirements are: angle < 60°, short vector not less than 40% of long vector;

[0135] 2. The distance between the two targets tracked continuously is not more than 50 (pixel plane distance); the size difference between the two targets is not more than 40%;

[0136] 3. The proportion of null nodes in any one trajectory is not more than 0.4;

[0137] 4. The trajectories under the same grandparent node (the parent node of the current node's parent node) can only be left with no more than 4;

[0138] 5. If the number of effective targets in the trajectory exceeds 5, the two targets are similar, the length of the inter-vector is similar, and the average angle difference is less than 30°; then as a possible target, it is prepared for output, etc.

[0139] Since the high-speed moving object has strong regularity and basically presents straight-line movement in the image, a large amount of screening can be performed in combination with the positional relationship between two targets (C. main work), in addition, the high-speed moving object appears in the video for a short time and has a short duration, and is very obvious relative to other target features, so that the possible target can be clearly screened in the D step, and the optional items having the same parent node or grandparent node are pruned; the above is a specific example of how the high-speed moving object affects the pruning, and there are many steps and details in the actual algorithm.

[0140] In summary, the above method provided by the application example uses a tree structure to describe and distinguish possible target trajectories, and analyzes the possible trajectories by traversing the pruned nodes. It can be understood that this point has certain universality and can be widely used in video stream target tracking problems, and has strong compatibility for technical details. (Convenient to combine other target detection means, motion features, etc.). The pruning algorithm of the high-speed motion feature is mainly used, and the tree is maintained by auxiliary operations such as layer control, so that the algorithm has moderate time and space complexity, and realizes target tracking. The concept of the second node is introduced in the tree structure to handle the situation of temporary loss of the target in the frame.

[0141] Based on this, the application example has at least the following beneficial effects:

[0142] 1. The tracking process will not be interrupted due to temporary loss of the target;

[0143] 2. Multiple targets can be tracked at the same time;

[0144] 3. It has better robustness and anti-interference ability: the sensitivity of the algorithm to the change of the preset parameter is greatly reduced;

[0145] 4. Focus on information in a longer time domain, make accurate judgments, and reduce the false detection rate.

[0146] The embodiment of the application also provides a computer device (i.e. electronic device), which can include a processor, a memory, a receiver and a transmitter, the processor is used to execute the high-speed moving target tracking method based on tree pruning mentioned in the above embodiment, wherein the processor and the memory can be connected through a bus or other means, for example, through a bus connection. The receiver can be connected with the processor and the memory through wired or wireless means. The computer device is in communication connection with the high-speed moving target tracking device based on tree pruning, to receive real-time motion data from the sensor in the wireless multimedia sensor network, and receive the original video sequence from the video acquisition device.

[0147] The processor can be a central processing unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, or a combination thereof.

[0148] The memory, as a non-transitory computer readable storage medium, can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the tree pruning based high-speed moving target tracking method in the embodiments of the present application. The processor executes various functions and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory, that is, implements the tree pruning based high-speed moving target tracking method in the above method embodiments.

[0149] The memory can include a program storage area and a data storage area. The program storage area can store an operating system and application programs required by at least one function; the data storage area can store data created by the processor and the like. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged with respect to the processor, and these remote memories can be connected to the processor through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0150] The one or more modules are stored in the memory and, when executed by the processor, perform the tree pruning based high-speed moving target tracking method in the embodiments.

[0151] In some embodiments of the present application, the user equipment can include a processor, a memory and a transceiver unit which can include a receiver and a transmitter, the processor, the memory, the receiver and the transmitter can be connected through a bus system, the memory is used to store computer instructions, and the processor is used to execute the computer instructions stored in the memory to control the transceiver unit to transceive signals.

[0152] As an implementation manner, the functions of the receiver and the transmitter in the present application can be realized by a transceiver circuit or a transceiver dedicated chip, and the processor can be realized by a dedicated processing chip, a processing circuit or a general-purpose chip.

[0153] As another implementation manner, the server provided by the embodiments of the present application can be implemented by using a general computer. That is, program codes for implementing the functions of the processor, the receiver and the transmitter are stored in the memory, and the general processor implements the functions of the processor, the receiver and the transmitter by executing the codes in the memory.

[0154] The embodiments of the present application further provide a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the steps of the aforementioned tree pruning based high-speed moving target tracking method. The computer readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0155] Those skilled in the art should understand that the exemplary components, systems and methods described in connection with the embodiments disclosed herein can be implemented in hardware, software or a combination thereof. The actual implementation depends on the specific application and design constraints imposed on the overall system. Those skilled in the art can use various methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a functional card, etc. When implemented in software, the elements of the present application are program or code segments used to perform the required tasks. The program or code segments can be stored in a machine readable medium or transmitted through a data signal carried in a carrier wave in a transmission medium or communication link.

[0156] It should be noted that the present application is not limited to the specific configurations and processes described above and shown in the drawings. For the sake of brevity, detailed descriptions of known methods are omitted. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present application are not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order of the steps, after understanding the spirit of the present application.

[0157] In the present application, the features described and / or illustrated for one embodiment can be used in the same way or in a similar way in one or more other embodiments, and / or in combination with or instead of features of other embodiments.

[0158] The above descriptions are only the preferred embodiments of the present application, and are not intended to limit the present application. The embodiments of the present application can be variously changed and modified by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of protection of the present application.

Claims

1. A tree-pruning based high-speed moving target tracking method, characterized by, The application comprises the following steps: According to the target detection result of the current frame of the video stream, the tracking tree corresponding to the video stream is updated by adding layers and inserting nodes in the newly-built layers, wherein the video stream contains at least one high-speed moving object, the tracking tree is used to represent the target tracking trajectory, each layer of the tracking tree corresponds to a video frame one by one, and each node in the tracking tree corresponds to a trajectory one by one through backtracking; The tracking tree is pruned based on the pruning algorithm corresponding to the high-speed motion characteristics of the object, and when the pruned tracking tree meets the preset layer number control rule, the structure of the tracking tree is maintained by reducing; The tracking tree comprises a plurality of longitudinally connected layers starting from a node, and each layer comprises at least one node, and each node is connected to a parent node in the layer above it. The node comprises a first node and a second node, wherein the first node is used to represent the frame identifier and the detected target identifier of the layer where it is located, and the second node is used to realize the relationship backtracking between the parent node of the layer above it and the child node of the layer below it. When the pruned tracking tree meets the preset layer number control rule, the structure of the tracking tree is maintained by reducing, comprising: Respectively acquiring the current depth of the tracking tree and the depth of the longest trajectory displayed in the tracking tree; If the depth of the current longest trajectory is less than the depth of the tracking tree, the root node of the tracking tree is transferred downward, and all nodes before the transferred root node are deleted to realize the structure reduction of the tracking tree.

2. The tree-pruning based high-speed moving target tracking method according to claim 1, characterized in that, The tracking tree is pruned based on the pruning algorithm corresponding to the high-speed motion characteristics of the object, comprising: In the tracking tree, the nodes that do not meet the high-speed motion characteristics of the object are pruned to complete the preliminary pruning of the tracking tree. 3.The tree-pruning based high-speed moving target tracking method according to claim 1, wherein, The tracking tree is pruned based on the pruning algorithm corresponding to the high-speed motion characteristics of the object, comprising: The nodes in the last layer of the tracking tree are completely backtracked to obtain the backtracking result containing the parameters of each target and the relationship data between the targets; According to the high-speed motion characteristics of the object and the backtracking result, the nodes in the tracking tree that belong to non-trajectories and the nodes that meet the similar suppression rule are pruned to complete the backtracking pruning of the tracking tree.

4. The tree-pruning based high-speed moving target tracking method according to claim 1, wherein, Before the tracking tree corresponding to the video stream is updated according to the target detection result of the current frame of the video stream by adding layers and inserting nodes in the newly-built layers, the following steps are further included: A video stream containing at least one high-speed moving object is received; A corresponding tracking tree is established for the video stream; During the continuous input of the video stream, the frames of the video stream are sequentially detected based on the preset target detection method, and after obtaining the target detection result of each frame, the target detection result is taken as the target detection result of the current frame to start the process of updating the tracking tree.

5. The tree-pruning based high-speed moving target tracking method according to any one of claims 1 to 4, characterized in that, Further comprising: If the current frame of the video stream is the last frame of the video stream, the target tracking trajectory data corresponding to the video stream is generated according to the structure of the current tracking tree.

6. A high-speed moving target tracking device based on tree pruning, characterized by, Comprising: The new target adding module is configured to update a tracking tree corresponding to a video stream according to a target detection result of a current frame of the video stream, by adding a layer and inserting a node into the newly added layer, wherein the video stream contains at least one high-speed moving object, the tracking tree is used to represent a target tracking trajectory, each layer of the tracking tree corresponds to a video frame one by one, and each node in the tracking tree corresponds to a trajectory one by one through backtracking; The pruning and layer number control module is configured to perform pruning processing on the tracking tree based on a pruning algorithm corresponding to a high-speed motion feature of the object, and perform structure reduction maintenance processing on the tracking tree when the tracking tree after pruning meets a preset layer number control rule; The tracking tree includes a plurality of longitudinally connected layers starting from one node, and each layer is composed of at least one node, and each node is connected to a parent node in a layer above it; The node includes a first node and a second node, wherein the first node is used to represent a frame identifier and a detected target identifier corresponding to the layer where the first node is located, and the second node is used to realize relationship backtracking between a parent node in a layer above the second node and a child node in a layer below the second node; When the tracking tree after pruning meets the preset layer number control rule, the structure reduction maintenance processing on the tracking tree includes: Respectively acquiring a current depth of the tracking tree and a depth of a longest trajectory displayed in the tracking tree; If the depth of the longest trajectory is less than the depth of the tracking tree, the root node of the tracking tree is transferred downward, and all nodes before the transferred root node are deleted to realize structure reduction of the tracking tree.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to realize the high-speed moving target tracking method based on tree pruning according to any one of claims 1 to 5.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the high-speed moving target tracking method based on tree pruning according to any one of claims 1 to 5.

Images