Target tracking methods, apparatus, computer equipment, storage media and software products

Abstract

This application relates to a target tracking method, apparatus, computer device, storage medium, and program product, belonging to the field of artificial intelligence technology. The method includes: extracting apparent features of the target to be tracked, and determining a first candidate position of the target in the current frame based on the result of the apparent feature extraction; predicting the motion trajectory of the target, and determining a second candidate position of the target in the current frame based on the result of the motion trajectory prediction; and determining the target position in the current frame from the first and second candidate positions. This method can improve the reliability of target tracking.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a target tracking method, apparatus, computer device, storage medium, and program product. Background Technology

[0002] Intelligent surveillance plays an important role in urban traffic, monitoring of key locations, and tracking of criminals. Among these, target tracking is the core research content of intelligent surveillance.

[0003] Existing target tracking methods match the target's appearance features between consecutive frames to track the target. Appearance features refer to the target's physical characteristics. For example, for a person, appearance features include information such as hair color, gender, hair length, height, and type of clothing.

[0004] However, existing target tracking methods often fail when the target is occluded or subjected to similar interference, resulting in low tracking reliability. Summary of the Invention

[0005] Therefore, it is necessary to provide a target tracking method, apparatus, computer equipment, storage medium, and program product to address the aforementioned technical problems.

[0006] In a first aspect, this application provides a target tracking method, which includes: extracting appearance features of the target to be tracked, and determining a first candidate position of the target in the current frame based on the result of the appearance feature extraction; predicting the motion trajectory of the target, and determining a second candidate position of the target in the current frame based on the result of the motion trajectory prediction; and determining the target position of the target in the current frame from the first candidate position and the second candidate position.

[0007] In one embodiment, the apparent feature extraction of the target to be tracked and the determination of the first candidate position of the target in the current frame based on the result of the apparent feature extraction include: obtaining a template frame, wherein the template frame is a video frame that is temporally preceding the current frame and contains the target; inputting the template frame and the current frame into the template branch and the detection branch of the Siamese network, respectively, to obtain the first apparent feature output by the template branch and the second apparent feature output by the detection branch; performing a cross-correlation operation on the first apparent feature and the second apparent feature, and determining the first candidate position based on the result of the cross-correlation operation.

[0008] In one embodiment, inputting the template frame and the current frame into the template branch and the detection branch of the Siamese network respectively includes: cropping the template frame based on the center of the bounding box in the template frame to obtain a first image block, the bounding box being used to outline the target to be tracked; cropping the current frame based on the center of the bounding box in the previous frame of the current frame to obtain a second image block; and inputting the first image block and the second image block into the template branch and the detection branch respectively.

[0009] In one embodiment, the size of the first image block is M*M, and the size of the second image block is 2M*2M, where M is calculated based on the width and height of the bounding box.

[0010] In one embodiment, performing a cross-correlation operation on the first appearance feature and the second appearance feature, and determining the first candidate position of the target in the current frame based on the result of the cross-correlation operation, includes: performing a cross-correlation operation on the first appearance feature and the second appearance feature to obtain a first response map; converting the first response map into a second response map based on a bicubic interpolation algorithm, wherein the response values ​​in the second response map represent the similarity between the first appearance feature and the second appearance feature; and determining the first candidate position based on the second response map.

[0011] In one embodiment, determining a first candidate location based on a second response map includes: obtaining location information corresponding to the maximum response value in the second response map; and determining the first candidate location based on the location information.

[0012] In one embodiment, the motion trajectory prediction of the target is performed, and the second candidate position of the target in the current frame is determined based on the motion trajectory prediction result. This includes: acquiring multiple historical frames, wherein the historical frames are video frames that are chronologically preceding the current frame and contain the target; inputting the position coordinates of the target in the multiple historical frames into a long short-term memory network in chronological order; and determining the second candidate position based on the output of the long short-term memory network.

[0013] In one embodiment, the position coordinates of the target in multiple historical frames are sequentially input into a Long Short-Term Memory (LSTM) network, and a second candidate position is determined based on the output of the LSTM network. This includes: inputting the position coordinates of the target in multiple historical frames sequentially into the LSTM network to obtain a first hidden state output by the LSTM network; inputting the first hidden state and the position coordinates of the target in the previous frame of the current frame into the LSTM network to obtain a second hidden state output by the LSTM network; and determining a second candidate position based on the second hidden state.

[0014] In one embodiment, determining the target position of the target in the current frame from the first candidate position and the second candidate position includes: calculating the average overlap accuracy between the current frame and the previous frame, where the average overlap accuracy is the intersection-union ratio of the areas of the bounding boxes in the current frame and the bounding boxes in the previous frame; if the maximum response value is not less than a first preset threshold or the average overlap accuracy is not less than a second preset threshold, then the first candidate position is determined as the target position of the target in the current frame; if the maximum response value is less than the first preset threshold and the average overlap accuracy is less than the second preset threshold, then the second candidate position is determined as the target position of the target in the current frame.

[0015] In one embodiment, calculating the average overlap accuracy between the current frame and the previous frame includes: calculating the intersection area of ​​the bounding boxes in the current frame and the bounding boxes in the previous frame; calculating the union area of ​​the bounding boxes in the current frame and the bounding boxes in the previous frame; and calculating the ratio of the intersection area to the union area to obtain the average overlap accuracy.

[0016] Secondly, this application also provides a target tracking device, which includes: a first determining module, used to extract appearance features of the target to be tracked, and determine a first candidate position of the target in the current frame based on the result of the appearance feature extraction; a second determining module, used to predict the motion trajectory of the target, and determine a second candidate position of the target in the current frame based on the result of the motion trajectory prediction; and a third determining module, used to determine the target position of the target in the current frame from the first candidate position and the second candidate position.

[0017] In one embodiment, the first determining module is specifically used for: obtaining a template frame, wherein the template frame is a video frame that is temporally preceding the current frame and contains the target; inputting the template frame and the current frame into the template branch and the detection branch of the Siamese network respectively to obtain a first appearance feature output by the template branch and a second appearance feature output by the detection branch; performing a cross-correlation operation on the first appearance feature and the second appearance feature, and determining a first candidate position based on the result of the cross-correlation operation.

[0018] In one embodiment, the first determining module is specifically used to: crop the template frame based on the center of the bounding box in the template frame to obtain a first image block, wherein the bounding box is used to outline the target to be tracked; crop the current frame based on the center of the bounding box in the previous frame to obtain a second image block; and input the first image block and the second image block into the template branch and the detection branch, respectively.

[0019] In one embodiment, the size of the first image block is M*M, and the size of the second image block is 2M*2M, where M is calculated based on the width and height of the bounding box.

[0020] In one embodiment, the first determining module is specifically used for: performing a cross-correlation operation on the first apparent feature and the second apparent feature to obtain a first response map; converting the first response map into a second response map based on a bicubic interpolation algorithm, wherein the response values ​​in the second response map represent the magnitude of the similarity between the first apparent feature and the second apparent feature; and determining a first candidate position based on the second response map.

[0021] In one embodiment, the first determining module is specifically used to: obtain the location information corresponding to the maximum response value in the second response map; and determine the first candidate location based on the location information.

[0022] In one embodiment, the second determining module is specifically used to: acquire multiple historical frames, wherein the historical frames are video frames that are in time sequence before the current frame and contain the target; input the position coordinates of the target in the multiple historical frames into the Long Short-Term Memory network in chronological order; and determine the second candidate position based on the output of the Long Short-Term Memory network.

[0023] In one embodiment, the second determining module is specifically used to: input the position coordinates of the target in multiple historical frames into the Long Short-Term Memory (LSTM) network in chronological order to obtain a first hidden state output by the LSTM network; input the first hidden state and the position coordinates of the target in the previous frame of the current frame into the LSTM network to obtain a second hidden state output by the LSTM network; and determine a second candidate position based on the second hidden state.

[0024] In one embodiment, the third determining module is specifically configured to: calculate the average overlap accuracy between the current frame and the previous frame, wherein the average overlap accuracy is the intersection-union ratio of the areas of the bounding boxes in the current frame and the bounding boxes in the previous frame; if the maximum response value is not less than a first preset threshold or the average overlap accuracy is not less than a second preset threshold, then determine the first candidate position as the target position of the target in the current frame; if the maximum response value is less than the first preset threshold and the average overlap accuracy is less than the second preset threshold, then determine the second candidate position as the target position of the target in the current frame.

[0025] In one embodiment, the third determining module is specifically used to: calculate the intersection area of ​​the bounding box in the current frame and the bounding box in the previous frame; calculate the union area of ​​the bounding box in the current frame and the bounding box in the previous frame; calculate the ratio of the intersection area to the union area to obtain the average overlap accuracy.

[0026] Thirdly, this application also provides a computer device, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of the method described in any of the first aspects above.

[0027] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any of the first aspects above.

[0028] Fifthly, this application also provides a computer program product. This computer program product includes a computer program that, when executed by a processor, implements the steps of the method described in any of the first aspects above.

[0029] The beneficial effects of the technical solutions provided in this application include at least the following:

[0030] In this embodiment, after extracting the apparent features of the target to be tracked and determining the first candidate position of the target in the current frame based on the results of the apparent feature extraction, the target's motion trajectory is also predicted, and a second candidate position of the target in the current frame is determined based on the results of the motion trajectory prediction. Finally, the target position in the current frame is determined from the first and second candidate positions. Therefore, this embodiment utilizes both apparent features and motion trajectories to track the target. This allows for improved tracking reliability, especially when the target is occluded or subject to similar interference. Attached Figure Description

[0031] Figure 1 A schematic diagram of an implementation environment provided for an embodiment of this application;

[0032] Figure 2 A flowchart illustrating a target tracking method provided in an embodiment of this application;

[0033] Figure 3 A flowchart illustrating a technical process for determining a first candidate position, provided as an embodiment of this application;

[0034] Figure 4 A flowchart illustrating a technical process for determining a second candidate position, provided as an embodiment of this application;

[0035] Figure 5 A flowchart illustrating a technical process for determining a target location, provided in an embodiment of this application;

[0036] Figure 6 A schematic diagram illustrating an average overlap accuracy provided in an embodiment of this application;

[0037] Figure 7 A flowchart illustrating a target tracking method provided in an embodiment of this application;

[0038] Figure 8A structural block diagram of target tracking that combines appearance features and motion trajectory is provided in an embodiment of this application;

[0039] Figure 9 A block diagram of a target tracking device provided in an embodiment of this application;

[0040] Figure 10 This is an internal structural diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0042] Intelligent surveillance plays an important role in urban traffic, monitoring of key locations, and tracking of criminals. Among these, target tracking is the core research content of intelligent surveillance.

[0043] Existing target tracking methods match the target's appearance features between consecutive frames to track the target. Appearance features refer to the target's physical characteristics. For example, for a person, appearance features include information such as hair color, gender, hair length, height, and type of clothing.

[0044] However, existing target tracking methods often fail when the target is occluded or subjected to similar interference, resulting in low tracking reliability.

[0045] In view of this, embodiments of this application provide a target tracking method, apparatus, computer device, storage medium, and program product, which can improve the reliability of target tracking.

[0046] Please see Figure 1 This diagram illustrates the implementation environment involved in the target tracking method provided in the embodiments of this application. Figure 1 As shown, the execution entity of the target tracking method provided in this application embodiment can be a single computer device or a cluster of computer devices composed of multiple computer devices. Different computer devices can communicate with each other via wired or wireless means. Wireless communication can be achieved through WIFI, carrier networks, NFC (Near Field Communication), or other technologies.

[0047] Please see Figure 2 The diagram illustrates a flowchart of a target tracking method provided in an embodiment of this application, which can be applied to... Figure 1 In the computer device shown. For example... Figure 2As shown, the target tracking method may include the following steps:

[0048] Step 201: The computer device extracts the appearance features of the target to be tracked and determines the first candidate position of the target in the current frame based on the results of the appearance feature extraction.

[0049] Here, appearance features refer to the physical characteristics of the target. Optionally, the appearance features of the target to be tracked can be extracted using a fully convolutional neural network, or other neural networks. This application does not limit this, as long as the appearance features can be extracted.

[0050] Optionally, the computer device can determine the first candidate position of the target in the current frame based on the target's appearance features. For example, if the target is a red triangular box, its appearance features are red, triangle, and box. The computer device can determine the first candidate position of the target in the current frame based on these three appearance features.

[0051] Step 202: The computer device predicts the motion trajectory of the target and determines the second candidate position of the target in the current frame based on the result of the motion trajectory prediction.

[0052] Optionally, in addition to confirming the target's position through its apparent features, a second candidate position for the target in the current frame can be predicted based on the target's motion trajectory in historical frames preceding the current frame. Optionally, the current frame and historical frames are video frames, which can be acquired in real time by monitoring devices such as cameras.

[0053] Step 203: The computer device determines the target position in the current frame from the first candidate position and the second candidate position.

[0054] Optionally, since the target's trajectory may change abruptly while the target's apparent features do not change much, the first candidate position determined by the apparent features can be used as the primary position, and the second candidate position determined by the trajectory prediction can be used as the secondary position. That is, if the first candidate position is accurate, the first candidate position is used as the target position; if the first candidate position is inaccurate, the accuracy of the second candidate position is analyzed. If the second candidate position is accurate, the second candidate position is used as the target position.

[0055] The main reasons for inaccurate first candidate positions include: occlusion of the target in the current frame, making it impossible to extract or fully extract the target's appearance features. The main reason for inaccurate second candidate positions is a sudden change in the motion trajectory, making it impossible to accurately predict the target's trajectory.

[0056] In this embodiment, after extracting the apparent features of the target to be tracked and determining the first candidate position of the target in the current frame based on the results of the apparent feature extraction, the target's motion trajectory is also predicted, and a second candidate position of the target in the current frame is determined based on the results of the motion trajectory prediction. Finally, the target position in the current frame is determined from the first and second candidate positions. Therefore, this embodiment utilizes both apparent features and motion trajectories to track the target. This allows for improved tracking reliability, especially when the target is occluded or subject to similar interference.

[0057] In an optional embodiment of this application, the target tracking method can be applied to the financial sector, such as banking. When applied to banking facilities, the target tracking method may include the following steps:

[0058] First, acquire video frames captured by the target cameras deployed by the bank within a preset time period.

[0059] The video frames include the current frame and multiple historical frames that precede the current frame in time. The preset duration can be manually determined according to actual needs. For example, if the target to be tracked may enter the shooting range of the target camera between 10:00 and 10:15 on January 20, 2021, then only the video frames captured by the target camera during that time period can be acquired.

[0060] Secondly, the apparent features of the target are extracted from a historical frame that includes the target to be tracked, and the first candidate position of the target in the current frame is determined based on the apparent features.

[0061] In this optional embodiment, it is necessary to obtain the target position in the current frame. To do this, the appearance features of the target can be extracted from a historical frame that includes the target. Based on these appearance features, the first candidate position of the target in the current frame can be determined. For example, if the target is a red triangular box, to determine the target position of the box in the current frame, the appearance features of the box are first extracted from a historical frame that includes the box, namely, red, triangle, and box. Then, the first candidate position of the box in the current frame is determined based on these three appearance features.

[0062] Next, based on the target's motion trajectory in historical frames, the target's motion trajectory is predicted, and the second candidate position of the target in the current frame is determined based on the motion trajectory prediction result.

[0063] Finally, the target position in the current frame is determined from the first candidate position and the second candidate position, that is, the target position in the bank is determined.

[0064] The first candidate position determined based on appearance features and the second candidate position predicted based on motion trajectory are not necessarily the target position in the current frame. This is because the target may be occluded in the current frame, causing the first candidate position determined based on appearance features to be inaccurate, or the target's motion trajectory may have a sudden change, causing the second candidate position predicted based on the motion trajectory to be inaccurate. Therefore, it is necessary to judge the accuracy of the first and second candidate positions. If the first candidate position is accurate, it is used as the target position in the current frame; if the first candidate position is inaccurate, the accuracy of the second candidate position is then judged. If the second candidate position is accurate, it is used as the target position in the current frame.

[0065] For example, in real life, banks, as high-risk locations, are key surveillance sites requiring real-time monitoring via cameras. In scenarios such as bank theft, it's necessary to track perpetrators based on the surveillance video captured by these cameras. Existing technologies typically determine a perpetrator's position in the current frame based on their physical characteristics in previous frames. However, if the perpetrator is obscured or a similar perpetrator is present in the current frame, the position cannot be determined, leading to tracking failure. In this embodiment, in addition to determining the perpetrator's position based on their physical characteristics, the perpetrator's position in the current frame can also be predicted using their motion trajectory in multiple previous frames. This motion trajectory-based prediction of the perpetrator's position in the current frame is unaffected by obscuration or the presence of similar perpetrators. Because the trajectory of motion may change abruptly, while the appearance of the actor does not change much, if there is no situation where the actor is occluded or there is a similar actor, the actor's position in the current frame can be determined based on the actor's appearance. If there is a situation where the actor is occluded or there is a similar actor, the actor's position in the current frame can be predicted based on the actor's trajectory. Therefore, compared with the prior art, the embodiments of this application can still track the actor's position in the current frame even when the actor is occluded or there is a similar actor, thus improving the reliability of target tracking.

[0066] Please see Figure 3 This illustrates a technical process for determining a first candidate position provided by an embodiment of this application. For example... Figure 3 As shown, the technical process may include the following steps:

[0067] Step 301: The computer device obtains the template frame.

[0068] The template frame is a video frame that is in the sequence preceding the current frame and contains the target.

[0069] Step 302: The computer device inputs the template frame and the current frame into the template branch and the detection branch of the twin network, respectively, to obtain the first appearance feature output by the template branch and the second appearance feature output by the detection branch.

[0070] The Siamese network consists of two structurally identical, weight-sharing fully convolutional networks: one as the template branch and the other as the detection branch. Each fully convolutional network comprises five convolutional layers and two pooling layers. Optionally, the template frame and the current frame are input into the template and detection branches of the Siamese network, respectively. This includes: cropping the template frame based on the center of the bounding box in the template frame to obtain a first image patch; cropping the current frame based on the center of the bounding box in the previous frame to obtain a second image patch; and inputting the first and second image patches into the template and detection branches, respectively. The bounding box is used to define the target to be tracked. Furthermore, during the cropping process, if the target exceeds the boundaries of the frame image, the average color of the frame image is used as fill.

[0071] Optionally, the size of the first image patch can be M*M, and the size of the second image patch can be 2M*2M, where M is calculated based on the width and height of the bounding box. The formula for calculating M is as follows:

[0072] (w+p)×(h+p)=M 2

[0073] Where w and h are the width and height of the bounding box, respectively, and p = (w + h) / 2.

[0074] Optionally, a first image patch of size M*M can be scaled to 127×127, and a second image patch of size 2M*2M can be scaled to 255×255. These 127×127 and 255×255 image patches are then input into the template branch and detection branch of the Siamese network, respectively, to obtain the first appearance feature output by the template branch and the second appearance feature output by the detection branch. The feature extraction process within the template branch and detection branch is shown in the table below:

[0075]

[0076] In this embodiment of the application, a first image block is input into the template branch to obtain a first appearance feature output by the template branch. The first appearance feature is the appearance feature corresponding to the first image block. Since the first image block includes a target, the first appearance feature is also the appearance feature corresponding to the target. A second image block is input into the detection branch to obtain a second appearance feature output by the detection branch. The second appearance feature is the appearance feature corresponding to the second image block.

[0077] Step 303: The computer device performs a cross-correlation operation on the first apparent feature and the second apparent feature, and determines the first candidate position based on the result of the cross-correlation operation.

[0078] The cross-correlation operation reflects the similarity between the first and second appearance features. Since the first appearance feature is the appearance feature corresponding to the target, the first candidate position can be determined based on the cross-correlation operation result between the second and first appearance features corresponding to the current frame.

[0079] In an optional embodiment of this application, performing a cross-correlation operation on the first apparent feature and the second apparent feature, and determining the first candidate position based on the result of the cross-correlation operation, may include: performing a cross-correlation operation on the first apparent feature and the second apparent feature to obtain a first response map; converting the first response map into a second response map based on a bicubic interpolation algorithm, where the response values ​​in the second response map represent the similarity between the first apparent feature and the second apparent feature; and determining the first candidate position based on the second response map. Optionally, determining the first candidate position based on the second response map includes: obtaining the position information corresponding to the maximum response value in the second response map; and determining the first candidate position based on the position information.

[0080] The calculation formula for cross-correlation is as follows:

[0081]

[0082] in, This represents the cross-correlation operation, where r is the result of the cross-correlation operation, and represents the first response plot. For the first appearance feature, This is the second appearance feature.

[0083] For example, image patches of size 127×127 and 255×255 are input into the template branch and detection branch of the Siamese network, respectively. The first appearance feature output by the template branch has a size of 6×6, and the second appearance feature output by the detection branch has a size of 22×22. The first appearance feature and the second appearance feature are cross-correlated to obtain a first response map r with a size of 17×17. Then, based on the bicubic interpolation algorithm, the first response map of size 17×17 is transformed into a second response map of size 272×272, and the maximum response value r is obtained. app The corresponding location information, based on which the first candidate position of the target in the current frame can be determined as l. app =(x app ,y app ).

[0084] In this embodiment of the application, cropping the template frame and the current frame respectively and then extracting the appearance features of the cropped image blocks can reduce the computational load of the computer device and improve the speed of appearance feature extraction.

[0085] Please see Figure 4 This illustrates a technical process for determining a second candidate position provided by an embodiment of this application. For example... Figure 4 As shown, the technical process may include the following steps:

[0086] Step 401: The computer device acquires multiple historical frames, which are time-series frames preceding the current frame and contain video frames of the target.

[0087] Step 402: The computer device inputs the position coordinates of the target in multiple historical frames into the Long Short-Term Memory network in chronological order, and determines the second candidate position based on the output of the Long Short-Term Memory network.

[0088] The input to the Long Short-Term Memory (LSTM) network is the target's position coordinates, hidden state, and memory unit. The output is the hidden state and memory unit. The output of the previous time step is used as the input of the next time step, and the calculation is repeated until the final output result is obtained.

[0089] In an optional embodiment of this application, inputting the position coordinates of the target in multiple historical frames sequentially into a Long Short-Term Memory (LSTM) network, and determining the second candidate position based on the output of the LSTM network, may include: inputting the position coordinates of the target in multiple historical frames sequentially into the LSTM network to obtain a first hidden state output by the LSTM network; inputting the first hidden state and the position coordinates of the target in the previous frame of the current frame into the LSTM network to obtain a second hidden state output by the LSTM network, and determining the second candidate position based on the second hidden state.

[0090] For example, the position coordinates X = {(x_i, x ... t ,y t )}, where t=t last-7 ,……,t last , t last This means that in the frame preceding the current frame, the eight position coordinates of the target are input into the Long Short-Term Memory (LSTM) network in chronological order to obtain the first hidden state output by the LTM network. Then, the first hidden state and the target's position coordinates from the previous frame are compared... The input is fed into the Long Short-Term Memory (LSTM) network to obtain the second hidden state output by the LSM network, and the second candidate position is determined as l based on the second hidden state. mot =(x mot ,y mot ).

[0091] In this embodiment, the second candidate position of the target in the current frame can be predicted by the position coordinates of the target in multiple historical frames, which makes up for the inaccuracy of taking the first candidate position as the target position and improves the reliability of target tracking.

[0092] The first candidate position and the second candidate position of the target in the current frame were determined above. In this embodiment, the first candidate position and the second candidate position are fused to obtain the target position of the target in the current frame. Please refer to [link to relevant documentation]. Figure 5 This illustrates a technical process for determining a target location provided by an embodiment of this application. For example... Figure 5 As shown, the technical process may include the following steps:

[0093] Step 501: The computer device calculates the average overlap accuracy between the current frame and the previous frame.

[0094] The average overlap accuracy is the intersection-union ratio (IUU) of the areas of the bounding boxes in the current frame and the bounding boxes in the previous frame. Optionally, calculating the average overlap accuracy between the current frame and the previous frame includes: calculating the intersection area of ​​the bounding boxes in the current frame and the bounding boxes in the previous frame; calculating the union area of ​​the bounding boxes in the current frame and the bounding boxes in the previous frame; and calculating the ratio of the intersection area to the union area to obtain the average overlap accuracy. The formula for calculating the average overlap accuracy is as follows:

[0095]

[0096] Where IOU is the average overlap precision, B curr B represents the bounding box region in the current frame. last This refers to the bounding box region in the previous frame.

[0097] To clearly explain the average overlap accuracy, we can use... Figure 6 denoted by , where |·| represents the area of ​​the region.

[0098] Step 502: If the maximum response value is not less than the first preset threshold or the average overlap accuracy is not less than the second preset threshold, then the first candidate position is determined as the target position in the current frame.

[0099] Step 503: If the maximum response value is less than the first preset threshold and the average overlap accuracy is less than the second preset threshold, then the second candidate position is determined as the target position in the current frame.

[0100] The first and second preset thresholds can be values ​​obtained based on a large amount of experimental data and experiments conducted on a certain platform. In this embodiment, the first and second preset thresholds can be fixed values. It should be noted that a lower response value indicates a lower probability of a correct tracking result, and a lower average overlap accuracy also indicates a lower probability of a correct tracking result. The formula for fusing the first and second candidate positions to determine the target position is as follows:

[0101]

[0102] Where, r app For the maximum response value, th r For the first preset threshold, th I Based on the second preset threshold, l is the target position, l app As the first candidate position, l mot The second candidate position; if r app <th r ,IOU <th I Then l mot As the target's location in the current frame; otherwise, use l app As the target's location in the current frame.

[0103] In this embodiment, the obtained first candidate position and second candidate position are fused, and the target position in the current frame is determined based on the joint result of the first candidate position and the second candidate position, thereby improving the reliability of target tracking.

[0104] Please see Figure 7 The diagram illustrates a flowchart of a target tracking method provided in an embodiment of this application, which can be applied to... Figure 1 In the computer device shown. For example... Figure 7 As shown, the target tracking method may include the following steps:

[0105] Step 701: The computer device acquires a template frame. The template frame is a video frame that is in the sequence before the current frame and contains the target video frame.

[0106] Step 702: The computer device crops the template frame based on the center of the bounding box in the template frame to obtain the first image block. The bounding box is used to outline the target to be tracked.

[0107] Step 703: The computer device crops the current frame based on the center of the bounding box in the previous frame to obtain a second image block.

[0108] Step 704: The computer device inputs the first image block and the second image block into the template branch and the detection branch respectively, and obtains the first appearance feature output by the template branch and the second appearance feature output by the detection branch.

[0109] Step 705: The computer device performs a cross-correlation operation on the first apparent feature and the second apparent feature to obtain the first response map.

[0110] Step 706: The computer device converts the first response map into a second response map based on the bicubic interpolation algorithm. The response values ​​in the second response map represent the similarity between the first and second apparent features.

[0111] Step 707: The computer device obtains the location information corresponding to the maximum response value in the second response map, and determines the first candidate location based on the location information.

[0112] Step 708: The computer device acquires multiple historical frames, which are time-series frames preceding the current frame and contain video frames of the target.

[0113] Step 709: The computer device inputs the position coordinates of the target in multiple historical frames into the Long Short-Term Memory network in chronological order to obtain the first hidden state output by the Long Short-Term Memory network.

[0114] Step 710: The computer device inputs the first hidden state and the target's position coordinates in the previous frame of the current frame into the Long Short-Term Memory network to obtain the second hidden state output by the Long Short-Term Memory network, and determines the second candidate position based on the second hidden state.

[0115] Step 711: The computer device determines the target position in the current frame from the first candidate position and the second candidate position.

[0116] Please see Figure 8 The diagram shown is a structural block diagram of a target tracking method that combines appearance features with motion trajectory, according to an embodiment of this application. Figure 8 As shown, image preprocessing is performed on the template frame and the current frame. Image preprocessing refers to cropping the template frame and the current frame, and then inputting the cropped image patches into the template branch and detection branch of the Siamese network, respectively. This yields the first appearance feature output by the template branch and the second appearance feature output by the detection branch. Next, cross-correlation is performed on the first and second appearance features to obtain a response map. Based on the position information corresponding to the maximum response value in the response map, the first candidate position l of the target in the current frame is determined. app Furthermore, the Long Short-Term Memory network determines the second candidate position of the target in the current frame based on the target's trajectory in historical frames. motThen, based on the combined result of the first candidate position and the second candidate position, the target position l in the current frame is determined.

[0117] In addition to using appearance features to track targets, this application embodiment also uses motion trajectories to track targets. Thus, when the target is occluded or there are similar interference problems, motion trajectories can be used to assist in tracking, improving the reliability of target tracking.

[0118] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0119] Please see Figure 9 The diagram illustrates a block diagram of a target tracking device 900 provided in an embodiment of this application, which can be configured in the aforementioned computer device. Figure 9 As shown, the target tracking device 900 includes a first determining module 901, a second determining module 902, and a third determining module 903.

[0120] The first determining module 901 is used to extract the appearance features of the target to be tracked and determine the first candidate position of the target in the current frame based on the result of the appearance feature extraction; the second determining module 902 is used to predict the motion trajectory of the target and determine the second candidate position of the target in the current frame based on the result of the motion trajectory prediction; the third determining module 903 is used to determine the target position of the target in the current frame from the first candidate position and the second candidate position.

[0121] In one embodiment, the first determining module 901 is specifically used to: obtain a template frame, wherein the template frame is a video frame that is in time sequence before the current frame and contains the target; input the template frame and the current frame into the template branch and the detection branch of the Siamese network respectively to obtain a first appearance feature output by the template branch and a second appearance feature output by the detection branch; perform a cross-correlation operation on the first appearance feature and the second appearance feature, and determine a first candidate position based on the result of the cross-correlation operation.

[0122] In one embodiment, the first determining module 901 is specifically used to: crop the template frame based on the center of the bounding box in the template frame to obtain a first image block, wherein the bounding box is used to outline the target to be tracked; crop the current frame based on the center of the bounding box in the previous frame to obtain a second image block; and input the first image block and the second image block into the template branch and the detection branch, respectively.

[0123] In one embodiment, the size of the first image block is M*M, and the size of the second image block is 2M*2M, where M is calculated based on the width and height of the bounding box.

[0124] In one embodiment, the first determining module 901 is specifically used for: performing a cross-correlation operation on the first apparent feature and the second apparent feature to obtain a first response map; converting the first response map into a second response map based on a bicubic interpolation algorithm, wherein the response values ​​in the second response map represent the magnitude of the similarity between the first apparent feature and the second apparent feature; and determining a first candidate position based on the second response map.

[0125] In one embodiment, the first determining module 901 is specifically used to: obtain the location information corresponding to the maximum response value in the second response map; and determine the first candidate location based on the location information.

[0126] In one embodiment, the second determining module 902 is specifically used to: acquire multiple historical frames, wherein the historical frames are video frames that are in time sequence before the current frame and contain the target; input the position coordinates of the target in the multiple historical frames into the Long Short-Term Memory network in chronological order; and determine the second candidate position based on the output of the Long Short-Term Memory network.

[0127] In one embodiment, the second determining module 902 is specifically used to: input the position coordinates of the target in multiple historical frames into the Long Short-Term Memory (LSTM) network in chronological order to obtain a first hidden state output by the LSTM network; input the first hidden state and the position coordinates of the target in the previous frame of the current frame into the LSTM network to obtain a second hidden state output by the LSTM network; and determine a second candidate position based on the second hidden state.

[0128] In one embodiment, the third determining module 903 is specifically configured to: calculate the average overlap accuracy between the current frame and the previous frame, wherein the average overlap accuracy is the intersection-union ratio of the areas of the bounding boxes in the current frame and the bounding boxes in the previous frame; if the maximum response value is not less than a first preset threshold or the average overlap accuracy is not less than a second preset threshold, then determine the first candidate position as the target position of the target in the current frame; if the maximum response value is less than the first preset threshold and the average overlap accuracy is less than the second preset threshold, then determine the second candidate position as the target position of the target in the current frame.

[0129] In one embodiment, the third determining module 903 is specifically used to: calculate the intersection area of ​​the bounding box in the current frame and the bounding box in the previous frame; calculate the union area of ​​the bounding box in the current frame and the bounding box in the previous frame; calculate the ratio of the intersection area to the union area to obtain the average overlap accuracy.

[0130] The target tracking device provided in this application embodiment can implement the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0131] Each module in the aforementioned target tracking device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0132] In one embodiment, a computer device is provided, the internal structure of which can be shown as follows: Figure 10 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a target tracking method. The display screen can be an LCD screen or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0133] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0134] In one embodiment of this application, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps: extracting appearance features of a target to be tracked, and determining a first candidate position of the target in the current frame based on the result of the appearance feature extraction; predicting the motion trajectory of the target, and determining a second candidate position of the target in the current frame based on the result of the motion trajectory prediction; and determining the target position of the target in the current frame from the first candidate position and the second candidate position.

[0135] In one embodiment of this application, when the processor executes the computer program, it further implements the following steps: obtaining a template frame, wherein the template frame is a video frame that is time-series preceding the current frame and contains the target; inputting the template frame and the current frame into the template branch and the detection branch of the Siamese network respectively to obtain a first appearance feature output by the template branch and a second appearance feature output by the detection branch; performing a cross-correlation operation on the first appearance feature and the second appearance feature, and determining a first candidate position based on the result of the cross-correlation operation.

[0136] In one embodiment of this application, when the processor executes the computer program, it further implements the following steps: cropping the template frame based on the center of the bounding box in the template frame to obtain a first image block, wherein the bounding box is used to outline the target to be tracked; cropping the current frame based on the center of the bounding box in the previous frame to obtain a second image block; and inputting the first image block and the second image block into the template branch and the detection branch, respectively.

[0137] In one embodiment of this application, the size of the first image block is M*M, and the size of the second image block is 2M*2M, where M is calculated based on the width and height of the bounding box.

[0138] In one embodiment of this application, when the processor executes the computer program, it further performs the following steps: performing a cross-correlation operation on the first apparent feature and the second apparent feature to obtain a first response map; converting the first response map into a second response map based on a bicubic interpolation algorithm, wherein the response value in the second response map represents the magnitude of the similarity between the first apparent feature and the second apparent feature; and determining a first candidate position based on the second response map.

[0139] In one embodiment of this application, when the processor executes the computer program, it further implements the following steps: obtaining the location information corresponding to the maximum response value in the second response map; and determining a first candidate location based on the location information.

[0140] In one embodiment of this application, when the processor executes the computer program, it further implements the following steps: acquiring multiple historical frames, wherein the historical frames are video frames that are in time sequence before the current frame and contain the target; inputting the position coordinates of the target in the multiple historical frames into the Long Short-Term Memory network in chronological order; and determining a second candidate position based on the output of the Long Short-Term Memory network.

[0141] In one embodiment of this application, when the processor executes the computer program, it further implements the following steps: inputting the position coordinates of the target in multiple historical frames into the Long Short-Term Memory (LSTM) network in chronological order to obtain a first hidden state output by the LSTM network; inputting the first hidden state and the position coordinates of the target in the previous frame of the current frame into the LSTM network to obtain a second hidden state output by the LSTM network; and determining a second candidate position based on the second hidden state.

[0142] In one embodiment of this application, when the processor executes the computer program, it further implements the following steps: calculating the average overlap accuracy between the current frame and the previous frame, wherein the average overlap accuracy is the intersection-union ratio of the areas of the bounding boxes in the current frame and the bounding boxes in the previous frame; if the maximum response value is not less than a first preset threshold or the average overlap accuracy is not less than a second preset threshold, then determining the first candidate position as the target position of the target in the current frame; if the maximum response value is less than the first preset threshold and the average overlap accuracy is less than the second preset threshold, then determining the second candidate position as the target position of the target in the current frame.

[0143] In one embodiment of this application, when the processor executes the computer program, it further implements the following steps: calculating the intersection area of ​​the bounding box in the current frame and the bounding box in the previous frame; calculating the union area of ​​the bounding box in the current frame and the bounding box in the previous frame; and calculating the ratio of the intersection area to the union area to obtain the average overlap accuracy.

[0144] The computer device provided in this application embodiment has a similar implementation principle and technical effect to the above method embodiment, and will not be described again here.

[0145] In one embodiment of this application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it performs the following steps: extracting appearance features of the target to be tracked, and determining a first candidate position of the target in the current frame based on the result of the appearance feature extraction; predicting the motion trajectory of the target, and determining a second candidate position of the target in the current frame based on the result of the motion trajectory prediction; and determining the target position of the target in the current frame from the first candidate position and the second candidate position.

[0146] In one embodiment of this application, when the computer program is executed by the processor, it further implements the following steps: obtaining a template frame, wherein the template frame is a video frame that is in time sequence before the current frame and contains the target; inputting the template frame and the current frame into the template branch and the detection branch of the Siamese network respectively to obtain a first appearance feature output by the template branch and a second appearance feature output by the detection branch; performing a cross-correlation operation on the first appearance feature and the second appearance feature, and determining a first candidate position based on the result of the cross-correlation operation.

[0147] In one embodiment of this application, when the computer program is executed by the processor, it further implements the following steps: cropping the template frame based on the center of the bounding box in the template frame to obtain a first image block, wherein the bounding box is used to outline the target to be tracked; cropping the current frame based on the center of the bounding box in the previous frame to obtain a second image block; and inputting the first image block and the second image block into the template branch and the detection branch, respectively.

[0148] In one embodiment of this application, the size of the first image block is M*M, and the size of the second image block is 2M*2M, where M is calculated based on the width and height of the bounding box.

[0149] In one embodiment of this application, when the computer program is executed by the processor, it further performs the following steps: performing a cross-correlation operation on the first apparent feature and the second apparent feature to obtain a first response map; converting the first response map into a second response map based on a bicubic interpolation algorithm, wherein the response value in the second response map represents the magnitude of the similarity between the first apparent feature and the second apparent feature; and determining a first candidate position based on the second response map.

[0150] In one embodiment of this application, when the computer program is executed by the processor, it further performs the following steps: obtaining the location information corresponding to the maximum response value in the second response map; and determining a first candidate location based on the location information.

[0151] In one embodiment of this application, when the computer program is executed by the processor, it further implements the following steps: acquiring multiple historical frames, wherein the historical frames are video frames that are in time sequence before the current frame and contain the target; inputting the position coordinates of the target in the multiple historical frames into the Long Short-Term Memory network in time sequence; and determining a second candidate position based on the output of the Long Short-Term Memory network.

[0152] In one embodiment of this application, when the computer program is executed by the processor, it further implements the following steps: inputting the position coordinates of the target in multiple historical frames into the Long Short-Term Memory (LSTM) network in chronological order to obtain a first hidden state output by the LSTM network; inputting the first hidden state and the position coordinates of the target in the previous frame of the current frame into the LSTM network to obtain a second hidden state output by the LSTM network; and determining a second candidate position based on the second hidden state.

[0153] In one embodiment of this application, when the computer program is executed by the processor, it further implements the following steps: calculating the average overlap accuracy between the current frame and the previous frame, wherein the average overlap accuracy is the intersection-union ratio of the areas of the bounding boxes in the current frame and the bounding boxes in the previous frame; if the maximum response value is not less than a first preset threshold or the average overlap accuracy is not less than a second preset threshold, then determining the first candidate position as the target position of the target in the current frame; if the maximum response value is less than the first preset threshold and the average overlap accuracy is less than the second preset threshold, then determining the second candidate position as the target position of the target in the current frame.

[0154] In one embodiment of this application, when the computer program is executed by the processor, it further performs the following steps: calculating the intersection area of ​​the bounding box in the current frame and the bounding box in the previous frame; calculating the union area of ​​the bounding box in the current frame and the bounding box in the previous frame; and calculating the ratio of the intersection area to the union area to obtain the average overlap accuracy.

[0155] The computer-readable storage medium provided in this embodiment is similar in principle and technical effect to the method embodiment described above, and will not be repeated here.

[0156] In one embodiment of this application, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps: extracting appearance features of a target to be tracked, and determining a first candidate position of the target in the current frame based on the result of the appearance feature extraction; predicting the motion trajectory of the target, and determining a second candidate position of the target in the current frame based on the result of the motion trajectory prediction; and determining the target position of the target in the current frame from the first candidate position and the second candidate position.

[0157] In one embodiment of this application, when the computer program is executed by the processor, it further implements the following steps: obtaining a template frame, wherein the template frame is a video frame that is in time sequence before the current frame and contains the target; inputting the template frame and the current frame into the template branch and the detection branch of the Siamese network respectively to obtain a first appearance feature output by the template branch and a second appearance feature output by the detection branch; performing a cross-correlation operation on the first appearance feature and the second appearance feature, and determining a first candidate position based on the result of the cross-correlation operation.

[0158] In one embodiment of this application, when the computer program is executed by the processor, it further implements the following steps: cropping the template frame based on the center of the bounding box in the template frame to obtain a first image block, wherein the bounding box is used to outline the target to be tracked; cropping the current frame based on the center of the bounding box in the previous frame to obtain a second image block; and inputting the first image block and the second image block into the template branch and the detection branch, respectively.

[0159] In one embodiment of this application, the size of the first image block is M*M, and the size of the second image block is 2M*2M, where M is calculated based on the width and height of the bounding box.

[0160] In one embodiment of this application, when the computer program is executed by the processor, it further performs the following steps: performing a cross-correlation operation on the first apparent feature and the second apparent feature to obtain a first response map; converting the first response map into a second response map based on a bicubic interpolation algorithm, wherein the response value in the second response map represents the magnitude of the similarity between the first apparent feature and the second apparent feature; and determining a first candidate position based on the second response map.

[0161] In one embodiment of this application, when the computer program is executed by the processor, it further performs the following steps: obtaining the location information corresponding to the maximum response value in the second response map; and determining a first candidate location based on the location information.

[0162] In one embodiment of this application, when the computer program is executed by the processor, it further implements the following steps: acquiring multiple historical frames, wherein the historical frames are video frames that are in time sequence before the current frame and contain the target; inputting the position coordinates of the target in the multiple historical frames into the Long Short-Term Memory network in time sequence; and determining a second candidate position based on the output of the Long Short-Term Memory network.

[0163] In one embodiment of this application, when the computer program is executed by the processor, it further implements the following steps: inputting the position coordinates of the target in multiple historical frames into the Long Short-Term Memory (LSTM) network in chronological order to obtain a first hidden state output by the LSTM network; inputting the first hidden state and the position coordinates of the target in the previous frame of the current frame into the LSTM network to obtain a second hidden state output by the LSTM network; and determining a second candidate position based on the second hidden state.

[0164] In one embodiment of this application, when the computer program is executed by the processor, it further implements the following steps: calculating the average overlap accuracy between the current frame and the previous frame, wherein the average overlap accuracy is the intersection-union ratio of the areas of the bounding boxes in the current frame and the bounding boxes in the previous frame; if the maximum response value is not less than a first preset threshold or the average overlap accuracy is not less than a second preset threshold, then determining the first candidate position as the target position of the target in the current frame; if the maximum response value is less than the first preset threshold and the average overlap accuracy is less than the second preset threshold, then determining the second candidate position as the target position of the target in the current frame.

[0165] In one embodiment of this application, when the computer program is executed by the processor, it further performs the following steps: calculating the intersection area of ​​the bounding box in the current frame and the bounding box in the previous frame; calculating the union area of ​​the bounding box in the current frame and the bounding box in the previous frame; and calculating the ratio of the intersection area to the union area to obtain the average overlap accuracy.

[0166] The computer program product provided in this embodiment has a similar implementation principle and technical effect to the method embodiment described above, and will not be repeated here.

[0167] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0168] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0169] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A target tracking method, characterized in that, The method includes: The apparent features of the target to be tracked are extracted, and the first candidate position of the target in the current frame is determined based on the results of the apparent feature extraction. The motion trajectory of the target is predicted, and a second candidate position of the target in the current frame is determined based on the result of the motion trajectory prediction. The target position of the target in the current frame is determined from the first candidate position and the second candidate position; The step of extracting appearance features from the target to be tracked and determining the first candidate position of the target in the current frame based on the result of the appearance feature extraction includes: Obtain a template frame, wherein the template frame is a video frame that is sequentially preceding the current frame and contains the target video frame; The template frame and the current frame are respectively input into the template branch and the detection branch of the Siamese network to obtain the first appearance feature output by the template branch and the second appearance feature output by the detection branch; A first response map is obtained by performing a cross-correlation operation on the first apparent feature and the second apparent feature. The first response map is converted into a second response map based on the bicubic interpolation algorithm, and the response value in the second response map represents the similarity between the first appearance feature and the second appearance feature. Obtain the location information corresponding to the maximum response value in the second response graph; Based on the location information, the first candidate location is determined; Determining the target position of the target in the current frame from the first candidate position and the second candidate position includes: Calculate the average overlap accuracy between the current frame and the previous frame, where the average overlap accuracy is the intersection-union ratio of the areas of the bounding box in the current frame corresponding to the first candidate position and the bounding box in the previous frame. If the maximum response value is not less than the first preset threshold or the average overlap accuracy is not less than the second preset threshold, then the first candidate position is determined as the target position of the target in the current frame; If the maximum response value is less than the first preset threshold and the average overlap accuracy is less than the second preset threshold, then the second candidate position is determined as the target position of the target in the current frame.

2. The method according to claim 1, characterized in that, The step of inputting the template frame and the current frame into the template branch and detection branch of the Siamese network, respectively, includes: Based on the center of the bounding box in the template frame, the template frame is cropped to obtain a first image block, wherein the bounding box is used to outline the target to be tracked. Based on the center of the bounding box in the previous frame of the current frame, the current frame is cropped to obtain a second image patch; The first image block and the second image block are respectively input into the template branch and the detection branch.

3. The method according to claim 2, characterized in that, The size of the first image block is M M, the size of the second image block is 2M 2M, where M is calculated based on the width and height of the bounding box.

4. The method according to claim 1, characterized in that, The step of predicting the motion trajectory of the target and determining the second candidate position of the target in the current frame based on the result of the motion trajectory prediction includes: Acquire multiple historical frames, wherein the historical frames are those that are in time sequence before the current frame, and contain video frames of the target; The location coordinates of the target in the multiple historical frames are input into the Long Short-Term Memory network in chronological order, and the second candidate position is determined based on the output of the Long Short-Term Memory network.

5. The method according to claim 4, characterized in that, The step of inputting the target's position coordinates from the multiple historical frames into a Long Short-Term Memory (LSTM) network in chronological order, and determining the second candidate position based on the output of the LSTM network, includes: The position coordinates of the target in the multiple historical frames are input into the Long Short-Term Memory network in chronological order to obtain the first hidden state output by the Long Short-Term Memory network. The first hidden state and the position coordinates of the target in the previous frame of the current frame are input into the Long Short-Term Memory network to obtain the second hidden state output by the Long Short-Term Memory network. The second candidate position is determined based on the second hidden state.

6. The method according to claim 1, characterized in that, The calculation of the average overlap accuracy between the current frame and the previous frame includes: Calculate the intersection area of ​​the bounding box in the current frame and the bounding box in the previous frame; Calculate the area of ​​the union of the bounding boxes in the current frame and the bounding boxes in the previous frame; The average overlap accuracy is obtained by calculating the ratio of the intersection area to the union area.

7. A target tracking device, characterized in that, The device includes: The first determining module is used to extract the appearance features of the target to be tracked, and determine the first candidate position of the target in the current frame based on the result of the appearance feature extraction. The second determining module is used to predict the motion trajectory of the target and determine the second candidate position of the target in the current frame based on the result of the motion trajectory prediction. The third determining module is used to determine the target position of the target in the current frame from the first candidate position and the second candidate position; The first determining module is specifically used to acquire a template frame, wherein the template frame is a video frame that is sequentially preceding the current frame and contains the target; input the template frame and the current frame into the template branch and the detection branch of the Siamese network, respectively, to obtain a first appearance feature output by the template branch and a second appearance feature output by the detection branch; perform a cross-correlation operation on the first appearance feature and the second appearance feature to obtain a first response map; convert the first response map into a second response map based on a bicubic interpolation algorithm, wherein the response value in the second response map represents the similarity between the first appearance feature and the second appearance feature; acquire the location information corresponding to the maximum response value in the second response map; and determine the first candidate location based on the location information. The third determining module is specifically used to calculate the average overlap accuracy between the current frame and the previous frame, wherein the average overlap accuracy is the intersection-union ratio of the areas of the bounding box in the current frame corresponding to the first candidate position and the bounding box in the previous frame; if the maximum response value is not less than a first preset threshold or the average overlap accuracy is not less than a second preset threshold, then the first candidate position is determined to be the target position of the target in the current frame; if the maximum response value is less than the first preset threshold and the average overlap accuracy is less than the second preset threshold, then the second candidate position is determined to be the target position of the target in the current frame.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images