Target tracking method and device and computer equipment

Abstract

The invention relates to a target tracking method and device and computer equipment. The method comprises the steps of obtaining current candidate frame information of a plurality of current candidate frames in a current image frame based on the current image frame and previous tracking frame information of a target in a previous image frame adjacent to the current image frame, and performing trajectory prediction according to the previous candidate frame information of the plurality of candidate frames in the previous image frame, determining prediction candidate frame information corresponding to each piece of previous candidate frame information in the current image frame, performing matching processing on each piece of current candidate frame information and each piece of prediction candidate frame information to obtain a matching result, obtaining a newest candidate frame information set according to the matching result, performing matching processing according to the newest candidate frame information set and the tracking frame information, and obtaining a tracking frame information set; and determining current tracking frame information of the target in the current image frame. By adopting the method, the accuracy and robustness of a target tracking result can be improved in a target tracking process.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a target tracking method, device and computer equipment. Background Art

[0002] With the continuous development of science and technology, target tracking technology has been widely used in video surveillance, video annotation, monitoring security, unmanned driving, human-computer interaction, virtual reality, human-computer interaction and other fields.

[0003] In the related art, the target selected in the first image frame is mainly used as the tracking object, and a tracking algorithm based on a deep neural network is used to obtain the position and size of the tracking object in multiple subsequent consecutive image frames to achieve target tracking.

[0004] However, in the process of using tracking algorithms to track targets in related technologies, there is a problem of low accuracy of target tracking results. Summary of the invention

[0005] Based on this, it is necessary to provide a target tracking method, device and computer equipment to address the above technical problems, which can improve the accuracy of target tracking results during the target tracking process.

[0006] In a first aspect, an embodiment of the present application provides a target tracking method, the method comprising:

[0007] Based on the previous tracking frame information of the target in the current image frame and the previous image frame, current candidate frame information of multiple current candidate frames in the current image frame is obtained; and trajectory prediction is performed according to the previous candidate frame information of the multiple candidate frames in the previous image frame to determine the predicted candidate frame information corresponding to each previous candidate frame information in the current image frame; the current image frame is the next adjacent image frame of the previous image frame;

[0008] Matching each current candidate box information with each predicted candidate box information to obtain a matching result, and obtaining the latest candidate box information set according to the matching result;

[0009] Matching is performed based on the latest candidate frame information set and the tracking frame information to determine the current tracking frame information of the target in the current image frame.

[0010] In one of the embodiments, the previous tracking frame information includes the tracking frame size and tracking frame position of the target in the previous image frame; based on the tracking frame information of the target in the current image frame and the previous image frame, obtaining current candidate frame information of multiple current candidate frames in the current image frame includes:

[0011] The current image frame, the target frame size and the tracking frame position are input into the pre-built filter to obtain a response map corresponding to the search area and the target area in the current image frame; the target frame size is determined according to the tracking frame size;

[0012] Determine the current candidate box positions of the multiple current candidate boxes in the current image frame according to the multiple response values ​​in the response map;

[0013] According to the position of each current candidate box and the size of the tracking box, the information of each current candidate box is determined.

[0014] In one embodiment, determining the current candidate box positions of multiple current candidate boxes in the current image frame according to multiple response values ​​in the response map includes:

[0015] Obtaining the mapping position of the response value greater than the preset threshold value in the current image frame;

[0016] Determine each mapping position as a position of a candidate frame to be processed corresponding to the candidate frame to be processed;

[0017] A non-maximum suppression method is adopted to screen multiple current candidate frames from each candidate frame to be processed according to the position of each candidate frame to be processed, and the current candidate frame position of each current candidate frame is determined according to the position of each candidate frame to be processed.

[0018] In one embodiment, the matching result includes the similarity between each current candidate box information and the corresponding predicted candidate box information; obtaining the latest candidate box information set according to the matching result includes:

[0019] If the similarity between each current candidate box information and the corresponding predicted candidate box information is greater than a preset similarity threshold, each current candidate box information is added to the candidate box information set to generate the latest candidate box information set.

[0020] In one embodiment, matching processing is performed based on the latest candidate frame information set and the tracking frame information to determine the current tracking frame information of the target in the current image frame, including:

[0021] For any current candidate box information in the latest candidate box information set, matching processing is performed on the current candidate box information and the tracking box information;

[0022] If the match is successful, then check whether there is tracking box information in the latest candidate box information set;

[0023] If it does not exist, the current candidate box information is determined as the current tracking box information of the target in the current image frame, and the tracking box information is deleted from the latest candidate box information set; if it exists, the latest candidate box information set is updated to obtain the latest candidate box information set.

[0024] In one embodiment, the method further comprises:

[0025] If each current candidate frame information in the latest candidate frame information set fails to match the tracking frame information or the current tracking frame information is not determined, the target in the current image frame is re-detected to obtain the current tracking frame information of the target in the current image frame.

[0026] In one embodiment, re-detecting the target in the current image frame to obtain current tracking frame information of the target in the current image frame includes:

[0027] Performing rasterization processing on the current image frame to obtain multiple raster area images;

[0028] For any grid region image, the grid region image is used as the current image frame, and the target tracking process is continued until the current tracking frame information of the target in the current image frame is determined.

[0029] In one embodiment, the method further comprises:

[0030] If the current tracking frame information of the target is not determined after the target in the current image frame is re-detected, the target tracking process is performed on the next adjacent image frame after the current image frame to track the target in the next adjacent image frame.

[0031] In a second aspect, an embodiment of the present application provides a target tracking device, the device comprising:

[0032] A determination module is used to obtain current candidate frame information of multiple current candidate frames in the current image frame based on previous tracking frame information of the target in the current image frame and the previous image frame; and perform trajectory prediction based on previous candidate frame information of multiple candidate frames in the previous image frame to determine predicted candidate frame information corresponding to each previous candidate frame information in the current image frame; the current image frame is the next adjacent image frame of the previous image frame;

[0033] A first matching module is used to match each current candidate box information with each predicted candidate box information to obtain a matching result, and obtain a latest candidate box information set according to the matching result;

[0034] The second matching module is used to perform matching processing based on the latest candidate frame information set and the tracking frame information to determine the current tracking frame information of the target in the current image frame.

[0035] In a third aspect, an embodiment of the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method of any one of the embodiments of the first aspect when executing the computer program.

[0036] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method of any embodiment in the above-mentioned first aspect are implemented.

[0037] In a fifth aspect, an embodiment of the present application further provides a computer program product, the computer program product comprising a computer program, which, when executed by a processor, implements the steps of the method of any embodiment of the first aspect above.

[0038] The embodiment of the present application provides a target tracking method, device and computer equipment, including: based on the current image frame and the previous tracking frame information of the target in the previous image frame adjacent to the current image frame, obtaining the current candidate frame information of multiple current candidate frames in the current image frame, and performing trajectory prediction according to the previous candidate frame information of multiple candidate frames in the previous image frame, determining the predicted candidate frame information corresponding to each previous candidate frame information in the current image frame, matching each current candidate frame information with each predicted candidate frame information to obtain a matching result, and obtaining the latest candidate frame information set according to the matching result, matching the latest candidate frame information set with the tracking frame information, and determining the current tracking frame information of the target in the current image frame. The above method is not limited to scenes without obstructions, so that the wide applicability of the target tracking method can be improved, and the accuracy and robustness of the target tracking results can be improved in the process of tracking the target; at the same time, the above method does not need to configure related additional equipment for shielding obstructions, and can directly track the target accurately, thereby reducing the implementation cost of the target tracking method, and does not require manual participation of users, which can reduce the human resources required to implement the target tracking process, thereby reducing the labor cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A diagram showing an application environment of a target tracking method in an embodiment;

[0040] Figure 2 A schematic diagram of a target tracking method according to an embodiment of the present invention;

[0041] Figure 3 is a flow chart of a target tracking method in another embodiment;

[0042] Figure 4 is a flow chart of a target tracking method in another embodiment;

[0043] Figure 5 is a flow chart of a target tracking method in another embodiment;

[0044] Figure 6 is a flow chart of a target tracking method in another embodiment;

[0045] Figure 7 A diagram showing the result of rasterization processing of a current image frame in one embodiment;

[0046] Figure 8 is a structural block diagram of a target tracking device in one embodiment;

[0047] Fig. 9 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0049] Target tracking technology is widely used in video surveillance, video annotation, surveillance security, unmanned driving, human-computer interaction, virtual reality, human-computer interaction and other fields. In the related art, the target selected in the first image frame is used as the tracking object, and the tracking algorithm based on deep neural network is used to obtain the position and size of the tracking object in subsequent consecutive multiple image frames to achieve target tracking. However, in the process of using tracking algorithms to track targets in the related art, there is a problem of low accuracy of target tracking results.

[0050] Based on this, an embodiment of the present application provides a target tracking method, which can improve the accuracy of target tracking results during the target tracking process.

[0051] The target tracking method provided in the embodiment of the present application can be applied to Figure 1 The application environment diagram shown. The application environment includes a target tracking system, which may include an image acquisition device and a computer device; the image acquisition device and the computer device may be connected by communication, and the communication method may be Wi-Fi, mobile network or Bluetooth connection, etc. Optionally, the image acquisition device may be a camera, a camera or an electronic device with a photo taking function; the computer device may be a personal computer, a desktop computer, a laptop computer, a tablet computer, an iPad, a smart watch, a smart bracelet and other devices. It should be noted here that the target tracking method can be applicable to scenarios where an obstruction suddenly obscures the target during target tracking. The following embodiment uses a computer device as the executor of the target tracking method to introduce the specific process of the target tracking method.

[0052] like Figure 2 FIG. 1 is a flow chart of a target tracking method provided in an embodiment of the present application, and the method can be implemented by the following steps:

[0053] S100, based on the current image frame and the previous tracking frame information of the target in the adjacent previous image frame of the current image frame, obtaining current candidate frame information of multiple current candidate frames in the current image frame; and, performing trajectory prediction based on the previous candidate frame information of the multiple candidate frames in the previous image frame, determining the predicted candidate frame information corresponding to each previous candidate frame information in the current image frame.

[0054] There may be multiple targets, but in the embodiment of the present application, tracking of a single target is taken as an example for explanation.

[0055] It should be noted here that the current image frame may contain a target or may not contain a target (the target is blocked by an occluder). At the same time, the adjacent previous image frame of the current image frame can be understood as the adjacent image frame captured by the image acquisition device before the current image frame is captured, and the acquisition time of the previous image frame is earlier than the acquisition time of the current image frame. In addition, the previous tracking frame information in the previous image frame can be understood as the tracking result corresponding to the target in the previous image frame determined after the target is tracked in the previous image frame.

[0056] Specifically, the computer device can obtain the current image frame captured by the image acquisition device, and then obtain the current candidate frame information of multiple current candidate frames in the current image frame based on the current image frame and the previous tracking frame information of the target in the previous image frame adjacent to the current image frame.

[0057] Optionally, each current candidate frame in the current image frame may contain a real target, or may contain a false target similar to the target, which is not limited in the embodiment of the present application. At the same time, the current candidate frame information of the current candidate frame may include the orientation information, corner point position, etc. of the current candidate frame.

[0058] In one embodiment, the computer device can pre-train an algorithm model, and then input the previous tracking frame information in the current image frame and the previous image frame into the algorithm model, and the algorithm model outputs the current candidate frame information of multiple current candidate frames in the current image frame.

[0059] In another embodiment, the computer device may map, convert, translate, and the like the previous tracking frame information in the current image frame and the previous image frame to obtain current candidate frame information of multiple current candidate frames in the current image frame.

[0060] At the same time, for any candidate box in the previous image frame, the computer device can use a trajectory prediction algorithm to perform trajectory prediction based on the previous candidate box information of the candidate box in the previous image frame to obtain the predicted candidate box information corresponding to the previous candidate box information in the current image frame.

[0061] Optionally, the trajectory prediction algorithm may be a convolutional neural network model, a Markov chain model, a particle filter algorithm, a long short-term memory network model, etc. It should be noted here that the previous candidate box information of the candidate box refers to the information of the candidate box.

[0062] In an embodiment of the present application, the computer device may use a Kalman filter algorithm to perform trajectory prediction based on the previous candidate frame information of the candidate frame in the previous image frame to obtain the predicted candidate frame information corresponding to the previous candidate frame information in the current image frame. The predicted candidate frame information may be understood as the trajectory of the previous candidate frame information in the current image frame.

[0063] S200 , performing matching processing on each current candidate box information and each predicted candidate box information to obtain a matching result, and obtaining a latest candidate box information set according to the matching result.

[0064] Specifically, the computer device may select part of the current candidate box information from the matching results to construct the latest candidate box information set. In another implementation, the matching results may be compared, analyzed, screened, etc. to obtain the latest candidate box information set.

[0065] In practical applications, for any current candidate box information, the computer device can use a matching algorithm to match the current candidate box information with the predicted candidate box information corresponding to the current candidate box information to obtain a matching result, and obtain the latest candidate box information set based on the matching result.

[0066] The matching algorithm may be a fuzzy matching algorithm, an exact matching algorithm, a K-nearest neighbor algorithm, a principal component analysis method, a regression matching algorithm, etc. In the embodiment of the present application, the matching algorithm is a Hungarian algorithm.

[0067] In one embodiment, the above-mentioned matching results include the similarity between each current candidate box information and the corresponding predicted candidate box information; the step of obtaining the latest candidate box information set based on the matching results in the above-mentioned S200 may include: if the similarity between each current candidate box information and the corresponding predicted candidate box information is greater than a preset similarity threshold, then each current candidate box information is added to the candidate box information set to generate the latest candidate box information set.

[0068] Specifically, for any current candidate box information, if the similarity between the current candidate box information and the corresponding predicted candidate box information is greater than a preset similarity threshold, the current candidate box information can be added to the candidate box information set. Until all the filtered current candidate box information is added to the candidate box information set, the latest candidate box information set is generated.

[0069] Optionally, the similarity threshold may be user-defined or determined based on historical experience, which is not limited in the embodiments of the present application.

[0070] It should be noted here that the latest candidate frame information set may include multiple current candidate frame information that meets the conditions in the current image frame, and may also include multiple candidate frame information that meets the conditions in multiple consecutive image frames before the current image frame.

[0071] S300 , performing matching processing according to the latest candidate frame information set and the tracking frame information to determine the current tracking frame information of the target in the current image frame.

[0072] Furthermore, the computer device may use a matching algorithm to perform matching processing based on the latest candidate frame information set and the tracking frame information to obtain current tracking frame information of the target in the current image frame.

[0073] In addition, the computer device may also pre-train a matching model, and then input the latest candidate frame information set and tracking frame information into the matching model, and the matching model performs matching processing based on the latest candidate frame information set and tracking frame information and then outputs the current tracking frame information of the target in the current image frame. Optionally, the matching model may be implemented by at least one of a convolutional neural network model, a fully connected neural network model, a long short-term memory neural network model, a recurrent neural network model, and the like.

[0074] The current tracking frame information of the target in the current image frame may be understood as the tracking result of the target in the current image frame.

[0075] In an embodiment of the present application, the computer device may use the Hungarian algorithm to perform matching processing based on the latest candidate frame information set and the tracking frame information to obtain the current tracking frame information of the target in the current image frame.

[0076] The technical solution in the embodiment of the present application is based on the current image frame and the previous tracking frame information of the target in the adjacent previous image frame of the current image frame, obtains the current candidate frame information of multiple current candidate frames in the current image frame, and performs trajectory prediction based on the previous candidate frame information of multiple candidate frames in the previous image frame, determines the predicted candidate frame information corresponding to each previous candidate frame information in the current image frame, matches each current candidate frame information with each predicted candidate frame information to obtain a matching result, and obtains the latest candidate frame information set based on the matching result, matches the latest candidate frame information set with the tracking frame information, and determines the current tracking frame information of the target in the current image frame; the above method is not limited to the scene without obstructions, so that the wide applicability of the target tracking method can be improved, and the accuracy and robustness of the target tracking result can be improved in the process of tracking the target; at the same time, the above method does not need to configure the relevant additional equipment for shielding the obstruction, and can directly track the target accurately, so that the implementation cost of the target tracking method can be reduced, and no manual participation of the user is required, which can reduce the human resources required to implement the target tracking process, thereby reducing the labor cost.

[0077] The following is a description of the process of obtaining the current candidate frame information of multiple current candidate frames in the current image frame based on the previous tracking frame information of the current image frame and the target in the previous image frame adjacent to the current image frame. In one embodiment, the previous tracking frame information includes the tracking frame size and tracking frame position of the target in the previous image frame; Figure 3 As shown, the steps in S100 above can be implemented by the following steps:

[0078] S110, inputting the current image frame, the target frame size and the tracking frame position into a pre-built filter to obtain a response map corresponding to the search area and the target area in the current image frame; the target frame size is determined according to the tracking frame size.

[0079] In practical applications, in order to utilize the spatiotemporal trajectory of the target neighborhood candidate box, the target box size can be equal to 1.5 times, 1.8 times, 2 times or 2.5 times the tracking box size, etc., but in the embodiment of the present application, the target box size is equal to 3 times the tracking box size for illustration.

[0080] Specifically, the computer device can input the current image frame, the target frame size, and the tracking frame position into a pre-built filter, and the filter performs convolution processing on the features of the search area in the current image frame and the features of the target area to obtain a response map corresponding to the search area in the current image frame and the target area. The response map can represent the similarity between the search area and the target area.

[0081] It should be noted that the search area refers to an area in the current image frame centered at the tracking frame position and having a size equal to the target frame size. The filter can be trained based on the feature information of the target in the first image frame.

[0082] S120 . Determine current candidate box positions of multiple current candidate boxes in the current image frame according to multiple response values ​​in the response map.

[0083] The size of the response map may be the same as or different from the size of the current image frame, which is not limited in the present embodiment. There is a certain mapping relationship between the current image frame and the response map.

[0084] Specifically, the computer device can obtain the response value corresponding to each position in the response map, and then perform analysis, conversion, mapping, matching and other processing according to each response value to obtain the current candidate box position of multiple current candidate boxes in the current image frame.

[0085] S130 . Determine information of each current candidate box according to the position of each current candidate box and the size of the tracking box.

[0086] Specifically, for any current candidate box, the computer device may determine the tracking box size as the current candidate box size of the current candidate box, and determine the current candidate box position and the current candidate box size of the current candidate box as the current candidate box information of the current candidate box.

[0087] In one embodiment, if Figure 4 As shown, the step of determining the positions of the current candidate boxes of the multiple current candidate boxes in the current image frame according to the multiple response values ​​in the response graph in the above S120 may include the following steps:

[0088] S121 . Obtain a mapping position in the current image frame corresponding to a response value greater than a preset threshold among the response values.

[0089] In practical applications, the computer device can compare each response value with a preset threshold, select a response value greater than the preset threshold from each response value based on the comparison result, and then obtain the position of the response value greater than the preset threshold in the response map corresponding to the position in the current image frame, that is, the mapping position, based on the mapping relationship between the corresponding position of each response value in the response map and the different positions in the current image frame.

[0090] Optionally, the above-mentioned preset threshold value may be user-defined or determined based on historical experience values, which is not limited in the embodiments of the present application.

[0091] S122: Determine each mapping position as a position of a candidate box to be processed corresponding to the candidate box to be processed.

[0092] Further, each mapping position may be determined as a position of a candidate box to be processed corresponding to the candidate box to be processed.

[0093] S123, using a non-maximum suppression method, screening multiple current candidate frames from each candidate frame to be processed according to the position of each candidate frame to be processed, and determining the current candidate frame position of each current candidate frame according to the position of each candidate frame to be processed.

[0094] In the embodiment of the present application, the computer device can use the non-maximum suppression method to select the candidate frames to be processed with low overlap from the candidate frames to be processed according to the positions of the candidate frames to be processed (that is, one candidate frame to be processed with high overlap is selected and retained, and other candidate frames to be processed with high overlap are deleted) as multiple current candidate frames. Further, the positions of the candidate frames to be processed can be determined as the current candidate frame positions of the current candidate frames.

[0095] The technical solution in the embodiment of the present application inputs the current image frame, the target frame size and the tracking frame position into a pre-built filter to obtain a response map corresponding to the search area and the target area in the current image frame; the target frame size is determined according to the tracking frame size, and the current candidate frame positions of multiple current candidate frames in the current image frame are determined according to multiple response values ​​in the response map, and the information of each current candidate frame is determined according to the position of each current candidate frame and the tracking frame size; the above method does not require the participation of complex algorithms, thereby simplifying the process of determining the information of each current candidate frame in the current image frame, and can improve the speed and efficiency of determining the information of each current candidate frame. On this basis, the speed of completing the target tracking process can be accelerated.

[0096] The following is an explanation of the process of determining the current tracking frame information of the target in the current image frame by matching the latest candidate frame information set and the tracking frame information. Figure 5 As shown, the steps in the above S300 may include the following steps:

[0097] S310 . For any current candidate box information in the latest candidate box information set, match the current candidate box information with the tracking box information.

[0098] Among them, for any current candidate box information in the latest candidate box information set, the computer device can use a matching algorithm to match the current candidate box information with the tracking box information.

[0099] Optionally, the above matching algorithm may be a fuzzy matching algorithm, an exact matching algorithm, a clustering algorithm, etc., but in the embodiment of the present application, the above matching algorithm may be a Hungarian algorithm.

[0100] S320: If the match is successful, find out whether there is tracking box information in the latest candidate box information set.

[0101] In practical applications, when the match is successful, it is possible to find out whether there is tracking box information in the latest candidate box information set.

[0102] S330, if it does not exist, determine the current candidate frame information as the current tracking frame information of the target in the current image frame, and delete the tracking frame information from the latest candidate frame information set; if it exists, update the latest candidate frame information set to obtain the latest candidate frame information set.

[0103] Among them, when it is determined that there is no tracking frame information in the latest candidate frame information set, the current candidate frame information can be determined as the current tracking frame information of the target in the current image frame, and the tracking frame information can be deleted from the latest candidate frame information set; when it is determined that there is tracking frame information in the latest candidate frame information set, the latest candidate frame information set can be updated to obtain the latest candidate frame information set.

[0104] The technical solution in the embodiment of the present application is to match the current candidate frame information with the tracking frame information for any current candidate frame information in the latest candidate frame information set. If the match is successful, it is checked whether the tracking frame information exists in the latest candidate frame information set. If not, the current candidate frame information is determined as the current tracking frame information of the target in the current image frame, and the tracking frame information is deleted from the latest candidate frame information set. If it exists, the latest candidate frame information set is updated to obtain the latest candidate frame information set. The above method can quickly reduce the occurrence of the tracking frame information of a false target with low reliability as the tracking frame information of the real target through matching processing, correct the tracking error, and improve the speed and efficiency of obtaining the actual tracking frame information of the target.

[0105] In practical applications, since the target is blocked by an obstruction when the image acquisition device acquires the image frame, the information of the tracking frame where the target is located in the image frame cannot be determined after the target tracking processing is performed on the image frame. The process of implementing the target tracking processing in this case is described below. In one embodiment, the above method may also include the following steps: if each current candidate frame information in the latest candidate frame information set fails to match the tracking frame information or the current tracking frame information is not determined, the target in the current image frame is re-detected to obtain the current tracking frame information of the target in the current image frame.

[0106] Among them, when each current candidate box information in the latest candidate box information set fails to match the tracking box information or the current tracking box information is not determined, the computer device can obtain a pre-trained re-detection model, and then input the identification information of the current image frame and the target into the re-detection model. The re-detection model re-detects the target in the current image frame and outputs the current tracking box information of the target in the current image frame.

[0107] Optionally, the identification information of the target may include information such as the appearance shape, color, orientation, size, etc. of the target. Optionally, the above re-detection model may be implemented by at least one of a convolutional neural network model, a long short-term memory neural network model, a fully connected neural network model, a residual neural network model, etc.

[0108] In some scenarios, considering the motion clues of the target, the search and positioning of the target in the image frame can be effectively realized. The image frame can be rasterized to realize the target tracking process. The process is described below. Figure 6 As shown, the above step of re-detecting the target in the current image frame and obtaining the current tracking frame information of the target in the current image frame may include:

[0109] S400: Perform rasterization processing on the current image frame to obtain a plurality of raster region images.

[0110] Specifically, the computer device can perform rasterization processing, i.e. segmentation processing, on the current image frame according to the target frame size, and retain the image of the region with insufficient edges to obtain multiple raster region images. The sizes of different raster region images can be the same or different, which is not limited in the embodiment of the present application.

[0111] For example, Figure 7 The figure shows a current image frame. After rasterization processing of the current image frame, 9 raster area images are obtained, among which the sizes of raster area images 1, 2, 4 and 5 are the same as the size of the target frame, and raster area images 3, 6, 7, 8 and 9 are all area images with insufficient edges in the current image frame.

[0112] S500 . For any grid region image, the grid region image is used as a current image frame, and the target tracking process is continued until current tracking frame information of the target in the current image frame is determined.

[0113] In practical applications, for any grid area image, the grid area image is taken as the current image frame, and the computer device can perform target tracking processing according to the previous tracking frame information in the grid area image and the previous image frame according to the steps in S100-S300 above, until the current tracking frame information of the target in the current image frame is determined.

[0114] In another embodiment, the above method may further include: if the current tracking frame information of the target is not determined after re-detection of the target in the current image frame, target tracking processing is performed on the next adjacent image frame after the current image frame to track the target in the next adjacent image frame.

[0115] Among them, when the current tracking frame information of the target has not been determined after re-detection of the target in the current image frame, after the image acquisition device captures the next adjacent image frame after the current image frame, the target tracking process can continue to be performed on the next adjacent image frame after the current image frame to achieve subsequent tracking.

[0116] According to the technical solution in the embodiment of the present application, if each current candidate frame information in the latest candidate frame information set fails to match the tracking frame information or the current tracking frame information is not determined, the target in the current image frame is re-detected to obtain the current tracking frame information of the target in the current image frame; in the case of the first tracking failure of the above method, the re-detection method can be used to re-capture the occluded target in the current image frame, which can avoid tracking failure or missed tracking as much as possible, thereby improving the accuracy of target tracking.

[0117] In one embodiment, the present application also provides a target tracking method, which is applied to a computer device. The method includes the following process:

[0118] (1) The current image frame, the target frame size, and the tracking frame position of the target in the previous image frame are input into a pre-built filter to obtain a response map corresponding to the search area in the current image frame and the target area; the target frame size is determined based on the tracking frame size;

[0119] (2) Obtaining the mapping position in the current image frame corresponding to the response value greater than a preset threshold among the response values ​​in the response map;

[0120] (3) Determine each mapping position as the position of the candidate frame to be processed corresponding to the candidate frame to be processed;

[0121] (4) Using the non-maximum suppression method, multiple current candidate frames are selected from each candidate frame to be processed according to the position of each candidate frame to be processed, and the current candidate frame position of each current candidate frame is determined according to the position of each candidate frame to be processed;

[0122] (5) determining information of each current candidate box according to the position of each current candidate box and the size of the tracking box; and,

[0123] (6) performing trajectory prediction based on previous candidate frame information of multiple candidate frames in the previous image frame, and determining predicted candidate frame information corresponding to each previous candidate frame information in the current image frame;

[0124] (7) Matching the current candidate box information with the predicted candidate box information to obtain a matching result;

[0125] (8) If the similarity between each current candidate box information and the corresponding predicted candidate box information in the matching result is greater than a preset similarity threshold, each current candidate box information is added to the candidate box information set to generate the latest candidate box information set;

[0126] (9) For any current candidate box information in the latest candidate box information set, match the current candidate box information with the tracking box information;

[0127] (10) If the match is successful, find out whether the tracking box information exists in the latest candidate box information set;

[0128] (11) If it does not exist, the current candidate box information is determined as the current tracking box information of the target in the current image frame, and the tracking box information is deleted from the latest candidate box information set; if it exists, the latest candidate box information set is updated to obtain the latest candidate box information set;

[0129] (12) If each current candidate frame information in the latest candidate frame information set fails to match the tracking frame information or the current tracking frame information is not determined, the current image frame is rasterized to obtain multiple raster area images;

[0130] (13) For any grid region image, the grid region image is used as the current image frame, and the target tracking process is continued until the current tracking frame information of the target in the current image frame is determined;

[0131] (14) If the current tracking frame information of the target is not determined after the target in the current image frame is re-detected, the target tracking process is performed on the next adjacent image frame after the current image frame to track the target in the next adjacent image frame.

[0132] The execution process of the above (1) to (14) can be specifically referred to the description of the above embodiment. The implementation principle and technical effect are similar and will not be repeated here.

[0133] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0134] Based on the same inventive concept, the embodiment of the present application also provides a target tracking device for implementing the target tracking method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more target tracking device embodiments provided below can refer to the limitations on the target tracking method above, and will not be repeated here.

[0135] In one embodiment, Figure 8 This is a schematic diagram of the structure of a target tracking device in one embodiment of the present application. The target tracking device provided in the embodiment of the present application can be applied to a computer device. Figure 8 As shown, the target tracking device of the embodiment of the present application may include: a determination module 11, a first matching module 12 and a second matching module 13, wherein:

[0136] The determination module 11 is used to obtain current candidate frame information of multiple current candidate frames in the current image frame based on the current image frame and the previous tracking frame information of the target in the previous image frame adjacent to the current image frame; and perform trajectory prediction according to the previous candidate frame information of the multiple candidate frames in the previous image frame to determine the predicted candidate frame information corresponding to each previous candidate frame information in the current image frame; the current image frame is the next adjacent image frame of the previous image frame;

[0137] A first matching module 12 is used to match each current candidate box information with each predicted candidate box information to obtain a matching result, and obtain a latest candidate box information set according to the matching result;

[0138] The second matching module 13 is used to perform matching processing according to the latest candidate frame information set and the tracking frame information to determine the current tracking frame information of the target in the current image frame.

[0139] The target tracking device provided in the embodiment of the present application can be used to execute the technical solution in the above-mentioned target tracking method embodiment of the present application. Its implementation principle and technical effect are similar and will not be repeated here.

[0140] In one embodiment, the previous tracking frame information includes the tracking frame size and tracking frame position of the target in the previous image frame; the determination module 11 includes: an acquisition unit, a first determination unit and a second determination unit, wherein:

[0141] An acquisition unit is used to input the current image frame, the target frame size and the tracking frame position into a pre-built filter to obtain a response map corresponding to the search area and the target area in the current image frame; the target frame size is determined according to the tracking frame size;

[0142] A first determining unit, configured to determine a current candidate box position of a plurality of current candidate boxes in a current image frame according to a plurality of response values ​​in the response map;

[0143] The second determining unit is used to determine the information of each current candidate box according to the position of each current candidate box and the size of the tracking box.

[0144] The target tracking device provided in the embodiment of the present application can be used to execute the technical solution in the above-mentioned target tracking method embodiment of the present application. Its implementation principle and technical effect are similar and will not be repeated here.

[0145] In one embodiment, the first determining unit is specifically configured to:

[0146] Obtaining the mapping position of the response value greater than the preset threshold value in the current image frame;

[0147] Determine each mapping position as a position of a candidate frame to be processed corresponding to the candidate frame to be processed;

[0148] A non-maximum suppression method is adopted to screen multiple current candidate frames from each candidate frame to be processed according to the position of each candidate frame to be processed, and the current candidate frame position of each current candidate frame is determined according to the position of each candidate frame to be processed.

[0149] The target tracking device provided in the embodiment of the present application can be used to execute the technical solution in the above-mentioned target tracking method embodiment of the present application. Its implementation principle and technical effect are similar and will not be repeated here.

[0150] In one embodiment, the matching result includes the similarity between each current candidate box information and the corresponding predicted candidate box information; the first matching module 12 is specifically used to:

[0151] If the similarity between each current candidate box information and the corresponding predicted candidate box information is greater than a preset similarity threshold, each current candidate box information is added to the candidate box information set to generate the latest candidate box information set.

[0152] The target tracking device provided in the embodiment of the present application can be used to execute the technical solution in the above-mentioned target tracking method embodiment of the present application. Its implementation principle and technical effect are similar and will not be repeated here.

[0153] In one embodiment, the second matching module 13 is specifically used for:

[0154] For any current candidate box information in the latest candidate box information set, matching processing is performed on the current candidate box information and the tracking box information;

[0155] If the match is successful, then check whether there is tracking box information in the latest candidate box information set;

[0156] If not, the current candidate frame information is determined as the current tracking frame information of the target in the current image frame, and the tracking frame information is deleted from the latest candidate frame information set to update the latest candidate frame information set to obtain the latest candidate frame information set.

[0157] The target tracking device provided in the embodiment of the present application can be used to execute the technical solution in the above-mentioned target tracking method embodiment of the present application. Its implementation principle and technical effect are similar and will not be repeated here.

[0158] In one embodiment, the target tracking device further includes: a re-detection processing module, wherein:

[0159] The re-detection processing module is used to re-detect the target in the current image frame and obtain the current tracking frame information of the target in the current image frame when each current candidate frame information in the latest candidate frame information set fails to match the tracking frame information or the current tracking frame information is not determined.

[0160] The target tracking device provided in the embodiment of the present application can be used to execute the technical solution in the above-mentioned target tracking method embodiment of the present application. Its implementation principle and technical effect are similar and will not be repeated here.

[0161] In one embodiment, the re-detection processing module is specifically used to:

[0162] Performing rasterization processing on the current image frame to obtain multiple raster area images;

[0163] For any grid region image, the grid region image is used as the current image frame, and the target tracking process is continued until the current tracking frame information of the target in the current image frame is determined.

[0164] The target tracking device provided in the embodiment of the present application can be used to execute the technical solution in the above-mentioned target tracking method embodiment of the present application. Its implementation principle and technical effect are similar and will not be repeated here.

[0165] In one embodiment, the target tracking device further includes: an execution module, wherein:

[0166] The execution module is used to perform target tracking processing on the next adjacent image frame after the current image frame when the current tracking frame information of the target is not determined after the target in the current image frame is re-detected, so as to track the target in the next adjacent image frame.

[0167] The target tracking device provided in the embodiment of the present application can be used to execute the technical solution in the above-mentioned target tracking method embodiment of the present application. Its implementation principle and technical effect are similar and will not be repeated here.

[0168] For the specific definition of the target tracking device, please refer to the definition of the target tracking method above, which will not be repeated here. Each module in the above-mentioned target tracking device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0169] In one embodiment, a computer device is also provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig. 9 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide processing power. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store a plurality of continuous image frames acquired by the image acquisition device. The network interface of the computer device is used to communicate with an external end point through a network connection. When the computer program is executed by the processor, a target tracking method is implemented.

[0170] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0171] In one embodiment, a computer device is also provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the technical solution in the above-mentioned target tracking method embodiment of the present application is implemented. The implementation principle and technical effect are similar and will not be repeated here.

[0172] In one embodiment, a computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, the technical solution of the target tracking method mentioned above in the present application is implemented. The implementation principle and technical effect are similar and will not be repeated here.

[0173] In one embodiment, a computer program product is also provided, including a computer program. When the computer program is executed by a processor, the technical solution of the target tracking method mentioned above in the present application is implemented. The implementation principle and technical effect are similar and will not be repeated here.

[0174] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0175] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0176] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A target tracking method, characterized in that: The method comprises: Based on the current image frame and the previous tracking frame information of the target in the previous image frame adjacent to the current image frame, current candidate frame information of multiple current candidate frames in the current image frame is obtained; and trajectory prediction is performed according to the previous candidate frame information of the multiple candidate frames in the previous image frame to determine the predicted candidate frame information corresponding to each of the previous candidate frame information in the current image frame; Performing matching processing on each of the current candidate box information and each of the predicted candidate box information to obtain a matching result, and acquiring a latest candidate box information set according to the matching result; Matching processing is performed according to the latest candidate frame information set and the tracking frame information to determine the current tracking frame information of the target in the current image frame.

2. The method according to claim 1, characterized in that The previous tracking frame information includes the tracking frame size and tracking frame position of the target in the previous image frame; the acquiring the current candidate frame information of the multiple current candidate frames in the current image frame based on the current image frame and the previous tracking frame information of the target in the adjacent previous image frame of the current image frame includes: The current image frame, the target frame size and the tracking frame position are input into a pre-built filter to obtain a response map corresponding to the search area in the current image frame and the area where the target is located; the target frame size is determined according to the tracking frame size; Determining a current candidate box position of the multiple current candidate boxes in the current image frame according to a plurality of response values ​​in the response map; The information of each current candidate box is determined according to the position of each current candidate box and the size of the tracking box.

3. The method according to claim 2, characterized in that The determining, according to the multiple response values ​​in the response map, the current candidate box positions of the multiple current candidate boxes in the current image frame comprises: Obtaining a mapping position in the current image frame corresponding to a response value greater than a preset threshold among the response values; Determine each of the mapping positions as a position of a candidate frame to be processed corresponding to the candidate frame to be processed; A non-maximum suppression method is adopted to filter the multiple current candidate frames from the candidate frames to be processed according to the positions of the candidate frames to be processed, and the current candidate frame position of each current candidate frame is determined according to the positions of the candidate frames to be processed.

4. The method according to any one of claims 1 to 3, characterized in that The matching result includes the similarity between each current candidate box information and the corresponding predicted candidate box information; and obtaining the latest candidate box information set according to the matching result includes: If the similarity between each current candidate box information and the corresponding predicted candidate box information is greater than a preset similarity threshold, each current candidate box information is added to the candidate box information set to generate the latest candidate box information set.

5. The method according to any one of claims 1 to 3, characterized in that: The matching process is performed according to the latest candidate frame information set and the tracking frame information to determine the current tracking frame information of the target in the current image frame, including: For any current candidate box information in the latest candidate box information set, matching processing is performed on the current candidate box information and the tracking box information; If the match is successful, then searching whether the tracking frame information exists in the latest candidate frame information set; If it does not exist, the current candidate frame information is determined as the current tracking frame information of the target in the current image frame, and the tracking frame information is deleted from the latest candidate frame information set; if it exists, the latest candidate frame information set is updated to obtain the latest candidate frame information set.

6. The method according to any one of claims 1 to 3, characterized in that The method further comprises: If each current candidate frame information in the latest candidate frame information set fails to match the tracking frame information or the current tracking frame information is not determined, the target in the current image frame is re-detected to obtain the current tracking frame information of the target in the current image frame.

7. The method according to claim 6, characterized in that The re-detecting the target in the current image frame to obtain current tracking frame information of the target in the current image frame includes: Performing rasterization processing on the current image frame to obtain a plurality of raster region images; For any grid region image, the grid region image is used as the current image frame, and the target tracking process is continuously performed until the current tracking frame information of the target in the current image frame is determined.

8. The method according to any one of claims 1 to 3, characterized in that The method further comprises: If the current tracking frame information of the target is not determined after the target in the current image frame is re-detected, target tracking processing is performed on the next adjacent image frame after the current image frame to track the target in the next adjacent image frame.

9. A target tracking device, characterized in that: The device comprises: A determination module, configured to obtain current candidate frame information of multiple current candidate frames in the current image frame based on the current image frame and the previous tracking frame information of the target in the previous image frame adjacent to the current image frame; and perform trajectory prediction according to the previous candidate frame information of the multiple candidate frames in the previous image frame to determine the predicted candidate frame information corresponding to each of the previous candidate frame information in the current image frame; the current image frame is the next adjacent image frame of the previous image frame; A first matching module, configured to perform matching processing on each of the current candidate box information and each of the predicted candidate box information to obtain a matching result, and obtain a latest candidate box information set according to the matching result; The second matching module is used to perform matching processing according to the latest candidate frame information set and the tracking frame information to determine the current tracking frame information of the target in the current image frame.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images