Target tracking method, device, equipment, storage medium and program product

Abstract

The application relates to a target tracking method, device, equipment, storage medium and program product. The method comprises the following steps: acquiring a tracking condition corresponding to a target at a current moment in a target tracking process; determining a tracking state of the target according to the tracking condition; the tracking state comprises tracking success or tracking failure; if the tracking state is tracking failure, performing a failure recovery process on the target based on a current frame image, and determining a tracking result corresponding to the target. The method can realize accurate tracking of a target such as a pet.

Description

Technical Field

[0001] This application relates to the field of target tracking technology, and in particular to a target tracking method, apparatus, device, storage medium, and program product. Background Technology

[0002] Target tracking refers to locating a target in a series of video frames. By tracking the target, it is possible to better display and monitor it.

[0003] Currently, in the field targeting pets, the common practice is to equip the pet with a specific device equipped with a communication module such as Bluetooth, like a collar; then, a positioning command is sent to the positioning module via Bluetooth, and the positioning module obtains the pet's location data based on the positioning command, ultimately completing the location of the pet, which is considered to have achieved pet tracking.

[0004] However, the aforementioned technologies cannot accurately track targets such as pets in videos. Summary of the Invention

[0005] Therefore, it is necessary to provide a target tracking method, device, equipment, storage medium, and program product that can accurately track targets such as pets, addressing the aforementioned technical problems.

[0006] Firstly, this application provides a target tracking method, which includes:

[0007] During target tracking, obtain the tracking conditions corresponding to the target at the current moment;

[0008] The tracking status of the target is determined based on the tracking conditions; the tracking status includes tracking success or tracking failure.

[0009] If the tracking status is tracking failure, then the target is processed for failure recovery based on the current frame image to determine the tracking result corresponding to the target.

[0010] In one embodiment, the tracking conditions include: tracking the target to a keyframe image, and / or, whether tracking the target fails for a consecutive preset number of frames; the keyframe image is an image whose timing follows the initial frame image.

[0011] In one embodiment, the tracking condition is to track the target to a keyframe image, and determining the tracking state of the target based on the tracking condition includes:

[0012] Obtain the initial features corresponding to the target in the initial frame image;

[0013] Obtain the re-identification features corresponding to the target in the keyframe image;

[0014] The tracking status of the target is determined based on the initial features and the re-identification features.

[0015] In one embodiment, determining the tracking state of the target based on the initial features and the re-identification features includes:

[0016] The initial features and the re-identified features are matched to obtain the first matching result;

[0017] The tracking status of the target is determined based on the first matching result.

[0018] In one embodiment, the above-described matching process between the initial features and the re-identified features to obtain a first matching result includes:

[0019] Calculate the similarity between the initial features and the re-identified features;

[0020] The first match result is determined based on similarity.

[0021] In one embodiment, determining the tracking status of the target based on the first matching result includes:

[0022] If the first matching result is a successful match, then the tracking status of the target is determined to be a successful tracking.

[0023] If the first matching result is a failure, then the target's tracking status is determined to be tracking failure.

[0024] In one embodiment, both the initial feature and the re-identification feature are color features, and the calculation of the similarity between the initial feature and the re-identification feature includes:

[0025] The value ranges corresponding to the three channels of the initial feature are divided into segments to obtain multiple initial segments; each initial segment includes a set number of pixels.

[0026] The value ranges corresponding to the three channels of the re-identification feature are divided into segments to obtain multiple re-identification segments; each re-identification segment includes a set number of pixels.

[0027] Based on each initial segment and each re-identified segment, the similarity between the initial features and the re-identified features is determined.

[0028] In one embodiment, determining the similarity between the initial features and the re-identified features based on each initial segment and each re-identified segment includes:

[0029] Remove segments with zero pixels from each initial segment to obtain the first color feature vector;

[0030] Remove segments with zero pixels from each re-identification segment to obtain the second color feature vector;

[0031] Calculate the similarity between the first color feature vector and the second color feature vector.

[0032] In one embodiment, the above-mentioned failure recovery processing of the target based on the current frame image to determine the tracking result corresponding to the target includes:

[0033] Perform a failure recovery operation, which includes: identifying and extracting features from each candidate object in the current frame image to obtain candidate features for each candidate object; and determining the tracking result corresponding to the target based on the initial features and each candidate feature.

[0034] In one embodiment, determining the tracking result corresponding to the target based on the initial features and each candidate feature includes:

[0035] The initial features and each candidate feature are matched to obtain the second matching results;

[0036] The tracking result corresponding to the target is determined based on each second matching result.

[0037] In one embodiment, determining the tracking result corresponding to the target based on each second matching result includes:

[0038] If all second matching results are unsuccessful, the next frame image after the current frame image is taken as the new current frame image, and the above failure recovery operation is performed again until the preset loop termination condition is reached, so as to obtain the tracking result corresponding to the target.

[0039] In one embodiment, the preset loop cutoff condition includes at least one of the following: one of the second matching results is a successful match; a preset recovery time; and a preset number of image frames.

[0040] In one embodiment, the target is a pet, and the initial features corresponding to the target include at least one of color features, histogram of oriented gradients (HOG) features, deep features, and category attributes.

[0041] In one embodiment, the method further includes:

[0042] During target tracking, the bounding box containing the target is displayed in each frame of the image.

[0043] In one embodiment, the method further includes:

[0044] The position of the gimbal is adjusted in real time according to the position of the target frame so that the target frame and the target within the target frame are displayed at the set position on the display interface.

[0045] In one embodiment, the method further includes:

[0046] The target within the target box is processed in slow motion and then displayed.

[0047] Secondly, this application also provides a target tracking device, which includes:

[0048] The condition acquisition module is used to acquire the tracking conditions corresponding to the target at the current moment during the target tracking process;

[0049] The status determination module is used to determine the tracking status of the target based on the tracking conditions; the tracking status includes tracking success or tracking failure.

[0050] The tracking module is used to perform failure recovery processing on the target based on the current frame image if the tracking status is tracking failure, and to determine the tracking result corresponding to the target.

[0051] Thirdly, this application also provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0052] During target tracking, obtain the tracking conditions corresponding to the target at the current moment;

[0053] The tracking status of the target is determined based on the tracking conditions; the tracking status includes tracking success or tracking failure.

[0054] If the tracking status is tracking failure, then the target is processed for failure recovery based on the current frame image to determine the tracking result corresponding to the target.

[0055] Fourthly, this application also provides a target tracking system, including:

[0056] Gimbal;

[0057] A shooting device connected to a gimbal, the shooting device including the computer equipment described in the third aspect above.

[0058] Fifthly, this application also provides a gimbal, including:

[0059] Filming equipment;

[0060] The computer device described above, which is connected to the shooting device.

[0061] Sixthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0062] During target tracking, obtain the tracking conditions corresponding to the target at the current moment;

[0063] The tracking status of the target is determined based on the tracking conditions; the tracking status includes tracking success or tracking failure.

[0064] If the tracking status is tracking failure, then the target is processed for failure recovery based on the current frame image to determine the tracking result corresponding to the target.

[0065] In a seventh aspect, this application also provides a computer program product, comprising a computer program that, when executed by a processor, performs the following steps:

[0066] During target tracking, obtain the tracking conditions corresponding to the target at the current moment;

[0067] The tracking status of the target is determined based on the tracking conditions; the tracking status includes tracking success or tracking failure.

[0068] If the tracking status is tracking failure, then the target is processed for failure recovery based on the current frame image to determine the tracking result corresponding to the target.

[0069] The aforementioned target tracking method, apparatus, device, storage medium, and program product acquire the tracking conditions corresponding to the target at the current moment during the target tracking process, determine the target's tracking status based on the tracking conditions, and if the tracking status is tracking failure, perform failure recovery processing on the target based on the current frame image to determine the corresponding tracking result; wherein, the tracking status includes tracking success or tracking failure. In this method, since the target's tracking status can be determined through the target's tracking conditions, and the target can be recovered when the tracking status is failure to obtain an accurate tracking result, the situation of target tracking loss leading to tracking errors can be avoided, achieving accurate and stable target tracking. Attached Figure Description

[0070] Figure 1 This is a diagram illustrating the application environment of a target tracking method in one embodiment;

[0071] Figure 2 This is a flowchart illustrating a target tracking method in one embodiment;

[0072] Figure 3 This is a flowchart illustrating the target tracking method in another embodiment;

[0073] Figure 4 This is a flowchart illustrating the target tracking method in another embodiment;

[0074] Figure 5 This is a flowchart illustrating the target tracking method in another embodiment;

[0075] Figure 6 This is a flowchart illustrating the target tracking method in another embodiment;

[0076] Figure 7 This is a structural block diagram of a target tracking device in one embodiment;

[0077] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0078] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0079] The target tracking method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, the gimbal 102 can have its own built-in shooting device 104 (not shown in this case), or it can be connected to an external shooting device 104 (the accompanying drawings in this embodiment mainly show this case). The shooting device 104 is mainly used to capture objects or environments in the scene to obtain corresponding single-frame images or video streams; the shooting device 104 can be a camera, webcam, radar, mobile phone, etc. The gimbal 102 is mainly used to prevent the shooting device 104 from shaking during the image capture process, that is, to ensure that the shooting device 104 can capture images stably. In addition, the shooting device 104 can have a built-in computer device, or the shooting device 104 can be connected to an external computer device. The computer device mainly performs image processing on the single-frame images or video streams captured by the shooting device 104, and sends control commands to the gimbal 102 based on the processing results, so that the gimbal 102 can control its own rotation based on the control commands, and drive the shooting device 104 to rotate, so that the shooting device 104 is aligned with the target and the target is tracked.

[0080] In one embodiment, such as Figure 2 As shown, a target tracking method is provided, which can be applied to... Figure 1 Taking computer equipment related to the shooting equipment as an example, the method may include the following steps:

[0081] S202, during the target tracking process, obtain the tracking conditions corresponding to the target at the current moment.

[0082] During the target tracking process, the first frame image in the video stream captured by the imaging device can be obtained first, which is recorded as the initial frame image, and objects of various categories therein are identified, so as to obtain the identified objects, the category of each object, and the detection frame where each object is located. When identifying the objects in the initial frame image, a multi-category detector can be pre-trained using convolutional neural network technology to locate the positions of the objects in the picture. For example, existing mature deep learning network architectures such as YOLO and SSD can be adopted. After the multi-category detector is trained, the initial frame image can be input into the detector for identification processing, so as to obtain the identified objects, the category of each object, and the detection frame where each object is located.

[0083] Assuming that a plurality of objects are identified in the initial frame image, the target can be selected from the objects according to the category of each object. For example, if the category of the target is a dog, and the categories of the respective objects are cat, dog, pedestrian, etc., then the object with the category of dog can be selected as the target.

[0084] In the target tracking process, the adopted tracking algorithms include, but are not limited to, the manners of frame-by-frame / skip-frame detection algorithm plus matching algorithm and the like. The detection algorithm is mainly implemented by using the multi-category detector trained above; by detecting the target on the frame image, the position of the target can be accurately identified, and accurate recovery can be realized after the target goes out of bounds or is occluded, so as to ensure the continuity and stability of target tracking. The matching algorithm mainly includes the following several types: First, center distance matching. A matching threshold Thr is set, and assuming that the distance between the center coordinates of the tracking frame Box and the detection frame is set as d, the Euclidean distance in the two-dimensional space can be adopted as the measure for d, so as to obtain the detection frame Det_box closest to the tracking frame, where the center distance is D at this time. If the calculated D<Thr, it is determined that the two frames are successfully matched, and the current detection frame is used as the new tracking frame. If D>=Thr, it is determined that there is no association between the two frames, and the matching fails.

[0085] Second, Intersection over Union (IOU) matching. A matching threshold Thr is set, and the calculation method of IOU is as follows: let the intersection of the two frames be intersection, and the union be Union; then intersection=Area(Box&Det_box), that is, the intersection is the area of the intersection of the tracking frame and the detection frame; Union=Area(Box)+Area(Det_box)-intersection, that is, the union is the sum of the area of the tracking frame and the area of the detection frame minus the area of the intersection; and then IOU=intersection / Union is calculated, that is, the Intersection over Union is the ratio of the area of the intersection to the area of the union. If the calculated iou>=Thr, it is determined that the two frames are successfully matched, and the current detection frame is used as the new tracking frame. If IOU<Thr, it is determined that there is no association between the two frames, and the matching fails.

[0086] If a match fails, the new tracking box position can be predicted using the Kalman prediction method based on the movement trend of the tracking box, or the tracking box can be left to hover in place (i.e., the position of the tracking box in the previous frame is used as the position of the tracking box in the next frame).

[0087] It should be noted that the above matching process refers to matching the detection boxes of the same category as the tracking boxes with the tracking boxes.

[0088] In summary, the target tracking algorithm described above can track the target in each frame of the image and update the position of the tracking box or target box in the current frame of the image in real time.

[0089] Of course, during the continuous tracking of the target, the current tracking conditions of the target can also be obtained, which are used to characterize the current tracking status of the target. As an optional embodiment, the above tracking conditions include: tracking the target to a key frame image, and / or, whether tracking of the target has failed for a consecutive preset number of frames; the above key frame image is an image whose time sequence follows the initial frame image.

[0090] In this process, target tracking reaches keyframe images, which are sequentially located after the initial frame image. Specifically, keyframe images are images following the initial frame image during target tracking. During target tracking, frames following the initial frame image are continuously tracked until the keyframe image is reached. The keyframe image can be one or more frames following the initial frame image. Specifically, the keyframe image can be determined based on a preset interval, which can be fixed or variable. This interval can be a frame interval (e.g., every 10 or 15 frames) or a time interval (e.g., every 500ms or 1000ms). Furthermore, whether target tracking fails for a preset number of consecutive frames refers to whether the tracking algorithm fails to detect or match the target in multiple consecutive frames.

[0091] By setting different tracking conditions, the subsequent target recovery process can be carried out in various scenarios, thus avoiding the loss of target tracking in various scenarios and improving the accuracy, continuity and stability of target tracking.

[0092] S204, determine the tracking status of the target based on the tracking conditions; the tracking status includes tracking success or tracking failure.

[0093] In this step, the tracking conditions are used to characterize the current tracking status of the target. Therefore, the tracking status of the target can be reflected by the different states of the tracking conditions.

[0094] Taking the tracking condition of whether the target tracking fails for a consecutive preset number of frames as an example, if the tracking algorithm is used to track the target for multiple consecutive frames, and the detection or matching fails in all of them, then the target's current tracking state can be considered as tracking failure; while if the result of one or more frames is not detection or matching failure, then the target's current tracking state can be considered as tracking success.

[0095] Taking the tracking condition of tracking a target to a keyframe image as an example, when tracking a target to a keyframe image, a target tracking algorithm can be used to track the target in the keyframe image, or other methods, such as threshold segmentation or feature matching, can be used to obtain the tracking status of the target in the keyframe image. If the tracking is successful, the target's current tracking state can be considered successful; otherwise, the target's current tracking state can be considered unsuccessful.

[0096] S206, If the tracking status is tracking failure, then perform failure recovery processing on the target based on the current frame image to determine the tracking result corresponding to the target.

[0097] Generally, if the current tracking status of the target is "tracking failure", it can be considered that the target has been lost. This will affect the target tracking process in subsequent frames. In order to avoid this problem and make the target tracking more stable in the future, the target that has failed to be tracked can be recovered so that it can be tracked again.

[0098] In this step, if the target is lost at the current moment, the current frame image can be acquired. A tracking algorithm can then be used to detect and match the current frame image to re-identify the target, thus recovering the target and enabling subsequent target tracking. Alternatively, subsequent frame images can be acquired based on the current frame image, and a tracking algorithm can be used to detect and match these frame images to re-identify the target, thus recovering the target and enabling subsequent target tracking.

[0099] In the aforementioned target tracking method, the tracking conditions corresponding to the target at the current moment are obtained during the target tracking process, and the target's tracking status is determined based on the tracking conditions. If the tracking status is tracking failure, failure recovery processing is performed on the target based on the current frame image to determine the corresponding tracking result. The tracking status includes successful tracking or tracking failure. This method, because the target's tracking status can be determined through the target's tracking conditions, and the target can be recovered when the tracking status is failed to obtain an accurate tracking result, avoids tracking errors caused by target loss and achieves accurate and stable target tracking.

[0100] The above embodiments mentioned that the current tracking state of the target can be determined by different tracking conditions. The following embodiments will explain in detail how to determine the tracking state of the target under the condition of tracking the target to the key frame image.

[0101] In another embodiment, another target tracking method is provided, based on the above embodiments, such as... Figure 3 As shown, the above S204 may include the following steps:

[0102] S302, Obtain the initial features corresponding to the target in the initial frame image.

[0103] In this step, the target in the initial frame image can be identified in the manner described in S202 above. After the target in the initial frame image is identified, the target can be feature extracted. Specifically, the target's bounding box can be used to extract features from the image to obtain the target's initial features. The initial features of the target can then be saved for subsequent re-identification.

[0104] As an optional embodiment, the target can be a pet, and the corresponding initial features can include at least one of the following: color features, Histogram of Oriented Gradients (HOG) features, Deep Features, and category attributes. The color features can be either HSL (Hue, Saturation, Brightness) or HSV (Hue, Saturation, Brightness) features.

[0105] S304, Obtain the re-identification features corresponding to the target in the keyframe image.

[0106] The keyframe images mentioned above are those whose timing follows that of the initial frame image. For an explanation of the keyframe images in this step, please refer to the relevant explanation in S202 above; it will not be repeated here. In short, the keyframe images for tracking can be determined during the tracking process.

[0107] After determining the keyframe image, the same process as tracking other frame images can be followed. The tracking algorithm can be used to continuously search for the region in the keyframe image that is most similar to the target in the initial frame image. The most similar region is obtained, that is, the target and the target box in which the target is located. The target box includes the position information of the target box, such as the coordinates of the center point of the target box, the length, width and height of the target box.

[0108] For example, taking a pet as the target, assuming the pet's coordinates in the current frame image are (x0, y0, w1, h1), when the pet moves to the left in the image, the system searches for or matches the region closest to the pet in the initial frame image based on the center point of the previous frame, obtaining the detection box where the pet is located, with coordinates (x1, y1, w1, h1). The position of the target box is continuously updated, and this process is the target tracking process.

[0109] The above explains that after determining that a keyframe image has been tracked, the target in the keyframe image can be tracked first. Then, feature extraction can be performed on the target in the keyframe image. Specifically, feature extraction can be performed on the image containing the target's bounding box to obtain the target's re-identification features. The specific types of these re-identification features can be the same as those in the initial features mentioned above, and will not be elaborated further here.

[0110] S306, Determine the tracking status of the target based on the initial features and the re-identification features.

[0111] In this step, after obtaining the initial features and re-identification features of the target, we can compare whether these two features of the target were close before, so as to determine whether the target in the keyframe image is the original target, that is, to obtain the tracking state corresponding to the target.

[0112] In this embodiment, when the target is tracked to a keyframe image, the initial features corresponding to the target in the initial frame image and the keyframe images during the target tracking process can be obtained. The re-identification features of the target in the keyframe images are then determined. The target's tracking state is then determined based on the initial features and the re-identification features. The keyframe image is an image whose time sequence follows the initial frame image. This approach, where re-identification is performed only on a specific target in the keyframe to determine the tracking state, rather than re-identifying all objects, reduces the computational load and shortens the processing time for a single frame image, resulting in higher processing efficiency and improved real-time image processing. Furthermore, since target re-identification is performed only on keyframes to determine the tracking state, rather than on every frame, the computational load is further reduced, further enhancing the real-time performance of image processing.

[0113] The above embodiments mentioned that the tracking status of a target can be obtained through the target's initial features and re-identification features. The following embodiments will explain in detail how to obtain the tracking status.

[0114] In another embodiment, another target tracking method is provided, such as Figure 4 As shown, based on the above embodiments, S306 may include the following steps:

[0115] S402, perform matching processing on the initial features and re-identified features to obtain the first matching result.

[0116] In this step, when matching the initial features and re-identified features of the target, it can be done by directly comparing whether the features in the two are the same, or by calculating the feature similarity between the two, or by other matching methods.

[0117] Taking the calculation of feature similarity between the two as an example, as an optional embodiment, the matching process in this step may include: calculating the similarity between the initial feature and the re-identified feature; and determining the first matching result based on the similarity.

[0118] Here, a preset similarity calculation method can be used to calculate the similarity between the initial features and the re-identified features. Then, the calculated similarity can be compared with a preset threshold Thr to obtain the first matching result. If the calculated similarity is greater than or equal to the preset threshold, it means that the initial features and the re-identified features of the target are relatively similar, and the first matching result is a successful match; if the calculated similarity is less than the preset threshold, it means that the initial features and the re-identified features of the target are not similar, and the first matching result is a failed match.

[0119] S404, determine the tracking status of the target based on the first matching result.

[0120] In this step, after obtaining the first matching result, the corresponding tracking status of the target can be obtained. As an optional embodiment, if the first matching result is a successful match, the tracking status of the target can be determined to be successful tracking; if the first matching result is a failed match, the tracking status of the target can be determined to be failed tracking.

[0121] In other words, successful tracking means the target has not been missed or lost, and the bounding box and its position can be updated to obtain a new bounding box position. Tracking failure means that the tracking area on the keyframe image has changed significantly from the features of the initial frame image, indicating that the target has been missed or lost. In this case, it is necessary to re-track or restore the target to enable it to be tracked again.

[0122] In this embodiment, the target's initial features and re-identified features are matched, and the target's tracking status is determined based on the matching result. This two-feature matching method allows for rapid acquisition of the target's tracking status, improving efficiency and real-time performance. Simultaneously, feature matching avoids target box offset or mistracking during the tracking process, enhancing the tracking's anti-interference capability and accuracy. Furthermore, the similarity between the initial and re-identified features can be calculated to obtain a matching result, leading to more accurate feature matching and improved target tracking status accuracy. Moreover, successful target tracking is determined when both features match successfully, and failed tracking is determined when they fail. This refines the process of obtaining the tracking status, resulting in more accurate tracking status results.

[0123] The above embodiments mention that the similarity between the initial features and the re-identified features of the target can be calculated. When both features are color features, the following embodiments will explain how to calculate the similarity between color features.

[0124] In another embodiment, another target tracking method is provided, such as Figure 5 As shown, based on the above embodiment, the process of calculating similarity in S402 may include the following steps:

[0125] S502, the value ranges corresponding to the three channels of the initial feature are divided into segments to obtain multiple initial segments; each initial segment includes a set number of pixels.

[0126] S504, the value ranges corresponding to the three channels of the re-identification feature are divided into segments to obtain multiple re-identification segments; each re-identification segment includes a set number of pixels.

[0127] In S502-S504, the initial frame image and the current frame image can first be converted from RGB or BGR to HSV color space, or alternatively to HSL color space. HSV and HSL both refer to chroma, saturation, and brightness. Taking HSV as an example, the corresponding H, S, and V channels can then be divided into several bins (for example, dividing pixels 1 to 100 into 10 bins, each bin containing 10 pixels). In other words, the value ranges corresponding to the three channels of the initial features and re-identification features can be divided into multiple segments, each segment acting as a container containing multiple pixels. These segments can be called the initial segment or the re-identification segment. The chroma H range is 0° to 360°, the saturation S range is 0.0 to 1.0, and the brightness V range is 0.0 (black) to 1.0 (white). Here, the number of segments divided in each of the three channels for the initial feature and the re-identified feature can be equal or unequal, and the number of pixels included in each segment can also be equal or unequal. In other words, the two set quantities mentioned above can be equal or unequal. Additionally, the extreme values ​​of the S and V channels can be saved as individual bins. S values ​​in the range of (0-30) represent pure white, and V values ​​in the range of (0-30) represent pure black. In these cases, the H color is inaccurate; extracting them separately makes the color similarity calculation more reliable. The segmentation of the HSL feature is the same as that of the HSV feature, and will not be repeated here.

[0128] It should be noted that there is no timing distinction between S502 and S504. S502 can be executed first, or S504 can be executed first, or S504 can be executed first and then S502, or S502 and S504 can be executed simultaneously.

[0129] S506, determine the similarity between the initial features and the re-identified features based on each initial segment and each re-identified segment.

[0130] In this step, in order to improve the accuracy of calculating color similarity, as an optional embodiment, segments with zero pixels in each initial segment can be removed to obtain a first color feature vector; segments with zero pixels in each re-identification segment can be removed to obtain a second color feature vector; and the similarity between the first color feature vector and the second color feature vector can be calculated.

[0131] In other words, after the segmentation is completed, some bins with a statistically zero number of pixels (i.e., the initial segment or the re-identified segment) can be removed. When calculating histogram similarity, each bin will have a certain weight. Therefore, if all bins are used to calculate histogram similarity, the result will tend to be flat, which is not conducive to highlighting the differences in histogram features. Removing bins with a zero number of pixels can make the feature differences more obvious and the calculated similarity more accurate.

[0132] After removing segments with zero pixels from both the initial and re-identified segments, the remaining segments each form a color feature vector, resulting in the first and second color feature vectors. Since class consistency is ensured during frame-by-frame matching in target tracking, class information is not needed when matching the initial and keyframe images; the similarity can be directly calculated using color histogram features. Therefore, the similarity between the first and second color feature vectors can be directly calculated and compared to a preset threshold. If the similarity is greater than or equal to the preset threshold, the two color feature vectors are considered close, meaning the initial and re-identified features are similar. In this case, the target corresponding to the initial and re-identified features is the same, indicating a successful target match. Otherwise, the match fails.

[0133] In this embodiment, the similarity between segments is calculated after dividing the initial features and re-identified features into color channel segments. This refines the similarity calculation process and makes the results more accurate. Furthermore, segments with zero pixels are removed from the divided segments before calculating the similarity, further highlighting the differences between the two features and resulting in more accurate similarity calculations.

[0134] The above embodiments mention that when target tracking fails, it will affect the target tracking process of subsequent frames. In order to avoid this problem and enable more stable target tracking in the future, it is proposed to perform recovery processing on targets that have failed to be tracked. The following embodiments will describe the failure recovery process in detail.

[0135] In another embodiment, another target tracking method is provided. If the target tracking fails, the method may further include the following steps based on the above embodiments:

[0136] Step A: Perform failure recovery operation. The failure recovery operation includes: identifying and extracting features from each candidate object in the current frame image to obtain candidate features of each candidate object; and determining the tracking result corresponding to the target based on the initial features and each candidate feature.

[0137] In this step, when the target tracking fails (i.e., the target has been missed or lost), a failure recovery process can be performed to re-track the target. Specifically, this involves re-identifying all candidate objects in the current frame image. This means using the detector trained in S202 to identify all objects in the current frame image, expanding the detection range, and obtaining the identified objects and their bounding boxes, all of which are recorded as candidate objects. Then, feature extraction can be performed on the images containing the bounding boxes of each candidate object to obtain the features corresponding to each candidate object, all of which are recorded as candidate features.

[0138] After identifying candidate objects and obtaining candidate features in the current frame image, failure recovery processing can be performed by combining the initial features of the target in the initial frame image. As an optional embodiment, such as... Figure 6 As shown, the specific recovery process can be carried out using the following steps:

[0139] S602, perform matching processing on the initial features and each candidate feature to obtain each second matching result.

[0140] S604, determine the tracking result corresponding to the target based on each second matching result.

[0141] When matching each candidate feature with the target's initial feature, the similarity between each candidate feature and the initial feature can be calculated separately to obtain the similarity for each candidate feature. Each similarity is then compared with a preset threshold to obtain a second matching result between each candidate feature and the initial feature. If the similarity between a candidate feature and the initial feature is greater than or equal to the preset threshold, it indicates that the candidate feature is similar to the initial feature, and the second matching result for that candidate feature is a successful match. If the similarity between another candidate feature and the initial feature is less than the preset threshold, it indicates that the other candidate feature is similar to the initial feature, and the second matching result for that other candidate feature is a failed match.

[0142] After obtaining the second matching results of each candidate object (also referred to as each candidate feature), as an optional embodiment, if each second matching result is a matching failure, the next frame image after the current frame image is taken as the new current frame image, and the above failure recovery operation is performed again until the preset loop termination condition is reached, so as to obtain the tracking result corresponding to the target.

[0143] In other words, candidate features and initial features of each candidate object in the current frame image can be matched. If any candidate feature matches successfully, it indicates that the target recovery result is successful, and the target can continue to be tracked, i.e., the real-time position of the target can be continuously updated. If all candidate features fail to match, it indicates that the target recovery result in the current frame image is unsuccessful. Then, the target recovery can continue in the frames after the current frame image. For example, if the current frame image is the Nth frame, then object recognition, feature extraction, and matching processing can be continuously performed in the N+1th, N+2th, N+3th, ... frames. Specifically, the N+1th frame image can be used as the new current frame image, and then the above steps A and detailed schemes S602-S604 can be returned to repeat the object recognition, feature extraction, and matching processing until the set loop cutoff condition is reached. If no match is found, the tracking ends.

[0144] The preset loop termination conditions include at least one of the following: at least one second matching result is successfully matched; a preset recovery time; and a preset number of image frames. Specifically, for the first condition, at least one second matching result is successfully matched, meaning that as long as one second matching result is successful, the target recovery is successful, and target tracking can continue. At this point, the target recovery process can be stopped. For the second condition, the preset recovery time, when repeating object recognition, feature extraction, and matching processing on subsequent frames, if a certain detection time is reached (e.g., 10 seconds), it indicates that the target has been completely lost and needs to be re-tracked, at which point the target recovery process can be stopped. For the third condition, the preset number of image frames, when repeating object recognition, feature extraction, and matching processing on subsequent frames, if a certain number of frames is reached (e.g., the 10th frame after the current frame is detected), it indicates that the target has been completely lost and needs to be re-tracked, at which point the target recovery process can be stopped.

[0145] It should be noted that the above process of identifying, extracting, and matching features from all objects in the current frame image assumes three iterations, i.e., identifying the current frame image and the two subsequent frames. Assuming six objects are identified in each frame (objects 1-6), then when selecting candidate objects for feature extraction and matching in the current frame image, objects 1 and 2 could be selected. Similarly, in the first frame image after the keyframe, objects 3 and 4 could be selected, and in the second frame image after the keyframe, objects 5 and 6 could be selected. By selecting different objects in each frame image, and using fewer than the total number of objects, the computational requirements during tracking are reduced, while ensuring feature extraction and matching are performed on all objects in the frame image, guaranteeing effective target recovery.

[0146] In this embodiment, if target tracking fails, a failure recovery operation is performed. This operation includes: re-identifying and extracting features from each candidate object in the current frame image, and determining the target recovery result using the candidate features and initial features of each candidate object. This expanded scope for re-detection and tracking allows for rapid target recovery, improving efficiency and accuracy. Furthermore, matching candidate features with target features yields a matching result to determine the target recovery result. Feature matching allows for rapid target recovery, improving efficiency and real-time performance. Even further, if target recovery is unsuccessful, continuous looping identification and feature matching can be performed in subsequent frame images to quickly recover the tracked target and ensure smooth execution of the tracking process. Setting different looping conditions can meet various scenario requirements, expanding the applicability of the recovery process.

[0147] In the aforementioned target tracking process, target tracking is generally performed on each frame of the image. This involves identifying and matching objects in each frame to obtain and track the target. During the identification process, a multi-class detector is typically used to identify objects in each frame. This yields the identified object, its category, the object's bounding box, and its position. For the target, this means obtaining the target, its category, and its bounding box, as well as its position. Based on this, in another embodiment, the target's bounding box can be displayed in each frame of the image during target tracking. Here, the user can manually adjust the camera (such as a mobile phone) to ensure the target remains centered in the frame. This real-time display of the bounding box (also called the tracking box) highlights the target's location, enhances the user's focus on the target, and thus improves the user experience.

[0148] In another embodiment, after obtaining the target, its bounding box, and the target's position, the gimbal position can be adjusted in real time based on the target's position to ensure that the target and the target within it are displayed at a predetermined location on the display interface. This predetermined location could be, for example, the center of the display interface or the upper right corner. Adjusting the gimbal position in real time based on the target's position means controlling the rotation of the gimbal's rotating components. Since the gimbal connects to and supports the shooting device, its rotation causes the shooting device to rotate, ensuring that the shooting device is always facing the target and thus continuously tracking it.

[0149] In another embodiment, after obtaining the target, its bounding box, and the box's position, the target within the bounding box can be processed in slow motion and then displayed. This slow-motion processing can involve embedding a target visual tracking algorithm into slow-motion tracking, slowing down the target's display to create a video with better visual effects and enhance the user experience.

[0150] To facilitate a detailed explanation of the technical solution of this application, a detailed embodiment is provided below to illustrate the technical solution of this application. Based on the above embodiment, taking a pet as an example, the method may include the following steps:

[0151] S1, during the pet tracking process, obtain the tracking conditions corresponding to the pet at the current moment; the tracking conditions include: tracking the pet to the key frame image, and / or, whether the pet tracking fails for a consecutive preset number of frames;

[0152] S2, if the tracking condition is to track the pet to the keyframe image, obtain the initial features corresponding to the pet in the initial frame image;

[0153] S3, Obtain the re-identification features of the pet in the keyframe image; the keyframe image is an image whose temporal sequence is after the initial frame image.

[0154] S4, divide the value range of each of the three channels of the initial feature into segments to obtain multiple initial segments; divide the value range of each of the three channels of the re-identified feature into segments to obtain multiple re-identified segments; each initial segment includes a set number of pixels; each re-identified segment includes a set number of pixels.

[0155] S5, remove segments with zero pixels from each initial segment to obtain the first color feature vector; remove segments with zero pixels from each re-identified segment to obtain the second color feature vector;

[0156] S6, calculate the first similarity between the first color feature vector and the second color feature vector, and compare the first similarity with a preset threshold;

[0157] S7. If the first similarity is greater than the preset threshold, the first matching result between the initial feature and the re-identified feature is a successful match; if the first similarity is not greater than the preset threshold, the first matching result between the initial feature and the re-identified feature is a failed match.

[0158] S8. If the first matching result is a successful match, then the tracking status of the pet is determined to be successful tracking; if the first matching result is a failed match, then the tracking status of the pet is determined to be failed tracking.

[0159] S9. If the tracking status is tracking failure, then identify and extract features of each candidate object in the current frame image to obtain the candidate features of each candidate object.

[0160] S10, Match the initial features and each candidate feature to obtain each second similarity;

[0161] S11, compare each second similarity with a preset threshold;

[0162] S12, if the second similarity is greater than the threshold, the second matching result between the initial feature and the candidate feature is a successful match; if the second similarity is not greater than the threshold, the second matching result between the initial feature and the candidate feature is a failed match.

[0163] S13, if all second matching results are unsuccessful, then the next frame image after the current frame image is taken as the new current frame image, and S9-S13 are executed again until the preset loop termination condition is met, and the tracking result corresponding to the pet is obtained; the preset loop termination condition includes at least one of the following: there is a second matching result that is a successful match; the preset recovery time; the preset number of image frames;

[0164] S14, Display the target box where the pet is located in each frame image during pet tracking;

[0165] S15, adjust the position of the gimbal in real time according to the position of the target box, so that the target box and the pet in the target box are displayed at the set position on the display interface;

[0166] S16, perform slow-motion processing on the pet in the target box and display it.

[0167] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0168] Based on the same inventive concept, this application also provides a target tracking device for implementing the target tracking method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more target tracking device embodiments provided below can be found in the limitations of the target tracking method described above, and will not be repeated here.

[0169] In one embodiment, such as Figure 7 As shown, a target tracking device is provided, including: a condition acquisition module, a state determination module, and a tracking module, wherein:

[0170] The condition acquisition module is used to acquire the tracking conditions corresponding to the target at the current moment during the target tracking process;

[0171] The status determination module is used to determine the tracking status of the target based on the tracking conditions; the tracking status includes tracking success or tracking failure.

[0172] The tracking module is used to perform failure recovery processing on the target based on the current frame image if the tracking status is tracking failure, and to determine the tracking result corresponding to the target.

[0173] Optionally, the above tracking conditions include: tracking the target to a keyframe image, and / or, whether tracking the target fails for a consecutive preset number of frames.

[0174] In another embodiment, another target tracking device is provided. Based on the above embodiments, the tracking condition is to track the target to a keyframe image, and the state determination module may include:

[0175] The initial feature acquisition unit is used to acquire the initial features corresponding to the target in the initial frame image;

[0176] The re-identification feature acquisition unit is used to acquire the re-identification features corresponding to the target in the keyframe image; the keyframe image is an image whose temporal sequence is after the initial frame image.

[0177] The state determination unit is used to determine the tracking state of the target based on the initial features and the re-identification features.

[0178] Optionally, the target is a pet, and the initial features corresponding to the target include at least one of color features, histogram of oriented gradients (HOG) features, deep features, and category attributes.

[0179] Optionally, the aforementioned state determination unit may include:

[0180] The first matching subunit is used to match the initial features and the re-identified features to obtain the first matching result;

[0181] The state determination subunit is used to determine the tracking state of the target based on the first matching result.

[0182] Optionally, the first matching subunit is specifically used to calculate the similarity between the initial features and the re-identified features; and to determine the first matching result based on the similarity.

[0183] Optionally, the aforementioned state determination subunit is specifically used to determine the target's tracking status as successful tracking if the first matching result is a successful match, and to determine the target's tracking status as failed tracking if the first matching result is a failed match.

[0184] In another embodiment, another target tracking device is provided. Based on the above embodiment, both the initial feature and the re-identification feature are color features. The first matching subunit is specifically used to divide the value ranges corresponding to the three channels of the initial feature into segments to obtain multiple initial segments; to divide the value ranges corresponding to the three channels of the re-identification feature into segments to obtain multiple re-identification segments; to determine the similarity between the initial feature and the re-identification feature based on each initial segment and each re-identification segment; each initial segment includes a set number of pixels; each re-identification segment includes a set number of pixels.

[0185] Optionally, the first matching subunit is specifically used to remove segments with zero pixels in each initial segment to obtain a first color feature vector; remove segments with zero pixels in each re-identification segment to obtain a second color feature vector; and calculate the similarity between the first color feature vector and the second color feature vector.

[0186] In another embodiment, another target tracking device is provided. Based on the above embodiments, the tracking module may include a failure recovery unit, which is used to perform a failure recovery operation. The failure recovery operation includes: identifying and extracting features of each candidate object in the current frame image to obtain candidate features of each candidate object; and determining the tracking result corresponding to the target based on the initial features and each candidate feature.

[0187] Optionally, the failure recovery unit described above may include:

[0188] The second matching subunit is used to match the initial features with each candidate feature to obtain each second matching result;

[0189] The tracking subunit is used to determine the tracking result corresponding to the target based on each second matching result.

[0190] Optionally, the aforementioned tracking subunit is specifically used to, if all second matching results are unsuccessful, take the next frame image after the current frame image as the new current frame image, return to perform the aforementioned failure recovery operation, until the preset loop termination condition is reached, and obtain the tracking result corresponding to the target.

[0191] Optionally, the preset loop termination condition includes at least one of the following: one of the second matching results is a successful match; a preset recovery time; and a preset number of image frames.

[0192] In another embodiment, a different target tracking device is provided, which, based on the above embodiments, further includes:

[0193] The display module is used to display the target bounding box of the target in each frame of the image during the target tracking process.

[0194] Optionally, it also includes: an adjustment module, used to adjust the position of the gimbal in real time according to the position of the target box, so that the target box and the target in the target box are displayed at the set position on the display interface.

[0195] Optionally, it also includes a processing module for slow-motion processing and displaying the target in the target box.

[0196] Each module in the aforementioned target tracking device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0197] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows:Figure 8 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a target tracking method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0198] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0199] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0200] During target tracking, the tracking conditions corresponding to the target at the current moment are obtained; the tracking status of the target is determined based on the tracking conditions; the tracking status includes tracking success or tracking failure; if the tracking status is tracking failure, the target is processed for failure recovery based on the current frame image to determine the tracking result corresponding to the target.

[0201] In one embodiment, the tracking conditions include: tracking the target to a keyframe image, and / or, whether tracking the target fails for a consecutive preset number of frames.

[0202] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0203] Obtain the initial features corresponding to the target in the initial frame image; obtain the re-identification features corresponding to the target in the key frame image; the key frame image is an image whose time sequence is after the initial frame image; determine the tracking state of the target based on the initial features and the re-identification features.

[0204] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0205] The initial features and the re-identified features are matched to obtain the first matching result; the tracking status of the target is determined based on the first matching result.

[0206] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0207] Calculate the similarity between the initial features and the re-identified features; determine the first matching result based on the similarity.

[0208] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0209] If the first matching result is a successful match, the target's tracking status is determined to be successful tracking; if the first matching result is a failed match, the target's tracking status is determined to be failed tracking.

[0210] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0211] The value ranges of the three channels of the initial feature are divided into segments to obtain multiple initial segments; the value ranges of the three channels of the re-identified feature are divided into segments to obtain multiple re-identified segments; the similarity between the initial feature and the re-identified feature is determined based on each initial segment and each re-identified segment; each initial segment includes a set number of pixels; each re-identified segment includes a set number of pixels.

[0212] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0213] Remove segments with zero pixels from each initial segment to obtain the first color feature vector; remove segments with zero pixels from each re-identification segment to obtain the second color feature vector; calculate the similarity between the first color feature vector and the second color feature vector.

[0214] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0215] Perform a failure recovery operation, which includes: identifying and extracting features from each candidate object in the current frame image to obtain candidate features for each candidate object; and determining the tracking result corresponding to the target based on the initial features and each candidate feature.

[0216] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0217] The initial features and each candidate feature are matched to obtain each second matching result; the tracking result corresponding to the target is determined based on each second matching result.

[0218] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0219] If all second matching results are unsuccessful, the next frame image after the current frame image is taken as the new current frame image, and the above failure recovery operation is performed again until the preset loop termination condition is reached, so as to obtain the tracking result corresponding to the target.

[0220] In one embodiment, the preset loop termination condition includes at least one of the following: one of the second matching results is a successful match; a preset recovery time; and a preset number of image frames.

[0221] In one embodiment, the target is a pet, and the initial features corresponding to the target include at least one of color features, histogram of oriented gradients (HOG) features, deep features, and category attributes.

[0222] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0223] During target tracking, the bounding box containing the target is displayed in each frame of the image.

[0224] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0225] The position of the gimbal is adjusted in real time according to the position of the target frame so that the target frame and the target within the target frame are displayed at the set position on the display interface.

[0226] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0227] The target within the target box is processed in slow motion and then displayed.

[0228] In one embodiment, a target tracking system is provided, see [link to relevant documentation]. Figure 1 The example diagram shown may include a gimbal; and a shooting device connected to the gimbal, which includes the aforementioned computer equipment.

[0229] In one embodiment, a gimbal is provided, including a shooting device and the aforementioned computer device connected to the shooting device.

[0230] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0231] During target tracking, the tracking conditions corresponding to the target at the current moment are obtained; the tracking status of the target is determined based on the tracking conditions; the tracking status includes tracking success or tracking failure; if the tracking status is tracking failure, the target is processed for failure recovery based on the current frame image to determine the tracking result corresponding to the target.

[0232] In one embodiment, the tracking conditions include: tracking the target to a keyframe image, and / or, whether tracking the target fails for a consecutive preset number of frames.

[0233] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0234] Obtain the initial features corresponding to the target in the initial frame image; obtain the re-identification features corresponding to the target in the key frame image; the key frame image is an image whose time sequence is after the initial frame image; determine the tracking state of the target based on the initial features and the re-identification features.

[0235] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0236] The initial features and the re-identified features are matched to obtain the first matching result; the tracking status of the target is determined based on the first matching result.

[0237] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0238] Calculate the similarity between the initial features and the re-identified features; determine the first matching result based on the similarity.

[0239] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0240] If the first matching result is a successful match, the target's tracking status is determined to be successful tracking; if the first matching result is a failed match, the target's tracking status is determined to be failed tracking.

[0241] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0242] The value ranges of the three channels of the initial feature are divided into segments to obtain multiple initial segments; the value ranges of the three channels of the re-identified feature are divided into segments to obtain multiple re-identified segments; the similarity between the initial feature and the re-identified feature is determined based on each initial segment and each re-identified segment; each initial segment includes a set number of pixels; each re-identified segment includes a set number of pixels.

[0243] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0244] Remove segments with zero pixels from each initial segment to obtain the first color feature vector; remove segments with zero pixels from each re-identification segment to obtain the second color feature vector; calculate the similarity between the first color feature vector and the second color feature vector.

[0245] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0246] Perform a failure recovery operation, which includes: identifying and extracting features from each candidate object in the current frame image to obtain candidate features for each candidate object; and determining the tracking result corresponding to the target based on the initial features and each candidate feature.

[0247] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0248] The initial features and each candidate feature are matched to obtain each second matching result; the tracking result corresponding to the target is determined based on each second matching result.

[0249] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0250] If all second matching results are unsuccessful, the next frame image after the current frame image is taken as the new current frame image, and the above failure recovery operation is performed again until the preset loop termination condition is reached, so as to obtain the tracking result corresponding to the target.

[0251] In one embodiment, the preset loop termination condition includes at least one of the following: one of the second matching results is a successful match; a preset recovery time; and a preset number of image frames.

[0252] In one embodiment, the target is a pet, and the initial features corresponding to the target include at least one of color features, histogram of oriented gradients (HOG) features, deep features, and category attributes.

[0253] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0254] During target tracking, the bounding box containing the target is displayed in each frame of the image.

[0255] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0256] The position of the gimbal is adjusted in real time according to the position of the target frame so that the target frame and the target within the target frame are displayed at the set position on the display interface.

[0257] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0258] The target within the target box is processed in slow motion and then displayed.

[0259] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0260] During target tracking, the tracking conditions corresponding to the target at the current moment are obtained; the tracking status of the target is determined based on the tracking conditions; the tracking status includes tracking success or tracking failure; if the tracking status is tracking failure, the target is processed for failure recovery based on the current frame image to determine the tracking result corresponding to the target.

[0261] In one embodiment, the tracking conditions include: tracking the target to a keyframe image, and / or, whether tracking the target fails for a consecutive preset number of frames.

[0262] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0263] Obtain the initial features corresponding to the target in the initial frame image; obtain the re-identification features corresponding to the target in the key frame image; the key frame image is an image whose time sequence is after the initial frame image; determine the tracking state of the target based on the initial features and the re-identification features.

[0264] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0265] The initial features and the re-identified features are matched to obtain the first matching result; the tracking status of the target is determined based on the first matching result.

[0266] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0267] Calculate the similarity between the initial features and the re-identified features; determine the first matching result based on the similarity.

[0268] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0269] If the first matching result is a successful match, the target's tracking status is determined to be successful tracking; if the first matching result is a failed match, the target's tracking status is determined to be failed tracking.

[0270] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0271] The value ranges of the three channels of the initial feature are divided into segments to obtain multiple initial segments; the value ranges of the three channels of the re-identified feature are divided into segments to obtain multiple re-identified segments; the similarity between the initial feature and the re-identified feature is determined based on each initial segment and each re-identified segment; each initial segment includes a set number of pixels; each re-identified segment includes a set number of pixels.

[0272] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0273] Remove segments with zero pixels from each initial segment to obtain the first color feature vector; remove segments with zero pixels from each re-identification segment to obtain the second color feature vector; calculate the similarity between the first color feature vector and the second color feature vector.

[0274] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0275] Perform a failure recovery operation, which includes: identifying and extracting features from each candidate object in the current frame image to obtain candidate features for each candidate object; and determining the tracking result corresponding to the target based on the initial features and each candidate feature.

[0276] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0277] The initial features and each candidate feature are matched to obtain each second matching result; the tracking result corresponding to the target is determined based on each second matching result.

[0278] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0279] If all second matching results are unsuccessful, the next frame image after the current frame image is taken as the new current frame image, and the above failure recovery operation is performed again until the preset loop termination condition is reached, so as to obtain the tracking result corresponding to the target.

[0280] In one embodiment, the preset loop termination condition includes at least one of the following: one of the second matching results is a successful match; a preset recovery time; and a preset number of image frames.

[0281] In one embodiment, the target is a pet, and the initial features corresponding to the target include at least one of color features, histogram of oriented gradients (HOG) features, deep features, and category attributes.

[0282] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0283] During target tracking, the bounding box containing the target is displayed in each frame of the image.

[0284] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0285] The position of the gimbal is adjusted in real time according to the position of the target frame so that the target frame and the target within the target frame are displayed at the set position on the display interface.

[0286] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0287] The target within the target box is processed in slow motion and then displayed.

[0288] It should be noted that all data involved in this application (including but not limited to data used for analysis, data stored, data displayed, etc.) are data that have been fully authorized by all parties.

[0289] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0290] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0291] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A target tracking method, characterized in that, The method includes: During target tracking, the tracking conditions corresponding to the target at the current moment are obtained; wherein, the tracking conditions include: tracking the target to a key frame image; the key frame image is determined according to a preset interval; the interval includes an image frame interval and a time interval; The step of determining the tracking status of the target based on the tracking conditions includes: acquiring initial features corresponding to the target in the initial frame image; acquiring re-identification features corresponding to the target in the key frame image; and determining the tracking status of the target based on the initial features and the re-identification features; the tracking status includes tracking success or tracking failure. If the tracking status is tracking failure, then a failure recovery process is performed on the target based on the current frame image to determine the tracking result corresponding to the target. This includes: performing a failure recovery operation, which includes: identifying and extracting features from each candidate object in the current frame image to obtain candidate features for each candidate object; determining the tracking result corresponding to the target based on the similarity between the initial feature and each candidate feature; wherein, if any candidate feature matches the initial feature successfully, it indicates that the target recovery result is successful, and tracking of the target continues; if all candidate features fail to match the initial feature, then the next frame image after the current frame image is taken as the new current frame image, and the failure recovery operation is returned to be performed; the initial feature represents the feature corresponding to the target in the initial frame image; wherein, when performing the failure recovery operation to identify and extract features from each candidate object in the current frame image, a number of candidate objects with different objects and less than all objects are selected in each frame image for feature extraction.

2. The method according to claim 1, characterized in that, The tracking conditions include: tracking the target to a keyframe image, and / or, whether tracking the target fails for a consecutive preset number of frames; the keyframe image is an image whose timing follows the initial frame image.

3. The method according to claim 1, characterized in that, Determining the tracking status of the target based on the initial features and the re-identification features includes: The initial features and the re-identified features are matched to obtain a first matching result; The tracking status of the target is determined based on the first matching result.

4. The method according to claim 3, characterized in that, The step of matching the initial features and the re-identified features to obtain a first matching result includes: Calculate the similarity between the initial features and the re-identified features; The first matching result is determined based on the similarity.

5. The method according to claim 3 or 4, characterized in that, Determining the tracking status of the target based on the first matching result includes: If the first matching result is a successful match, then the tracking status of the target is determined to be a successful tracking. If the first matching result is a matching failure, then the tracking status of the target is determined to be tracking failure.

6. The method according to claim 4, characterized in that, Both the initial feature and the re-identification feature are color features. The calculation of the similarity between the initial feature and the re-identification feature includes: The value ranges corresponding to the three channels of the initial feature are divided into segments to obtain multiple initial segments; each initial segment includes a set number of pixels. The value ranges corresponding to the three channels of the re-identification feature are divided into segments to obtain multiple re-identification segments; each re-identification segment includes a set number of pixels; The similarity between the initial features and the re-identified features is determined based on each of the initial segments and each of the re-identified segments.

7. The method according to claim 6, characterized in that, Determining the similarity between the initial features and the re-identified features based on each initial segment and each re-identified segment includes: Remove segments with zero pixels from each of the initial segments to obtain the first color feature vector; Remove segments with zero pixels from each of the re-identified segments to obtain the second color feature vector; Calculate the similarity between the first color feature vector and the second color feature vector.

8. The method according to claim 1, characterized in that, The step of determining the tracking result corresponding to the target based on the initial features and each of the candidate features includes: The initial features and each of the candidate features are matched to obtain the second matching results; The tracking result corresponding to the target is determined based on each of the second matching results.

9. The method according to claim 8, characterized in that, The step of determining the tracking result corresponding to the target based on each of the second matching results includes: If all the second matching results are unsuccessful, then the next frame image after the current frame image is taken as the new current frame image, and the failure recovery operation is performed again until the preset loop termination condition is reached, so as to obtain the tracking result corresponding to the target.

10. The method according to claim 9, characterized in that, The preset cycle cutoff condition includes at least one of the following: In each of the second matching results, there is one second matching result that is a successful match; Preset recovery time; The preset number of image frames.

11. The method according to any one of claims 3-4, characterized in that, The target is a pet, and the initial features corresponding to the target include at least one of the following: color features, oriented gradient histogram (HOG) features, deep features, and category attributes.

12. The method according to any one of claims 1-4, characterized in that, The method further includes: During target tracking, the bounding box containing the target is displayed in each frame of the image.

13. The method according to claim 12, characterized in that, The method further includes: The position of the gimbal is adjusted in real time according to the position of the target frame so that the target frame and the target within the target frame are displayed at the set position on the display interface.

14. The method according to claim 12, characterized in that, The method further includes: The target within the target box is processed in slow motion and then displayed.

15. A target tracking device, characterized in that, The device includes: The condition acquisition module is used to acquire the tracking conditions corresponding to the target at the current moment during the target tracking process; wherein, the tracking conditions include: tracking the target to a key frame image; the key frame image is determined according to a preset interval; the interval includes an image frame interval and a time interval; The state determination module is used to determine the tracking state of the target based on the tracking conditions, specifically: acquiring initial features corresponding to the target in the initial frame image; acquiring re-identification features corresponding to the target in the key frame image; and determining the tracking state of the target based on the initial features and the re-identification features; the tracking state includes tracking success or tracking failure. The tracking module is used to perform failure recovery processing on the target based on the current frame image if the tracking status is tracking failure, and to determine the tracking result corresponding to the target. Specifically, it is used to perform a failure recovery operation, which includes: identifying and extracting features from each candidate object in the current frame image to obtain candidate features for each candidate object; determining the tracking result corresponding to the target based on the similarity between the initial feature and each candidate feature; wherein, if any candidate feature matches the initial feature successfully, it indicates that the recovery result of the target is successful, and tracking of the target continues; if all candidate features fail to match the initial feature, the next frame image after the current frame image is taken as the new current frame image, and the failure recovery operation is returned to be performed; the initial feature represents the feature corresponding to the target in the initial frame image; wherein, when performing the failure recovery operation to identify and extract features from each candidate object in the current frame image, a number of candidate objects with different objects and less than all objects are selected in each frame image for feature extraction.

16. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 14.

17. A target tracking system, characterized in that, include: Gimbal; A shooting device connected to the gimbal, the shooting device including the computer device as described in claim 16.

18. A gimbal, characterized in that, include: A camera device, and a computer device as described in claim 17 connected to the camera device.

19. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 14.

20. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 14.

Images