A target tracking method, terminal and computer readable storage medium

Abstract

The present invention provides a target tracking method, a terminal and a computer-readable storage medium. The target tracking method performs target detection on an image to be processed to obtain key point information of a target object and at least one detection frame associated with the target object; determines a bounding box of the target object based on the key point information; and determines a tracking box of the target object from at least one detection box based on overlapping information of each detection box and the bounding box in at least one detection box. The present application detects the key point information of the target object when detecting the target object in the image to be processed, selects the detection box of the target object based on the key point information of the target object, thereby improving the reliability of the detection box of the target object, and determines the tracking box of the target object in a reliable tracking box, thereby improving the accuracy of target tracking and improving the instability of target object tracking.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a target tracking method, a terminal and a computer-readable storage medium. Background Art

[0002] Target tracking is a comprehensive application technology that integrates advanced achievements in various fields such as image processing, pattern recognition, artificial intelligence, automatic control, sensors, and navigation and positioning. It is a widely used technology in military and civilian fields. At present, the tracking of targets is mainly based on visual target detection, recognition, and tracking processing. Target detection and recognition mainly locates and classifies targets, while target tracking processing continuously assigns the same ID to the same target. When detecting targets in video frames, even if the entire video screen remains unchanged, the target stays in place, the aspect ratio of the detected targets in different video frames will change, and the position of the detected targets will also be different. Since the information such as the position and aspect ratio of the detected target is inaccurate, the result of target tracking is affected. Summary of the invention

[0003] The main technical problem solved by the present invention is to provide a target tracking method, a terminal and a computer-readable storage medium to solve the problem of unstable target tracking caused by low target detection accuracy in the prior art.

[0004] To solve the above technical problems, the first technical solution adopted by the present invention is: to provide a target tracking method, which includes: performing target detection on the image to be processed to obtain key point information of the target object and at least one detection frame associated with the target object; determining the circumscribed frame of the target object based on the key point information; and determining the tracking frame of the target object from at least one detection frame based on the overlapping information of each detection frame and the circumscribed frame in at least one detection frame.

[0005] Wherein, based on the overlapping information of each detection frame and the external frame in at least one detection frame, determining the tracking frame of the target object from the at least one detection frame includes: based on the overlapping information of each detection frame and the external frame in the at least one detection frame, screening out the detection frames whose overlapping information meets the preset overlapping conditions; and determining the tracking frame of the target object from the screened out detection frames.

[0006] The overlapping information includes an intersection-and-union ratio between a corresponding detection frame and an external frame; a detection frame that meets a preset overlapping condition includes a detection frame whose intersection-and-union ratio is greater than an intersection-and-union ratio threshold.

[0007] The key point information includes the contour key points of the target object.

[0008] Among them, determining the tracking frame of the target object from each filtered detection frame includes: grouping the filtered detection frames based on the overlapping information of every two detection frames in the filtered detection frames to obtain a detection frame group; matching each target object with the detection frame group respectively; and selecting the tracking frame of the target object in the image to be processed from the detection frame group matching the target object.

[0009] Among them, based on the overlapping information of every two detection frames in the filtered detection frames, the filtered detection frames are grouped to obtain a detection frame group, including: selecting one detection frame from the filtered detection frames, and determining the degree of overlap between the selected detection frame and other detection frames; clustering the detection frames corresponding to the overlap degrees exceeding the preset overlap degrees with the selected detection frames.

[0010] Among them, the image to be processed includes the current frame image in the video stream; each target object is matched with the detection frame group respectively, including: selecting a detection frame in each of the detection frame groups; determining the intersection and union ratio of the selected detection frame and the tracking frame in the previous frame image of the current frame image; selecting the detection frame group where the detection frame corresponding to the maximum intersection and union ratio is located and matching it with the target object corresponding to the tracking frame.

[0011] Among them, the image to be processed includes the current frame image in the video stream; the step of selecting the tracking frame of the target object in the image to be processed from the detection frame group matching the target object specifically includes: for each target object, performing the following operations respectively: determining a tracking frame of the target object in a reference video frame, the reference video frame includes a historical video frame before the current frame image in the video stream; based on the similarity between each detection frame in the detection frame group matching the target object and the tracking frame of the target object in the reference video frame, determining the tracking frame of the target object in the current frame image from the detection frame group.

[0012] Among them, based on the degree of similarity between each detection frame in the detection frame group matching the target object and the tracking frame of the target object in the reference video frame, the step of determining the tracking frame of the target object in the current frame image from the detection frame group specifically includes: predicting the first prediction frame of the target object in the current frame image according to the tracking frame of the target object in the reference video frame; respectively determining the loss value of each detection frame in the detection frame group and the first prediction frame; and selecting the detection frame corresponding to the smallest loss value as the tracking frame of the target object in the current frame image.

[0013] Among them, the loss value includes corner point offset loss, center point offset loss and area loss; the step of calculating the loss value of each detection frame in the detection frame group and the first prediction frame specifically also includes: weighted summing the corner point offset loss, center point offset loss and area loss to obtain the loss value.

[0014] Among them, the image to be processed includes the current frame image in the video stream; the target tracking method also includes: judging whether the degree of deviation of the position area of ​​the tracking frame of the target object in the current frame image reaches a threshold; if it does not exceed the threshold, determining that the target object is in a stationary state; if it exceeds the threshold, determining that the target object is in a moving state.

[0015] Among them, the key point information includes the central key point of the target object, and it is determined whether the offset degree of the position area of ​​the tracking box of the target object in the current frame image reaches a threshold, including: based on each frame image of the target object in the historical video frames before the current frame image in the video stream, determine the average position coordinates of the central key point of the target object; set a preset range according to the average position coordinates; determine whether the coordinates of the central key point of the target object in the current frame image meet the preset range; if it does not exceed the threshold, determine that the target object is in a stationary state, including: if the coordinates of the central key point of the target object meet the preset range, determine that the target object is in a stationary state; if it exceeds the threshold, determine that the target object is in a moving state, including: if the coordinates of the central key point of the target object do not meet the preset range, determine that the target object is in a moving state.

[0016] If the threshold is not exceeded, it is determined that the target object is in a stationary state, and it also includes: if the coordinates of the central key point of the target object meet the preset range, the tracking frame of the target object in the previous frame image is used as the tracking frame of the target object in the current frame image for updating.

[0017] Among them, if the threshold is exceeded, it is determined that the target object is in motion, and it also includes: if the coordinates of the central key point of the target object do not meet the preset range, determining the intersection-and-union ratio of the tracking frame of the target object in the previous frame image and the tracking frame of the target object in the current frame image; if the intersection-and-union ratio of the tracking frame of the target object in the previous frame image and the tracking frame of the target object in the current frame image exceeds the preset intersection-and-union ratio, the tracking frame of the target object in the current frame image is retained and updated.

[0018] Among them, before the step of performing target detection on the image to be processed to obtain the key point information of the target object and at least one detection frame associated with the target object, it also includes: using a target detection network model to perform target detection on the image to be processed; wherein the target detection network model is trained based on the key point information of the target.

[0019] Among them, the training process of the target detection network model includes: obtaining training samples, the training samples include a training video stream composed of the same image containing the target, the image includes a real frame of the labeled target and the real key points of the target; performing target detection on the image through the initial target detection network model to obtain the second predicted frame of the target and the predicted key points of the target; constructing a loss function through the real frame of the target and the second predicted frame of the target, the real key points of the target and the predicted key points of the target; and using the loss function to iteratively train the initial target detection network model to obtain the target detection network model.

[0020] To solve the above technical problems, the second technical solution adopted by the present invention is: to provide a terminal, which includes a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor is used to execute program data to implement the steps in the above target tracking method.

[0021] In order to solve the above technical problems, the third technical solution adopted by the present invention is: providing a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above target tracking method are implemented.

[0022] The beneficial effects of the present invention are as follows: Different from the prior art, a target tracking method, a terminal and a computer-readable storage medium are provided, wherein the target tracking method performs target detection on an image to be processed to obtain key point information of a target object and at least one detection frame associated with the target object; determines a bounding box of the target object based on the key point information; and determines a tracking box of the target object from at least one detection box based on overlapping information of each detection box and the bounding box in at least one detection box. The present application detects the key point information of the target object when detecting the target object in the image to be processed, selects the detection box of the target object based on the key point information of the target object, thereby improving the reliability of the detection box of the target object, and determines the tracking box of the target object in a reliable tracking box, thereby improving the accuracy of target tracking and improving the instability of target object tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 It is a flow chart of the target tracking method provided by the present invention;

[0025] Figure 2It is a flowchart of a specific embodiment of the target tracking method provided by the present invention;

[0026] Figure 3 yes Figure 2 A flowchart of a specific embodiment of step S21 in the target tracking method provided;

[0027] Figure 4 is a picture of a training sample annotated in the target tracking method provided by the present invention;

[0028] Figure 5 The present invention provides a method for performing target detection on the current frame image to obtain a picture of a corresponding detection frame;

[0029] Figure 6 yes Figure 2 A flowchart of a specific embodiment of step S25 in the target tracking method provided;

[0030] Figure 7 is a picture of a tracking frame of a target object in a current frame image provided by the present invention;

[0031] Figure 8 It is a flowchart of a specific embodiment of detecting and tracking a target object provided by the present invention;

[0032] Fig. 9 is a schematic block diagram of an implementation manner of a terminal provided by the present invention;

[0033] Fig.10 It is a schematic block diagram of an embodiment of a computer-readable storage medium provided by the present invention. DETAILED DESCRIPTION

[0034] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0035] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0036] The term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this article means two or more than two.

[0037] In order to enable those skilled in the art to better understand the technical solution of the present invention, a target tracking method provided by the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0038] See also Figure 1 , Figure 1 : is a flow chart of the target tracking method provided by the present invention. In this embodiment, a target tracking method is provided, and the target tracking method comprises the following steps.

[0039] S11: Performing target detection on the image to be processed to obtain key point information of the target object and at least one detection box associated with the target object.

[0040] Specifically, the image to be processed includes a current frame image in a video stream; the target object in the current frame image may be one or more. In a specific embodiment, a target detection network model is used to perform target detection on the current frame image in the video stream; wherein the target detection network model is trained based on the key point information of the target object. The target detection network model is used to perform target detection on the image to be processed to obtain the contour key points and center key points of the target object. And at least one detection frame associated with the target object is detected. Among them, one detection frame can be detected for one target object, and multiple detection frames can also be detected.

[0041] S12: Determine the bounding box of the target object according to the key point information.

[0042] Specifically, since the key point information includes the contour key points of the target object, the contour key points of the target object can be four or five. The number of contour key points can be set according to the different target objects. And the detected contour key points are at the edge of the target object, and the bounding box of the target object can be obtained according to the four contour key points.

[0043] S13: Determine a tracking frame of the target object from the at least one detection frame based on overlapping information between each detection frame and the circumscribed frame in the at least one detection frame.

[0044] Specifically, based on the overlapping information of each detection frame and the bounding box in at least one detection frame, the detection frames whose overlapping information meets the preset overlapping condition are screened out; and the tracking frame of the target object is determined from each screened detection frame. The overlapping information includes the intersection-and-union ratio of the corresponding detection frame and the bounding box; the detection frame that meets the preset overlapping condition includes the detection frame whose intersection-and-union ratio is greater than the intersection-and-union ratio threshold.

[0045] In a specific embodiment, based on the overlapping information of every two detection frames in the filtered detection frames, the filtered detection frames are grouped to obtain detection frame groups; each target object is matched with the detection frame group respectively; and the tracking frame of the target object in the image to be processed is selected from the detection frame group that matches the target object. In one embodiment, a detection frame in the filtered detection frames is selected, and the degree of overlap between the selected detection frame and other detection frames is determined; the detection frames corresponding to the overlap degrees exceeding the preset overlap degrees are clustered with the selected detection frames. Among them, when the target object is matched with the detection frame group, a detection frame is selected from each of the detection frame groups; the intersection and union ratio of the selected detection frame and the tracking frame in the previous frame image of the current frame image is determined; the detection frame group corresponding to the detection frame with the maximum intersection and union ratio is selected to match the target object corresponding to the tracking frame. For each target object, the following operations are performed respectively: determining a tracking frame of the target object in a reference video frame, the reference video frame including the historical video frames before the current frame image in the video stream; based on the similarity between each detection frame in the detection frame group matching the target object and the tracking frame of the target object in the reference video frame, determining the tracking frame of the target object in the current frame image from the detection frame group.

[0046] In one embodiment, a first prediction frame of the target object in the current frame image is predicted based on the tracking frame of the target object in the reference video frame; the loss value between each detection frame in the detection frame group and the first prediction frame is determined respectively; the detection frame corresponding to the minimum loss value is selected as the tracking frame of the target object in the current frame image. The loss value includes corner point offset loss, center point offset loss and area loss; the corner point offset loss, center point offset loss and area loss are weighted and summed to obtain the loss value.

[0047] In an optional embodiment, it is determined whether the degree of deviation of the position area of ​​the tracking frame of the target object in the current frame image reaches a threshold; if it does not exceed the threshold, it is determined that the target object is in a stationary state; if it exceeds the threshold, it is determined that the target object is in a moving state. In a specific embodiment, based on each frame image containing the target object in the historical video frame before the current frame image in the video stream, the average position coordinates of the central key point of the target object are determined; a preset range is set according to the average position coordinates; it is determined whether the coordinates of the central key point of the target object in the current frame image meet the preset range; if the coordinates of the central key point of the target object meet the preset range, it is determined that the target object is in a stationary state. If the coordinates of the central key point of the target object meet the preset range, the tracking frame of the target object in the previous frame image is updated as the tracking frame of the target object in the current frame image. If the coordinates of the central key point of the target object do not meet the preset range, it is determined that the target object is in a moving state. If the coordinates of the central key point of the target object do not meet the preset range, the intersection and union ratio of the tracking frame of the target object in the previous frame image and the tracking frame of the target object in the current frame image is determined; if the intersection and union ratio of the tracking frame of the target object in the previous frame image and the tracking frame of the target object in the current frame image exceeds the preset intersection and union ratio, the tracking frame of the target object in the current frame image is retained and updated.

[0048] The target tracking method provided in this embodiment performs target detection on the processed image to obtain key point information of the target object and at least one detection frame associated with the target object; determines the bounding box of the target object based on the key point information; and determines the tracking frame of the target object from at least one detection frame based on the overlapping information of each detection frame and the bounding box in at least one detection frame. This application detects the target object in the processed image, obtains the key point information of the target object, filters the detection frame of the target object based on the key point information of the target object, thereby improving the reliability of the target object detection frame, and determines the tracking frame of the target object in a reliable tracking frame, which can improve the accuracy of target tracking and improve the instability of target object tracking.

[0049] See also Figure 2 , Figure 2 1 is a flow chart of a specific embodiment of a target tracking method provided by the present invention. In this embodiment, a target tracking method is provided, and the target tracking method includes the following steps.

[0050] S21: Train and obtain the target detection network model.

[0051] Specifically, the target detection network model is trained based on the key point information of the target. Figure 3 , Figure 3 yes Figure 2The flow chart of step S21 in the target tracking method is provided as a specific embodiment. The training steps of the specific target detection network model are as follows.

[0052] S211: Obtain training samples.

[0053] Specifically, in order to improve the detection stability of the target detection network model, the training samples include a training video stream composed of the same image containing the target, and the image includes a real frame of the marked target and the real key points of the target. The real key points of the marked target are just close to the outline of the target, and all targets contained in the image must be marked to avoid missing labels. The targets in the training samples are vehicles or people. The real key points include real center key points and real outline key points. In one embodiment, it is also necessary to mark the real category of the target and the real position coordinates of the target. It is also necessary to mark the real distance between the real outline key points of the target and the corner points of the corresponding real frame, as well as the distance between the real center key point of the target and the center point of the real frame. Please refer to Figure 4 , Figure 4 is a picture of a training sample annotated in the target tracking method provided by the present invention. In a specific embodiment, the rectangular frame is the annotated real frame, K0, K1, K2, K3 are the real contour key points of the annotated target, K4 is the real center key point of the annotated target, and point C is the center point of the annotated target real frame.

[0054] S212: Performing target detection on the image through the initial target detection network model to obtain a second predicted box of the target and predicted key points of the target.

[0055] Specifically, the initial target detection network model detects each image in the training video stream to obtain the second prediction box of the target in the image and the predicted key points of the target. Among them, the second prediction box selects the target itself in the detected image, and the predicted key points of the target are the predicted contour key points of the target and the predicted center key points of the target. In a specific embodiment, target detection is performed on the image based on the YOLO (You Only Look Once)-v3 network structure. In an optional embodiment, the initial target detection network also detects the confidence of the second prediction box of the target, as well as the predicted category of the predicted target and the predicted position coordinates of the target.

[0056] S213: Construct a loss function through the real box of the target and the second predicted box of the target, and the real key points of the target and the predicted key points of the target.

[0057] Specifically, the prediction box prediction loss is calculated based on the predicted second prediction box of the target and the real box of the target, the key point prediction loss is calculated based on the predicted key points of the target and the real key points of the target, and then the loss function is constructed based on the prediction loss of the prediction box and the prediction loss of the key points. The key point prediction loss includes the center key point loss and the contour key point loss of the target. In one embodiment, the loss function can be constructed by the loss between the real category and the predicted category, the loss between the predicted distance between the predicted contour key point of the target and each corner point corresponding to the second prediction box and the real distance between the real contour key point of the target and the corresponding corner point of the real box, and the loss between the distance between the predicted center key point of the target and the center point of the second prediction box and the distance between the real center key point of the target and the center point of the real box. Due to the large difference in vehicle angles, there will be differences in the predicted distance between the target's predicted contour key points and the corresponding corner points of the second prediction box and the actual distance loss between the target's true contour key points and the corresponding corner points of the true box, and the distance between the target's predicted center key point and the center point of the second prediction box and the distance loss between the target's true center key point and the center point of the true box. In order to balance this difference, the square is used in the distance calculation to convert the distance loss into a positive number, and then the logarithm is used to narrow the distance difference between different points.

[0058] The specific loss function is shown in Formula 1.

[0059]

[0060] in, represents the prediction box coordinate loss of the target, λ coord It only indicates the meaning of the coordinates; Indicates the target width and height loss corresponding to the target prediction box. The previous λ coord It only indicates the meaning of the coordinates; Represents the confidence loss of the predicted target; Represents the confidence loss of predicting non-targets; represents the category loss of target prediction; represents the position loss of the five predicted key points, λ keypoint It only indicates the meaning of the coordinates; Represents the distance loss from the predicted central key point to the center point of the predicted box, μ distance It only indicates the meaning of distance;

[0061]

[0062] represents the distance loss from the four predicted contour key points to the corresponding corner points of the predicted box, μ distance It only means the distance; S is the grid dimension of the detection network division, and B is the dimension of the number of predicted targets.

[0063] By introducing contour key points and center key points to train the initial detection network model, not only can the initial detection network model predict the predicted contour key points and predicted center key points of the target, but also the combination of contour key points and center key points can promote the initial detection network model to be more stable.

[0064] By adding the predicted distance between the target's predicted contour key points and the corresponding corner points of the second prediction box and the actual distance loss between the target's true contour key points and the corresponding corner points of the true box into the loss function, the initial detection network model is made to predict the target's width and height more stably; by adding the distance between the target's predicted center key point and the center point of the second prediction box and the distance loss between the target's true center key point and the center point of the true box, the initial detection network model is made to predict the target's center point more stably.

[0065] S214: Using the loss function to iteratively train the initial target detection network model to obtain the target detection network model.

[0066] Specifically, the initial target detection network model is iteratively trained through the error value between the real frame of the target and the second predicted frame of the target, and the error value between the real key point of the target and the predicted key point of the target to obtain the target detection network model.

[0067] In an optional embodiment, the result of the initial target detection network model is back-propagated, and the weight of the initial target detection network model is modified according to the loss value fed back by the loss function. In an optional embodiment, the parameters in the initial target detection network model can also be modified to train the initial target detection network model to obtain the target detection network model.

[0068] The image is input into the initial target detection network model, and the initial target detection network model predicts the target position and the target key points in the image. When the sum of the error value between the target's true frame and the target's second predicted frame and the error value between the target's true key point and the target's predicted key point is less than a preset threshold value, which can be set by itself, such as 1%, 5%, etc., the training of the initial target detection network model is stopped and the target detection network model is obtained.

[0069] S22: Use the target detection network model to detect the acquired current frame image to obtain key point information of the target object and at least one detection box associated with the target object.

[0070] Specifically, the target detection network model obtained through training detects the acquired current frame image to obtain the detection frame of the target object and the contour key points and center key points of the target object. Among them, the target object can be one or more. The detection frame detected corresponding to a target object can be one or more. In a specific embodiment, the size of the current frame image collected is 416×416, then the number of original detection frames output by the target detection network model can be 13*13*3+26*26*3+52*52*3=10647.

[0071] S23: Determine the bounding box of the target object according to the key point information.

[0072] Specifically, since the key point information includes the contour key points of the target object, there may be four contour key points of the target object, and the detected contour key points are located at the edge of the target object, the bounding box of the target object can be obtained according to the four contour key points.

[0073] S24: Based on the overlapping information between each detection frame and the circumscribed frame in at least one detection frame, the detection frames whose overlapping information meets the preset overlapping condition are screened out.

[0074] Specifically, since most of the detection frames obtained by the target detection network model are invalid detection frames, it is necessary to filter the detection frames obtained. The specific filtering rule is to compare the overlapping information of the detection frame and the external frame with the threshold. The overlapping information includes the intersection of the corresponding detection frame and the external frame; the threshold in this step is the intersection of ... Figure 5 , Figure 5 The present invention provides a picture of a corresponding detection frame obtained by performing target detection on the current frame image. The detection frames screened by overlapping information include multiple ones.

[0075] S25: Determine a tracking frame of the target object from the screened detection frames.

[0076] Specifically, there may be one or two target objects.

[0077] See also Figure 6 and Figure 7 , Figure 6 yes Figure 2 A flowchart of a specific embodiment of step S25 in the target tracking method provided; Figure 7is a picture of the tracking frame of the target object in the current frame image provided by the present invention. In a specific embodiment, the steps of determining the tracking frame of the target object from the screened detection frames are as follows.

[0078] S251: Based on the overlapping information of every two detection frames in the filtered detection frames, the filtered detection frames are grouped to obtain a detection frame group.

[0079] Specifically, a detection frame among the filtered detection frames is selected to determine whether the category of the target object selected by the detection frame is the same as that of the other detection frame; if the categories are the same, the overlap degree between the selected detection frame and the other detection frames is determined; the detection frames corresponding to the overlap degree exceeding the preset overlap degree are clustered with the selected detection frame to obtain a detection frame group. The overlap degree includes the intersection-over-union ratio. A unique group ID (Identity) is assigned to each detection frame group.

[0080] S252: Match each target object with the detection frame group respectively.

[0081] Specifically, a detection frame is selected from each detection frame group, and the selected detection frame is matched with the target object. In a specific embodiment, the intersection and union ratio of the selected detection frame and the tracking frame in the previous frame image of the current frame image is calculated; the detection frame group corresponding to the detection frame with the maximum intersection and union ratio is selected to match the target object corresponding to the tracking frame. When a detection frame group is obtained, a detection frame is selected from the detection frame group, and the intersection and union ratio of the detection frame and all the detection frames in the previous frame image are calculated respectively, and the target object corresponding to the tracking frame corresponding to the maximum intersection and union ratio is selected to match the detection frame. When multiple detection frame groups are obtained, a detection frame is selected from each detection frame group, and the intersection and union ratio of the selected detection frame and all the tracking frames in the previous frame image are calculated respectively, and the target object corresponding to the tracking frame corresponding to the maximum intersection and union ratio is matched with the detection frame group to which the detection frame belongs.

[0082] S253: Selecting a tracking frame of the target object in the image to be processed from the detection frame group that matches the target object.

[0083] Specifically, a tracking frame of a target object in a reference video frame is determined, where the reference video frame includes historical video frames before a current frame image in a video stream; based on the degree of similarity between each detection frame in a detection frame group matching the target object and the tracking frame of the target object in the reference video frame, a tracking frame of the target object in the current frame image is determined from the detection frame group.

[0084] Specifically, the first prediction frame of the target object in the current frame image is predicted based on the tracking frame of the target object in the reference video frame; the loss value of each detection frame in the detection frame group and the first prediction frame is determined respectively; the detection frame corresponding to the smallest loss value is selected as the tracking frame of the target object in the current frame image. The loss value includes corner point offset loss, center point offset loss and area loss; the loss value is obtained by weighted summing the corner point offset loss, center point offset loss and area loss. Among them, the weight parameters corresponding to the corner point offset loss, center point offset loss and area loss are α, β, and γ, and the weight parameters are set to balance the loss. In this embodiment, α=0.5, β=0.5, and γ=0.1, which can also be set according to the specific situation.

[0085] The detection frame group is obtained by grouping the multiple detection frames obtained in step S24, and matching the corresponding target objects, and then selecting the tracking frame of the target object in the detection frame group, such as Figure 7 shown.

[0086] S26: Determine whether the degree of deviation of the position area of ​​the tracking frame of the target object in the current frame image reaches a threshold.

[0087] Specifically, based on each frame image of the target object in the historical video frames before the current frame image in the video stream, the average position coordinates of the central key point of the target object are determined; a preset range is set according to the average position coordinates; and it is determined whether the coordinates of the central key point of the target object in the current frame image are within the preset range, thereby determining the motion state of the target object.

[0088] If the threshold is not exceeded, it is determined that the target object is in a stationary state, and step S27 is directly executed; if the threshold is exceeded, it is determined that the target object is in a moving state, and step S28 is directly executed.

[0089] S27: updating the tracking frame of the target object in the previous frame image as the tracking frame of the target object in the current frame image.

[0090] Specifically, if the coordinates of the central key point of the target object meet the preset range, it is determined that the target object is in a stationary state. If the coordinates of the central key point of the target object meet the preset range, the tracking frame of the target object in the previous frame image is used as the tracking frame of the target object in the current frame image for updating.

[0091] S28: The tracking frame of the target object in the current frame image is retained and updated.

[0092] Specifically, if the coordinates of the central key point of the target object do not meet the preset range, it is determined that the target object is in motion. After the target object in the current frame image is updated and is in a stationary state, the detection frame in the detection frame group that is not used as a tracking frame is deleted.

[0093] If the coordinates of the central key point of the target object do not meet the preset range, the intersection and union ratio of the tracking frame of the target object in the previous frame image and the tracking frame of the target object in the current frame image is determined; if the intersection and union ratio of the tracking frame of the target object in the previous frame image and the tracking frame of the target object in the current frame image exceeds the preset intersection and union ratio, the tracking frame of the target object in the current frame image is retained and updated. Specifically, the average position of the target object is shown in Formula 2. This can prevent the instability caused by fluctuations in the current detection, so that a more stable detection frame can be found in the detection frame group.

[0094] avecenter=0.8*history+0.2*now (Formula 2)

[0095] Where avecenter is the average position value, history is the position of the center point of the target object in the historical video frame, and now is the position of the center point matched by the target object in the current frame image.

[0096] Specifically, the following updating method is adopted for the width and height, and see Formula 3 for details.

[0097] Nowtracker=0.5*historytracker+0.5*nowdetect (Formula 3)

[0098] Among them, Historytracker is the width and height of the target object in the historical video frame; Nowdetect is the width and height of the target object detected in the current frame image; Nowtracker is the width and height of the target object in the current frame image.

[0099] See also Figure 8 , Figure 8It is a flowchart of a specific embodiment of the detection and tracking of a target object provided by the present invention. In a specific embodiment, an image containing a vehicle is obtained, and a target detection network model is used to detect the vehicle in the image to obtain the contour key points and center key points of the vehicle, the detection frame of the vehicle, and the confidence of the corresponding detection frame. The external frame of the vehicle is obtained according to the contour key points of the vehicle, and the detection frame of the detected vehicle is screened according to the external frame of the vehicle and the confidence of the detection frame of the vehicle, and reliable and stable detection frames are retained. The retained detection frames are classified to obtain detection frame groups, and each detection frame group is matched with each vehicle respectively. When the target object is matched to the corresponding detection frame group, the tracking frame of the corresponding vehicle in the current frame image is selected from the detection frame group. The motion state of the vehicle is judged based on the center key point of the vehicle, and then a stable tracking result is returned based on the motion state of the vehicle to obtain a stable tracking frame of the vehicle in the current frame image.

[0100] The target tracking method provided in this embodiment performs target detection on the processed image to obtain key point information of the target object and at least one detection frame associated with the target object; determines the bounding box of the target object based on the key point information; and determines the tracking frame of the target object from at least one detection frame based on the overlapping information of each detection frame and the bounding box in at least one detection frame. This application detects the target object in the processed image, obtains the key point information of the target object, filters the detection frame of the target object based on the key point information of the target object, thereby improving the reliability of the target object detection frame, and determines the tracking frame of the target object in a reliable tracking frame, which can improve the accuracy of target tracking and improve the instability of target object tracking.

[0101] See also Fig. 9 , Fig. 9 1 is a schematic block diagram of an embodiment of a terminal provided by the present invention. The terminal 70 in this embodiment includes: a processor 71, a memory 72, and a computer program stored in the memory 72 and executable on the processor 71. When the computer program is executed by the processor 71, the above-mentioned object detection method is implemented. To avoid repetition, it is not described one by one here.

[0102] See also Fig.10 , Fig.10 It is a schematic block diagram of an embodiment of a computer-readable storage medium provided by the present invention.

[0103] A computer-readable storage medium 90 is also provided in an embodiment of the present application. The computer-readable storage medium 90 stores a computer program 901. The computer program 901 includes program instructions. The processor executes the program instructions to implement the object detection method provided in the embodiment of the present application.

[0104] The computer-readable storage medium 90 may be an internal storage unit of the computer device of the aforementioned embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium 90 may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device.

[0105] The above are only implementation modes of the present invention, and are not intended to limit the patent protection scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A target tracking method, characterized in that: The target tracking method comprises: Performing target detection on the image to be processed to obtain key point information of the target object and at least one detection box associated with the target object; Determining a bounding box of the target object based on the key point information; determining a tracking frame of the target object from the at least one detection frame based on overlapping information between each detection frame and the circumscribed frame in the at least one detection frame; The determining of the tracking frame of the target object from the at least one detection frame based on overlap information between each detection frame and the bounding box in the at least one detection frame includes: Based on the overlapping information between each detection frame in the at least one detection frame and the circumscribed frame, screening out the detection frames whose overlapping information meets a preset overlapping condition; Based on the overlapping information of every two detection frames in the filtered detection frames, the filtered detection frames are grouped to obtain a detection frame group; Matching each of the target objects with the detection frame group respectively; The tracking frame of the target object in the image to be processed is selected from the detection frame group that matches the target object.

2. The target tracking method according to claim 1, characterized in that The overlap information includes an intersection-over-union ratio between the corresponding detection frame and the circumscribed frame; The detection frame that meets the preset overlapping condition includes the detection frame whose intersection-over-union ratio is greater than an intersection-over-union ratio threshold.

3. The target tracking method according to claim 1, wherein: The key point information includes the contour key points of the target object.

4. The target tracking method according to claim 1, wherein: The step of grouping the filtered detection frames based on the overlapping information of every two detection frames in the filtered detection frames to obtain a detection frame group includes: selecting one of the filtered detection frames, and determining a degree of overlap between the selected detection frame and the other detection frames; The detection frames corresponding to the overlap degrees exceeding the preset overlap degree are clustered with the selected detection frames.

5. The target tracking method according to claim 1, wherein: The image to be processed includes a current frame image in the video stream; Matching each of the target objects with the detection frame groups includes: Selecting one detection frame from each detection frame group; Determine an intersection-over-union ratio between the selected detection frame and the tracking frame in the previous frame image of the current frame image; The detection frame group where the detection frame corresponding to the maximum intersection-over-union ratio is located is selected to match the target object corresponding to the tracking frame.

6. The target tracking method according to claim 1, characterized in that: The image to be processed includes a current frame image in the video stream; The step of selecting the tracking frame of the target object in the image to be processed from the detection frame group that matches the target object includes: For each target object, perform the following operations: Determine a tracking frame of the target object in a reference video frame, where the reference video frame includes a historical video frame before the current frame image in the video stream; Based on the degree of similarity between each detection frame in the detection frame group that matches the target object and the tracking frame of the target object in the reference video frame, the tracking frame of the target object in the current frame image is determined from the detection frame group.

7. The target tracking method according to claim 6, characterized in that: The determining, based on a degree of similarity between each detection frame in the detection frame group that matches the target object and the tracking frame of the target object in the reference video frame, the tracking frame of the target object in the current frame image from the detection frame group includes: Predicting a first prediction frame of the target object in the current frame image according to the tracking frame of the target object in the reference video frame; Determining a loss value between each detection frame in the detection frame group and the first prediction frame respectively; The detection frame corresponding to the minimum loss value is selected as the tracking frame of the target object in the current frame image.

8. The target tracking method according to claim 7, characterized in that: The loss values ​​include corner point offset loss, center point offset loss and area loss; The step of calculating the loss value between each detection frame in the detection frame group and the first prediction frame specifically further includes: The loss value is obtained by performing a weighted summation on the corner point offset loss, the center point offset loss, and the area loss.

9. The target tracking method according to claim 1, characterized in that: The image to be processed includes a current frame image in the video stream; The target tracking method further includes: Determining whether a degree of deviation of a position area of ​​the tracking frame of the target object in the current frame image reaches a threshold; If the threshold is not exceeded, determining that the target object is in a stationary state; If the threshold is exceeded, it is determined that the target object is in motion.

10. The target tracking method according to claim 9, characterized in that: The key point information includes the central key point of the target object, The determining whether the offset degree of the position area of ​​the tracking frame of the target object in the current frame image reaches a threshold includes: Determining the average position coordinates of the central key point of the target object based on each frame of the historical video frames before the current frame image in the video stream that contains the target object; Setting a preset range according to the average position coordinates; Determining whether the coordinates of the central key point of the target object in the current frame image are within the preset range; If the threshold value is not exceeded, determining that the target object is in a stationary state includes: If the coordinates of the central key point of the target object are within the preset range, it is determined that the target object is in a stationary state; If the threshold is exceeded, determining that the target object is in motion includes: If the coordinates of the central key point of the target object do not conform to the preset range, it is determined that the target object is in motion.

11. The target tracking method according to claim 10, characterized in that: If the threshold is not exceeded, determining that the target object is in a stationary state further includes: If the coordinates of the central key point of the target object are within the preset range, the tracking frame of the target object in the previous frame image is used as the tracking frame of the target object in the current frame image for updating.

12. The target tracking method according to claim 10, characterized in that: If the threshold is exceeded, determining that the target object is in motion further includes: If the coordinates of the central key point of the target object do not meet the preset range, determining an intersection-over-union ratio between the tracking frame of the target object in the previous frame image and the tracking frame of the target object in the current frame image; If the intersection-of-union (IoU) of the tracking frame of the target object in the previous frame image and the tracking frame of the target object in the current frame image exceeds a preset IoU, the tracking frame of the target object in the current frame image is retained and updated.

13. The target tracking method according to claim 1, characterized in that: Before the step of performing target detection on the image to be processed to obtain key point information of the target object and at least one detection frame associated with the target object, the following steps are further included: Performing target detection on the image to be processed using a target detection network model; The target detection network model is trained based on the key point information of the target.

14. The target tracking method according to claim 13, wherein: The training process of the target detection network model includes: Acquire a training sample, wherein the training sample includes a training video stream composed of the same image containing the target, wherein the image includes a real box annotating the target and real key points of the target; Performing target detection on the image using an initial target detection network model to obtain a second predicted frame of the target and predicted key points of the target; Constructing a loss function by combining the true frame of the target and the second predicted frame of the target, and the true key points of the target and the predicted key points of the target; The initial target detection network model is iteratively trained using the loss function to obtain the target detection network model.

15. A terminal, characterized in that: The terminal includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor is configured to execute program data to implement the steps of the target tracking method according to any one of claims 1 to 14.

16. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the target tracking method according to any one of claims 1 to 14 are implemented.

Images