Target tracking method, target tracking device, electronic device and storage medium

Abstract

The present disclosure relates to a target tracking method, a target tracking device, an electronic device and a storage medium. The target tracking method includes determining at least one first local area used to track a target to be tracked; determining the association relationship between the first local area and the entire target to be tracked; based on the first local area, performing local tracking between multiple image frames to obtain local tracking information; and obtaining global tracking information of the target to be tracked in different image frames based on the local tracking information and the association relationship. The target tracking method disclosed in the present disclosure can achieve target tracking more robustly by performing local tracking in the first local area to improve the success rate of target tracking.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a target tracking method, a target tracking device, an electronic device and a storage medium. Background Art

[0002] The target tracking problem is an important research direction in the field of computer vision. The target tracking problem can be divided into single target tracking and multi-target tracking according to the number of targets to be tracked. According to the complexity of the tracking process, it can be divided into short-term target tracking and long-term target tracking. At present, the research on target tracking has made many progress and breakthroughs, but there are still many problems to be solved. For example, in the process of long-term target tracking, the target is easily lost. Summary of the invention

[0003] In view of this, the embodiments of the present disclosure are intended to provide a target tracking method, a target tracking device, an electronic device, and a storage medium.

[0004] The technical solution of the present disclosure is achieved as follows:

[0005] In one aspect, the present disclosure provides a target tracking method.

[0006] The target tracking method provided by the embodiment of the present disclosure includes:

[0007] Determine at least one first local area used when tracking the target to be tracked;

[0008] Determining an association relationship between the first local area and the entire target to be tracked;

[0009] Based on the first local area, local tracking is performed between multiple image frames to obtain local tracking information;

[0010] The global tracking information of the target to be tracked in different image frames is obtained according to the local tracking information and the association relationship.

[0011] In some embodiments, determining at least one first local area used when tracking the target to be tracked includes:

[0012] determining at least one candidate local area for target tracking;

[0013] According to the statistical probability of each candidate local area being blocked, at least one candidate local area whose statistical probability of being blocked meets the first condition is selected as the first local area.

[0014] In some embodiments, the determining of at least one first local area used when tracking the target to be tracked further includes:

[0015] Determining the tracking quality of the local tracking information of the target to be tracked based on the first local area whose statistical probability of being blocked satisfies the first condition;

[0016] A local area whose tracking quality is not lower than a preset quality threshold is selected as the updated first local area.

[0017] In some embodiments, determining at least one candidate local area for target tracking includes:

[0018] At least one candidate local area for target tracking is determined according to characteristic attributes of different local areas of the target to be tracked, wherein the characteristic attributes include: the number of features and / or a significance parameter of a single feature.

[0019] In some embodiments, selecting at least one candidate local area whose statistical probability of being blocked meets a first condition as the first local area according to the statistical probability of each candidate local area being blocked includes:

[0020] According to the statistical probability of the candidate local area being blocked, a candidate local area whose statistical probability of being blocked meets the first condition and whose initial image frame is not blocked among the multiple image frames is selected as the first local area.

[0021] In some embodiments, the number of the first local areas is N, where N is a positive integer equal to or greater than 2 and less than P; P is the number of local areas included in the target to be tracked;

[0022] The performing local tracking between multiple image frames based on the first local area to obtain local tracking information includes:

[0023] Among the multiple image frames, local tracking is performed in the mth image frame based on the first local area that is not blocked in the m-1th image frame to obtain local tracking information of the target to be tracked in the mth image frame, wherein the mth image frame is the next image frame of the m-1th image frame, and m is a positive integer greater than 1.

[0024] In some embodiments, the method further comprises:

[0025] When it is detected that part of the first local areas are blocked in one or more of the image frames, unblocked local areas are selected as the updated N first local areas according to the blocking conditions.

[0026] In some embodiments, the selecting the unobstructed local areas as the updated N first local areas includes:

[0027] Determining the perspective relationship between different targets in each of the image frames; wherein the targets at least include the target to be tracked;

[0028] Determining a foreground target of the target to be tracked according to the perspective relationship;

[0029] Determine whether there is an interference object in the foreground target that blocks the candidate local area in the target to be tracked;

[0030] Select unobstructed candidate local regions as the updated N first local regions.

[0031] In some embodiments, determining whether there is an interference object in the foreground target that blocks the candidate local area in the target to be tracked includes:

[0032] Determining whether the position of the foreground target overlaps with the position of the candidate local area in the target to be tracked;

[0033] If so, it is determined that the candidate local area is blocked by the interference object in the image frame.

[0034] In some embodiments, determining the perspective relationship between different targets in each of the image frames, and determining the foreground target of the target to be tracked according to the perspective relationship, includes:

[0035] Determining the bounding boxes of the different targets;

[0036] Determine a first distance between a lower edge of a bounding box of the target to be tracked and a lower edge of the image frame, and determine a second distance between a lower edge of a bounding box of a target other than the target to be tracked and a lower edge of the image frame;

[0037] If the first distance is greater than the second distance, the target corresponding to the second distance is determined to be the foreground target of the target to be tracked.

[0038] In some embodiments, obtaining the global tracking information of the target to be tracked in different image frames according to the local tracking information and the association relationship includes:

[0039] Determining a relative position relationship of the first local area in the target to be tracked according to the position information of the first local area and the position information of the bounding box of the target to be tracked;

[0040] According to the relative position relationship, the local tracking information of the first local area in each of the image frames is converted into the global tracking information of the target to be tracked in each of the image frames.

[0041] On the other hand, the present disclosure provides a target tracking device. The target tracking device provided by the embodiment of the present disclosure includes:

[0042] A first processing unit, configured to determine at least one first local area used when tracking a target to be tracked;

[0043] A second processing unit, used for determining an association relationship between the first local area and the entire target to be tracked;

[0044] A third processing unit, configured to perform local tracking between a plurality of image frames based on the first local area to obtain local tracking information;

[0045] The fourth processing unit is used to obtain the global tracking information of the target to be tracked in different image frames according to the local tracking information and the association relationship.

[0046] In some embodiments, the first processing unit is used to determine at least one first local area used when tracking the target to be tracked, including:

[0047] The first processing unit is specifically configured to determine at least one candidate local area for target tracking;

[0048] According to the statistical probability of each candidate local area being blocked, at least one candidate local area whose statistical probability of being blocked meets the first condition is selected as the first local area.

[0049] In some embodiments, the first processing unit is used to determine at least one first local area used when tracking the target to be tracked, and further includes:

[0050] The first processing unit is specifically configured to determine the tracking quality of the local tracking information of the target to be tracked based on the first local area whose statistical probability of being blocked satisfies the first condition;

[0051] A local area whose tracking quality is not lower than a preset quality threshold is selected as the updated first local area.

[0052] In some embodiments, the first processing unit is used to determine at least one candidate local area for target tracking, including:

[0053] The first processing unit is specifically configured to determine at least one candidate local area for target tracking according to feature attributes of different local areas of the target to be tracked, wherein the feature attributes include: feature quantity and / or significance parameters of a single feature.

[0054] In some embodiments, the first processing unit is configured to select, according to the statistical probability of each candidate local area being blocked, at least one candidate local area whose statistical probability of being blocked meets the first condition as the first local area, including:

[0055] The first processing unit is specifically configured to select, according to the statistical probability of the candidate local area being blocked, a candidate local area whose statistical probability of being blocked meets the first condition and whose initial image frame is not blocked in the multiple image frames as the first local area.

[0056] In some embodiments, the number of the first local areas is N, where N is a positive integer equal to or greater than 2 and less than P; P is the number of local areas included in the target to be tracked;

[0057] The third processing unit is used to perform local tracking between multiple image frames based on the first local area to obtain local tracking information, including:

[0058] The third processing unit is specifically used to perform local tracking in the mth image frame among the multiple image frames based on the first local area that is not blocked in the m-1th image frame, so as to obtain local tracking information of the target to be tracked in the mth image frame, wherein the mth image frame is the image frame after the m-1th image frame, and m is a positive integer greater than 1.

[0059] In some embodiments, a fifth processing unit is further included for selecting unobstructed local areas as the updated N first local areas according to the occlusion conditions when it is detected that part of the first local areas are obstructed in one or more of the image frames.

[0060] In some embodiments, the fifth processing unit, for selecting the unobstructed local area as the updated N first local areas, includes:

[0061] The fifth processing unit is specifically used to determine the perspective relationship between different targets in each of the image frames; wherein the targets at least include the target to be tracked;

[0062] Determining a foreground target of the target to be tracked according to the perspective relationship;

[0063] Determine whether there is an interference object in the foreground target that blocks the candidate local area in the target to be tracked;

[0064] Select unobstructed candidate local regions as the updated N first local regions.

[0065] In some embodiments, determining whether there is an interference object in the foreground target that blocks the candidate local area in the target to be tracked includes:

[0066] Determining whether the position of the foreground target overlaps with the position of the candidate local area in the target to be tracked;

[0067] If so, it is determined that the candidate local area is blocked by the interference object in the image frame.

[0068] In some embodiments, the first processing unit is used to determine the perspective relationship between different targets in each of the image frames, and determine the foreground target of the target to be tracked according to the perspective relationship, including:

[0069] The first processing unit is specifically configured to determine the external frames of the different targets;

[0070] Determine a first distance between a lower edge of a bounding box of the target to be tracked and a lower edge of the image frame, and determine a second distance between a lower edge of a bounding box of a target other than the target to be tracked and a lower edge of the image frame;

[0071] If the first distance is greater than the second distance, the target corresponding to the second distance is determined to be the foreground target of the target to be tracked.

[0072] In some embodiments, the fourth processing unit is used to obtain the global tracking information of the target to be tracked in different image frames according to the local tracking information and the association relationship, including:

[0073] The fourth processing unit is specifically configured to determine a relative position relationship of the first local area in the target to be tracked according to the position information of the first local area and the position information of the external frame of the target to be tracked;

[0074] According to the relative position relationship, the local tracking information of the first local area in each of the image frames is converted into the global tracking information of the target to be tracked in each of the image frames.

[0075] On the other hand, the present disclosure also provides an electronic device.

[0076] An electronic device provided by an embodiment of the present disclosure includes: a processor and a memory for storing a computer program that can be run on the processor, wherein the processor, when running the computer program, executes the steps of the target tracking method provided by an embodiment of the present disclosure on one hand.

[0077] In yet another aspect, the present disclosure also provides a computer-readable storage medium.

[0078] The computer-readable storage medium provided by the embodiment of the present disclosure stores a computer program thereon, and when the computer program is executed by a processor, the steps of the target tracking method provided by the embodiment of the present disclosure are implemented on the one hand.

[0079] The disclosed embodiment locally tracks the target to be tracked between multiple image frames through at least one first local area, and obtains the global tracking information of the target in different image frames based on the obtained local tracking information and the association between the first local area and the entire target to be tracked. In this way, local tracking is performed through one or more first local areas to obtain local tracking information, and global tracking information is obtained based on the local tracking information. Compared with directly tracking the target as a whole globally, the number of features that need to be identified when the target can be tracked can be reduced. At the same time, when the target to be tracked is globally tracked as a whole, it is necessary to identify and match the entire area of ​​the target to be tracked. Once a part of the area is blocked, it may be impossible to match the entire area, causing the target to be unrecognizable, and then causing the target to be lost. The target to be tracked as a whole is relatively large compared to a local area in the target to be tracked, and its probability of being partially blocked is higher than that of the local area. Relatively speaking, the area that needs to be identified and matched is smaller when performing local tracking through the first local area. For the same obstacle, the probability of the local area being blocked relative to the global area is also smaller. Therefore, by performing local tracking in the first local area to obtain global tracking information, compared with directly performing global tracking on the entire target to be tracked, target tracking can be performed more robustly to improve the target tracking success rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 is a flow chart of a target tracking method according to an exemplary embodiment;

[0081] Figure 2 is a schematic diagram of occlusion relationship judgment according to an exemplary embodiment;

[0082] Figure 3 It is a schematic diagram of the process of the target to be tracked being lost;

[0083] Figure 4 It is a schematic diagram of possible situations where the target to be tracked is blocked;

[0084] Figure 5 is a schematic diagram of target tracking learning according to an exemplary embodiment;

[0085] Figure 6 is an overall flow chart of a target tracking method according to an exemplary embodiment;

[0086] Figure 7 is a schematic diagram of the structure of a target tracking device according to an exemplary embodiment;

[0087] Figure 8 The figure is a schematic diagram showing the structure of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0088] The technical solution of the present invention is further described in detail below in conjunction with the drawings and specific embodiments of the specification. The exemplary embodiments will be described in detail here, and examples thereof are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are only examples of devices and methods consistent with some aspects of the present invention as detailed in the attached claims.

[0089] At present, when the target tracking algorithm based on neural network tracks the target as a whole, it needs to process all the pixels and image features of the target to be tracked in a large number of images. The amount of calculation is too large, which puts high demands on the performance of the processor.

[0090] The present disclosure provides a target tracking method. Figure 1 FIG. 1 is a flow chart of a target tracking method according to an exemplary embodiment. Figure 1 As shown, the target tracking method includes:

[0091] Step 10: determining at least one first local area used when tracking the target to be tracked;

[0092] Step 11: determining the association relationship between the first local area and the entire target to be tracked;

[0093] Step 12: Based on the first local area, local tracking is performed between multiple image frames to obtain local tracking information;

[0094] Step 13: Obtain global tracking information of the target to be tracked in different image frames according to the local tracking information and the association relationship.

[0095] In this exemplary embodiment, the target to be tracked may be a movable target such as a human body, an animal, or a vehicle. The specific target to be tracked is not limited to any one.

[0096] In this exemplary embodiment, the first local area may be a local area of ​​the entire target to be tracked in the image frame, including identification features used by the tracking device for tracking and identification (for example, when the target to be tracked is a human body, the first local area may be an organ tissue of the human body in the image, including the head, limbs, torso, etc.).

[0097] In this exemplary embodiment, the association relationship between the first local area and the entire target to be tracked includes the positional relationship between the first local area and the entire target to be tracked in the image and the relative local area relationship, etc. The positional relationship may include the coordinate relationship between the regional center coordinates of the first local area and the regional center coordinates of the entire target to be tracked; the relative local area relationship may include the relative relationship between the contour and area of ​​the first local area and the contour and area of ​​the entire target to be tracked, for example, the first local area is which area of ​​which position of the entire target to be tracked.

[0098] In this exemplary embodiment, multiple image frames may be from the same video, and the video may be from a road monitoring device and / or a security monitoring device, etc. Of course, this is only an example of the device from which the video comes, and the specific implementation is not limited to these example road monitoring devices and / or security monitoring devices. The security monitoring equipment includes but is not limited to: surveillance cameras in residential areas, surveillance cameras in office buildings, surveillance equipment in shopping malls, and / or surveillance equipment in non-road public places such as parks.

[0099] In this exemplary embodiment, global tracking means that the tracking device identifies and locates the target according to the overall features of the target to be tracked (i.e., the target to be tracked is regarded as a whole). For example, when the target to be tracked is a person, all the features of the person (including the head, limbs, torso, etc.) are regarded as a whole for identification and location. Local tracking means that the tracking device identifies and locates the target according to some features of the target to be tracked. For example, when the target to be tracked is a person, only the head of the person can be identified and located to determine the position of the target to be tracked.

[0100] In this exemplary embodiment, the local tracking information may be the position information (including coordinate information) of the tracked local features in the image obtained based on the tracking of the target local features and / or the area information of the local features (including the morphological contour, area, etc. constituted by the local features).

[0101] The global tracking information may be the position information (including coordinate information) of the target to be tracked as a whole in the image obtained based on the tracking of the target's overall features and / or the region information of the target as a whole (including the morphological outline and area formed by the overall features).

[0102] In this exemplary embodiment, when the target to be tracked has obvious features that are easy to track and identify, at least one first local area (having obvious features that are easy to track and identify) can be locally tracked in multiple image frames to obtain local tracking information of the first local area, and then global tracking information of the target to be tracked is obtained based on the local tracking information.

[0103] In this exemplary embodiment, when the target to be tracked does not have obvious features that are easy to track and identify, multiple different first local areas can be tracked synchronously to obtain local tracking information of the multiple first local areas, and then the global tracking information of the target to be tracked can be obtained based on the local tracking information of the multiple first local areas to improve the tracking success rate.

[0104] The disclosed embodiment locally tracks the target to be tracked between multiple image frames through at least one first local area, and obtains the global tracking information of the target to be tracked in different image frames based on the obtained local tracking information and the association between the first local area and the entire target to be tracked. In this way, local tracking is performed through one or more first local areas to obtain local tracking information, and global tracking information is obtained based on the local tracking information. Compared with directly performing global tracking on the entire target to be tracked, the number of features that need to be identified when the target can be tracked can be reduced. At the same time, when the target to be tracked is globally tracked as a whole, it is necessary to identify and match the entire area of ​​the target to be tracked. Once a part of the area is blocked, it may be impossible to match the entire area, causing the target to be tracked to be unrecognizable, thereby causing the target to be lost. The target as a whole is relatively large compared to a local area in the target to be tracked, and its probability of being partially blocked is higher than that of the local area. Relatively speaking, the area that needs to be identified and matched is smaller when performing local tracking through the first local area. For the same obstacle, the probability of the local area being blocked is also smaller than that of the global area. Therefore, by performing local tracking in the first local area to obtain global tracking information, compared with directly performing global tracking on the entire target to be tracked, target tracking can be performed more robustly to improve the target tracking success rate.

[0105] In some embodiments, determining at least one first local area used when tracking the target to be tracked includes:

[0106] determining at least one candidate local area for target tracking;

[0107] According to the statistical probability of each candidate local area being blocked, at least one candidate local area whose statistical probability of being blocked meets the first condition is selected as the first local area.

[0108] In this exemplary embodiment, the candidate local regions are all local regions in the target global region in the image frame. In this exemplary embodiment, one or more candidate local regions for local tracking may be determined in advance.

[0109] In one embodiment, multiple local areas with a low probability of being blocked may be selected as candidate local areas based on big data statistics.

[0110] In this exemplary embodiment, the first condition may be that the probability of a local area being blocked is less than a preset probability value. Since the probability of each part being blocked is different during the movement of the target to be tracked, an alternative local area with a probability less than the preset probability value may be determined as the first local area for local tracking. For example, when the target to be tracked is a human body, the probability of the legs of the human body being blocked is generally greater than the probability of the head being blocked. The probability of the legs of the human body being blocked as an alternative local area is greater than the probability of the head being blocked as an alternative local area. Obviously, the head as an alternative local area is more likely to meet the first condition.

[0111] In one embodiment, if the target to be tracked has been tracked for a period of time, and this period of time exceeds a preset time, one or more local areas that have not been blocked or have been blocked less times can be selected as the candidate local areas according to the statistics of the target to be tracked itself being blocked. In this way, an area that is more suitable as a candidate local area in the current environment can be determined, thereby improving the tracking success rate.

[0112] In one embodiment, the size of the target to be tracked is determined according to the global tracking information of the target to be tracked in an initial image frame; wherein the initial image frame is the first image frame among the acquired multiple image frames;

[0113] If the size of the target to be tracked is greater than a preset size threshold, performing local tracking based on the first local area in the plurality of image frames;

[0114] If the size of the target to be tracked is not greater than a preset size threshold, global tracking is performed based on the entirety of the target to be tracked in the plurality of image frames.

[0115] In this exemplary embodiment, by comparing the size of the target to be tracked in the initial image frame with the preset size threshold, the overall size of the target to be tracked can be preliminarily obtained, and the possibility or probability of the target being lost when the target is globally tracked can be estimated. When the size of the target to be tracked is larger than the preset size threshold, the overall size of the target to be tracked is larger, the greater the probability of being blocked during the tracking process, the greater the possibility of being lost, and thus it is more appropriate to select local tracking to improve the success rate of target tracking. When the size of the target to be tracked is not larger than the preset size threshold, it means that the overall size of the target to be tracked is small, and when the target is globally tracked, the possibility of the target being lost is small, and global tracking can be directly selected.

[0116] In some embodiments, the determining of at least one first local area used when tracking the target to be tracked further includes:

[0117] Determining the tracking quality of the local tracking information of the target to be tracked based on the first local area whose statistical probability of being blocked satisfies the first condition;

[0118] A local area whose tracking quality is not lower than a preset quality threshold is selected as the updated first local area.

[0119] In this exemplary embodiment, while performing local tracking, the tracking quality of the first local area that meets the first condition based on the statistical probability of being blocked can be synchronously analyzed. Tracking quality can be an evaluation value for evaluating the tracking effect. The evaluation value can be used to indicate the tracking success rate and / or confidence of the target. At the same time, the tracking success rate and confidence are both positively correlated with the tracking quality. That is, the higher the tracking success rate, the higher the tracking quality; the higher the confidence, the higher the tracking quality. The tracking success rate can be expressed by the ratio of the number of image frames that successfully track the target to the total number of image frames. For example, if the total number of image frames is 100 frames and the number of image frames that track the target is 80, then the success rate of tracking the target based on the first local area is 80%, etc.

[0120] If the tracking quality is not good, for example, the feature recognition is not obvious and it is easy to identify errors, resulting in the tracking quality being lower than the preset quality threshold, a local area with a tracking quality not lower than the preset quality threshold can be reselected from the alternative local areas as the first local area after update. Then continue tracking the updated first local area. The preset quality threshold is a preset parameter value set in the system for evaluating the target tracking quality. For example, the preset parameter value is 50. When the tracking quality is good, for example, the selected local area has obvious features and is easy to identify, the tracking device can easily capture it, and the target is not easily lost. At this time, the tracking quality can be evaluated as 80, which is greater than the preset quality threshold of 50, and there is no need to reselect the first local area. If the tracking quality is evaluated as 20, which is less than the preset quality threshold of 50, then it is necessary to reselect the alternative local area as the first local area for local tracking.

[0121] In some embodiments, determining at least one candidate local area for target tracking includes:

[0122] At least one candidate local area for target tracking is determined according to characteristic attributes of different local areas of the target to be tracked, wherein the characteristic attributes include: the number of features and / or a significance parameter of a single feature.

[0123] In this exemplary embodiment, the candidate local areas can be determined according to the characteristic attributes of different local areas of the target. The characteristic attributes include: the number of features and / or the significance parameters of a single feature. The number of features refers to the number of features contained in the local area, and different numbers of features make the local area easier to identify. For example, when the target to be tracked is a human body, the local areas can be the head and the waist. When the head is used as a local area, the features included include the features of the facial features themselves, the features of the facial features, the head shape features, the hairstyle features, etc. Relatively speaking, when the waist is used as a local area, the features included are only the waist size and waist shape. The significance parameters of a single feature refer to some obvious differences in a single feature, and these differences can all be used as significance parameters. For example, the facial features of the human body have obvious scars, spots, etc. that are easy to distinguish. The higher the significance of the feature, the higher the degree of recognition. Through the identification of these characteristic attributes, tracking becomes easier and more accurate.

[0124] In some embodiments, selecting at least one candidate local area whose statistical probability of being blocked meets a first condition as the first local area according to the statistical probability of each candidate local area being blocked includes:

[0125] According to the statistical probability of the candidate local area being blocked, a candidate local area whose statistical probability of being blocked meets the first condition and whose initial image frame is not blocked among the multiple image frames is selected as the first local area.

[0126] In this exemplary embodiment, the first condition may be that the probability of a local area being blocked is less than a preset probability value. Since the probability of each local area being blocked is different during the movement of the target, an alternative local area less than the preset probability value can be determined as the first local area for local tracking. For example, when the target to be tracked is a human body, the probability of the legs of the human body being blocked is generally greater than the probability of the head being blocked. The probability of the legs of the human body being blocked as an alternative local area is greater than the probability of the head being blocked as an alternative local area. Obviously, the head is more likely to meet the first condition as an alternative local area. While the statistical probability satisfies the first condition, the alternative local area that is not blocked in the initial image frame in multiple image frames is used as the first local area for local tracking, which can improve the accuracy of target tracking.

[0127] In some embodiments, the number of the first local areas is N, where N is a positive integer equal to or greater than 2 and less than P; P is the number of local areas included in the target to be tracked;

[0128] The performing local tracking between multiple image frames based on the first local area to obtain local tracking information includes:

[0129] Among the multiple image frames, local tracking is performed in the mth image frame based on the first local area that is not blocked in the m-1th image frame to obtain local tracking information of the target to be tracked in the mth image frame, wherein the mth image frame is the next image frame of the m-1th image frame, and m is a positive integer greater than 1.

[0130] In this exemplary embodiment, there are multiple first local areas, and the accuracy of target tracking can be improved by synchronously tracking the multiple first local areas.

[0131] In some embodiments, the method further comprises:

[0132] When it is detected that part of the first local areas are blocked in one or more of the image frames, unblocked local areas are selected as the updated N first local areas according to the blocking conditions.

[0133] In this exemplary embodiment, during the target tracking process, if the first local area is blocked in one or more image frames, the target may be lost if the target is tracked with the first local area. Based on this, one or more local areas may be selected from the local areas where the target is not blocked, and the first local areas may be updated to improve the success rate of target tracking. When there are multiple first local areas, multiple first local areas may be updated synchronously.

[0134] In some embodiments, the selecting the unobstructed local areas as the updated N first local areas includes:

[0135] Determining the perspective relationship between different targets in each of the image frames; wherein the targets at least include the target to be tracked;

[0136] Determining a foreground target of the target to be tracked according to the perspective relationship;

[0137] Determine whether there is an interference object in the foreground target that blocks the candidate local area in the target to be tracked;

[0138] Select unobstructed candidate local regions as the updated N first local regions.

[0139] In this exemplary embodiment, each image of the target tracking can be obtained by means of camera, radar detection, etc. By analyzing the perspective relationship of each target in the image, it is confirmed whether the candidate local area in the tracking target is blocked by interference. Interference is a target other than the target in the image, including: people other than the tracked person, moving vehicles, fixed devices (including fences, etc.), etc.

[0140] In this exemplary embodiment, the perspective relationship refers to the spatial position, outline and projection of each target in the field of view, including relative relationships such as near is big and far is small, near is high and far is low, near is sparse and far is dense, near is wide and far is narrow, etc. In this exemplary embodiment, the spatial position, outline and other information of each target and each candidate local area in the field of view can be captured by a tracking device (e.g., a camera device, etc.) through video or photography.

[0141] In some embodiments, determining whether there is an interference object in the foreground target that blocks the candidate local area in the target to be tracked includes:

[0142] Determining whether the position of the foreground target overlaps with the position of the candidate local area in the target to be tracked;

[0143] If so, it is determined that the candidate local area is blocked by the interference object in the image frame.

[0144] In this exemplary embodiment, the foreground target refers to a target that is closer to the tracking device than the target to be tracked. The foreground target is one or more of the interference objects. That is, any interference object may become a foreground target.

[0145] In this exemplary embodiment, when the position of the foreground target overlaps with the position of the candidate local area in the target to be tracked, the interference object at the foreground target position blocks the tracking device from obtaining the field of view of the candidate local area. At this time, the candidate local area is blocked by the interference object in the image frame.

[0146] In some embodiments, determining the perspective relationship between different targets in each of the image frames, and determining the foreground target of the target to be tracked according to the perspective relationship, includes:

[0147] Determining the bounding boxes of the different targets;

[0148] Determine a first distance between a lower edge of a bounding box of the target to be tracked and a lower edge of the image frame, and determine a second distance between a lower edge of a bounding box of a target other than the target to be tracked and a lower edge of the image frame;

[0149] If the first distance is greater than the second distance, the target corresponding to the second distance is determined to be the foreground target of the target to be tracked.

[0150] In this exemplary embodiment, Figure 2 FIG. 1 is a schematic diagram of determining an occlusion relationship according to an exemplary embodiment. Figure 2As shown, in the acquired frame image, when the first distance is greater than the second distance, it means that the distance between the interference object and the tracking device is less than the distance between the target to be tracked (the obscured target in the figure) and the tracking device. At this time, the interference object is the foreground target of the target. At the same time, if the position of the foreground target overlaps with the position of the target to be tracked, the interference object obscures the target to be tracked. In this exemplary embodiment, the circumscribed frame is a feature capture frame circled according to the tracking feature (global feature or local feature) when the tracking device captures the target. The circumscribed frame can be any shape such as a rectangle or a circle.

[0151] In some embodiments, obtaining the global tracking information of the target to be tracked in different image frames according to the local tracking information and the association relationship includes:

[0152] Determining a relative position relationship of the first local area in the target to be tracked according to the position information of the first local area and the position information of the bounding box of the target to be tracked;

[0153] According to the relative position relationship, the local tracking information of the first local area in each of the image frames is converted into the global tracking information of the target to be tracked in each of the image frames.

[0154] In this exemplary embodiment, after local tracking information is acquired based on the first local area tracking, global tracking information of the target in different image frames is obtained according to the local tracking information and the association relationship between the first local area and the target as a whole.

[0155] For example, the local tracking information of the first local area in the mth frame image includes the position coordinates of the feature of the first local area in the mth frame image (for example, determining the position coordinates of the upper left corner of the outer bounding box encircling the local feature in the mth frame image: X' 局 , Y' 局 ; where X' 局 is the coordinate of the upper left corner of the bounding box that encloses the local feature in the X-axis direction in the m-th frame image, where Y' 局 is the Y-axis coordinate of the upper left corner of the bounding box that encloses the local feature in the m-th frame image);

[0156] Determine the position coordinates of the upper left corner of the external frame that encloses the overall features of the target to be tracked in the m-1th frame image: X 整 , Y 整 ; Among them, X 整 is the coordinate of the upper left corner of the external frame that encloses the overall features of the target to be tracked in the X-axis direction in the m-1th frame image, 整 The Y-axis coordinate of the upper left corner of the external frame that encloses the overall features of the target to be tracked in the m-1th frame image;

[0157] Determine the position coordinates of the upper left corner of the outer bounding box that encloses the first local area in the m-1th frame image: X 局 , Y 局 ; Among them, X 局 is the coordinate of the upper left corner of the outer frame of the first local area in the X-axis direction in the m-1th frame image, Y 局 is the Y-axis coordinate of the upper left corner of the outer bounding box that encloses the first local area in the (m-1)th frame image;

[0158] Determine the relative position relationship of the first local area that is not blocked in the m-1th frame image in the target: X 差 =X 整 -X 局 , Y 差 =Y 整 -Y 局 ; Among them, X 差 Y is the coordinate difference in the X-axis direction between the upper left corner of the bounding box that encloses the overall features of the target to be tracked and the upper left corner of the bounding box that encloses the first local area in the m-1th frame image, 差 is the coordinate difference in the Y-axis direction between the upper left corner of the outer bounding box enclosing the overall features of the target to be tracked and the upper left corner of the outer bounding box enclosing the first local area in the m-1th frame image;

[0159] Determine the global tracking information X' of the target to be tracked in the mth frame image 整 =X' 局 +X 差 , Y' 整 =Y' 局 +Y 差 ; where X' 整 Y' is the coordinate of the upper left corner of the external frame that encloses the overall features of the target to be tracked in the X-axis direction in the m-th frame image. 整 The Y-axis coordinate of the upper left corner of the external frame that encloses the overall features of the target to be tracked in the mth frame image.

[0160] Determine the width W of the bounding box that encircles the overall features of the target to be tracked in the m-1th frame image 整 and high H 整 At the same time, the width W' and height H' of the external frame encircling the overall features of the target to be tracked in the m-th frame image are the same as the width W and height H in the m-1-th frame image.

[0161] In this exemplary embodiment, radar, binocular vision and other methods can also be used to determine the occlusion relationship between targets or between targets and the environment. Combined with background modeling, optical flow and other algorithms, the unoccluded target area can be determined more accurately. The tracking algorithm is changed to use other feature models, deep learning, etc.

[0162] At the same time, in the field of computer vision, when tracking a target as a whole, it is easy for the tracking target to be lost due to occlusion. Figure 3 This is a schematic diagram of the process of tracking target loss. Figure 3 As shown, it includes an initial state, an occlusion state, and a tracking failure state. In the initial state, the tracking device can track the target to be tracked, and the target to be tracked is not occluded. In the occlusion state, the tracking device finds that the target to be tracked is occluded, and then the target to be tracked is lost, and the target frame (external frame) of the target to be tracked remains on the foreground target, resulting in tracking failure. Figure 4 It is a schematic diagram of the possible situation where the target to be tracked is blocked. Figure 4 As shown, the target to be tracked may be blocked by a human body, a fixed obstacle, or a vehicle.

[0163] The present disclosure can locally track a target by selecting an unobstructed candidate local area as the first local area, and obtain global tracking information of the target in different image frames based on the obtained local tracking information and the association between the first local area and the entire target to be tracked. This can reduce the situation where the target is lost when the target is partially obstructed and the target continues to be tracked as a whole. Because when the target to be tracked is partially obstructed, if the target to be tracked continues to be tracked as a whole, the tracking device will not be able to identify the target to be tracked as a whole due to the partial obstruction, resulting in the tracking result being judged as the absence of the target in the next frame of the image, which will further cause the target to be lost.

[0164] Figure 5 FIG. 1 is a schematic diagram of target tracking learning according to an exemplary embodiment. Figure 5 As shown in the figure, the weights of the tracking learning area for tracking the target to be tracked include the whole body and the upper body. The original tracking is based on the overall features of the target to be tracked, and the core learning area is at the center of the target, that is, the feature learning weight obtained at the center is the highest, and the learning weight at the edge is the weakest. When the weight of the tracking learning area is the whole body of the target to be tracked, the core tracking area is blocked, and the target to be tracked is easy to lose. When the weight of the tracking learning area is the upper body of the target to be tracked, since the probability of the lower body of the target to be tracked is generally higher, the core tracking area is not easy to be blocked, and the target to be tracked will not be easily lost. If the whole body is tracked, the core area of ​​the learning weight is in the waist and abdomen. The lower body is easily blocked due to alternating movements of the legs and occlusion of low objects, and the changes are large, which is not conducive to tracking. Therefore, using the upper body for tracking, moving the core area of ​​the learning weight to the chest part makes it easier to track stably. That is, it can improve the rapid changes of the legs that are not conducive to tracking, and can also solve the occlusion of some common objects, such as fences, non-motor vehicles, low cars, etc.

[0165] Figure 6 FIG. 1 is an overall flow chart of a target tracking method according to an exemplary embodiment. Figure 6 As shown, the target tracking method includes:

[0166] Video frame parsing: Get the m-1th frame image (detection frame image) and the mth frame image (tracking frame image).

[0167] Detection frame image analysis: In the m-1th frame image, obtain the position of the bounding box of the target to be tracked (for example, the coordinates of the upper left corner of the bounding box), and determine the occlusion relationship; select a local area for local tracking (obtain the coordinates of the upper left corner of the bounding box of the local area); the target tracking model learns to track the local area.

[0168] Tracking frame image analysis: Track the new position of the local area according to the local area model; obtain the global position according to the new position of the local area and the position and scaling relationship between the local area and the global area in the detection frame image.

[0169] If the frame image processing is completed, the task ends, otherwise the video frame parsing continues.

[0170] Among them, the models used in the above method can be various conventional models commonly used or already available in the field. For example, the target tracking model and the local area model can respectively be various target tracking models and local area models existing in the field. The update of related models is also an existing update method, which will not be described in detail here.

[0171] In another aspect, the present disclosure provides a target tracking device. Figure 7 FIG. 1 is a schematic diagram showing the structure of a target tracking device according to an exemplary embodiment. Figure 7 As shown, the target tracking device provided by the embodiment of the present disclosure includes:

[0172] A first processing unit 71, configured to determine at least one first local area used when tracking a target to be tracked;

[0173] A second processing unit 72 is used to determine the association relationship between the first local area and the entire target to be tracked;

[0174] A third processing unit 73 is used to perform local tracking between multiple image frames based on the first local area to obtain local tracking information;

[0175] The fourth processing unit 74 is configured to obtain the global tracking information of the target to be tracked in different image frames according to the local tracking information and the association relationship.

[0176] In this exemplary embodiment, the target to be tracked may be a movable target such as a human body, an animal, or a vehicle. The specific target to be tracked is not limited to any one.

[0177] In this exemplary embodiment, the first local area may be a local area of ​​the entire target in the image frame, including identification features used by the tracking device for tracking and identification (for example, when the target is a human body, the first local area may be an organ tissue of the human body in the image, including the head, limbs, torso, etc.).

[0178] In this exemplary embodiment, the association relationship between the first local area and the entire target to be tracked includes the positional relationship between the first local area and the entire target to be tracked in the image and the relative local area relationship, etc. The positional relationship may include the coordinate relationship between the regional center coordinates of the first local area and the regional center coordinates of the entire target to be tracked; the relative local area relationship may include the relative relationship between the contour and area of ​​the first local area and the contour and area of ​​the entire target to be tracked, for example, which area of ​​which position of the overall target is the first local area.

[0179] In this exemplary embodiment, multiple image frames may be from the same video, and the video may be from a road monitoring device and / or a security monitoring device, etc. Of course, this is only an example of the device from which the video comes, and the specific implementation is not limited to these example road monitoring devices and / or security monitoring devices. The security monitoring equipment includes but is not limited to: surveillance cameras in residential areas, surveillance cameras in office buildings, surveillance equipment in shopping malls, and / or surveillance equipment in non-road public places such as parks.

[0180] In this exemplary embodiment, global tracking means that the tracking device identifies and locates the target based on the overall features of the target (i.e., the target to be tracked is regarded as a whole). For example, when the target to be tracked is a person, all the features of the person (including the head, limbs, torso, etc.) are regarded as a whole for identification and location. Local tracking means that the tracking device identifies and locates the target based on partial features of the target. For example, when the target to be tracked is a person, only the head of the person can be identified and located to determine the position of the target.

[0181] In this exemplary embodiment, the local tracking information may be the position information (including coordinate information) of the tracked local features in the image obtained based on the tracking of the target local features and / or the area information of the local features (including the morphological contour, area, etc. constituted by the local features).

[0182] The global tracking information may be the position information (including coordinate information) of the entire target in the image and / or the area information of the entire target (including the morphological contour and area formed by the overall features) obtained based on the overall feature tracking of the target.

[0183] In this exemplary embodiment, when the target has obvious features that are easy to track and identify, at least one first local area (having obvious features that are easy to track and identify) can be locally tracked in multiple image frames to obtain local tracking information of the first local area, and then global tracking information of the target can be obtained based on the local tracking information.

[0184] In this exemplary embodiment, when the target does not have obvious features that are easy to track and identify, multiple different first local areas can be tracked synchronously to obtain local tracking information of the multiple first local areas, and then the global tracking information of the target can be obtained based on the local tracking information of the multiple first local areas to improve the tracking success rate.

[0185] The disclosed embodiment locally tracks the target to be tracked between multiple image frames through at least one first local area, and obtains the global tracking information of the target to be tracked in different image frames based on the obtained local tracking information and the association between the first local area and the entire target to be tracked. In this way, local tracking is performed through one or more first local areas to obtain local tracking information, and global tracking information is obtained based on the local tracking information. Compared with directly performing global tracking on the entire target to be tracked, the number of features that need to be identified when the target can be tracked can be reduced. At the same time, when the target to be tracked is globally tracked as a whole, it is necessary to identify and match the entire area of ​​the target. Once a part of the area is blocked, it may be impossible to match the entire area, causing the target to be unrecognizable, and then causing the target to be lost. The overall target to be tracked is relatively large compared to a part of the target to be tracked, and the probability of it being partially blocked is higher than that of the part. Relatively speaking, the area that needs to be identified and matched is smaller when performing local tracking through the first local area. For the same obstacle, the probability of the local area being blocked is also smaller than that of the global area. Therefore, by performing local tracking on the first local area to obtain global tracking information, compared with directly performing global tracking on the entire target, target tracking can be performed more robustly to improve the target tracking success rate.

[0186] In some embodiments, the first processing unit is used to determine at least one first local area used when tracking the target to be tracked, including:

[0187] The first processing unit is specifically configured to determine at least one candidate local area for target tracking;

[0188] According to the statistical probability of each candidate local area being blocked, at least one candidate local area whose statistical probability of being blocked meets the first condition is selected as the first local area.

[0189] In this exemplary embodiment, the candidate local regions are all local regions in the global region of the target to be tracked in the image frame. In this exemplary embodiment, one or more candidate local regions for local tracking may be determined in advance.

[0190] In one embodiment, multiple local areas with a low probability of being blocked may be selected as candidate local areas based on big data statistics.

[0191] In this exemplary embodiment, the first condition may be that the probability of a part being blocked is less than a preset probability value. Since the probability of each part being blocked is different during the movement of the target, an alternative local area with a probability less than the preset probability value may be determined as the first local area for local tracking. For example, when the target to be tracked is a human body, the probability of the legs of the human body being blocked is generally greater than the probability of the head being blocked. The probability of the legs of the human body being blocked as an alternative local area is greater than the probability of the head being blocked as an alternative local area. Obviously, the head as an alternative local area is more likely to meet the first condition.

[0192] In one embodiment, if the target to be tracked has been tracked for a period of time, and this period of time exceeds a preset time, one or more parts that have not been blocked or have been blocked less times can be selected as the candidate local areas according to the statistics of the target to be tracked being blocked. In this way, an area that is more suitable as a candidate local area in the current environment can be determined, thereby improving the tracking success rate.

[0193] In some embodiments, the first processing unit is used to determine at least one first local area used when tracking the target to be tracked, and further includes:

[0194] The first processing unit is specifically configured to determine the tracking quality of the local tracking information of the target to be tracked based on the first local area whose statistical probability of being blocked satisfies the first condition;

[0195] A local area whose tracking quality is not lower than a preset quality threshold is selected as the updated first local area.

[0196] In this exemplary embodiment, while performing local tracking, the tracking quality of the first local area that meets the first condition based on the statistical probability of being blocked can be synchronously analyzed. Tracking quality can be an evaluation value for evaluating the tracking effect. The evaluation value can be used to indicate the tracking success rate and / or confidence of the target. At the same time, the tracking success rate and confidence are both positively correlated with the tracking quality. That is, the higher the tracking success rate, the higher the tracking quality; the higher the confidence, the higher the tracking quality. The tracking success rate can be expressed by the ratio of the number of image frames that successfully track the target to the total number of image frames. For example, if the total number of image frames is 100 frames and the number of image frames that track the target is 80, then the success rate of tracking the target based on the first local area is 80%, etc.

[0197] If the tracking quality is not good, for example, the feature recognition is not obvious and it is easy to identify errors, resulting in the tracking quality being lower than the preset quality threshold, a local area with a tracking quality not lower than the preset quality threshold can be reselected from the alternative local areas as the updated first local area. Then continue tracking the updated first local area. The preset quality threshold is a preset parameter value set in the system for evaluating the target tracking quality. For example, the preset parameter value is 50. When the tracking quality is good, for example, the selected local features are obvious and easy to identify, the tracking device can easily capture it and the target is not easily lost. At this time, the tracking quality can be evaluated as 80, which is greater than the preset quality threshold of 50, and there is no need to reselect the first local area. If the tracking quality is evaluated as 20, which is less than the preset quality threshold of 50, then it is necessary to reselect the alternative local area as the first local area for local tracking.

[0198] In some embodiments, the first processing unit is used to determine at least one candidate local area for target tracking, including:

[0199] The first processing unit is specifically configured to determine at least one candidate local area for target tracking according to feature attributes of different local areas of the target to be tracked, wherein the feature attributes include: feature quantity and / or significance parameters of a single feature.

[0200] In this exemplary embodiment, the candidate local areas can be determined according to the characteristic attributes of different parts of the target. The characteristic attributes include: the number of features and / or the significance parameters of a single feature. The number of features refers to the number of features contained in the local area, and different numbers of features make the local area easier to identify. For example, when the target to be tracked is a human body, the local areas can be the head and the waist. When the head is used as a local area, the features included include the features of the facial features themselves, the features of the facial features, the head shape features, the hairstyle features, etc. Relatively speaking, when the waist is used as a local area, the features included are only the waist size and waist shape. The significance parameters of a single feature refer to some obvious differences in a single feature, and these differences can all be used as significance parameters. For example, the facial features of the human body have obvious scars, spots, etc. that are easy to distinguish. The higher the significance of the feature, the higher the degree of recognition. Through the identification of these characteristic attributes, tracking becomes easier and more accurate.

[0201] In some embodiments, the first processing unit is configured to select, according to the statistical probability of each candidate local area being blocked, at least one candidate local area whose statistical probability of being blocked meets the first condition as the first local area, including:

[0202] The first processing unit is specifically configured to select, according to the statistical probability of the candidate local area being blocked, a candidate local area whose statistical probability of being blocked meets the first condition and whose initial image frame is not blocked in the multiple image frames as the first local area.

[0203] In this exemplary embodiment, the first condition may be that the probability of a local area being blocked is less than a preset probability value. Since the probability of each local area being blocked is different during the movement of the target to be tracked, an alternative local area less than the preset probability value can be determined as the first local area for local tracking. For example, when the target to be tracked is a human body, the probability of the legs of the human body being blocked is generally greater than the probability of the head being blocked. The probability of the legs of the human body being blocked as an alternative local area is greater than the probability of the head being blocked as an alternative local area. Obviously, the head is more likely to meet the first condition as an alternative local area. While the statistical probability satisfies the first condition, the alternative local area that is not blocked in the initial image frame in multiple image frames is used as the first local area for local tracking, which can improve the accuracy of target tracking.

[0204] In some embodiments, the number of the first local areas is N, where N is a positive integer equal to or greater than 2 and less than P; P is the number of local areas included in the target to be tracked;

[0205] The third processing unit is used to perform local tracking between multiple image frames based on the first local area to obtain local tracking information, including:

[0206] The third processing unit is specifically used to perform local tracking in the mth image frame among the multiple image frames based on the first local area that is not blocked in the m-1th image frame, so as to obtain local tracking information of the target to be tracked in the mth image frame, wherein the mth image frame is the image frame after the m-1th image frame, and m is a positive integer greater than 1.

[0207] In this exemplary embodiment, there are multiple first local areas, and the accuracy of target tracking can be improved by synchronously tracking the multiple first local areas.

[0208] In some embodiments, a fifth processing unit is further included for selecting unobstructed local areas as the updated N first local areas according to the occlusion conditions when it is detected that part of the first local areas are obstructed in one or more of the image frames.

[0209] In this exemplary embodiment, during the target tracking process, if the first local area is blocked in one or more image frames, the target may be lost if the target is tracked with the first local area. Based on this, one or more local areas may be selected from the unblocked parts of the target to update the first local area to improve the success rate of target tracking. When there are multiple first local areas, multiple first local areas may be updated synchronously.

[0210] In some embodiments, the fifth processing unit, for selecting the unobstructed local area as the updated N first local areas, includes:

[0211] The fifth processing unit is specifically used to determine the perspective relationship between different targets in each of the image frames; wherein the targets at least include the target to be tracked;

[0212] Determining a foreground target of the target to be tracked according to the perspective relationship;

[0213] Determine whether there is an interference object in the foreground target that blocks the candidate local area in the target to be tracked;

[0214] Select unobstructed candidate local regions as the updated N first local regions.

[0215] In this exemplary embodiment, each image of the target tracking can be obtained by means of camera, radar detection, etc. By analyzing the perspective relationship of each target in the image, it is confirmed whether the candidate local area in the tracking target is blocked by interference. Interference is a target other than the target in the image, including: people other than the tracked person, moving vehicles, fixed devices (including fences, etc.), etc.

[0216] In this exemplary embodiment, the perspective relationship refers to the spatial position, outline and projection of each target in the field of view, including relative relationships such as near is big and far is small, near is high and far is low, near is sparse and far is dense, near is wide and far is narrow, etc. In this exemplary embodiment, the spatial position, outline and other information of each target and each candidate local area in the field of view can be captured by a tracking device (e.g., a camera device, etc.) through video or photography.

[0217] In some embodiments, determining whether there is an interference object in the foreground target that blocks the candidate local area in the target to be tracked includes:

[0218] Determining whether the position of the foreground target overlaps with the position of the candidate local area in the target to be tracked;

[0219] If so, it is determined that the candidate local area is blocked by the interference object in the image frame.

[0220] In this exemplary embodiment, the foreground target refers to a target that is closer to the tracking device than the tracking target, based on the tracking device. The foreground target is one or more of the interference objects. That is, any interference object may become a foreground target.

[0221] In this exemplary embodiment, when the position of the foreground target overlaps with the position of the candidate local area in the target, the interference object at the foreground target position blocks the tracking device from obtaining the field of view of the candidate local area. At this time, the candidate local area is blocked by the interference object in the image frame.

[0222] In some embodiments, the first processing unit is used to determine the perspective relationship between different targets in each of the image frames, and determine the foreground target of the target to be tracked according to the perspective relationship, including:

[0223] The first processing unit is specifically configured to determine the external frames of the different targets;

[0224] Determine a first distance between a lower edge of a bounding box of the target to be tracked and a lower edge of the image frame, and determine a second distance between a lower edge of a bounding box of a target other than the target to be tracked and a lower edge of the image frame;

[0225] If the first distance is greater than the second distance, the target corresponding to the second distance is determined to be the foreground target of the target to be tracked.

[0226] In this exemplary embodiment, Figure 2 FIG. 1 is a schematic diagram of determining an occlusion relationship according to an exemplary embodiment. Figure 2 As shown, in the acquired frame image, when the first distance is greater than the second distance, it means that the distance between the interference object and the tracking device is less than the distance between the tracking target (the blocked target in the figure) and the tracking device. At this time, the interference object is the foreground target of the target. At the same time, if the position of the foreground target overlaps with the position of the target, the interference object blocks the tracking target. In this exemplary embodiment, the circumscribed frame is a feature capture frame circled according to the tracking feature (global feature or local feature) when the tracking device captures the target. The circumscribed frame can be any shape such as a rectangle or a circle.

[0227] In some embodiments, the fourth processing unit is used to obtain the global tracking information of the target to be tracked in different image frames according to the local tracking information and the association relationship, including:

[0228] The fourth processing unit is specifically configured to determine a relative position relationship of the first local area in the target to be tracked according to the position information of the first local area and the position information of the external frame of the target to be tracked;

[0229] According to the relative position relationship, the local tracking information of the first local area in each of the image frames is converted into the global tracking information of the target to be tracked in each of the image frames.

[0230] In this exemplary embodiment, after local tracking information is acquired based on the first local area tracking, global tracking information of the target to be tracked in different image frames is obtained according to the local tracking information and the association relationship between the first local area and the entire target to be tracked.

[0231] For example, the local tracking information of the first local area in the mth frame image includes the position coordinates of the feature of the first local area in the mth frame image (for example, determining the position coordinates of the upper left corner of the outer bounding box encircling the local feature in the mth frame image: X' 局 , Y' 局 ; where X' 局 is the coordinate of the upper left corner of the bounding box that encloses the local feature in the X-axis direction in the m-th frame image, where Y' 局 is the Y-axis coordinate of the upper left corner of the bounding box that encloses the local feature in the m-th frame image);

[0232] Determine the position coordinates of the upper left corner of the external frame that encloses the overall features of the target to be tracked in the m-1th frame image: X 整 , Y 整 ; Among them, X 整 is the coordinate of the upper left corner of the external frame that encloses the overall features of the target to be tracked in the X-axis direction in the m-1th frame image, 整 The Y-axis coordinate of the upper left corner of the external frame that encloses the overall features of the target to be tracked in the m-1th frame image;

[0233] Determine the position coordinates of the upper left corner of the outer bounding box that encloses the first local area in the m-1th frame image: X 局 , Y 局 ; Among them, X 局 is the coordinate of the upper left corner of the outer frame of the first local area in the X-axis direction in the m-1th frame image, Y 局 is the Y-axis coordinate of the upper left corner of the outer bounding box that encloses the first local area in the (m-1)th frame image;

[0234] Determine the relative position relationship of the first local area that is not blocked in the m-1th frame image in the target: X 差 =X 整 -X 局 , Y 差 =Y 整 -Y 局 ; Among them, X 差Y is the coordinate difference in the X-axis direction between the upper left corner of the bounding box that encloses the overall features of the target to be tracked and the upper left corner of the bounding box that encloses the first local area in the m-1th frame image, 差 is the coordinate difference in the Y-axis direction between the upper left corner of the outer bounding box enclosing the overall features of the target to be tracked and the upper left corner of the outer bounding box enclosing the first local area in the m-1th frame image;

[0235] Determine the global tracking information X' of the target to be tracked in the mth frame image 整 =X' 局 +X 差 , Y' 整 =Y' 局 +Y 差 ; where X' 整 Y' is the coordinate of the upper left corner of the external frame that encloses the overall features of the target to be tracked in the X-axis direction in the m-th frame image. 整 The Y-axis coordinate of the upper left corner of the external frame that encloses the overall features of the target to be tracked in the mth frame image.

[0236] Determine the width W of the bounding box that encircles the overall features of the target to be tracked in the m-1th frame image 整 and high H 整 At the same time, the width W' and height H' of the external frame encircling the overall features of the target to be tracked in the m-th frame image are the same as the width W and height H in the m-1-th frame image.

[0237] In this exemplary embodiment, radar, binocular vision and other methods can also be used to determine the occlusion relationship between targets or between targets and the environment. Combined with background modeling, optical flow and other algorithms, the unoccluded target area can be determined more accurately. The tracking algorithm is changed to use other feature models, deep learning, etc.

[0238] The present disclosure also provides an electronic device. Figure 8 FIG. 1 is a schematic diagram showing the structure of an electronic device according to an exemplary embodiment. Figure 8 As shown, the electronic device provided by the embodiment of the present disclosure includes: a processor 830 and a memory 820 for storing a computer program that can be run on the processor, wherein the processor 830 executes the steps of the method provided in the above embodiments when running the computer program.

[0239] The present disclosure also provides a computer-readable storage medium. The computer-readable storage medium provided in the embodiments of the present disclosure stores a computer program, which, when executed by a processor, implements the steps of the method provided in the above embodiments.

[0240] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0241] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0242] In addition, all functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0243] In some cases, if any two of the above technical features do not conflict, they can be combined into a new method technical solution. In some cases, if any two of the above technical features do not conflict, they can be combined into a new device technical solution.

[0244] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks or optical disks, etc., various media that can store program codes.

[0245] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A target tracking method, characterized in that: The method comprises: Determine at least one first local area used when tracking the target to be tracked; Determining an association relationship between the first local area and the entire target to be tracked; Based on the first local area, local tracking is performed between multiple image frames to obtain local tracking information; Obtaining global tracking information of the target to be tracked in different image frames according to the local tracking information and the association relationship; The determining of at least one first local area used when tracking the target to be tracked includes: determining at least one candidate local area for target tracking; According to the statistical probability of each candidate local area being blocked, selecting at least one candidate local area whose statistical probability of being blocked meets the first condition as the first local area; The determining of at least one first local area used when tracking the target to be tracked further includes: Determining the tracking quality of the local tracking information of the target to be tracked based on the first local area whose statistical probability of being blocked satisfies the first condition; Selecting a local area whose tracking quality is not less than a preset quality threshold as the updated first local area; The determining of at least one candidate local area for target tracking comprises: Determine at least one candidate local area for target tracking according to characteristic attributes of different local areas of the target to be tracked, wherein the characteristic attributes include: the number of features and / or a significance parameter of a single feature; The selecting, according to the statistical probability of each candidate local area being blocked, at least one candidate local area whose statistical probability of being blocked satisfies a first condition as the first local area comprises: According to the statistical probability of the candidate local area being blocked, selecting a candidate local area whose statistical probability of being blocked meets the first condition and whose initial image frame is not blocked among the multiple image frames as the first local area; The number of the first local areas is N, where N is a positive integer equal to or greater than 2 and less than P; P is the number of local areas included in the target to be tracked; The performing local tracking between multiple image frames based on the first local area to obtain local tracking information includes: Among the multiple image frames, based on the first local area that is not blocked in the m-1th image frame, local tracking is performed in the m-th image frame to obtain local tracking information of the target to be tracked in the m-th image frame, wherein the m-th image frame is an image frame after the m-1th image frame, and m is a positive integer greater than 1; The method further comprises: When it is detected that part of the first local areas are blocked in one or more of the image frames, selecting the local areas that are not blocked as the updated N first local areas according to the blocking conditions; The selecting the unobstructed local areas as the updated N first local areas includes: Determining the perspective relationship between different targets in each of the image frames; wherein the targets at least include the target to be tracked; Determine the foreground target of the target to be tracked according to the perspective relationship; determine whether there is an interference object in the foreground target that blocks the candidate local area in the target to be tracked; Selecting unobstructed candidate local areas as the updated N first local areas; The determining whether there is an interference object in the foreground target that blocks the candidate local area in the target to be tracked includes: Determining whether the position of the foreground target overlaps with the position of the candidate local area in the target to be tracked; If yes, determining that the candidate local area is blocked by the interference object in the image frame; The determining of the perspective relationship between different targets in each of the image frames, and determining the foreground target of the target to be tracked according to the perspective relationship, comprises: Determining the bounding boxes of the different targets; Determine a first distance between a lower edge of a bounding box of the target to be tracked and a lower edge of the image frame, and determine a second distance between a lower edge of a bounding box of a target other than the target to be tracked and a lower edge of the image frame; If the first distance is greater than the second distance, determining that the target corresponding to the second distance is the foreground target of the target to be tracked; The obtaining, according to the local tracking information and the association relationship, the global tracking information of the target to be tracked in different image frames comprises: Determining a relative position relationship of the first local area in the target to be tracked according to the position information of the first local area and the position information of the bounding box of the target to be tracked; According to the relative position relationship, the local tracking information of the first local area in each of the image frames is converted into the global tracking information of the target to be tracked in each of the image frames.

2. A target tracking device, characterized in that: The device comprises: A first processing unit, configured to determine at least one first local area used when tracking a target to be tracked; A second processing unit, used for determining an association relationship between the first local area and the entire target to be tracked; A third processing unit, configured to perform local tracking between a plurality of image frames based on the first local area to obtain local tracking information; A fourth processing unit, configured to obtain global tracking information of the target to be tracked in different image frames according to the local tracking information and the association relationship; The first processing unit is used to determine at least one first local area used when tracking the target to be tracked, including: The first processing unit is specifically configured to determine at least one candidate local area for target tracking; and select, according to the statistical probability of each candidate local area being blocked, at least one candidate local area whose statistical probability of being blocked meets the first condition as the first local area; The first processing unit is used to determine at least one first local area used when tracking the target to be tracked, and further includes: The first processing unit is specifically configured to determine the tracking quality of the local tracking information of the target to be tracked based on the first local area whose statistical probability of being blocked satisfies the first condition; Selecting a local area whose tracking quality is not less than a preset quality threshold as the updated first local area; The first processing unit is used to determine at least one candidate local area for target tracking, including: The first processing unit is specifically configured to determine at least one candidate local area for target tracking according to characteristic attributes of different local areas of the target to be tracked, wherein the characteristic attributes include: the number of features and / or a significance parameter of a single feature; The first processing unit is configured to select, according to the statistical probability of each candidate local area being blocked, at least one candidate local area whose statistical probability of being blocked meets a first condition as the first local area, including: The first processing unit is specifically configured to select, according to the statistical probability of the candidate local area being blocked, a candidate local area whose statistical probability of being blocked meets the first condition and whose initial image frame is not blocked among the multiple image frames as the first local area; The number of the first local areas is N, where N is a positive integer equal to or greater than 2 and less than P; P is the number of local areas included in the target to be tracked; The third processing unit is used to perform local tracking between multiple image frames based on the first local area to obtain local tracking information, including: The third processing unit is specifically configured to perform local tracking in an m-th image frame among the multiple image frames based on the first local area that is not blocked in the m-1-th image frame, so as to obtain local tracking information of the target to be tracked in the m-th image frame, wherein the m-th image frame is an image frame subsequent to the m-1-th image frame, and m is a positive integer greater than 1; It also includes a fifth processing unit, which is used to select unobstructed local areas as the updated N first local areas according to the occlusion situation when it is detected that part of the first local areas are obstructed in one or more of the image frames; The fifth processing unit, used for selecting the unblocked local area as the updated N first local areas, comprises: The fifth processing unit is specifically used to determine the perspective relationship between different targets in each of the image frames; wherein the targets at least include the target to be tracked; Determining a foreground target of the target to be tracked according to the perspective relationship; Determine whether there is an interference object in the foreground target that blocks the candidate local area in the target to be tracked; Selecting unobstructed candidate local areas as the updated N first local areas; The determining whether there is an interference object in the foreground target that blocks the candidate local area in the target to be tracked includes: Determining whether the position of the foreground target overlaps with the position of the candidate local area in the target to be tracked; If yes, determining that the candidate local area is blocked by the interference object in the image frame; The first processing unit is used to determine the perspective relationship between different targets in each of the image frames, and determine the foreground target of the target to be tracked according to the perspective relationship, including: The first processing unit is specifically configured to determine the external frames of the different targets; Determine a first distance between a lower edge of a bounding box of the target to be tracked and a lower edge of the image frame, and determine a second distance between a lower edge of a bounding box of a target other than the target to be tracked and a lower edge of the image frame; If the first distance is greater than the second distance, determining that the target corresponding to the second distance is the foreground target of the target to be tracked; The fourth processing unit is used to obtain the global tracking information of the target to be tracked in different image frames according to the local tracking information and the association relationship, including: The fourth processing unit is specifically configured to determine a relative position relationship of the first local area in the target to be tracked according to the position information of the first local area and the position information of the external frame of the target to be tracked; According to the relative position relationship, the local tracking information of the first local area in each of the image frames is converted into the global tracking information of the target to be tracked in each of the image frames.

3. An electronic device, characterized in that: include: A processor and a memory for storing a computer program that can be run on the processor, wherein the processor is configured to perform the steps of the method of claim 1 when running the computer program.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to claim 1 are implemented.

Images