A multi-target tracking method, computer device and storage medium

Abstract

The application discloses a multi-target tracking method, a computer device and a storage medium. The method comprises the following steps: acquiring a current frame image and each track currently tracked, wherein the current frame image comprises a plurality of targets; determining a first motion matching cost and an appearance matching cost between each target in the current frame image and each track, and determining an occlusion condition of each target in the current frame image; adjusting the first motion matching cost between each target and each track by using the occlusion condition of each target, to obtain an occlusion-aware motion matching cost between each target and each track; fusing the appearance matching cost and the occlusion-aware motion matching cost between each target and each track, to obtain a total matching cost between each target and each track; and determining a matching relationship between each target and each track based on the total matching cost between each target and each track; wherein the matching relationship is used for updating the track. In the foregoing manner, the application can improve the target tracking accuracy.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a multi-target tracking method, computer device, and storage medium. Background Technology

[0002] Multi-target tracking (MOT) is a core task in intelligent visual perception, aiming to detect, correlate features, and continuously track the trajectories of multiple targets in consecutive video frames, outputting a stable and unique target ID and motion trajectory. Existing target tracking methods are prone to trajectory interruption and target omission when targets occlude each other or overlap significantly, resulting in low accuracy of target tracking results. Summary of the Invention

[0003] The main technical problem addressed by this application is to provide a multi-target tracking method, computer device, and storage medium that can improve target tracking accuracy.

[0004] To address the aforementioned technical problems, the first aspect of this application provides a multi-target tracking method. The method includes: acquiring a current frame image and currently tracked trajectories, wherein the current frame image includes several targets; determining a first motion matching cost and an appearance matching cost between each target and each trajectory in the current frame image, and determining the occlusion status of each target in the current frame image; the appearance matching cost is determined based on a first similarity between the trajectory appearance features of each trajectory and the current appearance features of each target, and a second similarity between the trajectory appearance enhancement features of each trajectory and the current appearance enhancement features of each target; the current appearance enhancement features of the target are determined by weighting each trajectory relative to the target using first attention weights and the trajectory appearance features of each trajectory, wherein each first attention weight is based on the current appearance features of the target and the first attention weights relative to the target. The first similarity between the trajectory appearance features of the trajectories is determined; the trajectory appearance enhancement features of the trajectories are obtained by weighting the second attention weights of each target relative to the trajectory and the current appearance features of each target, and the second attention weights are determined based on the first similarity between the trajectory appearance features of the trajectories and the current appearance features of each target; the first motion matching cost between each target and each trajectory is adjusted based on the occlusion status of each target to obtain the occlusion-aware motion matching cost between each target and each trajectory; the appearance matching cost and the occlusion-aware motion matching cost between each target and each trajectory are fused to obtain the total matching cost between each target and each trajectory; the matching relationship between each target and each trajectory is determined based on the total matching cost between each target and each trajectory; wherein, the matching relationship is used to update the trajectory.

[0005] Determining the occlusion status of each target in the current frame image includes: using the estimated position information of each trajectory in the current frame image to determine the occlusion status of the target corresponding to each trajectory in the current frame image.

[0006] The method involves using the estimated position information of each trajectory in the current frame image to determine the occlusion status of the target corresponding to each trajectory in the current frame image. This includes: determining the estimated bounding box region of the target corresponding to each trajectory in the current frame image based on the estimated position information; taking the target corresponding to each trajectory as the target to be judged, and determining the occlusion region of the target to be judged in the current frame image based on the overlap between the estimated bounding box region of the target to be judged and the estimated bounding box region of the target corresponding to other trajectories in the current frame image; and determining the final occlusion coefficient of the target to be judged based on the occlusion region of the target to be judged and the estimated bounding box region of the target to be judged. The final occlusion coefficient of the target to be judged represents the occlusion status of the target to be judged.

[0007] The final occlusion coefficient of the target to be judged is determined based on the occluded area of ​​the target to be judged and the estimated bounding box area of ​​the target to be judged. This includes: using the ratio between the area of ​​the occluded area and the area of ​​the estimated bounding box area as the initial occlusion coefficient of the target to be judged; and weighting the initial occlusion coefficient based on the distance between each pixel in the occluded area and the center point of the estimated bounding box of the target to be judged to obtain the final occlusion coefficient of the target to be judged. The distance is inversely proportional to the weighting weight.

[0008] Specifically, by utilizing the occlusion status of each target, the first motion matching cost between each target and each trajectory is adjusted to obtain the occlusion-aware motion matching cost between each target and each trajectory. This includes: obtaining the occlusion weight coefficient; wherein the occlusion weight coefficient represents the weight of the occlusion degree in the matching cost; for each trajectory, the final occlusion coefficient of the target corresponding to the trajectory and the first motion matching cost between the trajectory and any target are weighted using the occlusion weight coefficient to obtain the occlusion-aware motion matching cost between the trajectory and any target.

[0009] Specifically, the adjustment of the first motion matching cost between each target and each trajectory is based on the occlusion status of each target, and the targets in subsequent steps are some targets in the current frame image; wherein, some targets are targets with a detection confidence score higher than the confidence score threshold.

[0010] The process of determining the appearance matching cost between each target and each trajectory in the current frame image includes: enhancing the trajectory appearance features of each trajectory and the current appearance features of each target respectively, thereby obtaining the trajectory appearance enhancement features of each trajectory and the current appearance enhancement features of each target; and determining the appearance matching cost between each target and each trajectory based on the trajectory appearance enhancement features and the current appearance enhancement features.

[0011] The enhancement process for the trajectory appearance features of each trajectory and the current appearance features of each target includes: obtaining a first similarity matrix of the trajectory appearance features of each trajectory and the current appearance features of each target; wherein each row of the first similarity matrix corresponds to the current appearance features of the same target, and each column corresponds to the trajectory appearance features of the same trajectory; taking each row of the first similarity matrix as the current row, and performing temperature scaling on the similarity value of the current row to obtain the first attention weights of the current appearance features of the target corresponding to the current row on the trajectory appearance features of each trajectory; using the first attention weights to perform a weighted summation on the trajectory appearance features of each trajectory to obtain the current appearance enhancement features of the target corresponding to the current row; taking each column of the first similarity matrix as the current column, and performing temperature scaling on the similarity value of the current column to obtain the second attention weights of the trajectory appearance features of the trajectory corresponding to the current column on the current appearance features of each target; using the second attention weights to perform a weighted summation on the current appearance features of each target to obtain the trajectory appearance enhancement features of the trajectory corresponding to the current column.

[0012] The process of fusing the appearance matching cost and occlusion-aware motion matching cost between each target and each trajectory to obtain the total matching cost between each target and each trajectory includes: determining the fusion weight based on the occlusion status of each target; and using the fusion weight to weight the appearance matching cost and occlusion-aware motion matching cost between each target and each trajectory to obtain the total matching cost between each target and each trajectory.

[0013] The occlusion status of the target is represented by an occlusion coefficient. Based on the occlusion status of each target, the fusion weight is determined, including: averaging the occlusion coefficients of each target to obtain the total occlusion coefficient; and determining the fusion weight based on the total occlusion coefficient. The total occlusion coefficient and the fusion weight are directly proportional.

[0014] Specifically, the appearance matching cost and occlusion-aware motion matching cost between each target and each trajectory are weighted using fusion weights to obtain the total matching cost between each target and each trajectory. This includes: for any target and any trajectory, the appearance matching cost and occlusion-aware motion matching cost are weighted using fusion weights to obtain a first weighted result; the appearance matching cost is weighted using the difference between a preset value and the fusion weights to obtain a second weighted result; and the first weighted result and the second weighted result are summed to obtain the total matching cost between any target and any trajectory.

[0015] The process involves determining the matching relationship between each target and each trajectory based on the total matching cost between each target and each trajectory. This includes: performing a first matching using the Hungarian algorithm based on the total matching cost between each target and each trajectory to determine the targets that match each trajectory; where each target is a subset of targets, and the detection confidence of some targets is higher than a confidence threshold; in response to the first matching where a trajectory fails to match, calculating a second motion matching cost between the trajectory that failed to match in the first matching and the target that did not participate in the first matching; performing a second matching using the Hungarian algorithm based on the second motion matching cost to determine the matching relationship between the trajectory that failed to match in the first matching and the target that did not participate in the first matching; in response to the first matching where a target fails to match and the second matching where a trajectory fails to match, calculating a third motion matching cost between the trajectory that failed to match in the second matching and the target that failed to match in the first matching; performing a third matching using the Hungarian algorithm based on the third motion matching cost to determine the matching relationship between the trajectory that failed to match in the second matching and the target that failed to match in the first matching; and determining the final matching relationship between each target and each trajectory based on the matching relationships obtained from the three matchings.

[0016] The calculation of the second motion matching cost between the trajectory that failed to match for the first time and the target that did not participate in the matching for the first time includes: calculating the first intersection-union ratio between the estimated position information of the trajectory that failed to match for the first time in the current frame image and the actual position information of the target that did not participate in the matching for the first time, to obtain the second motion matching cost.

[0017] The calculation of the third motion matching cost between the second unmatched trajectory and the first unmatched target includes: obtaining the matching position information of the most recent match in the second unmatched trajectory; calculating the second intersection-union ratio between the matching position information and the actual position information of the first unmatched target to obtain the third motion matching cost.

[0018] To address the aforementioned technical problems, a second aspect of this application provides a computer device comprising a memory and a processor. The memory stores program instructions, and the processor executes the program instructions to implement the method provided in the first aspect.

[0019] To address the aforementioned technical problems, a third aspect of this application provides a computer-readable storage medium storing program instructions that can be executed by a processor to implement the method provided in the first aspect.

[0020] The beneficial effects of this application are as follows: Unlike existing technologies, this application determines the occlusion status of the target and uses this information to adjust the first motion matching cost between the target and each trajectory, thus obtaining the occlusion-aware motion matching cost between the target and each trajectory. This allows the occlusion-aware motion matching cost to reflect the true correlation probability of the target in complex scenes, eliminating matching bias caused by simply relying on position and improving the reliability of motion information. When determining the appearance matching cost, appearance enhancement features are introduced. The appearance matching cost is determined based on the first similarity between the trajectory appearance features of each trajectory and the current appearance features of each target, and the second similarity between the trajectory appearance enhancement features of each trajectory and the current appearance enhancement features of each target, which enhances the reliability of appearance information. Finally, the occlusion-aware motion matching cost and the appearance matching cost are fused to obtain the total matching cost. Based on the total matching cost, the matching relationship between each target and each trajectory is determined. By reliably matching motion information and appearance information, the accuracy of multi-target tracking can be improved. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in this application, the accompanying drawings required in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Among them: Figure 1 This is a flowchart illustrating one implementation of the multi-target tracking method provided in this application; Figure 2 This is a flowchart illustrating one embodiment of determining the occlusion status of each target in the current frame image provided in this application; Figure 3 This is a flowchart illustrating one implementation method for determining the appearance matching cost between each target and each trajectory in the current frame image provided in this application; Figure 4 This is a schematic diagram of the framework structure of one embodiment of the multi-target tracking device provided in this application; Figure 5 This is a schematic diagram of the framework structure of one embodiment of the computer device provided in this application; Figure 6 This is a schematic diagram of the framework structure of one embodiment of the computer-readable storage medium provided in this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0023] It should be noted that the embodiments of this application contain descriptions involving "first," "second," etc., which are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0024] In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. All directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of this application are only used to explain the relative positional relationships and movement of the components in a specific posture (as shown in the figures). If the specific posture changes, the directional indication will also change accordingly. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0025] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0026] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "several" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0027] This application provides the following embodiments, and each embodiment will be described in detail below.

[0028] Please see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the multi-target tracking method provided in this application. The method includes: S11: Obtain the current frame image and the currently tracked trajectories.

[0029] In one embodiment, the current frame image can be an image captured by an image acquisition device at the current moment, or an image to be processed at the current moment. The current frame image may include several targets. Targets include, but are not limited to, pedestrians, animals, and vehicles. The current frame image is input to a pre-trained target detection model, and the target detection model outputs the target detection results obtained by performing target detection on the current frame image. Exemplarily, the target detection results include the target's location information and detection confidence. The target's location information can be the coordinates of a bounding box that completely contains the target. Understandably, when the current frame image contains multiple targets, the target detection model outputs a set of target detection results, where each element in the set corresponds to a target. In a specific implementation scenario, a continuous video image sequence is input to the target detection model, which performs target detection on each frame of the continuous video image sequence to obtain the target detection results for each frame. Exemplarily, the target detection model can employ a YOLOv13n network.

[0030] The currently tracked trajectories are historical trajectories that are currently active. Each historical trajectory records the motion state of a target in multiple historical frames, the unique identifier (Identity, ID) of the trajectory, etc.

[0031] S12: Determine the first motion matching cost and appearance matching cost between each target and each trajectory in the current frame image, and determine the occlusion status of each target in the current frame image.

[0032] In one embodiment, the first motion matching cost between each target and each trajectory can be determined based on the actual position information of each target in the current frame image and the estimated position information of each trajectory in the current frame image.

[0033] The actual position information of each target in the current frame image is obtained through the target detection results output by the target detection model. The estimated position information of each trajectory in the current frame image is obtained by predicting and estimating the position of the target corresponding to the trajectory in the current frame using Kalman filtering. The estimated position information of each trajectory in the current frame image is the estimated position information of the target corresponding to each trajectory in the current frame image.

[0034] In one specific embodiment, for each target, the complete intersection-union ratio between the actual position information of the target and the estimated position information of each trajectory in the current frame image is calculated to obtain the first motion matching cost between the target and each trajectory.

[0035] In another specific embodiment, the first motion matching cost between each target and each trajectory can also be determined using Mahalanobis distance. For each target, the difference between the actual position information of the target and the estimated position information of each trajectory in the current frame image is calculated to obtain the prediction covariance matrix of each trajectory. The difference is then weighted and normalized using the inverse matrix of each prediction covariance matrix to obtain the first motion matching cost between the target and each trajectory. It is understood that in other embodiments, other methods may also be used to determine the first motion matching cost.

[0036] In one embodiment, the appearance matching cost between each target and each trajectory can be determined based on the trajectory appearance features of each trajectory and the current appearance features of each target. Specifically, for each target and each trajectory, the similarity between the current appearance features of the target and the trajectory appearance features of the trajectory can be calculated to obtain the appearance matching cost between the target and the trajectory.

[0037] In other embodiments, the trajectory appearance features of each trajectory and the current appearance features of each target can be enhanced first, resulting in enhanced trajectory appearance features for each trajectory and enhanced current appearance features for each target. Based on the enhanced appearance features of each trajectory and the enhanced current appearance features, the appearance matching cost between each target and each trajectory can be determined. For example, the similarity between the enhanced current appearance features of each target and the enhanced trajectory appearance features of each trajectory can be used as the appearance matching cost between each target and each trajectory.

[0038] In other embodiments, the appearance matching cost can also be determined based on a first similarity between the trajectory appearance features of each trajectory and the current appearance features of each target, and a second similarity between the trajectory appearance enhancement features of each trajectory and the current appearance enhancement features of each target. Specifically, the current appearance enhancement features of the target are determined by weighting each trajectory relative to the target with its first attention weights and the trajectory appearance features of each trajectory, where each first attention weight is determined based on the first similarity between the current appearance features of the target and the trajectory appearance features of each trajectory; the trajectory appearance enhancement features of the trajectory are obtained by weighting each target relative to the trajectory with its second attention weights and the current appearance features of each target, where each second attention weight is determined based on the first similarity between the trajectory appearance features of the trajectory and the current appearance features of each target.

[0039] In one embodiment, the occlusion status of the target corresponding to each trajectory in the current frame image can be determined using the estimated position information of each trajectory in the current frame image. For each trajectory, Kalman filtering is used to obtain the estimated position information of the target corresponding to that trajectory in the current frame image. In this way, the estimated position information of the target corresponding to all trajectories in the current frame image can be obtained. Taking the target corresponding to any trajectory as the target to be judged, the occlusion status of the target to be judged in the current frame image can be determined based on the estimated position information of the target to be judged and the estimated position information of the targets corresponding to other trajectories.

[0040] In other embodiments, the occlusion status of each target can also be determined directly based on the actual position information of each target in the current frame image. For example, a target is selected as the target to be judged, the intersection-union ratio (IUGR) between the actual position information of the target to be judged and the actual position information of other targets is calculated, and the other targets corresponding to the maximum IUGR are taken as occluded targets. The occlusion status of the target to be judged is determined based on the ratio of the overlap area between the occluded target and the target to be judged to the area of ​​the target to be judged. The above-described method for determining the occlusion status of each target in the current frame image is merely exemplary, and this application is not limited to the above method.

[0041] S13: Utilize the occlusion status of each target to adjust the first motion matching cost between each target and each trajectory, thereby obtaining the occlusion-aware motion matching cost between each target and each trajectory.

[0042] In one embodiment, the final occlusion coefficient of the target and the first motion matching cost can be weighted and summed to obtain the occlusion-aware motion matching cost. Specifically, an occlusion weight coefficient can be obtained, which represents the weight of the occlusion degree in the matching cost. For each trajectory, the final occlusion coefficient of the target corresponding to the trajectory and the first motion matching cost between the trajectory and any target are weighted using the occlusion weight coefficient to obtain the occlusion-aware motion matching cost between the trajectory and any target.

[0043] In one specific embodiment, the occlusion-aware motion matching cost is obtained using the following formula:

[0044] in, The cost of motion matching for occlusion perception between target i and trajectory j; Let be the final occlusion coefficient of the target corresponding to trajectory j; The occlusion weight coefficient should be set as needed, for example, to 0.3; The intersection-union ratio of the actual location information of target i with the trajectory j is given.

[0045] S14: Combine the appearance matching cost and occlusion-aware motion matching cost between each target and each trajectory to obtain the total matching cost between each target and each trajectory.

[0046] In one embodiment, the appearance matching cost and occlusion-aware motion matching cost between each target and each trajectory can be summed or weighted to obtain the total matching cost between each target and each trajectory. If weighted summation is performed, the weights corresponding to the appearance matching cost and occlusion-aware motion matching cost can be set as needed or determined based on the occlusion status of the target.

[0047] In one specific embodiment, fusion weights are determined based on the occlusion status of each target. These fusion weights are then used to weight the appearance matching cost and occlusion-aware motion matching cost between each target and each trajectory, resulting in the total matching cost between each target and each trajectory. The occlusion status of each target is represented by an occlusion coefficient, which can be either the initial occlusion coefficient or the final occlusion coefficient of the target.

[0048] In one embodiment, the occlusion coefficients of each target are averaged to obtain the total occlusion coefficient; based on the total occlusion coefficient, the fusion weight is determined; wherein the total occlusion coefficient and the fusion weight are proportional. For example, the numerical range of the total occlusion coefficient can be determined, and the weight corresponding to that range can be selected as the fusion weight. For instance, if the total occlusion coefficient is in a range greater than or equal to 0.5, the weight corresponding to that range is 0.7, and 0.7 is used as the fusion weight. The correspondence between the numerical range and the weight is preset. Alternatively, the total occlusion coefficient can be substituted into a mapping function to obtain the fusion weight, where the mapping function represents the difference between the target value and a preset multiple of the total occlusion coefficient. For example, the target value is 1.3, and the preset multiple is 2.

[0049] Furthermore, for the appearance matching cost and occlusion-aware motion matching cost between any target and any trajectory, the occlusion-aware motion matching cost is weighted using fusion weights to obtain a first weighted result; the appearance matching cost is weighted using the difference between a preset value and the fusion weights to obtain a second weighted result; the first weighted result and the second weighted result are summed to obtain the total matching cost between any target and any trajectory. For example, the preset value is equal to 1.

[0050] Specifically, as shown in the following formula:

[0051]

[0052]

[0053] in, The total occlusion coefficient is M, where M is the number of targets corresponding to the trajectory. To integrate weights, The cost of motion matching for occlusion perception between target i and trajectory j. The appearance matching cost between target i and trajectory j.

[0054] In other embodiments, the occlusion coefficient of a target can be directly weighted with the occlusion perception motion matching cost of the target to obtain a first weighted result; the appearance matching cost can be weighted using the difference between a preset value and the occlusion coefficient of a target to obtain a second weighted result; the first weighted result and the second weighted result can be summed to obtain the total matching cost between a target and a trajectory.

[0055] S15: Determine the matching relationship between each target and each trajectory based on the total matching cost between each target and each trajectory.

[0056] In one embodiment, based on the total matching cost between each target and each trajectory, the Hungarian algorithm is used for the first matching to determine the matching relationship between each target and each trajectory. This matching relationship is then used to update the trajectories.

[0057] In one embodiment, only the first matching may be performed, and the resulting matching relationship can be used as the final matching relationship. The targets participating in the first matching may include all targets, or only a subset of targets with a detection confidence level higher than a confidence threshold. In other embodiments, when only a subset of targets participate in the first matching, a second and third matching may be performed to avoid missed matches.

[0058] In one specific embodiment, based on the total matching cost between each target and each trajectory, a first matching is performed using the Hungarian algorithm to determine the targets that match each trajectory. At this point, each target is a subset of several targets, and the detection confidence of some targets is higher than a confidence threshold. In response to a failed trajectory match in the first matching, a second motion matching cost is calculated between the failed trajectory and the target not involved in the first matching. Based on the second motion matching cost, a second matching is performed using the Hungarian algorithm to determine the matching relationship between the failed trajectory and the target not involved in the first matching. In response to a failed target match in both the first and second matching, a third motion matching cost is calculated between the failed trajectory and the failed target. Based on the third motion matching cost, a third matching is performed using the Hungarian algorithm to determine the matching relationship between the failed trajectory and the failed target. Based on the matching relationships obtained from the three matchings, the final matching relationship between each target and each trajectory is determined.

[0059] The calculation of the second motion matching cost between the trajectory that failed to match in the first instance and the target that did not participate in the matching in the first instance includes: calculating the first intersection-union ratio (IUR) between the estimated position information of the trajectory that failed to match in the first instance in the current frame image and the actual position information of the target that did not participate in the matching in the first instance, to obtain the second motion matching cost. In the second matching, the IUR threshold can be set to 0.8.

[0060] The third motion matching cost is calculated between the second unmatched trajectory and the first unmatched target. This includes: obtaining the most recent matching position information in the second unmatched trajectory; calculating the second intersection-union ratio (IUU) between the matching position information and the actual position information of the first unmatched target to obtain the third motion matching cost. For the third matching, an IUU threshold of 0.3 can be set, and the most recent matching position information is the position information of the last successfully matched target in the second unmatched trajectory. This third matching method can resolve the trajectory disconnection problem caused by motion prediction drift and bounding box failure due to continuous target occlusion.

[0061] Finally, for successfully matched trajectories, update the trajectory position, motion state, and average appearance features; for unmatched targets with a detection confidence level higher than the confidence threshold, initialize and generate new trajectories; for unmatched trajectories, mark them as lost trajectories, set to retain them for τ frames for continuous prediction, and delete the trajectory from the tracking module if it fails to match within the timeout period. τ can be set to 15.

[0062] In one embodiment, the occlusion status of each target is used to adjust the first motion matching cost between each target and each trajectory, and the targets in subsequent steps are partial targets in the current frame image; wherein, partial targets are targets with a detection confidence score higher than a confidence score threshold. The confidence score threshold is set as needed.

[0063] The above method determines the occlusion status of the target and adjusts the first motion matching cost between the target and each trajectory to obtain the occlusion-aware motion matching cost between the target and each trajectory. This ensures that the occlusion-aware motion matching cost reflects the true correlation probability of the target in complex scenes, eliminating matching bias caused by relying solely on position and improving the reliability of motion information. When determining the appearance matching cost, appearance enhancement features are introduced. The appearance matching cost is determined based on the first similarity between the trajectory appearance features of each trajectory and the current appearance features of each target, and the second similarity between the trajectory appearance enhancement features of each trajectory and the current appearance enhancement features of each target. This enhances the reliability of appearance features. Finally, the occlusion-aware motion matching cost and the appearance matching cost are fused to obtain the total matching cost. Based on the total matching cost, the matching relationship between each target and each trajectory is determined. By reliably matching motion information and appearance information, the accuracy of multi-target tracking can be improved.

[0064] Please see Figure 2 , Figure 2 This is a flowchart illustrating an embodiment of determining the occlusion status of each target in the current frame image provided in this application. Determining the occlusion status of each target in the current frame image includes: S221: Based on the estimated location information, determine the estimated bounding box region of the target in the current frame image corresponding to each trajectory.

[0065] In one embodiment, each estimated location information includes the coordinates and dimensions of the estimated bounding box of the target in the current frame image. Therefore, the estimated bounding box region of each target in the current frame image can be determined directly based on the estimated location information.

[0066] S222: Take the target corresponding to each trajectory as the target to be judged, and determine the occluded area of ​​the target to be judged in the current frame image based on the overlap between the estimated bounding box area of ​​the target to be judged and the estimated bounding box area of ​​the target corresponding to other trajectories in the current frame image.

[0067] In one embodiment, the union of the estimated bounding box region of the target to be determined in the current frame image and the estimated bounding box regions of the targets corresponding to other trajectories in the current frame image is taken as the occluded region of the target to be determined in the current frame image.

[0068] S223: Based on the occluded area of ​​the target to be judged and the estimated bounding box area of ​​the target to be judged, determine the final occlusion coefficient of the target to be judged.

[0069] In one embodiment, the ratio between the area of ​​the occluded region and the area of ​​the estimated bounding box region is used as the initial occlusion coefficient of the target to be determined. The initial occlusion coefficient is weighted based on the distance between each pixel within the occluded region and the center point of the estimated bounding box of the target to be determined, resulting in the final occlusion coefficient of the target to be determined; wherein the distance is inversely proportional to the weighting. The final occlusion coefficient of the target to be determined characterizes the occlusion status of the target.

[0070] In one specific embodiment, the initial occlusion coefficient of the target can be expressed as:

[0071] in, Let be the initial occlusion coefficient of target i corresponding to trajectory i, and N be the number of occluded targets that occlude target i. The estimated bounding box region for target i. Let n be the estimated bounding box region of the target n corresponding to trajectory n. This represents the overlapping region between the estimated bounding box regions of target i and target n. Let represent the union of the estimated bounding box region of target i and the estimated bounding box regions of N targets. Indicates area.

[0072] Since the estimated bounding box of the target may contain background pixels, a Gaussian Map (GM) is introduced to mitigate the influence of the background. This refines the initial occlusion coefficients by adaptively weighting each pixel within the occluded region based on its distance from the center of the estimated bounding box of the target. Specifically, for the current image frame with N estimated bounding boxes, the GM value at pixel (x, y) within the estimated bounding box of the target is defined as:

[0073] in, , ( ) represents the centroid coordinates of the nth estimated bounding box, ( () represents the standard deviation of the target category in the horizontal and vertical directions. In this application, and The Gaussian kernel is set proportionally to the width and height of the estimated bounding box, allowing it to decay smoothly from the center to the edges.

[0074] The initial occlusion coefficient is weighted by the GM value of each pixel to obtain the final occlusion coefficient of the target to be judged.

[0075]

[0076] in, Let be the final occlusion coefficient of the target i to be judged.

[0077] In other embodiments, the initial occlusion coefficient can be directly used as the final occlusion coefficient.

[0078] In one embodiment, prior to step S222, depth sorting can be performed based on the estimated bounding boxes of each target in the current frame image to determine the positional relationship of each target, thereby determining whether a target is an occluded target or an occluded target. For example, if target i is occluded by target j, then target i is an occluded target and target j is an occluded target.

[0079] Specifically, the depth can be sorted according to the distance between the estimated bounding box of each target in the current frame image and the bottom edge of the current frame image, where the bottom edge of the current frame image is the edge containing the pixel with the largest pixel value. For example, the distance between the estimated bounding box of target i and the bottom edge of the current frame image is... The distance between the estimated bounding box of target j and the bottom edge of the current frame image is... .like We assume that target i is located behind target j. If there is an overlapping area between the two, we assume that target j occludes target i.

[0080] The estimated distance between the bounding box and the bottom edge of the current frame image can be the distance from the midpoint of the estimated bounding box to the bottom edge, or the distance from the nearest pixel in the bounding box to the bottom edge.

[0081] In this embodiment, the distance between the pixels in the occluded area and the center point of the estimated bounding box is used for inverse weighting. This makes the pixels closer to the center of the target contribute more to the occlusion coefficient, while the pixels farther away from the center contribute less. This effectively reduces the interference of background pixels and edge noise on the occlusion evaluation, so that the final occlusion coefficient can more accurately reflect the occlusion status of the core area of ​​the target. This is beneficial to improving the robustness of multi-target tracking methods in complex occlusion scenarios.

[0082] Please see Figure 3 , Figure 3 This is a flowchart illustrating an implementation method for determining the appearance matching cost between each target and each trajectory in the current frame image, as provided in this application. Determining the appearance matching cost between each target and each trajectory in the current frame image includes: S321: Enhance the trajectory appearance features of each trajectory and the current appearance features of each target respectively, and obtain the trajectory appearance enhancement features of each trajectory and the current appearance enhancement features of each target.

[0083] In one embodiment, a first similarity matrix is ​​obtained, consisting of trajectory appearance features of each trajectory and current appearance features of each target. Each row of the first similarity matrix corresponds to the current appearance features of the same target, and each column corresponds to the trajectory appearance features of the same trajectory. Each element in the first similarity matrix represents the first similarity between a target and a trajectory. The first similarity matrix can be represented by the inner product of the trajectory appearance features and the current appearance features.

[0084] The current appearance features of the target can be obtained by a re-identification (ReID) network model extracting features from the target detection results of the current frame image. The re-identification network model can adopt a lightweight OSNet network structure to extract features for each target detection result and output a 512-dimensional feature vector. The feature vectors corresponding to each target detection result are then normalized to obtain the current appearance feature set of each target.

[0085] The trajectory appearance features can be the average of all appearance features in the trajectory's appearance feature set. This set can include all image frames within the trajectory, or only the higher-quality image frames. In other embodiments, after matching the trajectory with targets in historical detection frames, the historical trajectory appearance features are updated using an exponential moving average to obtain the latest trajectory appearance features. When matching with targets in the current frame image, the latest trajectory appearance features are directly obtained.

[0086] Specifically, as shown in the following formula:

[0087] in, The trajectory appearance features of the trajectory in the t-th frame image are the latest trajectory appearance features. The trajectory appearance features of the trajectory in the (t-1)th frame image; Let be the appearance features of the target that matches the trajectory in the t-th frame image; This is a weight hyperparameter, which can be set to 0.9.

[0088] Furthermore, each row in the first similarity matrix is ​​taken as the current row, and the similarity value of the current row is temperature-scaled to obtain the first attention weights of the current appearance features of the target corresponding to the current row on the trajectory appearance features of each trajectory. The trajectory appearance features of each trajectory are then weighted and summed using these first attention weights to obtain the current appearance enhancement features of the target corresponding to the current row. Each column in the first similarity matrix is ​​taken as the current column, and the similarity value of the current column is temperature-scaled to obtain the second attention weights of the trajectory appearance features of the trajectory corresponding to the current column on the current appearance features of each target. The current appearance features of each target are then weighted and summed using these second attention weights to obtain the trajectory appearance enhancement features of the trajectory corresponding to the current column. After temperature scaling, normalization can be performed to obtain the weights.

[0089] Specifically, as shown in the following formula:

[0090]

[0091] in, The first attention weight of the current appearance features of target i on the trajectory appearance features of trajectory j; The element in the i-th row and j-th column of the first similarity matrix; This is a temperature parameter used to control the sharpness of attention distribution; it can be set to 100. The element in the i-th row and k-th column of the first similarity matrix; The second attention weight for the current appearance features of target i on the trajectory appearance features of trajectory j; The element in the k-th row and j-th column of the first similarity matrix; The trajectory appearance features of the j-th trajectory; Enhance the current appearance of target i; The current appearance features of target i; Enhance the trajectory appearance features of trajectory j.

[0092] In other embodiments, other methods can be used to enhance the trajectory appearance features and the current appearance features. For example, attention processing can be performed on the trajectory appearance features and the current appearance features separately to obtain corresponding attention features; the corresponding attention features can then be used to enhance the trajectory appearance features and the current appearance features to obtain trajectory appearance enhancement features and current appearance enhancement features. Specifically, enhancing the trajectory appearance features and the current appearance features using the corresponding attention features can be achieved by performing a weighted summation of the attention features with the corresponding trajectory appearance features or the current appearance features.

[0093] S322: Based on the appearance enhancement features of each trajectory and each current appearance enhancement feature, determine the appearance matching cost between each target and each trajectory.

[0094] In one embodiment, a second similarity is obtained between each trajectory appearance enhancement feature and each current appearance enhancement feature; based on each second similarity and each first similarity, the appearance matching cost between each target and each trajectory is obtained; wherein each first similarity is the similarity between the trajectory appearance feature of each trajectory and the current appearance feature of each target.

[0095] For example, for the current appearance features of each target and the trajectory appearance features of each trajectory, a first similarity is obtained between the two, and a second similarity is obtained between the current appearance enhancement features corresponding to the target and the trajectory appearance enhancement features corresponding to the trajectory. The second similarity and the first similarity are weighted and summed to obtain the appearance matching cost between the target and the trajectory; or, the difference between a preset value and the mean of the second similarity and the first similarity is used as the appearance matching cost between the target and the trajectory; or, the mean of the second similarity and the first similarity can be used as the appearance matching cost between the target and the trajectory.

[0096] In one specific embodiment, the appearance matching cost between the target and the trajectory is calculated using the following formula.

[0097]

[0098] in, Let $i$ be the cost of appearance matching between target $i$ and trajectory $j$. Let the first similarity be the current appearance features of target i and the trajectory appearance features of trajectory j. The second similarity is defined as the current appearance enhancement features of target i and the trajectory appearance enhancement features of trajectory j.

[0099] In this embodiment, by enhancing the trajectory appearance features of each trajectory and the current appearance features of each target, the appearance features can be integrated with intra-frame global context information, suppressing interference from occlusion areas and background redundancy information, thereby improving the ability to distinguish similar appearance targets. The appearance matching cost is determined based on the enhanced trajectory appearance features and the current appearance enhancement features, enabling the appearance matching cost to more accurately reflect the possibility of identity association between targets, improving the reliability of appearance information. This is beneficial for maintaining accurate association between targets and trajectories in scenarios where multiple targets occlude each other, causing the original appearance features to be missing or blurred.

[0100] It is understood that, in the specific implementation of the above method, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0101] Please see Figure 4 , Figure 4 This is a schematic diagram of the framework structure of one embodiment of the multi-target tracking device provided in this application.

[0102] The multi-target tracking device 40 includes an acquisition module 41, a parameter determination module 42, an adjustment module 43, a fusion module 44, and a matching module 45. The acquisition module 41 acquires the current frame image and the currently tracked trajectories, wherein the current frame image includes several targets. The parameter determination module 42 determines the first motion matching cost and appearance matching cost between each target and each trajectory in the current frame image, and determines the occlusion status of each target in the current frame image. The adjustment module 43 adjusts the first motion matching cost between each target and each trajectory based on the occlusion status of each target, obtaining the occlusion-aware motion matching cost between each target and each trajectory. The fusion module 44 fuses the appearance matching cost and occlusion-aware motion matching cost between each target and each trajectory to obtain the total matching cost between each target and each trajectory. The matching module 45 determines the matching relationship between each target and each trajectory based on the total matching cost between each target and each trajectory; wherein the matching relationship is used to update the trajectories.

[0103] The above scheme improves the accuracy of motion matching cost by adjusting the first motion matching cost between the target and the trajectory by utilizing the occlusion of the target. Furthermore, when matching the target and the trajectory based on the adjusted motion matching cost and appearance matching cost, the accuracy of target tracking can be improved.

[0104] It should be noted that the multi-target tracking device and the multi-target tracking method provided in the above embodiments belong to the same concept. The specific ways in which each module and unit performs operations have been described in detail in the method embodiments, and will not be repeated here. In practical applications, the multi-target tracking device provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0105] Please see Figure 5 , Figure 5 This is a schematic diagram of the framework structure of one embodiment of the computer device provided in this application.

[0106] Computer device 50 includes a memory 51 and a processor 52, which are coupled to each other. The memory 51 stores program instructions, and the processor 52 executes the program instructions stored in the memory 51 to implement the steps of any of the above method embodiments. In a specific implementation scenario, computer device 50 is an electronic device capable of executing program instructions, which may include, but is not limited to, a microcomputer or a server. In addition, computer device 50 may also include mobile devices such as laptops and tablets, which are not limited here.

[0107] Specifically, processor 52 controls itself and memory 51 to implement the steps in any of the above method embodiments. Processor 52 may also be referred to as a CPU (Central Processing Unit). Processor 52 may be an integrated circuit chip with signal processing capabilities. Processor 52 may also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. A general-purpose processor may be a microprocessor or any conventional processor. Furthermore, processor 52 may be implemented using integrated circuit chips.

[0108] Please see Figure 6 , Figure 6 This is a schematic diagram of the framework structure of one embodiment of the computer-readable storage medium provided in this application.

[0109] The computer-readable storage medium 60 stores program instructions 61 that can be executed by a processor, the program instructions 61 being used to implement the steps in any of the above method embodiments.

[0110] The storage media mentioned in this article include: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks or optical disks, and other media that can store program code.

[0111] This application also provides a computer program product comprising a computer program that, when executed by a processor, can implement the steps described in any of the foregoing method embodiments. Specifically, the computer program product can be a software or program product containing a computer program, capable of running on a computing device or stored on any available medium.

[0112] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0113] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0114] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0115] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0116] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods in the various embodiments of this application.

[0117] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, and thus stored in a computer-readable storage medium for execution by a computing device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Therefore, this application is not limited to any particular hardware and software combination.

[0118] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the individual through pop-up information or by asking the individual to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0119] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A multi-target tracking method, characterized in that, include: Acquire the current frame image and the currently tracked trajectories, wherein the current frame image includes several targets; A first motion matching cost and an appearance matching cost are determined between each target and each trajectory in the current frame image, and the occlusion status of each target in the current frame image is determined. The appearance matching cost is determined based on a first similarity between the trajectory appearance features of each trajectory and the current appearance features of each target, and a second similarity between the trajectory appearance enhancement features of each trajectory and the current appearance enhancement features of each target. The current appearance enhancement features of the target are determined by weighting each trajectory relative to the target with each first attention weight and each trajectory appearance feature, where each first attention weight is determined based on the first similarity between the current appearance features of the target and the trajectory appearance features of each trajectory. The trajectory appearance enhancement features of the trajectory are obtained by weighting each target relative to the trajectory with each second attention weight and each target's current appearance features, where each second attention weight is determined based on the first similarity between the trajectory appearance features of the trajectory and the current appearance features of each target. The first motion matching cost between each target and each trajectory is adjusted based on the occlusion status of each target to obtain the occlusion-aware motion matching cost between each target and each trajectory. The appearance matching cost and the occlusion-aware motion matching cost between each target and each trajectory are fused to obtain the total matching cost between each target and each trajectory; Based on the total matching cost between each target and each trajectory, a matching relationship between each target and each trajectory is determined; wherein, the matching relationship is used to update the trajectory.

2. The method according to claim 1, characterized in that, Determining the occlusion status of each target in the current frame image includes: Using the estimated position information of each trajectory in the current frame image, the occlusion status of the target corresponding to each trajectory in the current frame image is determined.

3. The method according to claim 2, characterized in that, The step of determining the occlusion status of the target corresponding to each trajectory in the current frame image using the estimated position information of each trajectory in the current frame image includes: Based on the estimated location information, the estimated bounding box region of the target corresponding to each trajectory in the current frame image is determined; The targets corresponding to each trajectory are respectively taken as targets to be judged. Based on the overlapping area between the estimated bounding box area of ​​the target to be judged and the estimated bounding box area of ​​the target corresponding to other trajectories in the current frame image, the occluded area of ​​the target to be judged in the current frame image is determined. Based on the occluded region of the target to be judged and the estimated bounding box region of the target to be judged, the final occlusion coefficient of the target to be judged is determined; wherein, the final occlusion coefficient of the target to be judged represents the occlusion status of the target to be judged.

4. The method according to claim 3, characterized in that, The step of determining the final occlusion coefficient of the target based on the occluded region of the target and the estimated bounding box region of the target includes: The ratio between the area of ​​the occluded region and the area of ​​the estimated bounding box region is used as the initial occlusion coefficient of the target to be judged. Based on the distance between each pixel within the occluded area and the center point of the estimated bounding box of the target to be judged, the initial occlusion coefficient is weighted to obtain the final occlusion coefficient of the target to be judged; wherein, the distance is inversely proportional to the weighting weight.

5. The method according to claim 1, characterized in that, The occlusion status of the target is characterized by the final occlusion coefficient of the target; The step of adjusting the first motion matching cost between each target and each trajectory based on the occlusion status of each target to obtain the occlusion-aware motion matching cost between each target and each trajectory includes: Obtain the occlusion weight coefficient; wherein, the occlusion weight coefficient represents the weight of the occlusion degree in the matching cost; For each trajectory, the final occlusion coefficient of the target corresponding to the trajectory and the first motion matching cost between the trajectory and any of the targets are weighted using the occlusion weight coefficient to obtain the occlusion-aware motion matching cost between the trajectory and any of the targets.

6. The method according to claim 1, characterized in that, The first motion matching cost between each target and each trajectory is adjusted by utilizing the occlusion status of each target, and the target in the subsequent steps is a portion of the target in the current frame image; The aforementioned target is a target with a detection confidence level higher than a confidence threshold.

7. The method according to claim 1, characterized in that, Determining the appearance matching cost between each target and each trajectory in the current frame image includes: The trajectory appearance features of each trajectory and the current appearance features of each target are enhanced respectively to obtain the trajectory appearance enhancement features of each trajectory and the current appearance enhancement features of each target. Based on the appearance enhancement features of each trajectory and the current appearance enhancement features, the appearance matching cost between each target and each trajectory is determined.

8. The method according to claim 7, characterized in that, The enhancement of the trajectory appearance features of each trajectory and the current appearance features of each target includes: Obtain a first similarity matrix of trajectory appearance features of each trajectory and current appearance features of each target; wherein, each row of the first similarity matrix corresponds to the current appearance features of the same target, and each column corresponds to the trajectory appearance features of the same trajectory. Each row in the first similarity matrix is ​​taken as the current row. The similarity value of the current row is temperature scaled to obtain the first attention weight of the current appearance feature of the target corresponding to the current row on the trajectory appearance feature of each trajectory. The trajectory appearance features of each trajectory are weighted and summed using the first attention weight to obtain the current appearance enhancement feature of the target corresponding to the current row. Each column in the first similarity matrix is ​​taken as the current column, and the similarity value of the current column is temperature scaled to obtain the second attention weight of the trajectory appearance feature of the trajectory corresponding to the current column to the current appearance feature of each target; the current appearance features of each target are weighted and summed using the second attention weight to obtain the trajectory appearance enhancement feature of the trajectory corresponding to the current column.

9. The method according to claim 1, characterized in that, The method of fusing the appearance matching cost and the occlusion-aware motion matching cost between each target and each trajectory to obtain the total matching cost between each target and each trajectory includes: Based on the occlusion status of each target, the fusion weights are determined; The appearance matching cost and the occlusion-aware motion matching cost between each target and each trajectory are weighted using the fusion weights to obtain the total matching cost between each target and each trajectory.

10. The method according to claim 9, characterized in that, The occlusion status of the target is represented by an occlusion coefficient; The determination of fusion weights based on the occlusion status of each target includes: The average occlusion coefficient of each target is calculated to obtain the total occlusion coefficient. The fusion weight is determined based on the total occlusion coefficient; wherein the total occlusion coefficient and the fusion weight are proportional. And / or, the step of using the fusion weights to weight the appearance matching cost and the occlusion-aware motion matching cost between each target and each trajectory to obtain the total matching cost between each target and each trajectory includes: For the appearance matching cost and the occlusion-aware motion matching cost between any target and any trajectory, the occlusion-aware motion matching cost is weighted using the fusion weight to obtain a first weighted result; the appearance matching cost is weighted using the difference between the preset value and the fusion weight to obtain a second weighted result. The sum of the first weighted result and the second weighted result is used to obtain the total matching cost between any target and any trajectory.

11. The method according to claim 1, characterized in that, Determining the matching relationship between each target and each trajectory based on the total matching cost between each target and each trajectory includes: Based on the total matching cost between each target and each trajectory, the Hungarian algorithm is used for the first matching to determine the target that matches each trajectory; wherein each target is a subset of the several targets, and the detection confidence of the subset of targets is higher than the confidence threshold; In response to the first match having a failed trajectory match, a second motion matching cost is calculated between the first failed trajectory and the first target not participating in the match. Based on the second motion matching cost, the Hungarian algorithm is used to perform a second matching to determine the matching relationship between the trajectory that failed to match in the first time and the target that did not participate in the first matching. In response to the first match where the target failed to match and the second match where the trajectory failed to match, a third motion matching cost is calculated between the trajectory that failed to match in the second match and the target that failed to match in the first match. Based on the third motion matching cost, the Hungarian algorithm is used to perform a third matching to determine the matching relationship between the trajectory that failed to match in the second time and the target that failed to match in the first time. Based on the matching relationships obtained from the three matching operations, the final matching relationship between each target and each trajectory is determined.

12. The method according to claim 11, characterized in that, The calculation of the second motion matching cost between the trajectory that failed to match in the first instance and the target that did not participate in the matching in the first instance includes: Calculate the first intersection-union ratio (IUU) between the estimated position information of the trajectory that failed to match for the first time in the current frame image and the actual position information of the target that did not participate in the first match, and obtain the second motion matching cost. And / or, the calculation of the third motion matching cost between the second unmatched trajectory and the first unmatched target includes: Obtain the most recent matching position information from the trajectory that failed to match the second time; Calculate the second crossover ratio (CRO) between the matched location information and the actual location information of the target that failed to match in the first attempt, and obtain the third motion matching cost.

13. A computer device, characterized in that, The method includes a memory and a processor, wherein the memory stores program instructions and the processor executes the program instructions to implement the method according to any one of claims 1 to 12.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program instructions that can be executed by a processor to implement the method of any one of claims 1 to 12.

Images