Similar interference resistant target recapture method and device

Abstract

The invention relates to the technical field of image tracking algorithms, and discloses a target recapturing method and device capable of resisting similar object interference. The method comprises the following steps: carrying out real-time target detection and full-situation monitoring on a current field-of-view image, and establishing an overall motion relation model of a detected target and a neighborhood target; judging whether the target is lost and judging a merging and splitting state of the target; when the target is lost, whether target recapture is started or not is judged according to the merging and splitting state; and when target recapture is started, determining the recapture target through the overall motion relation model of the neighborhood target corresponding to the target or predicting the recapture target through the long and short time motion model. According to the method, multi-target correlation tracking and full-situation monitoring are adopted, an overall motion relation model is established, the spatial relation between the targets is better understood and analyzed, the merging and splitting states between the targets are marked, a large amount of redundant interference can be eliminated during recapture, and the target recapture efficiency is improved. And the target recapture probability under the condition of similar object interference is improved, so that the accurate tracking precision is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image tracking algorithms, and in particular to a target recapture method and device that resists interference from similar objects. Background Art

[0002] Target tracking technology is a technology that uses sensors and algorithm technology to identify, locate and track moving targets in videos, images or other sensory data. The difficulties of target tracking technology are mainly problems such as memory tracking, anti-interference and re-capture of targets. In the process of target tracking, there are many interference problems, which may lead to reduced tracking accuracy or even tracking failure. The main interference problems include morphological changes, scale changes, illumination changes, occlusion problems, background interference, motion blur, etc.

[0003] Among them, the problem of interference from similar targets has a greater impact on tracking, which is mainly reflected in the following aspects: (1) Feature confusion: When there are targets with similar appearance in the scene, the tracking algorithm may confuse the features of these targets. For example, if two vehicles have the same color and appearance, the tracking algorithm may mistake them for the same target, resulting in incorrect tracking trajectories. (2) Target loss: Due to the interference of similar targets, the tracking algorithm may lose focus on the real target and instead track the wrong similar target. This will cause the trajectory of the real target to be suddenly interrupted or lost during the tracking process. (3) Trajectory breakage: In scenes where similar targets frequently appear or interact, the tracking algorithm may cause trajectory breakage due to confusion between targets. That is, the tracking trajectory that should have been continuous is mistakenly divided into multiple fragments due to the interference of similar targets. (4) ID switching: When similar targets are close to or intersecting each other in the scene, the tracking algorithm may assign the identity of one target to another target, resulting in an ID switching problem. This situation is particularly common when moving vehicles frequently overtake each other.

[0004] Tracking and recapture technology combines tracking technology and recapture technology to continuously track the target signal and recapture it after the signal is lost. It mainly solves the problem of tracking loss after the target is interfered with and re-tracking the correct target. At present, in order to solve the problem of similar target interference affecting target recapture, the commonly used target recapture methods are traditional segmentation detection, deep learning detection, and Re-ID technology based on deep learning.

[0005] However, although some anti-tracking interference algorithms have achieved certain results in dealing with interference problems in the target tracking process, they still have some obvious shortcomings. The main shortcomings include (1) Traditional segmentation detection methods are usually based on some basic attributes of the image, such as target color, texture, shape and other underlying features for distinction. These features cannot distinguish the target differences among multiple similar targets. (2) Target detection technology based on deep learning relies on models to learn features from a large amount of data for classification and positioning. When the shapes of two targets are highly similar, the model may find it difficult to obtain enough discriminative features for accurate distinction. (3) Re-ID technology mainly relies on extracting feature representations of the target for matching and identification. When the shapes of two targets are very similar, the extracted features are basically the same, making it difficult to distinguish. Summary of the invention

[0006] The main purpose of the present invention is to provide a target recapture method and device that are resistant to similar object interference, aiming to solve at least one of the above problems.

[0007] To achieve the above object, the present invention provides a target recapture method that resists similar interference, comprising:

[0008] Perform real-time target detection and full situation monitoring on the current field of view image, and establish an overall motion relationship model between the detected target and neighboring targets;

[0009] Determine whether the target is lost and determine the merge and split status of the target;

[0010] When the target is lost, judging whether to start recapture of the target according to the merge and split state;

[0011] When recapture of the target is turned on, the recapture target is confirmed by the overall motion relationship model of the neighborhood targets corresponding to the target or predicted by the long-short time motion model.

[0012] In some embodiments, the real-time target detection and full situation monitoring of the current field of view image and the establishment of an overall motion relationship model between the detected target and the neighboring targets include:

[0013] Performing real-time target detection on the current field of view image based on a deep learning detection algorithm to obtain multiple targets and assigning a unique identifier to each of the targets;

[0014] Performing multi-target tracking on the multiple targets based on a target tracking algorithm;

[0015] Performing full situation monitoring on the target and its neighboring targets, and recording the positional relationship between the target and its neighboring targets;

[0016] Establishing an overall motion relationship model between the target and the neighboring targets based on the position relationship;

[0017] The overall motion relationship model is updated in real time.

[0018] In some embodiments, the performing of full situation monitoring on the target and the neighboring targets of the target and recording the positional relationship between the target and the neighboring targets includes:

[0019] During multi-target tracking, the position and motion information of each target is recorded;

[0020] Recording the identifier, position, speed and movement direction of the neighboring target corresponding to each target, wherein the neighboring target is located within a neighborhood range of a preset radius centered on the target;

[0021] Counting the positional relationship between the target and the neighboring targets in the observation area;

[0022] The positional relationship of the neighboring targets corresponding to each target is stored to perform full situation monitoring.

[0023] In some embodiments, the determining whether the target is lost and determining the merge / split status of the target includes:

[0024] Traversing the target and the neighboring targets;

[0025] Determine whether there is a similar target moving in the same direction as the target;

[0026] If there is a similar target moving in the same direction, determining whether the target is lost;

[0027] If the target is lost, determining whether the neighborhood target is lost;

[0028] When the neighborhood target is not lost, tracking the neighborhood target and determining whether the IOU with the neighborhood target is greater than a first threshold and is in a close state when the target is lost;

[0029] If so, the merge split state is the target merge;

[0030] If not, the position relationship of the target is predicted and whether to start recapture is determined according to the merging and splitting status of the neighboring targets.

[0031] In some embodiments, the predicting of the position relationship of the target and judging whether to start recapture according to the merge and split status of the neighboring targets includes:

[0032] Determine whether an update target appears within a preset range of the neighborhood target;

[0033] If no update target appears, the merge split state is non-merge split, and recapture of the target is started;

[0034] If an update target appears, determine whether the neighborhood target is far away from the update target and count the IOU changes;

[0035] Determine whether there is a historical merge when the target is lost;

[0036] If there is no historical merge when the target is lost, the merge split state is non-merge split, and recapture of the target is enabled;

[0037] If there is a historical merge when the target is lost, determine whether the neighborhood target and the update target simultaneously satisfy the conditions that the IOU is less than the second threshold, are in a far-away state, and the far-away distance is greater than the third threshold;

[0038] If not, the merge-split state is target merge, and recapture of the target is not enabled;

[0039] If so, the split state is merged into a split state, and recapture of the target is started.

[0040] In some embodiments, if the target is lost, after determining whether the neighboring target is lost, the method further includes:

[0041] When the neighborhood target is lost, determining whether the IOU of the target with the neighborhood target is greater than a first threshold and is in a close state when the target is lost;

[0042] If yes, the merge-split state is target merge, and recapture of the target is not enabled;

[0043] If not, the merge-split state is non-merge-split, and recapture of the target is enabled.

[0044] In some embodiments, if there is a similar target moving in the same direction, after determining whether the target is lost, the method further includes:

[0045] If the target is not lost, tracking the target and determining whether the neighboring target is lost;

[0046] When the neighborhood target is not lost, tracking the neighborhood target and determining whether the target has a historical merger;

[0047] If the target has a historical merger, determine whether the neighborhood target and the target simultaneously satisfy the conditions that the IOU is less than the fourth threshold, are in a far-away state, and the far-away distance is greater than the fifth threshold;

[0048] If yes, the merge-split state is the target split, otherwise the merge-split state is the target merge;

[0049] If the target does not have a historical merge, the merge-split state is non-merge-split.

[0050] In some embodiments, if the target is not lost, after tracking the target and determining whether the neighboring target is lost, the method further includes:

[0051] When the neighborhood target is lost, determining whether the IOU of the neighborhood target with the target is greater than a sixth threshold and is in a close state when the neighborhood target is lost;

[0052] If so, the merge split state is the target merge;

[0053] If not, the merge-split state is non-merge-split.

[0054] In some embodiments, when the recapture of the target is enabled, the recapture target is confirmed by using the overall motion relationship model of the neighboring targets corresponding to the target or the recapture target is predicted by using the long-short time motion model, including:

[0055] When the target is lost and the neighboring target is not lost, recapture of the target is started and the positional relationship of the recaptured target is determined through the overall motion relationship model of the neighboring targets corresponding to the target;

[0056] When the target is lost and the neighboring target is lost, establishing a long-short time motion model and predicting the positional relationship of the recaptured target according to the long-short time motion model;

[0057] Performing feature weighted fusion according to the spatiotemporal information and visual feature information before the target is lost to obtain a target weighted feature value;

[0058] The recapture target is confirmed according to the merge and split state of the target, the target weighted characteristic value and the position relationship.

[0059] In addition, to achieve the above-mentioned purpose, the present invention also proposes a target re-capture device that resists interference from similar objects, comprising:

[0060] The detection and monitoring module is used to perform real-time target detection and full situation monitoring on the current field of view image, and establish an overall motion relationship model between the detected target and the neighboring targets;

[0061] A merge / split judgment module is used to judge whether the target is lost and to judge the merge / split status of the target;

[0062] A recapture start judgment module, used to judge whether to start the recapture of the target according to the merge and split state when the target is lost;

[0063] The target recapture module is used to confirm the recapture target through the overall motion relationship model of the neighborhood targets corresponding to the target or predict the recapture target through the long-short time motion model when the recapture of the target is turned on.

[0064] The present invention provides a target recapture method that resists interference from similar objects, including: performing real-time target detection and full-situation monitoring on the current field of view image, establishing an overall motion relationship model between the detected target and the neighboring target; judging whether the target is lost and judging the merge and split state of the target; when the target is lost, judging whether to turn on the recapture of the target according to the merge and split state; when turning on the recapture of the target, confirming the recapture target through the overall motion relationship model of the neighboring target corresponding to the target or predicting the recapture target through the long-short time motion model. In the present invention, the overall motion relationship model of the main tracking target and its neighboring target is established by multi-target association tracking and full-situation monitoring, which is helpful to analyze the relative position, distance and possible interactive behavior between the targets, and can better understand and analyze the spatial relationship between the targets, thereby improving the accuracy and stability of tracking, marking the merge and split states between all targets, and eliminating a large amount of redundant interference when recaptures the target, thereby significantly improving the probability of target recapture under the interference of similar objects, thereby improving the accurate tracking accuracy of the tracker, and solving the problems of tracking loss caused by interference from similar targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 A schematic flow chart of an embodiment of a target recapture method for resisting interference from similar objects according to the present invention;

[0066] Figure 2 This is an example flow chart of target recapture against similar interference involved in an embodiment of the present invention;

[0067] Figure 3 A schematic diagram of the size of the observation area involved in the embodiment of the present invention;

[0068] Figure 4 This is a main target merging and splitting judgment flow chart involved in the embodiment of the present invention;

[0069] Figure 5 A schematic diagram of a recapture process for judging the start of recapture according to the merge and split status of a neighborhood target involved in an embodiment of the present invention;

[0070] Figure 6 A schematic diagram of a process for establishing a long- and short-term motion model according to an embodiment of the present invention;

[0071] Figure 7 A flow chart of the relationship between the long-term and short-term motion models used in predicting the position of a main target according to an embodiment of the present invention;

[0072] Figure 8 It is a schematic diagram of the memory tracking and recapture process involved in the embodiment of the present invention;

[0073] Fig. 9 A schematic diagram of the interference tracking effect of similar targets involved in the embodiment of the present invention;

[0074] Fig.10 A schematic diagram of the tracking effect of the anti-similar target interference recapture algorithm involved in the embodiment of the present invention;

[0075] Fig.11 It is a structural block diagram of an embodiment of a target re-capture device for resisting similar interference of the present invention;

[0076] Fig.12 The present invention is a schematic diagram of the structure of an electronic device in a hardware operating environment according to an embodiment of the present invention.

[0077] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0078] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0079] It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0080] In addition, the descriptions of "first", "second", etc. in the present invention are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0081] The present invention provides a target recapture method and device that are resistant to similar object interference.

[0082] The embodiment of the present invention provides a target recapture method that resists interference from similar objects, referring to Figure 1 , Figure 1 Schematic diagram of a flow chart of an embodiment of a target recapture method for resisting similar interference of the present invention.

[0083] like Figure 1 As shown, the target recapture method against similar interference includes:

[0084] Step S100: performing real-time target detection and full situation monitoring on the current field of view image, and establishing an overall motion relationship model between the detected target and neighboring targets;

[0085] Step S200: determining whether the target is lost and determining the merge / split status of the target;

[0086] Step S300: when the target is lost, judging whether to start recapture of the target according to the merge and split state;

[0087] Step S400: when recapture of the target is turned on, the recapture target is confirmed by using the overall motion relationship model of the neighborhood targets corresponding to the target or predicted by using the long-short time motion model.

[0088] It should be noted that the execution subject in this embodiment may be an electronic device, which may be a computer device with data processing functions, or other devices that can achieve the same or similar functions. This embodiment does not limit this. In this embodiment, a computer device is used as an example for explanation.

[0089] It can be understood that this embodiment is explained by taking the ARM tracking hardware system applied to the motion device platform as an example, which is used to solve the problem of recapture against similar interference under the motion device platform, that is, the recapture technology problem after the target is lost due to the cross interference of similar objects in the target tracking process. In this embodiment, after verification by semi-physical simulation of the tracking system and field mounting test, the target recapture method against similar interference can significantly improve the target recapture probability under similar interference, thereby improving the accurate tracking accuracy of the tracker.

[0090] For example, reference Figure 2The target recapture method against similar interference includes: (1) an AI detection algorithm based on deep learning detects the target appearing in the current field of view in real time; (2) when the AI ​​detection algorithm detects the target, all targets are input into the target tracking algorithm to track all targets; (3) the whole situation monitoring of other targets around the target is carried out, the information of the surrounding neighboring targets is recorded for each target, the interference objects, the same direction and the opposite direction targets similar to the main tracking target are counted, the merge and split states between the targets are recorded, and the interference degree of the main target is judged by the merge and split states, and then whether the target is lost is judged; (4) the overall motion relationship model between the target and other surrounding targets (neighboring targets) is established, and it is updated in real time during the target movement process. After the target is lost, the position where the main target reappears is judged by the position relationship of the neighboring targets and the speed of the main target; (5) the target similarity value is integrated by different weighting methods for different features in combination with the feature information before the target is lost. The feature weighted fusion is based on the previous situation perception results, the importance and contribution of the features to improve the accuracy and reliability of the data. The recaptured target is finally confirmed by the target merge and split state, the target weighted feature value and the target position relationship. The implementation process of the target recapture method against similar object interference is described below with reference to specific steps and drawings.

[0091] In one embodiment, real-time target detection and full-situation monitoring are performed on the current field of view image, and an overall motion relationship model between the detected target and the neighboring targets is established, including: real-time target detection is performed on the current field of view image based on a deep learning detection algorithm to obtain multiple targets and assign a unique identifier to each of the targets; multi-target tracking is performed on the multiple targets based on a target tracking algorithm; full-situation monitoring is performed on the target and the neighboring targets of the target, and the positional relationship between the target and the neighboring targets is recorded; an overall motion relationship model between the target and the neighboring targets is established based on the positional relationship; and the overall motion relationship model is updated in real time.

[0092] Specifically, the AI ​​detection algorithm based on deep learning and the anti-similar multi-target tracking algorithm are run in parallel; first, the AI ​​detection algorithm based on deep learning detects the targets appearing in the current field of view in real time to obtain multiple targets; when the AI ​​detection algorithm detects the targets, all targets are input into the target tracking algorithm (anti-similar multi-target tracking algorithm) to track all targets; the latest feature information such as the multi-target tracking pipeline (target tracking chain) and target position are updated in real time. Here, the deep learning detection algorithm includes but is not limited to the AI ​​detection algorithm based on deep learning.

[0093] Exemplarily, the AI ​​detection algorithm based on deep learning detects the target in each frame and assigns a unique identifier (ID) to each detected target, which will remain unchanged during the subsequent tracking process to distinguish different targets. For each detected target, its visual features are extracted, including but not limited to color, texture, shape, etc., which will be used to identify and track the target in the subsequent process. The data association algorithm (such as the IOU data association algorithm) is used to match the target of the current frame with the target of the previous frame to achieve multi-target tracking in order to determine the position of each target in the current frame and maintain its trajectory. In the process of multi-target tracking, the trajectory information of each target is updated, including position, speed, acceleration, etc. At the same time, state estimation and prediction are performed according to the motion state of the target (such as stationary, uniform speed, acceleration, etc.) so as to track the target more accurately in the next frame.

[0094] In this embodiment, AI target detection, multi-target association tracking, full situation monitoring, and target speed and position prediction algorithms based on deep learning are adopted, and a position relationship model of the main tracking target and its neighboring targets is established at the same time. Due to the use of the AI ​​target detection algorithm, the target can be identified more accurately. Compared with the traditional algorithm, the AI ​​detection algorithm used in this embodiment has higher accuracy and reliability, avoiding error problems such as subjectivity and artificial setting of algorithm thresholds. The AI ​​target detection algorithm uses the capabilities of deep learning and machine learning, and can autonomously learn and adapt to new data and scenes. Whether it is processing complex visual features or coping with different environmental conditions, the AI ​​target detection algorithm has excellent adaptability.

[0095] In one embodiment, the target and the neighborhood targets of the target are fully monitored, and the positional relationship between the target and the neighborhood targets is recorded, including: in the multi-target tracking process, recording the position and motion information of each of the targets; recording the identifier, position, speed and motion direction of the neighborhood targets corresponding to each of the targets, wherein the neighborhood targets are located within a neighborhood range of a preset radius centered on the target; counting the positional relationship between the target and the neighborhood targets within the observation area; and storing the positional relationship between the neighborhood targets corresponding to each of the targets for full-situation monitoring.

[0096] Specifically, other targets around the target are fully monitored. In the process of multi-target tracking, not only the position and motion information of each target itself is recorded, but also the information of other targets in the surrounding neighborhood is recorded. In this embodiment, by defining a neighborhood range (for example, an area within a preset radius R centered on the target), other targets in the area are searched as neighborhood targets corresponding to the target. The relevant information of the neighborhood target includes but is not limited to the ID, position, speed, direction of movement, etc. of the neighborhood target. Each target will store the relevant information of its neighborhood target and monitor the motion situation of other targets in real time.

[0097] Exemplarily, the observation area is N times the width and height of the target, the center of the observation area is the center of the target in normal tracking, and is the long-term prediction center in memory tracking; targets outside the observation area are marked as non-primary tracking targets. Figure 3 The size of the observation area is as follows: during normal tracking, the positional relationship between the main tracking target and the neighboring targets is counted in the "observation area", and then the positional relationship is further analyzed to establish a positional relationship model (overall motion relationship model) to obtain the approach, distance, merger, and splitting trends of the main tracking target and the neighboring targets. The size of the observation area is N times the width and height of the main tracking target. The center of the observation area is the center of the main tracking target during normal tracking and the long-term prediction center during memory tracking. For targets outside the observation area, only "non-main tracking target marking" is performed without positional relationship statistics, which can reduce the amount of algorithm calculation and ensure real-time resistance to interference from similar targets.

[0098] It should be noted that the above-mentioned deep learning-based AI detection algorithm and the anti-similar multi-target tracking algorithm run in parallel. The synchronous operation of the two algorithms requires a large amount of hardware resources. The hardware system for running the method in this embodiment can be developed using the Feiteng 2000+ / 64 platform, ARMv8.0 architecture, 64 FTC-662 autonomous processing cores and 8 DDR42666MHz storage controllers, which can achieve a maximum memory capacity of 1T.

[0099] It is understandable that by establishing an overall motion relationship model, when a target is occluded or temporarily lost, the neighborhood target information previously recorded by the overall motion relationship model can be used for auxiliary tracking. If a target is occluded in a certain frame, the motion trajectory and position relationship of other targets in its neighborhood can be analyzed to predict the possible position of the occluded target. When the target reappears, this information can be used to re-establish tracking. In particular, for targets with the same appearance and color, when other external morphological features cannot be distinguished, the use of the target position relationship model (overall motion relationship model) can quickly determine the target to be recaptured.

[0100] In this embodiment, full situation monitoring is performed on other surrounding targets: the full situation monitoring used in the multi-target association tracking algorithm performs information statistics and status marking on the target pipeline (target tracking chain). The algorithm determines that the tracking pipeline pointed by the optical axis is the main tracking target, and other targets are all neighboring targets. The situation information such as approach, distance, overlap, and split between all targets is marked, so that a large amount of redundant interference can be eliminated first when the target is recaptured. The target pointed by the optical axis of the current motion device platform is the main target (main tracking target), and a position relationship model (overall motion relationship model) between the main target and the surrounding neighboring targets is established, which is helpful to analyze the relative position, distance and possible interaction between the targets, and can better understand and analyze the spatial relationship between the targets, thereby improving the accuracy and stability of tracking, and preventing the target tracking chain from being disconnected during tracking.

[0101] In one embodiment, determining whether the target is lost and determining the merge and split state of the target include: traversing the target and the neighborhood target; determining whether there is a similar target moving in the same direction as the target; if there is a similar target moving in the same direction, determining whether the target is lost; if the target is lost, determining whether the neighborhood target is lost; when the neighborhood target is not lost, tracking the neighborhood target and determining whether the IOU of the target with the neighborhood target is greater than a first threshold and is in a close state when the target is lost; if so, the merge and split state is target merge; if not, predicting the position relationship of the target and determining whether to turn on recapture based on the merge and split state of the neighborhood target.

[0102] Specifically, the target behavior is analyzed and predicted using spatiotemporal information, which includes but is not limited to the position and motion information of the target itself and the information of the neighboring targets (ID, position, speed, direction of motion, etc.). Predict the moving path of the target or the possible interaction. By recording the inter-frame position information and calculating the relative displacement, the close and distant relationship between the targets is obtained. At the same time, when the IOUs of the two targets intersect, the change in the IOU value is recorded, and the target merging and splitting states are judged. When judging the target merging and splitting states, the distance between the main target and the neighboring target, the IOU value, the close and distant state, the motion state and other comprehensive judgments are mainly used. In this embodiment, the target whose distance and IOU are greater than the first threshold and is in the close state among the targets moving in the same direction is mainly set. The interference degree of the target (main target) is judged by the merged state, and then whether the target is lost; at the same time, the lost target is judged by the split state whether it reappears, and the target recapture time is accurately grasped. During the tracking process, the target (main tracking target) and its neighboring targets are marked as "main target" and "neighboring target". When recaptured, most of the interference information of non-"main target" can be quickly excluded according to the marking situation.

[0103] It should be noted that IOU (Intersection over Union) is an indicator widely used in computer vision tasks such as target detection and image segmentation, which is used to evaluate the similarity between two bounding boxes (or two regions). IOU measures the degree of overlap between two regions, and its value range is between 0 and 1. Exemplarily, it is determined whether the IOU is greater than a first threshold, and the first threshold is an IOU threshold set according to actual conditions, such as 0.5 to 0.9.

[0104] In one example, reference Figure 4 The judgment process shown is as follows: Generally, similar targets moving in the opposite direction can be excluded by the feature of the moving direction, while similar targets moving in the same direction are difficult to exclude by factors such as appearance, speed, and moving direction. Therefore, this embodiment and method need to perform merge and split judgment on targets moving in the same direction. Merge and split mainly refers to the state where the main tracking target is interfered by the neighboring target and the intersection and union ratio occurs with the neighboring target.

[0105] like Figure 4 As shown in the figure, after judging the target merging and splitting process, the merged and splitting state is used to guide the subsequent target recapture. After judging the target merging, recapture is temporarily not performed to prevent the recapture of erroneous similar targets. After the target splitting condition (target splitting or non-merged splitting state) is met, recapture can be turned on after judging the target splitting completion through the position relationship of the neighboring targets, so as to ensure the accuracy of target recapture. When the main target is lost and the neighboring targets exist, the main target's position can be predicted based on the neighboring targets. At the same time, the merged and splitting state of the neighboring targets and the newly appeared targets is judged. In the "merged state", it is believed that the main target has not yet left the occlusion state, and recapture is not turned on. In the case of "target splitting", it is believed that the main target has left the occlusion state, and recapture can be turned on. In the case of "non-merged splitting state", it is believed that the main target is not disturbed by similar objects, and recapture can be turned on.

[0106] In one embodiment, reference Figure 5The process of starting recapture according to the merge and split state of the neighborhood target is shown as follows: predicting the position relationship of the target and judging whether to start recapture according to the merge and split state of the neighborhood target, including: judging whether an updated target appears within a preset range of the neighborhood target; if the updated target does not appear, the merge and split state is non-merge and split, and recapture of the target is started; if the updated target appears, judging whether the neighborhood target is far away from the updated target and counting the IOU changes; judging whether there is a historical merge when the target is lost; if there is no historical merge when the target is lost, the merge and split state is non-merge and split, and recapture of the target is started; if there is a historical merge when the target is lost, judging whether the neighborhood target and the updated target simultaneously meet the conditions that the IOU is less than a second threshold, are in a far away state, and the far away distance is greater than a third threshold; if not, the merge and split state is a target merge, and recapture of the target is not started; if so, the merge and split state is a split state, and recapture of the target is started.

[0107] In one embodiment, if the target is lost, after determining whether the neighborhood target is lost, it also includes: when the neighborhood target is lost, determining whether the IOU of the target with the neighborhood target is greater than a first threshold and is in a close state when the target is lost; if so, the merge and split state is target merge, and recapture of the target is not enabled; if not, the merge and split state is non-merge and split, and recapture of the target is enabled.

[0108] Specifically, Figure 4 As shown, when the main target is lost and the neighboring target is lost, it is determined whether the IOU of the main target with the neighboring target is greater than the first threshold and is in a close state when the main target is lost; if both conditions are met at the same time, the merge split state is target merge, and the recapture of the target is not enabled; if any one of the two conditions is not met or both conditions are not met, the merge split state is non-merge split, and the recapture of the target is enabled.

[0109] Exemplarily, it is determined whether the IOU is less than a second threshold value, the second threshold value is an IOU threshold value set according to actual conditions, such as 0.1 to 0.5; it is determined whether the distance is greater than a third threshold value, the third threshold value is a distance threshold value set according to actual conditions, such as 50 to 100. The specific values ​​of each threshold value can be set according to actual conditions, and this embodiment does not limit this.

[0110] In one embodiment, if there are similar targets moving in the same direction, after determining whether the target is lost, it also includes: if the target is not lost, tracking the target and determining whether the neighborhood target is lost; when the neighborhood target is not lost, tracking the neighborhood target and determining whether the target has a historical merge; if the target has a historical merge, determining whether the neighborhood target and the target simultaneously satisfy the conditions that the IOU is less than a fourth threshold, are in a far away state, and the far away distance is greater than a fifth threshold; if so, the merge-split state is target split, otherwise the merge-split state is target merge; if the target has no historical merge, the merge-split state is non-merge-split.

[0111] In one embodiment, if the target is not lost, after tracking the target and determining whether the neighborhood target is lost, it also includes: when the neighborhood target is lost, determining whether the IOU of the neighborhood target with the target is greater than a sixth threshold and is in a close state when the neighborhood target is lost; if so, the merge-split state is a target merge; if not, the merge-split state is a non-merge-split.

[0112] Specifically, refer to Figure 4 , when the main target is not lost, track the main target normally and determine whether the neighboring target is lost; when the neighboring target is not lost, it is necessary to first determine whether the main target has a historical merger; if the main target has a historical merger, it is necessary to determine whether the neighboring target and the main target meet the IOU less than the fourth threshold, are in a distant state, and the distance is greater than the fifth threshold. If all three conditions are met at the same time, the merge-split state is target split; if any of the three conditions are not met or all three conditions are not met, the merge-split state is target merge. Figure 4 As shown, when the main target is not lost but the neighboring target is lost, it is judged whether the IOU of the neighboring target with the main target is greater than the sixth threshold and is in a close state when the neighboring target is lost; if both conditions are met at the same time, the merge split state is target merge; if any of the two conditions is not met or both conditions are not met, it is a non-merge split state.

[0113] Here, specific values ​​of the fourth threshold, the fifth threshold, and the sixth threshold can be set according to actual conditions, and this embodiment does not limit this.

[0114] In one embodiment, when recapture of the target is turned on, the recapture target is confirmed by the overall motion relationship model of the neighborhood targets corresponding to the target or predicted by the long-short time motion model, including: when the target is lost and the neighborhood targets are not lost, recapture of the target is turned on and the positional relationship of the recapture target is determined by the overall motion relationship model of the neighborhood targets corresponding to the target; when the target is lost and the neighborhood targets are lost, a long-short time motion model is established and the positional relationship of the recapture target is predicted based on the long-short time motion model; feature weighted fusion is performed based on the spatiotemporal information and visual feature information before the target is lost to obtain a target weighted feature value; and the recapture target is confirmed based on the merge and split state of the target, the target weighted feature value and the positional relationship.

[0115] Specifically, after the target is lost, the relevant feature information of the target under normal tracking is found in the historical frame by backtracking, and then compared with the suspected target in the current frame, and multiple suspected targets with the smallest difference are obtained by various feature weighted fusion methods. The recapture target is determined by combining the target's merge and split state, position relationship, and "main target-neighborhood target" state mark.

[0116] For example, reference Figure 2 When the main tracking target is lost and the merge split state is judged to be target split or non-merge split, the target is recaptured. If the neighborhood target is lost, long-term and short-term motion prediction is performed. If the neighborhood target is not lost, the target position is determined according to the overall motion relationship model of the neighborhood target.

[0117] Specifically, refer to Figure 6 The long and short time motion model establishment process shown in Figure 7 The target motion prediction process shown in the figure starts when the main target is lost and the neighboring target is lost. In this embodiment, the long-short motion model can be used to predict the position relationship of the main target. The short-term and long-term predictions are combined. The short-term prediction can quickly respond to the acceleration and deceleration of the target, and the long-term prediction can filter the clutter noise during the motion process. Therefore, the use of the long-short prediction model can avoid the target in a single prediction mode. The prediction position error caused by data abnormality.

[0118] It is understandable that the program first executes Figure 6 After the function formulation process shown (establishing two modules, short-term motion model and long-term motion model), Figure 7 The process shown in the figure determines whether the target turns and the target speed changes according to the state before the main tracking target is lost, and selects the short-term prediction method or the long-term prediction method. Here, Figure 6The sequence of first establishing a short-term motion model (target short-term trajectory filtering and speed estimation) and then establishing a long-term motion model (target long-term trajectory filtering and speed estimation) is exemplarily shown in FIG. This embodiment does not limit the sequence of the specific function formulation process.

[0119] In this embodiment, the target speed position prediction algorithm used in the long- and short-time motion prediction is a long- and short-time prediction algorithm, and the position of the optical axis is predicted in combination with the optical axis angle. The long- and short-time prediction position can filter out singular point noise to prevent large prediction deviations.

[0120] For example, reference Figure 8 The memory tracking recapture process shown in the figure, after recapture is turned on, the target with the characteristics of target movement speed, direction, color, neighborhood position relationship, etc. that meet the conditions is searched among the suspected targets that have been roughly screened in the "neighborhood target" state marked by situation monitoring, and the recapture is completed when the continuous multi-frame recapture conditions are met. Specifically, Figure 8 As shown in the figure, all suspected targets are traversed. If they are unmarked neighborhood targets, the position relationship model (overall motion relationship model) is predicted to obtain the merge and split judgment result. If the target is merged, all suspected targets are traversed in the next frame; if the target is split or non-merged, recapture is turned on. Compare the target's spatiotemporal information and the visual features extracted during target detection (such as color, texture, shape, etc.). If multiple consecutive frames meet the comparison requirements, the target is determined to be a recaptured target.

[0121] In this embodiment, situational awareness is combined to determine the complexity of the scene and to adaptively adjust the feature weights. Combined with weighted fusion of target features, the accuracy of recapture of the target can be further guaranteed. By weighted fusion of different features, the advantages of various features can be fully utilized to improve the tracking system's ability to identify the target, thereby more accurately recapture the target. In this embodiment, situational monitoring can be used to perceive the complexity of the scene around the current target and adaptively adjust the feature weights to better capture the dynamic changes of the target.

[0122] It should be noted that, for example, reference Fig. 9 and Fig.10 A similar target occlusion example is given. Fig. 9 The similar targets shown in the figure interfere with the tracking effect. The commonly used tracking algorithm cannot distinguish between the two vehicles. Fig. 9 Figure a in the middle shows that there are similar targets in the image, namely 1-car-s and 2-car-s. Fig. 9 Figure b in the middle shows that car 1 (1-car-s) is blocked by car 2 (2-car-s). After car 1 is blocked by car 2, the algorithm continues to track and identifies car 2 as car 1 ( Fig. 9 (see Figure c), tracking error. Fig.10The similar target interference tracking effect shown in Fig.10 Figure a in the middle shows that there are two similar target vehicles in the image. Fig.10 In the middle figure b, it is determined that car 1 is blocked by car 2. The target recapture method of the embodiment with anti-similar interference is adopted to accurately track car 1 ( Fig.10 Figure c in the middle shows that the anti-similar target interference algorithm successfully recaptures).

[0123] In this embodiment, AI is used to detect targets, track and associate multiple targets, monitor other targets around the target, analyze the merge and split status of the target, and establish an overall position relationship model. When the target is interfered by similar objects, it can quickly and accurately judge, and locate the target through the position relationship of multiple targets and long- and short-term motion prediction methods, which can achieve rapid target recapture and tracking under the interference of similar objects. In this embodiment, the AI ​​algorithm detects targets accurately, and the position relationship model is simple and fast to establish. When multiple targets are visually similar, it can effectively recapture the correct target, effectively improving the accuracy of target recapture.

[0124] The present embodiment provides a target recapture method that resists interference from similar objects, including: performing real-time target detection and full-situation monitoring on the current field of view image, establishing an overall motion relationship model between the detected target and the neighboring target; judging whether the target is lost and judging the merge and split state of the target; when the target is lost, judging whether to turn on the recapture of the target according to the merge and split state; when turning on the recapture of the target, confirming the recapture target through the overall motion relationship model of the neighboring target corresponding to the target or predicting the recapture target through the long-short time motion model. In the present embodiment, the overall motion relationship model of the main tracking target and its neighboring targets is established by multi-target association tracking and full-situation monitoring, which is helpful to analyze the relative position, distance and possible interaction behavior between the targets, and can better understand and analyze the spatial relationship between the targets, thereby improving the accuracy and stability of tracking, marking the merge and split states between all targets, and eliminating a large amount of redundant interference when recaptures the target, thereby significantly improving the probability of target recapture under similar interference, thereby improving the accurate tracking accuracy of the tracker, and solving the problem of tracking loss caused by similar target interference.

[0125] In addition, an embodiment of the present invention further proposes a storage medium, on which is stored a target recapture program that resists interference from similar objects. When the target recapture program that resists interference from similar objects is executed by a processor, the steps of the target recapture method that resists interference from similar objects as described above are implemented.

[0126] Reference Fig.11 , Fig.11It is a structural block diagram of an embodiment of a target re-capture device for resisting similar object interference of the present invention.

[0127] like Fig.11 As shown, the target re-capture device that resists similar interference includes:

[0128] The detection and monitoring module 10 is used to perform real-time target detection and full situation monitoring on the current field of view image, and establish an overall motion relationship model between the detected target and the neighboring targets;

[0129] A merge / split judgment module 20 is used to judge whether the target is lost and to judge the merge / split status of the target;

[0130] A recapture start judgment module 30, used to judge whether to start the recapture of the target according to the merge and split state when the target is lost;

[0131] The target recapture module 40 is used to confirm the recapture target through the overall motion relationship model of the neighboring targets corresponding to the target or predict the recapture target through the long-short time motion model when the recapture of the target is turned on.

[0132] In this embodiment, AI is used to detect targets, track and associate multiple targets, monitor other targets around the target, analyze the merge and split status of the target, and establish an overall position relationship model. When the target is interfered by similar objects, it can quickly and accurately judge, and locate the target through the position relationship of multiple targets and long- and short-term motion prediction methods, which can achieve rapid target recapture and tracking under the interference of similar objects. In this embodiment, the AI ​​algorithm detects targets accurately, and the position relationship model is simple and fast to establish. When multiple targets are visually similar, it can effectively recapture the correct target, effectively improving the accuracy of target recapture.

[0133] The present embodiment provides a target recapture device that is resistant to interference from similar objects. In the present embodiment, multi-target association tracking and full-situation monitoring are used to establish an overall motion relationship model of the main tracking target and its neighborhood targets, which is helpful to analyze the relative positions, distances and possible interactive behaviors between targets, and can better understand and analyze the spatial relationships between targets, thereby improving the accuracy and stability of tracking. The merge and split states between all targets are marked, and a large amount of redundant interference can be eliminated during target recapture, thereby significantly improving the target recapture probability under interference from similar objects, thereby improving the accurate tracking accuracy of the tracker, and solving problems such as tracking loss due to interference from similar targets.

[0134] It should be noted that, for technical details that are not fully described in the embodiment of the target re-capture device against similar interference, reference can be made to the target re-capture method against similar interference as described above provided in any embodiment of the present invention, and they will not be repeated here.

[0135] In addition, to achieve the above-mentioned purpose, the present invention also proposes an electronic device, which includes: a memory, a processor, and a target recapture program that is resistant to similar interference and is stored in the memory and can be run on the processor, and the target recapture program that is resistant to similar interference and is configured to implement the target recapture method that is resistant to similar interference as described above.

[0136] Reference Fig.12 , Fig.12 The figure is a schematic diagram of the structure of an electronic device of the hardware operating environment involved in the embodiment of the present invention.

[0137] like Fig.12 As shown, the electronic device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (Wireless-Fidelity, Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM memory) or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0138] Those skilled in the art will understand that Fig.12 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0139] like Fig.12 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a target recapture program that resists similar interference.

[0140] exist Fig.12In the electronic device shown, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the electronic device of the present invention can be set in the electronic device, and the electronic device calls the target recapture program for resisting similar interference stored in the memory 1005 through the processor 1001, and executes the target recapture method for resisting similar interference provided by an embodiment of the present invention.

[0141] It should be understood that the above is only an example and does not constitute any limitation on the technical solution of the present invention. In specific applications, technicians in this field can make settings as needed, and the present invention does not limit this.

[0142] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present invention. In practical applications, technicians in this field can select part or all of them according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.

[0143] In addition, it should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0144] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0145] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0146] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A target recapture method against similar interference, characterized in that: include: Perform real-time target detection and full situation monitoring on the current field of view image, and establish an overall motion relationship model between the detected target and neighboring targets; Determine whether the target is lost and determine the merge and split status of the target; When the target is lost, judging whether to start recapture of the target according to the merge and split state; When recapture of the target is turned on, the recapture target is confirmed by the overall motion relationship model of the neighborhood targets corresponding to the target or predicted by the long-short time motion model.

2. The method according to claim 1, characterized in that The real-time target detection and full situation monitoring of the current field of view image are performed to establish an overall motion relationship model between the detected target and the neighboring targets, including: Performing real-time target detection on the current field of view image based on a deep learning detection algorithm to obtain multiple targets and assigning a unique identifier to each of the targets; Performing multi-target tracking on the multiple targets based on a target tracking algorithm; Performing full situation monitoring on the target and its neighboring targets, and recording the positional relationship between the target and its neighboring targets; Establishing an overall motion relationship model between the target and the neighboring targets based on the position relationship; The overall motion relationship model is updated in real time.

3. The method according to claim 2, characterized in that The step of performing full situation monitoring on the target and the neighboring targets of the target and recording the positional relationship between the target and the neighboring targets includes: During multi-target tracking, the position and motion information of each target is recorded; Recording the identifier, position, speed and movement direction of the neighboring target corresponding to each target, wherein the neighboring target is located within a neighborhood range of a preset radius centered on the target; Counting the positional relationship between the target and the neighboring targets in the observation area; The positional relationship of the neighboring targets corresponding to each target is stored to perform full situation monitoring.

4. The method according to claim 1, characterized in that The determining whether the target is lost and determining the merge / split state of the target includes: Traversing the target and the neighboring targets; Determine whether there is a similar target moving in the same direction as the target; If there is a similar target moving in the same direction, determining whether the target is lost; If the target is lost, determining whether the neighborhood target is lost; When the neighborhood target is not lost, tracking the neighborhood target and determining whether the IOU with the neighborhood target is greater than a first threshold and is in a close state when the target is lost; If so, the merge split state is the target merge; If not, the position relationship of the target is predicted and whether to start recapture is determined according to the merging and splitting status of the neighboring targets.

5. The method according to claim 4, characterized in that The predicting of the position relationship of the target and judging whether to start recapture according to the merging and splitting status of the neighboring targets includes: Determine whether an update target appears within a preset range of the neighborhood target; If no update target appears, the merge split state is non-merge split, and recapture of the target is started; If an update target appears, determine whether the neighborhood target is far away from the update target and count the IOU changes; Determine whether there is a historical merge when the target is lost; If there is no historical merge when the target is lost, the merge split state is non-merge split, and recapture of the target is enabled; If there is a historical merge when the target is lost, determine whether the neighborhood target and the update target simultaneously satisfy the conditions that the IOU is less than the second threshold, are in a far-away state, and the far-away distance is greater than the third threshold; If not, the merge-split state is target merge, and recapture of the target is not enabled; If so, the split state is merged into a split state, and recapture of the target is started.

6. The method according to claim 4, characterized in that If the target is lost, after determining whether the neighborhood target is lost, the method further includes: When the neighborhood target is lost, determining whether the IOU of the target with the neighborhood target is greater than a first threshold and is in a close state when the target is lost; If yes, the merge-split state is target merge, and recapture of the target is not enabled; If not, the merge-split state is non-merge-split, and recapture of the target is enabled.

7. The method according to claim 4, characterized in that If there is a similar target moving in the same direction, after determining whether the target is lost, the method further includes: If the target is not lost, tracking the target and determining whether the neighboring target is lost; When the neighborhood target is not lost, tracking the neighborhood target and determining whether the target has a historical merger; If the target has a historical merger, determine whether the neighborhood target and the target simultaneously satisfy the conditions that the IOU is less than the fourth threshold, are in a far-away state, and the far-away distance is greater than the fifth threshold; If yes, the merge-split state is the target split, otherwise the merge-split state is the target merge; If the target does not have a historical merge, the merge-split state is non-merge-split.

8. The method according to claim 7, characterized in that If the target is not lost, after tracking the target and determining whether the neighborhood target is lost, the method further includes: When the neighborhood target is lost, determining whether the IOU of the neighborhood target with the target is greater than a sixth threshold and is in a close state when the neighborhood target is lost; If so, the merge split state is the target merge; If not, the merge-split state is non-merge-split.

9. The method according to any one of claims 1 to 8, characterized in that When the recapture of the target is turned on, the recapture target is confirmed by using the overall motion relationship model of the neighboring targets corresponding to the target or the recapture target is predicted by using the long-short time motion model, including: When the target is lost and the neighboring target is not lost, recapture of the target is started and the positional relationship of the recaptured target is determined through the overall motion relationship model of the neighboring targets corresponding to the target; When the target is lost and the neighboring target is lost, establishing a long-short time motion model and predicting the positional relationship of the recaptured target according to the long-short time motion model; Performing feature weighted fusion according to the spatiotemporal information and visual feature information before the target is lost to obtain a target weighted feature value; The recapture target is confirmed according to the merge and split state of the target, the target weighted characteristic value and the position relationship.

10. A target recapture device that resists interference from similar objects, characterized in that: include: The detection and monitoring module is used to perform real-time target detection and full situation monitoring on the current field of view image, and establish an overall motion relationship model between the detected target and the neighboring targets; A merge / split judgment module is used to judge whether the target is lost and to judge the merge / split status of the target; A recapture start judgment module, used to judge whether to start the recapture of the target according to the merge and split state when the target is lost; The target recapture module is used to confirm the recapture target through the overall motion relationship model of the neighborhood targets corresponding to the target or predict the recapture target through the long-short time motion model when the recapture of the target is turned on.

Images