Head area three-dimensional re-identification and consistency tracking method based on binocular depth perception equipment

Abstract

The invention discloses a head area three-dimensional re-identification and consistency tracking method based on binocular depth perception equipment, and the method comprises the steps: obtaining tracking target data through employing binocular depth camera equipment, and determining the three-dimensional coordinates of a target head area through employing a four-stage rollback strategy; generating a combined vector as a main channel feature vector, introducing an auxiliary channel feature vector, and enabling the feature vector to enter a similarity measurement space; and calculating the target similarity based on the three-dimensional position gating threshold, recovering the original ID of the target meeting the similarity threshold, and storing the original ID into a memory entry. According to the invention, through fusing three-dimensional space information and multi-mode appearance features provided by binocular depth perception equipment, the problem of identity loss (ID) which is easy to occur when a traditional two-dimensional tracking method shields a target, changes illumination or leaves a field for a short time is effectively solved. The head three-dimensional positioning is carried out by adopting a four-stage rollback strategy, the head three-dimensional coordinate of the target can be stably output, and the robustness and continuity of target tracking in a complex scene are improved.

Description

Technical Field

[0001] This invention relates to the field of target tracking cross-identification and consistency maintenance using computer vision, and particularly to a method for three-dimensional re-identification and consistency tracking of the head region based on a binocular depth sensing device. Background Technology

[0002] Traditional target tracking technology based on two-dimensional images mainly relies on bounding box localization on the image plane and appearance descriptors (such as color, texture and other features) for identification and trajectory association.

[0003] While these methods are effective in ideal scenarios, they have significant limitations: when a target is occluded, briefly leaves the frame and then re-enters, the shooting angle changes drastically, or the ambient lighting is significantly disturbed, the system is highly prone to misidentifying the same target as a new object and assigning it a new IDSwitch. This error severely disrupts the continuity of the tracking context, leading to a significant reduction in the accuracy of subsequent downstream applications such as behavior analysis and statistical counting.

[0004] Existing re-identification (Re-ID) technologies attempt to piece together broken trajectories through similarity matching of appearance features. However, in the absence of absolute physical scale information, these methods typically rely on soft thresholds for similarity judgment. This strategy, which depends on pure appearance matching, has inherent drawbacks: First, in long-distance scenes, target appearance information is blurred, easily leading to mismatches; second, when groups of people with similar clothing appear in a scene (such as those in uniform), the high degree of confusion in the feature space makes it difficult for re-identification algorithms to accurately distinguish between different individuals, resulting in frequent identity confusion. Furthermore, the accuracy of 3D localization of the human head, as a key substructure containing rich identity information (such as facial features), is crucial. Some current solutions attempt to estimate the head's 3D coordinates in the world coordinate system using a single information source (e.g., using 2D keypoint backprojection or directly using the center of a 2D detection box). However, this single-path localization approach carries risks: when the SDK version is updated, causing changes in keypoint definitions, or when outliers occur due to sensor noise or poor local image quality, uncontrollable errors can occur in the localization results. This error at the geometric level can be amplified at the identity recognition level, causing irreversible ID drift and cascading errors, ultimately leading to the failure of the entire tracking system.

[0005] For example, invention application No. 202311434206.0 discloses a target recognition system based on a multi-band binocular camera. This application's solution uses a computing unit to fuse target detection results and binocular recognition results to obtain the target recognition result. However, its solution also suffers from the problems of insufficient robustness of head 3D localization and difficulty in maintaining long-term identity consistency.

[0006] Therefore, existing technologies face severe challenges in dealing with complex real-world scenarios in terms of the robustness of geometric positioning, the discriminative power of appearance matching, and the maintenance of long-term identity consistency. A method for 3D re-identification and consistency tracking of the head region based on binocular depth perception devices is needed to achieve accurate 3D positioning of the human head in complex scenarios, robust appearance feature matching, and long-term stable identity consistency maintenance. Summary of the Invention

[0007] To address the aforementioned problems, the present invention aims to provide a method for 3D re-identification and consistency tracking of the head region based on a binocular depth sensing device, thereby solving the problems of insufficient robustness of 3D head localization, limited appearance matching discrimination, and difficulty in maintaining long-term identity consistency caused by a single information source in the prior art.

[0008] This invention provides a method for three-dimensional re-identification and consistency tracking of the head region based on a binocular depth sensing device.

[0009] First aspect: A method for 3D re-identification and consistency tracking of the head region based on a binocular depth sensing device, including:

[0010] S1. Using a binocular depth camera, acquire target tracking data and use a four-level backtracking strategy to determine the three-dimensional coordinates of the target's head region;

[0011] S2. Extract the face ROI from the target head region, generate a combined vector as the main channel feature vector, and introduce the target height, clothing color histogram and body width-to-height ratio as the secondary channel feature vector. The feature vector is normalized and then entered into the similarity measurement space.

[0012] S3. Calculate target similarity based on three-dimensional position gating threshold. For targets that meet the similarity threshold, restore the original target ID, store it in memory entries, and maintain identity continuity.

[0013] In one embodiment of the present invention, the four-level backtracking strategy in S1 determines the three-dimensional coordinates of the target head region, including:

[0014] Prioritize reading the human body tracking interface If not, perform validity checks on key points such as the nasal alae, eyes, ears, and mouth, and calculate the average position; if still not found, calculate... Find the vertex center; if it is still not found, then calculate... If any level of the region center is established, the output will be the three-dimensional coordinates (X, Y, Z) of the target head region.

[0015] In one embodiment of the present invention, in step S2, a face ROI is extracted from the target head region, and a 470-dimensional combined vector is generated as the main channel feature vector, including: a 200-dimensional HOG vector, a 170-dimensional HSV histogram vector, and a 100-dimensional LBP vector.

[0016] In one embodiment of the present invention, the three-dimensional position gating threshold in S3 is that the three-dimensional distance between the target and the memory entry does not exceed 2.0 meters.

[0017] In one embodiment of the present invention, when merging existing memory entries, the three-dimensional distance between memory entries does not exceed 1.5 meters.

[0018] In one embodiment of the present invention, the appearance of the tracked target is accumulated to no less than 5 frames before it is stored in the memory entry. If more than 30 frames are lost consecutively, its memory entry is compressed into the memory bank and frozen.

[0019] In one embodiment of the present invention, the memory entries of the target are clustered. If the similarity of the targets is ≥0.85 and they are spatially proximate, then a merging and shrinking operation is performed, and each target retains at most three clusters of representative vectors.

[0020] In one embodiment of the present invention, a background scan of the memory entry database is performed at a cycle of 3 seconds to remove expired memory entries and merge redundant memory entries.

[0021] Second aspect: An electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, performs the steps of the method provided in the first aspect.

[0022] Third aspect: A non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method provided in the first aspect.

[0023] The beneficial effects of this invention are:

[0024] 1. This invention effectively solves the identity loss (ID) problem that easily occurs in traditional 2D tracking methods when the target is occluded, the lighting changes, or the target briefly leaves the scene, by fusing 3D spatial information provided by a binocular depth sensing device with multimodal appearance features. A four-level backtracking strategy is employed for 3D head localization, ensuring stable output of the target's head coordinates even with partial information loss or sensor noise interference. This strong geometric robustness provides reliable physical constraints for subsequent identity re-identification, thus significantly improving the continuity and stability of identity tracking in crowded and dynamically changing real-world scenarios. It enhances the robustness and continuity of target tracking in complex scenes.

[0025] 2. This invention employs a feature vector design combining main and secondary channels. The main channel integrates high-dimensional combined features of the face ROI, while the secondary channel introduces auxiliary features such as height, clothing color, and body aspect ratio. This multimodal feature combination greatly enriches the dimensions of the target's identity description. By introducing a three-dimensional positional gating threshold as a prerequisite for feature similarity calculation, a mechanism of physical prioritization and appearance verification is constructed. This a priori excludes invalid matches due to excessive spatial distance, performing fine feature comparison only between physically adjacent candidate targets. Therefore, in challenging scenarios such as groups of people with similar clothing, this significantly improves the accuracy of re-identification and effectively reduces the risk of false matches.

[0026] 3. This invention ensures the effectiveness of memory entries by setting rules such as storing entries only after a target has appeared for ≥5 frames and freezing entries after more than 30 consecutive lost frames. Simultaneously, a clustering and merging strategy is employed to retain a maximum of three representative feature clusters for each target, covering multiple pose variations while avoiding the decrease in discriminative power caused by the infinite expansion of the feature library. Furthermore, the system performs background scanning every 3 seconds, automatically cleaning up expired entries and merging redundant information. This effectively suppresses identity drift during long-term tracking, ensuring the long-term consistency of the output identity ID, and greatly improving its usability and reliability in downstream applications such as behavior analysis and security monitoring. Attached Figure Description

[0027] Figure 1 This is a schematic flowchart of the method of the present invention;

[0028] Figure 2 This is a schematic diagram of the four-level rollback strategy of the method of the present invention;

[0029] Figure 3 This is a schematic diagram of the structure of the electronic device of the present invention. Detailed Implementation

[0030] Embodiments of the present invention are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar symbols denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0031] This invention discloses a method for three-dimensional re-identification and consistency tracking of the head region based on a binocular depth sensing device. To make the purpose, technical solution and advantages of this disclosure clearer, the following detailed description is provided in conjunction with specific embodiments.

[0032] Example 1:

[0033] A method for 3D re-identification and consistency tracking of the head region based on a binocular depth sensing device, such as... Figure 1As shown, the steps include:

[0034] S1. Using a binocular depth camera, acquire tracking target data and use a four-level backtracking strategy to determine the three-dimensional coordinates of the target's head region.

[0035] Binocular depth camera devices simulate the stereoscopic vision principle of human eyes and calculate the three-dimensional coordinates of each point in the scene based on the parallax information between the left and right images through a built-in depth calculation chip or a matching image processing unit.

[0036] Based on data collected by a binocular depth camera, a four-level backtracking strategy is adopted to perform multi-source redundant head 3D localization of the target.

[0037] like Figure 2 As shown, the human body tracking interface is read first. If not, perform validity checks on key points such as the nostrils, eyes, ears, and mouth, and calculate the average position value; if still not found, calculate... The vertex center; if still unattainable, degenerate to the point where... The upper region is calculated. The center of the region forms a four-level backtracking chain: direct capture - key point - 3D frame center - 2D frame center. If any level is valid, the three-dimensional coordinates (X, Y, Z) of the target head region are output in meters.

[0038] S2. Extract the face ROI from the target head region and generate a combined vector as the main channel feature vector. Introduce the target height, clothing color histogram and body width-to-height ratio as the secondary channel feature vector. After normalization, the feature vectors are entered into the similarity measurement space.

[0039] The facial region of interest (ROI) is extracted from the target head region, and a 470-dimensional combined vector is generated using the facial ROI as the core. This 470-dimensional combined vector includes: HOG (200 dimensions, shape), HSV (170 dimensions, color), and LBP (100 dimensions, texture), and is used as the main channel feature vector. Features such as target height, clothing color histogram, and body aspect ratio are introduced as secondary channel feature vectors.

[0040] The feature vectors are normalized by standardizing the 470-dimensional combined vector of the main channel and the feature vectors of height, clothing color histogram and body width-to-height ratio of the sub-channel, so that the feature values ​​of each dimension are mapped to the [0,1] interval.

[0041] S3. Calculate target similarity based on three-dimensional position gating threshold. For targets that meet the similarity threshold, restore the original target ID, store it in memory entries, and maintain identity continuity.

[0042] After normalization, the feature vectors are entered into the similarity measurement space. Candidate target selection and conflict arbitration are then performed based on the feature vectors. Specifically:

[0043] First, candidate targets are screened by calculating the cosine similarity between the target and each memory entry in the memory bank on the main channel feature vector. Simultaneously, the Euclidean distance of the secondary channel feature vectors is used to construct a comprehensive similarity scoring model, initially selecting a set of memory entries with similarity scores higher than a set threshold. Next, conflict arbitration is performed. When multiple memory entries have comprehensive similarity scores close to the target, a dynamic weighting mechanism is introduced. The weight ratio of the main and secondary channel features is dynamically adjusted based on the discriminative power of the secondary channel feature vectors in the scene. For example, in scenes with drastic lighting changes, the weight of the clothing color histogram is increased; in scenes with significant occlusion, the weight of the body aspect ratio is enhanced. Through multiple rounds of iterative calculations, the optimal memory entry is finally determined.

[0044] Preferably, feature similarity calculation is only performed when the 3D distance between the observed target and the memory entry sample does not exceed 2.0 meters; if memory entries need to be merged, an upper limit of 1.5 meters is set to avoid cross-region fusion. Prior knowledge based on 3D distance physical gating can reduce false matches in open space and effectively control the redundant growth of the memory bank.

[0045] Specifically, when the observed target changes in a short period of time, it will not be too far away from the three-dimensional distance of the memory entry sample. Therefore, 2.0 meters is set as the physical gate threshold for entering the feature similarity calculation, ensuring that only the observed target within this distance range will trigger the feature matching process, thereby avoiding meaningless calculations for irrelevant targets at a distance.

[0046] In the memory entry merging stage, the upper limit of the three-dimensional distance is further tightened to 1.5 meters. This strictly limits the spatial range of entry merging in the memory bank, prevents similar target features in different areas from being erroneously merged, and improves the operating efficiency and stability of the entire tracking system in complex scenarios.

[0047] Furthermore, a tracked target must appear for at least 5 frames before being added to a memory entry. If more than 30 frames are lost consecutively, the memory entry is compressed into the memory bank and frozen. When the same target object is encountered again, if the similarity is ≥0.75 and the location gating is met, re-identification is triggered, the original ID is restored, and the reactivation count is incremented to maintain identity continuity.

[0048] Furthermore, the memory entries of the target are clustered. If the similarity of the targets is ≥0.85 and they are spatially adjacent, then a merging and shrinking operation is performed, and each target retains at most three clusters of representative vectors.

[0049] Clustering is performed on the features of the same target. If the similarity of the targets is ≥0.85 and they are spatially adjacent, merging and shrinking are performed. Each identity is limited to retaining a maximum of three clusters of representative vectors, which covers multiple poses and lighting conditions and prevents the dilution of discriminative power caused by feature expansion.

[0050] Furthermore, the memory entry database is scanned in the background every 3 seconds to perform periodic self-checks and memory preservation, remove expired memory entries, and merge redundant memory entries.

[0051] The method of this invention can significantly reduce the probability of ID reset and cross-time drift without changing the existing human tracking model of binocular depth camera (ZED) devices, by only using upper-level 3D physical gating, appearance multimodal coupling and memory topology rectification. In crowded, complex lighting and partially occluded scenarios, it can still output a continuous and consistent identity number and stable head 3D coordinates under the constraints of the physical world system, thereby improving the availability of downstream interactive, statistical and security applications.

[0052] Furthermore, it can be extended to multi-camera and cross-domain applications to continuously identify targets. By calibrating extrinsic parameters, the coordinates of multiple binocular depth sensing devices can be aligned to a unified world system. In cross-domain applications, adaptive color histogram normalization and local dynamic range compression are enabled to address color gamut drift and illumination differences.

[0053] The present invention also provides an electronic device, Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, such as... Figure 3 As shown, the electronic device may include a processor, a communications interface, memory, and a communication bus, wherein the processor, communications interface, and memory communicate with each other via the communication bus. The processor can invoke logical instructions from the memory, for example, to execute the following method:

[0054] S1. Using a binocular depth camera, acquire target tracking data and use a four-level backtracking strategy to determine the three-dimensional coordinates of the target's head region;

[0055] S2. Extract the face ROI from the target head region, generate a combined vector as the main channel feature vector, and introduce the target height, clothing color histogram and body width-to-height ratio as the secondary channel feature vector. The feature vector is normalized and then entered into the similarity measurement space.

[0056] S3. Calculate target similarity based on three-dimensional position gating threshold. For targets that meet the similarity threshold, restore the original target ID, store it in memory entries, and maintain identity continuity.

[0057] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0058] This invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is implemented to perform the methods provided in the above embodiments, including, for example:

[0059] S1. Using a binocular depth camera, acquire target tracking data and use a four-level backtracking strategy to determine the three-dimensional coordinates of the target's head region;

[0060] S2. Extract the face ROI from the target head region, generate a combined vector as the main channel feature vector, and introduce the target height, clothing color histogram and body width-to-height ratio as the secondary channel feature vector. The feature vector is normalized and then entered into the similarity measurement space.

[0061] S3. Calculate target similarity based on three-dimensional position gating threshold. For targets that meet the similarity threshold, restore the original target ID, store it in memory entries, and maintain identity continuity.

[0062] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0063] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0064] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for 3D re-identification and consistency tracking of the head region based on a binocular depth sensing device, characterized in that, include: S1. Using a binocular depth camera, acquire target tracking data and use a four-level backtracking strategy to determine the three-dimensional coordinates of the target's head region; S2. Extract the face ROI from the target head region, generate a combined vector as the main channel feature vector, and introduce the target height, clothing color histogram and body width-to-height ratio as the secondary channel feature vector. The feature vector is normalized and then entered into the similarity measurement space. S3. Calculate target similarity based on three-dimensional position gating threshold. For targets that meet the similarity threshold, restore the original target ID, store it in memory entries, and maintain identity continuity.

2. The method according to claim 1, characterized in that, The four-level backtracking strategy in S1 determines the three-dimensional coordinates of the target head region, including: Prioritize reading the human body tracking interface If not, perform validity checks on key points such as the nasal alae, eyes, ears, and mouth, and calculate the average position; if still not found, calculate... Find the vertex center; if it is still not found, then calculate... If any level of the region center is established, the output will be the three-dimensional coordinates (X, Y, Z) of the target head region.

3. The method according to claim 1, characterized in that, In step S2, the face ROI is extracted from the target head region, and a 470-dimensional combined vector is generated as the main channel feature vector, including: HOG 200-dimensional vector, HSV histogram 170-dimensional vector and LBP 100-dimensional vector.

4. The method according to claim 1, characterized in that, The three-dimensional position gating threshold in S3 is that the three-dimensional distance between the target and the memory entry does not exceed 2.0 meters.

5. The method according to claim 4, characterized in that, When merging existing memory entries, the three-dimensional distance between memory entries shall not exceed 1.5 meters.

6. The method according to claim 1, characterized in that, A tracked target is stored in a memory entry only if it appears for at least 5 frames. If more than 30 frames are lost consecutively, its memory entry is compressed into the memory bank and frozen.

7. The method according to claim 1, characterized in that, Cluster the memory entries of the target. If the similarity of the targets is ≥0.85 and they are spatially adjacent, then perform a merge shrinkage. Each target retains at most three cluster representative vectors.

8. The method according to claim 1, characterized in that, Perform a background scan of the memory entry database every 3 seconds to remove expired memory entries and merge redundant memory entries.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method as described in any one of claims 1 to 8.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 8.

Images