Method and system for tracking participant trajectory based on dynamic face recognition

Abstract

This invention discloses a method and system for participant trajectory tracking based on dynamic face recognition, belonging to the field of identity recognition technology. Based on assigned identity IDs, it connects target location points in consecutive frames to generate trajectories, and uses behavioral habit features to correlate and stitch cross-camera trajectories, generating a complete spatiotemporal trajectory stream of participants. This trajectory stream is mapped onto a meeting scene floor plan, generating reports on dwell time statistics, frequency of visits to interest areas, and social clustering relationship analysis. This invention effectively improves the accuracy and continuity of participant trajectory tracking in meeting scenarios, overcoming the limitations of single recognition technologies in complex scenarios. Through multimodal feature extraction and an edge-cloud collaborative decision-making mechanism guided by an uncertainty index, it balances the real-time performance and recognition accuracy of trajectory tracking, reducing edge-side resource consumption and cloud-side transmission pressure.

Description

Technical Field

[0001] This invention relates to the field of identity recognition technology, and in particular to a method and system for tracking the trajectory of attendees based on dynamic facial recognition. Background Technology

[0002] In current conference scenarios, participant tracking largely relies on single-face recognition technology or manual statistical methods, which have numerous limitations. Single-face recognition is susceptible to factors such as changes in lighting, occlusion by people, and long-distance shooting in conference settings, leading to decreased recognition accuracy and making it difficult to achieve continuous and stable trajectory tracking. Manual statistical methods are inefficient and cannot meet the dynamic tracking needs of large conferences with multiple participants and multiple areas, and are prone to statistical errors. Furthermore, existing technologies lack effective edge-cloud collaboration mechanisms; either insufficient edge processing capabilities result in excessive latency, or over-reliance on the cloud leads to resource waste, making it difficult to balance real-time performance and recognition accuracy.

[0003] In addition, existing trajectory tracking technologies lack the ability to dynamically maintain identity information, cannot effectively cope with situations such as changes in the scene of the participant or long periods of no observation, are prone to misidentification, and lack a sound privacy protection mechanism, thus failing to guarantee the security of the participant's personal information. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a method and system for tracking the trajectory of attendees based on dynamic face recognition. The technical solution adopted is as follows: The participant trajectory tracking method based on dynamic face recognition includes the following steps: Step 1: Acquire multiple synchronous video streams deployed in the conference scenario, perform spatiotemporal alignment preprocessing on the video frames, detect and extract pedestrian targets, and obtain detection results including human bounding boxes, head bounding boxes and skeletal key point sequences; Step 2: Perform multimodal feature extraction on the pedestrian target, and extract facial feature vectors respectively. Gait periodic eigenvectors Skeletal pose feature vector and clothing texture feature vector And calculate the uncertainty index U of the currently identified variable; Step 3: Based on the uncertainty index U, adopt an edge-cloud collaborative decision-making mechanism: If U is less than the first threshold, recognition and tracking are completed on the edge; if U is greater than or equal to the first threshold and less than or equal to the second threshold, the feature vector is uploaded to the cloud for enhanced matching; if U is greater than the second threshold, the original image is uploaded to the cloud for fine recognition using a large model. Step 4: Construct and dynamically maintain the identity imprint graph, which stores a multi-dimensional imprint vector for each participant. Step 5: For newly observed pedestrian targets, calculate the multimodal comprehensive similarity with all historical imprints in the identity imprint map, select the imprint ID with the highest similarity that exceeds the preset threshold as the current target identity, and create a new identity imprint if none of them exceed the threshold. Step 6: Based on the assigned identity ID, connect the target location points in consecutive frames to generate a trajectory, and use behavioral habit features to associate and stitch the cross-camera trajectories to generate a complete spatiotemporal trajectory stream of the participants. Step 7: Map the trajectory flow to the meeting scene floor plan and generate a report on dwell time statistics, frequency of visits to interest areas, and social gathering relationship analysis.

[0005] Optionally, the uncertainty index U in step 2 can be calculated as follows: ; in, This represents the maximum output probability of the face recognition classifier. The overall image quality score includes sharpness, occlusion, illumination uniformity, and angular deflection. This is the balance coefficient, and its value range is... The uncertainty index takes into account both the confidence level of the identification results and the image quality, providing a quantitative basis for edge-cloud collaborative decision-making.

[0006] Optionally, the gait cycle feature extraction in step 2 specifically includes: based on the skeletal keypoint sequence, extracting the hip joint angle change curve, stride length, and gait frequency features within a complete gait cycle, and encoding them into a 128-dimensional gait feature vector using LSTM. Gait features are unique to an individual and are not affected by changes in clothing, making them a primary basis for identification in long-distance or low-resolution scenarios.

[0007] Optionally, in step 4, the multidimensional imprint vector ;in It records the intensity weights, which are dynamically updated through the identity imprint forgetting function; The identity imprint forgetting function is as follows: ; in The initial confidence level is set to 1.0 when face recognition is successful, and 0.3-0.6 when recognition is based solely on soft biometric features. This is the time difference between the current time and the last observation time. The time decay factor has a range of values. ; The number of scene transitions experienced by the target; The scene attenuation factor has a range of values. A dual decay mechanism for identity imprints is implemented through a forgetting function to avoid erroneous identity inheritance due to prolonged lack of observation or scene changes.

[0008] Optionally, the multimodal synthesis similarity in step 5 The calculation formula is: ; in , , and These are the weight coefficients for the similarity of each feature vector. It is the similarity of facial feature vectors. It is the similarity of gait period feature vectors. It is the similarity of skeletal pose feature vectors. It is the similarity of clothing texture feature vectors.

[0009] Optionally, the edge-cloud collaborative decision-making mechanism in step 3 also includes: deploying a lightweight TensorRT inference engine on the edge for real-time object detection and feature extraction; deploying a large-model fine recognition network on the cloud for handling high-uncertainty samples; and implementing three-pipeline parallel processing through a dual-buffered queue: the first queue performs CUDA-accelerated image preprocessing, the second queue performs edge-side TensorRT inference, and the third queue completes feature uploading and cloud-side recognition result reception.

[0010] Optionally, the behavioral characteristics in step 6 include: the distribution of dwelling preference areas, the statistical characteristics of movement speed, and the social clustering relationship map; the social clustering relationship map is constructed by analyzing the spatiotemporal co-occurrence frequency of multiple targets appearing at the same time and moving in the same direction. When the target's face is completely obscured, the identity is inferred through the high-frequency co-occurrence IDs in its social relationship map, thus realizing the social relationship-assisted identity continuation.

[0011] Optionally, step 8, privacy protection and data lifecycle management, includes: setting a 24-hour automatic expiration mechanism for clothing texture features; retaining facial features only during the meeting and automatically encrypting and archiving them after the meeting; deleting all original video data within 7 days after generating the trajectory stream, and retaining only the anonymized statistical information.

[0012] Optionally, the trajectory association stitching in step 6 adopts a hierarchical association strategy: the first level is the association of continuous frames within the same camera, based on IoU and optical flow; the second level is the short-term cross-camera association between adjacent cameras, based on feature similarity and spatial topology constraints; the third level is the intelligent recovery of long-term broken trajectories, based on the memory strength in the identity imprint map, and automatically inherits the historical ID when the similarity between the reappearing target and the historical imprint exceeds the recovery threshold.

[0013] The participant trajectory tracking system based on dynamic face recognition is used to implement the participant trajectory tracking method based on dynamic face recognition. The system is characterized by including a video acquisition terminal, edge computing nodes, a cloud-based central server, and a visualization analysis platform. The video acquisition terminal is communicatively connected to the edge computing node and is used to acquire multiple synchronous video streams in a conference scenario, and transmit the acquired raw video streams to the edge computing node in real time. The edge computing node is connected to the cloud-side central server to perform multimodal feature extraction and uncertainty index calculation, and transmits the feature vectors to be processed on the cloud side, the original image and the preliminary processing results to the cloud-side central server. The cloud-side central server is communicatively connected to the edge computing nodes and the visualization analysis platform. It includes an identity imprint graph database, a forgetting function calculation engine, and a large model fine recognition module. It receives data transmitted from the edge computing nodes, performs identity imprint graph construction and maintenance, identity matching and new identity creation, trajectory generation and association stitching, privacy protection and data lifecycle management, completes global trajectory stitching, data storage and privacy control, and transmits the processed trajectory data and statistical analysis results to the visualization analysis platform. The visualization analysis platform communicates with the cloud-based central server, receives anonymized data transmitted from the cloud-based central server, and provides users with real-time trajectory heatmap display, single-target trajectory playback, social cluster relationship graph visualization, and privacy-anonymized data export functions, realizing the visualization presentation and data interaction of participant trajectories and related analysis results.

[0014] In summary, the present invention has at least one of the following beneficial technical effects: This invention provides a method and system for participant trajectory tracking based on dynamic face recognition, effectively improving the accuracy and continuity of participant trajectory tracking in conference scenarios and overcoming the limitations of single recognition technologies in complex situations. Through multimodal feature extraction and an edge-cloud collaborative decision-making mechanism guided by the uncertainty index, it balances the real-time performance of trajectory tracking with recognition accuracy, while reducing edge-side resource consumption and cloud-side transmission pressure.

[0015] The dynamically maintained identity imprint map can adapt to dynamic changes such as changes in participant scenarios and long periods of inactivity, reducing misidentification and ensuring the stable continuation of identity information. The hierarchical trajectory association and stitching strategy can effectively recover cross-camera and fragmented trajectories, generating a complete spatiotemporal trajectory stream of participants. A robust privacy protection and data lifecycle management mechanism effectively protects the security of participants' personal information and complies with data security standards.

[0016] Meanwhile, the visualization analysis function can intuitively present the participants' trajectories and related statistical information, providing data support for meeting management and process optimization, and improving the level of intelligence in meeting management. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the participant trajectory tracking method based on dynamic face recognition of the present invention. Figure 2 This is a schematic diagram of the participant trajectory tracking system based on dynamic face recognition of the present invention; Detailed Implementation

[0018] The present invention will be further described in detail below with reference to the accompanying drawings.

[0019] This invention discloses a method and system for tracking the trajectory of attendees based on dynamic face recognition.

[0020] Reference Figure 1 and Figure 2 Example 1, a participant trajectory tracking method based on dynamic face recognition, includes the following steps: Step 1: Acquire multiple synchronous video streams deployed in the conference scenario, perform spatiotemporal alignment preprocessing on the video frames, detect and extract pedestrian targets, and obtain detection results including human bounding boxes, head bounding boxes and skeletal key point sequences; Step 2: Perform multimodal feature extraction on the pedestrian target, and extract facial feature vectors respectively. Gait periodic eigenvectors Skeletal pose feature vector and clothing texture feature vector And calculate the uncertainty index U of the currently identified variable; Step 3: Based on the uncertainty index U, adopt an edge-cloud collaborative decision-making mechanism: If U is less than the first threshold, recognition and tracking are completed on the edge; if U is greater than or equal to the first threshold and less than or equal to the second threshold, the feature vector is uploaded to the cloud for enhanced matching; if U is greater than the second threshold, the original image is uploaded to the cloud for fine recognition using a large model. Step 4: Construct and dynamically maintain the identity imprint graph, which stores a multi-dimensional imprint vector for each participant. Step 5: For newly observed pedestrian targets, calculate the multimodal comprehensive similarity with all historical imprints in the identity imprint map, select the imprint ID with the highest similarity that exceeds the preset threshold as the current target identity, and create a new identity imprint if none of them exceed the threshold. Step 6: Based on the assigned identity ID, connect the target location points in consecutive frames to generate a trajectory, and use behavioral habit features to associate and stitch the cross-camera trajectories to generate a complete spatiotemporal trajectory stream of the participants. Step 7: Map the trajectory flow to the meeting scene floor plan and generate a report on dwell time statistics, frequency of visits to interest areas, and social gathering relationship analysis.

[0021] Example 2, the uncertainty index U in step 2 is calculated as follows: ; in, This represents the maximum output probability of the face recognition classifier. The overall image quality score includes sharpness, occlusion, illumination uniformity, and angular deflection. This is the balance coefficient, and its value range is... The uncertainty index takes into account both the confidence level of the identification results and the image quality, providing a quantitative basis for edge-cloud collaborative decision-making.

[0022] Example 3, step 2 of gait cycle feature extraction specifically includes: based on the skeletal keypoint sequence, extracting the hip joint angle change curve, stride length, and gait frequency features within a complete gait cycle, and encoding them into a 128-dimensional gait feature vector using LSTM. Gait features are unique to an individual and are not affected by changes in clothing, making them a primary basis for identification in long-distance or low-resolution scenarios.

[0023] By adopting the above technical solution, step 1 collects full-domain information of the conference scene through multiple synchronous video streams. Spatiotemporal alignment preprocessing can eliminate the time difference and spatial offset between different cameras, ensuring the consistency of video frames from multiple perspectives. At the same time, key information related to pedestrians is extracted through target detection, providing basic data support for subsequent feature extraction and identity recognition, and ensuring the accuracy and effectiveness of subsequent processing.

[0024] Step 2 overcomes the limitations of single-face recognition by integrating four differentiated features—face, gait, skeletal pose, and clothing texture—through multimodal feature extraction. The complementarity of these features enhances the robustness of identity recognition. Facial features provide the core basis for identity differentiation, gait and skeletal pose features are adapted for long-distance and low-resolution scenarios, and clothing texture features further strengthen the recognition effect. The calculation of the uncertainty index, combined with recognition confidence and image quality, quantifies the reliability of the current recognition result, providing an objective judgment standard for subsequent edge-cloud collaborative decision-making and avoiding the loss of accuracy or efficiency due to blind decision-making.

[0025] Step 3: Based on the uncertainty index, resources are allocated reasonably. Depending on the reliability of the identification, the edge or cloud processing mode is flexibly selected. Edge processing of low uncertainty samples can ensure real-time performance and reduce transmission costs, while cloud-side enhanced matching or fine identification of high uncertainty samples can make up for the insufficient edge processing capabilities, achieving a balance between real-time performance and identification accuracy.

[0026] Step 4 involves constructing an identity imprint graph to dynamically retain and update participant identity information. By integrating multimodal features and related auxiliary information of participants through multidimensional imprint vectors, a unified comparison benchmark is provided for subsequent identity matching. At the same time, the dynamic maintenance mechanism can adapt to changes in the participants' status during the meeting, ensuring the timeliness and accuracy of identity information.

[0027] Step 5 achieves accurate matching between new observation targets and historical identities through multimodal comprehensive similarity calculation. It comprehensively considers the similarity of four features to avoid misjudgment of identities caused by single feature bias. At the same time, it determines whether to create a new identity through a preset threshold to ensure the integrity and uniqueness of the identity imprint map and achieve full coverage tracking of the identities of all participants.

[0028] Step 6 uses the identity ID as a link to connect the target positions in consecutive frames to form a single segment trajectory. At the same time, it uses behavioral habit features to make up for the trajectory breakage problem in cross-camera tracking. Through the correlation of behavioral features, it achieves accurate stitching of the trajectory of the same participant under different cameras, and finally forms a complete spatiotemporal trajectory stream to ensure the continuity of trajectory tracking.

[0029] Step 7 transforms the abstract trajectory flow into concrete and usable statistical information. By mapping the meeting scene floor plan, it explores the participants' dwell patterns, interests, preferences, and social connections, and transforms the trajectory data into valuable analysis reports to provide data support for meeting management.

[0030] The uncertainty index is calculated by using the maximum output probability of the face recognition classifier to reflect the confidence level of the recognition result, and the overall image quality score to reflect the reliability of the input data. A balance coefficient is introduced to adjust the weights of both, ensuring that the uncertainty index can comprehensively and objectively reflect the current recognition status, providing accurate quantitative basis for edge-cloud collaborative decision-making. The principle of gait periodic feature extraction is to capture the periodic changes in human gait using skeletal keypoint sequences, extract gait feature parameters with unique identity, and transform the temporal gait features into a fixed-dimensional feature vector through LSTM encoding, making it suitable for identity recognition comparison requirements. Simultaneously, leveraging the characteristic that gait features are unaffected by clothing changes, it compensates for the shortcomings of facial features in complex scenarios.

[0031] Example 4, in step 4, the multidimensional imprint vector ;in It records the intensity weights, which are dynamically updated through the identity imprint forgetting function; The identity imprint forgetting function is as follows: ; in The initial confidence level is set to 1.0 when face recognition is successful, and 0.3-0.6 when recognition is based solely on soft biometric features. This is the time difference between the current time and the last observation time. The time decay factor has a range of values. ; The number of scene transitions experienced by the target; The scene attenuation factor has a range of values. A dual decay mechanism for identity imprints is implemented through a forgetting function to avoid erroneous identity inheritance due to prolonged lack of observation or scene changes.

[0032] By adopting the above technical solution, multimodal features and related auxiliary information of participants are integrated through multidimensional imprint vectors to form basic data that can be used for identity comparison. The memory strength weights are dynamically updated through an identity imprint forgetting function. This forgetting function combines time difference and scene switching frequency to achieve dual decay. Based on different initial confidence levels, the weights are adjusted over time and according to scene changes to avoid misjudgments due to prolonged periods without observation or scene switching, ensuring the accuracy and timeliness of the identity imprint map.

[0033] Example 5, Multimodal Synthesis Similarity in Step 5 The calculation formula is: ; in , , and These are the weight coefficients for the similarity of each feature vector. It is the similarity of facial feature vectors. It is the similarity of gait period feature vectors. It is the similarity of skeletal pose feature vectors. It is the similarity of clothing texture feature vectors.

[0034] By adopting the above technical solution, multimodal comprehensive similarity is calculated by weighted summation. Corresponding weight coefficients are set for the similarity of the four feature vectors respectively. The weights are assigned in combination with the different roles of each feature in identity recognition. The comparison results of the four features of face, gait, skeletal pose and clothing texture are integrated to improve the accuracy of identity matching and provide a reliable basis for the identity determination of new observation targets.

[0035] Example 6, the edge-cloud collaborative decision-making mechanism in step 3 also includes: deploying a lightweight TensorRT inference engine on the edge side for real-time target detection and feature extraction; deploying a large-model fine recognition network on the cloud side for handling high-uncertainty samples; and implementing three-pipeline parallel processing through a dual-buffer queue: the first queue performs CUDA-accelerated image preprocessing, the second queue performs edge-side TensorRT inference, and the third queue completes feature uploading and cloud-side recognition result reception.

[0036] By adopting the above technical solution, the simplified principle of the edge-cloud collaborative decision-making mechanism in step 3 is to adapt to different processing needs through edge-cloud division of labor. The edge deploys a lightweight inference engine to ensure the real-time performance of target detection and feature extraction, while the cloud deploys a large-model fine recognition network to cope with high-uncertainty samples and improve recognition accuracy. At the same time, a dual-buffered queue is used to realize parallel processing of three pipelines, which are responsible for image preprocessing, edge inference, feature uploading, and result reception, respectively, thereby improving the overall processing efficiency and further ensuring the real-time performance and reliability of edge-cloud collaboration.

[0037] Example 7, the behavioral habit features in step 6 include: the distribution of dwelling preference areas, the statistical features of movement speed, and the social clustering relationship map; the social clustering relationship map is constructed by analyzing the spatiotemporal co-occurrence frequency of multiple targets appearing at the same time and moving in the same direction. When the target's face is completely obscured, the identity is inferred with the help of high-frequency co-occurrence IDs in its social relationship map, so as to realize the social relationship-assisted identity continuation.

[0038] By adopting the above technical solution, the simplified principle of step 6, behavioral habit features, is to extract three types of behavioral features: participants' dwell preferences, movement speed, and social clustering relationships, which serve as auxiliary basis for cross-camera trajectory association and stitching. The social clustering relationship map is constructed by analyzing the spatiotemporal co-occurrence frequency of multiple targets. When a target's face is completely occluded and cannot be directly identified, the identity ID of the frequently co-occurring target is used to assist in inferring the current target's identity, thus achieving the continuity of identity information and further improving the accuracy and continuity of trajectory association and stitching.

[0039] Example 8 also includes step 8: privacy protection and data lifecycle management, specifically including: setting a 24-hour automatic expiration mechanism for clothing texture features; retaining facial features only during the meeting and automatically encrypting and archiving them after the meeting ends; deleting all original video data within 7 days after generating the trajectory stream, and retaining only the anonymized statistical information.

[0040] By adopting the above technical solutions, the simplified principle of step 8, privacy protection and data lifecycle management, is to formulate differentiated management strategies for data related to different types of participants. This is achieved by setting an automatic expiration mechanism for clothing texture features, limiting the retention time of facial features, and periodically deleting original video data while retaining only anonymized statistical information. This allows for full lifecycle control of data, preventing the leakage of participants' personal information, ensuring information security, and simultaneously balancing data usage needs with privacy protection requirements.

[0041] In Example 9, the trajectory association stitching in step 6 adopts a hierarchical association strategy: the first level is the association of continuous frames within the same camera, based on IoU and optical flow; the second level is the short-term cross-camera association between adjacent cameras, based on feature similarity and spatial topology constraints; the third level is the intelligent recovery of long-term broken trajectories, based on the memory strength in the identity imprint map, and automatically inherits the historical ID when the similarity between the reappearing target and the historical imprint exceeds the recovery threshold.

[0042] By adopting the above technical solution, the simplified principle of trajectory association and stitching in step 6 is to use a hierarchical association strategy to achieve accurate association and complete stitching of trajectories in three levels. The first level targets continuous frame trajectories within the same camera, relying on relevant algorithms to ensure the stability of continuous tracking; the second level targets short-term cross-camera trajectories between adjacent cameras, combining feature similarity and spatial constraints to achieve cross-camera connection; the third level targets long-term broken trajectories, leveraging the memory strength of the identity imprint map, inheriting historical identity when the target reappears and the similarity meets the standard, thus restoring the broken trajectory and ultimately ensuring the integrity of the participants' trajectories.

[0043] Example 10: A participant trajectory tracking system based on dynamic face recognition, used to implement a participant trajectory tracking method based on dynamic face recognition, characterized in that the system includes a video acquisition terminal, an edge computing node, a cloud-based central server, and a visualization analysis platform; The video acquisition terminal is communicatively connected to the edge computing node and is used to acquire multiple synchronous video streams in a conference scenario, and transmit the acquired raw video streams to the edge computing node in real time. The edge computing node is connected to the cloud-side central server to perform multimodal feature extraction and uncertainty index calculation, and transmits the feature vectors to be processed on the cloud side, the original image and the preliminary processing results to the cloud-side central server. The cloud-side central server is communicatively connected to the edge computing nodes and the visualization analysis platform. It includes an identity imprint graph database, a forgetting function calculation engine, and a large model fine recognition module. It receives data transmitted from the edge computing nodes, performs identity imprint graph construction and maintenance, identity matching and new identity creation, trajectory generation and association stitching, privacy protection and data lifecycle management, completes global trajectory stitching, data storage and privacy control, and transmits the processed trajectory data and statistical analysis results to the visualization analysis platform. The visualization analysis platform communicates with the cloud-based central server, receives anonymized data transmitted from the cloud-based central server, and provides users with real-time trajectory heatmap display, single-target trajectory playback, social cluster relationship graph visualization, and privacy-anonymized data export functions, realizing the visualization presentation and data interaction of participant trajectories and related analysis results.

[0044] The following specific embodiments illustrate the implementation principle of the present invention: This solution is applied to large-scale industry seminars, where the venue includes a main venue, three sub-venues, a rest area, and an exhibition area. It is necessary to track the entire process of more than 200 attendees and perform related data statistical analysis. The method and system of this technical solution are used to achieve this, and the specific implementation process is as follows.

[0045] First, the system deployment was completed, including the setup of video capture terminals, edge computing nodes, a cloud-based central server, and a visualization analysis platform. Each component was deployed and debugged according to its corresponding connection relationships. The video capture terminal uses 16 time-synchronized IPC network cameras, deployed at entrances, exits, passageways, and key areas of each meeting room to achieve full coverage of the meeting scenario. Each camera establishes a wired communication connection with the edge computing nodes. The edge computing nodes are deployed in the nearest server room to each meeting room, incorporating the TensorRT lightweight inference engine and a multimodal feature extraction model, and establishing a high-speed network communication connection with the cloud-based central server. The cloud-based central server is deployed in the back-end server room, incorporating an identity imprinting graph database, a forgetting function calculation engine, and a large-model fine-grained recognition module, and establishing bidirectional communication connections with both the edge computing nodes and the visualization analysis platform. The visualization analysis platform is deployed in the meeting management backend, supports web access, and maintains stable communication with the cloud-based central server.

[0046] Before the meeting begins, system initialization is completed, the first and second thresholds for edge-cloud collaborative decision-making are set, the preset thresholds for multimodal comprehensive similarity and trajectory recovery are set, the initial confidence level, time decay factor, and scene decay factor of the identity imprint forgetting function are configured, and the various weight coefficients of multimodal comprehensive similarity are configured. At the same time, privacy protection and data lifecycle management related settings are enabled.

[0047] During the meeting, the system initiated a full-process tracking workflow, executing the technical steps corresponding to each embodiment. In step 1, 16 IPC network cameras simultaneously acquired multiple video streams from the meeting scene. After being transmitted to the edge computing node, the edge computing node performed spatiotemporal alignment preprocessing on the video frames to eliminate time differences and spatial offsets between different cameras. Then, it extracted pedestrian targets from the video frames using a target detection algorithm, obtaining the human body bounding box, head bounding box, and skeletal keypoint sequence for each participant, providing basic data for subsequent processing.

[0048] In step 2, the edge computing nodes perform multimodal feature extraction on each participant target, extracting facial feature vectors, gait cycle feature vectors, skeletal pose feature vectors, and clothing texture feature vectors. Gait cycle feature extraction relies on skeletal keypoint sequences to capture hip joint angle changes, stride length, and stride frequency throughout the participant's complete gait cycle. This is then converted into a fixed-dimensional gait feature vector using LSTM encoding, adapting to recognition needs in long-distance and low-resolution scenarios. Simultaneously, the uncertainty index for each participant's current recognition is calculated. Combining facial recognition confidence with overall image quality, the reliability of the recognition results is quantified, providing a basis for edge-cloud collaborative decision-making.

[0049] In step 3, based on the calculated uncertainty index, an edge-cloud collaborative decision-making mechanism is used to handle the identification and tracking tasks of each participant. For participants with an uncertainty index less than the first threshold, edge computing nodes directly complete edge-side identification and real-time tracking to ensure real-time processing. For participants with an uncertainty index between the first and second thresholds, edge computing nodes upload the extracted multimodal feature vectors to the cloud-side central server for enhanced matching, improving identification accuracy. For participants with an uncertainty index greater than the second threshold, edge computing nodes upload the original image to the cloud-side central server for processing by the cloud-side large-model fine recognition network, ensuring identification accuracy. Simultaneously, a dual-buffered queue enables parallel processing across three pipelines, performing CUDA-accelerated image preprocessing, edge-side TensorRT inference, feature uploading, and cloud-side recognition result reception, improving overall processing efficiency and ensuring smooth simultaneous tracking of multiple participants.

[0050] In step 4, the cloud-based central server constructs and dynamically maintains an identity imprint map, assigning a unique identity ID to each successfully identified participant and storing the corresponding multi-dimensional imprint vector. This vector encompasses four feature vectors: face, gait, skeletal pose, and clothing texture, as well as the last observation time and memory strength weight. The memory strength weight is dynamically updated through an identity imprint forgetting function, undergoing a dual decay based on the difference between the participant's last observation time and the current time, and the number of scene transitions experienced. It is also differentiated based on the initial confidence level to avoid misidentification due to prolonged periods without observation or scene transitions, ensuring the accuracy and timeliness of the identity imprint map.

[0051] In step 5, for each newly observed pedestrian target, the cloud-based central server calculates the multimodal comprehensive similarity between its multimodal features and all historical imprints in the identity imprint map. The similarity results of the four features are integrated using a weighted summation method, and corresponding weights are assigned according to the role of each feature in identity recognition. The imprint ID with the highest similarity exceeding a preset threshold is selected as the current target's identity. If all similarities do not exceed the preset threshold, the target is determined to be a new participant, a new identity imprint is created in the identity imprint map, and a new identity ID is assigned, achieving full coverage tracking of all participants.

[0052] In step 6, based on the assigned identity ID, the system concatenates the target location points of participants in consecutive frames to generate continuous trajectories within the range of a single camera. Simultaneously, it utilizes the behavioral characteristics of participants to assist in the association and stitching of cross-camera trajectories, generating a complete spatiotemporal trajectory stream of the participants. These behavioral characteristics include the distribution of participants' preferred areas of stay, statistical characteristics of their movement speed, and a social clustering relationship graph. The social clustering relationship graph is constructed by analyzing the spatiotemporal co-occurrence frequency of multiple participants appearing simultaneously and moving in the same direction. When a participant's face is completely obscured and cannot be directly identified, the system uses high-frequency co-occurring identity IDs from their social relationship graph to assist in inferring their identity, thus achieving identity continuation. The trajectory association and stitching adopts a hierarchical association strategy. The first level relies on relevant algorithms to complete the association of continuous frame trajectories within the same camera, ensuring the stability of continuous tracking. The second level combines feature similarity and spatial topological constraints to complete the connection of short-term cross-camera trajectories between adjacent cameras. The third level relies on the memory strength in the identity imprint map. When a participant who has not been observed for a long time reappears and the similarity between their features and historical imprints exceeds the recovery threshold, the historical identity ID is automatically inherited to achieve intelligent recovery of broken trajectories, ensuring the integrity of the trajectory of each participant.

[0053] In step 7, the system maps the generated spatiotemporal trajectory stream of participants onto the conference scene floor plan. The cloud-based central server analyzes and processes the trajectory data to generate relevant statistical analysis reports, covering the dwell time of each participant in each area, the frequency of visits to each interest area (such as each booth in the exhibition area and sub-venues), and the social gathering relationships among participants. This provides data support for conference management and makes it easier for staff to understand the behavior patterns of participants and the effectiveness of the conference.

[0054] In step 8, the system implements privacy protection and data lifecycle management strategies throughout the process. A 24-hour automatic expiration mechanism is set for clothing texture features, which will no longer be used for identity recognition after expiration. Facial features are only retained during the meeting and are automatically encrypted and archived after the meeting to prevent information leakage. All original video data is automatically deleted within 7 days after the trajectory stream is generated, and only the anonymized statistical information is retained, which not only protects the personal information security of participants, but also takes into account the subsequent use needs of meeting data.

[0055] The visualization and analysis platform receives anonymized data from the cloud-based central server in real time, providing meeting administrators with an intuitive operation and viewing interface. It can display real-time heatmaps of participants' trajectories, clearly showing the gathering of participants in each area; it supports single-target trajectory playback, allowing users to query the entire movement trajectory of any participant; it can visualize social cluster relationship graphs, showing the connections between participants; and it also supports the export of privacy-anonymized data, facilitating subsequent meeting summaries and data analysis.

[0056] After the meeting, the system automatically stopped video capture and trajectory tracking, encrypted and archived facial features, deleted all original video data, and retained only the anonymized statistical analysis report and identity imprint map (with privacy-related features removed), thus completing the entire process of tracing the trajectory of the meeting participants.

[0057] The above are all preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Therefore, all equivalent changes made in accordance with the structure, shape and principle of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A participant trajectory tracking method based on dynamic face recognition, characterized in that: Includes the following steps: Step 1: Acquire multiple synchronous video streams deployed in the conference scenario, perform spatiotemporal alignment preprocessing on the video frames, detect and extract pedestrian targets, and obtain detection results including human bounding boxes, head bounding boxes and skeletal key point sequences; Step 2: Perform multimodal feature extraction on the pedestrian target, and extract facial feature vectors respectively. Gait periodic eigenvectors Skeletal pose feature vector and clothing texture feature vector And calculate the uncertainty index U of the currently identified variable; Step 3: Based on the uncertainty index U, adopt an edge-cloud collaborative decision-making mechanism: If U is less than the first threshold, recognition and tracking are completed on the edge; if U is greater than or equal to the first threshold and less than or equal to the second threshold, the feature vector is uploaded to the cloud for enhanced matching; if U is greater than the second threshold, the original image is uploaded to the cloud for fine recognition using a large model. Step 4: Construct and dynamically maintain the identity imprint graph, which stores a multi-dimensional imprint vector for each participant. Step 5: For newly observed pedestrian targets, calculate the multimodal comprehensive similarity with all historical imprints in the identity imprint map, select the imprint ID with the highest similarity that exceeds the preset threshold as the current target identity, and create a new identity imprint if none of them exceed the threshold. Step 6: Based on the assigned identity ID, connect the target location points in consecutive frames to generate a trajectory, and use behavioral habit features to associate and stitch the cross-camera trajectories to generate a complete spatiotemporal trajectory stream of the participants. Step 7: Map the trajectory flow to the meeting scene floor plan and generate a report on dwell time statistics, frequency of visits to interest areas, and social gathering relationship analysis.

2. The participant trajectory tracking method based on dynamic face recognition according to claim 1, characterized in that: The uncertainty index U in step 2 is calculated as follows: ;in, This represents the maximum output probability of the face recognition classifier. The overall image quality score includes sharpness, occlusion, illumination uniformity, and angular deflection. This is the balance coefficient, and its value range is... The uncertainty index takes into account both the confidence level of the recognition results and the image quality, providing a quantitative basis for edge-cloud collaborative decision-making.

3. The participant trajectory tracking method based on dynamic face recognition according to claim 2, characterized in that: Step 2, gait cycle feature extraction, specifically includes: based on the skeletal keypoint sequence, extracting the hip joint angle change curve, stride length, and gait frequency features within a complete gait cycle, and encoding them into a 128-dimensional gait feature vector using LSTM. Gait features are unique to an individual and are not affected by changes in clothing, making them the primary basis for identification in long-distance or low-resolution scenarios.

4. The participant trajectory tracking method based on dynamic face recognition according to claim 3, characterized in that: In step 4, the multidimensional imprint vector ;in It records the intensity weights, which are dynamically updated through the identity imprint forgetting function; The identity imprint forgetting function is as follows: ； in The initial confidence level is set to 1.0 when face recognition is successful, and 0.3-0.6 when recognition is based solely on soft biometric features. This is the time difference between the current time and the last observation time. The time decay factor has a range of values. ; The number of scene transitions experienced by the target; The scene attenuation factor has a range of values. A dual decay mechanism for identity imprints is implemented through a forgetting function to avoid erroneous identity inheritance due to prolonged lack of observation or scene changes.

5. The participant trajectory tracking method based on dynamic face recognition according to claim 4, characterized in that: Multimodal synthesis similarity in step 5 The calculation formula is: ； in , , and These are the weight coefficients for the similarity of each feature vector. It is the similarity of facial feature vectors. It is the similarity of gait period feature vectors. It is the similarity of skeletal pose feature vectors. It is the similarity of clothing texture feature vectors.

6. The participant trajectory tracking method based on dynamic face recognition according to claim 5, characterized in that: The edge-cloud collaborative decision-making mechanism in step 3 also includes: deploying a lightweight TensorRT inference engine on the edge for real-time target detection and feature extraction; deploying a large-model fine recognition network on the cloud for handling high-uncertainty samples; and implementing three-pipeline parallel processing through a dual-buffered queue: the first queue performs CUDA-accelerated image preprocessing, the second queue performs edge-side TensorRT inference, and the third queue completes feature uploading and cloud-side recognition result reception.

7. The participant trajectory tracking method based on dynamic face recognition according to claim 6, characterized in that: The behavioral characteristics in step 6 include: the distribution of dwelling preference areas, the statistical characteristics of movement speed, and the social clustering relationship map. The social clustering relationship map is constructed by analyzing the spatiotemporal co-occurrence frequency of multiple targets appearing at the same time and moving in the same direction. When the target's face is completely obscured, the identity is inferred by the high-frequency co-occurrence ID in its social relationship map, thus realizing the social relationship-assisted identity continuation.

8. The participant trajectory tracking method based on dynamic face recognition according to claim 7, characterized in that: It also includes step 8: privacy protection and data lifecycle management, which specifically includes: setting a 24-hour automatic expiration mechanism for clothing texture features; retaining facial features only during the meeting and automatically encrypting and archiving them after the meeting; deleting all original video data within 7 days after generating the trajectory stream, and retaining only the anonymized statistical information.

9. The participant trajectory tracking method based on dynamic face recognition according to claim 8, characterized in that: The trajectory association stitching in step 6 adopts a hierarchical association strategy: the first level is the association of consecutive frames within the same camera, based on IoU and optical flow; the second level is the short-term cross-camera association between adjacent cameras, based on feature similarity and spatial topology constraints. The third level is intelligent recovery of long-term fracture trajectories. Based on the memory strength in the identity imprint map, when the similarity between the reappearing target and the historical imprint exceeds the recovery threshold, the historical ID is automatically inherited.

10. A participant trajectory tracking system based on dynamic face recognition, used to implement the participant trajectory tracking method based on dynamic face recognition as described in claim 9, characterized in that, The system includes video capture terminals, edge computing nodes, cloud-based central servers, and a visualization and analysis platform; The video acquisition terminal is communicatively connected to the edge computing node and is used to acquire multiple synchronous video streams in a conference scenario, and transmit the acquired raw video streams to the edge computing node in real time. The edge computing node is connected to the cloud-side central server to perform multimodal feature extraction and uncertainty index calculation, and transmits the feature vectors to be processed on the cloud side, the original image and the preliminary processing results to the cloud-side central server. The cloud-side central server is communicatively connected to the edge computing nodes and the visualization analysis platform. It includes an identity imprint graph database, a forgetting function calculation engine, and a large model fine recognition module. It receives data transmitted from the edge computing nodes, performs identity imprint graph construction and maintenance, identity matching and new identity creation, trajectory generation and association stitching, privacy protection and data lifecycle management, completes global trajectory stitching, data storage and privacy control, and transmits the processed trajectory data and statistical analysis results to the visualization analysis platform. The visualization analysis platform communicates with the cloud-based central server, receives anonymized data transmitted from the cloud-based central server, and provides users with real-time trajectory heatmap display, single-target trajectory playback, social cluster relationship graph visualization, and privacy-anonymized data export functions, realizing the visualization presentation and data interaction of participant trajectories and related analysis results.

Images