A target tracking method based on semantic analysis model, intelligent template updating and abnormal event identification

Abstract

The application belongs to the technical field of artificial intelligence, and relates to a target tracking method for intelligent template updating and abnormal event identification based on a semantic analysis model. The method aims to solve the problems of template updating depending on low-level features, lacking semantic understanding, and the separation of abnormal identification and tracking in existing target tracking. The scheme outputs multi-dimensional scores through a semantic analysis model, constructs a comprehensive template evaluation mechanism, and realizes hierarchical management of a basic template pool and an event semantic template pool in combination with a tracking state machine, thereby guaranteeing tracking continuity while improving abnormal event identification accuracy and response efficiency.

Description

Technical Field

[0001] This invention relates to the fields of video surveillance and computer vision technology, and in particular to a target tracking method based on a semantic analysis model for intelligent template updating and abnormal event recognition. Background Technology

[0002] With the increasing prevalence of intelligent video surveillance systems, target tracking technology, as a core component for the continuous localization and trajectory analysis of dynamic targets such as people and vehicles, has become an indispensable technical support in fields such as public safety, intelligent transportation, and community management. Typical target tracking algorithms usually rely on one or more template images to characterize the target's appearance features, and then predict and update the target's position by performing template matching or feature comparison in subsequent video frames. To adapt to potential scale changes, posture adjustments, or lighting fluctuations that may occur during target movement, existing technologies generally introduce template update mechanisms to maintain the tracking model's adaptability to appearance changes. However, with the increasing complexity of application scenarios, especially the growing demand for high-reliability security monitoring, the technical paradigm upon which traditional template update strategies rely is facing more challenges.

[0003] Specifically, current mainstream target tracking systems primarily rely on quantitative indicators derived from low-level visual features, such as correlation filter response peaks, feature matching similarity, or tracking confidence scores, for template update decisions. When the matching confidence of a target region in a frame exceeds a preset threshold, the system often directly adopts the cropped region as the new template. While this strategy can effectively maintain tracking continuity in static or slowly changing scenes, its fundamental flaw lies in its lack of understanding of the high-level semantics of the image content.

[0004] Secondly, existing systems generally suffer from a disconnect between target tracking and anomaly detection functions in their architecture. In actual security monitoring deployments, besides stable target tracking, real-time identification and alerts are needed for high-risk anomalies such as "elderly people falling," "fights," and "knife attacks." However, current solutions operate on two separate fronts: a separate target tracker to maintain trajectory continuity and a dedicated behavior recognition model to detect anomalies. This fragmented architecture leads to two key flaws: the template update strategy cannot detect whether the current frame belongs to a high-risk behavior frame, thus failing to provide protective measures for critical but atypical posture frames such as "the moment of falling" or "the outbreak of conflict"; while the anomaly detection module can output behavior labels, it cannot intervene in the template management logic, potentially leading to high-risk frames being misused as base templates, severely weakening the robustness of subsequent tracking. Furthermore, the lack of a unified semantic-driven coordination mechanism makes it difficult for the system to balance tracking stability and event response accuracy when facing complex security events.

[0005] With the development of deep learning, especially visual-linguistic semantic analysis technology, models have emerged that can provide richer semantic interpretations of targets, scenes, and behaviors in images, such as recognizing behaviors like "running," "falling down," "holding a knife," and "rushing towards others." However, existing template update strategies in the field of target tracking have not fully utilized this semantic information: they do not use "behavioral semantics + risk level" to constrain and make decisions regarding template updates; even when high-risk events are detected, high-risk frames may still be used to update the base template, leading to potential deviations in subsequent tracking.

[0006] Therefore, how to construct a target tracking mechanism that can deeply integrate semantic analysis capabilities, explicitly model behavioral risks, and achieve collaborative optimization of template updates and abnormal event identification has become a key challenge and a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0007] The purpose of this invention is to propose a target tracking method based on a semantic analysis model for intelligent template updating and abnormal event recognition. This method aims to solve the problems of insufficient tracking stability and delayed event response caused by the template updating mechanism relying only on low-level visual features and lacking the ability to understand high-level semantics and behavioral risks in existing target tracking technologies, as well as the separation of target tracking and abnormal event recognition functions.

[0008] To achieve the above objectives, this invention proposes a target tracking method based on a semantic analysis model for intelligent template updating and anomaly event identification, comprising the following steps: S1: Receive the raw video stream and perform decoding and preprocessing; crop the target region to be tracked in each video frame to form a set of candidate templates, and record the timestamp, bounding box coordinates, scale parameters and tracking confidence of each candidate template; S2: Input the candidate template into the semantic analysis model to obtain the clarity score, occlusion score, background interference score, posture representativeness score, behavior state label and risk score; S3: Calculate the quality score, stability score, representativeness score and risk suppression factor for each candidate template based on the above scores, and then obtain the final template comprehensive score; S4: Dynamically switch the system operating state based on the tracking confidence and risk score. The system operating state includes normal tracking state, unstable tracking state and high-risk event state. Execute differentiated template update strategies in different system operating states. In the high-risk event state, freeze the basic template and prohibit high-risk frames from replacing the basic template. At the same time, store candidate templates with risk scores exceeding a preset threshold and their semantic information into the event semantic template pool. S5: Perform target tracking based on the current basic template in the basic template pool, and generate structured abnormal event records and push them to the management terminal when a preset abnormal behavior pattern is detected.

[0009] Furthermore, the quality score is calculated using the following formula: ; in, For the first Sharpness score of each candidate template image To score the degree of occlusion, Score the background interference. These are positive weighting parameters.

[0010] Furthermore, the stability score is calculated using the following formula: ; in, In order to be with the first The average tracking confidence of the target corresponding to each candidate template image in this frame and its neighboring frames. The time difference between the frame corresponding to the candidate template image and the current time. These are non-negative weight parameters.

[0011] Furthermore, the final template comprehensive score is obtained through the following methods: First, calculate the linear composite score: ; Multiply by the risk mitigation factor: ; Get the final template's overall score: ; in, These are non-negative weight parameters.

[0012] Furthermore, the switching rules for the system operating state include: when continuously The tracking confidence of the frame is lower than the preset confidence threshold. At that time, the tracking state changes from normal to unstable. Risk score for any frame Higher than the preset risk threshold At that time, it enters a high-risk event state; When in a high-risk event state, continuously The risk score of the frame is below the fallback threshold When the high-risk event status is exited, the tracking status will be restored to normal or unstable tracking status according to the current tracking confidence level.

[0013] Furthermore, under normal tracking conditions, when the final template comprehensive score of the current basic template is lower than the update threshold or reaches the fixed update cycle, the template with the highest final template comprehensive score and a risk score lower than the risk threshold is selected from the candidate template set. The candidate templates are used as new base templates, and several high-quality alternative templates are stored in the base template pool.

[0014] Furthermore, under unstable tracking conditions, the weighting of quality score and stability score in the overall template score is increased, the tolerance limit for risk score is reduced, and candidate templates that are close to the current frame in time are prioritized for updating the base template.

[0015] Furthermore, the basic template pool and the event semantic template pool adopt an independent storage structure. The basic template pool only stores templates that meet the conditions of low risk, high definition and typical posture, while the event semantic template pool specifically stores keyframe templates captured during high-risk events and their associated behavioral status labels, risk scores, timestamps and semantic description texts.

[0016] Furthermore, after the high-risk event ends, the template recalibration process is automatically initiated. From the candidate templates in the time period before and after the event, the template with the risk score lower than the fallback threshold and the highest final template comprehensive score is selected as the new base template, and the target tracker is reinitialized.

[0017] Furthermore, the abnormal event record includes a unique identifier for the target, start and end timestamps of the event, spatial location coordinates, risk level classification, behavior type encoding, and natural language semantic description, and is sent to the management terminal through a communication interface to trigger alarms of the corresponding level. The management terminal supports event list display, video clip playback, and multi-dimensional query.

[0018] Compared with existing technologies, the advantages of this invention are as follows: By deeply integrating semantic analysis capabilities and template management logic, this invention constructs a unified framework that integrates target tracking, template updating, and abnormal event recognition, effectively solving the problems of template pollution, tracking drift, and event response disconnect faced by traditional methods in complex security scenarios, and significantly improving the robustness, accuracy, and practicality of intelligent video surveillance systems. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the target tracking method based on a semantic analysis model for intelligent template updating and abnormal event identification according to the present invention. Figure 2 This is a block diagram of the system structure of the present invention; Figure 3 This is a schematic diagram illustrating the management relationship between the basic template pool and the event semantic template pool in this invention; Figure 4 This is a schematic diagram of the state transition tracking state machine in this invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the present invention will be further described below.

[0021] See Figure 1 This embodiment provides a target tracking method based on a semantic analysis model for intelligent template updating and abnormal event recognition. Its overall technical implementation revolves around the collaborative work of multiple modules, with each module functionally interconnected and forming a closed-loop control logic in the data flow.

[0022] Specifically, the steps include the following: Step S1: Video Input and Candidate Template Acquisition The video input module receives video streams from surveillance cameras, decodes and preprocesses them. Preprocessing includes: converting the color space from RGB to YUV or HSV to enhance illumination robustness; scaling the image to a preset normalized size (e.g., 640×480 pixels) using bilinear interpolation; and applying a non-local mean denoising algorithm to suppress sensor noise and compression artifacts. The preprocessed image frames are buffered in a frame buffer for subsequent modules to read as needed. The candidate template generation module uses target detection algorithms or existing target tracking results to locate the target to be tracked in each frame and crop the target region image. If the system is in the initial tracking stage, a lightweight target detector (such as YOLOv5s or MobileNet-SSD) is called to perform a full-image scan of the current frame and output the target bounding box and its category label. If an effective tracking trajectory has been established, the target center coordinates and scale parameters in the current frame are predicted based on the position of the previous frame fed back by the target tracking module and combined with the Kalman filter, thereby limiting the cropping area.

[0023] The system maintains a sliding time window of length N, within which the cropped target region image in each frame is used as a set of candidate template images. For each candidate template Record its corresponding timestamp Tracking confidence level Information such as the target's position and size in the image.

[0024] Step S2: Evaluation of the semantic analysis model The semantic analysis model module is a deep learning-based visual semantic understanding model. This model receives candidate template images as input and outputs various metrics related to the monitoring scene, including: Clarity rating This reflects whether the image is clear and free from obvious blur; it is calculated through frequency domain gradient energy or Laplacian variance, and after normalization, it is mapped to the [0,1] interval. The higher the value, the clearer the image.

[0025] Obstruction level rating , is obtained by inverse calculation of the intersection-union ratio of the target segmentation mask and the prior model of the complete target, and is also normalized to [0, 1], reflecting the proportion or degree of the target being occluded; Background interference score The KL divergence between the target area and the neighboring background area in terms of color histogram, LBP texture and edge direction distribution is comprehensively evaluated. After normalization, it also belongs to the [0,1] interval, reflecting whether the background is complex and whether there are a large number of similar interfering targets. Posture Representativeness Score Based on a predefined pose prototype library (such as standing, sideways, bending over, etc.), cosine similarity matching is performed, and the output value is between 0 and 1. 1 represents a complete match of a typical pose, which is used to reflect whether the target pose in the frame is typical and whether it is conducive to subsequent matching and recognition. Behavioral status labels, such as "standing", "walking", "running", "falling down", "fighting", "holding an object", "holding a knife", etc. Risk Score The data is generated by an independent risk regression head, whose training data annotations include quantitative scores (0 to 10 points) of the behavioral risk level from security experts, compressed to the [0,1] interval by the Sigmoid function. It reflects whether the frame contains any risky behaviors, such as "abnormal running," "suspected fall," or "suspected knife attack."

[0026] In one implementation, the semantic analysis model can employ a visual-language multimodal semantic analysis model, enabling it to simultaneously output structured scores and natural language descriptive text. However, the core of this invention lies not in the model structure, but in how to utilize the model's output to drive template updates and anomaly event recognition processes.

[0027] Step S3: Calculation of Multidimensional Scoring and Template-Based Comprehensive Scoring The template scoring and update decision module performs multi-dimensional scoring calculations on candidate templates.

[0028] (1) Quality rating: Taking into account sharpness, degree of occlusion and background interference, the quality rating is defined as: ; in, This is a positive weight parameter that can be configured according to the application scenario.

[0029] (2) Stability score: Considering the tracking confidence of the candidate template at the corresponding time and its nearby frames, as well as the time factor, the stability score is defined as: ; in, The average tracking confidence over a short time window centered on the candidate template frame. The time difference between the candidate template frame and the current time can be normalized. These are non-negative weight parameters.

[0030] (3) Representativeness score: Primarily based on posture representativeness score, defined as: ; In other embodiments, the representativeness score can be expanded by incorporating features such as changes in target size and viewing angle; however, for ease of explanation in this embodiment, it adopts... In the form of.

[0031] (4) Risk suppression factor: Based on the risk score, the template is subjected to exponential suppression, and the semantic safety factor is defined as: ; in, To adjust parameters, the higher the risk score, the corresponding... The closer it is to 0.

[0032] (5) Template-based comprehensive score: First, calculate the linear comprehensive score: ; in, The non-negative weight parameters are then used to obtain the final template comprehensive score: ; Through the above multi-dimensional scoring and comprehensive scoring calculation, the system can comprehensively consider factors such as image quality, temporal stability, pose representativeness and behavioral risk in the candidate templates, and sort and filter the templates.

[0033] Step S4: Tracking the state machine and template update strategy See Figure 4 In this embodiment, three tracking states are set: normal tracking state, unstable tracking state, and high-risk event state. The template scoring and update decision module performs state transitions based on tracking confidence and risk score: When the tracking confidence of M consecutive frames (M=5 in this embodiment) is lower than the preset confidence threshold Time (in this embodiment) =0.6), the system changes from normal tracking state to unstable tracking state; when the risk score of any frame is 0.6, Greater than the risk threshold ( When the risk score is 0.75, the event enters a high-risk event state; in the high-risk event state, when the risk score of L consecutive frames (L=5) is lower than the fallback threshold... ( When the value is 0.5, exit the high-risk event state and restore to the normal tracking state or unstable tracking state based on the current average tracking confidence level.

[0034] Employing differentiated template update strategies under different states: Normal tracking status: When the overall score of the current base template is lower than a preset threshold, or when a preset time period is reached, the candidate template set is... Sort them and select those with the highest overall score and the lowest risk score. The template is used as a new base template; a small number of high-quality alternative templates can be stored in the base template pool.

[0035] Unstable tracking state: Prioritize candidate templates within the most recent time frame and increase the weighting of quality and stability scores while reducing tolerance for risk scores; update the base template by selecting high-quality, low-risk templates that are more recent in time to help restore tracking stability.

[0036] High-risk event status: Freeze the current base template, disallowing direct replacement of the base template with high-risk frames to prevent template degradation due to abnormal postures or violent movements; set risk scores greater than [a certain threshold]. The templates are stored in the event semantic template pool. These templates are used to record abnormal events and are not used for updating the basic templates. After the risk status ends, candidate templates can be re-evaluated within the time period before and after the risk, and a suitable template can be selected to update the base template.

[0037] Through the aforementioned state machine and update strategy, the system can adaptively update the template in normal scenarios, quickly repair when tracking is unstable, and protect the current template from being contaminated by abnormal frames when high-risk behavior is detected.

[0038] Step S5: Target Tracking and Anomaly Output The target tracking module uses the current base template to predict and update the target's position in subsequent video frames. Meanwhile, the semantic analysis model continuously performs behavior recognition and risk assessment on target regions in new frames or keyframes.

[0039] The abnormal event management module, based on behavioral status tags and risk scores, generates an abnormal event record when it detects abnormal behaviors such as running, falling, or knife attacks. The record includes the target identifier, timestamp, spatial location, risk level, behavior type, and semantic description text. It then sends the corresponding event template image and event record to the management terminal, triggering an audio-visual or visual alarm of the appropriate level. The management terminal can display the current alarm information and supports searching historical events by time, target, or event type. Clicking on an event allows playback of the corresponding video clip.

[0040] See Figure 2 This embodiment also provides a target tracking system based on a semantic analysis model for intelligent template updating and abnormal event recognition, including: The video input module is used to receive and preprocess video surveillance streams; The candidate template generation module is used to crop candidate template images of the target to be tracked in consecutive frames from the video surveillance stream based on target detection or existing tracking results, and record timestamps and tracking confidence information. The semantic analysis model module is a deep learning-based visual semantic understanding model used to perform quality assessment and behavioral semantic analysis on the candidate template images, and outputs a sharpness score, an occlusion score, a background interference score, a pose representativeness score, a behavioral state label, and a risk score. The template scoring and update decision module is used to calculate the comprehensive template score based on the clarity score, occlusion score, background interference score, pose representativeness score, tracking confidence and risk score, maintain the tracking state machine, and decide whether to update the basic template and whether to store the template in the event semantic template pool based on the current state and the comprehensive template score. The target tracking module is used to track targets in subsequent video frames based on the current base template image; The abnormal event management module is used to generate abnormal event records based on the behavior status tags and risk scores, and send the abnormal event records to the management terminal for display or alarm. The management terminal is used to display target tracking results and abnormal event information, and supports historical event query and playback.

[0041] The semantic analysis model is deployed on edge computing nodes (such as NVIDIA Jetson AGX Orin), utilizing TensorRT or MindSpore Lite for model quantization and acceleration, keeping inference latency below 50 milliseconds, effectively reducing the load on the central server and network bandwidth consumption. In scenarios with fast-moving targets, the candidate template generation module enables a Kalman filter-based position prediction mechanism: the state vector includes position (x, y) and velocity (v). x ,v yThe observation matrix is ​​fused with the detection box and tracking prediction results, and the covariance matrix is ​​dynamically adjusted to adapt to sudden motion changes, thereby reducing pruning failures caused by instantaneous target occlusion. The weight parameters in the template scoring module support scene-adaptive calibration: when deployed in nursing homes, the risk weight δ for "falling" behaviors is increased to 0.3, and T is reduced. r Up to 0.6; In traffic intersection scenarios, an additional risk gain is assigned to the "running across" behavior, that is, when the behavior label is "running" and the target is located in the lane area, Automatically increase by 0.2. Track the threshold T of the state machine. c T r T r_back The counting parameters M and L can be dynamically optimized through an online learning mechanism.

[0042] Parameter selection and optional implementation methods In actual deployment, various parameters can be adjusted according to different monitoring scenarios (such as indoor corridors, outdoor squares, factory workshops, nursing homes, etc.), for example: The time window length N can be selected within the range of 10 to 100 frames; The number of consecutive low-confidence frames M can be set in the range of 3 to 10 frames; The exit threshold L for high-risk states can be set within the range of 5 to 20 frames; Confidence threshold Risk threshold Fallback threshold and weight parameters Adjustments can be made based on the experimental results.

[0043] The semantic analysis model can be deployed on a local server, a private cloud, or an edge computing node. In one alternative implementation, the semantic analysis model can be a pre-trained visual-language multimodal model that is fine-tuned with specific scenario data. However, this invention does not require a specific model structure. The main focus is on protecting the overall process and strategy of using semantic analysis results to drive template updates and anomaly event recognition.

[0044] In summary, this invention achieves deep integration of target tracking and abnormal event recognition by constructing a semantically aware template evaluation system, a state machine-driven differentiated update strategy, and a dual template pool isolation mechanism. The system is highly modular and configurable in engineering implementation, suitable for various security monitoring scenarios, and exhibits good real-time performance and robustness in edge computing environments.

[0045] The above are merely preferred embodiments of the present invention and do not constitute any limitation on the present invention. Any equivalent substitutions or modifications made by those skilled in the art to the technical solutions and content disclosed in the present invention without departing from the scope of the present invention shall be deemed to have remained within the protection scope of the present invention.

Claims

1. A target tracking method based on a semantic analysis model for intelligent template updating and abnormal event recognition, characterized in that, Includes the following steps: S1: Receive the raw video stream and perform decoding and preprocessing; crop the target region to be tracked in each video frame to form a set of candidate templates, and record the timestamp, bounding box coordinates, scale parameters and tracking confidence of each candidate template; S2: Input the candidate template into the semantic analysis model to obtain the clarity score, occlusion score, background interference score, posture representativeness score, behavior state label and risk score; S3: Calculate the quality score, stability score, representativeness score and risk suppression factor for each candidate template based on the above scores, and then obtain the final template comprehensive score; S4: Dynamically switch the system operating status based on the tracking confidence level and risk score. The system operating status includes normal tracking status, unstable tracking status and high-risk event status. Differentiated template update strategies are implemented under different system operating states. In the case of high-risk events, the basic template is frozen and high-risk frames are prohibited from replacing the basic template. At the same time, candidate templates with risk scores exceeding the preset threshold and their semantic information are stored in the event semantic template pool. S5: Perform target tracking based on the current basic template in the basic template pool, and generate structured abnormal event records and push them to the management terminal when a preset abnormal behavior pattern is detected.

2. The target tracking method according to claim 1, characterized in that, The quality score is calculated using the following formula: ； in, For the first Sharpness score of each candidate template image To score the degree of occlusion, Score the background interference. These are positive weighting parameters.

3. The target tracking method according to claim 1, characterized in that, The stability score is calculated using the following formula: ； in, In order to be with the first The average tracking confidence of the target corresponding to each candidate template image in this frame and its neighboring frames. The time difference between the frame corresponding to the candidate template image and the current time. These are non-negative weight parameters.

4. The target tracking method according to claim 1, characterized in that, The final template comprehensive score is obtained through the following methods: First, calculate the linear composite score: ; Multiply by the risk mitigation factor: ; Get the final template's overall score: ; in, These are non-negative weight parameters.

5. The target tracking method according to claim 1, characterized in that, The system's operating state switching rules include: when continuously The tracking confidence of the frame is lower than the preset confidence threshold. At that time, the tracking state changes from normal to unstable. Risk score for any frame Higher than the preset risk threshold At that time, it enters a high-risk event state; When in a high-risk event state, continuously The risk score of the frame is below the fallback threshold When the high-risk event status is exited, the tracking status will be restored to normal or unstable tracking status according to the current tracking confidence level.

6. The target tracking method according to claim 1, characterized in that, Under normal tracking conditions, when the final template comprehensive score of the current basic template is lower than the update threshold or reaches the fixed update cycle, the template with the highest final template comprehensive score and a risk score lower than the risk threshold is selected from the candidate template set. The candidate templates are used as new base templates, and several high-quality alternative templates are stored in the base template pool.

7. The target tracking method according to claim 1, characterized in that, In unstable tracking conditions, increase the weight of quality score and stability score in the overall template score, reduce the tolerance limit for risk score, and prioritize candidate templates that are close to the current frame in time for updating the base template.

8. The target tracking method according to claim 1, characterized in that, The basic template pool and the event semantic template pool adopt an independent storage structure. The basic template pool only stores templates that meet the conditions of low risk, high definition and typical posture. The event semantic template pool specifically stores keyframe templates captured during high-risk events and their associated behavioral status labels, risk scores, timestamps and semantic description texts.

9. The target tracking method according to claim 1, characterized in that, After a high-risk event ends, the template recalibration process is automatically initiated. From the candidate templates in the time period before and after the event, the template with the risk score lower than the fallback threshold and the highest final template comprehensive score is selected as the new base template, and the target tracker is reinitialized.

10. The target tracking method according to claim 1, characterized in that, The abnormal event record includes a unique identifier for the target, start and end timestamps of the event, spatial location coordinates, risk level classification, behavior type encoding, and natural language semantic description. It is sent to the management terminal through a communication interface to trigger alarms of the corresponding level. The management terminal supports event list display, video clip playback, and multi-dimensional query.

Images