Non-employee loitering and wandering intelligent detection system and method for a financial services area
The intelligent detection system, which integrates multi-feature fusion and spatiotemporal behavior analysis, solves the problems of employee and non-employee identification and abnormal behavior detection in financial service areas. It achieves accurate detection of non-employee loitering and wandering and provides real-time alarms, thereby improving the system's adaptability and security.
Patent Information
- Application Number
- CN202610460350.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-04-09
- Publication Date
- 2026-06-23
AI Technical Summary
Existing intelligent security systems in financial service areas cannot accurately distinguish between employees and non-employees, have a high false alarm rate, cannot effectively detect abnormal loitering behavior, lack multi-feature fusion identity recognition capabilities, cannot adapt to different scenario needs, have insufficient real-time performance, and lack intelligent detection mechanisms during clearing periods.
By employing multi-feature fusion employee identification, spatiotemporal behavior analysis, intelligent detection during clearing periods, and loitering behavior recognition technologies, and through system design including video acquisition, feature extraction, identity recognition, behavior analysis, and decision output layers, the system achieves accurate detection and real-time alerts for abnormal loitering and loitering behavior of non-employees.
It achieves accurate identification of employees and non-employees, reduces false alarm rate, improves detection accuracy and real-time performance, adapts to the scene characteristics of different financial service areas, ensures no security risks during clearing periods, and meets real-time security needs.
Smart Images

Figure CN122265946A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of intelligent video analysis technology, specifically relating to an intelligent detection system and method for non-employee loitering and lingering in financial service areas. Background Technology
[0002] With the increasing security demands of the financial industry, banks, financial institutions, and other financial service venues are placing increasingly stringent requirements on intelligent security systems. Financial service areas (such as self-service banking areas, ATM areas, and branch offices) are crucial financial service locations and must ensure that no non-employees remain during off-hours clearance periods to prevent security risks.
[0003] To address the aforementioned issues, existing technologies mainly include: traditional personnel detection systems; personnel identification systems based on facial recognition; anomaly detection systems based on behavior analysis; loitering detection systems based on trajectory analysis; lingering detection systems based on time windows; and traditional rule-based detection methods.
[0004] The main technical shortcomings of existing intelligent security technologies in financial service areas are: 1) Inability to accurately distinguish between employees and non-employees. Traditional personnel detection systems can only detect the presence of people but cannot identify their identities, resulting in a high false alarm rate and an inability to differentiate between employees on duty and non-employees loitering abnormally, failing to meet the needs of precise security; 2) Lack of multi-feature fusion for identity recognition. Single-feature (such as face, appearance, behavior) recognition has low accuracy and is greatly affected by factors such as lighting, angle, occlusion, and scene changes, resulting in poor robustness; 3) Inability to effectively detect abnormal loitering behavior. Traditional loitering detection only judges based on the time spent, failing to distinguish between employees on duty and non-employees loitering abnormally, and cannot accurately determine the presence of non-employees loitering during clearing periods; 4) Inaccurate detection of loitering behavior. The traditional loitering detection system cannot be combined with identity information, and it is easy to misjudge normal employee patrol behavior as abnormal loitering, resulting in a high false alarm rate; 5) It lacks an intelligent detection mechanism during the clearing period, and cannot automatically adjust the detection strategy according to business rules such as business hours and clearing time, and cannot effectively detect non-employee loitering during the clearing period; 6) It lacks real-time performance, and the traditional detection system has high processing latency, which cannot meet the real-time alarm requirements and the safety hazards are not detected in time; 7) It lacks the ability to comprehensively analyze spatiotemporal behavior, and only considers information from a single time point or space point, and cannot comprehensively analyze the behavior patterns of personnel in the spatiotemporal dimension, resulting in low detection accuracy; 8) It has poor scenario adaptability, and the scenario characteristics of different financial service areas (self-service area, ATM area, business hall) are very different, and the unified detection strategy cannot adapt to the needs of multiple scenarios. Summary of the Invention
[0005] This invention discloses an intelligent detection system and method for non-employee loitering and lingering in financial service areas. By integrating multiple features for employee identification, spatiotemporal behavior analysis, intelligent detection during clearing periods, and loitering behavior recognition, the system achieves accurate detection and real-time alarms for abnormal loitering and lingering behavior of non-employees in financial service areas. It effectively distinguishes between employees and non-employees, reduces false alarm rates, improves detection accuracy and real-time performance, and ensures the safe operation of financial service areas.
[0006] To achieve the above objectives, the technical solution of the present invention is as follows: The intelligent detection system for non-employee loitering and lingering in financial service areas includes a video acquisition layer, a feature extraction layer, an identity recognition layer, a behavior analysis layer, and a decision output layer connected in sequence. The video acquisition layer includes a multi-camera video stream acquisition module, a video preprocessing module, and a spatiotemporal synchronization module. The feature extraction layer includes an appearance feature extraction module, a behavior feature extraction module, a trajectory feature extraction module, and a scene feature extraction module. The identity recognition layer includes a multi-feature fusion employee identification module, an employee feature database management module, and an identity confidence assessment module. The behavior analysis layer includes a loitering behavior detection module, a lingering behavior detection module, a spatiotemporal behavior analysis module, and a clearing period detection module. The decision output layer includes an abnormal event determination module, an alarm generation module, and a result output module. These modules are connected via data streams and control signaling to form a complete intelligent detection system for non-employee loitering and lingering.
[0007] Preferably, in the video capture layer: The multi-camera video stream acquisition module is responsible for acquiring real-time video streams from multiple cameras in financial service areas (self-service areas, ATM areas, business halls, etc.), supporting multiple video stream protocols such as RTSP, RTMP, and HTTP; it adopts an asynchronous non-blocking I / O model to support concurrent acquisition of multiple video streams; it realizes video stream quality monitoring, automatically detects video stream anomalies (stream interruption, image freezing, image quality degradation, etc.) and triggers alarms; it supports video stream recording and playback functions for post-event analysis and evidence preservation. The video preprocessing module is used to preprocess the acquired video stream, including: (1) Image enhancement: Adaptive histogram equalization, noise reduction and contrast enhancement techniques are used to improve image quality and adapt to different lighting conditions; (2) Image correction: Correct camera distortion to ensure the accuracy of spatial coordinates and provide accurate spatial information for subsequent trajectory analysis; (3) Background modeling: Use deep learning-based background subtraction methods (such as background modeling algorithms based on convolutional neural networks) to build a background model and extract foreground targets; or use a Gaussian mixture model (GMM) for background modeling; (4) Target detection: The YOLOv8-based target detection algorithm is used to detect people in the video frame and output the bounding box of the people and its confidence score, or RetinaNet or other deep learning-based target detection models are used to detect the target; The aforementioned spatiotemporal synchronization module is used for spatiotemporal synchronization of video streams from multiple cameras, including: (1) Time synchronization: unify the timestamps of multiple video streams to ensure time consistency; (2) Spatial calibration: Through camera calibration and spatial mapping, the coordinate system of multiple cameras is converted to the world coordinate system to achieve cross-camera target tracking; (3) Target association: A target association algorithm based on appearance features and spatial location is adopted to realize target tracking and trajectory fusion in multi-camera scenarios.
[0008] Preferably, in the feature extraction layer: The appearance feature extraction module is used to extract the appearance features of personnel, including: (1) Clothing feature extraction: A deep learning-based clothing recognition model is used to identify whether people are wearing employee uniforms (such as bank uniforms, security uniforms, etc.) and extract clothing color, style and identification features; (2) Work permit feature extraction: Using target detection and OCR technology, the system detects and identifies whether personnel are wearing work permits and extracts the style and number features of the work permits; (3) Overall appearance feature extraction: The person re-identification (ReID) model based on deep learning is used to extract the overall appearance feature vector of the person, including height, body shape, hairstyle and backpack features; (4) Multi-scale feature fusion: Fusion of appearance features at different scales to improve the robustness of feature representation; The behavioral feature extraction module is used to extract the behavioral features of personnel, including: (1) Action feature extraction: An action recognition model based on I3D network is used to identify the types of actions of personnel (standing, walking, squatting, operating equipment, etc.). (2) Interaction feature extraction: Analyze the interaction behavior between people and equipment (ATM, self-service terminal, etc.) and extract the interaction frequency, interaction duration and interaction method features; (3) Posture feature extraction: A posture estimation model based on key point detection is used to extract the body posture and key point positions of the personnel and analyze the posture change pattern; (4) Behavioral sequence modeling: The behavioral features are serialized and the LSTM sequence model is used to model the temporal evolution pattern of the behavior; The trajectory feature extraction module is used to extract trajectory features of personnel, including: (1) Trajectory point extraction: Using a multi-target tracking algorithm, the trajectory point sequence of the person in the video sequence is extracted, including position coordinates, timestamp, speed and direction information; (2) Trajectory pattern analysis: Analyze the spatial distribution pattern of the trajectory (straight line, curve, loop, random, etc.) and extract the shape features of the trajectory; (3) Identification of dwelling areas: Based on trajectory density analysis, identify the dwelling areas and dwelling duration of personnel, and extract dwelling features; (4) Motion feature extraction: Calculate the velocity, acceleration, and rate of change of direction of the trajectory to analyze the motion pattern; The scene feature extraction module is used to extract scene context features, including: (1) Scene type recognition: Identify the current scene type (self-service area, ATM area, business hall, entrance / exit, etc.) and extract scene features; (2) Time feature extraction: Extract time features (business hours, closing time, nighttime, etc.) and combine them with business rules to determine the characteristics of the current time period; (3) Environmental feature extraction: Extract environmental features (lighting conditions, personnel density, equipment status, etc.) to provide contextual information for detection.
[0009] Preferably, in the identity recognition layer: The multi-feature fusion employee identification module is used to identify employee identity using a multi-feature fusion method, including: (1) Feature fusion strategy: Integrate multi-dimensional features such as appearance features (clothing, work ID, overall appearance), behavioral features (actions, interactions, postures), and trajectory features (trajectory patterns, dwelling areas) to construct a comprehensive feature vector; (2) Employee identification model: A deep learning-based multimodal fusion network is used to learn the distinguishing features between employees and non-employees; the network structure includes: feature extraction branch (extracting appearance, behavior and trajectory features respectively), feature fusion layer (using attention mechanism or feature splicing to fuse multimodal features), and classification layer; (3) Dynamic feature update: Based on the employee feature database and historical recognition results, the employee feature model is dynamically updated to adapt to changes in employee appearance (changing clothes, changing certificates, etc.). (4) Confidence calculation: Calculate the confidence score for identity recognition. When the confidence score is lower than the threshold, trigger manual review or use other auxiliary judgment methods. The employee characteristic database management module is used to establish and maintain the employee characteristic database, including: (1) Employee registration: Supports employee information registration, including basic information (name, employee number, department, etc.), appearance characteristics (uniform type, work permit style, etc.), and behavioral characteristics (typical behavior patterns). (2) Feature update: Based on the daily identification results, the employee features are automatically updated to adapt to changes in the employee's appearance and behavior; (3) Feature retrieval: Supports fast feature-based retrieval for identity recognition and verification; (4) Anomaly detection: Detect abnormal data in the employee feature database and promptly clean up expired or erroneous data; The identity confidence assessment module is used to assess the confidence level of identity recognition, including: (1) Multi-dimensional confidence calculation: The comprehensive confidence score is calculated by comprehensively considering factors such as appearance feature matching degree, behavior feature matching degree, and trajectory feature matching degree. (2) Confidence threshold setting: Set confidence thresholds according to business needs to distinguish between high confidence recognition (can be directly judged), medium confidence recognition (requires auxiliary judgment), and low confidence recognition (requires manual review). (3) Uncertainty handling: When the confidence level is low, multi-frame fusion and multi-view fusion methods are used to improve the recognition accuracy, or the manual review process is triggered.
[0010] Preferably, in the behavior analysis layer: The aforementioned loitering behavior detection module is used to detect loitering behavior of personnel, including: (1) Calculation of dwell time: Based on trajectory analysis, the dwell time of personnel in a specific area is calculated. When the dwell time exceeds the threshold, it is judged as a stay. (2) Identification of areas where people stay: Identify the areas where people stay (self-service area, ATM area, business hall, etc.) and judge the severity of the stay based on the importance of the area; (3) Distinguishing between employees and non-employees: Based on the identity recognition results, distinguish between employees on normal duty and non-employees loitering abnormally. Normal stay of employees in the duty area is not considered abnormal loitering; (4) Special handling during the clearing period: During the clearing period, all non-employee lingering will be judged as abnormal and an alarm will be triggered; The loitering behavior detection module is used to detect loitering behavior of personnel, including: (1) Trajectory pattern analysis: Analyze the spatial distribution pattern of personnel trajectory and identify circular, round-trip, and random wandering trajectory patterns; (2) Loitering area identification: Based on trajectory density analysis, identify the loitering area and loitering range of personnel; (3) Calculation of loitering time: Calculate the loitering time of personnel in the loitering area. When the loitering time exceeds the threshold, it is judged as abnormal loitering; (4) Employee patrol differentiation: Based on identity recognition and behavioral characteristics, distinguish between normal employee patrols and abnormal loitering by non-employees. Employee patrol behavior following a fixed route is not considered abnormal loitering; The aforementioned spatiotemporal behavior analysis module is used to comprehensively analyze personnel's behavioral patterns in the spatiotemporal dimension, including: (1) Spatiotemporal trajectory modeling: The trajectory of personnel is modeled as a spatiotemporal graph, where nodes represent spatial locations and edges represent temporal and spatial relationships, and spatiotemporal behavior patterns are analyzed; (2) Behavioral sequence analysis: Analyze the time sequence of personnel behavior to identify abnormal behavior patterns (such as leaving suddenly after a long stay, frequent entry and exit, etc.). (3) Multi-person interaction analysis: Analyze the interaction behavior among multiple people to identify abnormal clustering and following behavior; (4) Historical behavior comparison: Compare the current behavior with the historical normal behavior pattern to identify behaviors that deviate from the normal pattern; The aforementioned clearing period detection module is used to achieve intelligent detection of clearing periods, including: (1) Clearing time configuration: Supports configuring business rules for operating hours and clearing time, and automatically identifies whether the current time period is a clearing time period; (2) Detection strategy during clearing period: During the clearing period, a stricter detection strategy is adopted. All non-employee loitering and wandering are judged as abnormal and an alarm is triggered immediately. (3) Employee clearance confirmation: During the clearance period, a second confirmation is made for personnel identified as employees to ensure the accuracy of employee identification; (4) Clearance completion detection: Detect whether the clearing is completed. When no personnel are detected in the area or only confirmed employees are detected, the clearing is determined to be completed.
[0011] Preferably, in the decision output layer: The aforementioned abnormal event determination module is used to determine abnormal events by comprehensively considering the results of identity recognition and behavior analysis, including: (1) Determining the type of abnormal event: Determine the type of abnormal event (non-employee lingering, non-employee loitering, non-employees present during the clearing period, etc.); (2) Severity assessment of anomalies: The severity of the anomaly is assessed based on the type of anomaly, duration, and regional importance. (3) Multi-event correlation analysis: Analyze the correlation between multiple abnormal events and identify complex abnormal patterns; (4) False alarm filtering: Multi-frame verification and multi-view verification methods are used to filter false alarms and improve detection accuracy; The alarm generation module is used to generate abnormal alarm information, including: (1) Alarm information generation: Generate alarm records containing anomaly type, occurrence time, occurrence location, personnel information, and video clip information; (2) Alarm level setting: Set alarm levels (urgent, important, general) according to the severity of the abnormality, and use different alarm methods; (3) Alarm push: Push alarm information to relevant personnel (security personnel, management personnel, etc.), supporting multiple push methods (SMS, email, APP push, etc.); (4) Alarm record management: Save alarm records and support query, statistics and analysis functions; The result output module is used to output the detection results, including: (1) Real-time detection result output: Real-time output of personnel detection, identity recognition, and behavior analysis results for use by upper-level applications; (2) Visualization: Visualize the detection results, including personnel annotation, trajectory display, and highlighting of abnormal areas; (3) Data interface: Provides a standardized data interface to support integration with third-party systems; (4) Report generation: Generate detection and statistical reports, including abnormal event statistics, employee activity statistics, and regional safety assessment.
[0012] The intelligent detection method for non-employee loitering and lingering in financial services areas includes the following steps: Step 1: System Initialization and Configuration: After the system starts, initialize the video acquisition layer, feature extraction layer, identity recognition layer, behavior analysis layer, and decision output layer modules; load the employee feature library and establish the employee recognition model; configure the clearing time, detection threshold, alarm rules, and business parameters; initialize the multi-camera video stream connection and establish spatiotemporal synchronization; load the deep learning model, namely the target detection model, feature extraction model, and behavior analysis model. Step 2: Video Stream Acquisition and Preprocessing: The multi-camera video stream acquisition module acquires real-time video streams from multiple cameras in the financial services area; the video preprocessing module performs image enhancement, correction, and background modeling preprocessing on the video streams; a multi-target detection algorithm is used to detect personnel targets in the video frames, and outputs personnel bounding boxes and confidence scores; the spatiotemporal synchronization module performs spatiotemporal synchronization on multiple video streams to achieve cross-camera target tracking; Step 3: Multi-dimensional feature extraction: The appearance feature extraction module extracts clothing features, work permit features, and overall appearance features of personnel; the behavior feature extraction module extracts action features, interaction features, and posture features of personnel; the trajectory feature extraction module extracts personnel trajectories through multi-target tracking and analyzes trajectory patterns and motion features; the scene feature extraction module extracts contextual information such as scene type, time features, and environmental features. Step 4: Employee Identification: The multi-feature fusion employee identification module integrates appearance features, behavioral features, and trajectory features, and uses a deep learning model to identify employees and non-employees; the employee feature library management module provides employee feature retrieval and matching; the identity confidence assessment module calculates the confidence level of identity recognition, and triggers auxiliary judgment or manual review when the confidence level is low; Step 5: Behavior Analysis and Anomaly Detection: The loitering behavior detection module detects personnel loitering behavior based on trajectory analysis, and distinguishes between employees on duty and non-employees loitering abnormally based on identity recognition results; the wandering behavior detection module analyzes trajectory patterns to detect wandering behavior, and distinguishes between employees on patrol and non-employees wandering abnormally; the spatiotemporal behavior analysis module comprehensively analyzes personnel behavior patterns in the spatiotemporal dimension to identify abnormal behavior; the clearing period detection module determines whether it is a clearing period based on business rules, and adopts stricter detection strategies during clearing periods; Step 6: Abnormal Event Judgment and Alarm: The abnormal event judgment module integrates identity recognition and behavior analysis results to determine the type and severity of abnormal events; the alarm generation module generates alarm information and pushes it to relevant personnel; the result output module outputs the detection results, providing visualization and data interface; the system continuously monitors and detects abnormal loitering and wandering behavior of non-employees in real time.
[0013] The beneficial effects of the self-service financial service terminal and vault security facility anomaly detection system and method of the present invention are as follows: 1) Accurate employee / non-employee identification: Through multi-feature fusion, integrating appearance features (clothing, work permit, overall appearance), behavioral features (actions, interactions, postures), trajectory features, and other multi-dimensional features, accurate employee / non-employee identification is achieved, significantly improving accuracy and greatly reducing false alarm rate; 2) Effective abnormal loitering detection: Combining identity recognition results, it can accurately distinguish between employees on duty and non-employees loitering abnormally. During clearing periods, it can effectively detect non-employee loitering abnormally, triggering alarms in a timely manner to ensure the security of the financial service area; 3) Accurate loitering behavior identification: Through trajectory pattern analysis and behavioral feature recognition, it can accurately detect loitering behavior and distinguish between employees on normal patrols and non-employees loitering abnormally, reducing false alarm rate; 4) Intelligent clearing period detection mechanism: Automatically identifies clearing periods according to business rules. During the clearing period, a stricter detection strategy is adopted, and all non-employee loitering and wandering are judged as abnormal to ensure that there are no safety hazards during the clearing period; 5) Powerful spatiotemporal behavior analysis capability: Comprehensive analysis of personnel behavior patterns in the spatiotemporal dimension to identify complex abnormal behaviors and improve detection accuracy and robustness; 6) Excellent real-time performance: Utilizing efficient deep learning models and optimization algorithms, real-time detection and alarms are achieved to meet real-time security needs; 7) Good scene adaptability: It can adapt to the scene characteristics of different financial service areas (self-service area, ATM area, business hall), and maintain high performance in different scenarios through scene feature extraction and adaptive adjustment; 8) Perfect multi-feature fusion mechanism: Through multi-dimensional feature fusion, it overcomes the limitations of single features, improves recognition accuracy and robustness, and adapts to complex conditions such as lighting, angle, and occlusion. Attached Figure Description
[0014] Figure 1 This is the overall system architecture diagram.
[0015] Figure 2 This is a flowchart of the detection and processing procedure.
[0016] Figure 3 This is a diagram of the detection algorithm architecture.
[0017] Figure 4 Flowchart for multi-feature fusion employee identification.
[0018] Figure 5 This is a flowchart of the testing process during the clearing period. Detailed Implementation
[0019] The following description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
[0020] The following embodiments can be understood as illustrating a part of the structure or method of the present invention individually, or as combining the embodiments to explain the broader structure or method of the present invention.
[0021] Example 1: Intelligent detection systems for non-employee loitering and wandering in financial services areas, such as Figure 1 As shown, the system comprises a video acquisition layer, a feature extraction layer, an identity recognition layer, a behavior analysis layer, and a decision output layer connected in sequence. The video acquisition layer includes a multi-camera video stream acquisition module, a video preprocessing module, and a spatiotemporal synchronization module. The feature extraction layer includes an appearance feature extraction module, a behavior feature extraction module, a trajectory feature extraction module, and a scene feature extraction module. The identity recognition layer includes a multi-feature fusion employee identification module, an employee feature database management module, and an identity confidence assessment module. The behavior analysis layer includes a loitering behavior detection module, a wandering behavior detection module, a spatiotemporal behavior analysis module, and a clearing period detection module. The decision output layer includes an abnormal event determination module, an alarm generation module, and a result output module. These modules are connected via data streams and control signaling to form a complete intelligent detection system for non-employee loitering and wandering.
[0022] Example 2: like Figure 1 As shown, in the video acquisition layer: The multi-camera video stream acquisition module is responsible for acquiring real-time video streams from multiple cameras in financial service areas (self-service areas, ATM areas, business halls, etc.), supporting multiple video stream protocols such as RTSP, RTMP, and HTTP; it adopts an asynchronous non-blocking I / O model to support concurrent acquisition of multiple video streams; it realizes video stream quality monitoring, automatically detects video stream anomalies (stream interruption, image freezing, image quality degradation, etc.) and triggers alarms; it supports video stream recording and playback functions for post-event analysis and evidence preservation. The video preprocessing module is used to preprocess the acquired video stream, including: (1) Image enhancement: Adaptive histogram equalization, noise reduction and contrast enhancement techniques are used to improve image quality and adapt to different lighting conditions; (2) Image correction: Correct camera distortion to ensure the accuracy of spatial coordinates and provide accurate spatial information for subsequent trajectory analysis; (3) Background modeling: Use deep learning-based background subtraction methods (such as background modeling algorithms based on convolutional neural networks) to build a background model and extract foreground targets; or use a Gaussian mixture model (GMM) for background modeling; (4) Target detection: The YOLOv8-based target detection algorithm is used to detect people in the video frame and output the bounding box of the people and its confidence score, or RetinaNet or other deep learning-based target detection models are used to detect the target; The aforementioned spatiotemporal synchronization module is used for spatiotemporal synchronization of video streams from multiple cameras, including: (1) Time synchronization: unify the timestamps of multiple video streams to ensure time consistency; (2) Spatial calibration: Through camera calibration and spatial mapping, the coordinate system of multiple cameras is converted to the world coordinate system to achieve cross-camera target tracking; (3) Target association: A target association algorithm based on appearance features and spatial location is adopted to realize target tracking and trajectory fusion in multi-camera scenarios.
[0023] Example 3: like Figure 1 As shown, in the feature extraction layer: The appearance feature extraction module is used to extract the appearance features of personnel, including: (1) Clothing feature extraction: A deep learning-based clothing recognition model is used to identify whether people are wearing employee uniforms (such as bank uniforms, security uniforms, etc.) and extract clothing color, style and identification features; (2) Work permit feature extraction: Using target detection and OCR technology, the system detects and identifies whether personnel are wearing work permits and extracts the style and number features of the work permits; (3) Overall appearance feature extraction: The person re-identification (ReID) model based on deep learning is used to extract the overall appearance feature vector of the person, including height, body shape, hairstyle and backpack features; (4) Multi-scale feature fusion: Fusion of appearance features at different scales to improve the robustness of feature representation; The behavioral feature extraction module is used to extract the behavioral features of personnel, including: (1) Action feature extraction: An action recognition model based on I3D network is used to identify the types of actions of personnel (standing, walking, squatting, operating equipment, etc.). (2) Interaction feature extraction: Analyze the interaction behavior between people and equipment (ATM, self-service terminal, etc.) and extract the interaction frequency, interaction duration and interaction method features; (3) Posture feature extraction: A posture estimation model based on key point detection is used to extract the body posture and key point positions of the personnel and analyze the posture change pattern; (4) Behavioral sequence modeling: The behavioral features are serialized and the LSTM sequence model is used to model the temporal evolution pattern of the behavior; The trajectory feature extraction module is used to extract trajectory features of personnel, including: (1) Trajectory point extraction: Using a multi-target tracking algorithm, the trajectory point sequence of the person in the video sequence is extracted, including position coordinates, timestamp, speed and direction information; (2) Trajectory pattern analysis: Analyze the spatial distribution pattern of the trajectory (straight line, curve, loop, random, etc.) and extract the shape features of the trajectory; (3) Identification of dwelling areas: Based on trajectory density analysis, identify the dwelling areas and dwelling duration of personnel, and extract dwelling features; (4) Motion feature extraction: Calculate the velocity, acceleration, and rate of change of direction of the trajectory to analyze the motion pattern; The scene feature extraction module is used to extract scene context features, including: (1) Scene type recognition: Identify the current scene type (self-service area, ATM area, business hall, entrance / exit, etc.) and extract scene features; (2) Time feature extraction: Extract time features (business hours, closing time, nighttime, etc.) and combine them with business rules to determine the characteristics of the current time period; (3) Environmental feature extraction: Extract environmental features (lighting conditions, personnel density, equipment status, etc.) to provide contextual information for detection.
[0024] Example 4: like Figure 1 As shown, in the identity recognition layer: The multi-feature fusion employee identification module is used to identify employee identity using a multi-feature fusion method, including: (1) Feature fusion strategy: Integrate multi-dimensional features such as appearance features (clothing, work ID, overall appearance), behavioral features (actions, interactions, postures), and trajectory features (trajectory patterns, dwelling areas) to construct a comprehensive feature vector; (2) Employee identification model: A deep learning-based multimodal fusion network is used to learn the distinguishing features between employees and non-employees; the network structure includes: feature extraction branch (extracting appearance, behavior and trajectory features respectively), feature fusion layer (using attention mechanism or feature splicing to fuse multimodal features), and classification layer; (3) Dynamic feature update: Based on the employee feature database and historical recognition results, the employee feature model is dynamically updated to adapt to changes in employee appearance (changing clothes, changing certificates, etc.). (4) Confidence calculation: Calculate the confidence score for identity recognition. When the confidence score is lower than the threshold, trigger manual review or use other auxiliary judgment methods. The employee characteristic database management module is used to establish and maintain the employee characteristic database, including: (1) Employee registration: Supports employee information registration, including basic information (name, employee number, department, etc.), appearance characteristics (uniform type, work permit style, etc.), and behavioral characteristics (typical behavior patterns). (2) Feature update: Based on the daily identification results, the employee features are automatically updated to adapt to changes in the employee's appearance and behavior; (3) Feature retrieval: Supports fast feature-based retrieval for identity recognition and verification; (4) Anomaly detection: Detect abnormal data in the employee feature database and promptly clean up expired or erroneous data; The identity confidence assessment module is used to assess the confidence level of identity recognition, including: (1) Multi-dimensional confidence calculation: The comprehensive confidence score is calculated by comprehensively considering factors such as appearance feature matching degree, behavior feature matching degree, and trajectory feature matching degree. (2) Confidence threshold setting: Set confidence thresholds according to business needs to distinguish between high confidence recognition (can be directly judged), medium confidence recognition (requires auxiliary judgment), and low confidence recognition (requires manual review). (3) Uncertainty handling: When the confidence level is low, multi-frame fusion and multi-view fusion methods are used to improve the recognition accuracy, or the manual review process is triggered.
[0025] Example 5: like Figure 1 As shown, in the behavior analysis layer: The aforementioned loitering behavior detection module is used to detect loitering behavior of personnel, including: (1) Calculation of dwell time: Based on trajectory analysis, the dwell time of personnel in a specific area is calculated. When the dwell time exceeds the threshold, it is judged as a stay. (2) Identification of areas where people stay: Identify the areas where people stay (self-service area, ATM area, business hall, etc.) and judge the severity of the stay based on the importance of the area; (3) Distinguishing between employees and non-employees: Based on the identity recognition results, distinguish between employees on normal duty and non-employees loitering abnormally. Normal stay of employees in the duty area is not considered abnormal loitering; (4) Special handling during the clearing period: During the clearing period, all non-employee lingering will be judged as abnormal and an alarm will be triggered; The loitering behavior detection module is used to detect loitering behavior of personnel, including: (1) Trajectory pattern analysis: Analyze the spatial distribution pattern of personnel trajectory and identify circular, round-trip, and random wandering trajectory patterns; (2) Loitering area identification: Based on trajectory density analysis, identify the loitering area and loitering range of personnel; (3) Calculation of loitering time: Calculate the loitering time of personnel in the loitering area. When the loitering time exceeds the threshold, it is judged as abnormal loitering; (4) Employee patrol differentiation: Based on identity recognition and behavioral characteristics, distinguish between normal employee patrols and abnormal loitering by non-employees. Employee patrol behavior following a fixed route is not considered abnormal loitering; The aforementioned spatiotemporal behavior analysis module is used to comprehensively analyze personnel's behavioral patterns in the spatiotemporal dimension, including: (1) Spatiotemporal trajectory modeling: The trajectory of personnel is modeled as a spatiotemporal graph, where nodes represent spatial locations and edges represent temporal and spatial relationships, and spatiotemporal behavior patterns are analyzed; (2) Behavioral sequence analysis: Analyze the time sequence of personnel behavior to identify abnormal behavior patterns (such as leaving suddenly after a long stay, frequent entry and exit, etc.). (3) Multi-person interaction analysis: Analyze the interaction behavior among multiple people to identify abnormal clustering and following behavior; (4) Historical behavior comparison: Compare the current behavior with the historical normal behavior pattern to identify behaviors that deviate from the normal pattern; The aforementioned clearing period detection module is used to achieve intelligent detection of clearing periods, including: (1) Clearing time configuration: Supports configuring business rules for operating hours and clearing time, and automatically identifies whether the current time period is a clearing time period; (2) Detection strategy during clearing period: During the clearing period, a stricter detection strategy is adopted. All non-employee loitering and wandering are judged as abnormal and an alarm is triggered immediately. (3) Employee clearance confirmation: During the clearance period, a second confirmation is made for personnel identified as employees to ensure the accuracy of employee identification; (4) Clearance completion detection: Detect whether the clearing is completed. When no personnel are detected in the area or only confirmed employees are detected, the clearing is determined to be completed.
[0026] Example 6: like Figure 1 As shown, in the decision output layer: The aforementioned abnormal event determination module is used to determine abnormal events by comprehensively considering the results of identity recognition and behavior analysis, including: (1) Determining the type of abnormal event: Determine the type of abnormal event (non-employee lingering, non-employee loitering, non-employees present during the clearing period, etc.); (2) Severity assessment of anomalies: The severity of the anomaly is assessed based on the type of anomaly, duration, and regional importance. (3) Multi-event correlation analysis: Analyze the correlation between multiple abnormal events and identify complex abnormal patterns; (4) False alarm filtering: Multi-frame verification and multi-view verification methods are used to filter false alarms and improve detection accuracy; The alarm generation module is used to generate abnormal alarm information, including: (1) Alarm information generation: Generate alarm records containing anomaly type, occurrence time, occurrence location, personnel information, and video clip information; (2) Alarm level setting: Set alarm levels (urgent, important, general) according to the severity of the abnormality, and use different alarm methods; (3) Alarm push: Push alarm information to relevant personnel (security personnel, management personnel, etc.), supporting multiple push methods (SMS, email, APP push, etc.); (4) Alarm record management: Save alarm records and support query, statistics and analysis functions; The result output module is used to output the detection results, including: (1) Real-time detection result output: Real-time output of personnel detection, identity recognition, and behavior analysis results for use by upper-level applications; (2) Visualization: Visualize the detection results, including personnel annotation, trajectory display, and highlighting of abnormal areas; (3) Data interface: Provides a standardized data interface to support integration with third-party systems; (4) Report generation: Generate detection and statistical reports, including abnormal event statistics, employee activity statistics, and regional safety assessment.
[0027] Example 7: Based on the above embodiments, this embodiment discloses an intelligent detection method for non-employee loitering and lingering in financial service areas, such as... Figure 2 As shown, it includes the following steps: Step 1: System Initialization and Configuration: After the system starts, initialize the video acquisition layer, feature extraction layer, identity recognition layer, behavior analysis layer, and decision output layer modules; load the employee feature library and establish the employee recognition model; configure the clearing time, detection threshold, alarm rules, and business parameters; initialize the multi-camera video stream connection and establish spatiotemporal synchronization; load the deep learning model, namely the target detection model, feature extraction model, and behavior analysis model. Step 2: Video Stream Acquisition and Preprocessing: The multi-camera video stream acquisition module acquires real-time video streams from multiple cameras in the financial services area; the video preprocessing module performs image enhancement, correction, and background modeling preprocessing on the video streams; a multi-target detection algorithm is used to detect personnel targets in the video frames, and outputs personnel bounding boxes and confidence scores; the spatiotemporal synchronization module performs spatiotemporal synchronization on multiple video streams to achieve cross-camera target tracking; Step 3: Multi-dimensional feature extraction: The appearance feature extraction module extracts clothing features, work permit features, and overall appearance features of personnel; the behavior feature extraction module extracts action features, interaction features, and posture features of personnel; the trajectory feature extraction module extracts personnel trajectories through multi-target tracking and analyzes trajectory patterns and motion features; the scene feature extraction module extracts contextual information such as scene type, time features, and environmental features. Step 4: Employee Identification: The multi-feature fusion employee identification module integrates appearance features, behavioral features, and trajectory features, and uses a deep learning model to identify employees and non-employees; the employee feature library management module provides employee feature retrieval and matching; the identity confidence assessment module calculates the confidence level of identity recognition, and triggers auxiliary judgment or manual review when the confidence level is low; Step 5: Behavior Analysis and Anomaly Detection: The loitering behavior detection module detects personnel loitering behavior based on trajectory analysis, and distinguishes between employees on duty and non-employees loitering abnormally based on identity recognition results; the wandering behavior detection module analyzes trajectory patterns to detect wandering behavior, and distinguishes between employees on patrol and non-employees wandering abnormally; the spatiotemporal behavior analysis module comprehensively analyzes personnel behavior patterns in the spatiotemporal dimension to identify abnormal behavior; the clearing period detection module determines whether it is a clearing period based on business rules, and adopts stricter detection strategies during clearing periods; Step 6: Abnormal Event Judgment and Alarm: The abnormal event judgment module integrates identity recognition and behavior analysis results to determine the type and severity of abnormal events; the alarm generation module generates alarm information and pushes it to relevant personnel; the result output module outputs the detection results, providing visualization and data interface; the system continuously monitors and detects abnormal loitering and wandering behavior of non-employees in real time.
[0028] Working principle of the invention: In the entire system, the multi-feature fusion employee identification module is the core component, responsible for accurately distinguishing between employees and non-employees, providing accurate input for subsequent behavior analysis. The loitering behavior detection module and the wandering behavior detection module combine the identification results to accurately detect abnormal behavior of non-employees. The clearing period detection module enables intelligent detection during clearing periods, ensuring that no non-employees remain during these times. This innovative detection algorithm architecture (see details...) Figure 3 (Detection algorithm architecture diagram) The system can accurately detect and provide real-time alerts for abnormal loitering and wandering behavior of non-employees in financial service areas, effectively ensuring the safe operation of financial service areas.
[0029] like Figure 3 As shown in the detection algorithm architecture diagram, the system adopts a layered detection architecture design, which includes three core detection layers: Layer 1 - Feature Extraction Layer: Responsible for multi-dimensional feature extraction, including modules for appearance feature extraction, behavior feature extraction, trajectory feature extraction, and scene feature extraction. It adopts a deep learning-based feature extraction method to extract rich feature information from video streams, providing a data foundation for identity recognition and behavior analysis.
[0030] Layer 2 - Intelligent Analysis Layer: Responsible for identity recognition and behavior analysis, including modules such as multi-feature fusion employee recognition, lingering behavior detection, loitering behavior detection, spatiotemporal behavior analysis, and clearing period detection. It adopts multi-feature fusion and deep learning methods to achieve accurate identity recognition and abnormal behavior detection.
[0031] Layer 3 - Decision Output Layer: Responsible for anomaly event judgment and alarm output, including modules such as anomaly event judgment, alarm generation, and result output. It integrates identity recognition and behavior analysis results to generate alarm information and output detection results.
[0032] Each layer is connected according to the data flow path of "feature extraction → intelligent analysis → decision output", and information exchange is achieved through standardized interfaces to ensure the system's hierarchical detection, efficient collaboration and accurate identification characteristics.
[0033] like Figure 4 As shown in the flowchart (multi-feature fusion employee identification flowchart), the multi-feature fusion employee identification module uses a multi-feature fusion method to identify employee identity: Feature extraction stage: Extract multi-dimensional features such as appearance features (clothing, work ID, overall appearance), behavioral features (actions, interactions, postures), and trajectory features (trajectory patterns, dwell areas).
[0034] Feature fusion stage: A deep learning-based multimodal fusion network is used to fuse multi-dimensional features. The network structure includes: multiple feature extraction branches (processing appearance, behavior, and trajectory features respectively), a feature fusion layer (adaptively fusing multimodal features using an attention mechanism and learning the importance weights of different features), a feature dimensionality reduction layer (reducing the fused high-dimensional features to a low-dimensional feature space), and a classification layer (outputting the probability and confidence of employee / non-employee).
[0035] Identity Recognition Stage: Based on the fused feature vector and classification results, the identity of the person (employee / non-employee) is determined, and the recognition confidence score is calculated. When the confidence score is higher than the high threshold, the identity is directly determined; when the confidence score is in the middle threshold range, methods such as multi-frame fusion and multi-view fusion are used to improve the recognition accuracy; when the confidence score is lower than the low threshold, manual review is triggered.
[0036] Feature update phase: Based on the recognition results and the employee feature database, the employee feature model is dynamically updated to adapt to changes in employee appearance and behavior. For example... Figure 5As shown in the flowchart (clearance period detection flowchart), the clearance period detection module realizes intelligent detection of the clearance period: Time Determination Phase: Based on configured business rules such as operating hours and closing time, determine whether the current time period is a closing period. The closing period usually refers to a period of time after the end of operating hours, or a specific time period at night.
[0037] Inspection strategy during clearing period: A stricter inspection strategy will be adopted during the clearing period. (1) All non-employee stays are judged as abnormal and an alarm is triggered immediately, regardless of the stay duration.
[0038] (2) All non-employee loitering is judged as abnormal and an alarm is triggered immediately.
[0039] (3) A second verification is conducted on personnel identified as employees, using methods such as multi-frame verification and multi-view verification to ensure the accuracy of employee identification.
[0040] (4) Increase the detection frequency and shorten the detection cycle to ensure timely detection of abnormalities.
[0041] Clearance Completion Inspection: Continuously monitor the personnel situation within the area. When no personnel are detected in the area or only confirmed employees remain, the clearing is considered complete, and the clearing completion time is recorded.
[0042] Alarm Handling: When a non-employee is detected during the clearing period, an emergency alarm is immediately generated and pushed to security personnel and management personnel, providing detailed information such as video clips and location information to support rapid response.
[0043] It should be noted that: Regarding the technical solution of this invention, the following alternative solutions can also be considered: For employee identification technology, a face recognition-based method can be used, which identifies employees by pre-establishing an employee face database. However, this requires a complete face database, cannot identify temporary visitors, and is greatly affected by lighting, angle, and occlusion. Alternatively, an RFID or IC card-based identification method can be used, identifying employees through RFID tags or IC cards they wear. However, this requires additional hardware and is costly. Biometric identification methods (such as gait recognition and voiceprint recognition) can also be used, but the accuracy is relatively low and requires specific conditions. For feature extraction, traditional machine learning methods (such as HOG, SIFT, and other handcrafted features) can be used, which have low computational complexity but limited feature representation capabilities. A single deep learning-based feature extraction method can also be used, using only appearance or behavioral features, which is simple to implement but less accurate than multi-feature fusion methods. An unsupervised learning-based feature extraction method can also be used, which does not require labeled data but requires a large amount of training data. In terms of behavior detection, rule-based methods can be used to judge abnormal behavior according to manually set rules, which is simple to implement but lacks adaptability. Methods based on single behavioral features (such as judging lingering based solely on dwell time) can also be used, which is simple to implement but cannot distinguish between normal employee behavior and abnormal non-employee behavior. Statistical methods can also be used, identifying deviations from normal patterns through statistical analysis of normal behavior patterns, but require a large amount of historical data. Regarding clearance period detection, methods based on fixed-time rules can be used, detecting according to preset clearance times, which is simple to implement but lacks flexibility. Methods based on manual triggering can also be used, where clearance detection is triggered manually, but this requires manual operation and has poor real-time performance. Event-based methods can also be used, triggering clearance detection based on specific events (such as closing or locking doors), but this requires an additional event detection mechanism. For multi-camera fusion, single-camera-based detection methods can be used, which are simple to implement but have limited detection range and cannot achieve cross-camera tracking. Alternatively, simple time synchronization methods can be employed, performing only time synchronization without spatial calibration and target association, which are also simple to implement but cannot achieve cross-camera target tracking. Centralized processing methods can also be used, concentrating all video streams on a central server for processing, but this requires a high-bandwidth network and results in high latency. For anomaly detection, single-threshold-based methods can be used, judging anomalies based on a single indicator (such as dwell time), which are simple to implement but have low accuracy. Multi-rule combination methods can also be used, combining multiple rules for judgment, but rule design is complex and lacks adaptability. Machine learning-based classification methods can also be used, training a classifier to judge anomalies, but this requires a large amount of labeled data.In terms of alarm generation, a method based on a single alarm method (such as SMS alarm only) can be used, which is simple to implement but only provides a single alarm method; a method based on a fixed alarm level can also be used, where all anomalies use the same alarm level, which is simple to implement but cannot distinguish the severity of anomalies; a method based on delayed alarms can also be used, where alarms are generated after a certain delay, but the best response time may be missed.
[0044] The above examples illustrate only some equivalent substitution methods. Those skilled in the art should understand that any equivalent substitution of the technical solutions of the present invention without departing from the spirit and scope of the present invention falls within the protection scope of the present invention.
Claims
1. An intelligent detection system for non-employee loitering and lingering in financial service areas, characterized by: It includes a video acquisition layer, a feature extraction layer, an identity recognition layer, a behavior analysis layer, and a decision output layer, which are connected in sequence. The video acquisition layer includes a multi-camera video stream acquisition module, a video preprocessing module, and a spatiotemporal synchronization module; the feature extraction layer includes an appearance feature extraction module, a behavior feature extraction module, a trajectory feature extraction module, and a scene feature extraction module; the identity recognition layer includes a multi-feature fusion employee recognition module, an employee feature database management module, and an identity confidence assessment module; the behavior analysis layer includes a loitering behavior detection module, a wandering behavior detection module, a spatiotemporal behavior analysis module, and a clearing period detection module; the decision output layer includes an abnormal event judgment module, an alarm generation module, and a result output module; all modules are connected through data streams and control signaling to form a complete intelligent detection system for non-employee loitering and wandering.
2. The intelligent detection system for non-employee loitering and lingering in financial service areas as described in claim 1, characterized in that: In the video capture layer: The multi-camera video stream acquisition module is responsible for acquiring real-time video streams from multiple cameras in the financial services area, supporting multiple video stream protocols such as RTSP, RTMP, and HTTP; it adopts an asynchronous non-blocking I / O model and supports concurrent acquisition of multiple video streams; Enables video stream quality monitoring, automatically detects video stream anomalies and triggers alarms; It supports video stream recording and playback for post-event analysis and evidence preservation. The video preprocessing module is used to preprocess the acquired video stream, including: (1) Image enhancement: Adaptive histogram equalization, noise reduction and contrast enhancement techniques are used to improve image quality and adapt to different lighting conditions; (2) Image correction: Correct camera distortion to ensure the accuracy of spatial coordinates and provide accurate spatial information for subsequent trajectory analysis; (3) Background modeling: A background model is established by using a deep learning-based background subtraction method to extract the foreground target; or a Gaussian mixture model is used for background modeling. (4) Target detection: The YOLOv8-based target detection algorithm is used to detect people in the video frame and output the bounding box of the people and its confidence score, or RetinaNet or other deep learning-based target detection models are used to detect the target; The aforementioned spatiotemporal synchronization module is used for spatiotemporal synchronization of video streams from multiple cameras, including: (1) Time synchronization: unify the timestamps of multiple video streams to ensure time consistency; (2) Spatial calibration: Through camera calibration and spatial mapping, the coordinate system of multiple cameras is converted to the world coordinate system to achieve cross-camera target tracking; (3) Target association: A target association algorithm based on appearance features and spatial location is adopted to realize target tracking and trajectory fusion in multi-camera scenarios.
3. The intelligent detection system for non-employee loitering and lingering in financial service areas as described in claim 1, characterized in that: In the feature extraction layer: The appearance feature extraction module is used to extract the appearance features of personnel, including: (1) Clothing feature extraction: A deep learning-based clothing recognition model is used to identify whether personnel are wearing employee uniforms and extract clothing color, style, and identification features; (2) Work permit feature extraction: Using target detection and OCR technology, the system detects and identifies whether personnel are wearing work permits and extracts the style and number features of the work permits; (3) Overall appearance feature extraction: A deep learning-based pedestrian re-identification model is used to extract the overall appearance feature vector of the person, including height, body shape, hairstyle and backpack features; (4) Multi-scale feature fusion: Fusion of appearance features at different scales to improve the robustness of feature representation; The behavioral feature extraction module is used to extract the behavioral features of personnel, including: (1) Action feature extraction: An action recognition model based on I3D network is used to identify the types of actions of personnel; (2) Interaction feature extraction: Analyze the interaction behavior between people and equipment, and extract the interaction frequency, interaction duration and interaction method features; (3) Posture feature extraction: A posture estimation model based on key point detection is used to extract the body posture and key point positions of the personnel and analyze the posture change pattern; (4) Behavioral sequence modeling: The behavioral features are serialized and the LSTM sequence model is used to model the temporal evolution pattern of the behavior; The trajectory feature extraction module is used to extract trajectory features of personnel, including: (1) Trajectory point extraction: Using a multi-target tracking algorithm, the trajectory point sequence of the person in the video sequence is extracted, including position coordinates, timestamp, speed and direction information; (2) Trajectory pattern analysis: Analyze the spatial distribution pattern of the trajectory and extract its shape features; (3) Identification of dwelling areas: Based on trajectory density analysis, identify the dwelling areas and dwelling duration of personnel, and extract dwelling features; (4) Motion feature extraction: Calculate the velocity, acceleration, and rate of change of direction of the trajectory to analyze the motion pattern; The scene feature extraction module is used to extract scene context features, including: (1) Scene type recognition: Identify the current scene type and extract scene features; (2) Time feature extraction: Extract time features and combine them with business rules to determine the characteristics of the current time period; (3) Environmental feature extraction: Extract environmental features to provide contextual information for detection.
4. The intelligent detection system for non-employee loitering and lingering in financial service areas as described in claim 1, characterized in that: In the aforementioned identity recognition layer: The multi-feature fusion employee identification module is used to identify employee identity using a multi-feature fusion method, including: (1) Feature fusion strategy: fuse appearance features, behavioral features and trajectory features to construct a comprehensive feature vector; (2) Employee identification model: A deep learning-based multimodal fusion network is used to learn the distinguishing features between employees and non-employees; the network structure includes: feature extraction branch, feature fusion layer, and classification layer; (3) Dynamic feature update: Based on the employee feature database and historical recognition results, the employee feature model is dynamically updated to adapt to changes in employee appearance; (4) Confidence calculation: Calculate the confidence score for identity recognition. When the confidence score is lower than the threshold, trigger manual review or use other auxiliary judgment methods. The employee characteristic database management module is used to establish and maintain the employee characteristic database, including: (1) Employee registration: Supports employee information registration, including basic information, appearance characteristics, and behavioral characteristics; (2) Feature update: Based on the daily identification results, the employee features are automatically updated to adapt to changes in the employee's appearance and behavior; (3) Feature retrieval: Supports fast feature-based retrieval for identity recognition and verification; (4) Anomaly detection: Detect abnormal data in the employee feature database and promptly clean up expired or erroneous data; The identity confidence assessment module is used to assess the confidence level of identity recognition, including: (1) Multi-dimensional confidence calculation: The comprehensive confidence score is calculated by comprehensively considering factors such as appearance feature matching degree, behavior feature matching degree, and trajectory feature matching degree. (2) Confidence threshold setting: Set the confidence threshold according to business needs to distinguish between high confidence recognition, medium confidence recognition and low confidence recognition; (3) Uncertainty handling: When the confidence level is low, multi-frame fusion and multi-view fusion methods are used to improve the recognition accuracy, or the manual review process is triggered.
5. The intelligent detection system for non-employee loitering and lingering in financial service areas as described in claim 1, characterized in that: In the aforementioned behavior analysis layer: The aforementioned loitering behavior detection module is used to detect loitering behavior of personnel, including: (1) Calculation of dwell time: Based on trajectory analysis, the dwell time of personnel in a specific area is calculated. When the dwell time exceeds the threshold, it is judged as a stay. (2) Identification of areas where people are staying: Identify the areas where people are staying and determine the severity of the stay based on the importance of the areas; (3) Distinguishing between employees and non-employees: Based on the identity recognition results, distinguish between employees on normal duty and non-employees loitering abnormally. Normal stay of employees in the duty area is not considered abnormal loitering; (4) Special handling during the clearing period: During the clearing period, all non-employee lingering will be judged as abnormal and an alarm will be triggered; The loitering behavior detection module is used to detect loitering behavior of personnel, including: (1) Trajectory pattern analysis: Analyze the spatial distribution pattern of personnel trajectory and identify circular, round-trip, and random wandering trajectory patterns; (2) Loitering area identification: Based on trajectory density analysis, identify the loitering area and loitering range of personnel; (3) Calculation of loitering time: Calculate the loitering time of personnel in the loitering area. When the loitering time exceeds the threshold, it is judged as abnormal loitering; (4) Employee patrol differentiation: Based on identity recognition and behavioral characteristics, distinguish between normal employee patrols and abnormal loitering by non-employees. Employee patrol behavior following a fixed route is not considered abnormal loitering; The aforementioned spatiotemporal behavior analysis module is used to comprehensively analyze personnel's behavioral patterns in the spatiotemporal dimension, including: (1) Spatiotemporal trajectory modeling: The trajectory of personnel is modeled as a spatiotemporal graph, where nodes represent spatial locations and edges represent temporal and spatial relationships, and spatiotemporal behavior patterns are analyzed; (2) Behavioral sequence analysis: Analyze the time series of personnel behavior to identify abnormal behavioral patterns; (3) Multi-person interaction analysis: Analyze the interaction behavior among multiple people to identify abnormal clustering and following behavior; (4) Historical behavior comparison: Compare the current behavior with the historical normal behavior pattern to identify behaviors that deviate from the normal pattern; The clearing period detection module is used to achieve intelligent detection of clearing periods, including: (1) Clearing time configuration: Supports configuring business rules for operating hours and clearing time, and automatically identifies whether the current time period is a clearing time period; (2) Detection strategy during clearing period: During the clearing period, a stricter detection strategy is adopted. All non-employee lingering and loitering are judged as abnormal and an alarm is triggered immediately. (3) Employee clearance confirmation: During the clearance period, a second confirmation is made for personnel identified as employees to ensure the accuracy of employee identification; (4) Clearance completion detection: Detect whether the clearing is completed. When no personnel are detected in the area or only confirmed employees are detected, the clearing is determined to be completed.
6. The intelligent detection system for non-employee loitering and lingering in financial service areas as described in claim 1, characterized in that: In the aforementioned decision output layer: The aforementioned abnormal event determination module is used to determine abnormal events by comprehensively considering the results of identity recognition and behavior analysis, including: (1) Determining the type of abnormal event: Determine the type of abnormal event; (2) Severity assessment of anomalies: The severity of the anomaly is assessed based on the type of anomaly, duration, and regional importance. (3) Multi-event correlation analysis: Analyze the correlation between multiple abnormal events and identify complex abnormal patterns; (4) False alarm filtering: Multi-frame verification and multi-view verification methods are used to filter false alarms and improve detection accuracy; The alarm generation module is used to generate abnormal alarm information, including: (1) Alarm information generation: Generate alarm records containing anomaly type, occurrence time, occurrence location, personnel information, and video clip information; (2) Alarm level setting: Set alarm levels and adopt different alarm methods according to the severity of the anomaly; (3) Alarm push: Push alarm information to relevant personnel, supporting multiple push methods; (4) Alarm record management: Save alarm records and support query, statistics and analysis functions; The result output module is used to output the detection results, including: (1) Real-time detection result output: Real-time output of personnel detection, identity recognition, and behavior analysis results for use by upper-level applications; (2) Visualization: Visualize the detection results, including personnel annotation, trajectory display, and highlighting of abnormal areas; (3) Data interface: Provides a standardized data interface to support integration with third-party systems; (4) Report generation: Generate detection and statistical reports, including abnormal event statistics, employee activity statistics, and regional safety assessment.
7. An intelligent detection method for non-employee loitering and lingering in financial service areas, characterized by: The system described in any one of claims 1-6 includes the following steps: Step 1: System Initialization and Configuration: After the system starts, initialize the video acquisition layer, feature extraction layer, identity recognition layer, behavior analysis layer, and decision output layer modules; load the employee feature library and establish the employee recognition model; configure the clearing time, detection threshold, alarm rules, and business parameters; initialize the multi-camera video stream connection and establish spatiotemporal synchronization; load the deep learning model, namely the target detection model, feature extraction model, and behavior analysis model. Step 2: Video Stream Acquisition and Preprocessing: The multi-camera video stream acquisition module acquires real-time video streams from multiple cameras in the financial services area; The video preprocessing module performs image enhancement, correction, and background modeling preprocessing on the video stream; it uses a multi-target detection algorithm to detect people targets in the video frames and outputs the people bounding boxes and confidence scores; the spatiotemporal synchronization module performs spatiotemporal synchronization on multiple video streams to achieve cross-camera target tracking. Step 3: Multi-dimensional feature extraction: The appearance feature extraction module extracts the clothing features, work permit features, and overall appearance features of the personnel; The behavior feature extraction module extracts the action features, interaction features, and posture features of personnel; the trajectory feature extraction module extracts personnel trajectories through multi-target tracking and analyzes trajectory patterns and motion features; the scene feature extraction module extracts contextual information such as scene type, time features, and environmental features. Step 4: Employee Identification: The multi-feature fusion employee identification module integrates appearance features, behavioral features, and trajectory features, and uses a deep learning model to identify employees and non-employees; the employee feature library management module provides employee feature retrieval and matching; the identity confidence assessment module calculates the confidence level of identity recognition, and triggers auxiliary judgment or manual review when the confidence level is low; Step 5: Behavior Analysis and Anomaly Detection: The loitering behavior detection module detects personnel loitering behavior based on trajectory analysis and distinguishes between normal employee duty and abnormal non-employee loitering by combining the identity recognition results; the wandering behavior detection module analyzes trajectory patterns to detect wandering behavior and distinguishes between normal employee inspection and abnormal non-employee wandering; the spatiotemporal behavior analysis module comprehensively analyzes personnel behavior patterns in the spatiotemporal dimension to identify abnormal behavior. The clearing period detection module determines whether the current period is a clearing period based on business rules, and adopts a stricter detection strategy during the clearing period; Step 6: Abnormal Event Judgment and Alarm: The abnormal event judgment module integrates the results of identity recognition and behavior analysis to determine the type and severity of abnormal events; the alarm generation module generates alarm information and pushes it to relevant personnel; The results output module outputs the detection results, providing visualization and data interfaces; The system continuously monitors and detects abnormal loitering and wandering behavior by non-employees in real time.
