Intrusion detection method and system for security intelligent access control

Abstract

The invention provides an intrusion detection method and system of a security intelligent access control, and relates to the technical field of artificial intelligence security. The method comprises the following steps: acquiring multi-mode state signals of an access control card, biological recognition verification, a door magnet, a door lock and the like, and judging whether an abnormal state exists or not; if the state is an abnormal state, combining a historical verification record and a state vector, and based on a time window feature extraction and integration discrimination model, identifying an unauthorized intrusion event; meanwhile, performing target detection and tracking on video data of the access control area, generating a target trajectory, analyzing behavior characteristics, and counting the number of effective targets and an abnormal time period; and fusing the access control abnormal time period and the video abnormal time period, and realizing intelligent identification and automatic alarm of suspicious intrusion behaviors through preset rule weight and time consistency judgment. Compared with the prior art, the method has the advantages that the judgment precision and the multi-target recognition capability of abnormal events in a complex intrusion scene are improved, and the intelligence and the safety protection level of the access control system are effectively enhanced.

Description

Technical Field

[0001] This disclosure belongs to the field of artificial intelligence security technology, specifically relating to an intrusion detection method and system for intelligent security access control. Background Technology

[0002] With the development of IoT and AI technologies, the integration of video surveillance and multimodal sensor data based on access control systems has become an important direction for modern intelligent security. Traditional access control systems typically rely on single authentication methods such as access cards, fingerprints, or facial recognition to achieve basic control over personnel entering and exiting. However, existing systems often suffer from limited anomaly detection capabilities, high false alarm rates, and a lack of multi-source information fusion and discrimination when dealing with complex scenarios such as unauthorized intrusion, abnormal door locks / magnetic sensors, and abnormal video behavior.

[0003] Some existing technologies attempt to combine video target detection and behavior analysis to achieve automatic early warning of abnormal events in gate areas. However, they often only analyze a single data stream, making it difficult to fully reflect the diversity and concealment of actual intrusion behaviors. In addition, traditional anomaly detection methods mostly rely on manually set threshold rules and lack adaptive feature extraction and cross-modal joint discrimination mechanisms driven by big data. This makes it difficult to effectively deal with complex scenarios such as multi-target, multi-behavior, and gang intrusions, resulting in insufficient intelligence and robustness of the system.

[0004] Therefore, there is an urgent need to propose a technical solution that can integrate access control equipment, door magnetic and door lock status, video data and related multimodal sensor information, and can automatically identify and accurately determine abnormal intrusion behavior based on an intelligent discrimination model, so as to improve the intelligence level and security protection capability of access control and security systems. Summary of the Invention

[0005] A method for intrusion detection in a smart security access control system is disclosed. A system also performs the functions of this method. The method includes associating multi-source status signals acquired by access control devices, door sensors, door locks, and video acquisition units with the personnel operating in front of the access control point; detecting one or more abnormal status events; confirming the personnel's identity based on access control authentication records and multimodal sensor information; and combining access control abnormal events with video target detection and behavior recognition results to achieve intelligent judgment and alarm for intrusion behavior. Through the above methods, this disclosure can significantly improve the accuracy of access control systems in identifying unauthorized intrusion behavior and enhance their intelligent early warning capabilities, while reducing the false alarm rate.

[0006] In a first aspect of this disclosure, a method for intrusion detection in a smart security access control system is provided, comprising: acquiring historical access cards or biometric verification records; determining, based on the collected card swipe and door magnetic sensor / lock status signals, whether there is an abnormal state where the card swipe fails but the door magnetic sensor responds while the lock does not; if an abnormal state is present, access control authentication fails; if the access control identification result is an abnormal state, further determining, based on historical access cards or biometric verification records, whether the abnormal state is an unauthorized intrusion event, and performing multi-feature integrated analysis and discrimination on the status feature samples within each time window; detecting and tracking targets based on acquired video data, determining the number of valid targets and several abnormal video time periods within the identified access control area; and, based on the multi-feature integrated analysis and discrimination results, the number of valid targets, and several abnormal video time periods, using a preset rule weight and time consistency judgment standard, multi-modal fusion to determine whether the event is a suspicious intrusion behavior.

[0007] In a second aspect of this disclosure, an intrusion detection system for intelligent access control is provided. The system includes: an access control data acquisition module configured to acquire historical access cards or biometric verification records, and to acquire verification results, door magnetic sensor status signals, and door lock status signals in real time via access control devices; a video acquisition and target detection module configured to acquire video data in real time, detect and track valid targets within the access control area, determine the number of targets, and extract several abnormal video time periods; an access control anomaly detection module configured to determine, based on the acquired card swipe and door magnetic sensor / lock status signals, whether there is an abnormal state where card swipe fails but door magnetic sensor status responds while door lock does not, and whether access control authentication has failed; and a multi-feature integrated analysis module configured to analyze results where the access control identification result indicates an abnormal state. The system further determines whether the abnormal state is an unauthorized intrusion event based on historical access cards or biometric verification records, and performs multi-feature integrated analysis and discrimination on the state feature samples within each time window; the multi-target tracking and identity association module is configured to detect and track targets based on video data obtained by the video acquisition and target detection module, and determine the number of valid targets in the identified access control area and several abnormal video time periods; the fusion judgment module is configured to determine whether the event is a suspicious intrusion behavior based on the multi-feature integrated analysis and discrimination results obtained by the multi-feature integrated analysis module, the number of valid targets obtained by the multi-target tracking and identity association module, and several abnormal video time periods, based on preset rule weights and time consistency judgment criteria.

[0008] It should be understood that the summary section is not intended to identify key features of the claimed subject matter or to limit the scope of the claimed subject matter. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0009] The present disclosure will now be described in more detail with reference to embodiments and the accompanying drawings. Wherein:

[0010] Figure 1 A schematic block diagram of one embodiment of a system 100 for intrusion detection of a security smart access control system that implements a method according to an embodiment of the present disclosure is shown.

[0011] Figure 2 A schematic block diagram of one embodiment of computing device 110 is shown.

[0012] Figure 3 A schematic block diagram of another embodiment of the computing device 110 is shown.

[0013] Figure 4 A schematic flowchart of an embodiment of an intrusion detection method 200 for a security smart access control system disclosed herein is shown.

[0014] Figure 5 A schematic flowchart of an embodiment of the method 300 for determining whether an abnormal state is an unauthorized intrusion event is shown.

[0015] Figure 6 The access control status signal S is shown. m (t) and its moving average The situation of changes in segments over time series.

[0016] Figure 7 This diagram illustrates a time tree structure where sample feature values ​​within a time window are divided layer by layer according to a preset threshold.

[0017] Figure 8A The temporal importance curves of different features of the door lock state vector within the 0-100 time window are shown.

[0018] Figure 8B The time importance curves of different features of the gate magnetic state vector within the time window of 0-100 are shown.

[0019] Figure 8C The temporal importance curves of different features of face recognition results within the 0-100 time window are shown.

[0020] Figure 9 A schematic flowchart of an embodiment of the method 400 disclosed herein for training an integrated analysis model capable of extracting features across time windows based on an interval sample set and historical labeled samples is shown.

[0021] Figure 10 The comparison between the method of this disclosure and existing technologies under different feature distributions and segmentation decisions is shown.

[0022] Figure 11 A flowchart of one embodiment of the method 500 of this disclosure for determining the number of valid targets within an access control area and several abnormal video time periods is shown.

[0023] Figure 12 A flowchart illustrating an embodiment of the method 600 for continuous association and consistency matching of target identities between time-series frames of this disclosure is shown.

[0024] Figure 13 A flowchart of an embodiment of the method 700 for determining the number, correlation and collaborative operation relationship between targets disclosed herein is shown.

[0025] Figure 14 This diagram illustrates the trajectory discrimination of the same person target identified in step 706 of method 700.

[0026] Figure 15 The diagram illustrates the trajectory discrimination process in step 708 of method 700, where two valid targets are identified.

[0027] Figure 16 The diagram illustrates the trajectory discrimination process in step 710 of method 700, where a valid target is identified as one.

[0028] Figure 17 The diagram illustrates the trajectory discrimination of two targets identified in step 712 of method 700.

[0029] Figure 18 A flowchart of an embodiment of the method 800 for determining whether an event is a suspicious intrusion behavior using multimodal fusion of the present disclosure is shown. Detailed Implementation

[0030] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this disclosure.

[0031] Those skilled in the art will understand that various aspects of the embodiments can be implemented as a system, method, or program product. Therefore, embodiments can take the form of entirely hardware embodiments, entirely software embodiments (including firmware, resident software, microcode, etc.), or embodiments combining software and hardware aspects, all of which can be collectively referred to herein as “circuit,” “module,” or “system.” Furthermore, embodiments can take the form of a program product implemented using one or more computer-readable storage devices that store machine-readable code, computer-readable code, and / or program code, hereinafter referred to as code. The storage devices may not embody signals. In one embodiment, the storage device uses signals only for accessing the code.

[0032] Throughout this specification, references to "an embodiment," "embodiment," or similar language mean that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment. Therefore, the phrases "in one embodiment," "in an embodiment," and similar language appearing throughout this specification may, but do not necessarily, refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless explicitly stated otherwise. The terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless explicitly stated otherwise. The enumerated list of items does not imply that any or all items are mutually exclusive, unless explicitly stated otherwise. The terms "a" and "the" also mean "one or more," unless explicitly stated otherwise.

[0033] Current intelligent access control systems typically deploy various sensors, such as door magnets, locks, and card readers, along with video surveillance equipment, to achieve identity verification and security control at entrances and exits. However, in actual use, because different types of sensors often operate independently, they cannot effectively coordinate to detect abnormal behavior in complex scenarios, such as a card swipe not opening the door but a door magnet responding, or a door lock not responding. This makes it easy for unauthorized personnel to bypass the normal procedures and enter, posing a significant security risk to the access control area.

[0034] Currently, many security access control systems rely solely on single sensor data or manual inspections combined with passive video recording review. They lack intelligent integrated analysis of multimodal data, resulting in low accuracy in identifying complex intrusion scenarios (such as coordinated attacks, repeated card swipes, and tailgating), high false alarm and missed alarm rates, and high costs and slow response times for manual inspections. Therefore, how to achieve intelligent linkage between multi-source sensors and video streams to improve the accuracy and real-time performance of intrusion detection has become a pressing technical problem for the industry.

[0035] To address the aforementioned issues, this disclosure provides an intrusion detection method for intelligent security access control, comprising: acquiring historical access cards or biometric verification records; determining, based on the collected card swiping and door magnetic sensor / lock status signals, whether there exists an abnormal state where card swiping fails but the door magnetic sensor responds while the lock does not; if an abnormal state exists, access control authentication fails; if the access control identification result is an abnormal state, further determining whether the abnormal state is an unauthorized intrusion event based on historical access cards or biometric verification records, and performing multi-feature integrated analysis and discrimination on the status feature samples within each time window; detecting and tracking targets based on acquired video data, determining the number of valid targets and several abnormal video time periods within the identified access control area; and, based on the multi-feature integrated analysis and discrimination results, the number of valid targets, and several abnormal video time periods, using a preset rule weight and time consistency judgment standard, multi-modal fusion to determine whether the event is a suspicious intrusion behavior.

[0036] Figure 1 A schematic block diagram of one embodiment of a system 100 for implementing an intrusion detection method for a security smart access control system according to embodiments of the present disclosure is shown. System 100 includes a computing device 110 and a data storage device 120 and a computer network device 116 connected thereto. The computing device 110 is connected to an access control data acquisition module 102 and a video acquisition and target detection module 104 for data communication, enabling real-time acquisition, processing, and storage of multimodal information about the access area.

[0037] like Figure 1 As shown, system 100 includes a computing device 110 and a data storage device 120 and a computer network device 116 connected thereto. The computing device 110 is connected to the access control data acquisition module 102 and the video acquisition and target detection module 104 for data communication, so as to realize the real-time acquisition, processing and storage of multimodal information of the access area.

[0038] Figure 2 A schematic block diagram of one embodiment of computing device 110 is shown. As the central processing node of system 100, computing device 110 is responsible for the real-time operation of multi-feature data fusion, state discrimination, and control logic. Computing device 110 may integrate an access control anomaly detection module 112 and a multi-feature integrated analysis module 114 (including access control anomaly event time period T). eThe system includes a multi-feature extraction and analysis module, a time-series statistics module, a model discrimination module, a multi-target tracking and identity association module 116 (including a Kalman filter KF and an appearance feature extraction ReID model (also known as an appearance feature extractor), an action detection module, etc.), and a fusion judgment module 118 (which uses a multimodal decision method based on multiple consistent judgment results). These functional modules can be physically integrated on the same hardware platform or deployed in a distributed manner according to actual applications. In some embodiments, the access control anomaly detection module 112 and the multi-feature integration analysis module 114 receive data from multiple time windows within the detection period collected by the access control data acquisition module 102 via a network cable interface or wireless communication. The detection results of the access control anomaly detection module 112 can be directly uploaded to the data storage device 120 via the computer network device 116, or further transmitted to the multi-feature integration analysis module 114. The corresponding steps of the multi-feature integration analysis module 114 can be executed independently after the access control anomaly detection module 112 receives data from the access control data acquisition module 102, or it can receive the preliminary judgment results of the access control anomaly detection module 112 (i.e., whether the access control is in an abnormal state or whether the identity verification has failed) and then perform multi-feature integration analysis. The analysis results of the multi-feature integration analysis module 114 and the analysis results of the multi-target tracking and identity association module 116 are further transmitted to the fusion judgment module 118 for multi-module decision analysis. In other embodiments, the access control anomaly detection module 112... Figure 3 A schematic block diagram of another embodiment of the computing device 110 is shown. In other embodiments, Figure 3 The computing device 110 also includes a system linkage module 120, which is configured to output an alarm signal or link the access control management system to respond accordingly when the fusion judgment module 118 determines that a suspicious intrusion behavior has occurred, thereby enabling communication with downstream alarm and control devices.

[0039] The access control data acquisition module 102 is configured to acquire historical access cards or biometric verification records, and to collect verification results, door magnetic status signals, and door lock status signals in real time through the access control device. The access control data acquisition module 102 can be configured to include various types of access control status and identity recognition related sensors. Specifically, these may include, but are not limited to, door magnetic status detection sensors (for sensing the open / closed state of the door), door lock status recognition sensors (for determining the actual response and action state of the door lock), access control card readers (IC cards, ID cards, NFC cards, etc.), infrared human body sensors (for identifying whether someone is approaching or lingering at the door area), and facial recognition terminals (for biometric authentication). All of the above sensors can interact with the computing device 110 via wired or wireless means to achieve real-time acquisition of all key status information within the access control area.

[0040] The video acquisition and target detection module 104 is configured to acquire video data in real time, detect and track valid targets within the access control area, determine the number of targets, and extract several abnormal video time periods (Tv). In some embodiments, the method for detecting and tracking valid targets within the access control area is a deep learning target detection method (which can be implemented by a deep learning target detector). The video acquisition and target detection module 104 may include an integrated implementation of various types of video image acquisition devices and deep learning target detection methods. The video acquisition device may be a fixed-point or panoramic camera, or a smart camera with night vision, infrared, or other functions. The target detection algorithm may be deployed as a lightweight real-time detection model (Det) such as YOLO or DETR, used to automatically detect and annotate bounding boxes of targets such as people, vehicles, and objects in the video stream acquired by the camera. The above modules can interact with the computing device 110 locally or in the cloud in real time to achieve high-speed uploading, processing, and archiving of target detection results.

[0041] Regarding the access control anomaly detection module 112, it is configured to determine, based on the collected card swipe and door magnetic sensor / lock status signals (obtained through the access control data acquisition module), whether there is an abnormal state where the card swipe fails but the door magnetic sensor responds while the lock does not, and whether access control authentication has failed; in some embodiments, the access control anomaly detection module 112 is also configured to identify and obtain several access control anomaly event time periods T. e ;

[0042] Regarding the multi-feature integrated analysis module 114, it is configured to further determine whether the abnormal state is an unauthorized intrusion event based on the result of the access control identification being an abnormal state, and to perform multi-feature integrated analysis and discrimination on the state feature samples within each time window;

[0043] Regarding the multi-target tracking and identity association module 116, it is configured to detect and track targets based on the video data obtained by the video acquisition and target detection module, and determine the number of valid targets in the access control area and several abnormal video time periods;

[0044] Regarding the fusion judgment module 118, it is configured to determine whether the event is a suspicious intrusion behavior based on the multi-feature integration analysis and discrimination results obtained by the multi-feature integration analysis module, the number of effective targets obtained by the multi-target tracking and identity association module, and several abnormal video time periods, and based on the preset rule weights and time consistency judgment criteria.

[0045] Data storage device 120 and computing device 110 are interconnected via computer network device 116, supporting centralized storage, indexing, and scheduling of multi-source data such as historical access control card swipes, video streams, detection results, and abnormal events. Computer network device 116 can be a local area network (LAN), wide area network (WAN), or dedicated Internet of Things (IoT) gateway, and can also achieve remote interconnection via 4G / 5G public networks, WIFI, etc.

[0046] In a preferred embodiment of this disclosure, system 100 further includes various external execution terminals that are linked and controlled with computing device 110, including:

[0047] The audible and visual alarm 130 is configured to emit a high-decibel alarm sound and / or a strong flashing light when an abnormal intrusion event is detected, to deter and alert on site.

[0048] The smart door lock controller 132 is configured to work in conjunction with the door lock actuator to perform remote locking / unlocking operations based on the judgment result, or to cooperate with intrusion detection to achieve security protection actions such as automatic deadbolt.

[0049] The gate area warning light 134 is used to flash brightly or change color in the access control area to alert on-site personnel to the presence of risks or special conditions at the gate area.

[0050] The 136 security host or the 136 cloud platform serves as a local or cloud-based security host for communities / enterprises, facilitating the real-time uploading of access control detection information to the security management backend, supporting unified management, remote dispatch, and alarm handling.

[0051] The remote linkage terminal 138 can be a mobile APP for administrators, SMS, telephone notification module, etc. It is configured to automatically push alarm information to preset remote contact methods when a high-risk intrusion event is detected, so as to realize real-time linkage, emergency response and remote monitoring.

[0052] The system 100, through the collaborative operation of the aforementioned multi-level hardware modules and data channels, achieves innovative functions such as dual-channel fusion of access control status and video targets, intelligent multi-modal feature discrimination, rapid response to abnormal behavior, and multi-scenario security linkage. The various functional modules interact with each other through standardized interfaces and protocols, enabling both local closed-loop processing and facilitating large-scale distributed deployment and intelligent cloud expansion.

[0053] Figure 4 A schematic flowchart of an embodiment of an intrusion detection method 200 for a security smart access control system disclosed herein is shown.

[0054] In step 202, historical access control card or biometric (such as facial recognition, fingerprint, etc.) verification records are acquired. Card swiping behavior, door magnetic sensor status signals, and door lock status signals are collected in real time through the access control device, and the collected access control data is jointly analyzed. If an abnormal state is detected, such as a failed card swipe but a door magnetic sensor status response (e.g., the door is opened) and a door lock failure (not properly unlocked), the system automatically determines that the current access control authentication has failed and proceeds to the subsequent anomaly detection process. This step ensures preliminary screening of routine access and abnormal operations.

[0055] In step 204, if the access control authentication result is an abnormal state, then further analysis is conducted based on historical access card records or biometric information, considering both the current abnormal state and historical data, to determine whether it is an unauthorized intrusion event. For sample data such as access control status signals within each time window, a multi-feature ensemble analysis model is used to extract and identify features (such as mean, variance, duration, and sequence change trends) to improve the detection capability for complex or concealed abnormal behaviors. This step can be used not only for conventional threshold determination but also for intelligent anomaly recognition in multimodal, big data environments.

[0056] In step 206, video stream data of the access control area is further acquired. A deep learning object detection algorithm is used to detect and track people or targets in the video stream, automatically determining the actual number of valid targets within the access control area. Based on temporal trajectories and behavioral characteristics, the system can identify and mark several abnormal video time periods (such as abnormal loitering, multi-person collaboration, abnormal actions, etc.) within the access control area. This step enhances multimodal collaborative perception and improves real-time response capabilities to different intrusion scenarios.

[0057] In step 208, based on the integrated analysis results of multiple features, the number of valid targets, and information such as abnormal video time periods, a multimodal fusion analysis method is used to comprehensively determine whether the current event constitutes a suspicious intrusion behavior, based on preset rule weights and time consistency judgment criteria. This fusion judgment process can take into account the correlation between access control data and video data, achieving a more accurate and robust intrusion detection effect, and providing a highly reliable decision-making basis for subsequent alarms and linkage control.

[0058] In some embodiments, a quick judgment can be made based on the response status of the door lock and door sensor within a preset time period (if the door lock remains unresponsive while the door sensor remains responsive throughout the preset time period, the abnormal state is marked as an unauthorized intrusion event). This method is simple to operate and suitable for most common scenarios. The preset time period can be set to 10 to 30 seconds. That is, if the door lock remains unresponsive while the door sensor remains responsive within this time period, the current abnormal state is determined to be an unauthorized intrusion event, thereby enhancing the accuracy of identifying typical tailgating, forced entry, and other similar behaviors.

[0059] In other embodiments, the preset time period can be dynamically adjusted according to the security level, access frequency of the access control area, or device response speed of the actual application scenario. For example, in places with high security levels, the time period can be shortened to 5 to 15 seconds to improve the system's response sensitivity to high-risk behaviors; while in places with high traffic or large device response delays, the time period can be appropriately extended to 30 seconds or longer to take into account actual needs such as false alarm rate and false triggering.

[0060] This disclosure is not limited to the specific time period settings mentioned above. The specific values ​​of the preset time period can be flexibly configured by the system administrator according to actual application needs, equipment parameters, and on-site environment, and can be achieved through parameter settings or remote adjustments to ensure that the access control system has good adaptability and versatility. This implementation method can fully combine abnormal access control behavior detection and video multi-target perception to adapt to the intelligent security needs in different scenarios. The above steps can be implemented independently according to actual needs, or they can be combined to build an end-to-end fully automated system, which has high adaptability and scalability.

[0061] In other embodiments, an integrated analysis model based on multiple feature curves can be used to achieve intelligent classification of abnormal states through interval sample modeling, feature importance analysis, and majority voting mechanisms. Figure 5 A schematic flowchart of an embodiment of the method 300 for determining whether an abnormal state is an unauthorized intrusion event is shown.

[0062] In step 302, the access control state vector data is divided into multiple time windows of fixed length, and the i-th time window T is defined. i

[0063] In some embodiments, T i =2s, sampling period Δt s =100ms; Obtain the real-time acquired door magnetic state vector value S m (t), door lock state vector value S l (t), forming the state vector set X Ti :

[0064] X Ti ={S m (t),S l (t)]∣t∈[t i,1 ,t i,n ]};T i =t i,n -t i,1 +1;

[0065] The real-time acquired vector values ​​may also include the face recognition result V(t), indicating whether the face recognition was successful or unsuccessful, with V(t) = 1 or 0. V(t) = 1 indicates successful face recognition, and V(t) = 0 indicates unsuccessful face recognition. It may also include infrared sensing values ​​H(t), which can be acquired using a camera or infrared sensor with infrared functionality. H(t) = 1 or 0, with H(t) = 1 indicating that someone is approaching and H(t) = 0 indicating that no one is approaching. Furthermore, it may include the type of person or action type S detected in the video recognition. cam (t), S cam (t) = 1 or 0, S cam (t) = 1 indicates that the video recognition system detected someone. S cam (t) = 0 indicates that no one was detected by video recognition; it can also include the number of access control card swipe attempts (or failures) G(t), where G(t) = 1 represents a failed attempt and G(t) = 0 represents a successful attempt; therefore, the state vector set X Ti It can also be:

[0066] X Ti ={S m (t),S l (t),V(t),H(t),S cam (t),G(t)]∣t∈[t i,1 ,t i,n ]};

[0067] It can also be a required S. m (t), S l In addition to (t), V(t), H(t), S cam Any combination of vectors in S(t) and / or R(t) and S m (t), S l (t) combine to form X Ti .

[0068] In step 304, the set of gate magnetic state vectors {S} is obtained. m (t i,1 ),…,S m (t),…,S m (t i,n )}, the set of door lock state vectors {S l (t i,1 ),…,S l (t),…,S l (t i,n )}, similar to the first, for the set of state vectors X Ti The j-th vector x jMultiple vector values ​​within this time window can be obtained. By summarizing and calculating all vector values ​​within this event window, x can be obtained. j Mean characteristics Standard deviation characteristics and the slope features of the multi-state vector within the time window T

[0069]

[0070] For the i-th time window T i Multi-state vector slope features In other words,

[0071]

[0072] in, The average over time. This represents the mean of the access control feature vector (i.e., the door lock feature vector or the door magnetic sensor feature vector) over the time window.

[0073]

[0074] For the set of state vectors X Ti The j-th vector x j The first characteristic value, namely the mean. In other words, it indicates that within the time window [t] i，1 ,t i，n The j-th vector x in the [] j The mean value represents the average behavior of the door sensor / lock state during that time period (e.g., average state of being closed / open). For the set of state vectors X Ti The j-th vector x j The second characteristic value, namely the standard deviation. In other words, the standard deviation indicates the degree of state fluctuation within a time window, measuring the dispersion of the state vector at a certain moment from the mean of that time window. A large standard deviation indicates frequent state fluctuations (such as repeated opening and closing, door shaking, etc.). For a set of state vectors X... Ti The j-th vector x j The second eigenvalue, namely the slope feature of the multi-state vector. In other words, it measures the trend of state change. Multi-state vector slope feature. For the state vector x j (t) within the time window [t] i,1 ,t i,n Within the range, fit a least-squares straight line with a slope of [missing information]. Treating time t as the horizontal axis, and the state value x j(t) (i.e., the state of the door lock or door sensor) is considered as the vertical axis, and (t, x) j Linear regression is performed on the points (t) to obtain a trend line. If the slope is... A continuous rise indicates that within the time window [t] i,1 ,t i,n The inner door gradually opens; if the slope A decrease indicates that within the time window [t] i,1 ,t i,n The inner door gradually closes; if the slope If there is no obvious change, it indicates that the door is not opening or closing.

[0075] Therefore, for the first eigenvalue of each vector It is used to monitor and measure the persistence of behavior within a time window (such as abnormally persistent opening), for the second feature value. Used to monitor and measure unstable behaviors such as repeated operations or repeated opening and closing of a door within a time window, for the third eigenvalue. To monitor and measure changes in state trends (on→off or off→on), monitoring the direction of state changes and the rate of acceleration of these changes is a precursor to abnormal and sudden changes.

[0076] For a fixed sampling duration (t) i,1 ,t i,n The time window T) i Sliding is achieved by defining multiple time windows T1, T2, ..., T over the entire time series data. M As shown in Table 1, each slide yields the average value of a time window.

[0077] Table 1

[0078]

[0079] Therefore, we can obtain the result from As with T1, T2, ..., T M The vertical coordinates corresponding to each coordinate on the horizontal axis are used to form the mean time curve.

[0080] The original sequence S of gate magnetic states over time m When (t) is a vector, Figure 6 The access control status signal S is shown. m (t) and its moving average As the time series changes segmented, the multiple bars above represent the original sequence S of the gate magnetic state over time. m(t) uses black rectangular bars of different heights to display the door sensor status values ​​(such as open / closed degree, numerical encoding, etc.) at various times. It can be regarded as a discrete time domain, reflecting the actual state changes of the door sensor over a period of time. The moving average below is... To correspond to the mean curve obtained by sliding the time window four times, the height of each rectangle represents the height of a rectangle within a window (as shown by the arrows T1, T2, T3, and T4 below, where the arrows indicate the gradual sliding process of the window on the time axis). This height represents the value of S within that sliding window. m The average value of (t).

[0081] Each time window corresponds to a set of interval samples, and the i-th time window T i The inner interval sample set F i :

[0082]

[0083] For the interval sample set F i The “…” indicates that each interval sample set also includes the face recognition result V(t), which indicates whether the face recognition was successful or unsuccessful, with V(t) = 1 or 0; it may also include the infrared sensing value H(t), which can be collected by a camera or infrared sensor with infrared function, with H(t) = 1 or 0; and it may also include the type of person or action type S detected in the video recognition. cam (t), S cam (t) = 1 or 0; it can also include the number of access control card swipe attempts or failures R(t), then the interval sample set F is formed within the i-th time window. i It can be expanded to:

[0084]

[0085] It can also include only V(t), H(t), and S cam F is formed by combining the f1 mean, f2 standard deviation, and f3 multi-state vector slope of any of the features in G(t) and / or G(t). i .

[0086] In step 306, this step will be based on the interval sample set F extracted in step 304. i By combining historical labeled samples, a cross-window feature extraction and ensemble analysis model with anomaly recognition capabilities is trained.

[0087] For each interval sample set F i There is a set of state vectors X Ti The j-th vector x j Combined with the corresponding tag y i∈{normal, unauthorized intrusion, accidental touch, ...}, for example, y i A cross-window feature extraction ensemble analysis model is constructed for each region ∈{0,1,2,3,…}, with each region's decision tree based on interval feature partitioning criteria. For label y i ∈{0,1,2,3,…},y i =0 means normal passage through the door; y i =1, representing unauthorized intrusion; y i =2 indicates a mis-touch or failed card swipe; y i =3 represents other custom anomalies (such as system tests or false alarms). Each training sample is explicitly assigned to one of the discrete categories (classification task).

[0088] Figure 7 This diagram illustrates a time-tree structure where sample feature values ​​within a time window are divided layer by layer according to preset thresholds. It demonstrates how samples in different sub-segments are classified based on their feature mean across multiple levels of nodes, resulting in labels such as "normal entry," "accidental touch," and "unauthorized intrusion." Figure 7 As shown, the tree is recursively constructed from top to bottom at each node, and the partitioning is determined by the following criteria:

[0089]

[0090] Samples that meet this condition will be assigned to the left child node; otherwise, they will be assigned to the right child node.

[0091] in, This represents the set of state vectors X. Ti The j-th vector x in j The k-th type of feature (e.g., mean, standard deviation, slope), k = 1, 2, 3, x j =S m (t),S l (t),V(t),H(t),S cam (t), R(t). That is, the corresponding set of state vectors X. Ti Each vector in the set X has three types of features. When classifying based on the region decision tree, for the set of state vectors X Ti Each vector in the algorithm needs to have its feature value of the k-th class compared with the classification threshold. Let x be the vector within the i-th time window. j The threshold for the k-th feature. This is a threshold that is only effective for the current feature (i.e., the threshold for each feature is set independently for each split; different feature vectors may have different thresholds, and different nodes for the same feature may also have different thresholds). This is achieved by combining the label y. i For each interval sample set F iThresholding of vectors in the model can fully utilize the dynamic changing trends of multimodal time domains such as access control, door magnets, and door locks, rather than relying solely on static state values ​​at a single moment; through label attribution, the model can learn discrimination criteria for historical abnormal patterns.

[0092] For label y i The partitioning strategy, which improves entropy gain by introducing minimum spacing, is provided by manual annotation (during initial training samples) or historical audit logs to improve the discriminative sensitivity of the region decision tree.

[0093] In step 308, to improve the ability to locate abnormal time periods, statistics are calculated for each feature type f. k Use the i-th time window T in all tree nodes i The cumulative entropy gain of multiple sub-segments is defined as:

[0094]

[0095] Among them; Imp ij,k (T i The sum of entropy gains of the k-th class features of the j-th vector in the i-th time window Ti at time point t represents the importance factor of the k-th class features in determining whether the access control system is in an abnormal state within the time window Ti, i.e., based on Imp at time t. ij,k (T i The size is used to select whether it is used as a vector x within the i-th time window. j The selection features, if for S m For the vector (t), if its first eigenvalue The importance factor for screening (mean) is greater than that for the second eigenvalue. (standard deviation) and the third characteristic value The importance factor for screening (slope) is... Both greater than Also greater than Then for the i-th time window, S m For the time-domain filtering of the vector (t), the first feature value (mean) is selected as the filtering feature; for the (i+1)th time window, S m (t), if Both greater than Also greater than Then S in the (i+1)th time window m For the time-domain filtering of the vector (t), the second eigenvalue is selected. (Standard deviation) is used as a screening feature.

[0096] The cumulative entropy gain at a certain point in time is used to determine the tree by summarizing the region. Define the time importance function Impij,k (t), the time importance function for different time windows shows the different trends of the corresponding time points in classification, intuitively revealing which time intervals are most useful for distinguishing categories. It can generate time importance curves for three feature types: mean, standard deviation, and slope, clearly revealing when the system is most sensitive to classification.

[0097] Figure 8A The time importance curves of different features of the door lock state vector within the time window of 0-100 are shown. Figure 8B The temporal importance curves of different features of the gate magnetic state vector within the time window of 0-100 are shown. Figure 8C The temporal importance curves of different features of face recognition results within the 0-100 time window are shown.

[0098] about Figure 8A The three characteristic curves (mean, standard deviation, and slope) fluctuate gently with low amplitude and consistent trend in most areas. However, from the 45th to the 55th time window, i.e., T... 45 ~T 55 Within the specified segment, the standard deviation and slope characteristic curve showed a significant and sharp increase and steep fluctuations, exceeding the mean curve and exhibiting drastic changes. Conversely, from the 10th to the 20th time window T... 10 ~T 20 Within this segment, the fluctuation amplitudes of all three types of characteristics remain low and stable. Therefore, it can be preliminarily determined that T represents a generally stable and small-amplitude characteristic curve. 10 ~T 20 The section represents the normal passage phase of the door lock, conforming to normal entry conditions. For T, where the standard deviation and slope curve show a sharp increase and significant fluctuations... 45 ~T 55 The segment containing abnormally violent operations or forced intrusion behavior (such as forced unlocking or abnormal switching) is classified as an unauthorized intrusion event.

[0099] about Figure 8B In T 45 ~T 55 Within the segment, the temporal importance of all three types of features showed a synchronous and significant increase or fluctuation. In particular, the standard deviation and slope curves indicated that the door magnet status underwent a short-term drastic change or abnormal persistence, reflecting an abnormal event (such as being left open for a long time or forced intrusion).

[0100] about Figure 8C Within a time window of 0-100, only in T 10 ~T 20 In this segment, the mean, standard deviation, and slope characteristics all fluctuate relatively little, and the differences between curves are not significant, indicating continuous and stable data. This segment is determined to be a normal passage segment for face recognition. In T...45 ~T 55 Within the segment, the standard deviation characteristic curve shows a significant spike, with the amplitude far exceeding the mean and slope, indicating severe abnormal fluctuations. The spike in the standard deviation curve reflects the presence of face recognition failures, accidental touches, or abnormal face recognition operations in this segment, which constitutes abnormal intrusion behavior.

[0101] pass Figure 8A and Figure 8C The characteristic curves in the normal range are stable. Figures 8A to 8C The abnormal segments exhibit abrupt changes and synchronous increases, and the joint discrimination of three modes and multiple features can effectively improve the sensitivity of anomaly detection. Figure 7 The results demonstrate the effectiveness of the method 300 according to the above embodiments of this disclosure in multi-level interval sample feature extraction and screening. It can effectively and automatically select the optimal threshold and accurately classify events such as "normal entry" and "unauthorized intrusion" based on the constructed integrated analysis model (regional judgment tree). Through threshold division of multi-temporal features and cumulative entropy gain discrimination, abnormal behavior sensitive detection and adaptive classification recognition of the access control system can be achieved, reducing false alarms and missed alarms, and improving system security and intelligence.

[0102] In step 310, the judgment results of each region judgment tree in the cross-window feature extraction ensemble analysis model composed of multiple region judgment trees are statistically analyzed. Each tree completes the screening based on the independent segment feature division and optimal discrimination threshold selection process in steps 302-308. After each tree makes a polarity-independent category judgment on the feature sub-segments of the input sample, the preliminary predicted label of the m-th tree is output.

[0103] Or 0, The result of the screening judgment for the m-th tree is that it enters normally. The filtering result for the m-th tree indicates an unauthorized intrusion. m = 1, 2, ..., M;

[0104] Therefore, for the i-th time window, there are a total of M region decision tree predicted labels:

[0105] The final output category is obtained by using a majority voting mechanism. Right now:

[0106]

[0107] This indicates that the call is made by M substructures (Tree1~Tree) M The integrated voting results consist of... The final classification output, the output result of the decision tree for the m-th region.

[0108]

[0109] If formula (11) is satisfied, it indicates that the number of prediction trees labeled as unauthorized intrusion (label value 1) exceeds half of the total number of trees, and the final decision is: And trigger the reporting of abnormal events; otherwise, the final co-predicted category voting result will be output as follows. This indicates normal entry. II(·) is an indicator function; it takes the value 1 if the condition is true, and 0 otherwise. This indicates that when hour, The value is also 1.

[0110] If M is odd, then the final judgment result only needs to satisfy the following conditions:

[0111]

[0112] The final judgment will then be output as unauthorized intrusion. Indicates to The calculation result is rounded up.

[0113] If M is even, to ensure a unique voting result, the final judgment condition only needs to be met:

[0114]

[0115] It should be noted that the total number of predicted labels M of the region judgment tree in step 310 is not related to the total number of time windows in Table 1. That is, the two can be equal or not. The method disclosed in this paper does not require the two to be equal.

[0116] Figure 9 A schematic flowchart of an embodiment of the method 400 disclosed herein for training an integrated analysis model capable of extracting features across time windows based on an interval sample set and historical labeled samples is shown.

[0117] To improve the model's sensitivity to abnormal segments (such as door magnets always open or door locks not responding), in step 402, the model is further refined. Divide the time window into segments to obtain a total of N. ij,k Each sub-segment, as a total of N ij,k There are training samples, each training sample corresponding to a label y. i Each has a specific label, and thus N ij,k Each training sample is assigned a different label, resulting in a total of C. ij,k For each class of samples, the endpoints of each segment are taken as candidate thresholds, theoretically extending the time window T. i The j-th vector x j The kth feature Divided into N ij,k If there are N consecutive sub-segments of samples, it means that the feature value can form N sub-segments. ij,k There are N scalars (e.g., mean, variance, slope), which, when sorted by value, result in N... ij,k -1 gaps can be used as candidate thresholds, forming κ = N ij,k -1 candidate split points, meaning the theoretical number of candidate thresholds is κ. Often, a fixed κ << N is selected randomly or at equal intervals. ij,k There are 10 candidate partitioning points, that is, the total number of partitioning segments κ is set to a fixed constant B to improve efficiency, for example, B = 3 or 5.

[0118] Therefore, the optimal threshold can be selected by calculating κ times. Avoid sorting operations; define X corresponding to the set of state vectors. T The optimal segment for the k-th class feature of vector x in the vector is S. * It meets the following conditions:

[0119]

[0120] In step 404, for N ij,k Each sub-segment sample is further processed within a time window T, based on the label assigned to each sub-segment sample. i The sample types in the sub-segments within the time frame are classified, and the i-th time window T is calculated. i The set of internal state vectors X Ti The j-th vector x in j The proportion of class c samples with feature k and time window T i The information entropy E of the parent node of the judgment tree for the corresponding i-th region p,ij,k :

[0121]

[0122] The set of state vectors X corresponding to the time window Ti of the i-th region's decision tree T The j-th vector x in j The classification result of the k-th feature is to form a common C ij,k Class c samples, the sample proportion of class c samples is Therefore, through N ij,k The division of the samples into sub-segments results in a set of sample proportions.

[0123]

[0124] N represents the number of samples labeled as category c within this time window (i.e., the number of samples corresponding to the c-th child node). ij,k This represents the total number of all training samples within the event window.

[0125] With the i-th time window T i Internal state vector set X Ti The second vector S in l The second feature of (t) Taking a sample size of 20 sub-segments (i.e., the standard deviation feature of the door lock state vector) as an example, the mean range is between 0.12 and 0.91. The 20 feature sets are {0.12, 0.18, 0.23, 0.27, 0.32, 0.34, 0.36, 0.39, 0.41, 0.45, 0.48, 0.52, 0.57, 0.61, 0.65, 0.69, 0.73, 0.78, 0.84, 0.91}, and this set serves as the parent node. The parent node is selected based on the label y1 = 0 (normal entry) and the first threshold. The partitioning results in a first child node (left subset) and a second child node (right subset). The first child node contains the feature value set {0.12, 0.18, 0.23, 0.27, 0.32, 0.34}, and the second child node contains the feature value set {0.36, 0.39, 0.41, 0.45, 0.48, 0.52, 0.57, 0.61, 0.65, 0.69, 0.73, 0.78, 0.84, 0.91}.

[0126] Then based on the second threshold The partitioning results in a third child node (left subset) and a fourth child node (right subset). The mean set included in the third child node is {0.36, 0.39, 0.41, 0.45, 0.48, 0.52}, and the mean set included in the fourth child node is {0.57, 0.61, 0.65, 0.69, 0.73, 0.78, 0.84, 0.91}. The third child node contains 6 means, and the fourth child node contains 8 means.

[0127] Then based on the third threshold The labels y2=1 (unauthorized intrusion) and y3=2 (accidental touch or failed card swipe) are used to partition the mean set within the fourth child node, resulting in a fifth child node (left subset) and a sixth child node (right subset). The mean set within the fifth child node is {0.57, 0.61}, and the mean set within the sixth child node is {0.65, 0.69, 0.73, 0.78, 0.84, 0.91}. The fifth child node contains two means, and its mean set is obtained based on the label y2=1. The sixth child node contains six means, and its mean set is obtained based on the label y3=2.

[0128] Therefore, based on the above sub-segment division, 12 sub-segment samples are labeled y1=0 (normal entry), with a proportion of γ1=12 / 20=0.60; 6 sub-segment samples are labeled y2=1 (unauthorized intrusion), with a proportion of γ2=6 / 20=0.30; and 2 sub-segment samples are labeled y3=2 (accidental touch or card swipe failure), with a proportion of γ3=2 / 20=0.10. Therefore, for time window T... i The information entropy E of the parent node of the judgment tree for the corresponding i-th region p,i2,2 :

[0129] E p,i2,2 ≈1.2955.

[0130] In step 406, the entropy gain of the parent node and multiple child nodes is further calculated.

[0131] Each sub-segment sample corresponds to a partitioning threshold, in the i-th time window T i It contains several candidate thresholds The j-th state vector set X is sequentially processed through each candidate threshold (the b-th threshold, b = 1, 2, ..., B). T The j-th vector x in j The k-th feature is progressively divided into two subsets: left subset D L,ij,k and right subset D R,ij,k Each subset corresponds to a child node; the left subset D L,ij,k satisfy Right subset D R,ij,k satisfy For the initial time window T i The corresponding parent node has N as its left subset D L,ij,k With right subset D R,ij,k The sum of the number of samples in the left subset D L,ij,k The number of samples is n L,ij,kThe entropy is E L,ij,k Right subset D R,ij,k The number of samples is n R,ij,k The entropy is E R,ij,k :

[0132]

[0133] in, For the b-th threshold The entropy value of the current parent node in the partition (which is the entropy value of the left subset node or the right subset node in the (b-1)th threshold partition), when b is 1, The initial parent node information entropy, also known as E, is given by E. p,ij,k ; For the b-th threshold The total number of samples in the current parent node of the partition; To apply the b-th threshold to the current parent node The entropy value of the left subset obtained by partitioning, To apply the b-th threshold to the current parent node The number of samples in the left subset obtained by partitioning; To apply the b-th threshold to the current parent node The entropy value of the right subset obtained by partitioning, To apply the b-th threshold to the current parent node The number of samples in the right subset obtained by partitioning.

[0134]

[0135] in, For the b-th threshold The total number of samples in the current parent node of the partition. Right now It is a subset of the total samples corresponding to the parent node; Each threshold is applied to the current parent node. The resulting left subset sample set and right subset sample set.

[0136] In step 408, all candidate thresholds are evaluated based on entropy gain to improve the uniqueness and clarity of segmentation boundaries; in some embodiments, if the entropy gain of all candidate thresholds... If the values ​​are not equal, and an effective division can be achieved based on the threshold corresponding to the maximum entropy gain among all candidate thresholds, then in step 410, the region judgment tree can be divided solely based on the entropy gain.

[0137] Continuing with the example of the aforementioned 20 sub-segments, let's illustrate... The calculation method, if it passes the first threshold The partitioning results in 6 samples for the first child node (left subset) and 14 samples for the second child node (right subset). Therefore, the entropy of the left subset after the first threshold partitioning is... Right subset entropy The calculations are as follows:

[0138]

[0139]

[0140] Passing the second threshold The entropies of the third child node (left subset) and the fourth child node (right subset) obtained from the partition are respectively

[0141]

[0142] The parent node of the third and fourth child nodes is the second child node, therefore, Second threshold The entropy gain obtained from the partition is:

[0143]

[0144] Passing the third threshold The entropy of the fifth child node (left subset) and the sixth child node (right subset) obtained from the partitioning are respectively

[0145]

[0146] The parent node of the fifth and sixth child nodes is the fourth child node, therefore, The third threshold The entropy gain obtained from the partition is:

[0147]

[0148] Among the three thresholds, the second threshold (0.53) has the largest entropy gain (0.9852), which is significantly higher than the first (0.2813) and the third (0.8113), indicating that the second threshold of 0.53 is the best among the three candidate splitting thresholds.

[0149] Therefore, dividing the initial 20 samples using a single threshold (0.53) yields a left subset of 12 samples with values ​​less than or equal to the threshold of 0.53, all belonging to the "normal entry" category, with no accidental touches or intrusions. The right subset consists of 8 samples with feature values ​​greater than the threshold of 0.53, including 2 "accidental touches" and 6 "unauthorized intrusions".

[0150] Through maximum entropy gain Find the corresponding b-th threshold. As the optimal threshold This division ensures that the left subset is completely pure and requires no further division; the right subset mixes accidental touches and unauthorized intrusion categories, successfully separating normal entry from unauthorized intrusion categories, and excluding accidental touches from the normal entry category. represent

[0151] In some embodiments of method 400, when evaluating all candidate thresholds in step 408 based on the entropy gain of each candidate threshold (to filter and obtain the optimal threshold that can accurately divide the decision tree of the region where the parent node is located), if the entropy gains calculated by multiple candidate thresholds are equal or approximately equal, the division results of the two thresholds are different, and it is impossible to determine which threshold to choose as the optimal threshold, a minimum distance term Margin is introduced. ij,k Used to break up tiebreakers with the same entropy gain:

[0152]

[0153] Margin ij,k For the threshold of division The minimum distance to the sample value. That is, the sample value. With each division threshold Distance calculation results The minimum distance in the middle is Margin ij,k .

[0154] Furthermore, in step 408 of some other embodiments, the state vector set X of the i-th time window is obtained. Ti The j-th vector x in j The final segmentation evaluation index E of the k-th feature ij,k :

[0155]

[0156] Where α∈[1,5], it is only used to break a tie; this approach Margin with α-weight correction ij,k The method of calculating evaluation indicators by addition can prioritize the selection of clearer dividing boundaries. Weighting the margin with α is used. ij,k Weighting means making the margin ij,k They only participate in decision-making when the entropy gain is exactly the same.

[0157] Furthermore, in step 410 of some other embodiments, selecting the optimal segmentation threshold and generating segmentation conditions occurs during the completion of the i-th time window T in step 408 of this embodiment. i The j-th vector x j k-th type interval features (Right now (t1,t n The entropy gain ΔE (abbreviated as ) ij,k Margin with minimum distance ij,k Joint evaluation index E ij,k After calculation, the threshold for dividing all κ candidate nodes is determined. The corresponding evaluation index {E ij,k (1),E ij,k (2),...,E ij,k (κ)}, perform maximum value selection:

[0158]

[0159] Where b is the b-th candidate threshold, b = 1, 2, ..., B; the calculation result E is obtained by solving multiple candidate thresholds. ij,k (1),E ij,k (2),...,E ij,k The maximum value in (k) is used as the optimal partitioning threshold. Then, by using the optimal partitioning threshold The region decision tree is divided using the left and right subset partitioning criteria.

[0160] Soon As the optimal partitioning rule for the current region's decision tree node, the samples in the i-th time window are divided into left and right subsets according to this rule: if the feature value is less than or equal to a threshold, the sample is assigned to the left subset; otherwise, it is assigned to the right subset. In some embodiments, since each threshold... After partitioning, the calculated entropy gains may be the same or approximately the same, making it impossible to determine the optimal threshold. To effectively divide multiple samples, a margin is needed. ij,k Used to break up tiebreakers with the same entropy gain.

[0161] Figure 10 The comparative effects of the disclosed method and existing technologies under different feature distributions and segmentation decisions are shown, wherein Figure 10 (a) shows the original sample distribution. Figure 10 (b) The sample partitioning of the classification decision boundary after introducing the Margin joint evaluation index. Figure 10 (c) The region decision tree obtained by the Gini coefficient method divides the sample into decision boundaries.

[0162] Figure 10 In (a), the horizontal axis represents the index T of multiple sliding time windows. i The ordinate represents a certain characteristic value (such as linear slope). The red × represents normal category samples, and the blue × represents abnormal intrusion category samples. It can be seen that the two types of samples are significantly mixed in the feature space, with no clear linear boundary. Figure 10 (b) Add a minimum distance term Margin to the method disclosed herein. ij,k The decision boundary is determined by combining entropy gain and margin. Traditional methods fail to uniquely select the boundary point when the entropy gains of multiple candidate thresholds (cutoff points) are similar or even completely equal. This disclosure introduces the minimum distance term Margin. ij,k (i.e., the distance between the split point and the nearest sample), it tends to choose split points that are farther away from the samples (to avoid overfitting caused by splitting too close to the samples), thus obtaining a more robust and generalized boundary line. Entropy gain ΔE ij,k The calculation depends on the sample distribution throughout the time window. The weighting of the Margin term at different dividing points is a linear superposition. However, for the entire boundary curve, the optimal threshold for each time point adaptively changes with the feature distribution, and is not globally linear, thus forming a non-linear, curvilinear segmentation boundary. E is individually selected at each window position. ij,k (b) The maximum threshold point, the sliding of the window, and the non-uniformity of the sample distribution together cause the overall decision boundary to appear as a curve. Figure 10 (c) The region decision tree partitioning decision boundary is obtained using the Gini coefficient method, which means it cannot effectively partition the different categories in the original sample. It cannot maximize the entropy gain or add the minimum distance term Margin. ij,k The optimal threshold obtained by correcting the entropy-gain balance is used to effectively classify different categories of samples in the original sample.

[0163] Finally, the dividing index E is used to divide multiple time windows T. i The j-th vector x within a total of I time windows j Thresholding is performed on the feature values ​​of the k-th category, i.e., steps 402-410 are repeated three times for a total of I time windows for thresholding and cumulative entropy gain, which can complete the thresholding of the j-th vector x within the I time windows. j mean Standard deviation and the slope of the multi-state vector Threshold partitioning; for each vector x j By performing the above steps, the entire interval sample set F within a total of I time windows can be processed. i Threshold partitioning of the total set F.

[0164] In the process of building the cross-window feature extraction ensemble analysis model in Method 400, the samples are simply divided (into the left and right subtrees), which is equivalent to sample path splitting. Each child node still retains the complete feature vector of the original sample (such as the mean, standard deviation, slope, etc. of the door magnetic state). The division is just to route the samples into the child nodes for subsequent further division, rather than pruning the values ​​in the original samples.

[0165] For example, suppose the classification criteria are In this case, samples that meet the condition are placed in the left subtree during subtree construction, while the rest are placed in the right subtree—they all retain their original values ​​f1, f2, f3, etc., for the next level of partitioning.

[0166] When the card swipe fails, the door sensor remains high, and the door lock does not respond in time, the interval features collected by the model (such as high door sensor mean and slope close to 0) are likely to generate dividing points in multiple intervals.

[0167] The dividing boundary obtained by method 400 can distinguish the categories and is a point that is sufficiently far away from the observed abnormal samples, thus avoiding misclassification of boundary samples (such as occasional gate magnetic fluctuations, interference, etc.) and improving the robustness and sensitivity of the discrimination of the abnormal continuous segment.

[0168] Figure 11 A flowchart of one embodiment of the method 500 of this disclosure for determining the number of valid targets within an access control area and several abnormal video time periods is shown.

[0169] In Method 500, a deep learning-driven multi-target detection and tracking algorithm is used for the video stream in the access control area to accurately identify, associate, and determine the number of video targets appearing in front of the door.

[0170] In step 502, an advanced deep learning object detector (such as YOLOX, DETR, or a lightweight version thereof) is applied to the acquired video frame sequence to extract the two-dimensional bounding boxes of all candidate objects in each frame, resulting in a set of two-dimensional bounding boxes, Boud. t :

[0171] n t =1,2,…,N t .

[0172] t represents the time index or frame number of the currently processed video frame, t = 1, 2, ..., T; T is the total number of frames, for example, t = 5 represents the 5th frame. The relationship between frame and time (in seconds) is... FPS stands for video frame rate, measured in frames per second. Nt represents the total number of targets successfully detected in frame t. The number of targets may vary in different frames, therefore N...t It changes over time; for example, if two people are detected in front of the door in frame 5, then N5 = 2.

[0173] Let represent the two-dimensional bounding box of the nth candidate target in frame t. These are the bounding boxes. The coordinates of the top left corner or center point (depending on the model definition);

[0174] These are the bounding boxes. Width and height; The confidence score output by the object detection model represents the probability or confidence that the image within the bounding box is a person. For example: This indicates that the position of the second detection box in frame 5 is (320, 170), with a width of 60 pixels, a height of 140 pixels, and a confidence level of 0.93.

[0175] In step 504, based on the detection results, a cascaded association method (such as ByteTrack, BoT-SORT, Deep SORT, etc.) under the "detection-tracking separation" paradigm is used to perform identity consistency association on targets between time-series frames.

[0176] A cost matrix for target association is set, taking into account motion information (such as IoU and Kalman prediction) and appearance features (such as the cosine distance of Re-ID vectors), and the Hungarian algorithm is used to solve for the optimal allocation.

[0177] In step 506, a unique trajectory ID is generated for each continuously tracked target in time, and the p-th trajectory is τ. p A total of P trajectories form a trajectory set.

[0178]

[0179] Wherein, the p-th trajectory: Let the trajectory be τ p The target's bounding box sequence in each frame t (which may originate from a detection box in that frame (and the trajectory τ) p (Successful match) or it could be a trajectory prediction box (generated by the predictor when no target is detected in the frame, and it does not match the trajectory in frame t). For the p-th trajectory τ p The set of frames that have appeared represents the p-th trajectory τ. p The set of frames that have been validly encountered or predicted, i.e. It is not necessarily the nth detection box in frame t. But it may originate from it. That is, τ. p It does not traverse all N t Instead of a single frame, it records the set of frames that successfully match the trajectory p; if a frame does not match, that frame may be missing from the trajectory, or the Kalman filter prediction box may be retained as a record. In some embodiments, the corresponding trajectory ID is ID. p .

[0180] In multi-target tracking tasks, the trajectory ID refers to a unique identifier assigned to a continuously tracked target. It is usually represented by an integer number (e.g., 1, 2, 3...) to indicate the identity of the same target in consecutive video frames within the system. This number remains consistent over time. For example, if a person is detected with the number

[02] in frame 5, then if the target still exists in frames 6 and 7 and is successfully matched, it will remain as

[02] .

[0181] Suppose two people are continuously tracked in the video stream obtained from the door camera or the camera built into the smart lock:

[0182] Frame 5: The system detects two targets, creates trajectories τ1 and τ2, and assigns trajectory IDs ID1 and ID2 respectively;

[0183] Frame 6: The system detects two more people, the detection box and the trajectory are successfully matched, and the trajectories τ1 and τ2 are still maintained;

[0184] Frame 7: If one person leaves and the other remains, only update trajectory τ1, and the trajectory ID corresponding to trajectory τ1 remains unchanged as ID1.

[0185] If a new target enters, a new trajectory ID, such as ID3, is assigned to the new target.

[0186] In step 508, for the i-th time window T i Its length is W i The frame identifies the consistency of identity, cooperative operation relationship, or path separation behavior among the targets based on the degree of trajectory overlap, appearance feature similarity, and action behavior characteristics of the several targets. Specifically, it further determines the correlation between one or more identified targets, and then determines the number of targets and their trajectories within a certain time period.

[0187] Output the i-th time window T i (W i Number of valid targets within frame length And for each trajectory τ p Assign independent trajectory IDs and operation time periods;

[0188] If the number of effective targets If multiple targets participate in the operation in front of the door at the same time (i.e., there are any two trajectories that meet the judgment of multi-person collaborative operation), then it constitutes the basis for a suspected gang intrusion warning (when the subsequent fusion judgment module 118 performs step 208 (in some embodiments, it is to perform the subsequent method 800 provided in this disclosure) to perform fusion judgment, it serves as the basis for a suspected gang intrusion warning). For the i-th time window T i The number of valid targets determined after screening is determined by step 508 (in some embodiments, method 700 provided later in this disclosure);

[0189] If only one target continues to operate (potentially within a time window or a period formed by splicing together several consecutive time windows), or if multiple identified targets show a high degree of consistency (i.e., if a single person's trajectory continues to operate or a single person's trajectory reappears after being broken), then it constitutes suspicious repeated card swiping or multiple door breaking. All judgment results, trajectory IDs, operation types, and time periods are packaged and transmitted to the fusion judgment module 118.

[0190] Method 500 in this embodiment of the present disclosure decouples detection and tracking in the field of MOT (Multi-Object Tracking) and combines it with methods for appearance feature fusion and temporal trajectory consistency determination. It can effectively solve the problems that traditional access control video cannot accurately determine the number of targets, identity changes, and collaborative intrusions by relying solely on frame-level detection. It supports identity splitting and merging criteria in complex scenarios such as occlusion and passing by, which greatly improves the accuracy of security event discrimination. It supports quantitative statistics on the number of self-targets, operation time, trajectory distribution, etc., providing high-quality basic data for subsequent fusion judgment and intelligent alarm.

[0191] By introducing deep learning-driven multi-target detection and discrete tracking algorithms, this method achieves high-precision identification and dynamic identity association of all targets appearing in front of the door in the video stream of the access control area, effectively improving the accuracy of target quantity and identity determination in complex scenarios. By combining advanced target detection models (such as YOLOX and DETR) with efficient multi-target tracking algorithms (such as ByteTrack and Deep SORT), it can continuously and accurately assign a unique trajectory ID to each target, enabling precise monitoring of behaviors such as multi-person collaboration, target dispersion, and merging. Furthermore, through temporal trajectory analysis and operational relationship determination, the method supports real-time identification and hierarchical early warning of complex behaviors such as gang intrusion, multi-target collaboration, and repetitive single-person operations, significantly enhancing the intelligence level and practical protection capabilities of the access control system. In addition, this method possesses good occlusion robustness and trajectory coherence, effectively distinguishing target identities in scenarios of occlusion, separation, and merging, greatly reducing false alarms and false negatives, and providing a high-quality, structured data foundation for subsequent multimodal fusion and intelligent alarms.

[0192] Figure 12 A flowchart illustrating an embodiment of the method 600 for continuous association and consistency matching of target identities between time-series frames of this disclosure is shown.

[0193] In step 602, the detection bounding boxes for each frame of the video stream obtained in step 502 are... Extracting appearance feature vectors from targets within the target area (Extracted using the ReID model, typically a 128 or 256-dimensional vector), and further processed using a Kalman filter (KF) to analyze the trajectories in the historical trajectory set (all trajectories in the historical trajectory set are either in an "active" state or a "lost but not invalid" state). p Perform Kalman prediction to obtain the predicted bounding box of the p-th trajectory in frame t. The predicted position vector of the p-th trajectory is given by the Kalman filter KF, which predicts the center position and size, and is also a 4-dimensional column vector.

[0194] In step 604, the detection box of each frame image is taken. The first four vectors As a computation vector, the detection bounding box of each frame image is calculated. The predicted bounding box of the p-th trajectory in the t-th frame Trajectory association matching distance d mot (n t ,p):

[0195]

[0196] in The predicted location of the historical trajectory p (derived from KF's prediction of past trajectories); for The transpose of S p,t This is the state covariance matrix between the detection boxes and the trajectory prediction boxes, with dimensions of 4×4. For S p,t The inverse matrix;

[0197] Calculate d mot (a, b) is actually used to determine the detection box of the nth candidate target in frame t. Is it possible that the current new detection result belongs to trajectory p, that is, the trajectory p of the current new detection result? Compared with existing trajectory prediction results Perform association matching.

[0198] In some embodiments, It can also be simplified to

[0199] The detection box for the nth candidate target in frame t. The spatial overlap index between the trajectory of the p-th trajectory and the trajectory of the t-th frame.

[0200] In step 606, the appearance matching distance between the detected target and the trajectory is calculated:

[0201]

[0202] E q The historical appearance features of the p-th trajectory (usually the most recent frames or a weighted average vector);

[0203] Appearance features With E q Cosine similarity;

[0204]

[0205] in, For calculation and The vector dot product, |||2: represents the Euclidean norm (i.e., L2 norm) of the independent variable vector; the cosine similarity ranges from [-1, 1], but after L2 normalization, the value of the RelD vector usually falls in [0, 1], where 1 represents complete similarity and 0 represents complete dissimilarity.

[0206] In step 608, the matching cost matrix C(n) is constructed. t b):

[0207] C(n t,p)=λ·d mot (n t ,p)+(1-λ)·d app (n t ,p) (27)

[0208] Wherein, λ∈[0,1] is used to adjust the relative contribution of motion and appearance information; in some embodiments, λ=0.6~0.8;

[0209] In other embodiments, the specific value of λ is selected by the developer based on the characteristics of the task. When the environmental complexity increases (poor lighting, more occlusion), the value of λ is appropriately reduced in the matching cost matrix C(n). t The value of λ in (b) is adjusted to enhance the stability of ReID features; when the target movement is intense and there is little occlusion, the value of λ in the matching cost matrix C(n) is appropriately increased. t The value of λ in (b) can be adjusted to improve motion constraint capability; in some other embodiments, the matching cost matrix C(n) can also be adjusted as needed. t In b), λ is designed as a function λ(t) that is adjusted based on environmental metrics or frame signal-to-noise ratio (e.g., dynamic strategies in deep reinforcement learning).

[0210] In step 610, the Hungarian Algorithm is executed to perform the optimal allocation.

[0211] Based on the matching cost matrix C(n) t The Hungarian algorithm is called to perform minimum weight matching, resulting in three types of results:

[0212] First type of result: The detection box of the nth candidate target in frame t. The predicted bounding box of the p-th trajectory in the t-th frame Matching; multiple matches result in detection boxes with corresponding trajectory prediction boxes, and the targets corresponding to these boxes form the first type of target set;

[0213] Second type of result: Targets corresponding to detection boxes that did not match the corresponding trajectory form a second type of target set;

[0214] The third type of result: The trajectories of the unmatched detection boxes form a set of unmatched trajectories.

[0215] It should be noted that the matching cost matrix C(n) t b) represents each detection box The matching cost between each trajectory p is determined by the Hungarian algorithm, which finds the one-to-one assignment with the minimum total cost among all possible matching combinations.

[0216]

[0217] Where Q is a set of unique matching pairs (for each detection box). The corresponding n t The p-th trajectory can only be assigned once, that is, (n t ,p)∈Q represents each detection box in set Q. The p-th trajectory is matched only once.

[0218] The core process of the Hungarian algorithm is as follows:

[0219] 1) Matching cost matrix C(n) t Convert p) to a two-dimensional array supported by the Hungarian algorithm (such as n×m, which usually needs to be converted to a square matrix and padded with zeros);

[0220] 2) Applying the Hungarian algorithm:

[0221] Perform a row / column subtraction operation to construct a zero-cost operation;

[0222] Construct a zero-element graph;

[0223] Find the maximum matching coverage;

[0224] If the matching is not completed, adjust the cost matrix and continue iterating;

[0225] 3) Obtain a set of optimal matching pairs (n) with the minimum total cost. t ,p).

[0226] 4) Output Result Classification

[0227] A. Matching pair set Q: Detection target: The nth candidate target in frame t. t Successfully associated with the p-th trajectory τ p ;

[0228] B. Set of unmatched detected targets U det U det The target in the target will be used for new trajectory initialization;

[0229] C. Set of unmatched trajectories U trk For set U trk The trajectory will determine whether it is "lost" or enters the deletion stage.

[0230] In step 612, the trajectory state of the current t-th frame image is updated.

[0231] For the trajectory in the first type of target set, use the detection box of the nth candidate target in the t-th frame. Trajectory prediction box in Kalman filter KF Update the position of the nth candidate target in frame t by adding the detection box. appearance features Add a buffer to the trajectory, complete the historical data update of the ReID appearance feature extractor, keep the trajectory active and clear the lost counter of the trajectory; in some embodiments, the matching pair set Q of the A class output in step 4) of the above Hungarian algorithm is the first class classification target set;

[0232] For targets in the second target set, increment the original value of the lost counter for that trajectory by 1, and then check whether the result after incrementing by 1 exceeds the maximum number of unmatched frames threshold T. lost If so, then delete the target corresponding to the detection box that did not match the corresponding trajectory in the second type of target set; in some embodiments, the unmatched detection target set U of the B type output in step 4) of the above Hungarian algorithm is... det That is, the set of second-class classification targets;

[0233] For a set of unmatched trajectories, retain the predicted bounding box of that trajectory. and its position; increment the lost counter for this trajectory by 1, if it exceeds T lost If the value is not exceeded, it is marked as "deleted"; if it does not exceed the threshold, the trajectory is retained for matching in the next frame; the trajectory still retains its appearance feature E. p This continues to be used as a historical appearance embedding feature for association in frame t+1. In some embodiments, the set of unmatched trajectories U output by class C in step 4) of the Hungarian algorithm described above... trk This refers to the set of unmatched trajectories.

[0234] The target corresponding to the trajectory for which no matching detection box was found is taken as the new target, and a new trajectory is initialized. The features E of the trajectory for which no matching detection box was found are then used. p The predicted bounding box of the p-th trajectory in frame t that did not match the corresponding detection box. Feature E p Save the results for future use as historical results of appearance feature extraction and for association with the detection bounding boxes in frame t+1.

[0235] Each execution of steps 602-612 completes the multi-target tracking of the current frame. There is no iteration process until convergence is achieved and the iteration ends. Steps 602-612 are executed again for the (t+1)th frame to update the historical data once. In other words, the repeated execution of steps 602-612 performs an update loop for different frames, updating only once.

[0236] This invention, Method 600, establishes a video target detection and identity tracking method for multiple targets by introducing multi-information fusion of Kalman filter prediction and deep ReID appearance feature extraction, combined with adaptive weights for motion information and appearance features. This method utilizes a Kalman filter to improve the spatiotemporal continuity and occlusion robustness of target trajectories, leverages high-dimensional ReID features to effectively distinguish between similar-looking or cross-frame targets, and employs a joint matching cost matrix and the Hungarian algorithm to achieve globally optimal target-trajectory allocation, significantly reducing the probability of mismatch, missed detection, and identity drift. The system can dynamically adjust the motion and appearance weights λ to adapt to environmental changes and occlusion situations in different access control scenarios, and can maintain real-time trajectory state updates and data buffers, supporting target addition, loss, and recovery. Compared with traditional single motion or single appearance association methods, this scheme significantly improves the accuracy and robustness of target identity tracking in complex scenarios, and is particularly suitable for practical access control applications such as occlusion, multi-person collaboration, and frequent entry and exit, providing a high-quality continuous trajectory and identity data foundation for subsequent intelligent recognition of abnormal behavior.

[0237] This invention, Method 600, proposes a globally optimal allocation strategy that weights and fuses motion and appearance features in the field of multi-target access control video identity tracking. It organically combines covariance-based Kalman trajectory prediction with appearance matching of deep ReID feature vectors. Through adaptive matching weights and a dynamically adjustable association mechanism, it achieves consistent maintenance of target identities under adverse factors such as environmental changes, lighting interference, and short-term occlusion. The Hungarian algorithm is used to optimally solve the joint cost matrix, effectively avoiding identity mismatch and trajectory loss problems caused by local greedy matching.

[0238] Figure 13 A flowchart of an embodiment of the method 700 for determining the number, correlation and collaborative operation relationship between targets disclosed herein is shown.

[0239] In the method 700 disclosed herein, in order to accurately assess the number and identity independence of targets operating in front of the gate, a multi-trajectory statistical and spatial / appearance joint correlation determination method based on time windows is adopted.

[0240] In step 702, for the i-th time window T i Each trajectory within the window is filtered to determine if it is a valid trajectory. If the following two conditions are met, it is recorded as a valid trajectory: Condition 1: The trajectory exists continuously for ≥θ frames within the window. dur Condition 2: Confidence of the associated detection box θ dur In some embodiments, θ is used as a threshold for determining the number of frames the trajectory continues to run. dur The frame rate is 3 to 10 frames.

[0241] Finally, count the number of trajectories within the window that meet the above conditions, and denot them as follows: Used to determine how many main operational targets exist in front of the door.

[0242] Condition two means that within the analysis time window, the confidence score of each detection bounding box corresponding to the p-th trajectory. (confidence score) is not lower than the preset detection confidence threshold det τ In other words, all valid detection boxes within the trajectory are high-confidence targets, excluding trajectories formed by low-quality or false detections, thus ensuring the validity and reliability of the trajectory.

[0243] In step 704, a trajectory identity association degree Assoc(p,q) calculation model is constructed as the basis for determining the p-th trajectory τ. p With the q-th trajectory τ q The criteria for determining whether something belongs to the same physical target in a time series.

[0244]

[0245] Where q = 1, 2, ..., P. Represents the p-th trajectory τ p The q-th trajectory τ q They appear simultaneously in the same frame, and the p-th trajectory τ p With the q-th trajectory τ q Spatial overlap degree IoU pq ≥IoU min The p-th trajectory τ p With the q-th trajectory τ q Sim pq ≥Sim τ The set of frames; sim pq =cos(F p E q );

[0246] Represents the p-th trajectory τ p It appears but is related to the q-th trajectory τ q Spatial overlap degree IoU pq and the similarity of their appearance features (Sim) pq The above conditions are not met (i.e., IoU) pq <IoU min And Sim pq <Sim τ IoU pq ≥IoU min And Sim pq <Sim τ , or IoUpq <IoU min And Sim pq ≥Sim τ Any of the cases in the p-th trajectory τ p The set of frames (in the case of occurrence), or the p-th trajectory τ p The q-th trajectory τ appears q A set of frames that does not exist;

[0247] This indicates that the q-th trajectory has τ. q But with the p-th trajectory τ q Spatial overlap degree IoU pq and the similarity of their appearance features (Sim) pq The above conditions are not met (i.e., IoU) pq <IoU min And Sim pq <Sim τ IoU pq ≥IoU min And Sim pq <Sim τ , or IoU pq <IoU min And Sim pq ≥Sim τ Any of the cases in the equation and the q-th trajectory τ q (The situation that occurs), or the q-th trajectory τ q The p-th trajectory τ appears p A set of frames that does not exist.

[0248] IoU min The IoU is the bounding box spatial overlap threshold. In some embodiments, it is... min The value is 0.3–0.5; Sim τ Sim is the appearance cosine similarity threshold. τ =0.5~0.7.

[0249] In step 706, the p-th trajectory τ is determined. p With the q-th trajectory τ q Does it simultaneously satisfy the condition that the number of intersecting frames θ within a time period composed of several time windows is equal? pq ≥θ overlap IoU pq ≥IoU min Sim pq ≥Sim τ And Assoc(p,q)≥β same If both conditions are met, then the p-th trajectory τ is determined. p With the q-th trajectory τ qTrajectories belonging to the same target indicate that the target has reappeared after being separated from or occluded along its path, thus determining the number of valid targets. The value is 1. Path separation refers to the situation where the trajectory of the same actual target in consecutive video frames is split into two or more discontinuous trajectories by the system due to reasons such as occlusion, detection failure, or association interruption. In other words, the trajectory that should originally belong to the same person is identified as two different trajectories τ during the initial identification. p τ q This disclosure enhances the stability and accuracy of continuous multi-target tracking by setting a threshold for the number of intersection frames, spatial IoU, and appearance similarity criteria to identify path separation and process trajectory merging. same To determine the minimum correlation ratio threshold for identifying the target of the same person in each frame of a video stream, β same It ranges from 0.6 to 0.65;

[0250] The `card()` function is used to count the cardinality of a set, that is, to count the number of elements in the set; θ overlap The threshold number of trajectory intersection frames set for the system, θ overlap The time frame ranges from 3 to 6 frames. For a time period consisting of several time windows, each window has a fixed frame length or is a time period determined by motion detection events. Multiple windows can be stitched together to form continuous or discontinuous analysis time intervals, used to enhance the accuracy of statistical analysis and judgment of overlapping behaviors and trajectory co-occurrence relationships between targets. It should be noted that the statistical time period in step S3 is the same.

[0251] In step 708, the p-th trajectory τ is determined. p With the q-th trajectory τ q Does the trajectory identity correlation degree Assoe(p,q) < β simultaneously satisfy the condition that the correlation degree between trajectory identities within a time period consisting of several time windows is less than β? diff If two trajectories are active simultaneously and partially overlap in the operating area, and the action detection module identifies that the two people's actions are inconsistent (e.g., one person bends over, the other pulls a door), then the number of valid targets detected is determined. The value is 2. Furthermore, it was determined that the two individuals engaged in collaborative activity. β diff To determine the maximum correlation ratio threshold for identifying two targets as distinct targets in each frame of a video stream, β diff The value is 0.35–0.4, β same +β diff =1.

[0252] The condition for determining that two trajectories partially overlap within the operating region space is that the IoU (Interval in U+) is satisfied. pq ≥IoU min And Sim pq <Sim minThe video acquisition unit is used to acquire real-time image sequences of the gate area. The action recognition module is connected to the video acquisition unit and is used to analyze the acquired video stream, extract key action features of the target, and output behavior labels, which are used in conjunction with trajectory information to determine intrusion behavior.

[0253] In step 710, the p-th trajectory τ is determined. p In the i-th time window T i Frame length W i The number of consecutive frames less than θ dur And with the main trajectory τ main The number of overlapping frames is very small, and at the same time, it is in close contact with the main trajectory τ. main The trajectory identity correlation degree Assoc(p,main) < β pass Then determine the p-th trajectory τ p The target being identified was a brief passerby and does not constitute a valid intrusion participant; this is related to the number of valid targets. It has no impact. β pass β is the threshold for the maximum similarity between passing targets. pass =0.2. θ dur θ is the threshold for determining the duration of a trajectory in frames. dur = 5 to 10 frames, or 5 to 10 time frames.

[0254] Main trajectory τ main Refers to the i-th time window T used for analysis i Within the system, the target trajectory with the most persistent frames and the highest correlation confidence is typically obtained by filtering the valid frames of all trajectories within the statistical window and the detection box confidence. The main trajectory generally corresponds to the target that performs the main operation or stays for the longest time in front of the door, and serves as the reference benchmark for the system to distinguish between passing targets and core intrusion targets.

[0255] The method 700 in this embodiment further includes a determination and identification step for determining whether the newly appearing trajectory and the disappeared trajectory are the same target when a trajectory in the video stream disappears and then a new trajectory appears:

[0256] In step 712, if the p-th trajectory τ p It disappears at some point, and a new u-th trajectory τ u If the trajectories appear in similar locations and within the same time period, calculate the spatial overlap (IoU) between the spatial location of the u-th trajectory in the t-th frame where it appears and the last frame of the p-th trajectory. pu and appearance feature similarity Sim pu (That is, each frame in which the u-th trajectory appears performs an IoU operation with the last frame of the p-th trajectory.) pu Similarity to appearance features (sim) pu(calculation of the p-th trajectory τ), and then continue to use the judgment criteria and trajectory identity correlation formula from step 704 to calculate the p-th trajectory τ. p With the u-th trajectory τ u Trajectory identity correlation degree Assoc(p,u):

[0257]

[0258] For card That is, statistics The number of cardinalities in the set; For card That is, statistics The number of cardinalities in the set; Let p be the set of frames in which the p-th trajectory appears; Let u be the set of frames in which the u-th trajectory appears;

[0259] Determine if Assoc(p,u)>β merge Whether it is true or not, and at the same time determine the u-th trajectory τ u The time of occurrence and the p-th trajectory τ p The time interval Δt ≤ θ when the time disappears switch If so, it is determined to be a trajectory break and merging, β merge β is the threshold for determining trajectory breakage and merging. merge The value is 0.4 to 0.55; where u≠p and u≠p, θ switch θ is the threshold for trajectory breakage interval. switch For 3 to 4 frames, for example in a 30fps video, θ switch =3 frames ≈ 0.1 seconds. In some other embodiments, step 712 can be executed independently of steps 706, 708, and / or 710 to handle situations where the trajectory is temporarily interrupted and regenerated due to occlusion, frame loss, etc. When a trajectory disappears at a certain moment, a new trajectory appears in a similar spatiotemporal region, and the correlation between the two exceeds a preset threshold β. merge When this happens, the system can automatically merge the two into the same target trajectory, restoring trajectory continuity. Step 712 is adapted to the determination in method 700 of whether the disappeared trajectory and the newly appeared trajectory are repeated operations by the same person in the scenario of non-simultaneous trajectory interruption.

[0260] Figure 14 This diagram illustrates the trajectory discrimination of the same person target identified in step 706 of method 700. Figure 14 The blue trajectory represents trajectory 1 formed by target 1, which appears between time frames 7 and 15. The orange trajectory represents trajectory 2 formed by target 2, which exists continuously from time frame 7 to time frame 13 and disappears from time frame 14 to time frame 20. A threshold θ is set for the number of trajectory intersection frames.overlap The number of intersection frames θ between trajectory 1 and trajectory 2 from time frame 1 to time frame 20 is 3 frames. pq Satisfying θ pq >θ overlap By analyzing the spatial overlap (IoU) of trajectory 1 and trajectory 2 at each time frame... 12 Sim, appearance feature similarity pq Calculations were performed to determine the spatial overlap (IoU) between trajectory 1 and trajectory 2 within the light blue highlighted area (i.e., time frames 8 to 13). 12 Always satisfy IoU 12 ≥IoU min (Setting IoU) min The similarity of appearance features is 0.5, and the similarity is Sim 12 ≥Sim τ (Set to 0.55), and satisfying Assoc(1,2)≥β same ,β same Set to 0.65. Figure 14 The trajectory identity correlation degree is calculated as follows:

[0261]

[0262] (Total number of time frames for trajectory 1 or trajectory 2 within the light blue highlighted area). Although both trajectory 2 and trajectory 1 appear within time frame 7, the spatial overlap (IoU) between trajectory 1 and trajectory 2 is significant. 12 and the similarity of their appearance features (Sim) 12 All are less than the corresponding threshold, therefore Similarly (That is, at time frames 14 and 15, trajectory 1 appeared but trajectory 2 did not appear), therefore, Assoc(1,2)>β same ,therefore, Figure 14 Trajectory 1 and trajectory 2 both belong to the same target. Figure 14 The corresponding number of effective targets

[0263] Figure 15 The diagram illustrates the trajectory discrimination process in step 708 of method 700, where two valid targets are identified. Figure 15 The spatial overlap and appearance feature similarity of each time frame are labeled at the corresponding time frame location. Figure 15 The blue trajectory remains the motion trajectory formed by target 1, and the orange trajectory remains the motion trajectory formed by target 2. At the 6th time frame and the 13th time frame, trajectories 1 and 2 coexist in the doorway operation area and partially overlap, meaning they satisfy the IoU within the yellow highlighted area. 12≥0.3, Sim 12 <0.6, Figure 15 The trajectory identity correlation degree is calculated as follows:

[0264]

[0265] (Total number of time frames for trajectory 1 or trajectory 2 within the yellow highlighted area). Although both trajectory 2 and trajectory 1 appear within the area of ​​time frames 1 to 6, the spatial overlap (IoU) between trajectory 1 and trajectory 2 is relatively small. 12 and the similarity of their appearance features (Sim) 12 All are less than the corresponding threshold; the same is true in the region of time frames 14–20, therefore, Therefore, Assoc(1,2)<β diff ,β diff Set to 0.4, therefore Figure 15 The situation shown satisfies the recognition result of step S344, which is one of two cases for the target. Figure 15 The corresponding number of effective targets

[0266] Figure 16 The diagram illustrates the trajectory discrimination process in step 710 of method 700, where a valid target is identified as one. The second trajectory, represented by the orange trajectory (trajectory 2), appears for four consecutive frames (time frames 6 to 9), which is less than the trajectory duration threshold θ. dur Furthermore, the spatial overlap IoU between trajectory 2 and trajectory 1 is only observed within time frames 6 to 9. 12 and the similarity of their appearance features (Sim) 12 All are less than the corresponding threshold (marked in red at each time frame), and neither satisfies the above conditions at the remaining time frames. Trajectory 1 appears consistently from time frame 1 to time frame 20; therefore, trajectory 1 is taken as the main trajectory τ. main ,therefore, (The total number of time frames for trajectory 1 or trajectory 2 within the green highlighted area), therefore, Figure 16 The trajectory identity correlation degree is calculated as follows:

[0267]

[0268] Set the maximum similarity threshold β for passing targets pass =0.2, which satisfies the condition, Assoc(2,main) = β pass Therefore, trajectory 2 is determined to be merely a brief passage and does not constitute a valid intruder. Figure 16 The corresponding number of effective targets

[0269] Figure 17 The diagram illustrates the trajectory discrimination of two targets identified in step 712 of method 700. Figure 17 Trajectory 1 appears from time frame 1 to time frame 8, and trajectory 2 appears from time frame 11 to time frame 20. The time interval Δt between the time frame when trajectory 1 disappears and the time frame when trajectory 2 first appears is 3 frames, satisfying Δt ≤ θ. switch Furthermore, the spatial overlap and appearance similarity of each frame from time frame 11 to time frame 2 (where trajectory 2 appears) with the last frame of trajectory 1 are compared to obtain the IoU. 12 With Sim 12 All are marked in the figure, satisfying IoU. 12 <IoU min And Sim 12 <Sim τ , Set the threshold β for determining trajectory breakage and merging. merge If the value is 0.45, then Assoc(1,2) > β. merge Combined with the condition that Δt≤θ switch As a result, therefore, Figure 17 The trajectory 1 and trajectory 2 shown represent the same target disappearing in time frame 8 and then reappearing in time frame 11, indicating a trajectory break-merge. Therefore, it can be determined that... Figure 17 This refers to the situation where the same person performs repeated operations in a scenario where the trajectory is interrupted at different times.

[0270] Figures 14-17 The vertical axis of the graph represents spatial location, used to reflect the relative positional change trend of the target in the video stream in the vertical direction or a defined axis.

[0271] Although the ordinate varies within the range of [1.0, 2.4], the spatial location of the trajectory represented by the ordinate is not directly involved in the numerical calculation of trajectory identity association degree Assoc(p,q), spatial location overlap degree IoU, or appearance similarity degree Sim. However, it provides the relative positional evolution of the target trajectory, which helps to intuitively show whether there is a significant path intersection or similar movement trend between two targets. It can also be used as the spatial location overlap degree IoU. pq The geometric reference is used. The way this coordinate is represented in the simulation image depends on the target's dynamic behavior in the actual video frame, such as whether it moves away from or towards the camera, or whether it moves parallel to the camera. However, it does not interfere with the numerical calculation process itself. In short: the spatial position of the trajectory as the vertical axis represents the relative position of two target trajectories, used to visualize the relative motion relationship between the two targets, but it does not affect the mathematical logic of the calculation formula.

[0272] Figure 18A flowchart of an embodiment of the method 800 for determining whether an event is a suspicious intrusion behavior using multimodal fusion of the present disclosure is shown.

[0273] In step 802, for the time period T to be detected ab =[t a ,t b ], determine the time period T of several access control abnormal events obtained from the identification e Compared with the identified several abnormal video time periods T v All are related to the time period T to be detected ab Perform intersection verification when a certain access control abnormal event occurs within time period T. e With the time period T to be detected ab satisfy Meanwhile, during a certain abnormal video period T v With the time period T to be detected ab Satisfied and satisfied Then it is assumed that the detection period T ab There are time-consistent access control anomaly events and video anomaly events within the memory; the above judgment should be made for all identified T... e With T v Perform intersection verification one by one. Represents the empty set. This indicates that during a certain access control anomaly event period T... e With the time period T to be detected ab There are overlapping time periods. The meaning is similar and will not be elaborated upon here.

[0274] Access control anomaly event T e It is the recognition result obtained from the time importance curve generated by method 300, such as door magnet / door lock / card swipe attempt status / face / video action type, etc., within its own time range (e.g., from the i-th time window T). i Up to the h-th time window T h Then the access control abnormal event period T e =t h,n -t i,1 +1),

[0275] According to the definition in Method 300, T i =t i,n -t i,1 +1, then T h =t h,n -t h,1 +1, therefore, from the i-th time window T i Up to the h-th time window T h The duration range T of the identified access control anomaly events e (or time period) is t h,n-t i,1 +1.

[0276] For the abnormal video period Tv, following the previous method 500, the duration of the video consisting of each frame in the abnormal video stream from frame t to a certain frame can be detected. The abnormal video period can then be obtained by converting the video frame to time using FPS.

[0277] In step 804, the two-dimensional bounding boxes of all identified valid targets are calculated and compared with the entrance restricted area during the detection period T. ab =[t a ,t b The visible overlap rate ρ within the restricted area s If for a single valid target there exists Then it is determined that the valid target has spatial consistency. The threshold for spatial consistency judgment.

[0278] For the trajectory corresponding to the identified target, taking the p-th trajectory as an example, its two-dimensional bounding box is: Determine that the restricted area frame in the video is R, which corresponds to the designated restricted area at the entrance. Then it is considered that the p-th trajectory at time t coincides with the access control, and the statistics of T are calculated. ab =[t a ,t b The visible overlap rate of restricted areas within the specified time period:

[0279]

[0280] in, It is an existential quantifier, indicating that at least one exists; The meaning is to calculate within the set of valid trajectories. The p-th trajectory τ p During the detection period T ab =[t a ,t b Within the total duration of the time frame t that appears, The meaning is in Under the constraints of the effective trajectory set There exists at least one trajectory within the current time period T: the p-th trajectory; therefore, the statistics for the current time period T are as follows: ab Within the window, the number of frames t where at least one trajectory's bounding box intersects with the access control restricted area box R is calculated, then divided by the total number of frames t in the window. b -t a +1, and the resulting ratio is the visible overlap rate of the restricted area, ρ. s .

[0281] In step 806, it is determined that the valid target is within the detection time period T. abThe total duration T of abnormal intrusion behavior imp-peak That is, the time importance curve obtained in step S2 is used in the detection period T. ab The duration of occurrence of the characteristic value, i.e., the peak value of the characteristic value appears from the i-th time window to the r-th time window, then T imp-peak =t r,n -t i,1 +1, and further calculate the video abnormal period T. v The system calculates the ratio of overlapping peak values ​​in the characteristic states of the access control anomaly vector, and determines whether this ratio is greater than the behavioral anomaly consistency judgment threshold η. If it is, the system is ultimately judged to meet the behavioral anomaly consistency requirement. The anomaly consistency judgment threshold η is generally between 0.65 and 0.7.

[0282]

[0283] The duration() function is used to calculate the duration of a given time period. imp-peak ∩T v The duration (T) represents the time interval during which the target appears in the restricted area and its status becomes abnormal. v This refers to the duration of the abnormal period in the video. For T... imp-peak and T e In other words, The former is an important sub-period with high temporal weight in the latter.

[0284] For a valid target (the identification result of step 706, step 710, or step 712 in method 700), T v This refers to the set of all consecutive time periods in a video stream where a target exhibits abnormal behavior; of course, a video stream may contain multiple abnormal video periods T. v To handle several access control anomaly events during time periods T e Compared with the identified several abnormal video time periods T v Verification was carried out for each time period to be tested.

[0285] In the case of multiple targets (the identification results of step 708 in method 700), T v This refers to the time segment corresponding to the total duration of abnormal actions occurring in the video stream for the two identified targets. Specifically, it involves extracting the abnormal video segment for each target separately. Take the union of all target abnormal time periods to ensure that the abnormal behavior of everyone is included in the total T. v . S represents the total number of targets, and s represents the s-th target.

[0286] T e This refers to the time period during which the event itself occurs. For example: the card swipe failure time period is from the 10th to the 12th second; the door sensor opening time period is from the 15th to the 23rd second; the door lock not responding time period is from the 14th to the 18th second. (T can be...) e This can be understood as the event itself, i.e., when the door sensor changes and when the door lock does not respond.

[0287] T imp-peak T is the time period when the event's abnormal characteristics are significant. imp-peak Based on Te, a time importance curve is formed by combining a time window Ti with sliding analysis and using time characteristic quantities (such as the average value of the door magnet and variance). Then, the peak segments in the curve are identified, and the time set composed of these peak segments is called T. imp-peak T can be used imp-peak It can be understood as those crucial moments when things happen particularly violently or unusually.

[0288] In step 808, for the time period T to be detected ab Determine whether one or more valid targets simultaneously satisfy temporal consistency, spatial consistency, and behavioral anomaly consistency. If so, determine that the target is within the detection period T. ab A suspicious intrusion by one or more people occurred inside.

[0289] This disclosure enables deep fusion of multi-source access control data and video behavioral information. It can not only promptly detect unauthorized intrusions such as unresponsive door locks but abnormally opened door sensors, but also automatically identify suspicious intrusion events in complex scenarios by combining target detection and behavioral trajectory analysis. Compared to traditional methods relying solely on single access control or video information, this disclosure significantly improves the intelligent discrimination capability against abnormal events such as group intrusions and obstructed passage. Furthermore, by introducing multi-dimensional cross-validation based on temporal consistency, spatial consistency, and behavioral anomaly consistency, it significantly reduces the probability of false alarms and missed alarms, effectively enhancing the practicality and security level of the access control system.

[0290] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0291] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical applications, or technical improvements to the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. An intrusion detection method for a security intelligent access control system, characterized in that, include: Obtain historical access control cards or biometric verification records. Based on the collected card swipe and door magnetic sensor / lock status signals, determine whether there is an abnormal state where the card swipe fails but the door magnetic sensor responds and the lock does not respond. If there is an abnormal state, the access control authentication fails. If the access control identification result is abnormal, further determine whether the abnormal state is an unauthorized intrusion event based on historical access cards or biometric verification records, and perform multi-feature integrated analysis and discrimination on the state feature samples in each time window; Based on the acquired video data, the target is detected and tracked, and the number of valid targets and several abnormal video periods within the access control area is determined. Based on the results of multi-feature integrated analysis and discrimination, the number of valid targets, and several abnormal video time periods, multimodal fusion is used to determine whether the event is a suspicious intrusion behavior, based on preset rule weights and time consistency judgment criteria.

2. The method according to claim 1, characterized in that, The criterion for determining whether an abnormal state is an unauthorized intrusion event is that if the door lock remains unresponsive while the door sensor remains responsive for a preset time period, the abnormal state is marked as an unauthorized intrusion event.

3. The method according to claim 1, characterized in that, The steps to determine whether an abnormal status is an unauthorized intrusion event also include: The access control status vector data is divided into multiple fixed-length time windows, and the door magnetic state, door lock state and related multimodal sensor status vector sets are collected and constructed within each time window; For each state vector within a time window, calculate the various feature parameters corresponding to each state vector to form a multi-feature sample set for the corresponding interval; Based on interval sample sets and historical labeled samples, an integrated analysis model capable of extracting features across time windows is trained to achieve intelligent identification of abnormal patterns. The cumulative entropy gain of each feature within different time windows is statistically analyzed to generate feature time importance curves, and key time segments that distinguish between normal and abnormal states are selected. The judgment results of multiple regional judgment trees are merged by majority vote, and the final output is the classification judgment result of normal passage or unauthorized intrusion event in this time window.

4. The method according to claim 3, characterized in that, The ensemble analysis model trained based on interval sample sets and historical labeled samples, capable of extracting features across time windows, includes: The feature data within the time window is subdivided into multiple sub-segments of training samples, and multiple candidate segmentation thresholds are set based on different category labels for subsequent selection of the optimal dividing point; For each sub-segment, the proportion of training samples by label type is statistically analyzed, and the category proportion of each feature and the information entropy of the parent node are calculated based on the sub-segment distribution to measure the feature's ability to distinguish categories. For each candidate splitting threshold, calculate the information entropy and entropy gain of the parent node and each child node, and evaluate the effect of sample purity improvement after splitting in turn. All candidate thresholds are evaluated based on entropy gain to improve the uniqueness and clarity of segmentation boundaries; The threshold with the highest evaluation index is selected as the optimal partitioning threshold for the current node. The best partitioning rule for the feature is determined, and the sample distinction within the time window is completed, providing an accurate partitioning basis for the growth of the region judgment tree.

5. The method according to claim 4, characterized in that, In the step of training an integrated analysis model capable of extracting features across time windows based on interval sample sets and historical labeled samples, when evaluating all candidate thresholds, in the case of multiple divisions with equal or approximately equal entropy gains, a minimum distance term is introduced as an auxiliary criterion, prioritizing the selection of boundary points farther from the sample distribution to enhance robustness.

6. The method according to claim 1, characterized in that, The step of detecting and tracking targets based on the acquired video data, and determining the number of valid targets within the access control area and several abnormal video time periods, includes: Deep learning object detection methods are applied to the video stream of the access control area to achieve efficient identification and boundary localization of target objects; Continuously associate and match the target identities across time frames; Assign a unique identifier to each continuously tracked target and generate a corresponding time-series trajectory record; Based on trajectory overlap, appearance features, and action behavior, the number, correlation, and collaborative operation relationship between targets are determined; Output the number of valid targets and their behavioral characteristics within each time window, which can be used for subsequent automatic alarm linkage analysis of different types of intrusion events.

7. The method according to claim 6, characterized in that, The continuous association and consistency matching of target identities across time frames includes: Extract the appearance features of the detected targets in each frame of the video stream and use a motion prediction model to estimate the position of each historical trajectory; Based on the positional relationship between the detection box and the predicted trajectory, the motion correlation degree is calculated to assist in the determination of target attribution; The similarity between the appearance features of the detected target and the historical trajectory is measured to achieve appearance association matching; By fusing motion and appearance information, a matching cost matrix is ​​constructed to weigh the correlation contributions of different features; Perform a globally optimal match between the target and the trajectory to achieve a consistent association between the old and new target identities; The status of each trajectory is updated in real time based on the matching results, enabling the addition, maintenance, and handling of lost targets.

8. The method according to claim 6, characterized in that, The determination of the number, correlation, and collaborative relationship between targets includes: Based on the continuity of trajectories within the time window and the detection confidence, the number of valid target trajectories is screened and counted. Establish an identity correlation model among multiple trajectories to accurately determine whether targets belong to the same physical individual; Based on the number of overlapping frames, spatial location, and appearance similarity, the system identifies and merges the separated trajectories of the same target caused by occlusion and other reasons, thereby correcting the number of effective targets. By analyzing trajectory identity correlation, spatial behavior and action differences, the system can determine the independence and collaborative relationships among multiple targets, thereby improving the identification of abnormal behaviors such as gang intrusion. Trajectories that pass by briefly and have low correlation with the main trajectory are removed to avoid interference from irrelevant targets in determining the number of abnormal events; Cross-temporal identity clustering and fragmentation merging are performed on disappearing and newly emerging trajectories to improve trajectory coherence and the accuracy of multi-target recognition in complex scenarios.

9. The method according to claim 1, characterized in that, The multimodal fusion method for determining whether an event constitutes a suspicious intrusion includes: By performing temporal domain intersection verification on the time periods of access control abnormal events and video abnormal events, the temporal consistency of abnormal events can be determined. Statistical analysis of the visual overlap rate of targets in the restricted area at the entrance, to determine the spatial consistency between the target and the access control area; Based on the overlap ratio between the feature peaks of the access control state vector and the abnormal time periods in the video, the consistency of the target's behavior is determined to be abnormal. Based on the combined judgment results of temporal consistency, spatial consistency, and behavioral anomaly consistency, the final judgment result of a suspected intrusion event involving one or more persons is output.

10. An intrusion detection system for intelligent security access control, characterized in that, The system includes: The access control data acquisition module is configured to acquire historical access cards or biometric verification records, and to collect verification results, door magnetic status signals, and door lock status signals in real time through the access control device; The video acquisition and target detection module is configured to acquire video data in real time, detect and track valid targets within the access control area, determine the number of targets, and extract several abnormal video time periods. The access control anomaly detection module is configured to determine, based on the collected card swipe and door magnetic sensor and door lock status signals, whether there is an abnormal state where the card swipe fails but the door magnetic sensor responds but the door lock does not respond, and whether the access control authentication has failed. The multi-feature integrated analysis module is configured to further determine whether the abnormal state is an unauthorized intrusion event based on the result of access control identification being an abnormal state, and to perform multi-feature integrated analysis and discrimination on the state feature samples within each time window; The multi-target tracking and identity association module is configured to detect and track targets based on video data acquired by the video acquisition and target detection module, and determine the number of valid targets within the access control area and several abnormal video time periods. The fusion judgment module is configured to determine whether an event is a suspicious intrusion behavior based on the multi-feature integration analysis and discrimination results obtained by the multi-feature integration analysis module, the number of effective targets obtained by the multi-target tracking and identity association module, and several abnormal video time periods, and based on preset rule weights and time consistency judgment criteria.

Images