A method and system for identifying abnormal cat behaviors

Abstract

This application provides a method and system for identifying abnormal behavior in cat-eye surveillance, applied in the fields of security monitoring and behavior recognition. By acquiring multi-source heterogeneous data, it provides a comprehensive foundation for subsequent analysis. These data are extracted into discrete atomic events, achieving dimensionality reduction and structuring of complex data, facilitating efficient processing in resource-constrained environments. Atomic events are stored chronologically and the event sequence within a sliding time window is maintained, enabling correlation analysis of discontinuous behaviors and overcoming the limitations of judging short-term continuous data. Based on preset rules, the event sequence is analyzed, and state transitions and abnormal behavior discrimination are performed, achieving accurate modeling and identification of complex, multi-stage abnormal behaviors. The abnormal behavior type, confidence level, and timestamp are output, providing clear and actionable information for users or subsequent systems. Therefore, the proposed solution has the advantages of multi-dimensional perception, efficient processing, and accurate identification of complex abnormal behaviors.

Description

Technical Field

[0001] This application relates to the fields of security monitoring and behavior recognition technology, and in particular to a method and system for recognizing abnormal cat behavior. Background Technology

[0002] A peephole system, an important residential security device, is typically installed on the front door of a house. Its main function is to monitor activity in the area in front of the door and identify potentially abnormal behavior. This system usually incorporates an image sensor to acquire video streams of the area in front of the door. The system processes and analyzes the acquired video streams in real time to identify preset behavioral patterns.

[0003] Modern peephole systems often interconnect with smart home devices, especially smart locks. The current state of the lock (e.g., locked, unlocked, or unlocked with authorized credentials) provides crucial contextual information for behavior assessment. For example, when the lock is unlocked with authorized credentials, a target's activity within the lock area should generally not be considered abnormal. Conversely, if the lock is locked and no authorized credentials are expected, an unauthorized attempt to operate within the lock area should be considered abnormal. Therefore, peephole systems need to acquire and effectively utilize the state information from external devices such as smart locks, integrating it into the behavior recognition process.

[0004] Given that many peephole systems operate on edge computing units, their computing resources are typically limited. Under such constraints, identifying abnormal behavioral sequences that involve long time spans, intermittent activity, complex interactions, and require the integration of external state information poses a significant challenge to the system's ability to continuously track targets, maintain behavioral states over extended periods, and understand multi-stage behavioral patterns. Judgments based solely on a single image frame or a short time window are prone to false alarms or missed detections.

[0005] Therefore, how to accurately model and identify complex, discontinuous, and externally-state-related doorway behavior sequences at resource-constrained edge environments is a pressing technical challenge that needs to be addressed.

[0006] To address the aforementioned issues, existing technologies urgently need improvement. Summary of the Invention

[0007] In view of the shortcomings of the prior art, this application provides a method and system for identifying abnormal behavior of cat eyes, which is applied to the fields of security monitoring and behavior recognition technology. It has the advantages of improving the recognition accuracy of cat eye system in complex environments, improving the accuracy and robustness of cat eye system judgment, effectively identifying abnormal behaviors with long time span, intermittent and complex combinations, and being suitable for resource-constrained environments.

[0008] In a first aspect, a method for identifying abnormal cat eye behavior includes the following steps:

[0009] S1: Obtain the video stream, audio stream, and smart lock status of the area in front of the door;

[0010] S2: Extract multiple atomic events based on the video stream, the audio stream, and the smart lock status;

[0011] S3: Store atomic events in chronological order to the event buffer and maintain the sequence of atomic events within the sliding time window;

[0012] S4: Based on preset abnormal behavior pattern rules, analyze the atomic event sequence in the event buffer, and perform state transition and abnormal behavior discrimination;

[0013] S5: Based on the final state or judgment result after state transition, output the identified abnormal behavior type, confidence level, and occurrence timestamp.

[0014] This application proposes a method for identifying abnormal behavior in smart locks. Step S1 acquires multi-source heterogeneous data, including video streams, audio streams, and smart lock status, enabling multi-dimensional perception of the door-front environment and incorporating external contextual information, providing a comprehensive foundation for subsequent analysis. Step S2 extracts these data into discrete atomic events, achieving dimensionality reduction and structuring of complex data, facilitating efficient processing in resource-constrained environments. Step S3 stores atomic events in chronological order and maintains the event sequence within a sliding time window, enabling correlation analysis of discontinuous behaviors and overcoming the limitations of judging short-term continuous data. Step S4 analyzes the event sequence based on preset rules and performs state transitions and abnormal behavior discrimination, achieving accurate modeling and identification of complex, multi-stage abnormal behaviors. Step S5 outputs the abnormal behavior type, confidence level, and timestamp, providing clear and actionable information for users or subsequent systems. Therefore, the proposed solution has the advantages of multi-dimensional perception, efficient processing, and accurate identification of complex abnormal behaviors.

[0015] Furthermore, step S2 includes:

[0016] S21: Decode the video stream and extract a foreground image containing the moving target;

[0017] S22: Perform target detection on the foreground image to obtain target category and location information, and calculate the distance between the target and the door lock area and the overlapping area between the target and the door lock area; if the distance is less than a preset threshold, generate a target approach event; if the overlapping area is detected to be greater than a preset area for N consecutive frames, generate a target contact event, where N is a positive integer greater than or equal to 3.

[0018] S23: Perform noise reduction processing on the audio stream and extract sound events;

[0019] S24: Real-time acquisition of the smart lock status; if the lock status changes, a lock status event is generated.

[0020] The atomic events include the target proximity event, the target contact event, the sound event, and the door lock status event.

[0021] This application proposes a method for identifying abnormal behavior in cat eyes. By processing three different types of data—video, audio, and smart lock status—it generates a variety of atomic events with clear physical meaning and behavioral correlation, laying the foundation for subsequent state transitions and anomaly detection.

[0022] Furthermore, step S22 includes:

[0023] S221: Reduce the resolution of the foreground image using an image pyramid to obtain a multi-scale image;

[0024] S222: Perform target detection on the multi-scale image to obtain multi-scale target category and location information;

[0025] S223: Fuse the multi-scale target category and location information to obtain the target detection result;

[0026] S224: Calculate the distance between the target and the door lock area and the overlapping area between the target and the door lock area based on the target detection result. If the distance is less than a preset threshold, generate a target approach event. If the overlapping area is detected to be greater than a preset area for N consecutive frames, generate a target contact event. Here, N is a positive integer greater than or equal to 3.

[0027] The proposed method for identifying abnormal cat-eye behavior improves the target detection process by introducing multi-scale target detection technology, thereby enhancing the detection accuracy for targets of different sizes. This leads to more accurate generation of target approach events and target contact events, solving the problem of poor performance of single-scale detection for targets of different sizes.

[0028] Furthermore, step S224 includes:

[0029] S2241: Extract the pixel coordinates of the target detection box based on the target detection results;

[0030] S2242: Calculate the Euclidean distance between the center point of the target detection box and the center point of the smallest outer rectangle of the door lock area to obtain the distance;

[0031] S2243: Calculate the intersection-union ratio of the target detection box and the smallest outer rectangle of the door lock area to obtain the overlapping area;

[0032] S2244: If the distance is less than a preset threshold, a target approach event is generated; if the overlapping area is detected to be greater than a preset area for N consecutive frames, a target contact event is generated, where N is a positive integer greater than or equal to 3.

[0033] This application proposes a method for identifying abnormal behavior in cat eyes, aiming to provide a specific, accurate, and robust method to quantify the spatial relationship between the target and the door lock area, thereby improving the accuracy of target approach and target contact atomic events, and thus enhancing the reliability of subsequent abnormal behavior identification.

[0034] Furthermore, step S3 includes:

[0035] S31: Store atomic events in chronological order to the event buffer, and set the priority of the atomic events;

[0036] S32: When the event buffer is full, compare the priority of the latest atomic event with the priority of the lowest priority atomic event in the event buffer. If the priority of the latest atomic event is higher, replace the lowest priority atomic event; otherwise, discard the latest atomic event.

[0037] S33: When an atomic event entering the event buffer is detected to be a sound event, the sliding window is adjusted to a preset long time window threshold; otherwise, a short time window threshold is used to maintain the atomic event sequence within the sliding time window.

[0038] Furthermore, step S31 includes:

[0039] S311: Identify the current working mode of the peephole system, which includes home mode, away mode and alert mode;

[0040] S312: Adjust the priority of atomic events according to the recognition working mode. In home mode, the priority of sound events is increased; in away mode, the priority of video events is increased; in alert mode, the priority of door lock status events is increased.

[0041] S313: Store atomic events in chronological order to the event buffer, and set the priority of the atomic events according to the adjusted priority.

[0042] Furthermore, step S4 includes:

[0043] S41: Construct a state transition network containing multiple state nodes. The state nodes include the initial state, intermediate state and final state. The state nodes represent the stage of the cat eye system. The initial state represents the system startup state and the final state represents the abnormal behavior recognition completion state.

[0044] S42: Read the atomic event sequence from the event buffer, analyze the atomic event sequence according to the preset abnormal behavior pattern rules of the current state node, if the preset abnormal behavior pattern rules are met, then perform state transition according to the smart door lock state and update the current state node; otherwise, keep the current state node unchanged.

[0045] S43: Repeat S43 until the state transitions to the final state or the atomic event sequence in the event buffer is analyzed. If the state transitions to the final state, it is determined that abnormal behavior has occurred; otherwise, it is determined that no abnormal behavior has occurred.

[0046] Furthermore, the preset abnormal behavior pattern rules include the atomic event combinations, occurrence order, time interval, smart door lock status, and cat eye system response status.

[0047] Furthermore, step S42 includes:

[0048] S421: Read the atomic event sequence from the event buffer and construct a directed acyclic graph of the atomic event sequence;

[0049] S422: Based on the preset abnormal behavior pattern rules of the current state node, extract the candidate event chain that matches the preset abnormal behavior pattern rules from the directed acyclic graph;

[0050] S423: For each extracted candidate event chain, calculate its matching score with the preset abnormal behavior pattern rule;

[0051] S424: If the matching score of the candidate event chain with the highest matching score exceeds the preset threshold, it is determined that the preset abnormal behavior pattern rule is met, and the state is transitioned according to the smart lock state to update the current state node; otherwise, the current state node remains unchanged.

[0052] Secondly, a cat-eye abnormal behavior recognition system, applied in the steps of any of the methods described above, the system comprising:

[0053] Data acquisition module: Acquires video streams, audio streams, and smart lock status of the area in front of the door;

[0054] Event extraction module: Extracts multiple atomic events based on the video stream, the audio stream, and the smart lock status;

[0055] Event storage module: Stores atomic events in chronological order to the event buffer and maintains the sequence of atomic events within the sliding time window;

[0056] Event Judgment Module: Based on preset abnormal behavior pattern rules, analyzes the atomic event sequence in the event buffer, and performs state transition and abnormal behavior discrimination;

[0057] Results output module: Based on the final state after state transition or the judgment result, output the identified abnormal behavior type, confidence level and occurrence timestamp.

[0058] Beneficial Effects: This application proposes a method and system for identifying abnormal behavior in smart locks. By acquiring multi-source heterogeneous data, including video streams, audio streams, and smart lock status, it achieves multi-dimensional perception of the door-front environment and introduces external contextual information, providing a comprehensive foundation for subsequent analysis. Extracting these data into discrete atomic events achieves dimensionality reduction and structuring of complex data, facilitating efficient processing in resource-constrained environments. Storing atomic events in chronological order and maintaining the event sequence within a sliding time window enables correlation analysis of discontinuous behaviors, overcoming the limitations of judging short-term continuous data. Analyzing event sequences based on preset rules and performing state transitions and abnormal behavior discrimination achieves accurate modeling and identification of complex, multi-stage abnormal behaviors. Outputting the abnormal behavior type, confidence level, and timestamp provides clear and actionable information for users or subsequent systems. Therefore, the proposed solution has the advantages of multi-dimensional perception, efficient processing, and accurate identification of complex abnormal behaviors. Attached Figure Description

[0059] Figure 1 This is a flowchart of a method for identifying abnormal cat eye behavior proposed in this application.

[0060] Figure 2 This is a structural diagram of a cat-eye abnormal behavior recognition system proposed in this application.

[0061] Figure 3 This is an architecture diagram of a cat-eye abnormal behavior recognition system proposed in this application.

[0062] Figure 4 This application proposes a directed acyclic graph.

[0063] Labeling Explanation: 201, Data Acquisition Module; 202, Event Extraction Module; 203, Event Storage Module; 204, Event Judgment Module; 205, Result Output Module. Detailed Implementation

[0064] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. The components of the embodiments of this application described and marked in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0065] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0066] Please refer to Figure 1 A method for identifying abnormal cat behavior, the method includes the following steps:

[0067] S1: Obtain the video stream, audio stream, and smart lock status of the area in front of the door;

[0068] S2: Extract multiple atomic events based on the video stream, audio stream, and smart lock status;

[0069] S3: Store atomic events in chronological order to the event buffer and maintain the sequence of atomic events within the sliding time window;

[0070] S4: Based on preset abnormal behavior pattern rules, analyze the atomic event sequence in the event buffer, and perform state transition and abnormal behavior discrimination;

[0071] S5: Based on the final state or judgment result after state transition, output the identified abnormal behavior type, confidence level, and occurrence timestamp.

[0072] In step S1, acquiring the video stream, audio stream, and smart lock status of the area in front of the door refers to obtaining information about the area from different sensors and devices, including visual, auditory, and the current status of the lock. This can be achieved using a camera, microphone, and a communication interface with the smart lock. In this solution, acquiring multi-source data provides a more comprehensive environmental perception and crucial contextual information than a single data source, which helps to more accurately understand activities occurring in front of the door, particularly distinguishing between normal and abnormal behavior.

[0073] In step S2, extracting multiple atomic events refers to transforming continuous raw data and discrete state information into discrete basic event units with clear semantics. This can be achieved using image processing, sound analysis, and state monitoring techniques. In this scheme, extracting atomic events abstracts and structures complex raw data, reducing the complexity of subsequent processing, making it suitable for analysis in resource-constrained environments, and enabling the capture of key activities reflected by different types of sensors.

[0074] In step S3, storing atomic events to the event buffer in chronological order means saving the extracted atomic events in a temporary storage area according to their chronological order of occurrence. This can be implemented using a queue or list structure in memory. In this scheme, storing atomic events is to establish a historical record of events, providing an event sequence within a certain time range for subsequent analysis, thereby enabling the analysis of non-continuous behavioral patterns.

[0075] In step S4, maintaining the atomic event sequence within the sliding time window means focusing on and processing only the atomic event sequence that occurred within the most recent time period in the event buffer. This can be achieved by managing the buffer size and event timestamps. In this scheme, maintaining the sliding time window is to focus on recently relevant events, filter out outdated information, and simultaneously correlate events that occurred scattered over a period of time for analysis, overcoming the limitations of making judgments based solely on instantaneous or extremely short-term data.

[0076] Predefined abnormal behavior pattern rules refer to predefined patterns that describe the atomic events, their combinations, sequences, temporal relationships, and contextual states that constitute a specific abnormal behavior. These patterns can be represented using rule sets, state machines, or graph structures. In this scheme, the predefined rules serve as the basis for identifying abnormal behavior; the system determines the existence of anomalies by matching observed event sequences against these rules.

[0077] Analyzing the atomic event sequence in the event buffer refers to checking and comparing the atomic event sequence within a sliding time window in the event buffer according to preset rules. This can be achieved using pattern matching, sequence alignment, or graph analysis algorithms. In this solution, analyzing the event sequence is the core process for identifying abnormal behavior. By examining the type, order, time interval, and correlation with contextual information such as door lock status of the events, it is determined whether they conform to a pattern of abnormal behavior.

[0078] State transition refers to the system moving from one behavioral stage to another based on the analysis of the current sequence of atomic events. This can be implemented using a state machine model. In this scheme, state transition is used to model complex, multi-stage anomalous behavior. The system progressively advances its understanding of the behavior based on observed events until it reaches a final anomalous state or determines that the behavior is not anomalous.

[0079] Abnormal behavior discrimination refers to determining whether a currently occurring behavior belongs to a preset abnormal behavior type based on the results of state transitions or the final analysis of an event sequence. Specifically, this can be achieved by judging whether an abnormal state has been reached or whether the matching degree exceeds a threshold. In this scheme, abnormal behavior discrimination is the final judgment step in the identification process, directly outputting a conclusion on whether the behavior is abnormal.

[0080] In step S5, based on the final state or judgment result after state transition, outputting the identified abnormal behavior type, confidence level, and occurrence timestamp means presenting the identified abnormal behavior information in a structured form. This can be implemented using data structures or message formats. In this solution, the output is intended to provide users or subsequent systems with clear and actionable information about abnormal events, including the type of abnormality, the confidence level of the judgment, and the time of occurrence, facilitating further action.

[0081] The core innovation of this application lies in integrating multi-source heterogeneous data (including video, audio, and key smart lock status), abstracting them into discrete atomic events, and analyzing the atomic event sequence within a sliding time window through a state transition mechanism based on preset rules. This enables the accurate identification of complex, discontinuous abnormal behaviors in front of the door that involve external states in resource-constrained environments.

[0082] This method aims to address the technical challenge of accurately identifying complex, discontinuous, and externally-involved gate-front anomalous behavior sequences in resource-constrained environments. Its core idea lies in integrating multi-source heterogeneous data, abstracting it into discrete atomic events, and identifying anomalies by analyzing the time series and state transitions of these events.

[0083] Furthermore, step S2 includes:

[0084] S21: Decode the video stream and extract the foreground image containing the moving target;

[0085] S22: Perform target detection on the foreground image to obtain target category and location information, and calculate the distance between the target and the door lock area and the overlapping area between the target and the door lock area; if the distance is less than a preset threshold, a target approach event is generated; if the overlapping area detected in N consecutive frames is greater than a preset area, a target contact event is generated, where N is a positive integer greater than or equal to 3.

[0086] S23: Perform noise reduction processing on the audio stream and extract sound events;

[0087] S24: Real-time acquisition of smart lock status; if the lock status changes, a lock status event is generated.

[0088] Atomic events include target proximity events, target contact events, sound events, and door lock status events.

[0089] This solution provides a specific implementation method for extracting atomic events for abnormal behavior identification from multi-source data. By processing three different types of data—video, audio, and smart lock status—a variety of atomic events with clear physical meaning and behavioral correlation are generated, laying the foundation for subsequent state transitions and anomaly detection.

[0090] Specifically, step S21 decodes the video stream and extracts foreground images containing moving targets, focusing on changes and activities in the area in front of the door. This effectively reduces the amount of data that needs to be processed and improves the efficiency of subsequent analysis. Video stream decoding can be implemented using a standard video decoder.

[0091] In step S22, object detection can be implemented using a deep learning-based model, such as YOLO or SSD. These models can identify pre-defined categories of targets, such as pedestrians and packages, in the image and output their bounding box coordinates. Based on this, the distance and overlap area between the target and the door lock region are calculated. These parameters directly quantify the degree of interaction between the target and this core region of the door lock. The door lock region can be pre-defined as a fixed area in the image, such as a rectangle. The distance between the target and the door lock region can be calculated by the Euclidean distance between the center point of the target's bounding box and the center point of the door lock region. The overlap area between the target and the door lock region can be calculated by the intersection-union ratio (IUGR) of the target's bounding box and the door lock region.

[0092] Generating target proximity events based on distances less than a preset threshold allows for timely detection of early warning signs of a target approaching a door lock. For example, the preset threshold can be set to 0.5 meters. Generating target contact events based on N consecutive frames detecting an overlap area greater than a preset area allows for more robust identification of actual contact between the target and the door lock, such as attempts to operate the lock. The preset area can be set to 0.7, indicating that the target bounding box overlaps with the door lock area by more than 70%. Continuous frame detection avoids false alarms caused by single-frame false detections. N can be set to 4, 5, or 10. These extracted target proximity and target contact events are treated as atomic events and sent to subsequent event buffers for storage and sequence maintenance.

[0093] Step S23 involves denoising the audio stream and then extracting sound events, using the sound information to supplement the video information. The denoising process can be implemented using spectral subtraction or a deep learning denoising model to reduce the impact of environmental noise on sound event recognition. Sound event extraction can be achieved using a machine learning or deep learning-based sound classification model to identify preset sound types such as knocking sounds, lock picking sounds, glass breaking sounds, and unusual shouts. These sound events enrich the perception of the environment in front of the door and are sent to the event buffer as atomic events.

[0094] Step S24: Acquire the smart lock status in real time and generate a lock status event when the status changes. The smart lock status can be obtained through communication with the smart lock, such as via Bluetooth or Wi-Fi connection. The lock status can include locked, unlocked, unlocking, locking, alarm, etc. The lock status is key contextual information for determining whether actions at the door are legal. A change in the lock status itself (such as an unauthorized unlocking attempt) is an important anomaly signal, and it can be directly used as an atomic event in subsequent anomaly detection. The lock status event is sent to the event buffer as an atomic event.

[0095] Furthermore, step S22 includes:

[0096] S221: Use image pyramids to reduce the resolution of the foreground image to obtain a multi-scale image;

[0097] S222: Perform target detection on multi-scale images to obtain multi-scale target category and location information;

[0098] S223: Integrate multi-scale target category and location information to obtain target detection results;

[0099] S224: Calculate the distance between the target and the door lock area and the overlapping area between the target and the door lock area based on the target detection results. If the distance is less than a preset threshold, a target approach event is generated. If the overlapping area detected in N consecutive frames is greater than a preset area, a target contact event is generated, where N is a positive integer greater than or equal to 3.

[0100] In step S221, the resolution of the foreground image is reduced using an image pyramid to obtain multi-scale images. This is to overcome the problem of poor adaptability of single-scale detection to targets of different sizes. By generating images with different resolutions, a foundation is laid for subsequent detection at different scales. For example, the original foreground image can be downsampled layer by layer using Gaussian pyramids or Laplacian pyramids, with the resolution of each layer decreasing by a fixed percentage, such as 0.5, relative to the previous layer. This results in a series of images with decreasing resolution, forming an image pyramid.

[0101] Step S222 performs detection on images at different resolutions, allowing the detection algorithm to better adapt to targets of different sizes. For example, large distant targets are easier to detect on low-resolution images, while small nearby targets are easier to detect on high-resolution images. The detector at each scale runs independently on the corresponding resolution image, outputting the target bounding box and class confidence at that scale.

[0102] Step S223 integrates the detection results from different scales, combining the advantages of each scale to improve the overall accuracy and robustness of the detection, and reducing false negatives or missed detections that may result from single-scale detection. Fusion methods may include non-maximum suppression (NMS) applied across scales.

[0103] Specifically: the fusion process is as follows:

[0104] Suppose that in the image pyramid, we obtain object detection results at three scales, corresponding to the original image, the 1 / 2 resolution image, and the 1 / 4 resolution image, respectively. At each scale, an object was detected, represented by a bounding box and a confidence score.

[0105] First, all detection results at all scales are mapped back to the coordinate system of the original image. Then, all bounding boxes mapped back to the original image coordinate system are sorted by confidence level. The bounding box with the highest confidence level is selected and added to the final detection result list. Next, the intersection-union ratio (IoU) of this bounding box with all other bounding boxes at all scales is calculated. If the IoU of a bounding box with the highest confidence level is greater than a preset threshold (e.g., 0.5), it is considered that the bounding box and the highest confidence level bounding box represent the same target and should be suppressed. Therefore, these bounding boxes with IoU greater than the threshold are removed from the list of bounding boxes to be processed. The above process is repeated, selecting the highest confidence level bounding box from the remaining bounding boxes, adding it to the final detection result list, and removing all bounding boxes with IoU greater than the threshold. This continues until the list of bounding boxes to be processed is empty.

[0106] In this way, NMS cross-scale application can effectively suppress redundant detection results from images of different scales, retain the detection results with the highest confidence and most representative of the real target, thereby improving the accuracy and robustness of target detection.

[0107] By fusing the data, a final list containing all detected targets and their positions and categories in the original foreground image coordinate system can be obtained.

[0108] Furthermore, step S224 includes:

[0109] S2241: Extract the pixel coordinates of the target detection box based on the target detection results;

[0110] S2242: Calculate the Euclidean distance between the center point of the target detection box and the center point of the smallest outer rectangle of the door lock area to obtain the distance;

[0111] S2243: Calculate the intersection-union ratio of the target detection box and the smallest outer rectangle of the door lock area to obtain the overlapping area;

[0112] S2244: If the distance is less than a preset threshold, a target approach event is generated. If the overlapping area detected in N consecutive frames is greater than a preset area, a target contact event is generated, where N is a positive integer greater than or equal to 3.

[0113] Step S2241 processes the target detection results to obtain the target's position information in the image, specifically represented as a set of pixel coordinates of the target detection box. These coordinates constitute the basic data describing the target boundary, providing input for subsequent spatial relationship calculations. For example, a target detection box can be determined by the pixel coordinates of its upper left and lower right corners.

[0114] Step S2242 first determines the coordinates of the center point of the target detection box, and simultaneously determines the coordinates of the center point of the pre-defined minimum outer rectangle of the door lock area. Then, the Euclidean distance between these two center points is calculated. This yields a quantified value representing the distance between the overall position of the target and the door lock area. As a preferred implementation, the minimum outer rectangle of the door lock area can be determined through configuration or calibration during system initialization.

[0115] Step S2243 calculates the overlapping area. The intersection-union ratio (IUGR) between the target detection box and the smallest outer rectangle of the door lock region is calculated to obtain a proportional value that measures the degree of overlap between the two rectangular regions. The IUGR is calculated by dividing the area of ​​the intersection of the two rectangles by the area of ​​their union. This proportional value reflects the degree of spatial intersection between the target and the door lock region on the image plane.

[0116] If the calculated distance is less than a preset distance threshold in step S2244, it is determined that the target has approached the door lock area, thus generating a target approach event. Furthermore, to determine whether the target has actually interacted or made contact with the door lock area, continuous overlap needs to be detected. Therefore, if the system detects that the intersection-union ratio (i.e., the overlap area) of the target detection box and the smallest outer rectangle of the door lock area is greater than a preset area threshold for N consecutive frames (N is a positive integer greater than or equal to 3), a target contact event is generated.

[0117] In this way, the judgment of consecutive frames is introduced into the logic for generating contact events, making the generated events more reflective of the continuous spatial relationship between the target and the door lock area. These generated atomic events, namely target approach events and target contact events, are then used to construct event sequences, providing the basic input for subsequent abnormal behavior recognition. By providing specific methods for calculating distance and overlap area, as well as event generation logic based on consecutive frame judgment, this scheme provides an implementation approach for accurately capturing the spatial interaction between the target and the door lock area.

[0118] Furthermore, step S3 includes:

[0119] S31: Store atomic events in chronological order to the event buffer and set the priority of the atomic events;

[0120] S32: When the event buffer is full, compare the priority of the latest atomic event with the priority of the lowest priority atomic event in the event buffer. If the priority of the latest atomic event is higher, replace the lowest priority atomic event; otherwise, discard the latest atomic event.

[0121] S33: When an atomic event entering the event buffer is detected to be a sound event, the sliding window is adjusted to a preset long time window threshold; otherwise, a short time window threshold is used to maintain the atomic event sequence within the sliding time window.

[0122] Step S31 involves storing atomic events in chronological order and assigning priorities. Atomic events can be assigned different priority values ​​based on their type, source, or relevance to potential abnormal behavior. For example, events related to changes in door lock status can be assigned higher priority, while certain background sound events can be assigned lower priority. The stored procedure ensures that events maintain their chronological order within the buffer. The priority setting provides a basis for subsequent event management when buffer space is limited.

[0123] In step S32, when a new atomic event is generated and needs to be stored in the buffer, this step is triggered if the buffer has reached its capacity limit. The priority of the latest atomic event is obtained and compared with the lowest priority event among all atomic events in the current buffer. If the priority of the latest event is higher than that of the lowest priority event, the lowest priority event is removed from the buffer, and the latest event is stored. If the priority of the latest event is not higher than that of the lowest priority event, the latest event is discarded and not stored in the buffer. This priority-based replacement mechanism avoids simply discarding the oldest event in chronological order, ensuring that higher priority events can be retained when buffer space is limited, thereby improving the effectiveness of the event sequence in the buffer.

[0124] In step S33, the sliding time window is used to define the time range of the atomic event sequence that needs to be analyzed for abnormal behavior. When a newly entered atomic event in the event buffer is detected to be an audio event, the size of the sliding window is set to a preset long-term threshold, such as 10 seconds or longer. This is because audio events, especially certain types of sounds (such as continuous knocking sounds), may require a longer period of contextual information to accurately determine their meaning or whether they constitute an anomaly. When the newly entered atomic event is not an audio event, the size of the sliding window is set to a preset short-term threshold, such as 2 seconds or 5 seconds. This method of adjusting the window size according to the event type allows subsequent abnormal behavior analysis to obtain appropriate temporal context based on the characteristics of different types of events, improving the accuracy and efficiency of the analysis. For example, for audio anomalies that require a long period of context, a sufficient event sequence is provided; for other anomalies that may only require a brief interaction to determine, a shorter window is used, reducing unnecessary computation.

[0125] Furthermore, step S31 includes:

[0126] S311: Identify the current working mode of the peephole system, which includes home mode, away mode and alert mode;

[0127] S312: Adjust the priority of atomic events according to the recognition working mode. In home mode, the priority of sound events is increased; in away mode, the priority of video events is increased; in alert mode, the priority of door lock status events is increased.

[0128] S313: Store atomic events in chronological order to the event buffer, and set the priority of atomic events according to the adjusted priority.

[0129] Specifically, the peephole system first determines its current operating mode, such as whether it is at home, away from home, or on alert. Then, based on the determined operating mode, the system dynamically adjusts the priority of different types of atomic events (such as sound events, video events, and door lock status events). Finally, when newly generated atomic events are stored in the event buffer, their priority is set to the priority adjusted according to the current operating mode.

[0130] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0131] The system identifies the current operating mode of the doorbell system. For example, the operating mode can be manually set by the user in the accompanying mobile application, or it can automatically switch through linkage with a smart home hub; for instance, when the user leaves home, the system automatically switches to "away mode." The identified operating mode can be "home mode," "away mode," or "alarm mode."

[0132] Based on the identified current operating mode, the system dynamically adjusts the priority of different types of atomic events. For example, a priority mapping table can be set. In home mode, sound events are given a higher priority, while other event types (such as video events and door lock status events) have relatively lower priorities. In away mode, video events (such as target approach events and target contact events) are given a higher priority, while sound events and door lock status events have relatively lower priorities. In alert mode, door lock status events are given a higher priority, while video events and sound events have relatively lower priorities. Priorities can be represented numerically, with higher values ​​indicating higher priorities.

[0133] Atomic events are stored in the event buffer in chronological order of their occurrence. When storing each atomic event, its priority is assigned based on the current system operating mode and preset priority adjustment rules. For example, when the system is in home mode, newly generated audio events are assigned the higher priority value corresponding to audio events in home mode, while newly generated video events are assigned the lower priority value corresponding to video events in home mode. Thus, the atomic events in the event buffer carry priority information dynamically set according to the current system state.

[0134] Furthermore, step S4 includes:

[0135] S41: Construct a state transition network containing multiple state nodes. The state nodes include the initial state, intermediate state and final state. The state nodes represent the stage of the cat eye system. The initial state represents the system startup state and the final state represents the abnormal behavior recognition completion state.

[0136] S42: Read the atomic event sequence from the event buffer, analyze the atomic event sequence according to the preset abnormal behavior pattern rules of the current state node, if the preset abnormal behavior pattern rules are met, then perform state transition according to the smart door lock state and update the current state node; otherwise, keep the current state node unchanged.

[0137] S43: Repeat S43 until the state transitions to the final state or the atomic event sequence in the event buffer is analyzed. If the state transitions to the final state, it is determined that abnormal behavior has occurred; otherwise, it is determined that no abnormal behavior has occurred.

[0138] Step S41 decomposes the complex abnormal behavior identification process into a series of discrete states and transitions between states by constructing a state transition network. This network includes an initial state, one or more intermediate states, and a final state. The initial state represents the starting point where the system begins analyzing the behavior sequence. Intermediate states represent different stages in the development of abnormal behavior, such as "target approaching," "target touching the lock," and "attempt to unlock." The final state represents that the abnormal behavior has been identified. Each state node represents a specific stage in the behavior identification process of the peephole system. This structure provides a framework for analyzing long-term, multi-stage behavior sequences. For example, a network can be constructed where the initial state transitions to the intermediate state of "target approaching," then to the intermediate state of "target touching" based on subsequent events, and may eventually transition to the final state of "abnormal unlocking attempt."

[0139] Step S42 retrieves a sequence of atomic events arranged chronologically from the event buffer. These atomic events are extracted and stored from the video stream, audio stream, and smart lock status in previous steps. At the current state node, the system applies preset abnormal behavior pattern rules associated with that state node to analyze the retrieved atomic event sequence. These rules define the event types, order, time intervals, and possible external states that need to be considered at the current stage. For example, in the "target approach" state, the rules might focus on the occurrence of a "target contact event." If the event sequence satisfies the rules of the current state node, a state transition occurs. The state transition decision further incorporates the smart lock status. For example, even if a target contact event is detected, if the smart lock status displays "legitimate unlock," it may not transition to a state indicating an anomaly, but instead transition to another intermediate state or remain in the current state. Thus, the smart lock status serves as important contextual information, correcting behavior judgments and improving accuracy. If the event sequence does not satisfy the rules of the current state node, the system remains in the current state node, waiting for subsequent events. This analysis and transition mechanism, based on the current state, event sequence, and external state, enables the system to gradually track and understand the development process of complex behaviors.

[0140] Step S43 repeats step S42. The system continuously reads new atomic events from the event buffer and analyzes and transitions the state according to the rules of the current state node. This process continues until the system state transitions to the final state, or all atomic event sequences in the event buffer have been analyzed. If the state successfully transitions to the final state, it indicates that the system has identified a complete abnormal behavior sequence that conforms to a preset pattern, and therefore determines that abnormal behavior has occurred. If the event sequence analysis is completed, but the system fails to transition to the final state, it is determined that no abnormal behavior has occurred.

[0141] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0142] Construct a state transition network to identify abnormal behavior within the monitoring area of ​​the peephole system. This network contains multiple state nodes; for example, the initial state can be set to "idle," intermediate states include "target approaching," "target lingering," and "potential contact," and the final state is "abnormal behavior occurring." Each state node is associated with a set of preset abnormal behavior pattern rules.

[0143] Specifically, when the system starts, it enters an "idle" state. The system reads a sequence of atomic events arranged in chronological order from the event buffer. In the "idle" state, preset rules may focus on "target proximity events". If a "target proximity event" is detected, the system state transitions to the "target proximity" state according to the rules.

[0144] In the "Target Approach" state, preset rules may focus on "Target Stay Events" or "Target Contact Events." If a "Target Stay Event" is continuously detected over a period of time, the state may remain in the "Target Approach" state or transition to the "Target Stay" state. If a "Target Contact Event" is detected, the smart lock state is further checked. For example, if the smart lock state is "Locked" and a "Target Contact Event" is detected, the preset rules for the current state node are satisfied, and the system state transitions to the "Potential Contact" state. If the smart lock state is "Legitimate Unlocked," even if a "Target Contact Event" is detected, the rules may determine that the abnormal mode is not met, and the state remains unchanged or transitions back to the "Idle" state.

[0145] In the "potential contact" state, the preset rules may focus on "audio events" (such as the sound of picking the lock) or continuous "target contact events," and then combine this with the smart lock status for judgment. For example, if the smart lock status is still "locked," and an "audio event" or continuous "target contact event" is detected, then the rule is satisfied, and the system state transitions to the final state of "abnormal behavior occurs."

[0146] The system repeatedly executes the process of reading events, analyzing rules, and performing state transitions based on the smart lock's status until the final state of "abnormal behavior occurring" is reached, or the atomic event sequence in the event buffer is analyzed. If the final state is reached, abnormal behavior is determined to have occurred. If the event sequence analysis is completed but the final state is not reached, abnormal behavior is determined not to have occurred.

[0147] Furthermore, the preset abnormal behavior pattern rules include atomic event combinations, occurrence order, time interval, smart lock status, and cat eye system response status.

[0148] Specifically, the preset abnormal behavior pattern rules can include specific combinations of atomic events, the order in which these events occur, the time intervals between events, specific smart lock states, and the response state of the peephole system. For example, a rule might stipulate that in the "target contact" state, if a "target contact event" is detected shortly followed by an "audio event" (such as a lock-picking sound), and the smart lock state is "locked," then the rule is satisfied, and the system may transition to the "abnormal attempt" state. As a preferred implementation, the rule can define a pattern of event sequences, for example, using regular expressions or finite state machines to describe allowed event sequences. The smart lock state can be directly included as part of the rule, for example, "when the state is X and event Y is detected, if the lock state is Z, then transition to the new state."

[0149] Furthermore, step S42 includes:

[0150] S421: Read the atomic event sequence from the event buffer and construct a directed acyclic graph of the atomic event sequence;

[0151] S422: Based on the preset abnormal behavior pattern rules of the current state node, extract the candidate event chain that matches the preset abnormal behavior pattern rules from the directed acyclic graph.

[0152] S423: For each extracted candidate event chain, calculate its matching score with the preset abnormal behavior pattern rule;

[0153] S424: If the matching score of the candidate event chain with the highest matching score exceeds the preset threshold, it is determined that the preset abnormal behavior pattern rule is met, and the state transition is performed according to the smart lock state to update the current state node; otherwise, the current state node remains unchanged.

[0154] Specifically, this method constructs a directed acyclic graph (DAG) of atomic event sequences to structurally represent discrete atomic events and their temporal relationships. Next, using pre-defined abnormal behavior pattern rules as templates, it searches and extracts potential event sequence fragments, i.e., candidate event chains, within the constructed graph structure. Subsequently, each extracted candidate event chain is quantitatively compared with the pre-defined rules, calculating a matching score. This score reflects the degree of conformity between the event chain and the rules, allowing for a certain degree of deviation. Finally, by comparing the highest matching score with a pre-defined threshold, it determines whether the current atomic event sequence is sufficient to trigger a state transition, and combines this with the smart lock's state information to update the system's current state node, thereby advancing the abnormal behavior identification process.

[0155] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0156] Please refer to Figure 4Obtain the atomic event sequence from the event buffer. For example, the sequence contains events occurring in chronological order: target approach event (T1), sound event (T2), target contact event (T3), and door lock state change event (T4, from locked to unlocked). Construct a directed acyclic graph (DAG) of this atomic event sequence. Nodes in the graph represent atomic events, and edges represent the chronological order between events. For example, edges pointing from T1 to T2, T3, and T4; edges pointing from T2 to T3 and T4; and edges pointing from T3 to T4.

[0157] Suppose the preset abnormal behavior pattern rule corresponding to the current state node is: "When the door lock is locked, after a target approach event, a target contact event occurs within 5 seconds." Extract candidate event chains matching this rule from the constructed directed acyclic graph. For example, one candidate event chain might be: target approach event (T1) -> target contact event (T3). Another candidate event chain might contain other events, but be related to the rule.

[0158] For each extracted candidate event chain, calculate its matching score against preset abnormal behavior pattern rules. For the candidate chain "Target Approach Event (T1) -> Target Contact Event (T3)," calculate its matching score. The score considers the following factors: whether the event types match (yes, approach and contact), whether the event order matches (yes, approach precedes contact), whether the time interval is within the specified range (e.g., check if the time difference between the occurrence time of T3 and the occurrence time of T1 is less than 5 seconds), and whether the smart lock status meets the requirements (e.g., check if the lock is locked during T1 to T3). A scoring function can be set, for example, a base score is given for a perfect match of event type and order, extra points are given for a time interval within the specified range, and extra points are given for a lock status that meets the requirements. If the time interval is outside the range but close, partial points can be given. If the lock status does not meet the requirements, zero points or a deduction is given for that item. Calculate the total matching score for the candidate chain. Repeat this process for all extracted candidate event chains.

[0159] Compare the matching scores of all candidate event chains and find the one with the highest score. If the highest matching score exceeds a preset threshold (e.g., a threshold of 0.8), the current atomic event sequence is determined to satisfy a preset abnormal behavior pattern rule. Therefore, state transition is performed based on the smart lock's state. For example, if the lock is locked, it transitions to the "Abnormal Behavior in Progress" state node; if the lock is unlocked, it may not transition or may transition to another state node. The current state node is then updated. Otherwise, if the highest matching score does not exceed the preset threshold, it is determined that the preset abnormal behavior pattern rule is not satisfied, and the current state node remains unchanged.

[0160] Please refer to Figure 2 , Figure 3 A cat-eye abnormal behavior recognition system, applied in the steps of any of the above methods, the system comprising:

[0161] Data acquisition module 201: Acquires video stream, audio stream, and smart lock status of the area in front of the door;

[0162] Event extraction module 202: Extracts multiple atomic events based on the video stream, audio stream, and smart lock status;

[0163] Event storage module 203: Stores atomic events in chronological order to the event buffer and maintains the sequence of atomic events within the sliding time window;

[0164] Event Judgment Module 204: Based on preset abnormal behavior pattern rules, analyze the atomic event sequence in the event buffer, and perform state transition and abnormal behavior discrimination;

[0165] Result output module 205: Based on the final state after state transition or the judgment result, output the identified abnormal behavior type, confidence level and occurrence timestamp.

[0166] The data acquisition module 201 is the system's perception layer, responsible for acquiring various types of raw data from the area in front of the door, including video streams, audio streams, and the status of the smart lock. By acquiring this multi-source heterogeneous data, the system can comprehensively perceive the environment in front of the door, providing a rich information foundation for subsequent abnormal behavior identification and overcoming the limitations of relying on only a single data source.

[0167] The event extraction module 202 receives the raw data and transforms it into atomic events with clear semantics. Extracting atomic events based on video streams, audio streams, and smart lock status abstracts continuous, low-level perceptual data into discrete, high-level event representations.

[0168] The event storage module 203 manages the extracted atomic events. It stores atomic events in chronological order in the event buffer and maintains the event sequence within a sliding time window. Storing events in chronological order ensures that the temporal relationships of events are preserved, which is crucial for understanding the order in which actions occur. Maintaining the sliding time window allows the event judgment module to focus on analyzing the most recently occurring event sequence most relevant to the current action, improving the efficiency and accuracy of the analysis.

[0169] The event judgment module 204 is the core intelligent part of the system. Based on preset abnormal behavior pattern rules, it analyzes the atomic event sequence provided by the event storage module and identifies abnormal behavior through state transitions. Analyzing the atomic event sequence based on preset rules involves combining discrete atomic events and attempting to match known abnormal behavior patterns. State transitions allow the system to track the progress of behavior and understand complex behaviors in multiple stages, such as from "target approach" to "target contact" and then to "door lock state change".

[0170] The result output module 205 is the system's external interface. Based on the final state or judgment result of the event judgment module, it outputs the identified abnormal behavior type, confidence level, and occurrence timestamp. Outputting the type, confidence level, and occurrence time of the abnormal behavior provides users or external systems with crucial information about the abnormal event, facilitating timely response and handling.

[0171] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0172] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A cat eye abnormal behavior recognition method, characterized in that, The method comprises steps of: S1: acquiring a video stream, an audio stream and a smart door lock state of a door area; S2: extracting a plurality of atomic events according to the video stream, the audio stream and the smart door lock state; S3: storing the atomic events in an event buffer in chronological order and maintaining an atomic event sequence in a sliding time window; S4: analyzing the atomic event sequence in the event buffer based on a preset abnormal behavior pattern rule, performing state transition and abnormal behavior discrimination; S5: outputting an abnormal behavior type, a confidence and a time stamp of recognition according to a final state after state transition or a discrimination result; Step S2 comprises: S21: decoding the video stream to extract a foreground image containing a moving target; S22: performing target detection on the foreground image to obtain target category and position information, and calculating a distance between the target and a door lock area and an overlapping area between the target and the door lock area; if the distance is less than a preset threshold, a target approaching event is generated, and if the overlapping area is greater than a preset area for N consecutive frames, a target contact event is generated, where N is a positive integer greater than or equal to 3; S23: performing noise reduction processing on the audio stream to extract a sound event; S24: acquiring the smart door lock state in real time, and generating a door lock state event if the door lock state changes; the atomic events include the target approaching event, the target contact event, the sound event and the door lock state event.

2. The cat eye abnormal behavior recognition method of claim 1, wherein, Step S22 comprises: S221: reducing the resolution of the foreground image by using an image pyramid to obtain a multi-scale image; S222: performing target detection on the multi-scale image to obtain multi-scale target category and position information; S223: fusing the multi-scale target category and position information to obtain a target detection result; S224: calculating the distance between the target and the door lock area and the overlapping area between the target and the door lock area according to the target detection result, generating a target approaching event if the distance is less than a preset threshold, and generating a target contact event if the overlapping area is greater than a preset area for N consecutive frames, where N is a positive integer greater than or equal to 3.

3. The cat eye abnormal behavior recognition method of claim 2, wherein, Step S224 comprises: S2241: extracting pixel coordinates of a target detection box according to the target detection result; S2242: calculating the Euclidean distance between the center point of the target detection box and the center point of the minimum bounding rectangle of the door lock area to obtain the distance; S2243: calculating the intersection-over-union of the target detection box and the minimum bounding rectangle of the door lock area to obtain the overlapping area; S2244: generating a target approaching event if the distance is less than a preset threshold, and generating a target contact event if the overlapping area is greater than a preset area for N consecutive frames, where N is a positive integer greater than or equal to 3.

4. The cat eye abnormal behavior recognition method of claim 1, wherein, Step S3 comprises: S31: storing the atomic events in the event buffer in chronological order and setting a priority of the atomic events; S32: When the event buffer is full, compare the priority of the latest atomic event with the lowest priority atomic event in the event buffer, if the priority of the latest atomic event is higher, replace the lowest priority atomic event, otherwise, discard the latest atomic event; S33: When it is detected that the atomic event entering the event buffer is a sound event, adjust the sliding window to a preset long time window threshold, otherwise, use a short time window threshold, to maintain the sequence of atomic events in the sliding time window.

5. The cat eye abnormal behavior recognition method of claim 4, wherein, Step S31 includes: S311: Identify the current working mode of the cat eye system, the working mode including home mode, away mode and alert mode; S312: According to the identified working mode, adjust the priority of the atomic event, in the home mode, the priority of the sound event is improved; in the away mode, the priority of the video event is improved; in the alert mode, the priority of the door lock state event is improved; S313: Store the atomic event in the event buffer in chronological order, and set the priority of the atomic event according to the adjusted priority.

6. The cat eye abnormal behavior recognition method of claim 1, wherein, Step S4 includes: S41: Construct a state transition network including a plurality of state nodes, the state nodes including initial state, intermediate state and final state, the state nodes representing the stage of the cat eye system, the initial state representing the system startup state, and the final state representing the completion state of the abnormal behavior recognition; S42: Read the sequence of atomic events from the event buffer, analyze the sequence of atomic events according to the preset abnormal behavior mode rule of the current state node, if the preset abnormal behavior mode rule is met, perform state transition according to the state of the intelligent door lock, update the current state node, otherwise, keep the current state node unchanged; S43: Repeat S43 until the state is transferred to the final state or the sequence of atomic events in the event buffer is analyzed, if the state is transferred to the final state, it is determined that the abnormal behavior occurs, otherwise, it is determined that the abnormal behavior does not occur.

7. The cat eye abnormal behavior recognition method of claim 6, wherein, In step S42, the preset abnormal behavior mode rule includes the combination of the atomic events, the occurrence order, the time interval, the state of the intelligent door lock and the response state of the cat eye system.

8. The cat eye abnormal behavior recognition method of claim 7, wherein, Step S42 includes: S421: Read the sequence of atomic events from the event buffer, and construct a directed acyclic graph of the sequence of atomic events; S422: According to the preset abnormal behavior mode rule of the current state node, extract a candidate event chain matching the preset abnormal behavior mode rule from the directed acyclic graph; S423: For each extracted candidate event chain, calculate the matching score with the preset abnormal behavior mode rule; S424: If the matching score of the candidate event chain with the highest matching score exceeds the preset threshold, it is determined that the preset abnormal behavior mode rule is met, the state is transferred according to the state of the intelligent door lock, and the current state node is updated, otherwise, the current state node is kept unchanged.

9. A cat eye abnormal behavior recognition system characterized by, The system is applied to the steps of the method of any one of the above claims 1-8, and includes: Data acquisition module: acquire video stream, audio stream and intelligent door lock state in front of the door area; Event extraction module: extract a plurality of atomic events according to the video stream, the audio stream and the intelligent door lock state; The event storage module stores atomic events in time sequence to an event buffer and maintains a sequence of atomic events within a sliding time window; The event judgment module analyzes the sequence of atomic events in the event buffer based on preset abnormal behavior pattern rules, performs state transition and abnormal behavior discrimination; The result output module outputs the recognized abnormal behavior type, confidence and occurrence timestamp according to the final state after state transition or the discrimination result.

Images