A security event analysis method and a security event analysis system

Abstract

The application discloses a security event analysis method and a security event analysis system, and relates to the technical field of security analysis. The method comprises the following steps: acquiring a security video to be analyzed, performing structured perception by using a first model to obtain structured context information comprising a perception result of a target object in a security event; then determining a target reasoning path from a plurality of candidate reasoning paths; when the target reasoning path does not call a second model, generating an event analysis result based on the perception result, wherein the parameter scale of the second model is larger than that of the first model; when the target reasoning path calls the second model, generating input information of the second model according to the structured context information and performing semantic reasoning to obtain a semantic description result, and generating the event analysis result based on the perception result and the semantic description result. The application can match a corresponding reasoning path for a security event, reduce reasoning cost, and improve the accuracy and reliability of an event analysis result.

Description

Technical Field

[0001] This invention relates to the field of security analysis technology, and in particular to a security incident analysis method and a security incident analysis system. Background Technology

[0002] Currently, security video analytics is evolving from traditional single-model object detection to a collaborative approach combining small and large models. Small models refer to lightweight perception models with a small parameter scale, such as lightweight convolutional neural network (CNN) detectors and person re-identification (ReID) models. These excel at low-cost, high-frequency structured perception tasks, such as human detection, face recognition, and trajectory tracking. Large models refer to models with a large parameter scale, such as large visual models and large multimodal language models (LLMs), which excel at complex semantic understanding and behavioral interpretation. Furthermore, the video input sources for security video analytics systems are becoming increasingly diverse, including real-time camera streams, local video files, network-transmitted fragments, and captured image streams. Systems typically use a unified media abstraction layer to standardize heterogeneous inputs into temporal media objects with timestamps, frame indices, and device identifiers, allowing downstream AI analysis processes to consume them uniformly.

[0003] However, existing collaborative analysis schemes for large and small models are typically general collaborative frameworks that do not differentiate between application scenarios. Examples include general architectures where the small model generates structured context to drive inference in the large model, or parallel collaborative evaluation frameworks for large and small models based on a single confidence threshold. Neither is specifically designed for the actual needs of security scenarios, and they suffer from at least the following problems: the collaborative strategies lack adaptation to security scenarios, do not consider the priority differences between security events, and use the same inference path for all events. This can lead to delays for high-value events (e.g., a stranger approaching the door) because computing power is consumed by low-value events (e.g., leaves blowing in the wind). Furthermore, the input sources for security video analysis systems are becoming increasingly diverse, including real-time camera streams, local video files, network-transmitted fragments, and captured image streams. In existing technologies, different input formats typically require different parsing logics, and the lack of a frame quality evaluation mechanism that incorporates model feedback results in a lack of quality-based criteria for keyframe selection, affecting the reliability of subsequent analysis results. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a security incident analysis method and a security incident analysis system, which solves the problem that existing collaborative analysis schemes using large and small models employ the same inference path for all security incidents, leading to a mismatch between inference resource allocation and incident value, and insufficient reliability of analysis results. This invention can match appropriate inference paths to security incidents, reducing inference costs and improving the accuracy of incident analysis results. The specific solution is as follows: Firstly, this application provides a security incident analysis method, including: The security video to be analyzed is acquired, and the security video to be analyzed is subjected to structured perception using a first model to obtain structured context information; wherein, the structured context information is used to describe the security events in the security video to be analyzed, and the structured context information includes the perception results of the target objects in the security events; Based on the structured context information, the target reasoning path for the security event is determined from a set of multiple candidate reasoning paths. When the target inference path does not invoke the second model, the event analysis result of the security event is generated based on the perception result; the parameter size of the second model is larger than the parameter size of the first model; When the second model is invoked in the target reasoning path, the input information of the second model is generated according to the structured context information, and the input information is input into the second model for semantic reasoning to obtain the semantic description result of the security event. Based on the perception result and the semantic description result, the event analysis result of the security event is generated.

[0005] Secondly, this application provides a security incident analysis system, characterized in that it includes: Image capture equipment, used to acquire security videos for analysis; A server, communicatively connected to the image capture device, includes a first processor and a second processor; wherein the first processor is programmed to execute a first model to perform structured perception on the security video to be analyzed, and the second processor is programmed to execute a second model to perform semantic reasoning on the security video to be analyzed; the server is used to implement the security event analysis method as described in any one of claims 1 to 9; A user terminal, which is connected to the server, is used to receive and display the event analysis results generated by the server.

[0006] Based on the above technical solution, after acquiring the security video to be analyzed, this application first uses a first model to perform structured perception on the security video to obtain structured context information describing the security event. Since the structured context information includes the perception results of the target object in the security event, a target inference path matching the security event can be determined from multiple candidate inference paths based on the perception results of the target object in the security event, instead of using the same inference path for all security events. On this basis, when the target inference path does not call the second model, the event analysis result is directly generated based on the perception results output by the first model. Since the parameter scale of the second model is larger than that of the first model, that is, the single inference cost of the second model is higher than that of the first model, this part of the security event does not need to consume the inference resources of the second model. The analysis can be completed, avoiding the second model's reasoning resources being occupied by such events, thereby reducing the overall reasoning cost and ensuring the real-time nature of the event response. When the second model is called in the target reasoning path, the input information of the second model is generated by structured context information. That is, the second model performs semantic reasoning based on the perception results of the first model, rather than directly analyzing the original security video. On the one hand, this reduces the amount of information that the second model needs to process, and on the other hand, it allows the semantic reasoning of the second model to focus on the target object perceived by the first model, improving the relevance between the semantic description results and the security event. Furthermore, the event analysis results are generated jointly by the perception results and the semantic description results, so that the event analysis results simultaneously contain structured perception information and semantic understanding information, thereby improving the accuracy and reliability of the event analysis results. Attached Figure Description

[0007] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0008] Figure 1 A flowchart of a security incident analysis method provided in this application; Figure 2 A schematic diagram of a security incident analysis system framework provided in this application; Figure 3 This application provides a schematic diagram of the overall process for security incident analysis. Figure 4 A schematic diagram of a reasoning path determination process provided in this application; Figure 5 This application provides an architecture diagram for a security incident analysis system. Detailed Implementation

[0009] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0010] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0011] Currently, in security video analytics, the scheduling strategy for collaborative analysis of large and small models is usually centered on a single dimension, without considering the priority differences between security events. All events are treated with the same inference path, which can lead to delays for high-value events due to computing power being consumed by low-value events. This application, however, first uses a smaller-scale first model to perform structured perception of the security video to be analyzed. Then, based on the structured context information obtained from the structured perception, it matches appropriate target inference paths for the security events. Only when necessary is the larger-scale second model called for semantic inference. The perception results of the first model and the semantic description results of the second model are then fused, thereby reducing inference costs, ensuring real-time response, and improving the accuracy of event analysis results.

[0012] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0013] See Figure 1 As shown, this embodiment of the invention discloses a security event analysis method, including: Step S11: Obtain the security video to be analyzed, and use the first model to perform structured perception on the security video to be analyzed to obtain structured context information; wherein, the structured context information is used to describe the security events in the security video to be analyzed, and the structured context information includes the perception results of the target objects in the security events.

[0014] In this embodiment, the security video to be analyzed may include, but is not limited to, real-time camera streams based on Real-Time Streaming Protocol (RTSP) or Web Real-Time Communication (WebRTC), local video files in MP4 (MPEG-4 Part 14, a digital multimedia container format), network transmission fragments in Transport Stream (TS) format, and captured image streams. It is understood that, as... Figure 2 As shown, to enable the unified processing of heterogeneous security videos from different sources, the input layer can standardize these videos into temporal media objects with timestamps, frame indices, and device identifiers; that is, the security video to be analyzed can specifically be a standardized temporal media object. Furthermore, after acquiring the security video to be analyzed, frame quality assessment can be performed. The assessment content can include image underlying quality, model usability feedback, and cross-frame temporal consistency.

[0015] The first model is a perception model with a small parameter scale, which can be referred to as a small model in this embodiment; correspondingly, such as Figure 2 As shown, a model layer composed of one or more first models can be called a small model layer or a structured perception layer. The first model can specifically include human detection models, such as one or more of the following: a lightweight CNN detector, a face recognition model, a ReID cross-camera tracking model, a vehicle detection model, a package detection model, and a region state analysis model. Correspondingly, the first model is used to perform structured perception on the security video to be analyzed; that is, the first model performs low-cost, high-frequency structured perception on the standardized temporal media objects. Structured perception can specifically include human detection, face recognition, ReID cross-camera tracking, vehicle / package detection, and region state analysis. It is understandable that, due to the small parameter size and low single inference cost of the first model, it can handle more than 90% of daily inference tasks.

[0016] In one specific implementation, the output of the first model can be uniformly encoded as a StructuredContext object, such as a JSON (JavaScript Object Notation) formatted structured object; that is, the structured context information in this embodiment can specifically be the information in the StructuredContext object. Specifically, the StructuredContext can include event identifier (event_id), device identifier (device_id), time window (time_window), keyframe index (keyframe_indices), target list (target_list), area status (area_status), parcel status (parcel_state), and historical event summary; wherein, each item in the target list corresponds to a target object in a security event, and can specifically include the target object's trajectory identifier (track_id), identity candidate (person_id candidate), bounding box coordinates, trajectory direction, behavior candidate, credibility, and frame quality score.

[0017] It should be noted that the perception results in this embodiment, i.e., the information in the structured context information used to describe the detection, identification, and analysis results of the target object, may specifically include the target object's detection box coordinates, identity candidates, trajectory direction, and behavior candidates, as well as information such as the region state and package state related to the target object. In one implementation, this embodiment can also determine the credibility information of the above perception results, i.e., the credibility output by the first model for each perception result, such as the credibility field corresponding to each target object in the target list. In this way, the structured context information can describe the security events occurring in the security video to be analyzed, as well as the target objects involved in the security events and the states of each target object, so as to realize the decision of the inference path based on the perception results of the target objects.

[0018] In one specific implementation, the structured context information also includes a conflict flag field (conflict_flag) and a conflict type field (conflict_type). When the first model layer contains multiple concurrently operating perception models (e.g., a face recognition model and a ReID model), the first model layer can first perform a lightweight consistency check, including comparing whether the face recognition result matches the ReID recognition result, checking trajectory continuity, and evaluating the quality and temporal consistency of the bounding box. If the conflict is minor and the confidence level of a certain perception result is significantly higher than that of other results, the first model layer can internally resolve the conflict and output an adjusted target confidence level. If the conflict cannot be reliably resolved internally within the first model layer, for example, if the face recognition result identifies a family member A but the ReID result is significantly inconsistent with the historical trajectory of family member A, then the first model layer does not internally determine the identity, but instead outputs structured context information containing the conflict flag and conflict type, for example: { "conflict_flag": true, "conflict_type": "face_vs_reid", "face_candidate": "family_A", "reid_candidate": "unknown", "effective_confidence": 0.55 }; After receiving the structured context information with conflict markers, the scheduling engine identifies the event as a low-confidence conflict event, which can further trigger parallel verification or serial enhancement of the second model, thereby avoiding the first model layer from outputting incorrect conclusions due to unreliable internal resolution.

[0019] Step S12: Based on the structured context information, determine the target reasoning path for the security event from a set of multiple candidate reasoning paths.

[0020] In this embodiment, the candidate inference paths are pre-set inference paths distinguished by whether a second model is called and the method of calling the second model, such as paths that only call the small model. Figure 2 As shown, the process of determining the target inference path can be executed by the scheduling engine (InferenceRouter). The inference path is dynamically selected based on multi-dimensional factors, ensuring that a small model is used when necessary and a large model is used when necessary. In other words, the appropriate inference path is matched for the security event based on the actual situation of the security event reflected by the structured context information, rather than using the same inference path for all security events.

[0021] Step S13: When the target inference path does not call the second model, generate the event analysis result of the security event based on the perception result; the parameter scale of the second model is larger than the parameter scale of the first model.

[0022] It is understandable that the second model is a model with a larger parameter scale than the first model; in this embodiment, it can be referred to as a large model, such as a large visual model, a large multimodal language model, etc.; correspondingly, such as Figure 2 As shown, the model layer containing the second model can be called the large model layer or the semantic reasoning layer. When the target inference path does not call the second model, it means that the current security event can be analyzed solely based on the structured perception of the first model. In this case, the event analysis result of the security event is directly generated based on the perception result. For example, the event analysis result can be an event report or event notification output to the user, such as generating an event notification "Family member A has arrived home" based on the identity candidate and trajectory information in the perception result. Thus, since the inference process of the second model is not involved in this path, the latency of the event analysis depends only on the lightweight inference time of the first model, and the expected latency can be controlled within 50ms.

[0023] Step S14: When the second model is invoked in the target reasoning path, the input information of the second model is generated according to the structured context information, and the input information is input into the second model for semantic reasoning to obtain the semantic description result of the security event. Based on the perception result and the semantic description result, the event analysis result of the security event is generated.

[0024] In this embodiment, when the second model is invoked in the target inference path, the second model does not directly perform indiscriminate analysis on the original security video to be analyzed. Instead, it receives highly structured input information generated based on structured context information. Specifically, the input information may include keyframe sequences cropped from ROIs (Regions of Interest Cropping) with bounding boxes, textual descriptions of structured context information, historical behavior summaries, and device metadata. The second model is invoked when triggering conditions are met to perform deep semantic understanding of high-risk or low-confidence events. Therefore, it can perform semantic inference on security events based on the input information and output semantic description results of the security events. The semantic description results are usually descriptive information expressed in natural language, such as a stranger approaching the door at night and lingering for about 30 seconds, indicating suspicious behavior, suggesting that the user pay attention. Finally, the event analysis results of the security events are generated based on the perception results and the semantic description results.

[0025] In one implementation, a unified analysis of cross-device event aggregation can be performed. For example, when multiple related devices such as the front door camera, back door camera, indoor doorbell, and garage camera detect related events within the same time window, the scheduling engine packages the structured context information of each of these devices and triggers a unified second model analysis. The second model then outputs a global event summary across devices.

[0026] In this embodiment, when generating the input information for the second model based on structured context information, firstly, keyframe images containing the target object are extracted from the security video to be analyzed based on the perception results, and the image region where the target object is located is marked in the keyframe images. In this embodiment, several frames containing the target object and of high quality are selected from the security video to be analyzed as keyframe images based on the keyframe index and frame quality score of each frame in the structured context information, for example, 3 to 10 frames are selected; then, based on the detection box coordinates of the target object in the perception results, the image region where the target object is located is marked in the keyframe images. The marking method includes superimposing detection boxes on the keyframe images and / or cropping the ROI of the keyframe images according to the detection boxes. Furthermore, in order to control the amount of input data for the second model, the marked keyframe images can be low-resolution compressed images, such as JPEG (Joint Photographic Experts Group) images with a resolution of 640×360, with each frame having a data size of approximately 30 to 50 KB.

[0027] In one specific implementation, before extracting keyframe images, frame rate adaptation processing can be performed for low frame rate input scenarios or image stream input scenarios. Specifically, the video frame rate of the security video to be analyzed can be determined, and the trajectory information of the target object in the security video to be analyzed can be determined. The trajectory information of the target object is output by the first model during structured perception, including the detection box position of the target object in each frame and its timestamp. When the video frame rate is detected to be lower than a preset frame rate threshold (e.g., lower than 15 frames / second), it indicates that the time interval between adjacent frames is large, and the displacement of the target object between frames may be large. Directly extracting keyframes may result in discontinuous target object positions or incomplete behavior segments. Therefore, trajectory interpolation can be performed on the security video to be analyzed based on the trajectory information of the target object to obtain interpolated video frames. The interpolation method can be linear interpolation or motion model-based interpolation, supplementing one or more interpolated frames between adjacent original frames. Each interpolated frame contains the interpolated position of the target object and its corresponding timestamp. After trajectory interpolation is completed, keyframe images containing the target object are extracted from the interpolated video frames based on the perception results. Through the above trajectory interpolation processing, even in low frame rate input or image stream input scenarios, the temporal continuity of key frame images and the smoothness of target object positions can be guaranteed, providing high-quality images for semantic reasoning of the second model and avoiding incomplete behavior understanding or misjudgment due to low frame rate.

[0028] Furthermore, this embodiment can generate text description information based on structured context information; this text description information is used to describe security events. In this embodiment, the text description information is a textualized description of the structured context information, that is, converting the perception results in the StructuredContext into a textual form that the second model can directly understand, such as detecting that person A enters the garage, approaches the vehicle, and there are no abnormalities in the package area; the typical length of the text description information can be 500 to 2000 characters. Further, the input information may also include historical behavior summaries generated based on historical event summaries, as well as device metadata of the collected devices, so that the second model can combine historical behavior patterns and device context for judgment during semantic reasoning.

[0029] In one specific implementation, the text description information can be generated according to a predefined prompt template structure, enabling the semantic reasoning of the second model to be performed under a standardized input format. The prompt template structure may include the following components: task description, indicating the type of task the second model needs to perform; device information, including device identifier (device_id), area type (e.g., entrance, garage, courtyard), and current time period (e.g., day / night / early morning); event information, including event identifier (event_id), event level (if preliminarily determined by the scheduling engine), and time window (time_window); target list, including information for each item in the target list (target_list), including track identifier (track_id), bounding box coordinates (bbox), candidate identity (person_id), candidate behavior, and corresponding credibility; historical summary, including contextual information such as whether the same target has appeared in the recent time window and historical behavior records; and output requirements, used to constrain the output format of the second model, such as only outputting structured JSON format, including event category, risk explanation, credibility, and the cited evidence frame number. The specific rules for ROI cropping are as follows: cropping is performed centered on the target object's bounding box (bbox); the area is expanded outward by 1.5 to 2 times the bounding box to preserve the surrounding context information; for key areas such as package placement areas, door handle areas, and vehicle parking areas, ROI cropping can be performed separately; for low-resolution scenes, both panoramic frames and local ROI frames can be preserved simultaneously; track identifiers (track_id) and bounding box markers can be overlaid in the cropped image. The keyframe selection logic is as follows: for P0 and P1 events, select 3 to 10 keyframe images; for P2 events, select 3 to 5 keyframe images; for offline batch analysis, more keyframes can be selected; and when selecting keyframes, priority is given to keyframes at three time points: the event initiation stage, the target behavior occurrence stage, and the result change stage, while filtering out low-quality frames that are blurred, overexposed, or severely occluded.

[0030] Furthermore, the input information for the second model can be generated based on the annotated keyframe images and textual descriptions. That is, the input information for the second model includes the annotated keyframe images for the image modality and the textual descriptions for the text modality. Understandably, by extracting keyframe images, the second model does not need to process all video frames of the security video to be analyzed, effectively reducing the inference time and token consumption of the second model. Simultaneously, by annotating the image regions where the target object is located, the semantic inference of the second model focuses on the target object itself, avoiding interference from irrelevant areas in the image. Moreover, the textual descriptions can inject the structured perception results of the first model into the second model in text form, allowing the semantic inference of the second model to use the perception results as prior constraints and cues, thereby improving the consistency between the semantic description results and the actual situation of the security event. In this way, the second model can output semantic description results highly relevant to the security event with lower inference overhead.

[0031] When generating event analysis results, this embodiment can first perform time alignment and object alignment on the perception results and semantic description results, and detect whether there are any conflicts between the aligned perception results and semantic description results. In this embodiment, time alignment is time window alignment, which specifically involves aligning the frame interval described by the semantic description results with the time window in the structured context information, so that the perception results and semantic description results participating in the comparison correspond to the same time window; the window length of time alignment can be dynamically adapted according to the event level. For example, for P0 events, the time window can be set to ±0.5 seconds to ±1 second before and after the event occurs to ensure the sensitivity of real-time response; for P1 events, the time window can be set to ±2 seconds to ±5 seconds to cover the complete process before and after the suspicious behavior occurs; for P2 events, the time window can be set to ±5 seconds to ±10 seconds; for offline video recording analysis, the time window can be extended to the complete event segment. Object alignment, also known as object reference alignment, involves establishing a correspondence between the descriptive objects (people, vehicles, packages, etc.) mentioned in the semantic description results and the target objects corresponding to the trajectory identifiers in the perception results, ensuring that the comparison conclusions point to the same target object. The main criteria for object alignment include: trajectory identifiers (track_id), i.e., the target tracking identifiers output by the first model; the spatial overlap of detection boxes, i.e., whether the Intersection over Union (IoU) reaches a preset threshold; timestamps, i.e., confirming that the target objects appear within the same time window; ROI references, i.e., detecting whether the content described by the second model references the same ROI or keyframe image; appearance features or ReID features, i.e., calculating the similarity between the appearance feature vectors of people or vehicles; and spatial relationships, i.e., detecting whether the relative positions of the target object and key reference objects such as doors, packages, and vehicles match. Furthermore, the fusion output layer can prioritize using trajectory identifiers for fast alignment; if the semantic description results of the second model do not directly output trajectory identifiers, secondary alignment is performed using keyframe numbers, detection box positions, target description information, and ReID feature similarity to ensure the robustness of the alignment. After completing time alignment and object alignment, we check whether there is a conflict between the two, that is, whether the conclusions of the first model and the second model are contradictory for the same target object within the same time window.

[0032] In one specific implementation, when there is no conflict between the perception results and the semantic description results, the perception results and the semantic description results are merged to generate an event analysis result. That is, when the two conclusions are consistent, the structured perception results and the natural language form of the semantic description results are merged into a unified event analysis result. For example, the perception results are used as the structured fields of the event analysis result, and the semantic description results are used as the semantic summary of the event analysis result.

[0033] In another specific implementation, when a conflict exists between the perception result and the semantic description result, the conflict type is determined, and the perception result and semantic description result are compared according to the conflict type to obtain a comparison result. An event analysis result is then generated based on the comparison result. For example, conflict detection and resolution can employ a hierarchical arbitration mechanism, specifically including: conflict type judgment, which may include identity conflict, behavior conflict, object conflict, and risk level conflict; source reliability comparison, where the reliability of the first model side can be measured based on detection credibility and trajectory stability, and the reliability of the second model side can be measured based on the consistency between the frame index referenced by the semantic description result and the actual keyframe, as well as the contextual consistency of the semantic description result; and weighted fusion or threshold adjudication is performed based on the comparison result to generate the event analysis result.

[0034] In one specific implementation, source reliability comparison can be achieved through a quantitative scoring method. The reliability score of the first model can be calculated by weighting parameters such as the target credibility after correction by the correction factor and conflict penalty factor, the stability of the target object's recognition results across consecutive frames, the image quality score of the keyframes, the known reliability coefficients of the currently used first model version, and the continuity of the target object's trajectory. The reliability score of the second model can be calculated by weighting parameters such as the consistency between the keyframe numbers referenced in the semantic description results and the actual input keyframes, the logical consistency between the semantic description results and the structured context information, the credibility of the second model's own output, and the level of detail of specific evidence (e.g., reference to specific frame numbers or specific detection boxes) in the semantic description results. Furthermore, the weights in the weighted decision-making stage can be dynamically adjusted according to the conflict type. final_score=w_small×small_model_result_score+w_large×large_model_result_score; Here, w_small and w_large are the fusion weights of the results from the first and second models, respectively, with w_small + w_large = 1. For identity-related conflicts (e.g., inconsistencies between face recognition and ReID results, or conflicts with the semantic description of the second model), if the credibility of the first model is high (e.g., not lower than 0.85), then w_small takes a larger value (e.g., 0.7), retaining the identity conclusion of the first model; if the credibility of the first model is low (e.g., lower than 0.6), then w_small takes a smaller value (e.g., 0.3), prioritizing the judgment of the second model. For complex behavior understanding-related conflicts (e.g., the first model only detects people lingering, while the second model identifies abnormal loitering behavior), then w_large takes a larger value (e.g., 0.7), prioritizing the semantic analysis of the second model. For target existence-related conflicts (e.g., the first model detects a target but the second model does not mention it in its description), then w_small takes a larger value (e.g., 0.8), prioritizing the detection result of the first model. For conflicts in risk level interpretation (e.g., the simple rule judgment of the first model and the semantic understanding of the second model have different assessments of risk level), w_large is taken as a larger value (e.g., 0.65), and the comprehensive semantic understanding of the second model is taken as the standard.

[0035] In this way, different weight allocations can be applied to different conflict types, ensuring that the basis for the ruling matches the characteristics of the conflict scenario, thereby improving the rationality of the ruling. For example, when the first model's face recognition confidence is 0.95, the lighting is good, and the trajectory is stable, while the second model only judges that the person might be a stranger based on a vague description, the ruling prioritizes retaining the identity conclusion of the first model. When the first model's face recognition confidence is 0.55 and there is a ReID conflict, while the second model consistently points out inconsistencies in gait and body shape using five keyframes, the ruling increases the judgment weight of the second model, outputting "Identity pending confirmation, suspected not to be the person." The above conflict ruling rules can be shown in Table 1 below: Table 1 Conflict Adjudication Rules

[0036] Furthermore, when neither the perception result nor the semantic description result meets the preset reliability conditions, the event analysis result is set to a pending confirmation state so that the preset server can review the security event. The preset reliability conditions are conditions constructed based on the credibility of the perception result and the semantic description result. For example, the preset reliability condition can be that the credibility is not lower than the corresponding reliability threshold. That is, when the credibility of both the perception result and the semantic description result is lower than their respective reliability thresholds, both parties fail to meet the preset reliability conditions and cannot make a decision. In this case, a definitive conclusion is not forcibly output, but the event analysis result is set to a pending confirmation state. The preset server can be a cloud server, so the cloud server can review the security event or further prompt the user to confirm the security event.

[0037] Understandably, the aforementioned time alignment and object alignment ensure that conflict detection occurs on the same target object within the same time window, avoiding misjudging conclusions from different time periods or different objects as conflicts. Furthermore, layered adjudication based on conflict type allows for different adjudication criteria to be applied to conflicts of different natures. For example, in identity-related conflicts, the first model's face recognition and ReID-like structured perception are more effective, so identity conclusions are retained when the first model has high confidence. Conversely, in behavior-related conflicts, the second model is more effective in semantic understanding of multi-frame behaviors, so its weight is increased when the first model has low confidence. This ensures that the adjudication result is a more reliable model output in different conflict scenarios, ultimately resulting in an adjudicated conclusion for the event analysis. Additionally, when neither party meets the preset reliability conditions, a pending confirmation state prevents the output of erroneous conclusions; instead, a preset server verifies the results, further guaranteeing the reliability of the event analysis results.

[0038] Based on the above embodiments, in one specific implementation, after generating the event analysis result of the security event, an event evidence chain for the security event can also be generated. The event evidence chain includes the identification information of the security video to be analyzed, the perception result, the target inference path, and the event analysis result. Correspondingly, when the target inference path calls the second model, the event evidence chain also includes input information and semantic description results. In this implementation, the identification information of the security video to be analyzed can specifically include the input media object identifier and keyframe identifier. Furthermore, the event evidence chain can also record the frame extraction strategy and its parameters, the version information of the first model and its complete output, the version information of the fusion rules, the conflict handling result, and the final notification content and its credibility. When the target inference path calls the second model, the event evidence chain also records the input context (i.e., input information) of the second model and the text output by the second model (i.e., semantic description results). It should be noted that regardless of whether the target inference path calls the second model, a corresponding event evidence chain is generated each time the analysis process outputs the event analysis result. In this way, the input and output of each link from the original media object to the event analysis result can be recorded through the event evidence chain, allowing users to trace back from the event analysis result to the original video frame step by step, thereby seeing the complete reasoning basis for why the system determined that this was a suspicious behavior; compared with the black box solution that only outputs the final conclusion, the analysis process of this embodiment has interpretability and evidence integrity in security scenarios, and the event analysis result is auditable.

[0039] Based on the above technical solution, see Figure 2 The security incident analysis system framework shown includes an input layer, a small model layer (structured perception layer) composed of a first model, a scheduling engine, a large model layer (semantic reasoning layer) composed of a second model, and a fusion output layer. The small model layer is the first-level reasoning node in the entire analysis process. Its output StructuredContext serves as the input factor for the scheduling engine's path decision-making and also as the structured constraint context when the second model is triggered. The scheduling engine receives the StructuredContext and frame quality score output from the small model layer. Its decision results control the triggering of the large model layer and the selection of reasoning paths, making it the core hub for unified scheduling of cost efficiency, real-time performance, and reliability. The large model layer is triggered on demand by the scheduling engine, performing deep semantic understanding only on high-risk or low-confidence security incidents. Its semantic output and the structured output of the first model are aligned and conflict-resolved at the fusion output layer to form a unified conclusion. The fusion output layer is the final convergence node of the entire analysis process. The results of all upstream reasoning paths are processed uniformly here. The semantic output of the second model and the structured output of the first model are aligned and conflict-resolved at the fusion output layer to form a unified conclusion, ultimately outputting the incident analysis results and the incident evidence chain.

[0040] For example, this embodiment can adopt a three-tiered deployment scheduling architecture of end-edge-cloud. The first model is deployed entirely on the edge chip of the device, the intermediate model with parameter scale between the first and second models is deployed on the home gateway, and the second model is deployed in the cloud. The scheduling engine automatically selects the inference node according to network conditions and latency requirements. When the network is connected, the second model in the cloud is called. When the network is disconnected, the intermediate model on the gateway side is used as a backup. In extreme cases, only the first model on the end side is used to complete the analysis. In this way, the continuity of security event analysis services can be guaranteed in deployment environments with unstable network conditions. Furthermore, this embodiment can introduce cascading triggering logic for sparse activation of the second model. That is, the first model first filters out suspicious frames with a confidence level below a threshold and only sends suspicious frames to the intermediate model. After further filtering by the intermediate model, the second model is only triggered when it is determined to be highly suspicious. Through three-level cascading, the call rate of the second model can be further reduced to below 1%, thereby further compressing the inference cost in large-scale deployment scenarios.

[0041] And see Figure 3 As shown, the security event analysis method in this embodiment may include the following steps: Step 1, First model prior: Regardless of whether the second model is triggered subsequently, all media objects entering the analysis process must first undergo structured perception processing of the small model layer. The StructuredContext output by the small model layer is both the core input of the scheduling decision and the semantic constraint context of the second model; Step 2, Multi-dimensional scheduling decision: The scheduling engine selects the inference path based on multi-dimensional factors such as event level, credibility, SLA (Service Level Agreement) response time constraints, and inference cost, according to the preset scheduling strategy table; Step 3, Hierarchical triggering of the second model: The second model is only triggered when preset triggering conditions are met (e.g., high-risk events and low confidence of the first model, high-risk events requiring natural language summarization, user-initiated semantic analysis, etc.). In typical scenarios, the invocation rate of the second model is less than 5%; Step 4, Layered conflict resolution: The output results of the first model and the second model are aligned in time and objects to detect and resolve conflicts between them; Step 5, Evidence chain generation: Each output records the complete inference chain from the original media object to the final conclusion.

[0042] In summary, after acquiring the security video to be analyzed, this embodiment first uses a first model to perform structured perception on the security video to obtain structured context information describing the security event. Since the structured context information includes the perception results of the target object in the security event and the credibility of the perception results, when determining the target inference path from multiple candidate inference paths, it can determine the credibility of the event content in the perception results, thereby determining the target inference path that matches the security event, rather than using the same inference path for all security events. Based on this, when the target inference path does not call the second model, the event analysis result is directly generated based on the perception results output by the first model. Since the parameter scale of the second model is larger than that of the first model, that is, the single inference cost of the second model is higher than that of the first model, this part of the security event does not need to consume the second model. The analysis can be completed using the inference resources of the second model, avoiding the occupation of the second model's inference resources by such events, thereby reducing the overall inference cost and ensuring the real-time nature of the event response. When the second model is called in the target inference path, the input information of the second model is generated by structured context information. That is, the second model performs semantic inference based on the perception results of the first model, rather than directly analyzing the original security video. On the one hand, this reduces the amount of information that the second model needs to process, and on the other hand, it allows the semantic inference of the second model to focus on the target object perceived by the first model, improving the relevance between the semantic description results and the security event. Furthermore, the event analysis results are generated jointly by the perception results and the semantic description results, so that the event analysis results contain both structured perception information and semantic understanding information, thereby improving the accuracy and reliability of the event analysis results.

[0043] Based on the above embodiments, the following embodiments of this application further describe the process of determining the target reasoning path. In one specific implementation, see... Figure 4 As shown, the process of determining the target reasoning path described above may include: Step S21: Determine the event level of the security event based on the structured context information; wherein the event level is used to characterize the processing priority of the security event.

[0044] In this embodiment, the event level of a security event can be determined based on the perception results in the structured context information. For example, the security event can be determined as the corresponding event level based on information such as whether the target object's identity candidate matches a registered person, the behavior category of the behavior candidate, the area status, and the package status. In a specific implementation, the event level can be divided into four levels, P0 to P3, and the quantification rules for each event level are shown in Table 2 below: Table 2 Event Level Quantification Rules

[0045] It should be noted that the above event level classification method reflects the priority logic for security scenarios in this embodiment: P0 level events related to personal safety are always scheduled and allocated computing resources first to ensure that critical security scenarios are not crowded out by low-value events; while P3 level events such as wind blowing leaves, changes in light, and known family members entering and leaving are only processed by the first model and do not occupy the inference resources of the second model.

[0046] It should be noted that the event level is determined by the scheduling engine based on the perception result fields in the structured context information. The first model layer only outputs structured perception facts (such as target type, behavior candidate, region type, dwell time, etc.) and candidate behavior labels, and does not directly determine the final event level. The scheduling engine determines the P0 to P3 levels based on the following combinations of factors: For P0 events, the determination rules include detecting violent behavior, forced entry (behavior_candidate is forced_entry and area type is door or window), fall detection, distress calls, weapon detection, or scenarios involving personal safety such as children being in dangerous areas; For P1 events, the determination rules include strangers loitering at the door at night (person_type is stranger, area_type is door_close, duration is greater than 30 seconds, time_period is night), packages being taken, abnormal vehicle movement, or strangers entering restricted areas; For P2 events, the determination rules include unknown personnel passing by (unmatched personnel passing through the monitored area), vehicles entering, animals appearing, or brief stays in low-risk areas; For P3 events, the determination rules include known family members entering and leaving, environmental changes such as leaves blowing in the wind, changes in light, or low-value background movement.

[0047] Furthermore, the scheduling engine also features a dynamic escalation and de-escalation mechanism for event levels. When the perception result of a target object in the structured context information changes, the scheduling engine adjusts the current event level of the security event based on the changed information. For example, if the event duration increases, the target behavior escalates from normal behavior to suspicious behavior, or the target moves from a low-risk area to a high-risk area, the scheduling engine escalates the event level from P2 to P1, or from P1 to P0. Conversely, if the target is identified as a registered person or authorized visitor in a subsequent frame, the scheduling engine de-escalates the event level from P1 to P3 or P2. By dynamically adjusting the event level, the priority of security events can change with the evolution of the scenario, avoiding insufficient inference resource allocation due to an initial low judgment level, or missed detections or delays due to high-priority events failing to obtain sufficient inference resources in a timely manner after the scenario changes. For example, the scheduling engine adjusts the event level triggering conditions based on changes in the perception results of the target object, including but not limited to: if the target object stays in the monitored area for more than a preset time threshold, the scheduling engine upgrades the event level from P2 (general security event) to P1 (high-value asset / privacy event); if the target object enters a high-risk area from a low-risk area (e.g., from a gate to the entrance area), the scheduling engine upgrades the event level from P3 or P2 to P1; if the target object's behavioral candidate label changes from "passing by" to "loitering" or "forcibly entering," the scheduling engine upgrades the event level to P0 or P1. Conversely, if the face recognition result successfully matches a registered person in a subsequent frame, the scheduling engine downgrades the event level from P1 or P2 to P3; if the target object leaves the monitored area for more than a preset time threshold, the scheduling engine reduces the processing priority of the event. Furthermore, it can be understood that the event level adjustment results can be fed back to the scheduling engine as a basis for subsequent inference path re-decision, enabling the scheduling strategy to adapt to real-time changes in the security scenario.

[0048] Step S22: Obtain the credibility information of the perception result based on the structured context information, and determine the target credibility of the perception result.

[0049] In this embodiment, the credibility information of the perception result can be obtained based on the structured context information, such as the credibility field in the target list of the structured context object in the above embodiment, thereby determining the target credibility of the perception result. In a specific implementation, when determining the target credibility, the quality assessment information of the security video to be analyzed can be obtained; wherein, the quality assessment information includes one or more of image quality information, illumination information, occlusion information, temporal stability information, and acquisition device reliability information; correction parameters are determined based on the quality assessment information, and the credibility information is corrected using the correction parameters to obtain the target credibility of the perception result.

[0050] For example, the target confidence level can be the effective confidence level (effective_confidence), and the correction parameters can specifically be correction factors corresponding to various types of quality assessment information. The original confidence level (model_confidence) output by the first model is corrected by the cascade of correction factors to obtain the target confidence level (effective_confidence). The formula for calculating the target confidence level can be: effective_confidence=model_confidence×image_quality_factor×lighting_factor×occlusion_factor×temporal_stability_factor×device_reliability_factor; Among them, image_quality_factor is the image quality factor corresponding to image quality information, with a value range of 0.5 to 1.0, and a lower value is taken when the image quality is poor; lighting_factor is the lighting factor corresponding to illumination information, with a value range of 0.6 to 1.0, and a lower value is taken when there is low light or overexposure; occlusion_factor is the occlusion factor corresponding to occlusion information, with a value range of 0.3 to 1.0, and a significant reduction is taken when the target object is severely occluded; temporal_stability_factor is the temporal stability factor corresponding to temporal stability information, with a value range of 0.4 to 1.0, and a lower value is taken when the recognition result is unstable between consecutive frames. For example, this factor can be determined based on the consistency of the recognition results within N frames (N is typically 30 frames) of cross-frame buffer; device_reliability_factor is the device reliability factor corresponding to the reliability information of the acquisition device, with a value range of 0.7 to 1.0, and a moderate reduction is taken when the acquisition device is a low-end device.

[0051] Furthermore, when the structured context information carries a conflict marker, a conflict penalty factor, `conflict_penalty_factor`, can be introduced on top of the target confidence level to obtain the final target confidence level, `final_confidence`. final_confidence=effective_confidence×conflict_penalty_factor; conflict_penalty_factor=1-conflict_score; Here, `conflict_score` is a conflict score characterizing the degree of conflict between various perception results. It can be calculated from one or more factors, including inconsistency between face recognition and ReID results, cross-frame identity switching frequency, instability of target detection boxes, and trajectory continuity. For example, `conflict_score = 0.4 × face_reid_mismatch + 0.3 × temporal_identity_switch + 0.2 × bbox_instability + 0.1 × low_track_continuity`. When there is no conflict, `conflict_penalty_factor` is close to 1, and the target confidence is unaffected. When the conflict is severe, the target confidence is significantly reduced, and the scheduling engine identifies this event as a low-confidence event, triggering a second model verification or outputting a pending confirmation state. This avoids making incorrect path decisions based on the confidence of the first model when multiple perception results are inconsistent.

[0052] Understandably, the raw confidence level output by the first model only reflects the model's confidence in the judgment result and does not consider the quality of the input data itself. When the image quality is poor, the lighting conditions are inadequate, the target object is occluded, the recognition result fluctuates in time, or the reliability of the acquisition equipment is low, the model may assign a high confidence level to a result obtained based on low-quality input. Therefore, this embodiment can use correction parameters corresponding to the quality assessment information to correct the raw confidence level, so that the target confidence level reflects both the model's judgment confidence and the reliability of the input data. Using the target confidence level rather than the raw confidence level as the basis for path decision-making can avoid incorrectly selecting lightweight inference paths under low-quality input conditions, thereby reducing missed detections and misjudgments and improving the reliability of path decision-making.

[0053] Step S23: Obtain the response time limit requirements for the security event and the inference cost of each candidate inference path.

[0054] In this embodiment, the response time limit requirement is the upper limit of the allowed time for the analysis process of security events. Specifically, it can be determined according to the Service Level Agreement (SLA), which clearly specifies the minimum performance level and indicators that the service must achieve. If the service fails to meet these standards, a corresponding remedial mechanism is usually triggered. In one specific implementation, the response time limit requirement can be divided into three categories: real-time notification (no more than 300ms), near real-time summary (no more than 2000ms), and offline analysis (no time limit). The inference cost is an estimated value of the resource overhead of each candidate inference path, which can specifically include one or more of the following: estimated token consumption, inference duration, and current queue load.

[0055] It should be noted that in this embodiment, there is a priority relationship among event level, response time limit requirement, target credibility, and inference cost. In the security scenario, the priority order is: event level > response time limit requirement > target credibility > inference cost. Among them, the security event priority (P0) has the highest priority. For example, when the response time limit requirement of a high-priority event (e.g., P0 event) (e.g., no more than 300ms) conflicts with the inference latency required to call the second model (e.g., 800ms), that is, the inference of the second model may exceed the response time limit requirement specified by the SLA, the scheduling engine adopts a two-stage response: the first stage is an immediate response, that is, within the response time limit requirement, an emergency notification is issued based on the perception results and security alarm template output by the first model, and video recording markers or local audio-visual alarms are activated to ensure that the emergency is not delayed; the second stage is a latency enhancement, that is, after the immediate response of the first stage is completed without interference, the second model is asynchronously called in the background to perform semantic inference on the stored keyframes and structured context information, generate detailed event explanations, risk summaries and evidence supplements, and update the event analysis results and event evidence chain after the second model inference is completed. In other words, for P0 events, the response is not delayed until the second model completes inference; instead, it responds first and then enhances the response. This satisfies the real-time SLA constraint while achieving deep semantic understanding through the second model. For example, if a suspected forced entry is detected, the small model immediately triggers a P0 alarm, while the background calls the large model to analyze keyframes. After the large model completes its analysis, it updates the event summary and evidence chain in the client. Furthermore, for P1 or P2 events, if the estimated inference time of calling the second model may exceed the response time limit, the scheduling engine can, based on the inference cost score and the current queue load, choose to output only the first model, use a medium-sized model (with parameter sizes between the first and second models) for rapid supplementary analysis, or enter asynchronous summary mode (i.e., output the first model's results first, and then update the semantic summary asynchronously in the background).

[0056] Step S24: Determine the target inference path for the security event from multiple candidate inference paths based on the event level, the target credibility, the response time limit requirement, and the inference cost.

[0057] In one specific implementation, a scheduling strategy table can be pre-built. This table records the correspondence between combinations of conditions such as event level, target credibility, response time limit requirements, and inference cost, and their corresponding inference paths. The scheduling engine makes path decisions based on this scheduling strategy table. For example, the core rules of the scheduling strategy table can be shown in Table 3 below: Table 3 Scheduling Strategy Table

[0058] For inputs in the form of low frame rates or image streams, keyframe completion and trajectory interpolation can be performed first to restore temporal continuity. Then, the first model performs structured perception, and the second model is triggered as needed. For offline MP4 files or cloud recordings, since latency is not critical, the first model can process them in batches before the second model is triggered for analysis, thus scheduling the inference task of the second model to be executed during periods of low system load. For example, for inputs in the form of low frame rates or image streams, when the input source is a low frame rate video (e.g., frame rate less than 15 frames / second) or an image stream (e.g., a sequence of JPEG images captured at second-level intervals), the first model outputs the detection box positions of the target objects in each frame during the structured perception process. The scheduling engine determines the trajectory information of the target objects based on the detection box positions and determines whether the current frame rate is lower than a preset frame rate threshold. If the frame rate is below a preset threshold, the scheduling engine triggers a trajectory interpolation process. This involves interpolating the position of the detection box at the intermediate moment between two temporally adjacent original frames, based on the displacement of the target object's detection box center coordinates between adjacent frames, and generating interpolated frames. The number of interpolated frames can be determined based on the ratio of the original frame rate to a preset frame rate threshold. After interpolation, if the frame rate of the security video to be analyzed reaches or exceeds the preset frame rate threshold, frames containing the target object and meeting the quality requirements can be selected from the interpolated video frames as keyframe images. Thus, this embodiment is compatible with video input from low-cost acquisition ports such as low-frame-rate cameras or timed capture devices, ensuring the quality of keyframes required for semantic reasoning in the second model without increasing hardware acquisition costs.

[0059] Furthermore, when making path decisions based on the scheduling strategy table, the inference cost constraints of each candidate inference path can also be considered, such as the estimated token consumption required to call the second model, the model inference duration, and the current inference queue load. The scheduling engine can calculate the cost score (inference_cost_score) for each candidate inference path based on the above factors. inference_cost_score=token_cost_estimate+model_latency_estimate+queue_load_factor+user_plan_cost_factor; Here, `token_cost_estimate` represents the cost level corresponding to the estimated number of tokens required to call the second model for semantic inference; `model_latency_estimate` is the estimated time required for the second model's inference; `queue_load_factor` is the load level corresponding to the number of tasks pending in the current inference queue; and `user_plan_cost_factor` is the cost budget factor corresponding to the user's subscribed service package. Therefore, the scheduling engine can jointly determine whether to call the second model, and what scale of second model to call, based on the event level, target credibility, response time requirements, and inference cost score. For example, for a P3 event where the first model has high credibility and the current second model queue is busy, even if the inference cost score is low, the scheduling engine will still prioritize the first model-only path to avoid wasting inference resources on low-value events. For a P2 event where the target credibility is medium and the user has not subscribed to the advanced semantic summarization service, the first model output can be combined with a short template summary without triggering the second model. For a P1 event where the target credibility is below the threshold and the cost budget is sufficient, the scheduling engine will trigger the second model for supplementary analysis after the first model. P0 events are not subject to ordinary cost budget constraints and are forced to enter the highest priority inference path, which means that the second model must be called for semantic enhancement. In this case, the cost only affects the size of the second model selected or whether to output a template alarm first and then perform asynchronous supplementary analysis, without affecting whether to respond to the security event.

[0060] As shown above, in determining the target inference path in this embodiment, the event level represents the processing priority of the security event, reflecting the value of the event to the security target; the target credibility represents the reliability of the first model's perception result, reflecting whether the event needs further confirmation or enhancement by the second model; the response time limit requirement represents the upper limit of the allowed time consumption of the event's analysis process, constraining the latency range of the selectable inference path; and the inference cost represents the resource overhead of each candidate inference path, constraining the cost of path selection. Decision-making based on these dimensions ensures that high-priority security events receive more sufficient inference resources, security events with insufficient perception reliability are confirmed or enhanced by the second model, the analysis time of security events does not exceed their response time limit requirement, and the consumption of inference resources matches the value of the security event. Compared to scheduling methods based solely on a single credibility threshold, the dynamic scheduling in this embodiment enables precise model scheduling. High-value events will not experience delays due to low-value events occupying computing power, and low-value events will not consume the inference resources of the second model, thereby reducing the overall inference cost while ensuring analysis quality.

[0061] Furthermore, in one specific implementation, the multiple candidate inference paths may include a first inference path, a second inference path, and a third inference path. The first inference path is a path that generates event analysis results based on perception results without calling the second model. It can also be called the small model-only path and is suitable for security events with low event levels and reliable perception results from the first model, such as P3 events with high target credibility. The second inference path is a path that serially calls the second model for semantic reasoning after obtaining structured context information. It can also be called the serial enhancement path. That is, the first model completes structured perception first, and then the input information generated based on the structured context information is given to the second model for semantic enhancement. It is suitable for security events that require deep semantic understanding, such as P0 events, P1 events with target credibility below the threshold, and P1 / P2 events that require the generation of natural language summaries. The third inference path is a path that calls the second model in parallel to verify the perception results. It can also be called the parallel verification path. That is, while generating preliminary analysis results based on perception results, the input information is given to the second model for analysis in parallel. The semantic description results of the second model are used to verify the perception results of the first model. It is suitable for security events where the perception results of the first model are ambiguous (e.g., identity ambiguity).

[0062] Understandably, the first inference path handles the most frequent, routine events with the lowest cost and latency; the second inference path enables the semantic reasoning of the second model to be based on the structured priors of the first model, achieving deep semantic understanding; the third inference path provides verification for ambiguous perception results through independent analysis of the second model, and since the second model executes in parallel, the verification process does not affect the output of the perception results. In this way, the smaller model can handle more than 90% of the daily inference tasks (detection, tracking, basic recognition), while the larger model only intervenes when necessary, significantly reducing inference costs. In typical scenarios, the second model's call rate is less than 5%, while in existing large-small model collaboration solutions, the call rate of the larger model is generally between 20% and 50%. Therefore, this embodiment can use an event-level hierarchical scheduling strategy to precisely concentrate the second model's inference resources on high-value events, significantly reducing inference costs.

[0063] To facilitate understanding of the technical solution of this application, the following uses a smart doorbell security scenario as an example to illustrate the security event analysis method provided in this application. In this application scenario, the smart doorbell device includes a camera and a microphone, is deployed at the door of the home, and is connected to the home gateway via WiFi; the system receives RTSP real-time video streams, and performs collaborative inference between large and small models after standardization by the unified media abstraction layer (input layer).

[0064] In one specific embodiment, a first inference path (daily family member entry and exit) is adopted. Family member A returns home normally during the day. The first model completes human detection (confidence 0.95), face recognition (person_id matches family member A), and trajectory tracking (normal path from the yard gate to the door). Due to sufficient daytime lighting, good image quality, no occlusion, and stable recognition results across consecutive frames, all correction factors are close to 1.0, resulting in high target confidence. The scheduling engine evaluates: event level is P3 (known family member entry and exit), target confidence is high, response time requirement is real-time (no more than 300ms), and inference cost is acceptable. The decision result is the first inference path, i.e., only the first model path, with an analysis latency of less than 50ms. The system pushes a notification "Family member A has arrived home," without triggering the second model throughout the process. The event evidence chain records the input frame identifier, the first model version, the recognition result, and its confidence.

[0065] In another specific embodiment, a second inference path (stranger loitering at the door) is adopted. Stranger B appears in the doorway area at 10 PM. The first model completes human detection (confidence 0.72), face recognition (no matching person_id), and area analysis (entering the care area and staying for more than a preset threshold). Due to poor lighting conditions at night, the lighting factor further reduces the original confidence, and the target confidence drops below the threshold of 0.7. The scheduling engine evaluates: the event level is P1 (loitering at the doorway, nighttime, stranger), the target confidence is low to medium, and the response time limit allows for enhancement by the second model (no more than 2000ms). The decision result is the second inference path, i.e., the serial enhancement path. The system generates input information based on structured context information: 5 keyframe images with labeled detection boxes and ROI cropping, and a textual description of StructuredContext; after semantic inference, the second model outputs a semantic description result: "A stranger approached the doorway at night and lingered for about 30 seconds, the behavior is suspicious, and it is recommended that the user pay attention." After fusion, the system pushes an alarm notification with semantic summary, and the overall analysis latency is about 800ms. The event evidence chain fully records the inputs, outputs, and fusion process of the first and second models.

[0066] In another specific embodiment, a third inference path and conflict resolution (identity ambiguity) are employed. Person C appears at the door. The first model's facial recognition matches Person C as family member D (confidence 0.65), but the ReID features differ from family member D's historical template. The scheduling engine evaluates: the event level is P2, the target confidence is low, and the identity is ambiguous. The decision result is the third inference path, i.e., the parallel verification path: the first model outputs preliminary results according to the normal process, while the second model receives the labeled keyframe images and text description information in parallel, performs independent identity and behavior analysis, and outputs the semantic description result: "The person's facial features differ from family member D, and the gait and body shape are inconsistent, possibly not family member D." After time alignment and object alignment, the fusion output layer detects an identity conflict and initiates hierarchical arbitration: the first layer determines the conflict type as an identity conflict; the second layer compares the reliability of the sources. The confidence of the first model is 0.65, which is low, while the 5 keyframes referenced by the second model consistently describe inconsistent features, indicating high contextual consistency; the third layer prioritizes the second model's judgment, and finally outputs "The identity of the person at the door is pending confirmation, suspected to be not family member D." The chain of evidence for an event is marked with the type of conflict, the basis for the ruling, and the final conclusion.

[0067] In the above application scenario, for example, the parameters can be set as follows: The first model includes a human detection model (lightweight CNN, inference latency 10-30ms), a face recognition model (inference latency 15-40ms), and a ReID model (inference latency 10-20ms); the second model trigger condition for P0 events in the scheduling engine is unconditional triggering, and the low confidence trigger threshold for P1 events is a target confidence level of less than 0.7; the typical value of the number of frames N in the cross-frame buffer is 30 frames; the typical input of the second model is 3-10 frames of low-resolution JPEG images with a resolution of 640×360 (approximately 30-50KB per frame), and text description information of approximately 500-2000 characters.

[0068] Therefore, in this embodiment, the second model is invoked zero-time in routine scenarios, with an analysis latency of less than 50ms; the second model is precisely intervened in suspicious scenarios, with an analysis latency of approximately 800ms, meeting the response time requirements for near real-time alarms; in ambiguous scenarios, conflicts between the first and second models are correctly detected and adjudicated, avoiding the output of incorrect conclusions to the user; and the event evidence chain is complete and traceable. Thus, the security event analysis method in this embodiment can balance cost-effectiveness, real-time performance, output reliability, and auditability in security scenarios with high real-time requirements, such as smart doorbells.

[0069] See Figure 5 As shown in the embodiments, this application also discloses a security incident analysis system, including: Image capture device 11 is used to acquire security videos to be analyzed; Server 12 is communicatively connected to the image capture device. The server includes a first processor and a second processor. The first processor is programmed to execute a first model to perform structured perception on the security video to be analyzed, and the second processor is programmed to execute a second model to perform semantic reasoning on the security video to be analyzed. The server is used to implement the security event analysis methods in the above embodiments. User terminal 13, which is connected to the server, is used to receive and display the event analysis results generated by the server.

[0070] Specifically, the image capture device 11 can be a camera device deployed in the security monitoring area, such as a network camera or a smart doorbell camera. There can be one or more image capture devices 11, deployed in locations requiring monitoring, such as the front door, yard, garage, and indoor corridor. Each image capture device 11 establishes a communication connection with the server 12 via a wired or wireless network, transmitting the acquired real-time video stream or recorded video files to the server 12. Furthermore, the security video to be analyzed acquired by the image capture device 11 includes one or more of the following: real-time camera streams, local recorded video files, network-transmitted segments, and captured image streams.

[0071] Server 12 can be a local server, an edge computing device (such as a home gateway or network video recorder), or a cloud server or cloud server cluster deployed in the cloud; this application does not specifically limit it in this regard. Server 12 can process video streams from image capture device 11 in real time, or it can perform offline batch processing of stored video files after image capture device 11 has finished recording.

[0072] User terminal 13 can be a terminal device with display and interaction capabilities, such as a smartphone, tablet, personal computer, or smart wearable device. User terminal 13 runs security application software that works in conjunction with server 12. This application software can receive event analysis results pushed by server 12 and display these results to the user in the form of notification messages, event lists, and event detail pages. User terminal 13 is also used to receive query commands, confirmation commands, or configuration commands input by the user and send these commands to server 12 so that server 12 can perform corresponding response operations based on the user commands (such as re-analyzing security events, adjusting event level judgment rules, or updating inference path scheduling parameters).

[0073] Based on the above system architecture, in this embodiment, the image capture device first acquires the security video to be analyzed within the monitored area and transmits the video to the server. In multi-device deployment scenarios, each image capture device pushes its acquired video stream to the server in real time, and the server performs independent analysis and processing on each video stream.

[0074] After receiving the security video to be analyzed, the server first standardizes the security videos from different image capture devices and different protocol formats through the input layer, converting them into temporal media objects with timestamps, frame indexes, and device identifiers. Subsequently, the server executes the first model (i.e., a lightweight perception model with a small parameter scale) through the first processor to perform structured perception on the standardized temporal media objects and output structured context information.

[0075] After obtaining structured context information, the server's scheduling engine determines the target inference path for the security event from a set of pre-defined candidate inference paths. The scheduling engine comprehensively evaluates the event level, target credibility, response time requirements, and the inference cost of each candidate path, and determines the target inference path that matches the current security event based on a pre-defined scheduling strategy table. If the target inference path does not invoke the second model (i.e., a large model with a larger parameter scale than the first model, such as a large visual model or a multimodal large language model), the server directly generates the event analysis result of the security event based on the perception result output by the first model and pushes the event analysis result to the user terminal. Under this path, the second model is not triggered, and the event analysis latency depends only on the lightweight inference time of the first model. The expected latency can be controlled within 50ms, suitable for low-value routine events or events where the perception result of the first model is highly reliable.

[0076] If the target inference path calls the second model, the server generates input information for the second model based on structured context information. Specifically, the server extracts keyframe images containing the target object from the security video to be analyzed based on the perception results, and annotates the image region where the target object is located in the keyframe images; simultaneously, the server generates text description information based on structured context information; subsequently, the server combines the annotated keyframe images and text description information as input information for the second model. The server inputs the input information into the second model, which is then executed by the second processor to perform semantic inference. The second model performs a deep semantic understanding of high-risk or low-confidence events based on the keyframe images and text description information, and outputs a semantic description of the security event.

[0077] After the server obtains the semantic description results output by the second model, the fusion output layer performs time and object alignment on the perception results of the first model and the semantic description results of the second model to ensure that the perception results and semantic description results involved in the comparison correspond to the same target object within the same time window. After alignment, the fusion output layer detects whether there is a conflict between the perception results and the semantic description results. If there is no conflict, the perception results and semantic description results are fused to generate an event analysis result; if there is a conflict, the conflict type is determined, and the source reliability of the perception results and semantic description results is compared according to the conflict type, and an event analysis result is generated based on the comparison result. When neither the perception results nor the semantic description results meet the preset reliability conditions, the fusion output layer determines the event analysis result to be in a pending confirmation state and triggers the server to send the pending confirmation event analysis result to a preset cloud server or user terminal for verification.

[0078] Finally, the server pushes the generated event analysis results and corresponding event evidence chain to the user terminal. After receiving the event analysis results, the user terminal displays them to the user through security application software in the form of notification messages, event lists, or event detail pages. The user terminal also allows users to view the complete event evidence chain corresponding to the event analysis results. Users can view each step of the reasoning basis from the original video frames to the final conclusion by clicking on the evidence chain link or detail entry in the event analysis results, including the perception results of the first model, the semantic description results of the second model, the fusion adjudication process, and the final credibility. If the event analysis results are in a pending confirmation state, the user terminal also provides a confirmation or review operation entry for users to manually confirm the actual situation of the security event and return the user's confirmation result to the server. The server updates the event analysis results and event evidence chain based on the user's confirmation result.

[0079] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. Regarding the security event analysis system disclosed in the embodiments, since it corresponds to the security event analysis method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0080] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0081] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A security incident analysis method, characterized in that, include: The security video to be analyzed is acquired, and the security video to be analyzed is subjected to structured perception using a first model to obtain structured context information; wherein, the structured context information is used to describe the security events in the security video to be analyzed, and the structured context information includes the perception results of the target objects in the security events; Based on the structured context information, the target reasoning path for the security event is determined from a set of multiple candidate reasoning paths. When the target inference path does not invoke the second model, the event analysis result of the security event is generated based on the perception result; the parameter size of the second model is larger than the parameter size of the first model; When the second model is invoked in the target reasoning path, the input information of the second model is generated according to the structured context information, and the input information is input into the second model for semantic reasoning to obtain the semantic description result of the security event. Based on the perception result and the semantic description result, the event analysis result of the security event is generated.

2. The security incident analysis method according to claim 1, characterized in that, The step of determining the target inference path for the security event from a set of preset candidate inference paths based on the structured context information includes: Based on the structured context information, the event level of the security event is determined; wherein, the event level is used to characterize the processing priority of the security event; The credibility information of the perception result is obtained based on the structured context information, and the target credibility of the perception result is determined. Obtain the response time limit requirements for the security event and the inference cost of each candidate inference path; Based on the event level, the target credibility, the response time limit requirement, and the inference cost, the target inference path for the security event is determined from the multiple candidate inference paths.

3. The security incident analysis method according to claim 2, characterized in that, After determining the event level of the security event based on the structured context information, the method further includes: In response to changes in the perception results of the target object, the event level of the security event is adjusted.

4. The security incident analysis method according to claim 2, characterized in that, Determining the target credibility of the perception result includes: Obtain quality assessment information of the security video to be analyzed; wherein, the quality assessment information includes one or more of the following: image quality information, illumination information, occlusion information, temporal stability information, and acquisition device reliability information; Based on the quality assessment information, correction parameters are determined, and the credibility information is corrected using the correction parameters to obtain the target credibility of the perception result.

5. The security incident analysis method according to claim 2, characterized in that, The multiple candidate reasoning paths include a first reasoning path, a second reasoning path, and a third reasoning path; wherein, the first reasoning path is a reasoning path that generates the event analysis result based on the perception result without calling the second model; the second reasoning path is a reasoning path that serially calls the second model to perform semantic reasoning after obtaining the structured context information; and the third reasoning path is a reasoning path that calls the second model in parallel to verify the perception result.

6. The security incident analysis method according to claim 1, characterized in that, The step of generating the input information for the second model based on the structured context information includes: Based on the perception results, keyframe images containing the target object are extracted from the security video to be analyzed, and the image region where the target object is located is marked in the keyframe images; Text description information is generated based on the structured context information; the text description information is used to describe the security event; The input information for the second model is generated based on the labeled keyframe images and the text description information.

7. The security incident analysis method according to claim 6, characterized in that, Before extracting keyframe images containing the target object from the security video to be analyzed based on the perception result, the method further includes: Determine the video frame rate of the security video to be analyzed, and determine the trajectory information of the target object in the security video to be analyzed; In response to the video frame rate being determined to be lower than a preset frame rate threshold, the security video to be analyzed is interpolated according to the trajectory information of the target object to obtain the interpolated video frame. Based on the perception results, the keyframe image containing the target object is extracted from the interpolated video frame.

8. The security incident analysis method according to claim 1, characterized in that, The process of generating event analysis results for the security event based on the perception results and the semantic description results includes: The perception result and the semantic description result are aligned in time and by object, and the conflict between the aligned perception result and the semantic description result is detected. When there is no conflict, the perception result and the semantic description result are fused to generate the event analysis result; When a conflict exists, the conflict type is determined, the perception result and the semantic description result are compared according to the conflict type to obtain a comparison result, and the event analysis result is generated based on the comparison result. Where neither the perception result nor the semantic description result meets the preset reliability conditions, the event analysis result is determined to be in a pending confirmation state so that the preset server can review the security event; the preset reliability conditions are conditions constructed based on the credibility of the perception result and the semantic description result.

9. The security incident analysis method according to any one of claims 1 to 8, characterized in that, After generating the event analysis results for the security event, the process also includes: Generate the event evidence chain for the security incident; The event evidence chain includes the identification information of the security video to be analyzed, the perception result, the target inference path, and the event analysis result. Correspondingly, when the target inference path calls the second model, the event evidence chain also includes the input information and the semantic description result.

10. A security incident analysis system, characterized in that, include: Image capture equipment, used to acquire security videos for analysis; A server, communicatively connected to the image capture device, includes a first processor and a second processor; wherein the first processor is programmed to execute a first model to perform structured perception on the security video to be analyzed, and the second processor is programmed to execute a second model to perform semantic reasoning on the security video to be analyzed; the server is used to implement the security event analysis method as described in any one of claims 1 to 9; A user terminal, which is connected to the server, is used to receive and display the event analysis results generated by the server.

Images