Video image dangerous behavior recognition alarm method, device and equipment and storage medium

Abstract

The application discloses a video image dangerous behavior recognition alarm method and device, equipment and storage medium, relates to the technical field of video image intelligent analysis, and the method comprises the steps that a video image is acquired, target objects in the video image are detected and trajectory analysis is performed, and the motion trajectory and behavior sequence of the target objects are generated; visual semantic analysis is performed on the motion trajectory and behavior sequence, multi-dimensional semantic features are extracted, and multi-feature semantic objects are constructed according to the multi-dimensional semantic features; the multi-feature semantic objects are subjected to integrity evaluation and consistency evaluation, and integrity scores and consistency scores are obtained; according to the integrity scores and the consistency scores, a rule reasoning path or an uncertain reasoning path is executed; alarm information is generated according to the reasoning result and is output. The application can effectively improve the accuracy and reliability of dangerous behavior recognition in a complex scene. The application can improve the accuracy and reliability of video image dangerous behavior recognition.

Description

Technical Field

[0001] This application relates to the field of intelligent video image analysis technology, and in particular to methods, devices, equipment and storage media for identifying and alarming dangerous behaviors in video images. Background Technology

[0002] With the rapid development of intelligent video image analysis technology, hazardous behavior recognition and alarm technology has been widely applied in various public safety and industrial control scenarios. To improve the accuracy of hazardous behavior recognition, existing technologies typically integrate multiple intelligent models such as scene recognition, target detection and trajectory analysis, behavior recognition, and visual semantic analysis to comprehensively analyze and determine the target state and behavior in video footage.

[0003] To avoid decreased recognition accuracy due to insufficient information from a single model, current methods typically employ multi-model feature fusion or rule matching to comprehensively determine dangerous behaviors. Multi-model feature fusion integrates multi-dimensional feature information from models such as target detection, target trajectory analysis, and behavior recognition, using feature concatenation or weighted fusion for comprehensive judgment. Rule matching relies on pre-configured scene, region, target, and behavior alarm rules, triggering alarms when video analysis results meet preset conditions. However, these solutions still have shortcomings in real-world complex scenarios: the output information formats, data structures, and semantic expressions of different algorithm models are inconsistent, making it difficult to establish effective semantic relationships between heterogeneous information such as scenes, targets, and behaviors; the two core recognition systems, feature fusion and rule matching, are independent and lack a collaborative decision-making mechanism; and existing technologies lack fault-tolerance mechanisms when model output information is missing or multi-model recognition results conflict, easily leading to missed or false alarms.

[0004] Therefore, how to achieve unified collaboration and fault-tolerant decision-making of multi-model information, and improve the accuracy and reliability of dangerous behavior identification, is an urgent problem to be solved.

[0005] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0006] The main objective of this application is to provide a video image dangerous behavior recognition and alarm method, device, equipment and storage medium, which aims to solve the technical problem of how to achieve unified coordination and fault-tolerant decision-making of multi-model information, and improve the accuracy and reliability of dangerous behavior recognition.

[0007] To achieve the above objectives, this application proposes a video image dangerous behavior recognition and alarm method, the method comprising: Acquire video images, detect and perform continuous trajectory analysis on target objects in the video images, and generate motion trajectories and behavior sequences of the target objects; Visual semantic analysis is performed on the motion trajectory and behavior sequence to extract multi-dimensional semantic features, and a multi-feature semantic object is constructed based on the multi-dimensional semantic features; The semantic integrity and semantic consistency of the multi-feature semantic object are evaluated to obtain an integrity score and a consistency score. Based on the integrity score and the consistency score, a rule-based reasoning path or an uncertain reasoning path is executed, wherein the rule-based reasoning path is based on condition matching based on deterministic reasoning rules, and the uncertain reasoning path is based on probability determination based on multi-dimensional evidence fusion. Based on the reasoning results, generate danger event alarm information, and issue alarms based on the danger event alarm information; The step of executing a rule-based reasoning path or an uncertain reasoning path based on the integrity score and the consistency score includes: When the integrity score is greater than or equal to the first threshold and the consistency score is greater than or equal to the second threshold, the rule reasoning path is executed. When the integrity score is less than the first threshold or the consistency score is less than the second threshold, an uncertain inference path is executed.

[0008] In one embodiment, the step of executing the rule inference path includes: Read the candidate rule list from the multi-feature semantic object; The multi-feature semantic object is conditionally matched with the rules in the candidate rule list; If a rule exists that perfectly matches the multi-feature semantic object, then dangerous behavior is determined to exist. If there is no rule that completely matches the multi-feature semantic object, but there is a rule that partially matches the multi-feature semantic object, then calculate the partial matching degree. The partial matching degree and the rules of partial matching are used as evidence and injected into the fused evidence vector of the uncertain inference path to execute the uncertain inference path.

[0009] In one embodiment, the step of executing an uncertain inference path includes: Obtain a fused evidence vector, wherein the fused evidence vector includes at least a scene evidence score, a target evidence score, a behavioral evidence score, and an uncertainty evidence score; Based on the fused evidence vector, the probability of danger is calculated using the fusion function; If the probability of danger is greater than or equal to a preset risk threshold, then dangerous behavior is determined to exist.

[0010] In one embodiment, the step of performing semantic integrity evaluation and semantic consistency evaluation on the multi-feature semantic object to obtain integrity score and consistency score includes: Iterate through the pre-defined set of integrity semantic slots; If a corresponding semantic slot exists in the multi-feature semantic object, the preset weights corresponding to the semantic slots are accumulated to obtain an integrity score. Iterate through the pre-defined set of consistency constraints; If the multi-feature semantic object does not violate the consistency constraint of the current traversal, the preset weights corresponding to the consistency constraint are accumulated to obtain a consistency score.

[0011] In one embodiment, the step of constructing a multi-feature semantic object based on the multi-dimensional semantic features includes: Based on the multi-dimensional semantic features, scene semantic features, target semantic features, and behavior semantic features are obtained; The scene semantic features, the target semantic features, and the behavior semantic features are semantically aligned according to the same target identifier and the same time window; A multi-feature semantic object is constructed based on the semantically aligned scene semantic features, target semantic features, and behavior semantic features.

[0012] In one embodiment, the step of generating the fused evidence vector includes: Obtain scene semantics, target semantics, and behavior semantics from the multi-feature semantic object; Calculate the scene evidence score based on the scene confidence and scene risk level in the scene semantics; Calculate the target evidence score based on the trajectory analysis confidence level and target location risk coefficient in the target semantics; Calculate the behavioral evidence score based on the behavioral confidence and behavioral risk coefficient in the behavioral semantics. Calculate the uncertainty evidence score based on the integrity score and the consistency score; The fused evidence vector is generated based on the scenario evidence score, the target evidence score, the behavioral evidence score, and the uncertainty evidence score.

[0013] Furthermore, to achieve the above objectives, this application also proposes a video image dangerous behavior recognition and alarm device, which includes: The target monitoring module is used to acquire video images, detect and continuously analyze target objects in the video images, and generate motion trajectories and behavior sequences of the target objects. The semantic analysis construction module is used to perform visual semantic analysis on the motion trajectory and behavior sequence, extract multi-dimensional semantic features, and construct a multi-feature semantic object based on the multi-dimensional semantic features. The quality assessment module is used to perform semantic integrity evaluation and semantic consistency evaluation on the multi-feature semantic object, and obtain integrity score and consistency score; The reasoning module is used to execute a rule-based reasoning path or an uncertain reasoning path based on the integrity score and the consistency score, wherein the rule-based reasoning path is based on condition matching based on deterministic reasoning rules, and the uncertain reasoning path is based on probability determination based on multi-dimensional evidence fusion. The alarm module is used to generate dangerous event alarm information based on the reasoning results, and to issue alarms based on the dangerous event alarm information; The reasoning module is also used to execute a rule-based reasoning path when the integrity score is greater than or equal to a first threshold and the consistency score is greater than or equal to a second threshold. When the integrity score is less than the first threshold or the consistency score is less than the second threshold, an uncertain inference path is executed.

[0014] In addition, to achieve the above objectives, this application also proposes a video image dangerous behavior recognition and alarm device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the video image dangerous behavior recognition and alarm method described above.

[0015] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the video image dangerous behavior recognition and alarm method described above.

[0016] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the video image dangerous behavior recognition and alarm method described above.

[0017] This application provides a method for identifying and alerting dangerous behaviors in video images. The method involves acquiring video images, detecting and continuously analyzing the trajectory of a target object to generate its motion trajectory and behavior sequence; performing visual semantic analysis on the motion trajectory and behavior sequence to extract multi-dimensional semantic features, and constructing a multi-feature semantic object based on these features; evaluating the semantic integrity and consistency of the multi-feature semantic object to obtain integrity and consistency scores; executing a rule-based reasoning path or an uncertain reasoning path based on the integrity and consistency scores, wherein the rule-based reasoning path uses deterministic reasoning rules for conditional matching, and the uncertain reasoning path uses multi-dimensional evidence fusion for probability determination; generating dangerous event alert information based on the reasoning results, and issuing an alert based on the dangerous event alert information.

[0018] In summary, this application assesses the quality of multi-feature semantic objects through integrity scoring and consistency scoring, and adaptively selects rule-based reasoning paths or uncertain reasoning paths based on the assessment results. This solves the technical problem of lacking an effective fault-tolerant reasoning mechanism when information is missing or conflicting, thereby improving the accuracy and reliability of dangerous behavior identification in complex scenarios. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort. All actions of acquiring signals, information, or data in this application are performed in compliance with the relevant data protection regulations of the locality and with authorization from the owner of the corresponding device.

[0021] Figure 1 This is a flowchart illustrating an embodiment of the video image dangerous behavior recognition and alarm method of this application. Figure 2 This is a flowchart illustrating Embodiment 2 of the video image dangerous behavior recognition and alarm method of this application; Figure 3 A simplified flowchart illustrating the dual-path reasoning decision-making process provided in Embodiment 2 of this application; Figure 4 This is a schematic diagram of the module structure of the video image dangerous behavior recognition and alarm device according to an embodiment of this application; Figure 5This is a schematic diagram of the device structure of the hardware operating environment involved in the video image dangerous behavior recognition and alarm method in the embodiments of this application.

[0022] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0023] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0024] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0025] The main solution of this application embodiment is as follows: acquiring video images, detecting and continuously analyzing the target objects in the video images to generate the motion trajectory and behavior sequence of the target objects; performing visual semantic analysis on the motion trajectory and behavior sequence to extract multi-dimensional semantic features, and constructing multi-feature semantic objects based on the multi-dimensional semantic features; performing semantic integrity evaluation and semantic consistency evaluation on the multi-feature semantic objects to obtain integrity scores and consistency scores; executing rule-based reasoning paths or uncertain reasoning paths based on the integrity scores and consistency scores, wherein the rule-based reasoning path performs condition matching based on deterministic reasoning rules, and the uncertain reasoning path performs probability determination based on multi-dimensional evidence fusion; generating dangerous event alarm information based on the reasoning results, and issuing alarms based on the dangerous event alarm information.

[0026] Currently, multi-model feature fusion or rule matching is commonly used to comprehensively determine dangerous behaviors. Multi-model feature fusion integrates multi-dimensional feature information from models such as target detection, target trajectory analysis, and behavior recognition, using feature concatenation or weighted fusion for comprehensive judgment. Rule matching relies on pre-configured scene, region, target, and behavior alarm rules, triggering alarms when video analysis results meet preset conditions. However, these solutions still have shortcomings in real-world complex scenarios: the output information formats, data structures, and semantic expressions of different algorithm models are inconsistent, making it difficult to establish effective semantic relationships between heterogeneous information such as scenes, targets, and behaviors; the two core recognition systems, feature fusion and rule matching, are independent and lack a collaborative decision-making mechanism; when model output information is missing or multi-model recognition results conflict, existing technologies lack fault-tolerance mechanisms, easily leading to missed or false alarms.

[0027] In summary, this application assesses the quality of multi-feature semantic objects through integrity scoring and consistency scoring, and adaptively selects rule-based reasoning paths or uncertain reasoning paths based on the assessment results. This solves the technical problem of lacking an effective fault-tolerant reasoning mechanism when information is missing or conflicting, thereby improving the accuracy and reliability of dangerous behavior identification in complex scenarios.

[0028] It should be noted that the executing entity in this embodiment can be a video image dangerous behavior recognition and alarm system, or a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or a thermal management device capable of implementing the above functions of a video image dangerous behavior recognition and alarm system, etc. This embodiment does not specifically limit it. The following uses a video image dangerous behavior recognition and alarm system as the executing entity as an example to describe this embodiment and the following embodiments. All actions of acquiring signals, information, or data in this application are performed in compliance with the relevant data protection regulations of the locality and with authorization from the owner of the corresponding device.

[0029] Based on this, embodiments of this application provide a method for identifying and alerting dangerous behaviors in video images, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the video image dangerous behavior recognition and alarm method of this application.

[0030] In this embodiment, the video image dangerous behavior recognition and alarm method includes steps S10~S50: Step S10: Acquire video images, detect and perform continuous trajectory analysis on target objects in the video images, and generate motion trajectories and behavior sequences of the target objects; It should be noted that the target object refers to the entity in the video scene that needs to be identified and its trajectory analyzed, including but not limited to people, vehicles, or other objects with the ability to move. In this embodiment, people are used as the target object, but the invention is not limited to this. Detection refers to using a target detection model (such as a deep learning-based target detector) to locate and classify target objects in video frames, outputting the bounding box coordinates and target category of the target object. Trajectory analysis refers to using a target trajectory analysis model (such as a trajectory analyzer based on Kalman filtering or correlation filtering) to continuously associate and maintain the consistency of the same target object in consecutive video frames, outputting the trajectory sequence of the target object (i.e., the set of position coordinates at each time point) and trajectory analysis status (such as stable trajectory analysis, brief loss, recapture, etc.). Through the coordinated operation of detection and trajectory analysis, the system can generate the motion trajectory (including motion attributes such as position, speed, and direction) and behavior sequence (including the temporal behavior changes of the target) of each target object within a continuous time window.

[0031] Additionally, it should be noted that in real-world video scenarios, systems typically access multiple video streams, and the fields of view of different cameras may overlap or complement each other. By loading camera topology configurations, including camera position coordinates, field of view, overlapping region relationships, and region type classification, the system can associate and map target identifiers across multiple cameras, ensuring that the trajectory and behavior of the same target in different camera views can be traced uniformly. Furthermore, this step performs preprocessing operations after obtaining the original model output, including aligning the outputs of each model with timestamps according to a unified time window, and labeling low-quality results with detection box confidence below a preset lower limit and keypoint missing ratio exceeding a preset upper limit, but not discarding them directly, to retain some evidence for subsequent uncertain inference paths.

[0032] Understandably, given the complexities of real-world video scenes, such as occlusion, rapid movement, and changes in lighting, detection results at a single moment may be unstable or experience brief loss. Therefore, step S10 generates the target object's motion trajectory and behavior sequence within a continuous time window through continuous trajectory analysis. This accumulates the target's motion information over time, preventing target loss or misjudgment due to single-frame detection failures, thus providing stable and continuous target motion state data for subsequent steps. Simultaneously, timestamp alignment ensures the consistency of output data from multiple models—scene recognition, target detection and trajectory analysis, and behavior recognition—within the same time window, providing a reliable data foundation for subsequent cross-model semantic association and fusion inference of multi-feature semantic objects.

[0033] Step S20: Perform visual semantic analysis on the motion trajectory and behavior sequence, extract multi-dimensional semantic features, and construct a multi-feature semantic object based on the multi-dimensional semantic features; In this step, the system performs visual semantic analysis on the motion trajectory and behavior sequence generated in step S10, extracts multi-dimensional semantic features, and constructs a multi-feature semantic object based on these features. Specifically, the system uses a behavior recognition model and / or a visual semantic discrimination model to understand and abstract the motion trajectory and behavior sequence of the target object at the semantic level, extracting semantic features that can characterize the target's behavior pattern. Then, the system fuses the extracted semantic features with the scene semantic information output by the scene recognition model and the target semantic information output by the target detection model and trajectory analysis model, constructing a structured multi-feature semantic object according to a unified data structure.

[0034] It should be noted that visual semantic analysis refers to the process of semantically understanding and abstracting the motion trajectory and behavior sequence of a target object using behavior recognition models and / or visual semantic discrimination models, and extracting semantic features that can characterize the target's behavioral patterns. Behavior recognition models are used to identify the target object's action categories, such as running, falling, fighting, or climbing; visual semantic discrimination models are used to generate natural language risk descriptions or semantic tendency scores for the target's behavior. Only the behavior recognition model can be deployed in the system, or both can be deployed simultaneously; this embodiment does not impose specific limitations on this. Visual semantic analysis can be executed independently within each time window or fused across time windows; this embodiment does not impose specific limitations on this either. Multi-dimensional semantic features refer to the collective term for semantic information extracted from multiple dimensions such as scene, target, and behavior, including scene semantic features, target semantic features, and behavioral semantic features. Multi-feature semantic objects refer to structured semantic units formed by organizing the above multi-dimensional semantic features according to a unified data structure. These units contain multiple semantic slots such as scene semantics, target semantics, behavioral semantics, uncertainty semantics, and reasoning semantics, serving as a data bridge connecting the perception layer and the reasoning layer. Scene semantic features are obtained by standardizing the output of the scene recognition model through an adaptation unit, and include scene category labels, scene confidence, region type labels, scene risk level, and time status. Target semantic features are obtained by standardizing the output of the target detection model and target trajectory analysis model through an adaptation unit, and include target category, bounding box coordinates, detection confidence, trajectory sequence, current speed, direction of motion, and trajectory analysis status. Behavioral semantic features are obtained by standardizing the output of the behavior recognition model through an adaptation unit, and include action category labels, behavior confidence, behavior duration, and behavior direction.

[0035] Additionally, it should be noted that before extracting multi-dimensional semantic features, the system performs a standardized transformation on the original outputs of each model through a dedicated adaptation unit. This includes the following three operations: First, label mapping, mapping the unique labels of each model's output to a globally unified label library; second, coordinate normalization, aligning the coordinate data of different model outputs to a unified reference frame of the camera's field of view; and third, confidence level unification, normalizing the confidence levels of different dimensions of each model's output to the range of 0 to 1. The standardized transformation of each model's output achieves uniformity in semantic expression and data structure, providing a standardized data source for the subsequent construction of multi-feature semantic objects. Furthermore, for missing scenarios where there is no output due to model malfunction, the system distinguishes between two categories: inference failure missing and empty output without a target, which are marked as different missing types in subsequent steps.

[0036] Understandably, since the output information formats, data structures, and semantic expressions of multi-source heterogeneous models such as scene recognition, target detection and trajectory analysis, and behavior recognition are different and lack a unified correlation, step S20 extracts behavioral semantic features through visual semantic analysis and constructs a structured multi-feature semantic object by combining scene semantic features and target semantic features. This can unify heterogeneous information from different models within the same data framework, providing a complete and operable data foundation for subsequent semantic integrity evaluation, semantic consistency evaluation, and dual-path reasoning decision-making. This solves the technical problem that it is difficult to establish effective semantic associations and achieve efficient collaborative reasoning for heterogeneous information from multiple models.

[0037] In one feasible implementation, constructing a multi-feature semantic object based on the multi-dimensional semantic features may include steps A21-A23: Step A21: Based on the multi-dimensional semantic features, obtain scene semantic features, target semantic features, and behavior semantic features; It should be noted that scene semantic features refer to the semantic description information of a video scene, including scene category labels, scene confidence, region type labels, scene risk level, and time status. The scene category label indicates the scene type of the current video frame, such as a platform, corridor, or playground; the region type label is determined based on the region division in the camera topology configuration, such as edge-sensitive area, restricted area, or movement area; the scene risk level is comprehensively assessed based on the region type and the current time period, such as high risk, medium risk, or low risk; and the time status is determined based on the time period of the current time window, such as daytime peak or nighttime. Target semantic features refer to the semantic description information of the state and motion attributes of a target object, including target category, detection box position coordinates, detection confidence, trajectory sequence, current speed, direction of motion, and trajectory analysis status. The target category indicates the type of target object, such as a person or vehicle; the detection box position coordinates indicate the target's position in the image; the detection confidence indicates the reliability of the target detection; the trajectory sequence contains the set of position coordinates of the target within a continuous time window; and the trajectory analysis status includes status information such as trajectory stability, brief loss, or recapture. Behavioral semantic features refer to the semantic description information of the target object's behavior, including action category label, behavior confidence, behavior duration, and behavior direction. Among them, the action category label indicates the type of action the target is currently performing, such as running, falling, fighting, or climbing; the behavior duration is calculated based on the number of consecutive time windows in which the current behavior is first detected and the length of the time windows; the behavior direction is determined based on the target's trajectory direction.

[0038] Additionally, it should be noted that when a certain type of semantic feature cannot be obtained due to the corresponding model not outputting it or the output being unqualified, the system marks that type of semantic feature as missing, but does not discard other valid semantic information within that time window. The missing feature marker will be written into the uncertain semantic slot of the multi-feature semantic object in subsequent steps, for uncertain inference paths to process according to partial evidence. In video scenarios where multiple targets coexist, the system independently performs the above parsing operations for each target object, ensuring that each target object has independent scene semantic features, target semantic features, and behavioral semantic features.

[0039] Step A22: Semantically align the scene semantic features, the target semantic features, and the behavior semantic features according to the same target identifier and the same time window; It should be noted that semantic alignment refers to aggregating feature information from different models and semantic dimensions according to unified target attribute information and time labels. This allows scene information, target information, and behavioral information, originally scattered across different data structures, to be attributed to the same target object within the same time window, forming a complete semantic description. The same target identifier refers to the stable target identifier output by the target trajectory analysis model, used to uniquely identify a target object; the same time window refers to the unified time window preprocessed in step S10, used to ensure the consistency of the output data of each model in the time dimension. Through semantic alignment, the system can accurately distinguish the semantic information of different targets in complex scenarios with multiple targets coexisting, avoiding semantic confusion and erroneous associations between different targets.

[0040] Additionally, it should be noted that when a target first appears without a stable target identifier, the system first generates a temporary candidate target identifier, which is then replaced with a stable target identifier after confirmation by the trajectory analysis model. When a target reappears after a brief period of loss, the system restores the association of the target identifier through trajectory prediction and feature matching, ensuring the continuity of the alignment operation. For scenarios with overlapping fields of view from multiple cameras, the system uses the overlapping region relationship in the camera topology configuration to associate the identifiers of the same target across different cameras, ensuring the continuity and consistency of semantic information of the target when it crosses the field of view of a camera.

[0041] Step A23: Construct a multi-feature semantic object based on the semantically aligned scene semantic features, target semantic features, and behavior semantic features.

[0042] It should be noted that the multi-feature semantic object comprises five semantic slots: scene semantics, target semantics, behavioral semantics, uncertainty semantics, and inference semantics. Scene semantics stores scene-related information such as scene category, region type, scene risk level, and scene confidence. Target semantics stores target-related information such as target identifier, target category, location coordinates, trajectory sequence, speed, direction, and trajectory analysis status. Behavioral semantics stores behavioral information such as action category, behavior confidence, behavior duration, and behavior direction. Uncertainty semantics stores quality marker information such as semantic missing masks and semantic conflict masks. Inference semantics stores inference-related information such as candidate rule lists and fused evidence vectors. The multi-feature semantic object is a structured data unit. Essentially, it encapsulates raw output information from multiple heterogeneous models, each with different formats, into a unified data structure with fixed and indexable fields after standardization and semantic alignment. This data structure supports direct access to corresponding semantic information via field names, supports marking and tracking missing and conflicting fields, and supports dynamic updating and supplementation of stored semantic information during inference.

[0043] Additionally, it should be noted that during the construction of multi-feature semantic objects, the system also performs the following operations simultaneously: (1) Semantic Missing Detection. When a certain type of semantic slot is missing due to the model not outputting or the output being unqualified, the corresponding slot is marked as missing in the missing mask of uncertain semantics, and the quality score and missing type are recorded. Specifically, let the set of semantic slots required for completeness be E. For any slot r∈E, its existence is marked as is_exist(r). If the slot exists, it is marked as 1, and if the slot is missing, it is marked as 0. The missing mask records the name of the missing slot and the missing type. The missing type includes two types: inference failure missing and empty output without target.

[0044] (2) Semantic Conflict Determination. When a conflict exists between different semantic slots, the conflict pair is marked in the conflict mask of the uncertain semantics, and the conflict type and conflict intensity are recorded. Specifically, let the set of consistency constraints be H. For any constraint h∈H, a conflict determination function is defined as is_conflict(h). If a conflict is triggered, it is marked as 1; otherwise, it is marked as 0. The conflict intensity is quantified according to the degree of conflict, for example, taking a value in the range of 0 to 1. A typical example of a conflict is as follows: if the behavioral semantics is "falling down" and the trajectory in the target semantics is continuous high-speed movement (speed exceeding a preset high-speed threshold), then it is marked as a conflict. Constraints marked as conflict are included in the penalty item of subsequent consistency scoring according to a preset weight.

[0045] It is understandable that, since the multi-feature semantic object unifies the semantic information of scene, target, behavior and other elements that were originally scattered in the output of different models, and also includes auxiliary information such as uncertain semantics and reasoning semantics, step A23 can provide a complete data carrier for subsequent semantic integrity evaluation and semantic consistency evaluation. At the same time, it can also provide a list of candidate rules that can be directly read for rule reasoning paths and a fusion evidence vector that can be directly used for uncertain reasoning paths, thereby ensuring that the data transmission between the perception layer and the reasoning layer is efficient and unambiguous.

[0046] Step S30: Perform semantic integrity evaluation and semantic consistency evaluation on the multi-feature semantic object to obtain integrity score and consistency score; It should be noted that semantic completeness evaluation refers to the process of quantitatively assessing the completeness of each semantic slot in a multi-feature semantic object, used to measure whether the key semantic information required for reasoning is complete and usable. Semantic consistency evaluation refers to the process of quantitatively assessing the logical consistency between each semantic slot in a multi-feature semantic object, used to detect whether there are contradictions or conflicts between different semantic dimensions. The completeness score is used to quantify the completeness of semantic information; the higher the score, the more complete the semantic information. The consistency score is used to quantify the degree of logical conflict between semantics; the higher the score, the more consistent the semantic information.

[0047] Additionally, it should be noted that completeness evaluation and consistency evaluation are two independent evaluation dimensions. Completeness evaluation only focuses on the existence of semantic slots and does not involve the logical relationships between semantic content; consistency evaluation only focuses on logical conflicts between different semantic slots and does not involve the completeness of semantic slots. Both quantitatively assess the semantic quality of multi-feature semantic objects from different perspectives, jointly constituting the decision-making basis for inference path selection. The calculation of both completeness and consistency scores is completed independently within the current time window and does not depend on data from historical time windows.

[0048] It is understandable that, in real video scenarios, situations where model output information is missing or multiple model recognition results conflict frequently occur. If semantic information with missing or conflicting information is used directly for reasoning, it is very easy to cause missed or false alarms of dangerous behaviors. Therefore, step S30 can quantitatively evaluate the quality of semantic information in the input reasoning link by evaluating the semantic integrity and semantic consistency of multi-feature semantic objects. This provides a reliable quantitative decision basis for the adaptive selection of subsequent reasoning paths, thereby avoiding erroneous reasoning caused by missing or conflicting information and improving the accuracy and reliability of dangerous behavior recognition.

[0049] In one feasible implementation, step S30 may include steps A31 to A34: Step A31: Traverse the preset set of integrity semantic slots; It should be noted that the integrity semantic slot set refers to a pre-defined set of semantic slot names that participate in the integrity scoring calculation, denoted as E. This set lists all semantic slots that have significant reference value for determining dangerous behaviors, including but not limited to region type slots in scene semantics, location / trajectory slots in target semantics, and action category slots in behavior semantics. The integrity semantic slot set can be pre-established based on rule conditions in the rule configuration library during system initialization, or it can be dynamically adjusted during operation based on the current scene type or candidate rule list. This embodiment does not impose specific limitations on this.

[0050] Additionally, it should be noted that the preset basis for the complete semantic slot set is the semantic slots involved in the conditional predicates of all rules in the rule configuration library. The system iterates through each rule in the rule configuration library, extracts the semantic slots that each conditional predicate of that rule depends on, and takes the union of all the semantic slots that all rules depend on to form the complete semantic slot set. Different scene types or different region types correspond to different rule subsets, therefore the complete semantic slot set can be dynamically filtered according to the scene type or region type in the current scene semantics, retaining only the semantic slots that the rules that may be triggered in the current scene depend on.

[0051] Step A32: If there is a corresponding semantic slot in the multi-feature semantic object, then the preset weights corresponding to the semantic slots are accumulated to obtain an integrity score. It should be noted that the corresponding semantic slot refers to the actual slot in the multi-feature semantic object corresponding to each semantic slot in the integrity semantic slot set E. Specifically, during the traversal of the integrity semantic slot set E, for each slot r in the set, the system checks whether the slot exists in the multi-feature semantic object and is not marked as missing (i.e., not marked as missing in the missing mask of uncertain semantics). If it exists, the system assigns the preset weight w corresponding to the slot. r Included in the cumulative value; if it does not exist or has been marked as missing, it is not included. Integrity score The calculation method is as follows:

[0052] Here, `is_exist(r)` is a binary function that takes a value of 1 if the semantic slot exists in the multi-feature semantic object, and a value of 0 otherwise. Weight w rBased on the preset importance of the semantic slot for judging dangerous behavior, different semantic slots can be assigned different weight values. For example, for edge region rules, the region type slot in scene semantics and the position / trajectory slot in target semantics can be assigned higher weights; for fighting rules, the action category slot in behavior semantics can be assigned higher weights.

[0053] Additionally, it should be noted that the completeness score is a normalized value within the range of 0 to 1. A higher score indicates that the multi-feature semantic object contains more complete key semantic information required for reasoning, making it more suitable for entering the rule-based reasoning path for precise condition matching. A lower score indicates more severe semantic information loss, requiring entry into the uncertain reasoning path for probabilistic judgment based on partial evidence. In actual deployment, the sum of the weights of each slot in the same scenario is generally normalized to 1, but different weight allocation strategies can be configured for different scenario types.

[0054] Step A33: Traverse the preset set of consistency constraints; It should be noted that the consistency constraint set refers to a pre-defined set of constraint rules, denoted as H, used to detect whether there are logical conflicts between different semantic slots. Each constraint h in this set defines a logical mutual exclusion relationship or contradiction judgment condition between one or more pairs of semantic slots. For example, constraint h1 can be defined as: if the behavioral semantic is "falling down" and the speed in the target semantic exceeds a preset high-speed threshold (such as 3 meters / second), then it is judged as a logical conflict; constraint h2 can be defined as: if the behavioral semantic is "stationary" and the trajectory in the target semantic continues to move, then it is judged as a logical conflict. The consistency constraint set can be pre-configured based on expert experience during system initialization, or it can be dynamically adjusted according to the scenario type during operation. This embodiment does not impose specific limitations on this.

[0055] Additionally, it should be noted that the consistency constraint set is pre-defined based on known physical laws and logical common sense regarding different semantic dimensions. For example, falling should not be accompanied by continuous high-speed movement, and a stationary state should not be accompanied by continuous trajectory changes. The system formalizes these physical laws and logical common sense into binary decision rules, with each rule defining a conflict condition and a corresponding conflict intensity weight. The consistency constraint set cross-constrains the relationships between the three dimensions of scene semantics, target semantics, and behavior semantics, covering conflict detection between three types of semantic pairs: scene-target, target-behavior, and scene-behavior.

[0056] Step A34: If the multi-feature semantic object does not violate the consistency constraint of the current traversal, then the preset weights corresponding to the consistency constraint are accumulated to obtain a consistency score. It should be noted that "not violating the consistency constraint being traversed" means that the scene semantics, target semantics, and behavior semantics in the multi-feature semantic object do not constitute a conflict condition defined by the consistency constraint. Specifically, during the traversal of the consistency constraint set H, for each constraint h in the set, the system checks whether the scene semantics, target semantics, and behavior semantics in the multi-feature semantic object satisfy the conflict condition defined by the constraint. If a conflict condition is met, the constraint is marked as "violated"; if no conflict condition is met, the constraint is marked as "not violated". Consistency Score The calculation method is as follows:

[0057] Where is_conflict(h) is a binary function that takes the value 1 when a multi-feature semantic object violates the consistency constraint, and takes the value 0 otherwise; The preset weights are the values ​​corresponding to this consistency constraint.

[0058] Additionally, it should be noted that the consistency score is a normalized value within the range of 0 to 1. A higher score indicates better logical consistency among the semantic dimensions of a multi-feature semantic object, making it more suitable for entering the rule-based inference path for precise condition matching; a lower score indicates more severe semantic conflict, requiring entry into the uncertain inference path for probabilistic judgment based on multi-dimensional evidence fusion. When multiple conflicting constraints exist, the preset weights of each conflicting constraint are... The higher the cumulative value, the greater the deduction in the consistency score, and the higher the system's sensitivity to conflicts. In actual deployments, the sum of the weights of each consistency constraint in the same scenario is generally normalized to 1. The integrity score and consistency score independently evaluate the semantic quality of multi-feature semantic objects from two dimensions: "information completeness" and "logical consistency," respectively. Both scores must reach a preset threshold (e.g., ...). =0.7, When the score is 0.7, the semantic information is complete and consistent, and the system enters the rule-based reasoning path; when any score is lower than the preset threshold, the semantic information is missing or conflicting, and the system enters the uncertain reasoning path.

[0059] Step S40: Based on the integrity score and the consistency score, execute a rule-based reasoning path or an uncertain reasoning path, wherein the rule-based reasoning path is based on condition matching based on deterministic reasoning rules, and the uncertain reasoning path is based on probability determination based on multi-dimensional evidence fusion. It should be noted that deterministic reasoning rules refer to a set of conditional logics used to determine dangerous behavior, such as "if people are running in the edge area of ​​a platform, then dangerous behavior is determined to exist." Multi-dimensional evidence refers to evidence information extracted from multiple dimensions such as scene semantics, target semantics, and behavioral semantics, including scene evidence scores, target evidence scores, behavioral evidence scores, and uncertain evidence scores. The first preset threshold is the basis for judging whether the completeness of semantic information meets the execution conditions of the rule reasoning path, and the second preset threshold is the basis for judging whether the consistency of semantic information meets the execution conditions of the rule reasoning path.

[0060] Additionally, it should be noted that the rule-based inference path and the uncertain inference path are not executed simultaneously, but rather one is executed based on the comparison results of the integrity score and the consistency score with preset thresholds. When the integrity score is greater than or equal to the first preset threshold and the consistency score is greater than or equal to the second preset threshold, the rule-based inference path is executed; when the integrity score is less than the first preset threshold or the consistency score is less than the second preset threshold, the uncertain inference path is executed.

[0061] It is understandable that since the quality of semantic information may vary in different time windows, using a single reasoning path may lead to a decrease in reasoning accuracy. Therefore, step S40 adaptively selects a rule-based reasoning path or an uncertain reasoning path based on the integrity score and consistency score, which can avoid misjudgment or omission caused by the mismatch between the reasoning path and the information quality, thereby improving the accuracy and robustness of dangerous behavior identification.

[0062] Step S50: Generate a danger event alarm message based on the reasoning result, and issue an alarm based on the danger event alarm message; It should be noted that the reasoning result refers to the judgment conclusion output by the rule-based reasoning path or the uncertain reasoning path in step S40, that is, the judgment result of whether a dangerous behavior exists. Dangerous event alarm information refers to comprehensive alarm data used to describe dangerous events, including at least alarm identifier, alarm timestamp, hazard type, and hazard level. Alarm refers to the process of sending the generated alarm information to on-duty personnel or business systems.

[0063] Additionally, it should be noted that when the reasoning result indicates no dangerous behavior is detected, the system will not generate a danger event alarm or trigger an alarm output. Alarms can be triggered in various ways, including but not limited to pushing alarm notifications to security terminals, displaying pop-up prompts on the management center interface, linking audible and visual warning devices, and sending alarm information to the superior management platform.

[0064] Understandably, since the judgment results output by the dangerous behavior recognition system need to be promptly transmitted to the on-duty personnel or business system to achieve effective alarm, step S50 generates dangerous event alarm information based on the reasoning results and issues an alarm, which can transform the dangerous behavior judgment conclusion into a perceptible and transmittable alarm notification, so that the dangerous behavior recognition results can reach the responding subject in a timely manner, forming a complete closed loop of recognition-judgment-alarm technology.

[0065] This embodiment provides a method for identifying and alerting dangerous behaviors in video images. The method involves acquiring video images, detecting and continuously analyzing the trajectory of a target object to generate its motion trajectory and behavior sequence. Visual semantic analysis is then performed on the motion trajectory and behavior sequence to extract multi-dimensional semantic features, and a multi-feature semantic object is constructed based on these features. Semantic integrity and consistency evaluations are performed on the multi-feature semantic object to obtain integrity and consistency scores. Based on the integrity and consistency scores, a rule-based reasoning path or an uncertain reasoning path is executed. The rule-based reasoning path uses deterministic reasoning rules for conditional matching, while the uncertain reasoning path uses multi-dimensional evidence fusion for probability determination. Finally, dangerous event alert information is generated based on the reasoning results, and an alert is issued based on this information.

[0066] In summary, this embodiment assesses the quality of multi-feature semantic objects through integrity scoring and consistency scoring, and adaptively selects rule-based reasoning paths or uncertain reasoning paths based on the assessment results. This solves the technical problem of lacking an effective fault-tolerant reasoning mechanism when information is missing or conflicting, thereby improving the accuracy and reliability of dangerous behavior identification in complex scenarios.

[0067] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 Step S40 also includes steps S41 to S42: Step S41: When the integrity score is greater than or equal to the first threshold and the consistency score is greater than or equal to the second threshold, execute the rule reasoning path; It should be noted that the rule-based reasoning path is used to perform conditional matching on multi-feature semantic objects based on preset deterministic reasoning rules to arrive at a judgment conclusion when the semantic information is complete and consistent.

[0068] Additionally, it should be noted that when both the integrity score and the consistency score reach their respective thresholds, it indicates that the semantic information quality meets the preconditions for rule-based reasoning, and the system enters the rule-based reasoning path; otherwise, it enters the uncertain reasoning path. The first and second thresholds can be preset to fixed values ​​(e.g., the first threshold is 0.7 and the second threshold is 0.7), or they can be adaptively adjusted according to the scenario type or regional risk level. This embodiment does not impose specific limitations on this.

[0069] It is understandable that, since the rule reasoning path can quickly obtain the judgment result through accurate condition matching when the semantic information is complete and consistent, step S41 can ensure that the rule reasoning path is executed only when the quality of the semantic information meets the requirements, avoiding rule matching errors caused by missing or conflicting semantic information, thereby improving the accuracy and efficiency of dangerous behavior judgment.

[0070] In one feasible implementation, executing the rule-based reasoning path may include steps A411-A415: Step A411: Read the candidate rule list from the multi-feature semantic object; It should be noted that the candidate rule list refers to the set of candidate dangerous rules that are pre-selected based on the area type and risk level of the current scenario and written into the inference semantics. Each rule is associated with an applicable set of area types and a set of risk levels.

[0071] Additionally, it should be noted that the candidate rule list may contain one or more rules. When the candidate rule list is empty, the process directly proceeds to an uncertain inference path. This embodiment does not impose any specific limitations on this.

[0072] Step A412: Perform conditional matching between the multi-feature semantic object and the rules in the candidate rule list; It should be noted that conditional matching refers to the process of comparing the actual values ​​of each semantic slot in a multi-feature semantic object with the conditional predicates defined in the rule one by one. Each rule contains one or more conditional predicates. For example, the rule "platform edge + personnel + running" contains three conditional predicates (region type is platform edge, target type is personnel, and behavior category is running).

[0073] Additionally, it should be noted that conditional matching can be exact matching, meaning the actual value of the semantic slot must be completely consistent with the rule condition; or it can be fuzzy matching, meaning a match is considered to have a similarity between the actual value of the semantic slot and the rule condition reaching a preset similarity threshold. This embodiment does not impose specific limitations on either. For numerical conditions (such as speed greater than 3 m / s), the matching method is to compare the actual value with the threshold.

[0074] Step A413: If there is a rule that completely matches the multi-feature semantic object, then it is determined that there is dangerous behavior; It should be noted that a complete match means that there exists a rule in the candidate rule list where all conditional predicates of that rule are satisfied by the current value in the multi-feature semantic object.

[0075] Additionally, it should be noted that when a perfect match is achieved, the system directly outputs a rule-based hazard event. The hazard event includes information such as hazard type (the name of the matching rule), hazard level, occurrence time window, associated target identifier, and associated camera number, and is sent to step S50 for alarm output.

[0076] Step A414: If there is no rule that completely matches the multi-feature semantic object, but there is a rule that partially matches the multi-feature semantic object, then calculate the partial matching degree. It should be noted that partial matching refers to a situation where there are no completely matching rules in the candidate rule list, but at least one rule has a partial conditional predicate that is satisfied by the current value in the multi-feature semantic object. Partial matching degree The calculation formula is:

[0077] For example, if the rule "platform edge + people + running → danger" has a weight of [0.4, 0.3, 0.3], and only "platform edge + people" matches, then... .

[0078] Additionally, it should be noted that when there are multiple partially matching rules, the partial matching degree of each rule can be calculated separately, or only the rule with the highest partial matching degree can be selected for subsequent processing. This embodiment does not impose any specific limitations on this.

[0079] Step A415: The partial matching degree and the rules of partial matching are used as evidence and injected into the fused evidence vector of the uncertain inference path to execute the uncertain inference path; It should be noted that injection refers to converting the conditional information that has been successfully matched in the partial matching process into evidence data and writing it into the corresponding component of the fused evidence vector as input for subsequent uncertain reasoning paths.

[0080] Additionally, it should be noted that after the evidence is injected, the system directly switches to an uncertain reasoning path, making a probability determination by integrating multi-dimensional evidence.

[0081] Step S42: When the integrity score is less than the first threshold or the consistency score is less than the second threshold, execute the uncertain inference path; It should be noted that the uncertain reasoning path refers to a reasoning method that relies on the fusion of multi-dimensional evidence to make probabilistic judgments in order to arrive at a conclusion when semantic information is missing or conflicting. Multi-dimensional evidence includes at least scenario evidence, target evidence, behavioral evidence, and uncertainty evidence.

[0082] Additionally, it should be noted that when the completeness score is less than the first threshold or the consistency score is less than the second threshold, it indicates that the semantic information is incomplete or conflicting. In this case, the matching accuracy of the rule-based reasoning path cannot be guaranteed. Therefore, an uncertain reasoning path is switched to, which uses probabilistic judgment by fusing multi-source evidence. The uncertain reasoning path can still complete effective reasoning under uncertain conditions of missing or conflicting information.

[0083] Understandably, rule matching may produce incorrect judgments when information is missing or conflicting. Therefore, step S42, by switching to an uncertain reasoning path, can still complete effective reasoning under conditions of incomplete or conflicting information, thereby improving the robustness of dangerous behavior identification in complex working conditions.

[0084] In one feasible implementation, executing an uncertain reasoning path may include steps A421-A423: Step A421: Obtain the fused evidence vector, wherein the fused evidence vector includes at least the scene evidence score, the target evidence score, the behavior evidence score, and the uncertainty evidence score; It should be noted that the fused evidence vector refers to a collection of multi-dimensional evidence used for uncertain reasoning paths.

[0085] In one feasible implementation, the fused evidence vector is generated according to the following steps: Step A01: Obtain scene semantics, target semantics, and behavior semantics from the multi-feature semantic object; It should be noted that the acquisition method can be either direct reading from the corresponding fields of the multi-feature semantic object or extraction by calling a data interface; this embodiment does not impose specific limitations on this. When a certain type of semantic data in the multi-feature semantic object is empty or marked as missing, the missing marker is passed to the subsequent scoring calculation steps.

[0086] Step A02: Calculate the scene evidence score based on the scene confidence and scene risk level in the scene semantics; It should be noted that the scene evidence score is C. S The calculation method is to multiply the scene confidence score by the scene risk coefficient. Here, the scene confidence score refers to the credibility of the current scene category judgment output by the scene recognition model; the scene risk level refers to the risk level of the video scene determined by a combination of region type and current time period, and the scene risk coefficient is determined based on the scene risk level.

[0087] Additionally, it should be noted that the scene risk coefficient is calculated as follows: 1 corresponds to high risk, 0.7 to medium risk, and 0.3 to low risk. Other mapping methods can also be used for scene risk levels and coefficients depending on the actual scene requirements; this embodiment does not impose specific limitations on this. Scene evidence scoring is used to characterize the degree to which the current video scene supports hazard assessment.

[0088] Step A03: Calculate the target evidence score based on the trajectory analysis confidence and target location risk coefficient in the target semantics; It should be noted that the target evidence score C O The calculation method is to multiply the trajectory analysis confidence level by the target location risk coefficient. The trajectory analysis confidence level refers to the degree of confidence the target trajectory analysis model has in continuously analyzing the trajectory of the current target; the target location risk coefficient is determined based on the type of area where the target is located.

[0089] Additionally, it should be noted that the target location risk coefficient is calculated as follows: 1 for edge areas and 0.2 for core areas. Other mapping methods can also be used for the target location risk coefficient depending on the specific needs of the scenario; this embodiment does not impose any specific limitations on this. The target evidence score is used to characterize the degree to which the current state of the target object supports the hazard assessment.

[0090] Step A04: Calculate the behavioral evidence score based on the behavioral confidence and behavioral risk coefficient in the behavioral semantics; It should be noted that the behavioral evidence score C A The calculation method is to multiply the behavior confidence score by the behavior risk coefficient. The behavior confidence score refers to the reliability of the behavior recognition model's judgment of the current action category; the behavior risk coefficient is determined based on the action category.

[0091] Additionally, it should be noted that the behavioral risk coefficient is calculated as follows: 0.9 for running and 0.2 for walking. Other mapping methods can also be used for the behavioral risk coefficient depending on the specific needs of the scenario; this embodiment does not impose any specific limitations on this. The behavioral evidence score is used to characterize the degree to which the current target behavior supports the risk assessment.

[0092] Step A05: Calculate the uncertainty evidence score based on the integrity score and the consistency score; It should be noted that the uncertainty evidence score C UThis is used to characterize the overall quality of current semantic information and its impact on risk assessment. In this embodiment, when the semantic information is complete and consistent, the sum of the missing slot weights and the sum of the conflict intensity are both 0, and the uncertainty evidence score is 1. When the semantic information is missing or there is severe conflict, the sum of the missing slot weights and the sum of the conflict intensity increase, and the uncertainty evidence score approaches 0. The uncertainty evidence score can also be calculated based on a weighted combination of the completeness score and the consistency score; this embodiment does not specifically limit this calculation.

[0093] Step A06: Generate the fused evidence vector based on the scene evidence score, the target evidence score, the behavior evidence score, and the uncertainty evidence score; It should be noted that the fused evidence vector E is in the following form: Among them, C S The scene evidence score is C. O Score the target evidence, C A For scoring behavioral evidence, C U Scoring for evidence of uncertainty This represents the partial matching degree of each candidate rule.

[0094] Additionally, it should be noted that the generation method of the fused evidence vector is not limited to combining the evidence scores into an array in a fixed order; it can also be organized using key-value pairs or structured data objects. This embodiment does not specifically limit this approach. (Partial matching degree of each candidate rule) When partial matching evidence is injected in step A415, a corresponding value is assigned; initially, all values ​​are set to 0. The generated fusion evidence vector is used for subsequent fusion function calculations.

[0095] Specifically, the scores for each piece of evidence are calculated as follows: C S (Scene Evidence) = Scene Confidence × Scene Risk Coefficient (High = 1, Medium = 0.7, Low = 0.3); C O (Target Evidence) = Trajectory Analysis Confidence Level × Target Location Risk Coefficient (Edge = 1, Core = 0.2); C A (Behavioral evidence) = Behavioral confidence level × Behavioral risk coefficient (running = 0.9, walking = 0.2); C U (Uncertainty Evidence) = 1 - (Sum of missing slot weights + Sum of conflict strengths) / Total weight; : The partial matching degree of each candidate rule is assigned a value when there is partial matching evidence injected in step A215. Initially, all values ​​are set to 0.

[0096] Additionally, it should be noted that the scores of each piece of evidence are combined into a fused evidence vector after acquisition, for use in subsequent fusion function calculations. The scene risk coefficient is determined based on the scene risk level, the target location risk coefficient is determined based on the type of area where the target is located, and the behavior risk coefficient is determined based on the action category; this embodiment does not impose specific limitations on these.

[0097] Step A422: Calculate the risk probability based on the fusion function according to the fused evidence vector; It should be noted that the fusion function refers to the function used to synthesize multi-dimensional evidence from the fusion evidence vector to obtain the probability of danger. The calculation function is used. In this embodiment, the fusion function adopts a weighted scoring method, with the risk probability... The calculation formula is:

[0098] in, ,like .

[0099] Additionally, it should be noted that the fusion function is not limited to the weighted scoring method. It can also be replaced by DS evidence theory fusion, Bayesian fusion, fuzzy inference, or learning-based fusion networks. This embodiment does not make specific limitations on this.

[0100] Step A423: If the probability of danger is greater than or equal to a preset risk threshold, then it is determined that there is a dangerous behavior. It should be noted that the preset risk threshold This is the threshold value used to determine whether to trigger an alarm, for example, 0.7. When the probability of danger... At that time, output a fusion-type hazardous event.

[0101] Additionally, it should be noted that when When a dangerous behavior is detected, a fused dangerous event is output and sent to step S50 for alarm output; when When the time comes, the data is sent to the continuous observation queue without triggering an alarm. It can be preset to a fixed value, or it can be dynamically adjusted according to the scenario type or regional risk level. This embodiment does not make specific limitations on this.

[0102] Optionally, if the probability of danger is lower than a preset risk threshold, the following operations may be further performed: In one alternative implementation, when the danger probability output by the inference path is uncertain... Below the output threshold In this case, the corresponding event enters the continuous observation queue instead of being discarded directly. The core mechanism of the continuous observation queue includes two sub-mechanisms: trend accumulation determination and conflict resolution accumulation.

[0103] It should be noted that the continuous observation queue is used to continuously monitor target events that have not reached the alarm threshold but pose a potential risk.

[0104] For the trend accumulation determination mechanism: for the same target identifier j, define its danger probability sequence within K consecutive time windows (K=3 in the example). The conditions for determining an upward trend are:

[0105] in, This indicates a logical AND operation; if... It is true and the current probability of danger has not reached But it exceeds the warning threshold (0.35 in the example), then an early warning notification is generated, pushed to the on-duty personnel and marked "rising trend warning".

[0106] For the conflict resolution accumulation mechanism: when a semantic conflict causes a decrease in the consistency score and the system temporarily suppresses the alarm, a conflict resolution count is defined. This represents the number of consecutive conflict-free time windows since the conflict was first identified. The formula for correcting the cumulative danger probability after conflict resolution is as follows:

[0107] in, The minimum number of consecutive conflict-free time windows required for complete recovery (3 in the example). When When the probability of danger decreases proportionally (the first conflict-free window decreases to 1 / 3 of its original value, the second to 2 / 3); when When the coefficient is 1, the risk probability returns to the original calculated value, and the alarm judgment returns to normal.

[0108] Additionally, it should be noted that the system also performs queue cleanup operations. If the trend is interrupted, i.e., any window appears... ≤ After the observation window duration T is exceeded obs (In this example, 5 time windows) The observation record for the target is automatically cleared after each window. When the probability of danger decreases (i.e., the trend declines) within two consecutive time windows, or when the recording duration exceeds a preset upper limit, the observation record for the target is automatically cleared. In this way, the system can issue early warnings when the risk continues to rise, avoiding false alarms caused by premature alerts, and preventing the omission of potential risks.

[0109] This embodiment provides a method for identifying and alerting dangerous behaviors in video images. Building upon the first embodiment, this embodiment first executes a rule-based reasoning path in step S41 when the integrity score is greater than or equal to a first threshold and the consistency score is greater than or equal to a second threshold. In this path, a candidate rule list is read from the multi-feature semantic object, and the multi-feature semantic object is conditionally matched with the rules in the candidate rule list. If a completely matching rule exists, a dangerous behavior is directly determined. If no completely matching rule exists but a partially matching rule exists, a partial matching degree is calculated, and the partial matching degree and the matched conditions are injected as evidence into the fused evidence vector, transitioning to an uncertain reasoning path. Further, in step S42, this embodiment executes an uncertain reasoning path when the integrity score is less than the first threshold or the consistency score is less than the second threshold. In this path, a fused evidence vector containing scene evidence score, target evidence score, behavior evidence score, and uncertain evidence score is obtained. A danger probability is calculated based on a fusion function, and a dangerous behavior is determined when the danger probability is greater than or equal to a preset risk threshold, thus enabling effective reasoning even when information is missing or conflicting.

[0110] In summary, this embodiment adaptively selects between rule-based inference paths and uncertain inference paths through integrity and consistency scores, and achieves dual-path collaboration by injecting evidence and switching to uncertain inference paths when rules are partially matched. This solves the technical problem in the prior art where a single path cannot adapt to different semantic information quality conditions and lacks an effective fault tolerance mechanism when information is missing or conflicting. It achieves both accuracy and robustness in dangerous behavior recognition under different information conditions, thereby further improving the accuracy and reliability of dangerous behavior recognition in complex scenarios.

[0111] For example, to help understand the implementation process of the dual-path reasoning decision-making in this embodiment combined with the above embodiment two, please refer to... Figure 3 , Figure 3 This is a simplified flowchart illustrating the dual-path reasoning decision-making process provided in Embodiment 2 of this application. Specifically: First, obtain the multi-feature semantic object F constructed in step S20. This multi-feature semantic object is the core data structure connecting the perception layer and the reasoning layer. It contains multiple semantic slots such as scene semantics, target semantics, behavior semantics, uncertainty semantics and reasoning semantics, and carries all the semantic information required for subsequent dual-path reasoning decision-making.

[0112] Then, the system performs semantic integrity evaluation and semantic consistency evaluation on the multi-feature semantic object, and calculates the integrity score. and consistency score Among them, semantic integrity evaluation is used to quantify the completeness of semantic information, while semantic consistency evaluation is used to quantify the degree of logical conflict between semantic elements.

[0113] Subsequently, the system determines whether, based on the integrity score and the consistency score, it is correct. and .

[0114] If the judgment result is yes, that is, the integrity score is greater than or equal to the first threshold and the consistency score is greater than or equal to the second threshold, then the system executes the rule reasoning path. In the rule reasoning path, the system reads a candidate rule list from the multi-feature semantic object and performs conditional matching between the multi-feature semantic object and the rules in the candidate rule list. If there is a rule that completely matches the multi-feature semantic object, then a dangerous behavior is determined to exist, and a dangerous event is output. If there is no rule that completely matches the multi-feature semantic object, but there is a rule that partially matches the multi-feature semantic object, then the partial matching degree is calculated. The partial matching degree and the rules of the partial matching are then used as evidence and injected into the fused evidence vector of the uncertain inference path, and then the process is transferred to the uncertain inference path.

[0115] If the judgment result is negative, i.e., the integrity score is less than the first threshold or the consistency score is less than the second threshold, the system executes an uncertain inference path. In the uncertain inference path, the system obtains a fused evidence vector. ,in Scoring of scene evidence, Score the target evidence. Scoring behavioral evidence, Scoring for evidence of uncertainty This represents the partial matching degree of each candidate rule. Based on the fused evidence vector, the system calculates the hazard probability using a fusion function. and judge Whether it is valid or not.

[0116] like If a dangerous behavior is detected, a dangerous event is output, and the reasoning result is sent to step S50 for alarm output. If the observation record is not found, it will be added to the continuous observation queue for subsequent trend analysis and early warning.

[0117] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the video image dangerous behavior recognition and alarm method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0118] This application also provides a video image dangerous behavior recognition and alarm device, please refer to... Figure 4The video image dangerous behavior recognition and alarm device includes: The target monitoring module 10 is used to acquire video images, detect and continuously analyze target objects in the video images, and generate motion trajectories and behavior sequences of the target objects. The semantic analysis construction module 20 is used to perform visual semantic analysis on the motion trajectory and behavior sequence, extract multi-dimensional semantic features, and construct a multi-feature semantic object based on the multi-dimensional semantic features. The quality assessment module 30 is used to perform semantic integrity evaluation and semantic consistency evaluation on the multi-feature semantic object to obtain integrity score and consistency score; The reasoning module 40 is used to execute a rule-based reasoning path or an uncertain reasoning path based on the integrity score and the consistency score, wherein the rule-based reasoning path is based on condition matching of deterministic reasoning rules, and the uncertain reasoning path is based on probability determination of multi-dimensional evidence fusion. The alarm module 50 is used to generate dangerous event alarm information based on the reasoning results, and to issue an alarm based on the dangerous event alarm information; The reasoning module 10 is also used to execute a rule reasoning path when the integrity score is greater than or equal to the first threshold and the consistency score is greater than or equal to the second threshold. When the integrity score is less than the first threshold or the consistency score is less than the second threshold, an uncertain inference path is executed.

[0119] In one embodiment, the inference module 40 is further configured to read a list of candidate rules from the multi-feature semantic object; The multi-feature semantic object is conditionally matched with the rules in the candidate rule list; If a rule exists that perfectly matches the multi-feature semantic object, then dangerous behavior is determined to exist. If there is no rule that completely matches the multi-feature semantic object, but there is a rule that partially matches the multi-feature semantic object, then calculate the partial matching degree. The partial matching degree and the rules of partial matching are used as evidence and injected into the fused evidence vector of the uncertain inference path to execute the uncertain inference path.

[0120] In one embodiment, the reasoning module 40 is further configured to obtain a fused evidence vector, wherein the fused evidence vector includes at least a scene evidence score, a target evidence score, a behavioral evidence score, and an uncertainty evidence score; Based on the fused evidence vector, the probability of danger is calculated using the fusion function; If the probability of danger is greater than or equal to a preset risk threshold, then dangerous behavior is determined to exist.

[0121] In one embodiment, the quality assessment module 30 is further configured to traverse a preset set of integrity semantic slots; If a corresponding semantic slot exists in the multi-feature semantic object, the preset weights corresponding to the semantic slots are accumulated to obtain an integrity score. Iterate through the pre-defined set of consistency constraints; If the multi-feature semantic object does not violate the consistency constraint of the current traversal, the preset weights corresponding to the consistency constraint are accumulated to obtain a consistency score.

[0122] In one embodiment, the semantic analysis construction module 20 is further configured to obtain scene semantic features, target semantic features, and behavioral semantic features based on the multi-dimensional semantic features; The scene semantic features, the target semantic features, and the behavior semantic features are semantically aligned according to the same target identifier and the same time window; A multi-feature semantic object is constructed based on the semantically aligned scene semantic features, target semantic features, and behavior semantic features.

[0123] In one embodiment, the reasoning module 40 is further configured to obtain scene semantics, target semantics, and behavioral semantics from the multi-feature semantic object; Calculate the scene evidence score based on the scene confidence and scene risk level in the scene semantics; Calculate the target evidence score based on the trajectory analysis confidence level and target location risk coefficient in the target semantics; Calculate the behavioral evidence score based on the behavioral confidence and behavioral risk coefficient in the behavioral semantics. Calculate the uncertainty evidence score based on the integrity score and the consistency score; The fused evidence vector is generated based on the scenario evidence score, the target evidence score, the behavioral evidence score, and the uncertainty evidence score.

[0124] The video image dangerous behavior recognition and alarm device provided in this application, employing the video image dangerous behavior recognition and alarm method described in the above embodiments, can solve the technical problem of how to achieve unified coordination and fault-tolerant decision-making of multi-model information, thereby improving the accuracy and reliability of dangerous behavior recognition. Compared with the prior art, the beneficial effects of the video image dangerous behavior recognition and alarm device provided in this application are the same as those of the video image dangerous behavior recognition and alarm method provided in the above embodiments, and other technical features in the video image dangerous behavior recognition and alarm device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0125] This application provides a video image dangerous behavior recognition and alarm device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the video image dangerous behavior recognition and alarm method in the above embodiment 1.

[0126] The following is for reference. Figure 5 This document illustrates a structural schematic diagram of a video image dangerous behavior recognition and alarm device suitable for implementing embodiments of this application. The video image dangerous behavior recognition and alarm device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), tablets, PMPs (Portable Media Players), and vehicle terminals (e.g., vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 5 The video image dangerous behavior recognition and alarm device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0127] like Figure 5As shown, the video image dangerous behavior recognition alarm device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the video image dangerous behavior recognition alarm device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the video image dangerous behavior recognition alarm device to exchange data with other devices wirelessly or via wired communication. Although the figure shows a video image dangerous behavior recognition alarm device with various systems, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0128] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0129] The video image dangerous behavior recognition and alarm device provided in this application, employing the video image dangerous behavior recognition and alarm method described in the above embodiments, can solve the technical problem of how to achieve unified coordination and fault-tolerant decision-making of multi-model information, thereby improving the accuracy and reliability of dangerous behavior recognition. Compared with the prior art, the beneficial effects of the video image dangerous behavior recognition and alarm device provided in this application are the same as those of the video image dangerous behavior recognition and alarm method provided in the above embodiments, and other technical features in this video image dangerous behavior recognition and alarm device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0130] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0131] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0132] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the video image dangerous behavior recognition and alarm method in the above embodiments.

[0133] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0134] The aforementioned computer-readable storage medium may be included in the video image dangerous behavior recognition alarm device; or it may exist independently and not be assembled into the video image dangerous behavior recognition alarm device.

[0135] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by the video image dangerous behavior recognition and alarm device, the video image dangerous behavior recognition and alarm device performs the following actions: acquires video images; detects and continuously analyzes the target objects in the video images to generate the motion trajectory and behavior sequence of the target objects; performs visual semantic analysis on the motion trajectory and behavior sequence to extract multi-dimensional semantic features, and constructs multi-feature semantic objects based on the multi-dimensional semantic features; performs semantic integrity evaluation and semantic consistency evaluation on the multi-feature semantic objects to obtain integrity scores and consistency scores; executes a rule-based reasoning path or an uncertain reasoning path based on the integrity scores and consistency scores, wherein the rule-based reasoning path performs condition matching based on deterministic reasoning rules, and the uncertain reasoning path performs probability determination based on multi-dimensional evidence fusion; generates dangerous event alarm information based on the reasoning results, and issues an alarm based on the dangerous event alarm information.

[0136] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0137] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0138] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0139] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described video image dangerous behavior recognition and alarm method. This solves the technical problem of how to achieve unified coordination and fault-tolerant decision-making of multi-model information, thereby improving the accuracy and reliability of dangerous behavior recognition. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the video image dangerous behavior recognition and alarm method provided in the above embodiments, and will not be repeated here.

[0140] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the video image dangerous behavior recognition and alarm method described above.

[0141] The computer program product provided in this application can solve the technical problem of how to achieve unified coordination and fault-tolerant decision-making of multi-model information, thereby improving the accuracy and reliability of dangerous behavior identification. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the video image dangerous behavior identification and alarm method provided in the above embodiments, and will not be repeated here.

[0142] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for identifying and alerting dangerous behaviors in video images, characterized in that, The method includes: Acquire video images, detect and perform continuous trajectory analysis on target objects in the video images, and generate motion trajectories and behavior sequences of the target objects; Visual semantic analysis is performed on the motion trajectory and behavior sequence to extract multi-dimensional semantic features, and a multi-feature semantic object is constructed based on the multi-dimensional semantic features; The semantic integrity and semantic consistency of the multi-feature semantic object are evaluated to obtain an integrity score and a consistency score. Based on the integrity score and the consistency score, a rule-based reasoning path or an uncertain reasoning path is executed, wherein the rule-based reasoning path is based on condition matching based on deterministic reasoning rules, and the uncertain reasoning path is based on probability determination based on multi-dimensional evidence fusion. Based on the reasoning results, generate danger event alarm information, and issue alarms based on the danger event alarm information; The step of executing a rule-based reasoning path or an uncertain reasoning path based on the integrity score and the consistency score includes: When the integrity score is greater than or equal to the first threshold and the consistency score is greater than or equal to the second threshold, the rule reasoning path is executed. When the integrity score is less than the first threshold or the consistency score is less than the second threshold, an uncertain inference path is executed.

2. The method as described in claim 1, characterized in that, The steps of the execution rule inference path include: Read the candidate rule list from the multi-feature semantic object; The multi-feature semantic object is conditionally matched with the rules in the candidate rule list; If a rule exists that perfectly matches the multi-feature semantic object, then dangerous behavior is determined to exist. If there is no rule that completely matches the multi-feature semantic object, but there is a rule that partially matches the multi-feature semantic object, then calculate the partial matching degree. The partial matching degree and the rules of partial matching are used as evidence and injected into the fused evidence vector of the uncertain inference path to execute the uncertain inference path.

3. The method as described in claim 1, characterized in that, The steps for executing the uncertain inference path include: Obtain a fused evidence vector, wherein the fused evidence vector includes at least a scene evidence score, a target evidence score, a behavioral evidence score, and an uncertainty evidence score; Based on the fused evidence vector, the probability of danger is calculated using the fusion function; If the probability of danger is greater than or equal to a preset risk threshold, then dangerous behavior is determined to exist.

4. The method as described in claim 1, characterized in that, The steps of performing semantic integrity evaluation and semantic consistency evaluation on the multi-feature semantic object to obtain integrity score and consistency score include: Iterate through the pre-defined set of integrity semantic slots; If a corresponding semantic slot exists in the multi-feature semantic object, the preset weights corresponding to the semantic slots are accumulated to obtain an integrity score. Iterate through the pre-defined set of consistency constraints; If the multi-feature semantic object does not violate the consistency constraint of the current traversal, the preset weights corresponding to the consistency constraint are accumulated to obtain a consistency score.

5. The method as described in claim 1, characterized in that, The step of constructing a multi-feature semantic object based on the multi-dimensional semantic features includes: Based on the multi-dimensional semantic features, scene semantic features, target semantic features, and behavior semantic features are obtained; The scene semantic features, the target semantic features, and the behavior semantic features are semantically aligned according to the same target identifier and the same time window; A multi-feature semantic object is constructed based on the semantically aligned scene semantic features, target semantic features, and behavior semantic features.

6. The method as described in any one of claims 2 or 3, characterized in that, The steps for generating the fused evidence vector include: Obtain scene semantics, target semantics, and behavior semantics from the multi-feature semantic object; Calculate the scene evidence score based on the scene confidence and scene risk level in the scene semantics; Calculate the target evidence score based on the trajectory analysis confidence level and target location risk coefficient in the target semantics; Calculate the behavioral evidence score based on the behavioral confidence and behavioral risk coefficient in the behavioral semantics. Calculate the uncertainty evidence score based on the integrity score and the consistency score; The fused evidence vector is generated based on the scenario evidence score, the target evidence score, the behavioral evidence score, and the uncertainty evidence score.

7. A video image dangerous behavior recognition and alarm device, characterized in that, The device includes: The target monitoring module is used to acquire video images, detect and continuously analyze target objects in the video images, and generate motion trajectories and behavior sequences of the target objects. The semantic analysis construction module is used to perform visual semantic analysis on the motion trajectory and behavior sequence, extract multi-dimensional semantic features, and construct a multi-feature semantic object based on the multi-dimensional semantic features. The quality assessment module is used to perform semantic integrity evaluation and semantic consistency evaluation on the multi-feature semantic object, and obtain integrity score and consistency score; The reasoning module is used to execute a rule-based reasoning path or an uncertain reasoning path based on the integrity score and the consistency score, wherein the rule-based reasoning path is based on condition matching based on deterministic reasoning rules, and the uncertain reasoning path is based on probability determination based on multi-dimensional evidence fusion. The alarm module is used to generate dangerous event alarm information based on the reasoning results, and to issue alarms based on the dangerous event alarm information; The reasoning module is also used to execute a rule-based reasoning path when the integrity score is greater than or equal to a first threshold and the consistency score is greater than or equal to a second threshold. When the integrity score is less than the first threshold or the consistency score is less than the second threshold, an uncertain inference path is executed.

8. A video image dangerous behavior recognition and alarm device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the video image dangerous behavior recognition and alarm method as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the video image dangerous behavior recognition and alarm method as described in any one of claims 1 to 6.

Images