Industrial scene anti-violation active safety monitoring method, device, medium, equipment and product

Abstract

This disclosure relates to a method, device, medium, equipment, and product for proactive safety monitoring against violations in industrial scenarios, belonging to the field of industrial internet technology. It can identify cross-system logical violations and implement proactive tool invocation planning. The method includes: acquiring video streams and event signals from the industrial scenario; performing abnormal event detection based on the video stream and event signals; generating event objects when candidate abnormal events are detected; and extracting audio and video slices corresponding to the event objects from the video stream. A context snapshot corresponding to the event object is determined, and the audio and video slices are converted into semantic descriptions. A tool invocation sequence is generated based on the event object, semantic description, context snapshot, and a preset tool registry. The tool invocation sequence is then subjected to consistency verification according to preset constraint rules to obtain a verification result. If the verification result indicates that the consistency verification has passed, an evidence package is generated, and relevant equipment in the industrial scenario is controlled according to the risk level of the evidence package.

Description

Technical Field

[0001] This disclosure relates to the field of industrial internet technology, specifically to a method, device, medium, equipment, and product for proactive safety monitoring against violations in industrial scenarios. Background Technology

[0002] In related technologies, monitoring methods for safe production in industrial settings mainly rely on two types of systems: control and monitoring systems and video surveillance AI systems. Control and monitoring systems focus on the acquisition of physical parameters and interlocking protection, lacking the ability to perceive unstructured data from the field. Video surveillance AI systems focus on visual appearance recognition and cannot understand complex industrial business logic. Summary of the Invention

[0003] To address the aforementioned technical issues, this disclosure provides a method, device, medium, equipment, and product for proactive safety monitoring against violations in industrial scenarios.

[0004] To achieve the above objectives, firstly, this disclosure provides a proactive safety monitoring method for preventing violations in industrial scenarios, including: Acquire video streams and event signals from an industrial scene, and perform abnormal event detection based on the video streams and event signals to obtain event detection results; When the event detection result indicates that a candidate abnormal event has been detected, an event object is generated. An audio and video slice corresponding to the event object is extracted from the video stream. The event object includes an event identifier, a time window, a region identifier or camera identifier, an event type or action candidate, a confidence level, and a segment reference. Determine the context snapshot corresponding to the event object, convert the audio and video slices into semantic descriptions, generate a tool call sequence based on the event object, the semantic description, the context snapshot, and a preset tool registry, and perform consistency verification on the tool call sequence according to preset constraint rules to obtain the verification result; If the verification result indicates that the consistency verification has passed, an evidence package is generated, and the relevant equipment in the industrial scenario is controlled according to the risk level of the evidence package.

[0005] Secondly, this disclosure provides an active safety monitoring device for preventing violations in industrial settings, comprising: The acquisition module is configured to acquire video streams and event signals from an industrial scene, and to perform abnormal event detection based on the video streams and event signals to obtain event detection results. The execution module is configured to generate an event object when the event detection result indicates that a candidate abnormal event has been detected, and to extract an audio and video slice corresponding to the event object from the video stream. The event object includes an event identifier, a time window, a region identifier or camera identifier, an event type or action candidate, a confidence level, and a segment reference. The verification module is configured to determine the context snapshot corresponding to the event object, convert the audio and video slices into semantic descriptions, generate a tool call sequence based on the event object, the semantic description, the context snapshot, and a preset tool registry, and perform consistency verification on the tool call sequence according to preset constraint rules to obtain the verification result. The control module is configured to generate an evidence package if the verification result indicates that the consistency verification has passed, and to control the relevant equipment in the industrial scenario according to the risk level of the evidence package.

[0006] Thirdly, this disclosure provides a non-transitory computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the method described in the first aspect.

[0007] Fourthly, this disclosure provides an electronic device, comprising: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the method described in the first aspect.

[0008] Fifthly, this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0009] The above technical solution detects abnormal events based on video streams and event signals in industrial scenarios. When a candidate abnormal event is detected, an event object is generated, and the corresponding audio and video slices are extracted from the video stream to achieve proactive detection and evidence collection of abnormal events. A context snapshot corresponding to the event object is determined, and the audio and video slices are converted into semantic descriptions, breaking down the barriers between video and data and enabling cross-system data acquisition. Based on the event object, semantic description, context snapshot, and a preset tool registry, a tool call sequence is generated. The tool call sequence is then validated for consistency according to preset constraint rules, and the validation result is obtained. Tools in the tool registry can be automatically invoked, and the consistency of the tool call sequence is validated based on rule constraints, ensuring the integrity of the evidence chain. Upon passing the consistency validation, an evidence package is generated, and relevant equipment in the industrial scenario is controlled according to the risk level of the evidence package, achieving automated control of relevant equipment in the industrial scenario without relying on any specific system or platform.

[0010] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0011] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart illustrating an industrial scenario proactive safety monitoring method for preventing violations, based on an exemplary embodiment of this disclosure.

[0012] Figure 2 This is a flowchart illustrating another proactive safety monitoring method for preventing violations in industrial scenarios, based on an exemplary embodiment of this disclosure.

[0013] Figure 3 This is a block diagram of an active safety monitoring device for preventing violations in an industrial setting, as illustrated by an exemplary embodiment of this disclosure.

[0014] Figure 4 This is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0015] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this disclosure.

[0016] It should be noted that all actions involving the acquisition of signals, information, or data in this disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with authorization from the owner of the relevant device.

[0017] As mentioned in the background section, in video detection in industrial scenarios, lightweight visual models are often used to achieve real-time edge detection (e.g., safety helmets, area intrusions, etc.), and when necessary, more powerful multimodal semantic models are invoked to perform semantic parsing on candidate events to obtain structured information such as actions, objects, number of people, and clothing. Industrial scenarios include, but are not limited to, high-risk industrial environments with strict safety regulations, such as thermal power plants, substations, coal mines, and chemical plants, where multi-source heterogeneous data (e.g., tickets, DCS, videos) exists.

[0018] Taking thermal power plants as an example, in the safe production of thermal power plants, traditional monitoring methods mainly rely on two types of systems: 1. Control and monitoring systems: such as SCADA (Supervisory Control And Data Acquisition) and DCS (Distributed Control System). These systems focus on the acquisition and interlocking protection of physical parameters (such as temperature, pressure, and switching quantities), but lack the ability to perceive unstructured data from the field (such as personnel behavior and equipment appearance).

[0019] 2. Traditional video surveillance AI (Artificial Intelligence): Focuses on the recognition of visual appearances (such as not wearing a safety helmet, area intrusion), but cannot understand complex industrial business logic (such as "the operation is compliant, but electronic invoice authorization has not been obtained" or "dual supervision is lacking").

[0020] Among related technologies, Chinese patent CN118334604A discloses a method and device for accident detection and dataset construction based on a multimodal large model. Its implementation typically includes: visual small model detection: using models such as YOLO to extract frames from the video stream and identify people, vehicles, or specific objects. Large model verification: when the confidence level of the small model is insufficient, the multimodal model is invoked for secondary confirmation. Alarm output: once visual recognition indicates an "abnormality," an alarm is directly triggered or a text report is generated. It is evident that this solution is essentially a unidirectional, open-loop visual perception system, its core logic being image interpretation. The large model verification and subsequent handling are usually implemented through manually preset trigger conditions and fixed procedures, lacking the ability for automated planning and incremental evidence collection under tool constraints.

[0021] In summary, the inventors have discovered the following technical defects in the relevant technology: 1. Lack of cross-system status acquisition and time alignment capabilities: It is difficult to distinguish the compliance differences of the same action under different authorizations or different device states based solely on video.

[0022] 2. Lack of rule-based logical verification: Most solutions only output detection results or alarms, and cannot make a consistent judgment between action semantics and constraints such as authorization, status or regional risk.

[0023] 3. Incomplete chain of evidence and lack of security grading in handling: Alarms usually only contain pictures or video clips, lacking snapshots of the current state and the basis for the hit rules, and lacking security gates and degrading strategies for control-related handling.

[0024] 4. Lack of automated tool planning and incremental evidence collection closed loop: Related technologies usually only call a small number of interfaces under preset rules or fixed processes; when the evidence is insufficient or the results are conflicting, it is impossible to automatically generate and execute an incremental "evidence collection, query, verification, and disposal" tool call sequence according to the current context, making it difficult to form a self-verifiable closed-loop disposal.

[0025] It is evident that the relevant technologies cannot identify cross-system logical violations, lack proactive automated tool planning capabilities, and suffer from incomplete chains of evidence.

[0026] In view of this, this disclosure provides a method, device, medium, equipment and product for proactive safety monitoring of violations in industrial scenarios, which can identify cross-system logical violations and realize proactive automatic tool invocation planning.

[0027] Figure 1 This is a flowchart illustrating an industrial scenario proactive safety monitoring method for preventing violations, based on an exemplary embodiment of this disclosure. Figure 1 As shown, this proactive safety monitoring method for preventing violations in industrial scenarios may include the following steps: In step S11, video streams and event signals from the industrial scene are acquired, and abnormal event detection is performed based on the video streams and event signals to obtain event detection results.

[0028] In step S12, when the event detection result indicates that a candidate abnormal event has been detected, an event object is generated. The audio and video slices corresponding to the event object are extracted from the video stream. The event object includes an event identifier, a time window, a region identifier or camera identifier, an event type or action candidate, a confidence level, and a segment reference.

[0029] In step S13, the context snapshot corresponding to the event object is determined, and the audio and video slices are converted into semantic descriptions. A tool call sequence is generated based on the event object, semantic description, context snapshot, and preset tool registry. The consistency of the tool call sequence is checked according to preset constraint rules to obtain the check result.

[0030] In step S14, if the verification result characterization passes the consistency verification, an evidence package is generated, and the relevant equipment in the industrial scenario is controlled according to the risk level of the evidence package.

[0031] The event object includes at least the event identifier (event_id), time window (time_window), area identifier (area_id) or camera identifier (camera_id), event type (event_type) or action candidate (action_candidate), confidence level, and clip reference (clip_ref).

[0032] It is worth noting that in this embodiment, the industrial scenario of a thermal power plant is mainly used as an example. Video streams from the thermal power plant can be acquired through cameras, and event signals can be obtained through the plant's edge gateway, sensors, or monitoring systems. A lightweight model deployed at the edge is used to monitor abnormal events in real time based on the video streams and event signals. This lightweight model includes, but is not limited to, the YOLOv8 model (You Only Look Once version 8) and motion detection algorithm models.

[0033] It should be understood that the planner automatically selects the required tools from the preset tool registry based on the event object, semantic description, and context snapshot, and generates the tool call sequence plan=[call1, call2, ...].

[0034] In this embodiment, the audio and video slices of the event object are generally 5-10 seconds long. The audio and video slices are input into the Vision-Language Model (VLM) to obtain the semantic description output by the Vision-Language Model, such as {“Subject”: “A maintenance worker”, “Action”: “Operating the handle of the No. 2 disconnect switch”, “Wearing”: “Safety helmet is worn”, “Time”: “10:05”}. The Vision-Language Model includes, but is not limited to, Qwen-VL (Qwen Vision-Language Model).

[0035] It is worth noting that proactive safety supervision refers to the automatic generation of supplementary certification plans and the use of tools to complete multi-dimensional evidence collection and verification after an incident is triggered, without manual scheduling, thereby achieving closed-loop handling; it does not refer to the advance prediction of accidents. The tools used include query, evidence collection, and handling interfaces. The tool call sequence is automatically generated by the planner under tool constraints, and operators do not need to preset fixed workflows.

[0036] The above technical solution detects abnormal events based on video streams and event signals in industrial scenarios. When a candidate abnormal event is detected, an event object is generated, and the corresponding audio and video slices are extracted from the video stream to achieve proactive detection and evidence collection of abnormal events. A context snapshot corresponding to the event object is determined, and the audio and video slices are converted into semantic descriptions, breaking down the barriers between video and data and enabling cross-system data acquisition. Based on the event object, semantic description, context snapshot, and a preset tool registry, a tool call sequence is generated. The tool call sequence is then validated for consistency according to preset constraint rules, and the validation result is obtained. Tools in the tool registry can be automatically invoked, and the consistency of the tool call sequence is validated based on rule constraints, ensuring the integrity of the evidence chain. Upon passing the consistency validation, an evidence package is generated, and relevant equipment in the industrial scenario is controlled according to the risk level of the evidence package, achieving automated control of relevant equipment in the industrial scenario without relying on any specific system or platform.

[0037] To facilitate a better understanding of the proactive safety monitoring method for preventing violations in industrial scenarios provided in this disclosure, the following detailed explanation of the proactive safety monitoring method for preventing violations in industrial scenarios is provided.

[0038] In one feasible embodiment, the preset tool registry includes tools and tool descriptors, and the proactive safety monitoring method for preventing violations in industrial scenarios may further include: If the verification result fails the consistency check, the tool descriptor corresponding to the tool call sequence is backfilled into the context snapshot to obtain a new context snapshot; A new tool invocation sequence is generated based on the event object, semantic description, new context snapshot, and preset tool registry; The new tool call sequence is subjected to consistency verification according to the preset constraint rules until the new tool call sequence passes the consistency verification.

[0039] It's worth noting that each tool in the default tool registry corresponds to a set of machine-resolvable tool descriptors. These descriptors include at least a tool identifier (tool_id), an input parameter structure (input_schema), an output structure (output_schema), a precondition, a risk level (risk_level), and an interface type (e.g., query, forensics, control). The tool registry may also include allowed call ranges and security constraints to restrict the planner to only generating tool call sequences within the allowed set.

[0040] Among them, the event object, tool descriptor and context snapshot are preferably encoded and interacted using the I-MCP (Industrial ModelContext Protocol) protocol message format; its transmission carrier can be HTTP / gRPC, message bus, event bus or industrial protocol gateway, etc. In other implementation methods, I-MCP can be used as an internal unified object model and converted into external protocol messages by the gateway.

[0041] It is worth noting that the executor invokes the tool according to the call sequence and fills the returned results back into the context snapshot; if the tool returns results as an I-MCP response message and fills the context snapshot, it can trigger incremental replanning to achieve closed-loop evidence collection and verification.

[0042] In one feasible embodiment, the evidence package includes a target context snapshot, video keyframes, a hit rule ID, and a timestamp; Controlling relevant equipment in industrial scenarios based on the risk level of the evidence package can include: If the risk level of the evidence package is low, a work order is generated based on the evidence package and sent to the central control room of the relevant equipment in the industrial scenario. If the risk level of the evidence package is high, and the confidence level of the evidence package is greater than the first preset value, then a control command is sent to the corresponding equipment in the industrial scene. If the confidence level of the evidence package is less than or equal to the first preset value, then an alarm message is generated and broadcast. The control command includes at least one of the following: stop command, reverse control command, deceleration control command, blocking control command, lock control command, parameter adjustment command, robot scheduling command, and autonomous operation command.

[0043] It is worth noting that the evidence package also includes a hash digest or digital signature, as well as a generation timestamp, which can be used to verify the integrity of the evidence content and for audit traceability. The first preset value can be preset according to the safety requirements of the thermal power plant; in this embodiment, the first preset value is set to 0.99.

[0044] It should be understood that the target context snapshot, video keyframes, hit rule ID, and timestamp corresponding to the event object are packaged into an immutable evidence package to solidify the evidence.

[0045] It is worth noting that, taking a thermal power plant as an example, the risk level of an evidence package can be determined through a safety gate. Specifically, the safety gate determines the potential risk of the event corresponding to the evidence package, as well as the thermal power plant's tolerance and handling capabilities for faults, to classify the risk level of the evidence package. In this embodiment, the risk level of the evidence package includes low-risk and high-risk levels. When the risk level of the evidence package is low-risk, a corresponding work order is generated based on the evidence package and pushed to the thermal power plant's control system. When the risk level of the evidence package is high-risk, a safety degradation strategy is triggered. The safety degradation strategy includes: if the confidence level of the evidence package is greater than 0.99 (and receives secondary manual confirmation), a shutdown command is issued to the corresponding equipment in the thermal power plant; if the confidence level of the evidence package is less than or equal to 0.99, it is automatically downgraded to a broadcast alarm or broadcast notification.

[0046] In one feasible embodiment, determining the context snapshot corresponding to the event object may include: Retrieve state data from different sources corresponding to the event object and map the state data from different sources to a unified state key-value item; Align multiple state key-value items to the time window in the event object, and determine the time deviation corresponding to each state key-value item; If a conflict is determined among multiple state key-value items based on the time deviations corresponding to the multiple state key-value items, the multiple state key-value items are fused according to the source priority and time freshness of the state data of the corresponding state key-value items to generate a context snapshot. The context snapshot includes a snapshot timestamp, a source identifier, a set of state key-value items, and a quality identifier.

[0047] It is worth noting that, based on the adapter, at least one authoritative context source can be accessed in a pluggable manner to obtain state data from different sources. Semantic mapping, event alignment, source credibility labeling and conflict resolution are performed on the state data from different sources to generate a context snapshot for verification.

[0048] Semantic mapping can include mapping raw fields from different sources (such as tags, point tables, and API return fields) to unified state key-value items (state_items) (containing key, value, unit, and timestamp). Time alignment can include aligning state_items to the event time window (e.g., using nearest neighbor, interpolation, or range aggregation) and recording time deviations. Source credibility labeling and conflict resolution can include attaching source_id, proofance, quality_flag, and confidence to each state key-value item; when multiple sources conflict for the same key, selection or merging is performed according to a preset strategy (e.g., source priority + time freshness + consistency check), and conflict records are retained.

[0049] In one feasible embodiment, the status data includes physical status data of the industrial scenario, authorization status data, and equipment status data. Obtaining status data from different sources corresponding to the event object may include: Physical status data of industrial scenarios can be obtained through edge gateways, IoT sensors, equipment, instruments, or camera modules. Authorization status data in industrial settings can be obtained through electronic ticketing systems, access control systems, and personnel qualification management systems. Obtain equipment status data in industrial settings through control and monitoring systems or historical databases.

[0050] It is worth noting that authoritative contextual sources include, but are not limited to: 1. Direct connection sensing channel: The physical status of the thermal power plant is directly acquired through edge gateways, IoT sensors (such as vibration, temperature, gas), equipment-side interfaces, or instrument reading cameras; 2. Electronic Authorization Channel: Access to electronic invoices, access control, personnel qualification management systems, etc., to obtain authorization status data of thermal power plants, such as authorization status, qualification status, and access status; 3. Existing system channels: Access control and monitoring systems or historical databases (such as SCADA, DCS, historical databases, etc.) to obtain equipment status data of thermal power plants, such as measurement point status and operation records.

[0051] It should be understood that when any channel is unavailable, the adapter can generate a degraded snapshot based on the remaining channels, which does not constitute a necessary dependency on a single system.

[0052] In one feasible embodiment, determining the context snapshot corresponding to the event object may include: Determine the confidence level of the event object and the initial context snapshot corresponding to the event object; If the confidence level of the event object is less than the second preset value or if there are missing fields in the initial context snapshot, a complete evidence collection strategy is generated. The complete evidence collection strategy is executed to confirm the new confidence level of the event object and the new context snapshot corresponding to the event object, until the new confidence level of the event object is greater than or equal to the second preset value and the new context snapshot does not have any missing fields. The new context snapshot is then used as the context snapshot corresponding to the event object.

[0053] It is worth noting that when a context snapshot contains missing fields or its quality identifier is below a threshold, an active evidence collection mechanism is triggered to generate a supplementary evidence strategy. The second threshold can be preset based on the accuracy of safety supervision; this disclosure does not impose any limitations on it.

[0054] Specifically, when the confidence level of an event object is lower than the threshold or when there are missing fields in the context snapshot, the planner automatically generates a supplementary evidence collection strategy. The supplementary evidence collection strategy includes, but is not limited to, calling the zoom evidence collection tool, calling the identity verification tool, and calling the regional risk reading tool to supplement the minimum evidence set required for verification.

[0055] The following describes the complete process of the proactive safety monitoring method for preventing violations in industrial scenarios provided in this disclosure. This proactive safety monitoring method for preventing violations in industrial scenarios may include: generating event objects at the edge and extracting video segments corresponding to the event objects from the video stream of a thermal power plant; converting the video segments into semantic descriptions using a multimodal semantic model; obtaining a context snapshot that is time-aligned with the event object through an authoritative context adapter; and having a planner driven by a large language model automatically generate and execute a tool call sequence for evidence collection, verification, and disposal under the constraints of a tool registry, completing the consistency verification of the tool call sequence according to preset constraint rules, generating the corresponding evidence package, and triggering the risk-level disposal of the evidence package through a safety gate.

[0056] Specifically, the aforementioned proactive safety monitoring method for preventing violations in industrial scenarios adopts a three-tier architecture of sentinel, analysis, and decision-making. Figure 2 As shown, the process is as follows: I, L1 Edge Sentinel Filtering a. Acquire full video streams from the power plant via cameras, as well as event signals from edge gateways, sensors, or monitoring systems.

[0057] b. Real-time monitoring of candidate abnormal events based on video streams and event signals using lightweight models (such as YOLOv8 and motion detection algorithms) deployed at the edge.

[0058] c. Upon detecting a candidate abnormal event (such as personnel entering the booster station or abnormal vibration sounds in the coal mill area), a structured event object is generated, and a 5-10 second audio-visual slice is extracted from the video stream. The structured event object includes at least: an event identifier (event_id), a time window (time_window), an area or camera identifier (area_id / camera_id), an event type or action candidate (event_type / action_candidate), a confidence score, and a clip reference (clip_ref).

[0059] II. L2 Semantic Analysis The system invokes a cloud-based or server-side multimodal semantic model (VLM, such as Qwen-VL) to convert audio and video slices into semantic descriptions, such as {“Subject”: “A maintenance worker”, “Action”: “Operating the handle of the No. 2 isolating switch”, “Wearing”: “Safety helmet is worn”, “Time”: “10:05”}. The VLM does not perform violation determination; it only converts unstructured footage into structured semantic descriptions.

[0060] III. L3 Consistency Check a. The tool invocation planning and execution module obtains the context snapshot corresponding to the event object through the authority context adapter.

[0061] Specifically, the authoritative context adapter connects to at least one authoritative context source in a pluggable manner, performing semantic mapping, time alignment, source credibility labeling, and conflict resolution on state data from different sources, generating a context snapshot for verification. Authoritative context sources include, but are not limited to: direct-connection sensing channels, electronic authorization channels, and existing system channels. When any channel becomes unavailable, the authoritative context adapter can generate a degraded snapshot based on the remaining channels, without constituting a necessary dependency on a single system.

[0062] b. The tool call planning and execution module includes a planner and an executor. The planner is used to automatically select the required tools from the tool registry and generate a tool call sequence plan=[call1, call2, ...] based on the event object, semantic description and context snapshot. The executor is used to call the tools according to the tool call sequence and fill the returned results back into the context snapshot. If the tool returns a result and fills the context snapshot with an I-MCP response message, it can trigger incremental replanning to achieve closed-loop evidence collection and verification.

[0063] Each tool in the tool registry corresponds to a set of machine-resolvable tool descriptors. Each tool descriptor includes at least: a tool identifier (tool_id), an input parameter structure (input_schema), an output structure (output_schema), a precondition for invocation, a risk level (risk_level), and an interface type (e.g., query, forensics, control). The tool registry also includes allowed invocation scope and security constraints to restrict the planner to only generating tool invocation sequences within the allowed set. Event objects, context snapshots, and tool descriptors are preferably encoded and interacted using the I-MCP (Industrial Model Context Protocol) message format; its transmission bearer can be HTTP / gRPC, a message bus / event bus, or an industrial protocol gateway, etc. Alternatively, in other implementations, I-MCP can be used as an internal unified object model and converted into external protocol messages by the gateway.

[0064] c. Dynamic evidence collection mechanism: When the confidence level of an event object is lower than the threshold or there are missing fields in the context snapshot, the planner automatically generates a supplementary evidence collection strategy. The supplementary evidence collection strategy includes, but is not limited to, calling the zoom evidence collection tool, calling the identity verification tool, and calling the regional risk reading tool to supplement the minimum evidence set required for verification.

[0065] d. Perform consistency checks on the tool call sequence based on preset constraint rules. For example, when operating live equipment, a valid operation ticket is required. The comparison process is as follows: visual action = "operation" + ticket status = "no ticket" + direct voltage sensor = "live" = constraint violation. The conclusion is: "Logical violation due to no ticket".

[0066] IV. Level 4 Safety Classification and Evidence Consolidation a. Evidence Enclosure: Package "video keyframes + context snapshots (such as tickets, sensor values) + hit rule ID + timestamp" to generate an immutable evidence package. The evidence package includes a hash digest or digital signature and records the generation timestamp for integrity verification and audit traceability of the evidence content.

[0067] b. Threshold verification and degradation strategy: Based on the security gate, graded handling is performed according to the risk level of the evidence package.

[0068] c. Output the results of the graded treatment.

[0069] It is worth noting that L1 can be replaced by traditional video analysis (background modeling, motion detection), rule-based triggering, or edge-based lightweight detection networks (such as YOLO series, RT-DETR, PP-YOLO, etc.); it can also be replaced by sensor events such as acoustics / vibration to replace some visual triggering. L2 can be replaced by arbitrary visual language models (VLM) or video understanding models; it can also be replaced by a multi-model cascade approach such as "object detection / human pose / behavior recognition / OCR" to generate structured semantics (e.g., detecting personnel + key points + action classification + equipment nameplate OCR) to replace end-to-end VLM. Regarding preset constraint rules, vector retrieval, knowledge graph retrieval, structured rule base, or traditional rule engines (such as Drools) can be used; it can also be maintained by configuration files or database tables. L4 evidence package solidification can be replaced by hash chains, digital signatures, trusted timestamp services, or third-party audit and evidence storage systems.

[0070] The automatic planning and execution module for tool calls can be implemented by a generative planning model (such as a large language model based on the Transformer architecture) with function or tool call capabilities. This allows it to handle unknown or complex abnormal scenarios based on tool descriptors, dynamically generate and execute non-preset call plans, and perform adaptive incremental replanning after obtaining intermediate results. In some scenarios, a hybrid approach of generative models and rule constraints can also be used; or a non-generative automatic planner (such as a finite state machine (FSM), behavior tree, or HTN planning) can be used to achieve some of the ability to automatically generate call sequences.

[0071] The authoritative context adapter can be implemented using message bus / event bus (MQTT / Kafka, etc.), OPC UA / Modbus and other industrial protocol gateways, or various system API adaptations. Optionally, I-MCP can draw on the tool discovery / invocation paradigm of the general MCP (Model Context Protocol) in terms of interaction abstraction, but adds or strengthens extensions such as event objects, time-aligned snapshots, source credibility semantics, and risk classification / security gate constraints for industrial safety monitoring scenarios, thus distinguishing it from the general tool invocation specification of the general MCP.

[0072] The following three scenarios illustrate the proactive safety monitoring methods for preventing violations in industrial settings.

[0073] Scenario 1: Interception of device operation compliance based on electronic invoice verification It is worth noting that this scenario demonstrates how the system integrates IT (invoices) and OT (video) data to identify hidden violations such as "compliant actions but no authorization".

[0074] 1. Event Structure: L1 Sentinel detects personnel with significant physical movements in the "No. 2 Booster Station Isolation Switch" area and captures video. L2 Parses and outputs semantics: {"subject": "Worker_A", "action": "operating_handle" (operating handle), "target": "Switch_201", "time": "14:30"}.

[0075] 2. Multi-source context acquisition: The consistency verification engine generates an automated forensics plan through the authoritative context adapter. a. Verify identity: Use facial recognition tool to confirm that Worker_A is "Maintenance Team Leader Li Si".

[0076] b. Query Authorization (Channel B): Query the electronic ticket system via the adapter. Snapshot returned: Ticket_Status: "Expired" (Ticket expired).

[0077] c. Check status (Channel A): Connect the voltage sensor directly to the adapter to confirm that Switch_201 is currently in the "powered" state.

[0078] 3. Constraint verification: a. Loading rules: Operating live equipment MUST HAVE a valid operation ticket.

[0079] b. Logical comparison: Visual action ("operation") + Ticket status ("expired") = violation.

[0080] 4. Safety classification and handling: a. Triggering the safety gate: The risk level is determined to be "extremely high".

[0081] b. The system directly triggers a loudspeaker broadcast on-site via the direct connection channel: "Li Si, please note that the current operation ticket has expired and operation is prohibited!" At the same time, a red pop-up alarm is pushed to the central control room.

[0082] Scenario 2: Verification of a dual guardianship system It is worth noting that no invoice system is required.

[0083] 1. Visual perception: L2's VLM recognizes "high-voltage room switching operation, only 1 person in the picture".

[0084] 2. Rule retrieval: The consistency verification engine retrieved the procedure and found that "the switching operation must be supervised by two people (at least two people are present)".

[0085] 3. Logical verification: The actual number of people (1) < the requirement of the procedure (2).

[0086] 4. Action taken: The violation is determined to be "violation due to lack of supervision," triggering an alarm.

[0087] Scenario 3: Electronic Fence Verification Based on Regional Risk It is worth noting that no ticketing system is required; the sensor is directly connected.

[0088] 1. Event triggered: The sentry detects that a person has crossed the electronic fence and entered area A.

[0089] 2. Context Acquisition: The adapter reads the toxic gas sensor in area A through the direct connection channel, and the value is displayed as "high toxicity".

[0090] 3. Visual analysis: L2's VLM identifies that the person is wearing a "regular mask".

[0091] 4. Logical verification: Area status (highly toxic) + wearing (ordinary mask) != Procedure requirements (positive pressure respirator).

[0092] 5. Handling: If the violation is deemed as "insufficient protection", an audible and visual alarm will be triggered. If the handling strategy determines it to be a high-risk action, an access control request will be generated according to the safety gate mechanism and executed after the preset safety threshold is met and (optionally) manual confirmation is required. Otherwise, it will automatically be downgraded to a broadcast alarm / work order notification.

[0093] The above technical solution achieves proactive safety monitoring by invoking the automatic planning and execution module through tool calls. The authoritative context adapter breaks down barriers between video, data, and invoices, resolving "cross-system logical violations." A safety gate mechanism is introduced to ensure industrial-grade safety when AI intervenes in control, improving the reliability and security of anti-violation safety monitoring results. It supports direct connection to sensing channels and pluggable multi-source context access, operating independently without relying on any specific legacy system or vendor platform. The authoritative context adapter supports pluggable access to multiple authoritative context sources, optionally reusing existing monitoring / control / historical database / invoice system interfaces as one of the context sources to utilize existing data and interface capabilities, ensuring compatibility and interoperability.

[0094] Based on the same inventive concept, this disclosure also provides an active safety monitoring device for industrial scenarios to prevent violations, such as... Figure 3 As shown, the industrial scenario anti-violation active safety monitoring device 300 includes: The acquisition module 301 is configured to acquire video streams and event signals from an industrial scene, and perform abnormal event detection based on the video streams and event signals to obtain event detection results. Execution module 302 is configured to generate an event object when the event detection result indicates that a candidate abnormal event has been detected, and to extract an audio and video slice corresponding to the event object from the video stream. The event object includes an event identifier, a time window, a region identifier or camera identifier, an event type or action candidate, a confidence level, and a segment reference. The verification module 303 is configured to determine the context snapshot corresponding to the event object, convert the audio and video slices into semantic descriptions, generate a tool call sequence based on the event object, the semantic description, the context snapshot, and a preset tool registry, and perform consistency verification on the tool call sequence according to preset constraint rules to obtain a verification result. The control module 304 is configured to generate an evidence package when the verification result indicates that the consistency verification has passed, and to control the relevant equipment in the industrial scenario according to the risk level of the evidence package.

[0095] Optionally, the preset tool registry includes tools and tool descriptors. The control module 304 is also configured to, in the case that the verification result characterization fails the consistency verification, fill the tool descriptor corresponding to the tool call sequence back into the context snapshot to obtain a new context snapshot. A new tool invocation sequence is generated based on the event object, the semantic description, the new context snapshot, and the preset tool registry. The new tool call sequence is subjected to consistency verification according to preset constraint rules until the new tool call sequence passes the consistency verification.

[0096] Optionally, the evidence package includes a target context snapshot, video keyframes, hit rule ID, and timestamp; the control module 304 is configured to generate a work order based on the evidence package when the risk level of the evidence package is low risk level, and send the work order to the central control room in the industrial scenario; If the risk level of the evidence package is high, and the confidence level of the evidence package is greater than a first preset value, a control command is sent to the corresponding device in the industrial scenario. If the confidence level of the evidence package is less than or equal to the first preset value, an alarm message is generated and broadcast. The control command includes at least one of the following: stop command, reverse control command, deceleration control command, blocking control command, lock control command, parameter adjustment command, robot scheduling command, and autonomous operation command.

[0097] Optionally, the verification module 303 is configured to obtain state data from different sources corresponding to the event object and map the state data from different sources to a unified state key-value item. Align the multiple state key-value items to the time window in the event object, and determine the time deviation corresponding to each state key-value item; If a conflict is determined among the multiple state key-value items based on the time deviations corresponding to the multiple state key-value items, the multiple state key-value items are fused according to the source priority and time freshness of the state data corresponding to the state key-value items to generate a context snapshot. The context snapshot includes a snapshot timestamp, a source identifier, a set of state key-value items, and a quality identifier.

[0098] Optionally, the status data includes the physical status data, authorized status data, and equipment status data of the thermal power plant. The verification module 303 is configured to acquire the physical status data of the industrial scenario through the edge gateway, IoT sensor, equipment instrument, or camera module in the industrial scenario. The authorization status data of the industrial scenario is obtained through the electronic ticketing system, access control system, and personnel qualification management system in the industrial scenario. The equipment status data of the industrial scenario is obtained through the control and monitoring system or historical database in the industrial scenario.

[0099] Optionally, the verification module 303 is further configured to determine the confidence level of the event object and the initial context snapshot corresponding to the event object; If the confidence level of the event object is less than the second preset value or if the initial context snapshot has missing fields, a completion evidence collection strategy is generated. The completion evidence collection strategy is executed to confirm the new confidence level of the event object and the new context snapshot corresponding to the event object, until the new confidence level of the event object is greater than or equal to the second preset value and the new context snapshot does not have any missing fields, and the new context snapshot is used as the context snapshot corresponding to the event object.

[0100] Regarding the industrial scenario anti-violation active safety monitoring device in the above embodiments, the specific methods of each module's operation have been described in detail in the embodiments of the relevant method, and will not be elaborated here.

[0101] Based on the same inventive concept, this disclosure also provides an electronic device, comprising: A memory on which computer programs are stored; A processor is used to execute the computer program in the memory to implement the above-described proactive safety monitoring method for preventing violations in industrial scenarios.

[0102] Figure 4 This is a block diagram illustrating an electronic device 400 according to an exemplary embodiment. Figure 4As shown, the electronic device 400 may include a processor 401 and a memory 402. The electronic device 400 may also include one or more of a multimedia component 403, an input / output (I / O) interface 404, and a communication component 405.

[0103] The processor 401 controls the overall operation of the electronic device 400 to complete all or part of the steps in the aforementioned proactive safety monitoring method for preventing violations in industrial scenarios. The memory 402 stores various types of data to support the operation of the electronic device 400. This data may include, for example, instructions for any application or method operating on the electronic device 400, as well as application-related data, such as video streams and event signals from the industrial scenario, event detection results, audio and video slices and context snapshots corresponding to event objects, verification results, preset constraint rules, and evidence packages, etc. The memory 402 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The multimedia component 403 may include a screen and an audio component. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in the memory 402 or transmitted via the communication component 405. The audio component also includes at least one speaker for outputting audio signals. I / O interface 404 provides an interface between processor 401 and other interface modules, such as keyboards, mice, and buttons. These buttons can be virtual or physical. Communication component 405 is used for wired or wireless communication between the electronic device 400 and other devices. Wireless communication includes, for example, Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IoT, eMTC, or other 5G technologies, or combinations thereof, and is not limited herein. Therefore, the corresponding communication component 405 may include: a Wi-Fi module, a Bluetooth module, an NFC module, etc.

[0104] In an exemplary embodiment, the electronic device 400 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described proactive safety monitoring method for preventing violations in industrial scenarios.

[0105] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided. When executed by a processor, these program instructions implement the steps of the above-described proactive safety monitoring method for preventing violations in industrial scenarios. For example, the computer-readable storage medium may be the memory 402 including the program instructions, which may be executed by the processor 401 of the electronic device 400 to complete the above-described proactive safety monitoring method for preventing violations in industrial scenarios.

[0106] In another exemplary embodiment, a computer program product is also provided, which includes a computer program executable by a programmable device, the computer program having a code portion for performing the above-described proactive safety monitoring method for preventing violations in industrial scenarios when executed by the programmable device.

[0107] The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and these simple modifications all fall within the protection scope of this disclosure.

[0108] It should also be noted that the various specific technical features described in the above embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, this disclosure will not describe the various possible combinations separately.

[0109] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.

Claims

1. An industrial scene anti-violation active safety monitoring method, characterized in that, include: Acquire video streams and event signals from an industrial scene, and perform abnormal event detection based on the video streams and event signals to obtain event detection results; When the event detection result indicates that a candidate abnormal event has been detected, an event object is generated. An audio and video slice corresponding to the event object is extracted from the video stream. The event object includes an event identifier, a time window, a region identifier or camera identifier, an event type or action candidate, a confidence level, and a segment reference. Determine the context snapshot corresponding to the event object, convert the audio and video slices into semantic descriptions, generate a tool call sequence based on the event object, the semantic description, the context snapshot, and a preset tool registry, and perform consistency verification on the tool call sequence according to preset constraint rules to obtain the verification result; If the verification result indicates that the consistency verification has passed, an evidence package is generated, and the relevant equipment in the industrial scenario is controlled according to the risk level of the evidence package.

2. The industrial scene anti-violation proactive safety monitoring method according to claim 1, characterized in that, The preset tool registry includes tools and tool descriptors, and the method further includes: If the verification result indicates that the consistency verification has failed, the tool descriptor corresponding to the tool call sequence is backfilled into the context snapshot to obtain a new context snapshot; A new tool invocation sequence is generated based on the event object, the semantic description, the new context snapshot, and the preset tool registry. The new tool call sequence is subjected to consistency verification according to preset constraint rules until the new tool call sequence passes the consistency verification.

3. The method for proactive safety monitoring in industrial scenarios to prevent violations, as described in claim 1, is characterized in that... The evidence package includes a target context snapshot, video keyframes, hit rule IDs, and timestamps. The step of controlling relevant equipment in the industrial scenario based on the risk level of the evidence package includes: If the risk level of the evidence package is low, a work order is generated based on the evidence package and the work order is sent to the central control room of the industrial scenario. If the risk level of the evidence package is high, and the confidence level of the evidence package is greater than a first preset value, a control command is sent to the corresponding device in the industrial scenario. If the confidence level of the evidence package is less than or equal to the first preset value, an alarm message is generated and broadcast. The control command includes at least one of the following: stop command, reverse control command, deceleration control command, blocking control command, lock control command, parameter adjustment command, robot scheduling command, and autonomous operation command.

4. The method for proactive safety monitoring in industrial scenarios to prevent violations, as described in claim 1, is characterized in that... Determining the context snapshot corresponding to the event object includes: Obtain state data from different sources corresponding to the event object, and map the state data from different sources to a unified state key-value item; Align the multiple state key-value items to the time window in the event object, and determine the time deviation corresponding to each state key-value item; If a conflict is determined among the multiple state key-value items based on the time deviations corresponding to the multiple state key-value items, the multiple state key-value items are fused according to the source priority and time freshness of the state data corresponding to the state key-value items to generate a context snapshot. The context snapshot includes a snapshot timestamp, a source identifier, a set of state key-value items, and a quality identifier.

5. The method for proactive safety monitoring in industrial scenarios to prevent violations, as described in claim 4, is characterized in that... The status data includes physical status data, authorization status data, and equipment status data of the industrial scenario. Obtaining status data from different sources corresponding to the event object includes: The physical state data of the industrial scenario is obtained through edge gateways, IoT sensors, equipment, instruments, or camera modules in the industrial scenario. The authorization status data of the industrial scenario is obtained through the electronic ticketing system, access control system, and personnel qualification management system in the industrial scenario. The equipment status data of the industrial scenario is obtained through the control and monitoring system or historical database in the industrial scenario.

6. The method for proactive safety monitoring in industrial scenarios to prevent violations, as described in claim 1, is characterized in that... Determining the context snapshot corresponding to the event object includes: Determine the confidence level of the event object and the initial context snapshot corresponding to the event object; If the confidence level of the event object is less than the second preset value or if the initial context snapshot has missing fields, a completion evidence collection strategy is generated. The completion evidence collection strategy is executed to confirm the new confidence level of the event object and the new context snapshot corresponding to the event object, until the new confidence level of the event object is greater than or equal to the second preset value and the new context snapshot does not have any missing fields, and the new context snapshot is used as the context snapshot corresponding to the event object.

7. An active safety monitoring device for preventing violations in industrial settings, characterized in that, include: The acquisition module is configured to acquire video streams and event signals from an industrial scene, and to perform abnormal event detection based on the video streams and event signals to obtain event detection results. The execution module is configured to generate an event object when the event detection result indicates that a candidate abnormal event has been detected, and to extract an audio and video slice corresponding to the event object from the video stream. The event object includes an event identifier, a time window, a region identifier or camera identifier, an event type or action candidate, a confidence level, and a segment reference. The verification module is configured to determine the context snapshot corresponding to the event object, convert the audio and video slices into semantic descriptions, generate a tool call sequence based on the event object, the semantic description, the context snapshot, and a preset tool registry, and perform consistency verification on the tool call sequence according to preset constraint rules to obtain the verification result. The control module is configured to generate an evidence package if the verification result indicates that the consistency verification has passed, and to control the relevant equipment in the industrial scenario according to the risk level of the evidence package.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1-6.

9. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the method of any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1-6.

Images