Safety and health closed-loop management method and system based on hybrid state space and managed autonomy

Abstract

The application discloses a safety and health closed-loop management method and system based on a hybrid state space and managed autonomy. The method comprises the following steps: acquiring heterogeneous time sequence data such as video streams, inertial physiological signals and environmental indicators in a perception layer, and performing LZ4 lossless compression and disaster recovery storage on the edge side; constructing a theme data pool, capturing the semantic association of violation behaviors by adopting a hybrid Mamba cross-attention architecture with an interleaving embedding fusion strategy; configuring a managed autonomy state machine, migrating among the four states of stability, metacognition, collaboration and control according to the reasoning consistency entropy value, and performing task scheduling by adopting a batch processing query optimization strategy; calling a homography transformation algorithm to realize image space geometric alignment verification, and constructing a HybridRAG index based on a domain ontology to perform model parameter updating. The application solves the problems of industrial field perception distortion, low multi-modal research and judgment precision and agent decision illusion, and realizes real-time closed-loop and knowledge self-evolution of safety and health management.

Description

Technical Field

[0001] This invention relates to the fields of industrial safety production management, artificial intelligence, and Internet of Things communication technology. Specifically, it relates to a closed-loop management method and system for Environment, Health, and Safety (EHS) based on a hybrid state space and regulated autonomy. Background Technology

[0002] Traditional health, safety, and environmental (SHE) management models primarily rely on manual inspections, paper-based records, and post-incident remedial measures. While some companies have introduced IoT monitoring systems, they still face the following significant technical bottlenecks in practical applications: First, the perception dimensions are limited and the data is distorted. Existing security monitoring often uses scattered sensor deployments, which cannot achieve comprehensive access to all elements such as personnel behavior, physiological indicators, equipment status, and environmental physical parameters. At the same time, due to the harsh industrial environment, network fluctuations often lead to the loss or time-series gaps of high-frequency sampling data (such as IMU inertial signals and high frame rate video), and there is a lack of effective local governance and disaster recovery mechanisms at the edge.

[0003] Second, the cross-modal collaborative judgment capability is insufficient. Traditional systems mostly rely on threshold exceeding alarms from a single sensor (such as gas concentration exceeding limits), which are highly susceptible to environmental noise interference, resulting in extremely high false alarm and false negative rates. Although multimodal architectures such as Transformer have made breakthroughs in visual understanding, their quadratic complexity O(L2) leads to huge memory overhead when processing long-term industrial physical signals, and it is difficult to solve the depth ambiguity problem when extracting 3D human poses from 2D videos.

[0004] Third, there is the risk of uncontrollable autonomy for intelligent agents. Existing AI agent architectures are mostly based on the assumption of "boundaryless autonomy," meaning the system often maintains "false confidence" in its output even when perceptual signals are ambiguous or internal knowledge is lacking, leading to decision-making illusions. In the high-risk field of safe production, there is a lack of a managed autonomy architecture capable of dynamically transitioning between autonomous decision-making, expert collaboration, and controlled shutdown based on reasoning confidence assessment.

[0005] Fourth, fragmented business processes and outdated knowledge. Monitoring, early warning, rectification, review, and emergency response modules operate independently, failing to form a closed loop of automation technology. Traditional AI models, once deployed, often become static black boxes, unable to perform incremental evolution based on the latest safety procedures, laws and regulations, and on-site feedback, resulting in insufficient continuous adaptability of the system to industry scenarios.

[0006] To address the aforementioned issues, there is an urgent need for a safety, health, and environmental management solution that integrates cloud-edge collaborative sensing, hybrid state-space reasoning, managed autonomy control, and knowledge self-evolution and iteration, in order to achieve proactive and intelligent monitoring of the entire lifecycle of industrial sites. Summary of the Invention

[0007] To address the aforementioned technical problems, this invention proposes a closed-loop management method for health, safety, and environmental (SHE) based on a hybrid state space and regulated autonomy, comprising: S1: Configure a cloud-edge collaborative perception layer to acquire heterogeneous time-series data containing video streams, inertial physiological signals, and physical indicators, and perform lossless compression and disaster recovery storage based on a local ring database at the edge; so as to output a holographic feature package that is doubly aligned in physical time step and three-dimensional spatial coordinate system when the network link is restored. S2: The cloud-based brain receives the holographic feature package, extracts the employee's unique identification code and business action code, and executes an asymmetric XOR hash partitioning algorithm to construct a theme data pool with consistent order. The theme data pool is input into the safety, health, and environmental protection-specific large model, which captures the cross-modal semantic association between visual scenes and physical time sequence through an interleaved embedding fusion strategy and a hybrid Mamba cross-attention architecture, and outputs the violation behavior judgment decision result carrying the inference consistency entropy value. S3: Configure the managed autonomous state machine and the agent scheduling layer to receive the violation behavior judgment decision results and their reasoning consistency entropy values. Based on the reasoning consistency entropy values, perform transitions between four autonomous states: stable, metacognitive, collaborative, and controlled to determine the security decision boundary. Based on the transitioned permission state, for the violation behavior pointed to by the decision results, use a batch processing query optimization strategy to perform topology orchestration and computing power scheduling on the directed acyclic graph of the rectification work order tasks, and dispatch precise rectification work order instructions to the site. S4: For the work order after rectification according to the precise rectification work order instruction, call the homography transformation algorithm to realize the spatial geometric alignment verification of the images before and after rectification; after the verification is passed, the rectification process is used as a compliance knowledge sample, a hybrid RAG index is constructed based on domain ontology, and the parameters of the safety, health and environment-specific large model are fine-tuned and updated in combination with the quantized low-rank adaptation scheme to realize the incremental self-evolution closed loop of knowledge to the specific large model.

[0008] Specifically, step S2 includes: capturing cross-modal semantic association between visual scenes and physical time sequence through an interleaving embedding fusion strategy, wherein: without introducing additional trainable parameters, a slot-based interleaving splicing method is used to interleave and splice a two-dimensional visual token sequence representing the visual characteristics of the work site environment with a physical time sequence vector representing the physiological abnormalities of the worker's limb tilt angle and heart rate at the same physical time step index; the slot-based interleaving splicing method includes: using a self-attention mechanism to calculate the fine-grained interactive association between the spatial distribution of hazards in the two-dimensional visual token and the physiological movement changes in the physical time sequence vector, which is used to identify the unstable fall or abnormal state of personnel working in confined spaces or at heights in environments with strong light, dense smoke, or blind spots.

[0009] Specifically, in step S2, the cross-modal semantic association between visual scenes and physical time sequence is captured through a hybrid Mamba cross-attention architecture. This architecture includes two core components: a temporal feature extraction component and a modal alignment and fusion component, which are applied to the assessment of concealed violations under occluded conditions. The temporal feature extraction component uses a selective state-space model to perform linear complexity modeling on high-frequency inertial measurement signals representing continuous limb movement of workers to obtain long-term limb state features in the latent space. The modal alignment and fusion component utilizes a multi-head cross-modal cross-attention mechanism to use the occluded two-dimensional video features as a query matrix and perform cross-modal association with the long-term limb state features in the latent space. It uses inertial features unaffected by visual occlusion to complete the missing depth information in the two-dimensional video image, eliminating ambiguity in action judgment caused by perspective occlusion, thereby accurately determining concealed limb violations caused by non-compliance with procedures behind blind spots or physical barriers in industrial sites.

[0010] Specifically, the state transition triggering mechanism in step S3 includes: when the internal reasoning consistency entropy value exceeds a preset threshold, forcing a transition from a stable state to a metacognitive state to perform self-diagnosis and suppressing the output of external decision signals.

[0011] Specifically, the controlled state includes: when the uncertainty of the disaster risk exceeds the safety envelope, the system actively withdraws the intelligent agent's execution authority and transfers the operation authority to external manual supervision or executes a preset controlled shutdown sequence.

[0012] Specifically, the batch query optimization strategy includes: identifying repeated prefixes and shared subgraphs in concurrent work order tasks, reducing memory overhead through a key-value pair cache reuse mechanism, and executing non-large model operators in batches through pipeline pre-filling.

[0013] Specifically, step S3 further includes task splitting and merging of a dynamic directed acyclic graph: based on the real-time distribution of disaster risk areas, the static contingency plan task is instantly decomposed into parallel sub-task branches, and a synchronization obstacle node is generated after the branches are executed.

[0014] Specifically, in step S4, the hybrid RAG index includes: constructing a knowledge graph covering the hierarchical relationship of security compliance procedures, and combining it with vector cosine similarity retrieval, and generating rectification suggestions with legal authority through a reordering algorithm.

[0015] Specifically, the updated model parameters include: keeping the large model basis weights frozen, and using a quantized low-rank adaptation scheme to perform incremental parameter fine-tuning only on the bypass low-rank matrix corresponding to the multimodal projection layer and the self-attention matrix.

[0016] The present invention also discloses a cloud-edge collaborative sensing module, which is configured with a cloud-edge collaborative sensing layer for acquiring heterogeneous time-series data containing video streams, inertial physiological signals and physical indicators, and performing lossless compression and disaster recovery storage based on a local ring database at the edge side, so as to output a holographic feature package that is doubly aligned in physical time step and three-dimensional spatial coordinate system when the network link is restored. The big data governance and analysis module is configured to receive the holographic feature package from the cloud brain, extract the unique employee identification code and business action code, and execute an asymmetric XOR hash partitioning algorithm to construct a theme data pool with consistent order. The theme data pool is input into the safety, health and environmental protection-specific big model, which captures the cross-modal semantic association between visual scenes and physical time sequence through an interleaved embedding fusion strategy and a hybrid Mamba cross-attention architecture, and outputs the violation behavior analysis and decision results carrying the inference consistency entropy value. The autonomous control and scheduling module is configured to receive the violation judgment decision results and their reasoning consistency entropy values ​​from the managed autonomous state machine and the intelligent agent scheduling layer. Based on the reasoning consistency entropy values, it performs migration between four autonomous states: stable, metacognitive, collaborative, and controlled, to determine the security decision boundary. Based on the permission state after migration, for the violation behavior pointed to by the decision results, it adopts a batch processing query optimization strategy to perform topology orchestration and computing power scheduling on the directed acyclic graph of the rectification work order tasks, and dispatches precise rectification work order instructions to the site. The closed-loop review and knowledge evolution module is configured to perform spatial geometric alignment verification of images before and after rectification based on the precise rectification work order instruction. After the verification is passed, the rectification process is used as a compliance knowledge sample. A hybrid RAG index is constructed based on domain ontology, and the parameters of the safety, health, and environmental protection-specific large model are fine-tuned and updated in combination with a quantized low-rank adaptation scheme. This realizes the incremental self-evolution of knowledge into the specific large model, and the automated AI review and knowledge self-evolution are executed in a closed loop. The homography transformation algorithm is used to perform image spatial geometric alignment verification, and a hybrid RAG index is constructed based on domain ontology. The model parameters are updated in combination with a bypass low-rank adaptation scheme.

[0017] The method of this invention achieves the following technical effects: 1. Deep coupling of heterogeneous features and improved recognition accuracy Eliminating the modal semantic gap: Through an interleaved embedding fusion (IEF) strategy, this invention forces a large model's self-attention mechanism at the index level to capture fine-grained correlations between "visual scenes" and "physical indicators" without introducing additional trainable parameters. Compared to traditional post-processing fusion schemes, this low-level fusion can effectively identify long-tail hazards (e.g., when video is obscured by strong light or smoke and the movement of people cannot be determined, IMU inertial signals serve as supplementary information, improving the accuracy of limb violation detection by approximately 30%).

[0018] Real-time processing capability for long sequences: The HybridMamba architecture is introduced, leveraging its ability to model time-series signals with linear complexity O(L), solving the problem of exploding memory overhead when traditional Transformers process long sequences of industrial high-frequency sampling (100Hz+). This ensures that the system can process the continuous operation characteristics of devices for minutes or even hours in milliseconds while maintaining high frame rate video analysis.

[0019] 2. Active inhibition of cognitive safety and decision-making illusion Quantitative epistemological confidence: By monitoring the state of the large model in real time through the reasoning consistency entropy value H(σ), the system can identify the "confusion level" of its internal reasoning. When the entropy value exceeds the limit and triggers "epistemological drift", the system is forced to migrate from a stable state to a metacognitive self-diagnostic state.

[0020] Deterministic Authority Management: Unlike the "boundaryless autonomy" of traditional AI agents, the Managed Autonomy State Machine (SMARt) establishes clearly defined execution boundaries. State Boundaries (SMARt): Clearly includes four autonomous states: Steady state (S): Automatic execution under high confidence.

[0021] Metacognitive state (M): When the inference entropy value is too high, external dispatch is suspended and internal self-diagnosis is performed.

[0022] Collaborative State (A): When self-diagnosis cannot eliminate ambiguity, external collaboration (such as HybridRAG) is introduced.

[0023] Controlled state (Rt): When the risk exceeds the safety envelope, the authority is withdrawn and manual takeover or preset lockout is applied.

[0024] This mechanism reduces the false alarm rate by more than 50% and ensures that in extremely high-risk conditions (such as when flash explosion signs are detected), the system can forcibly relinquish autonomy to a controlled state based on non-Zeno progressive logic, thus guaranteeing the absolute authority of human operation and the ultimate safety of the system.

[0025] 3. High-concurrency transaction processing and computing resource optimization Exponential throughput acceleration: The agent scheduling layer uses the Halo batch query optimization strategy to identify repeated prefixes and shared subgraphs in concurrent work order tasks, thereby accelerating end-to-end inference.

[0026] Eliminating idle computing resources: The system effectively masks the latency of I / O-intensive tasks (such as querying APIs and retrieving databases) through pipeline pre-filling and work-stealing mechanisms, significantly reducing the cumulative GPU-seconds.

[0027] Flexible emergency dynamic orchestration: The dynamic DAG task splitting and merging mechanism enables emergency response to be adjusted instantaneously according to real-time risk distribution (such as the number of personnel in different workshops locked by UWB), ensuring the parallelism and logical rigor of the command chain in the event of a plant-wide emergency.

[0028] 4. Effectiveness of closed-loop review and the ability of knowledge to evolve independently Eradicating "paper-based rectification": By utilizing a homography transformation algorithm, the system eliminates perspective distortion in photos transmitted from non-fixed camera positions, enabling pixel-level feature comparison between alarm areas and rectification areas. Combined with multi-source sensor numerical coupling verification, the confidence level of hazard rectification verification is increased by more than 25%.

[0029] Knowledge retrieval precision: The HybridRAG hybrid search engine, combined with domain ontology, significantly improves the precision of legal compliance reviews involving the hierarchical relationship of "parent law, subsidiary law, and implementing rules" compared to pure vector retrieval, greatly suppressing knowledge gaps or logical deviations in the generated rectification suggestions.

[0030] Low-cost model hot update: Based on the Quantized Low-Rank Adaptive Scheme (QLoRA), the parameter incremental fine-tuning scheme enables the system to quickly absorb the latest incident reports and compliance standards on a weekly basis. Since only a very small number of low-rank bypass matrices need to be trained, the system achieves "learning on the go" without interrupting operation, giving the safety, health, and environmental management capabilities a continuously evolving vitality.

[0031] 5. Resilience and Disaster Recovery Reliability of the Sensing Layer Data preservation in extreme environments: The edge side employs LZ4 lossless block compression coupled with a local ring database, reducing data transmission volume and establishing a millisecond-level triggered breakpoint resumption mechanism. This ensures the temporal integrity of the perception token sequence received by the cloud-based system under the normal conditions of high-frequency fluctuations in industrial field networks. Attached Figure Description

[0032] Figure 1 This is a flowchart of the closed-loop management method for health, safety, and environmental health based on hybrid state space and managed autonomy proposed in this invention. Figure 2 This is a block diagram of the safety, health, and environmental closed-loop management method based on hybrid state space and managed autonomy proposed in this invention. Detailed Implementation

[0033] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0034] This invention proposes a closed-loop management method for health, safety, and environmental (SHE) systems based on a hybrid state space and managed autonomy, such as... Figure 1 As shown, it includes: S1: Configure a cloud-edge collaborative perception layer to acquire heterogeneous time-series data containing video streams, inertial physiological signals, and physical indicators, and perform lossless compression and disaster recovery storage based on a local ring database at the edge; so as to output a holographic feature package that is doubly aligned in physical time step and three-dimensional spatial coordinate system when the network link is restored.

[0035] In a preferred embodiment of the present invention, the sampling frequency of the visual acquisition terminal (AI camera) is typically fixed at [frequency value missing]. to In order to capture highly dynamic human limb features, the IMU (Inertial Measurement Unit) sensors integrated into wearable devices typically have sampling frequencies as high as [missing information - likely referring to a specific frequency range]. to This asymmetry in sampling frequency can lead to "motion capture lag" or "feature alignment misalignment" in subsequent large models during fusion inference.

[0036] Therefore, the system executes the following specific alignment algorithm process on the edge gateway or sensing terminal side: S1.1. Based on global clock timestamp normalization, the system establishes a unified NTP (Network Time Protocol) synchronization mechanism to ensure that all sensing terminals share a millisecond-level synchronized global reference clock. Each frame of video data and each group of IMU signals is timestamped with high precision at the moment of generation. and .

[0037] S1.2. Linear interpolation and alignment of scalar physical signals: For scalar signals such as heart rate, blood oxygen, and acceleration in the IMU, the system employs a linear resampling algorithm based on time deviation compensation. When the system needs to obtain the corresponding video frame time... When determining the physical characteristic values, first lock the values ​​that satisfy the conditions in the IMU timing buffer. Two adjacent sampling points. Aligned feature values. Calculated using the following formula: Variable definition and parameter meaning: Align to video moment The target physical characteristic value; , :time Features of the two closest original sampling points; Defined as the normalized time step factor, it is used to quantify the relative position of the current video frame within the inertial signal period.

[0038] S1.3. Spherical Linear Interpolation (SLERP) for Limb Rotational Attitudes Since the human limb posture output by the IMU is usually represented in quaternion form, which follows non-Euclidean geometry, traditional linear interpolation can lead to the non-orthogonality of the rotation matrix. The system uses the SLERP algorithm for attitude alignment to ensure the shortest rotation path and constant angular velocity: in Representing the angle between two rotational states, the algorithm ensures that the posture characteristics of a person's movements are mathematically continuous and smooth within the extremely short time of video frame switching.

[0039] S1.4. Visual-Inertial Spatial Geometric Unification (Coordinate System Alignment) After completing time-step alignment, the system also needs to perform spatial dimension calibration. The system utilizes the T-pose normalization calibration process, rotating the skeleton in the IMU coordinate system using the following rotation chain formula. Unified mapping to camera coordinate system : Parameter definition: The relative offset rotation between the IMU sensor and the corresponding human skeleton; The transformation matrix from the IMU local coordinate system to the field global coordinate system; Camera extrinsic matrix.

[0040] Technical Results: Through the combined application of the above algorithms, the system reconstructs discrete and disordered "bitstreams" into "holographic feature packets" within the same physical time step and spatial reference frame. This not only solves the depth ambiguity problem (i.e., the difficulty in determining the true bending angle of a limb based solely on video), but also reduces the recognition response latency. The above provides high-confidence data input for the "interleaved embedding fusion" of the large model in the subsequent S2 step.

[0041] S2: The cloud-based brain receives the holographic feature package, extracts the employee's unique identification code and business action code, and executes an asymmetric XOR hash partitioning algorithm to construct a theme data pool with consistent order. The theme data pool is input into the safety, health, and environmental protection-specific large model. Through an interleaving embedding fusion strategy and a hybrid Mamba cross-attention architecture, the cross-modal semantic association between visual scenes and physical time sequence is captured, and the violation behavior judgment decision result carrying the inference consistency entropy value is output. The cross-modal semantic association between visual scenes and physical time sequence is captured through the interleaving embedding fusion strategy, wherein: without introducing additional trainable parameters, a slot-style interleaving splicing is used to interleave and splice the two-dimensional visual token sequence representing the visual characteristics of the work site environment with the physical time sequence vector representing the physiological abnormalities of the worker's limb tilt angle and heart rate at the same physical time step index. The slot-type interleaving and splicing includes: using a self-attention mechanism to calculate the fine-grained interactive correlation between the spatial distribution of hazards in the two-dimensional visual token and the physiological action changes in the physical time-series vector, in order to identify unstable falls or abnormal states of personnel in confined spaces or working at heights in environments with strong light, dense smoke, or blind spots. The method employs a hybrid Mamba cross-attention architecture to capture cross-modal semantic associations between visual scenes and physical time sequences. This architecture comprises two core components: a temporal feature extraction component and a modal alignment and fusion component, applied to the assessment of concealed violations under occluded conditions. The temporal feature extraction component utilizes a selective state-space model to perform linear complexity modeling on high-frequency inertial measurement signals representing continuous limb movement of workers, thereby obtaining long-term temporal limb state features in the latent space. The modal alignment and fusion component utilizes a multi-head cross-modal cross-attention mechanism, using occluded 2D video features as a query matrix to perform cross-modal association with the long-term temporal limb state features in the latent space. It uses inertial features unaffected by visual occlusion to supplement the missing depth information in the 2D video image, eliminating ambiguity in action judgment caused by perspective occlusion, thus accurately determining concealed limb violations not performed according to procedures behind blind spots or physical barriers in industrial sites.

[0042] Edge side: The storage adaptive optimization edge computing layer based on LZ4 compression ratio calculation serves as the system's data outpost. By monitoring the compression efficiency of the data payload in real time, it achieves efficient management of the limited local storage space (ring database).

[0043] S2.1 Real-time LZ4 Compression Ratio Calculation Algorithm: When heterogeneous time-series data streams enter the edge gateway, the system starts the LZ4 lossless block compression engine. For each data block to be stored... The system calculates its space-saving gain and compression ratio in real time. : Parameter definition: : Current data block The LZ4 compression ratio reflects the redundancy of the physical signal within that time period; Size of the original bitstream before compression (unit: bytes); : The size of the compressed load after processing by the LZ4 algorithm.

[0044] S2.2 Adaptive Ring Storage Strategy Based on Compression Ratio Feedback: The local ring database adopts a "first-in, first-out, cyclic overwrite" logic. The system introduces... Weighting factor, dynamically adjusts the retention priority of data segments: if ( The mean, (where the standard deviation is 0.5%) indicates that the on-site signal is highly stable and the information entropy is extremely low. In this case, the ring database will shorten the retention priority of this type of low-entropy data, prioritizing the freeing up of physical sectors for it. High-entropy data blocks close to 1 (representing drastic fluctuations in on-site physical indicators or containing complex behavioral characteristics) thus retain more "effective mutation features" without changing the total capacity.

[0045] S2.3 Backhaul Priority Scheduling Based on Converged Compression Ratio: After the uplink recovers from an interruption, the system uses the following formula to correct the retransmission priority score. : Beneficial effects: Highly compressed data packets occupy fewer physical bits during transmission but cover a longer time span. By increasing their priority, the system can quickly complete the "historical background knowledge graph" of the cloud-based large model within a very short link recovery time, avoiding misjudgments of causal chains due to gaps in historical data.

[0046] Cloud Brain: After the queue partitioning governance data based on asymmetric XOR hashing is aggregated to the cloud message queue, the system must ensure that the judgment of violations for the same operating entity has atomicity and time-order rigor.

[0047] The system extracts the employee's unique identification code from the payload. With business action codes Using partition mapping functions Routing the data stream to Exclusive subqueues: Technical principle: The XOR operation is used to achieve semantic entanglement of features before hash mapping. Since the distribution patterns of identity codes and action codes are different, the XOR operation can significantly reduce hash collisions (DataSkew) caused by overload of single-dimensional data, distribute the load evenly to each processing unit, and ensure that the action sequence of the same responsible party (such as "illegally crossing" followed by "damaging the fence") is logically strictly sequential according to physical time steps, eliminating the illusion of reasoning caused by multi-threaded competition.

[0048] Feature Layer: The Interleaved Embedding Fusion (IEF) strategy is implemented at the input layer of the large model. The system needs to eliminate the sampling asymmetry between the 2D video token and the 1D physical temporal signal. The system adopts the Interleaved Embedding Fusion (IEF) strategy to interleave features in slots at the index level.

[0049] If the visual sequence is The physical time series is The fused holographic sequence satisfy: Technical Effects: This operation does not introduce additional trainable parameters. By changing the position index, it forces the self-attention mechanism to establish a fine-grained association between "visual scene" and "physical indicator" at the underlying level (e.g., making "flame visual features" and "temperature surge token" directly adjacent in the latent space), significantly improving the sensitivity of long-tail hazard identification. In this embodiment, the interleaved embedding fusion strategy is applied to the judgment of specific behavioral violations by high-risk workers. Specifically, on the same physical time step index, the spatial distribution of hazard sources (such as holes and edges) in the two-dimensional visual token and the fine-grained interactive association between physiological action changes (such as sudden acceleration changes and abnormal heart rate) in the physical time sequence vector are calculated. This enables accurate identification of unstable falls or abnormal violations by personnel in confined spaces or at heights in environments with strong light, dense smoke, or blind spots.

[0050] Model Layer: The Hybrid Mamba-Cross-Attention Architecture (HMAO) judgment system calls upon a dedicated large-scale model for health, safety, and environmental monitoring. The HMAO architecture performs a full-domain scan using the Hybrid Mamba-Cross-Attention architecture. It is defined as a deep neural network that heterogeneously cascades a Selective State Space Model (SSM) with a Multi-Head Cross-Attention (MHCA) operator. Its technical boundaries and operational logic are defined by the following two core components: 1. Temporal Feature Extraction Component (Mamba Branch) This component utilizes a state-space model with a selective scanning mechanism (i.e., the Mamba operator) to perform feature compression on high-frequency inertial measurement unit (IMU) signals and physical time-series data. Its core lies in capturing long-range dependencies in the latent space through a computational complexity O(L) that increases linearly with the sequence length L. Its state transition equation is defined as: .

[0051] in, , The discretized state transition matrix is ​​used to extract the deep temporal representation of the high-frequency physical indicators above 100Hz obtained in step S1.

[0052] 2. Modal alignment and fusion component (cross-attention branch) This component achieves semantic space alignment between video features and physical features through a cross-attention mechanism. The system maps visual token sequences to a query matrix. The physical feature sequence output by the Mamba branch is mapped to a key matrix. Sum matrix The following multi-head cross-modal cross-attention (MHCA) formula is used for joint evaluation: Variable meaning: For video token query matrix; The key / value matrix for mapping physical features; It is a spatiotemporal mask matrix. This is a scaling factor used to maintain the stability of the training gradient.

[0053] This architecture eliminates depth ambiguity when extracting 3D human poses from 2D videos. When visual tokens are missing due to object occlusion or strong light interference, IMU inertial signals are used as supplementary information through cross-attention weights, enabling the model to accurately identify subtle violations of human body movements in hazardous areas. Specifically, this hybrid Mamba cross-attention architecture is applied to the assessment of concealed violations under occlusion conditions. When equipment or barriers in an industrial setting obstruct the line of sight, causing specific visual tokens to be missing in the 2D video feature matrix, cross-attention weights are used to supplement the missing depth information in the 2D video image using inertial measurement signals (IMU high-frequency skeletal motion features) that are not affected by visual occlusion. This eliminates the ambiguity in action judgment caused by perspective occlusion, thereby accurately identifying concealed violations of body movements not in accordance with procedures behind blind spots of equipment or physical barriers in industrial settings. The selective state-space model (Mamba) is used to process high-frequency physical signals, and its computational complexity increases with sequence length. It grows linearly. This effectively solves the problem of traditional Transformer architecture in processing long-term industrial data. The problem of exploding computing power overhead.

[0054] Cross-modal spatial complementarity: Perform multi-head cross-modal cross-attention (MHCA) computation: S3: The managed autonomous state machine and the agent scheduling layer receive the violation judgment decision results and their reasoning consistency entropy values. Based on the reasoning consistency entropy values, they perform transitions between four autonomous states—stable, metacognitive, collaborative, and controlled—to determine the safety decision boundary. According to the transitioned permission state, for the violation pointed to by the decision results, a batch processing query optimization strategy is used to perform topology orchestration and computing power scheduling on the directed acyclic graph of rectification work order tasks, and to dispatch precise rectification work order instructions to the site. This includes dynamic directed acyclic graph task splitting and merging; based on the real-time distribution of disaster risk areas, static contingency plan tasks are instantaneously decomposed into parallel sub-task branches, and a synchronization obstacle node is generated after the branch execution is complete. The state transition triggering mechanism includes: when the internal reasoning consistency entropy value exceeds a preset threshold, a forced transition from the stable state to the metacognitive state is initiated to perform self-diagnosis, and external decision signal output is suppressed.

[0055] The controlled state includes: when the uncertainty of disaster risk exceeds the safety envelope, the system actively withdraws the agent's execution authority and transfers the operation authority to external manual supervision or executes a preset controlled shutdown sequence. The batch processing query optimization strategy includes: identifying repeated prefixes and shared subgraphs in concurrent work order tasks, reducing memory overhead through a key-value pair cache reuse mechanism, and pre-filling pipelines to execute non-large model operators in batches.

[0056] This step is the core of the system's achievement of "cognitive security" and "high computing efficiency." It addresses the uncontrollability of AI decision-making through a managed autonomy architecture, while maximizing system throughput by utilizing batch query optimization strategies.

[0057] In a preferred embodiment of the present invention, the system constructs a two-layer management architecture at the cloud brain layer: the bottom layer is a parallel scheduling engine based on DAG, and the top layer is a managed autonomy logic state machine.

[0058] S3.1. SMARt State Transition Logic Based on Inference Consistency Entropy: The managed autonomous state machine and the agent scheduling layer receive the violation behavior judgment decision result and its inference consistency entropy value output from step S2. To prevent the safety, health, and environmental agent from experiencing "decision illusions" when faced with highly uncertain environments or ambiguous perceived signals, the system is equipped with a SMARt four-level state machine. Based on this entropy value, the state machine triggers permission transitions between four autonomous states—stable, metacognitive, cooperative, and controlled—to determine safety decision boundaries. Before executing a task, the agent needs to calculate the aforementioned inference consistency entropy value. Assess execution permissions: Parameter meanings and technical principles: The large model outputs the predicted entropy value of the token sequence based on the current safety, health, and environmental situation, which is used to quantify the uncertainty in the reasoning process; Under the current thought chain reasoning path, the first... The predicted probability distribution of each candidate decision branch; state transition discrimination: the system presets a hysteresis trigger threshold. .like This triggers an "epistemological drift," forcing the state machine to transition from a stable state to a metacognitive state and suspending external work order dispatch, while initiating internal self-diagnostic logic. Technical effect: This mechanism elevates safety, health, and environmental (SHE) management from "probabilistic identification" to "deterministic access control," effectively suppressing false alarms due to video lighting interference or sensor noise, and ensuring that AI remains under control even in high-risk conditions.

[0059] S3.2. Specific Implementation of Batch Processing Query Optimization Strategy (Halo): After determining the execution permissions, the agent scheduling layer is responsible for mapping the task DAG to heterogeneous computing resources. The system adopts the Halo batch processing scheduling algorithm, which optimizes computing power by identifying repeated prefixes in concurrent tasks.

[0060] Workflow batch processing speedup formula: Variable definition and parameter meaning: Speedup compared to traditional single-task serial mode; Total time taken for all safety and health tasks to be executed independently; The bottleneck computing unit's processing latency is used to compress the shortest task completion time; Fixed scheduling overhead regularization term, used to penalize excessively fragmented task splitting; Balance factor: Adjusts the weights between pipeline fluidity and computing power utilization.

[0061] S3.3. Dependency Decoupling and Parallel Scheduling of Task DAG: The scheduling engine performs dependency decoupling transformation, extracting task nodes such as work order dispatch, API retrieval, and image verification from the underlying large model to form independent CPU operator sequences. Utilizing KV cache reuse technology, the system can automatically merge safety and health ring instruction streams with the same prefix logic, reducing parallel processing overhead.

[0062] Technical Effects: By deploying this scheduling algorithm, this invention can achieve up to [percentage missing] when handling high-concurrency safety, health, and environmental monitoring tasks such as shift changes across the entire plant. It achieves a 100% end-to-end inference acceleration and can utilize pipelined bubbles to execute I / O-intensive tasks, completely resolving the "white screen response" problem under high concurrency in traditional systems. Therefore, after determining the safety decision boundaries, it can issue precise rectification work orders to the site for violations indicated by the decision results.

[0063] S4: For the work orders after rectification according to the precise rectification work order instructions, the homography transformation algorithm is called to verify the spatial geometric alignment of the images before and after rectification. After the verification is passed, the rectification process is used as a compliance knowledge sample. A hybrid RAG index is constructed based on domain ontology, and the parameters of the safety, health, and environment-specific large model are fine-tuned and updated using a quantized low-rank adaptation scheme to achieve an incremental self-evolutionary closed loop of knowledge to the specific large model. In step S4, the hybrid RAG index includes: constructing a knowledge graph covering the hierarchical relationship of safety compliance procedures, and combining vector cosine similarity retrieval to generate rectification suggestions with legal authority through a re-ranking algorithm.

[0064] S4.1. Image spatial geometric alignment verification based on homography transformation: The closed-loop verification and knowledge evolution module receives the rectification work order dispatched and executed in step S3, and receives the hazard rectification photos returned by the operators. Due to deviations in shooting position, angle, and focal length, the system first matches feature points through feature extraction operators, and then uses homography transformation to complete pixel-level spatial alignment.

[0065] Parameter meanings and variable definitions: Homogeneous coordinates of the center of the hazard source area in the initial alarm image; : The corresponding target coordinates in the mapped verification image; : The homography matrix is ​​solved iteratively using the RANSAC algorithm. Technical effect: This transformation eliminates perspective distortion caused by camera position, and AI can complete secondary feature comparison at precise locations. Combined with sensor data, it can comprehensively determine whether the hidden danger is closed-loop.

[0066] S4.2. The HybridRAG hybrid retrieval and knowledge evolution system based on domain ontology is launched. The HybridRAG hybrid retrieval architecture based on domain ontology is used to perform semantic-aware segmentation based on the safety, health and environmental ontology model to ensure the integrity of knowledge fragments.

[0067] Parameter meanings and variable definitions: Candidate text With the current situation The final relevance score; Cosine similarity in vector space represents surface semantic relationships; Knowledge graph multi-hop reasoning score; : Ontology consistency penalty item, which reduces the weight if the ontology rules are violated; Dynamic adjustment coefficient. Technical effect: Compared with pure vector retrieval, this hybrid retrieval scheme improves accuracy. The search results are used as prompts and injected into the large model to support review and judgment.

[0068] S4.3. The bypass low-rank adaptation model hot update system freezes the basis weights of the large model and only uses the Quantized Low-Rank Adaptation (QLoRA) scheme for incremental fine-tuning. The update formula is as follows: in , This is the trainable adapter matrix after low-rank decomposition.

[0069] Technical benefits: This hot update method significantly reduces the overhead of video memory and computing power. The system can periodically absorb new cases and new regulations, enabling continuous iteration of safety, health and environmental management capabilities.

[0070] This invention also discloses a closed-loop management system for health, safety, and environmental health based on a hybrid state space and managed autonomy, such as... Figure 2 As shown, it includes: a cloud-edge collaborative sensing module, which is configured with a cloud-edge collaborative sensing layer to acquire heterogeneous time-series data containing video streams, inertial physiological signals and physical indicators, and performs lossless compression and disaster recovery storage based on a local ring database on the edge side, so as to output a holographic feature package that is doubly aligned in physical time step and three-dimensional spatial coordinate system when the network link is restored. The big data governance and analysis module is configured to receive the holographic feature package from the cloud brain, extract the unique employee identification code and business action code, and execute an asymmetric XOR hash partitioning algorithm to construct a theme data pool with consistent order. The theme data pool is input into the safety, health and environmental protection-specific big model, which captures the cross-modal semantic association between visual scenes and physical time sequence through an interleaved embedding fusion strategy and a hybrid Mamba cross-attention architecture, and outputs the violation behavior analysis and decision results carrying the inference consistency entropy value. The autonomous control and scheduling module is configured to receive the violation judgment decision results and their reasoning consistency entropy values ​​from the managed autonomous state machine and the intelligent agent scheduling layer. Based on the reasoning consistency entropy values, it performs migration between four autonomous states: stable, metacognitive, collaborative, and controlled, to determine the security decision boundary. Based on the permission state after migration, for the violation behavior pointed to by the decision results, it adopts a batch processing query optimization strategy to perform topology orchestration and computing power scheduling on the directed acyclic graph of the rectification work order tasks, and dispatches precise rectification work order instructions to the site. The closed-loop review and knowledge evolution module is configured to perform spatial geometric alignment verification of images before and after rectification based on the precise rectification work order instruction. After the verification is passed, the rectification process is used as a compliance knowledge sample. A hybrid RAG index is constructed based on domain ontology, and the parameters of the safety, health, and environmental protection-specific large model are fine-tuned and updated in combination with a quantized low-rank adaptation scheme. This realizes the incremental self-evolution of knowledge into the specific large model, and the automated AI review and knowledge self-evolution are executed in a closed loop. The homography transformation algorithm is used to perform image spatial geometric alignment verification, and a hybrid RAG index is constructed based on domain ontology. The model parameters are updated in combination with a bypass low-rank adaptation scheme.

[0071] It should be understood that the processor in the embodiments of the present invention may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor described above can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0072] It is understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0073] It should be understood that the above-described memory is exemplary but not restrictive. For example, the memory in the embodiments of this application may also be static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM), etc. That is to say, the memory in the embodiments of this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0074] This application also provides a computer-readable storage medium for storing computer programs.

[0075] Optionally, the computer-readable storage medium can be applied to the terminal device in the embodiments of this application, and the computer program causes the computer to execute the corresponding processes implemented by the mobile terminal / terminal device in the various methods of the embodiments of this application. For the sake of brevity, it will not be described in detail here.

[0076] This application also provides a computer program product, including computer program instructions.

[0077] Optionally, the computer program product can be applied to the terminal device in the embodiments of this application, and the computer program instructions cause the computer to execute the corresponding processes implemented by the mobile terminal / terminal device in the various methods of the embodiments of this application. For the sake of brevity, they will not be described in detail here.

[0078] This application also provides a computer program.

[0079] Optionally, the computer program can be applied to the vehicle autonomous driving device in the embodiments of this application. When the computer program is run on a computer, it causes the computer to execute the corresponding processes implemented by the terminal device in the various methods of the embodiments of this application. For the sake of brevity, it will not be described in detail here.

[0080] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0081] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0082] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0083] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

Claims

1. A closed-loop management method for health, safety, and environmental health (SHE) based on a hybrid state space and managed autonomy, characterized in that, include: S1: Configure a cloud-edge collaborative perception layer to acquire heterogeneous time-series data containing video streams, inertial physiological signals, and physical indicators, and perform lossless compression and disaster recovery storage based on a local ring database on the edge side so as to output a holographic feature package that is doubly aligned in physical time step and three-dimensional spatial coordinate system when the network link is restored. S2: The cloud-based brain receives the holographic feature package, extracts the employee's unique identification code and business action code, and executes an asymmetric XOR hash partitioning algorithm to construct a theme data pool with consistent order. The theme data pool is input into the safety, health, and environmental protection-specific large model, which captures the cross-modal semantic association between visual scenes and physical time sequence through an interleaved embedding fusion strategy and a hybrid Mamba cross-attention architecture, and outputs the violation behavior judgment decision result carrying the inference consistency entropy value. S3: Configure the managed autonomous state machine and the agent scheduling layer to receive the violation behavior judgment decision results and their reasoning consistency entropy values. Based on the reasoning consistency entropy values, perform transitions between four autonomous states: stable, metacognitive, collaborative, and controlled to determine the security decision boundary. Based on the transitioned permission state, for the violation behavior pointed to by the decision results, use a batch processing query optimization strategy to perform topology orchestration and computing power scheduling on the directed acyclic graph of the rectification work order tasks, and dispatch precise rectification work order instructions to the site. S4: For the work order after rectification by the precise rectification work order instruction, call the homography transformation algorithm to realize the spatial geometric alignment verification of the images before and after rectification. After verification, the rectification process is used as a compliance knowledge sample. A hybrid RAG index is constructed based on domain ontology, and the parameters of the safety, health and environment-specific large model are fine-tuned and updated in combination with a quantitative low-rank adaptation scheme to realize the incremental self-evolution closed loop of knowledge to the specific large model.

2. The method according to claim 1, characterized in that, Step S2 specifically includes: The method of capturing cross-modal semantic association between visual scenes and physical time sequence through interleaving embedding fusion strategy includes: without introducing additional trainable parameters, using slot-based interleaving splicing to interleave and splice a two-dimensional visual token sequence representing the visual characteristics of the work site environment with a physical time sequence vector representing the physiological abnormalities of the worker's limb tilt angle and heart rate at the same physical time step index; the slot-based interleaving splicing includes: using a self-attention mechanism to calculate the fine-grained interactive association between the spatial distribution of hazards in the two-dimensional visual token and the physiological movement changes in the physical time sequence vector, which is used to identify the unstable fall or abnormal state of personnel working in confined spaces or at heights in environments with strong light, dense smoke or blind spots.

3. The method according to claim 1, characterized in that, In step S2, the cross-modal semantic association between visual scenes and physical time sequence is captured through a hybrid Mamba cross-attention architecture. This architecture includes two core components: a temporal feature extraction component and a modal alignment and fusion component, which are applied to the assessment of concealed violations under occluded conditions. The temporal feature extraction component uses a selective state-space model to perform linear complexity modeling on high-frequency inertial measurement signals representing continuous limb movement of workers to obtain long-term limb state features in the latent space. The modal alignment and fusion component utilizes a multi-head cross-modal cross-attention mechanism, using the occluded two-dimensional video features as a query matrix, and performs cross-modal association with the long-term limb state features in the latent space. It uses inertial features unaffected by visual occlusion to complete the missing depth information in the two-dimensional video image, eliminating ambiguity in action judgment caused by perspective occlusion, thereby accurately determining concealed limb violations caused by non-compliance with procedures behind blind spots or physical barriers in industrial sites.

4. The method according to claim 1, characterized in that, In step S3, the state transition triggering mechanism includes: when the internal reasoning consistency entropy value exceeds a preset threshold, forcing a transition from a stable state to a metacognitive state to perform self-diagnosis and suppressing the output of external decision signals.

5. The method according to claim 1, characterized in that, The controlled state includes: when the uncertainty of disaster risk exceeds the safety envelope, the system actively withdraws the agent's execution authority and transfers the operation authority to external manual supervision or executes a preset controlled shutdown sequence.

6. The method according to claim 1, characterized in that, The batch query optimization strategy includes: identifying repeated prefixes and shared subgraphs in concurrent work order tasks, reducing memory overhead through key-value pair cache reuse mechanism, and executing non-large model operators in batches through pipeline pre-filling.

7. The method according to claim 1, characterized in that, Step S3 further includes the task splitting and merging of the dynamic directed acyclic graph: based on the real-time distribution of disaster risk areas, the static contingency plan task is instantly decomposed into parallel sub-task branches, and a synchronization obstacle node is generated after the branch is completed.

8. The method according to claim 1, characterized in that, In step S4, the hybrid RAG index includes: constructing a knowledge graph covering the hierarchical relationship of security compliance procedures, and combining it with vector cosine similarity retrieval, and generating rectification suggestions with legal authority through a re-ranking algorithm.

9. The method according to claim 1, characterized in that, The updated model parameters include: keeping the large model basis weights frozen, and using a quantized low-rank adaptation scheme to perform incremental parameter fine-tuning only on the bypass low-rank matrix corresponding to the multimodal projection layer and the self-attention matrix.

10. A closed-loop management system for health, safety, and environmental health based on a hybrid state space and managed autonomy, characterized in that, include: The cloud-edge collaborative sensing module is configured with a cloud-edge collaborative sensing layer to acquire heterogeneous time-series data containing video streams, inertial physiological signals and physical indicators. It performs lossless compression and disaster recovery storage based on a local ring database at the edge side to output a holographic feature package that is doubly aligned in physical time step and three-dimensional spatial coordinate system when the network link is restored. The big data governance and analysis module is configured to receive the holographic feature package from the cloud brain, extract the unique employee identification code and business action code, and execute an asymmetric XOR hash partitioning algorithm to construct a theme data pool with consistent order. The theme data pool is input into the safety, health and environmental protection-specific big model, which captures the cross-modal semantic association between visual scenes and physical time sequence through an interleaved embedding fusion strategy and a hybrid Mamba cross-attention architecture, and outputs the violation behavior analysis and decision results carrying the inference consistency entropy value. The autonomous control and scheduling module is configured to receive the violation judgment decision results and their reasoning consistency entropy values ​​from the managed autonomous state machine and the intelligent agent scheduling layer. Based on the reasoning consistency entropy values, it performs migration between four autonomous states: stable, metacognitive, collaborative, and controlled, to determine the security decision boundary. Based on the permission state after migration, for the violation behavior pointed to by the decision results, it adopts a batch processing query optimization strategy to perform topology orchestration and computing power scheduling on the directed acyclic graph of the rectification work order tasks, and dispatches precise rectification work order instructions to the site. The closed-loop review and knowledge evolution module is configured to perform spatial geometric alignment verification of images before and after rectification by calling the homography transformation algorithm for the work order after rectification based on the precise rectification work order instruction. After verification, the rectification process is used as a compliance knowledge sample. A hybrid RAG index is constructed based on domain ontology. The parameters of the safety, health and environment-specific large model are fine-tuned and updated by combining a quantized low-rank adaptation scheme. This realizes the incremental self-evolution closed loop execution of knowledge to the specific large model, automated AI review and knowledge self-evolution. The homography transformation algorithm is called to realize the geometric alignment verification of image space. A hybrid RAG index is constructed based on domain ontology, and the model parameters are updated by combining a bypass low-rank adaptation scheme.

Images