Limited space operation safety control method and system

Abstract

The invention discloses a limited space operation safety control method and system, and relates to the technical field of operation safety control, and the method comprises the steps: recognizing the modal dominant change in a limited space operation process, and adjusting the interpretation priority and judgment reference weight of each piece of modal information according to the change of an operation stage; according to the modal dominant change of the current stage, judging whether the acquired multi-modal sensing signal is in an overall silent or abnormal silent state or not; establishing a time sequence causal chain between the behavior triggering event and the modal response to identify potential dangers; the cooperative stability among the multiple types of rhythm signals in the operation process is monitored; a response observation time window is set after the abnormal state occurs, and whether a safety intervention operation is triggered or not is judged by monitoring whether a multi-mode signal presents a self-recovery trend or not in the time window, and the safety intervention operation is performed. According to the invention, the problem that safety accidents cannot be timely and accurately identified and handled in the prior art is solved.

Description

Technical Field

[0001] This invention relates to the field of work safety control technology, and in particular to a method and system for work safety control in confined spaces. Background Technology

[0002] Confined space operations are prevalent in industrial sectors such as urban pipeline networks and energy storage and transportation. Due to their inherent characteristics such as enclosed spaces, poor ventilation, and restricted access, toxic and harmful gases can easily accumulate and oxygen-deficient environments can occur during operations. These operations are also accompanied by risks such as restricted personnel movement, poor communication, and complex equipment operation, making them historically high-risk scenarios for safety accidents. To improve the safety of confined space operations, various sensor network-based monitoring systems have emerged to collect and alarm on data such as the vital signs of workers, ambient gas concentrations, and equipment operating status in real time.

[0003] Currently, more advanced monitoring solutions employ multi-sensor fusion technology, such as the intelligent monitoring system for confined space operations based on multi-sensor fusion disclosed in patent document CN119380258A. Although significant technological advancements have been made in sensor fusion, real-time monitoring, and mobile device support, several key shortcomings and drawbacks remain, particularly in the lack of a complete closed loop regarding multimodal perception logic judgment mechanisms, abnormal state evolution trend modeling, self-recovery capability identification, and the intelligence of system intervention strategies. Specifically: First, the core idea of ​​this patent remains based on the traditional fusion paradigm of parallel acquisition and information display of multi-sensor data. While it achieves multi-dimensional perception of workers' vital signs, ambient gases, equipment operating status, and behavioral trajectories, the modeling of relationships between various data sources remains at the level of parallel combination rather than intermodal causal linkage modeling. This means the system can only respond to abnormal events through threshold triggering mechanisms, lacking the ability to understand and predict the shift of dominant relationships, trend evolution paths, and anomalous logical chains between modes. Consequently, when facing atypical risk modes or combined unstable situations, the system's response is slow, inaccurate, or even results in false alarms or missed alarms. Secondly, the patent does not address the dynamic identification and interpretation weight adjustment mechanism for modal dominance. In actual confined space operations, the dominance of modal signals will change over time (e.g., initially dominated by vision and environment, shifting to vital signs and actions in the middle stage, and voice signals becoming the key feedback in the later stage). Traditional monitoring systems lack this capability, leading to fixed modal priorities and making it easy to misjudge system stability or ignore key anomalies when certain modal signals weaken. Furthermore, although existing systems claim to have behavior detection capabilities, this function is mostly accomplished through image recognition and changes in motion trajectories. Their ability to judge the completeness of the behavior chain response is limited, and they cannot determine whether a certain behavior is effectively fed back and closed by other modalities in the system. For example, whether a person's sudden stagnation is caused by factors such as confusion, gas asphyxiation, equipment failure, or sudden changes in lighting, the system cannot verify and attribute the cause based on the behavior-modal feedback chain. This deficiency will severely limit the system's ability to identify potential hidden events. Furthermore, the patent does not mention any form of system self-recovery capability judgment mechanism, which means that once the warning or intervention logic is triggered, the system will intervene according to fixed rules, lacking the judgment support of whether the system itself has the ability to converge to a stable trend; for example, in the case of short-term disturbances but overall trend stabilization, traditional systems may still mistakenly trigger overreaction, interfere with the continuity of operations, and reduce system robustness.

[0004] Therefore, there is an urgent need to design a confined space operation safety management system with high judgment and response accuracy, and capable of accurately determining whether the system state is in the disturbance, self-recovery, critical or collapse range. Summary of the Invention

[0005] This invention provides a method and system for safety control in confined space operations, which addresses the problem that existing technologies, due to their shortcomings in multimodal information fusion, abnormal trend prediction, and intelligent intervention, cannot identify and handle safety accidents in a timely and accurate manner.

[0006] In view of this, the first aspect of the present invention provides a safety control method for confined space operations, the method comprising:

[0007] S1. Dynamically identify modal dominance changes during confined space operations based on the operation sequence, and adjust the interpretation priority and judgment reference weight of each modal information according to the changes in the operation stage;

[0008] S2. Based on the adjusted interpretation priority and judgment reference weight, determine whether the collected multimodal sensing signals are in a state of overall silence or abnormal silence according to the modal dominance change in the current stage. If the multimodal sensing signals are in a state of overall silence or abnormal silence, execute steps S3 and S4 respectively.

[0009] S3. Establish a temporal causal chain between the behavior triggering event and the modal response, wherein the temporal causal chain is verified by: continuously retaining multimodal state snapshots before and after the behavior triggering event within a set first time window, and detecting whether the behavior triggering event causes delayed feedback that conforms to the expected logic in other modalities based on the multimodal state snapshots;

[0010] S4. Monitor the coordination stability among multiple rhythmic signals during the operation. When rhythm breakage or imbalance is detected and no effective modal compensation is generated, it is determined to be an unstable critical state.

[0011] S5. In response to the failure of the behavior feedback verification through the time-series causal chain, or the determination of the unstable critical state, a second time window is set as a response observation window. By monitoring whether the multimodal sensing signal shows a self-recovery trend within the second time window, it is determined whether a safety intervention operation is triggered. When it is determined that the self-recovery condition is not met, response control matching the current modality dominance is executed.

[0012] Optionally, the step of dynamically identifying modal dominance changes during confined space operations based on the operation time sequence includes:

[0013] A time segment template for modality dominance transition is constructed based on historical operational experience data. The time segment template includes the dominant priority sequence and weight change curve of each modality under different operational stages.

[0014] The modal distribution characteristics of the current operation stage are matched with the time segment template. When the deviation exceeds the preset dominance transfer deviation threshold, a re-evaluation of modal dominance is triggered.

[0015] When multiple modalities compete for dominance, the difference in reaction time delay from event triggering to signal response of each modality helps to determine the modal interpretation priority at the current stage.

[0016] Optionally, the verification process of the temporal causal chain includes:

[0017] If no expected response features matching the behavior triggering event are detected based on the multimodal state snapshot within three consecutive modal update cycles, the behavior feedback is deemed to have failed.

[0018] When a modal response related to the behavior triggering event is detected, but the activation time order of each modal response is inconsistent with the preset standard response order, modal logic verification is performed. The modal logic verification is used to determine whether the current response order belongs to an atypical but safe behavior response chain based on the work context and behavior goal.

[0019] Optionally, the determination of rhythm breakage includes:

[0020] Time-frequency domain feature analysis is performed on the periodic rhythmic signal, and based on at least one of the periodic stability, frequency perturbation rate and amplitude continuity obtained from the analysis, it is determined whether an irreversible frequency abrupt change has occurred.

[0021] When an irreversible frequency abrupt change is detected in at least two different rhythmic signals, and the frequency abrupt change persists for more than one complete rhythmic cycle, it is determined to be a systemic rhythmic collapse.

[0022] Optionally, the determination of effective modal compensation includes:

[0023] After determining that the systemic rhythm collapse has occurred, it is detected whether other modes produce alternative responses to the mode whose rhythmic signal has been broken; if the alternative responses are detected, a safety sensitivity difference score is calculated between the mode that produces the alternative responses and the mode whose rhythmic signal has been broken.

[0024] When the security sensitivity difference score exceeds the set compensation tolerance threshold, it is determined to be invalid modal compensation, and the determination of rhythm collapse is maintained.

[0025] Optionally, it also includes: quantifying system stability through a higher-order perturbation response function;

[0026] The expression for the higher-order disturbance response function is:

[0027] ;

[0028] In the formula, This is a high-order disturbance response intensity index function of the system at the current moment, used to determine whether it has entered the unrecoverable region; This represents the total number of modes currently participating in the calculation; For modality The risk sensitivity coefficient is set based on historical performance or safety level. For modality In time The signal strength sequence at each moment; For modality The acceleration term of the signal represents the degree of intensification of its changing trend; For modality The rate of change of the signal direction vector represents the speed of disturbance in its trend direction; For modality The maximum amplitude difference within the current time window reflects whether the mode is in an amplified fluctuation state; For modality The trend exponential amplification factor is used to enhance the system's sensitivity to sudden nonlinear amplification; This is to apply nonlinear exponential penalty or amplification to modal amplitude perturbations.

[0029] Optionally, the control method further includes refined discrimination of the degree of modal collapse, including:

[0030] Establish a temporal dependency model between modes based on historical operating condition data;

[0031] The actual collaborative relationship between each modality and the mutual dependency mismatch rate between the temporal dependency relationship model are calculated in real time. When the mutual dependency mismatch rate exceeds a preset threshold, the response observation window is shortened to enter the intervention determination stage.

[0032] If some modal trends diverge but key safety modes remain stable, the diverging modal trends are determined to be non-safety-related interference trends and are kept under observation.

[0033] When a sudden decoupling is detected between two or more key modes and the interdependence mismatch rate exceeds a threshold, the length of the second time window is shortened.

[0034] Optionally, the monitoring of the self-recovery trend includes:

[0035] Monitor the rate of change of the modal signal, and determine the trend direction of the modal signal based on the change of the modal signal;

[0036] When three or more modal signals maintain the same trend direction in three consecutive update cycles, it is marked as a weak recovery state;

[0037] When the modal signal exhibits high-frequency oscillations with a regular increasing magnitude, it is determined to be nonlinear instability, and the second time window is skipped to directly trigger response control.

[0038] Optionally, the execution response control includes:

[0039] Prioritize applying stabilization intervention signals in the opposite direction to the identified fluctuation source modes;

[0040] If the monitoring system responds to the stabilization intervention signal and the system shows a convergence in the trend direction or a decrease in the intensity of the disturbance, it will switch to the auxiliary recovery path.

[0041] If the system does not exhibit a structural response or the trend diverges further, mandatory response controls, including audible and visual warnings, forced ventilation, or evacuation orders, should be implemented immediately.

[0042] A second aspect of the present invention provides a safety control system for confined space operations, the system comprising:

[0043] The identification unit is used to dynamically identify modal dominance changes during confined space operations based on the operation sequence, and adjust the interpretation priority and judgment reference weight of each modal information according to the changes in the operation stage.

[0044] The judgment unit is used to determine whether the acquired multimodal sensing signal is in a state of overall silence or abnormal silence based on the current stage modal dominance change, according to the adjusted interpretation priority and judgment reference weight. When the multimodal sensing signal is in a state of overall silence or abnormal silence, the establishment unit and the judgment unit are triggered respectively.

[0045] The first analysis unit is used to establish a temporal causal chain between the behavior triggering event and the modal response, wherein the temporal causal chain is verified by: continuously retaining multimodal state snapshots before and after the behavior triggering event within a set first time window, and detecting whether the behavior triggering event causes delayed feedback that conforms to the expected logic in other modalities based on the multimodal state snapshots;

[0046] The second analysis unit is used to monitor the coordinated stability among multiple rhythmic signals during the operation. When a rhythm break or imbalance is detected and no effective modal compensation is generated, it is determined to enter an unstable critical state.

[0047] The control unit is configured to, in response to the determination of behavioral feedback failure through the time-series causal chain verification, or the determination of the unstable critical state, set a second time window as a response observation window, and determine whether to trigger a safety intervention operation by monitoring whether the multimodal signal shows a self-recovery trend within the second time window. When the determination does not meet the self-recovery condition, the control unit executes a response control that matches the current modality dominance.

[0048] As can be seen from the above technical solutions, the present invention has the following advantages:

[0049] 1) By dynamically identifying modal dominant changes during operation and making linked judgments based on the interpretation priority and response sequence between modalities, it is possible to effectively identify "silent anomalies" or "atypical behavioral chains" that are difficult to detect using traditional methods, thus enabling early perception of latent instability and potential accident signs. By analyzing multi-dimensional indicators such as directional perturbation, acceleration, and fluctuation amplitude of modal signals to construct a high-order perturbation function, the observation window, intervention response rhythm, and control strategy are dynamically adjusted under different modal trend states, thereby enhancing real-time environmental adaptability.

[0050] 2) By introducing a modal collapse degree and modal decoupling discrimination mechanism, a systemic collapse trend can be quickly identified when a critical mode fails. Furthermore, through modal compensation tolerance assessment, it can be determined whether a safe and effective alternative mode can be introduced, thus maintaining the overall system's stable monitoring capability even under conditions of partial modal distortion or delay. The introduction of micro-fluctuation direction consistency analysis and a change-direction failure identification mechanism enables fine-grained judgment of whether the system possesses trend-based self-recovery capabilities, avoiding unnecessary interruptions due to misjudgments and improving the accuracy of system fault tolerance and continuity assessment. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 A flowchart illustrating a safety control method for confined space operations provided in an embodiment of the present invention;

[0053] Figure 2 A simplified flowchart of the confined space operation safety management method provided in this embodiment of the invention;

[0054] Figure 3 This is a flowchart of multimodal signal evaluation and intervention provided in an embodiment of the present invention;

[0055] Figure 4 This is a flowchart of multimodal instability monitoring and intervention provided in an embodiment of the present invention;

[0056] Figure 5 A simplified flowchart of multimodal monitoring and intervention response for confined space operations provided in Embodiment 2 of the present invention;

[0057] Figure 6This is a flowchart of multimodal monitoring and intervention response for gas pipeline maintenance operations provided in Embodiment 3 of the present invention.

[0058] Figure 7 This is a schematic diagram of a confined space operation safety control system provided in an embodiment of the present invention. Detailed Implementation

[0059] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0060] Example 1:

[0061] Please see Figure 1 and 2 The present invention provides a confined space operation safety control method, comprising:

[0062] Step 101: Dynamically identify modal dominant changes in the confined space operation process according to the operation sequence, and adjust the interpretation priority and judgment reference weight of each modal information according to the changes in the operation stage.

[0063] In one embodiment, step 101, dynamically identifying modal dominance changes during confined space operations based on the operation time sequence, includes: constructing a time segment template for modal dominance transition based on historical operation experience data, wherein the time segment template contains the dominance priority sequence and weight change curve of each modality under different operation stages; matching the modal distribution characteristics of the current operation stage with the time segment template, and triggering a reassessment of modal dominance when the deviation exceeds a preset dominance transition deviation threshold; and, when multiple modalities compete for dominance, assisting in determining the modal interpretation priority of the current stage based on the difference in reaction delay from event triggering to signal response of each modality.

[0064] Specifically, during confined space operations, various modal dominant changes are dynamically identified based on the operation time sequence. These include, but are not limited to, environmental gas concentration modalities, worker vital sign modalities, behavioral modalities, voice and audio modalities, and visual image modalities. The modal signals are uniformly mapped to the time-series perception management channel within the system (which can be understood as the system that implements the confined space operation safety control method; the term "system" will not be repeated below). The system first dynamically identifies the modal dominant signals that should be emphasized at the current stage based on the temporal evolution characteristics of the operation. That is, it judges the degree of dependence on a certain type of modality at the current time period based on dimensions such as operation process nodes, risk density distribution, and correlation with historical accidents. For example, in the initial stage of entering the confined space, the focus is on monitoring oxygen and hydrogen sulfide modalities, while in the middle and later stages, more emphasis is placed on behavioral and vital sign modalities. While identifying modal dominance, the system automatically adjusts the interpretation priority and judgment reference weight of each modality, so that the multimodal collaborative fusion model has adaptive interpretation capabilities at different stages.

[0065] Step 102: Based on the adjusted interpretation priority and judgment reference weight, determine whether the collected multimodal sensing signal is in an overall silent or abnormally silent state according to the modal dominance change in the current stage. If the multimodal sensing signal is in an overall silent or abnormally silent state, then execute steps 103 and 104 respectively.

[0066] Understandably, potential hidden risks can be identified by comparing the degree of deviation between the expected response trajectory and the actual signal response of each modality's dominant changes under current environmental and behavioral conditions.

[0067] Specifically, the system further performs a real-time overall activity assessment of the acquired multimodal signal status, that is, to determine whether all modes are currently in a silent or near-silent state. This judgment is achieved by comparing the deviation between the typical response trajectory of each mode under normal operating conditions and the actual acquired signal. The system presets the response intensity, trend of change, and synchronous fluctuation characteristics of each mode under specific operating scenarios, and generates expected response templates based on the historical coupling behavior between modes. If multiple main modes do not show the expected response characteristics within the expected time window, or the response trend is significantly weakened and asynchronous, the system will determine the current state as abnormal silence, and trigger the marking and warning of potential hidden risks, such as non-sudden safety hazards that cannot be immediately manifested, such as workers falling into a coma, equipment silent failure, and gas sensing failure, can be identified in advance under this mechanism.

[0068] Step 103: Establish a temporal causal chain between the behavior triggering event and the modal response, wherein the temporal causal chain is verified by the following method: continuously retain multimodal state snapshots before and after the behavior triggering event within a set first time window, and detect whether the behavior triggering event causes delayed feedback in other modalities that conforms to the expected logic based on the multimodal state snapshots.

[0069] In one embodiment, step 103, the verification process of the temporal causal chain, includes: when no expected response feature matching the behavior triggering event is detected based on the multimodal state snapshot within three consecutive modal update cycles, the behavior feedback is determined to be invalid; when a modal response related to the behavior triggering event is detected, but the activation time order of each modal response is inconsistent with the preset standard response order, modal logic verification is performed; the modal logic verification is used to determine whether the current response order belongs to an atypical but safe behavior response chain based on the job context and behavior goal.

[0070] Specifically, this mechanism constructs a temporal causal chain between behavioral trigger events and modal responses to determine whether there is a reasonable linkage between work behaviors and multimodal signals. After a work behavior is identified and triggered, the system continuously tracks whether the behavior triggers a modal response in other modalities within a preset time observation window. For example, when a worker completes a significant bending motion, the system uses this behavior as the starting point of the chain and enters the state monitoring phase. During the state monitoring period, the system searches for signal segments such as a slight upward trend in the heart rate modality, a corresponding change in displacement angle in the posture modality, a disturbance in the respiratory sound frequency in the speech modality, or a slight increase in CO2 in the environmental modality. If the system continuously tracks a multimodal response sequence within this time window, and the temporal order and amplitude change direction of these responses are highly consistent with the historical behavioral response model, then the system will be considered successful. If the response chain triggered by the behavior is closed, the system's judgment capability is considered effective. Conversely, if the system fails to receive any effective modal response related to the behavior within the time window, or if the response signal exhibits a logical error (such as reversed sequence, reversed amplitude, or missing response), the system determines that the behavior chain is not closed and further marks it as a behavior feedback failure. This situation may mean that the behavior has occurred but has not been perceived by other modalities, such as the sensor failing to track after personnel operation, equipment reacting slowly, modal signal channels being blocked, or personnel becoming disabled. Based on this, the system will perform further risk assessment and intervention decisions. This chain-based judgment method can effectively improve the system's ability to identify weak anomalies and non-explosive risks, overcome the shortcomings of traditional multimodal systems that are too sensitive to single-point-of-time state errors, and realize the temporal integration judgment and dynamic mismatch detection of the operation state in a confined space.

[0071] The system continues to monitor the rhythmic behavior of various modal signals during operation to determine the dynamic coordination stability of the current system state. Rhythmic signals include, but are not limited to, the breathing rhythm, gait rhythm, intonation rhythm of workers, repetitive work rhythms, and periodic or quasi-periodic dynamic signals such as equipment operation sounds or environmental background sounds. The system extracts features from these rhythmic signals and performs time-series comparisons to construct a rhythm mapping matrix, which is used to determine whether there is coordinated resonance or overall rhythmic stability among the various modes. When the system detects a break, frequency disorder, discontinuous beat, or significant deviation from its historical rhythm spectrum in one or more key rhythmic signals, it determines that the mode has experienced a rhythmic breakdown. At this point, the system further searches for other modal signals that actively undergo rhythmic compensation behavior, such as a decrease in speech rhythm but a stabilization in gait rhythm, or a uniformization of feedback noise within the equipment's operating cycle. If no modal signal is found to coordinately adjust or substitute for equilibrium within a specified time window, the system classifies the overall state as an unstable critical state, indicating that the system may be about to enter a critical state. In the event of an irreversible safety instability process, a pre-emptive risk response mechanism must be activated. In this state, the system automatically sets a response observation window, the length of which can be dynamically adjusted according to the current modal misalignment level. Within the observation window, the system continuously tracks the trend behavior of multimodal signals to determine whether there is a self-recovery trend. That is, it observes whether the rhythmic signals gradually return to their original rhythm or form a new rhythmic stability structure. If the system determines that the multimodal signals show signs of trend convergence, rhythm resynchronization, or natural convergence of fluctuation intensity within the observation window, it is considered that the system has endogenous self-recovery capability, and it can choose not to perform intervention operations, but only record the critical instability event. If the system detects that the modal signals continue to diverge, the rhythmic break chain expands, or some modal recovery fails and produces reverse fluctuation amplification within the observation window, it can be determined that the system recovery path has been interrupted and it does not have trend self-balancing capability. At this time, a safety intervention decision is immediately triggered. The system selects an appropriate response control method according to the current modal dominance, such as issuing audible and visual warnings, remotely linking forced ventilation, and emergency broadcasting to prompt personnel evacuation, to achieve real-time containment of the potential evolution into an accident state within a limited space with minimal delay.

[0072] Combined with appendix Figure 3As shown in the process flow, structured mining is performed based on a large amount of historical operational experience data. Multimodal signal evolution behavior is clustered and analyzed according to operation type, process flow, time sequence nodes, and typical event labels. This results in the construction of a temporal segment template for modal dominance transition. This template uses time as the main axis to divide the limited space operation cycle into multiple stage segments. Each segment records the dominant priority sequence, weight change curve, and expected dominant mode switching time point for different modes under normal conditions. It also includes risk characteristics of dominant mode mismatch in accident samples. During the current operation, the system matches the collected information such as operation rhythm, personnel behavior, and environmental changes in real time. When the modal distribution characteristics of the current operation stage deviate from the expected distribution in the template, and the deviation exceeds the dominant mode transition deviation threshold set by the system, the system triggers a modal dominance reassessment mechanism. Under this mechanism, the system fuses and scores multiple indicators such as the effectiveness, signal integrity, change frequency, and historical response confidence of all current modes, initially forming a dominant mode ranking.

[0073] If multiple modal scores are close and competing for dominance during the ranking process, the system further introduces modal response delay differences as an auxiliary judgment criterion. That is, it calculates the delay difference between the occurrence of the triggering event and the generation of the signal response for each modality, and combines it with the average expected response delay of the modality in historical scenarios. If a modality responds more quickly and has a clear trend, it is given priority to be given the dominant interpretation right in the current stage. Conversely, if a modality responds lagging, fluctuates violently, or is unstable, it is downgraded. This mechanism ensures that the dominance adjustment process has a dual decision-making basis based on behavior-driven and time series accuracy constraints, avoiding the system misjudging the dominant modality or over-relying on low-quality signals in the case of modal competition.

[0074] When a behavioral change event is detected, the system automatically takes snapshots of the multimodal data state within several time steps before and after the behavioral change, forming a behavioral state segment. This snapshot includes multiple dimensions such as changes in the worker's posture, voice loudness and semantic fluctuations, environmental gas disturbance trends, equipment response delay curves, and changes in vital signs. Its time window generally covers 1 to 2 modal sampling cycles before and after the behavioral event, ensuring that the system can fully capture the starting point of the signal disturbance caused by the behavior and its early signs of impact on the system. After the behavioral state segment is generated, the system will enter a continuous three-round modal update window monitoring period. Within this window, the system actively compares whether multiple modalities produce response characteristics that match the behavioral segment within the set time delay range. If all major modalities do not produce the expected response within the three consecutive update cycles, or if the generated response is not significantly related to the behavioral characteristics, it is determined that the behavior has not been fully recognized by the system or the modal feedback path is interrupted, and thus it is marked as a feedback failure state.

[0075] Step 104: Monitor the coordinated stability among multiple rhythmic signals during the operation. When a rhythm break or imbalance is detected and no effective modal compensation is generated, it is determined to have entered an unstable critical state.

[0076] In one embodiment, the determination of rhythm breakdown includes: performing time-frequency domain feature analysis on the periodic rhythmic signal, and determining whether an irreversible frequency mutation has occurred based on at least one of the periodic stability, frequency perturbation rate, and amplitude continuity obtained from the analysis; when the irreversible frequency mutation is detected in at least two rhythmic signals from different sources, and the frequency mutation persists for more than one complete rhythm cycle, it is determined to be a systemic rhythm collapse.

[0077] The determination of effective modal compensation includes: after determining that the systemic rhythm collapse has occurred, detecting whether there are other modes that produce alternative responses to the mode whose rhythmic signal has been broken; if the alternative responses are detected, calculating the safety sensitivity difference score between the mode that produces the alternative responses and the mode whose rhythmic signal has been broken; when the safety sensitivity difference score exceeds a set compensation tolerance threshold, it is determined to be invalid modal compensation, and the determination of rhythm collapse is maintained.

[0078] Specifically, to prevent misjudgment due to inconsistent modal responses, the system does not immediately make a feedback failure judgment when it determines that a modal signal exists but the response logic is disordered. Instead, it performs modal logic verification processing based on the current behavioral event and its modal response chain. This modal logic verification processing establishes a standard response sequence model based on the work context and behavioral goal, and performs logical consistency verification according to parameters such as the activation time sequence of each modality under behavior-driven, signal peak delay, and response direction consistency. If it is found that although the modal signal exists, its sequence is seriously disordered from the standard model, such as the voice response preceding the posture change or the heart rate rise occurring before the action is completed, it will further determine whether the response chain belongs to an atypical behavioral response chain. Modal disordered patterns caused by special behaviors such as fatigue, psychological stress, non-standard operation, or work deviation will be recorded into the atypical behavioral response library and handed over to expert rules or machine learning auxiliary engines for secondary analysis.

[0079] By performing high-frequency sampling analysis on the time-frequency domain characteristics of key modal signals, the dominant rhythm parameters, including period stability, frequency perturbation rate, and amplitude continuity, are calculated in real time. The determination of rhythm breakage does not rely on frequency changes in a single modality. Instead, it requires the system to detect unrecoverable frequency abrupt changes in multiple key modalities (at least three types, such as vital sign modality, behavioral / action modality, and speech modality). This unrecoverability is defined as a frequency offset exceeding twice the standard deviation of the modality's historical stable interval and persisting for more than one complete rhythm cycle. If this abrupt change occurs not only in a single modality but also spans two or more independent modal rhythms and simultaneously exhibits a trend of non-automatic regression, the system immediately determines it as a systemic rhythm collapse, no longer waiting for natural rhythm recovery or modal self-adjustment compensation, and automatically enters an abnormal intervention preparation state. Simultaneously, to avoid system... To address false alarms related to rhythm disturbances, a modal compensation capability assessment mechanism is further implemented to determine whether other modalities can provide alternative responses to the collapsed modality. These alternative modalities must not only possess temporal synchronicity but also have acceptable functional equivalence in terms of safety sensitivity. Therefore, the system constructs a modal alternative sensitivity scoring model, comparing the risk identification accuracy and signal feature analysis capability of the original modality with the identification confidence of the alternative modality in the current scenario, and calculating its safety sensitivity difference score. If the score does not exceed the system's set compensation tolerance threshold, the alternative modality compensation is considered effective, and intervention measures can be temporarily suspended while trend monitoring continues. If the difference score significantly exceeds the threshold, indicating insufficient coverage of core risk factors or severe asymmetry in information expression dimensions by the alternative modality, the system rejects the alternative compensation strategy, maintains the rhythm collapse state determination, and escalates the intervention priority.

[0080] In one embodiment, it further includes: using a higher-order perturbation response function Quantifying system stability: ;

[0081] In the formula, This is a high-order disturbance response intensity index function of the system at the current moment, used to determine whether it has entered the unrecoverable region; This represents the total number of modes currently participating in the calculation; For modality The risk sensitivity coefficient is set based on historical performance or safety level. For modality In time The signal strength sequence at each moment; For modality The acceleration term of the signal represents the degree of intensification of its changing trend; For modality The rate of change of the signal direction vector represents the speed of disturbance in its trend direction; For modality The maximum amplitude difference within the current time window reflects whether the mode is in an amplified fluctuation state; For modality The trend exponential amplification factor is used to enhance the system's sensitivity to sudden nonlinear amplification; This is to apply nonlinear exponential penalty or amplification to modal amplitude perturbations.

[0082] Specifically: in conjunction with the appendix Figure 4 As shown in the flowchart, in order to achieve quantitative identification of risk states and adaptive response intervention control, the system introduces a higher-order disturbance response function. This function serves as a comprehensive indicator of the changing trend of modal cooperative stability. It is specifically used to determine whether the system is in a recoverable state, a critical observation state, or has entered an unrecoverable state, and accordingly dynamically adjusts the response observation window and the timing of triggering intervention operations. In specific implementation, the system first performs a check on all modalities during operation. signal Real-time data collection and trend modeling are performed, and the derivative behavior is decomposed and analyzed. Indicates the trend velocity of the modal signal. The acceleration term of the signal is used to measure whether its trend is intensifying or about to undergo a nonlinear abrupt change, while also introducing the direction vector of the mode in the state space. This variable reflects the directional characteristics of modal signal evolution, and its first derivative... The product of these two terms represents the directional perturbation rate. It can characterize the overall risk intensity of the current mode as the trend intensifies and the direction becomes unstable, thereby reflecting the possibility that the mode is in a state of dynamic instability or divergence.

[0083] The system further analyzes the maximum signal disturbance amplitude of the mode within a set time window. This is used to quantify the fluctuation intensity of modal signals, and a modal characteristic amplification factor is introduced based on this. Through nonlinear exponential terms By nonlinearly combining fluctuation intensity and trend change, the system maintains robustness in judging weak modal perturbations while sensitively amplifying their weights during severe fluctuations, thus constructing a complete higher-order perturbation term for each mode. The system then assigns a corresponding risk sensitivity coefficient to each mode. This is used to adjust for the differences in the contribution of different modes to the overall system stability. The system normalizes and sums the above higher-order perturbation expressions of all modes to form the system's higher-order perturbation response function. (See the expression above).

[0084] The function value is continuously calculated during runtime for monitoring system stability trends. When the system is in a stable, low-disturbance state, the modal signals, though exhibiting slight variations, maintain consistency or convergence in direction, velocity, and amplitude, indicating that it is currently in the self-recovery zone and requires no intervention; when exist Fluctuations within the range indicate that the system has a certain tendency to be disturbed but has not yet evolved into an unstable state. At this time, the system enters the critical observation region. The length of the response observation window is dynamically set according to the current function value growth rate, the number of participating instability modes, and their fluctuation contribution rate to observe whether the system has a natural recovery trend; if Rapid growth and exceeding the upper limit threshold at several consecutive time points. If the directional disturbance, acceleration, and fluctuation intensity of at least two or more modes simultaneously reach unstable peak values, the system can determine that the current state has entered the unrecoverable zone and needs to immediately trigger safety intervention operations, including control commands such as voice alarms, personnel evacuation, automatic ventilation, and on-site lighting switching. This function realizes unified modeling of multiple risk characteristics such as modal evolution trends, rhythm breakage, response disorder, and recovery interruption, improving the system's perception depth and judgment accuracy of the unstable state of complex multimodal signals.

[0085] In one embodiment, the method further includes a control method, which further includes a refined judgment of the degree of modal collapse, including: calculating in real time the actual collaborative relationship between each modality and the mutual dependency mismatch rate between the time-series dependency relationship model; when the mutual dependency mismatch rate exceeds a preset threshold, shortening the response observation window to enter the intervention judgment stage; when a divergence trend of some modalities is detected but a key safety modality is detected to remain stable, the diverging modalities are determined to be non-safety-related interference trends and are kept under observation.

[0086] Specifically, a refined discrimination mechanism for modal collapse is introduced. This mechanism not only considers the perturbation intensity, directional abrupt changes, and fluctuation amplitude of individual modes, but also comprehensively considers the changes in the interdependence between modal signals. The system first establishes a temporal dependency model between modes based on historical operating data. This model includes a modal cooperative feature matrix composed of parameters such as signal activation sequence, response coupling delay, resonant frequency band, and trend direction consistency. This matrix represents the stable correlation structure between multiple modes under normal operating conditions. During system operation, the system calculates the mismatch between the current modal cooperative structure and the standard matrix in real time, paying particular attention to whether the dependencies between key modes (such as vital sign modalities and behavioral action modalities, gas concentration modalities and visual abnormality modalities) undergo sudden decoupling. Specifically, this means that modal pairs that should have been activated or evolved synchronously under the same behavior or event exhibit unilateral fluctuations, asynchronous activation, or complete trend deviation in the current state. The system quantifies this decoupling behavior. The system derives a modal interdependence mismatch rate index. When two or more key modes are detected to suddenly decouple and their coordination consistency drops below a preset threshold, the system immediately determines that the current modal collapse has reached the criteria for early intervention preparation. It then automatically shortens the originally set response observation window, allowing the system to quickly enter the intervention judgment stage to prevent the continued spread of modal decoupling from causing systemic information structure collapse and delaying risk response. In addition, to avoid misjudgment intervention caused by a single modal anomaly or external environmental disturbance, the system sets a stability priority verification mechanism for key safety modes when processing modal divergence trends. That is, when the system detects that some modes (such as equipment acoustic modes or video light modes) show divergence trends, but at the same time, key modes such as vital signs modes, main behavior modes, or gas safety modes remain stable and have not experienced frequency abrupt changes or trend reversals, the system marks this type of divergence trend as a non-safety-related interference trend and judges that its source may be non-accident factors such as background noise, non-related equipment actions, or sensor accuracy fluctuations.

[0087] Step 105: In response to the determination of behavioral feedback failure or unstable critical state through the temporal causal chain verification, a second time window is set as the response observation window. By monitoring whether the multimodal signal shows a self-recovery trend within the second time window, it is determined whether a safety intervention operation is triggered. When the determination does not meet the self-recovery condition, response control that matches the current modality dominance is executed.

[0088] In one embodiment, step 105, monitoring the self-recovery trend includes: monitoring the rate of change of the modal signal, determining the trend direction of the modal signal based on the change of the modal signal; when three or more modal signals maintain the same trend direction in three consecutive update cycles, they are marked as a weak recovery state; when the modal signal exhibits a high-frequency oscillation with a regular increasing magnitude, it is determined to be nonlinear instability, and the second time window is skipped to directly trigger response control.

[0089] Specifically: A direction reversal consistency verification algorithm is introduced. Within a set dynamic time window, the direction vector of the modal signal change trend is continuously tracked. When a certain mode reverses its trend direction in two consecutive time segments, i.e., from rising to falling and then reversing to rising, or changing in the opposite direction, the system will record the reversal behavior and further determine whether the direction after the two reversals converges again or shows a relatively stable unidirectional trend. If the directions after the reversal are still divergent or do not overlap significantly, the mode is considered to be in a state of trend direction disorder. If such trend disorder occurs concurrently in multiple key modes and lasts for more than two complete rhythm cycles, the system marks the state as a systemic trend divergence and enters the response observation window adjustment mechanism and intervention preparation process accordingly. Conversely, if the system recognizes that the trend gradually converges after the reversal, even if there are local fluctuations, it will not be immediately identified as an disordered state, thereby avoiding misjudgment and excessive intervention. Regarding trend recovery, the system introduces a consistency judgment rule for modal recovery trends. When the system is detected to be in an unstable critical region or has triggered the response observation window, the system continuously monitors the micro-fluctuation direction of all main modes, extracts their first-order change direction vectors, and statistically judges whether their trend directions show consistency within a short period. If three or more modes maintain consistent trend directions in three or more consecutive update cycles, that is, simultaneously exhibit convergent evolution characteristics such as rising, falling, or weakening oscillations, then this state will be marked as a weak recovery state. This indicates that although the system has not yet fully recovered to a stable state, it has shown signs of active coordination and modal recoupling. At this stage, the system will prioritize maintaining the response observation state rather than immediately implementing control intervention, and continue to evaluate whether the trend consistency expands or stabilizes. This mechanism uses high-dimensional indicators such as the periodicity, consistency, and reversal overlap of trend directions to semantically analyze the modal state evolution process.

[0090] When the system detects that one or more modes oscillate at high frequency within a preset time period, and the amplitude of the oscillation shows a regular increasing trend, that is, the difference between the peak and trough of the amplitude of each oscillation cycle continuously increases with time, the system no longer classifies the state as temporary jitter or initial fluctuation of recovery, but directly identifies it as a typical nonlinear instability phenomenon. Such nonlinear instability usually means that the system has lost its internal buffer mechanism, and the disturbance is amplified rather than attenuated in the modal feedback path, with a very high possibility of systemic risk evolution. Therefore, after detecting this feature, the system does not enter the observation window stage, but immediately prioritizes the execution of response control mechanisms, such as activating safety prompts, cutting off dangerous processes, or starting the forced ventilation system, in order to minimize the response delay from identification to control.

[0091] Furthermore, when judging the trend direction of modal coordination, in order to solve the problem that some modalities may have biased trend judgments due to high historical noise, frequent false triggers, or instability differences, the system introduces a dynamic modal weight redistribution strategy. In each trend consistency analysis, not only the disturbance intensity and direction vector of the current modality are considered, but also the stability evaluation coefficient is calculated based on indicators such as the fluctuation confidence interval, early warning misjudgment rate, and typical stable window duration of the modality in the historical operating environment. For modalities with low stability, trend direction bias correction is performed, that is, by shrinking their trend disturbance vector or reducing their voice in the trend voting mechanism, so as to avoid them dominating the system's trend direction judgment due to short-term anomalies.

[0092] When a modal signal exhibits two distinct fluctuations in the recovery direction after entering the response observation window—meaning its trend direction, amplitude changes, or stability indicators show typical recovery characteristics such as convergence, slowdown, or mean reversion within a short period—but the system subsequently finds, in the trend direction analysis after the end of each fluctuation cycle, that the modal signal reverses back to the divergent direction, and the divergent characteristics show increasing trends in speed, direction perturbation, or frequency amplitude, the system marks this state as a failed change of direction. It no longer considers it a normal fluctuation process in the initial recovery state but directly identifies it as a recovery interruption. This type of judgment avoids misjudging false recovery processes and is particularly suitable for risky scenarios where there is a short-term stabilization phenomenon but ultimately slips into an uncontrollable state. In this state, the system will exit the observation mechanism and immediately execute a control response. Simultaneously, in setting the response observation window, the system considers not only the degree of modal collapse (such as higher-order perturbation response functions) but also... The initial time window length is determined by the value of the modal decoupling number. An adaptive adjustment strategy for modal historical response delay is also introduced. That is, the system backtracks and models the response delay time of each modality in each working condition, calculates its average delay response period and superimposes its fluctuation confidence interval. In the observation window setting, a longer trend evaluation time tolerance is given to this type of modality to avoid misjudging it as unrecovered because individual slow response modes do not form trend signals in time.

[0093] In one embodiment, step 105, performing response control includes: preferentially applying a small amount of stabilization intervention signal in the opposite direction to the identified fluctuation source mode; monitoring the system's response to the stabilization intervention signal; if the system shows convergence in trend direction or a decrease in disturbance intensity, switching to an auxiliary recovery path; if the system does not show a structural response or the trend further diverges, immediately executing mandatory response control including audible and visual warnings, forced ventilation, or personnel evacuation commands.

[0094] Specifically: In this invention, after entering the response observation window, the system performs first-order directional derivative analysis on all main modes, extracts their trend direction vectors, and dynamically calculates the angle between the trend vectors of each pair of modes. When at least two modes exhibit consistent direction vectors (angle less than a set threshold) in three consecutive cycles, and the difference in the amplitude of their recovery trend is less than the system's preset tolerance range for the cooperative ratio (i.e., the proportion of the synchronization amplitude difference to the average amplitude of the two modes is less than the threshold), the system determines that the current mode has a cooperative recovery trend, thus identifying that the system as a whole has the possibility of self-recovery. In this state, the observation window remains valid, and the trend expansion and stability enhancement characteristics continue to be monitored. When the system determines that the recovery trend is interrupted (i.e., the mode continues to diverge, fails to change direction, or the cooperative index falls out of the tolerance range), and response control needs to be executed, in order to avoid the operation interruption or false triggering caused by direct rigid intervention, the system is designed with passive adjustment detection. The mechanism prioritizes core modes identified as sources of fluctuation and applies a small amount of stabilizing intervention signal to the reverse channel of their signal evolution direction. Intervention signals include frequency perturbation damping, weakening of environmental balance perturbation, or semantic intervention. After applying the signal, the system detects the response effect of other modes to the reverse intervention within a short period of time. If the system shows obvious trend convergence, decreased perturbation intensity, or modal recoupling behavior, it can be determined that the system still has a certain degree of passive adjustment capability. The system will switch to the auxiliary recovery path and continue to execute the progressive recovery guidance mechanism, postponing the execution of mandatory response control. Conversely, if the system's modes do not show any structural response after applying the intervention signal, or if the trend further diverges, it is determined that the system's adjustment capability has been lost, and the system immediately enters the response execution process. This judgment logic introduces a state-guided repair strategy while ensuring the real-time nature of safe intervention, thereby improving the overall elasticity of the system's identification capability and the level of refined control.

[0095] Example 2:

[0096] Combined with appendix Figure 5As shown in the process flow, during the cleaning operation inside a chemical plant tank, the workers need to enter the closed storage tank to flush the inner wall with high-pressure water. This operation is identified as a typical confined space operation, which has a complex risk of oxygen deficiency, accumulation of toxic gases, obstruction of vision and personnel fatigue. The system is equipped with a multimodal sensing and analysis unit, including voice signal (V), action recognition (A), vital signs (B, such as heart rate and respiratory rate), ambient gas concentration (G) and local illumination / visual modality (L), forming a five-modal structure. By structurally summarizing the multimodal sequence data from 180 past operations, the system constructed a time segment template for modality dominance transition. This template is divided into five stages: preparation for entry (T1), initial contact (T2), high-intensity operation (T3), interruption / buffering (T4), and final evacuation (T5). During T2-T3, the vital signs modality (B) and environmental modality (G) are set as the dominant modalities. In T1 and T4, the voice modality (V) dominates (such as intercom confirmation and calling for rest). In T5, the action modality (A) dominates, indicating the evacuation rhythm and changes in posture. The template also defines the confidence weight threshold change curve of each modality dominance and the corresponding deviation tolerance of the starting time point, with a maximum of ±30 seconds.

[0097] During a specific operation, the system identified that it had entered the 20th minute of the operation, which, according to the historical template, should be the T3 high-intensity operation phase. This phase should be dominated by the vital signs modality (B) and the environmental modality (G). However, the system currently detected an abnormally high activity frequency of the speech modality (V), with rapid speech and continuous calling, while the B modality remained at a low to medium level, without corresponding increases in heart rate or shortness of breath. The system matched the current modality dominance distribution with the template and found that the V modality weight deviated from the template by more than 0.35, while the B modality weight was 0.28 lower than the expected value, deviating from the overall index by more than the system's preset dominance. When the gender transfer bias threshold (0.3) was reached, the system immediately triggered a dominant adjustment mechanism. During the adjustment of interpretation priority, it was found that the V and G modes were continuously active for 5 seconds, competing for dominant interpretation rights. The system activated the reaction delay difference judgment mechanism and found that the average delay from triggering to system parsing of the speech mode was 0.6 seconds, while the delay of the gas mode was 1.8 seconds due to the longer sensor cycle. Combining the dominant weights of the two, the system decided to temporarily increase the interpretation priority of the V mode in the current abnormal stage, perform deep semantic analysis, and identify it as a keyword for difficulty breathing. Combined with the gas mode, it was found that the hydrogen sulfide concentration rose rapidly (from 3 ppm to 22 ppm). The system judged it as a high-risk event, and then triggered the safety response chain, started the ventilation system, and issued an evacuation command through the headset. Fortunately, the personnel responded in time and no accident occurred.

[0098] This event can also be calculated using the dominant deviation index: at the current moment, the weight of mode V is 0.52, the template expectation is 0.23, and the deviation value is 0.29; the current weight of mode B is 0.25, the template expectation is 0.53, and the deviation value is 0.28; the combined deviation value of the two is 0.57, which is much higher than the threshold of 0.3, and the reaction delay difference is 1.2 seconds. Based on this, the system makes a comprehensive judgment to prioritize the dominant explanation of the mode with a faster response and stronger activity.

[0099] In this embodiment, maintenance worker Li needed to enter a large storage tank to inspect the high-pressure pipeline interface. This storage tank is a typical confined space. A multimodal sensing system was pre-installed, including voice modality (V), action modality (A), vital signs modality (B), gas environment modality (G), and visual illumination modality (L). The system continuously collected data from these five modalities at a 1-second interval. At the 15-minute mark of the operation, the system detected Li suddenly standing up from a squatting position (a sudden change in acceleration of 1.2 m / s²). Simultaneously, the voice modality showed the use of the interjection "uh..." without clear meaning, and the environmental modality showed the hydrogen sulfide concentration rising from 2 ppm to 8 ppm. Although this did not exceed the alarm threshold, it showed a fluctuating upward trend. The system automatically marked this action change as a behavior trigger event and generated a behavior state fragment according to the mechanism of this invention, recording a multimodal snapshot for 5 seconds before and after the event.

[0100] After the behavioral segment was recorded, the system immediately entered three consecutive rounds of modal update monitoring (1 second per round) to track whether a response chain matching the behavior logic appeared. However, no heart rate change was detected in the vital signs modality (always 84 bpm) within 3 consecutive seconds, the speech modality had no further language feedback, only weak noise, and the environmental modality remained in a steady rising state (to 10 ppm). The system thus determined that a no-response state had been formed, that is, the behavioral feedback failed, and initiated the causal chain logic verification.

[0101] After entering the modal response sequence logic judgment module, the system calls the historical causal chain model: under similar actions, the expected modal response sequence should be that the action modality is activated first, followed by an increase in heart rate within 1 second, verbal expression indicating completion of the action or feedback signal, accompanied by slight visual or gaseous environmental disturbances within 1-2 seconds. However, in this detection, the response sequence was: action → speech (noise) → gaseous changes → blank → unchanged heart rate, a severely disordered sequence, with a logic consistency score of only 0.42 (historical threshold is 0.75). The system could not match the existing behavioral chain template, but the signal still existed, so it entered the modal logic verification module. This module compared the current chain with the dizziness prodromal symptoms stored in the atypical behavioral response library and found that the similarity feature matching degree reached 0.81 (standard is 0.8). The system judged that Li might have a physiological dysfunction prodromal, triggered a level 2 safety prompt, and sent the status to the rhythm stability monitoring module.

[0102] At this point, the system begins to perform rhythmic signal breakdown judgment. In the vital signs modality, the system detects that Li's respiratory rhythm fluctuates from 18 breaths / min to 26 breaths / min within 6 seconds, and the frequency curve shows non-periodic fluctuations. The calculated frequency mutation value reaches Δf = +0.67Hz (continuously exceeding twice the standard deviation of the normal baseline). Simultaneously, it is found that the gas mode fluctuation frequency also suddenly increases from 0.2Hz to 0.5Hz within 3 seconds. Both have exceeded the rhythm stability threshold of their respective modes and have undergone synchronous mutations across modes. According to the judgment criteria of this invention, the system identifies it as a systemic rhythm collapse state, skips the delayed observation, and enters the intervention judgment.

[0103] At this point, the system determines whether intervention can be postponed based on the modal compensation judgment mechanism: the system detects that the visual modality L is stable during this period (average illumination 300 lux) and the motion modality remains unchanged, but attempts to find alternative judgment indicators from the visual modality. The system calls up the modal safety sensitivity scoring model and compares the vital signs modality (B) and the visual modality (L). The B modality is 0.95 (high sensitivity) and the L modality is 0.61, with a difference of 0.34. The system sets the compensation tolerance threshold to 0.25. The actual difference has exceeded the tolerance range. Based on this, the system rejects modal compensation and concludes that there is currently no credible alternative path.

[0104] Ultimately, the system executed a response control strategy, issuing a voice alarm indicating an abnormal operation and requesting immediate feedback. It also sent a signal to the external control terminal to trigger emergency ventilation. Simultaneously, it alerted the worker via vibration through a wearable terminal. Upon being alerted, Mr. Li realized his dizziness had worsened and proactively evacuated the tank, preventing further escalation of the risk. Subsequent analysis confirmed that Mr. Li's brief dizziness was caused by slippery conditions and an unstable standing posture while working at the high-pressure water pipe interface. The lack of immediate response in vital signs was due to delayed neural heart rate feedback. This system, relying on behavioral state fragments and causal chain logic for accurate identification, avoided omissions in risk assessment caused by modal lag.

[0105] Example 3:

[0106] Combined with appendix Figure 6 As shown in the process, during the maintenance of underground gas pipelines at a gas company, worker Wang entered a small underground valve well. Inside, a multimodal sensing system was installed to collect real-time data on the worker's vital signs (heart rate B1), movement (limb posture A1), gas environment (combustible gas concentration G1), voice (voice energy V1), and visual (spatial light intensity L1). The five modalities (N=5) of signals were continuously uploaded at a frequency of one sample per second.

[0107] At the 30-minute mark of the operation, the system detected abnormal fluctuations in both gas mode G1 and vital signs mode B1: the gas concentration rapidly increased from an initial stable value of 0.1% to 0.9%, with the fluctuations intensifying; the heart rate of the personnel fluctuated drastically from 70 beats / min to 95 beats / min and then fluctuated repeatedly. To determine the system's recovery capability and whether intervention was necessary, the system employed the higher-order perturbation response function proposed in this invention. Perform the calculation.

[0108] The function takes the following form:

[0109]

[0110] The meanings of each symbol are as follows: It is a high-order disturbance response strength index. For the total number of modes, The modal risk sensitivity coefficients are represented as follows: B1 = 0.9, G1 = 0.85, A1 = 0.7, V1 = 0.6, and L1 = 0.4. For the first Each mode in Time signal value, The acceleration of the modal change, The perturbation velocity is the modal trend direction. The maximum perturbation difference of the current mode. The trend index amplification factor is set to 1.5.

[0111] Detailed calculation process (using G1 and B1 as examples):

[0112] First, the signal values ​​of vital sign modality B1 were calculated to be 70 bpm → 82 bpm → 95 bpm over the last 3 seconds. Then:

[0113] First derivative: ,

[0114] Second derivative: ,

[0115] Trend direction disturbance speed: Because the heart rate trend direction is steadily increasing, at this time... ,

[0116] Maximum disturbance value: .

[0117] Similarly, for the gas mode G1 value over the last 3 seconds: 0.1% → 0.5% → 0.9%, then:

[0118] First derivative: ,

[0119] Second derivative: (The fluctuations are relatively linear)

[0120] Trend direction disturbance speed: Due to the rapid and continuous changes in the trend, it is set to 0.5 rad / s.

[0121] Maximum disturbance value: .

[0122] Substituting into the formula, we get:

[0123]

[0124] Considering that the main contributions are from modes B1 and G1, while the contributions from other modes to transient stability are very small, an approximate calculation is performed:

[0125]

[0126] Calculated values:

[0127] Calculation for item B1: ,

[0128] G1: Since the acceleration is 0, the contribution is 0.

[0129] therefore: .

[0130] The system's preset threshold is:

[0131] .

[0132] The calculation result is significantly higher than the threshold. This indicates that the current system is severely unstable, with a strong divergence in modal coordination trends and no self-recovery capability. The system skips the observation window and directly triggers the highest level intervention response mechanism: immediately starts forced ventilation and exhaust in the well, and urgently issues an evacuation order to Mr. Wang through voice alarms and wearable terminals. Mr. Wang then evacuated. It was later confirmed that there was a gas leak in the well, but no one was injured.

[0133] Another scenario in this embodiment: At the 33rd minute of operation, the system continuously detected drastic fluctuations in the vital signs modality (B1): the heart rate rose from 85 bpm to 102, then fell back to 76, and rose again to 98. The action modality (A1) showed rapid switching between standing up, squatting, and standing up within a short period of time. Although the gaseous modality (G1) remained stable, the energy of the speech modality (V1) gradually increased, and the semantic content included the words "feeling dizzy." The light intensity remained unchanged in the visual modality (L1) during this period. The system immediately calculated the intermodal dependency mismatch rate: signal trend matching was performed between B1 and A1, and the consistency of their change directions was maintained for less than 2 seconds within a 10-second time window; the cross-correlation coefficient between B1 and V1 dropped to 0.31 (normal is 0.72). The system determined that a sudden decoupling phenomenon had occurred between the key modalities. According to the logic of this invention, the response observation window was immediately shortened to 30% of the original planned 20 seconds, that is, only a 6-second observation window was retained for the intervention judgment stage.

[0134] The system then judged the trend direction of each modal signal: within a 6-second time window, the B1 modality showed two consecutive reversals from decreasing to increasing to decreasing again, but the peak and trough values ​​did not coincide (the difference exceeded 8 bpm), which met the systemic misalignment standard of two consecutive reversals that did not coincide. The A1 modality also showed a trend of switching back and forth with micro-acceleration, and the V1 speech spectrum energy also fluctuated repeatedly. The system initially determined that the modal trend direction was divergent and that an intervention mechanism needed to be triggered. However, the system further judged that if the system still had the possibility of self-recovery, it could be temporarily observed.

[0135] At this point, the system performs trend consistency analysis on L1, G1, and V1 based on the micro-fluctuation direction consistency mechanism proposed in this invention. The results show that the fluctuation direction of the three has remained synchronously downward in the past three consecutive cycles (L1 illuminance slightly decreased, G1 concentration dropped by 0.2%, and V1 speech energy weakened). The direction consistency score is 0.91 (higher than the threshold of 0.75). The system thus marks the state as a weak recovery state and does not intervene immediately.

[0136] However, during the subsequent 4-second detection period, the system detected continuous high-frequency oscillations in the B1 mode heart rate with small, incremental increases (84→90→93→98), and the standard deviation showed an upward trend. Superimposed FFT spectral analysis revealed that the dominant frequency increased from 0.1Hz to 0.4Hz without any stable decline, indicating a typical nonlinear instability mode. Based on the design mechanism of this invention, the system skipped the observation window and directly executed the response control strategy.

[0137] Before initiating control, the system re-verifies the effectiveness of the intervention strategy. Using the modal weight redistribution mechanism proposed in this invention, the historical stability score of the B1 modality is found to be 0.58 (lower than the mean of 0.7). The system introduces a bias correction factor to adjust its trend direction, shifting towards a more conservative approach, confirming that the current risk level is high. Next, it checks for trend reversal failures. It is found that the B1 heart rate twice changed from decreasing to increasing within 10 seconds, only to quickly drop back, indicating a failed trend reversal, consistent with the proposed logic of "failed trend reversal = recovery interruption."

[0138] Ultimately, the system decided to implement active intervention, but considering the system's potential for passive adjustment, a reverse intervention mechanism was first used: the system induced the operator to produce self-regulation feedback by issuing a deep breathing prompt (via an earpiece) to the B1 modality, while simultaneously reducing the wellhead fan speed to determine if the system exhibited trend responsiveness. After 3 seconds, the system detected that the B1 modality's heart rate had dropped to 85, and the voice modality had stabilized, confirming the existence of regulatory capacity. The system then marked the entry into the assisted recovery path, continuing dynamic monitoring without triggering higher-level interventions.

[0139] In summary, this example, based on real-time modal data (B1, A1, G1, V1, L1), comprehensively demonstrates the highly sensitive, highly logical, and self-learning safety management capabilities achieved by this invention in extreme confined space operation scenarios based on multimodal signal fusion. The process includes identifying sudden decoupling due to modal mismatch rate, determining misalignment through continuous trend reversal, judging weak recovery based on micro-fluctuation direction consistency, judging nonlinear instability through spectrum analysis, confirming recovery interruption through trend reversal failure, and verifying the system's passive capabilities through reverse intervention. All judgment mechanisms are data-driven, function-supported, and can be implemented in practical applications, showcasing the originality and engineering feasibility of this invention in the field of multimodal dynamic fusion.

[0140] The confined space operation safety management method based on multimodal fusion provided in this invention, during confined space operations, determines whether the collected multimodal sensing signals are in a state of overall silence or abnormal silence based on the current stage's modal dominance, compares the deviation between the responses of each modal signal to identify potential hidden risks; establishes a temporal causal chain between behavioral triggering events and modal responses, continuously retains the state fragments of previous behaviors within a set time window and detects whether they trigger delayed feedback in other modalities; if the behavioral response chain cannot form an effective closed loop, it is considered that the behavioral feedback has failed; monitors the cooperative stability between multiple rhythmic signals during the operation to determine whether the system has entered an unstable critical state; and determines whether to trigger a safety intervention operation. If the modal signals cannot achieve trend self-balancing within a set time threshold, or if their recovery path is interrupted, response control is implemented. This solves the problem that existing systems, due to deficiencies in multimodal information fusion, abnormal trend prediction, and intelligent intervention, cannot timely and accurately identify and handle safety accidents.

[0141] The above is a confined space operation safety control method provided in the embodiments of the present invention. The following is a confined space operation safety control system provided in the embodiments of the present invention.

[0142] Please see Figure 7 The confined space operation safety control system provided in this embodiment of the invention includes:

[0143] The identification unit 201 is used to dynamically identify modal dominance changes in the confined space operation process according to the operation sequence, and adjust the interpretation priority and judgment reference weight of each modal information according to the changes in the operation stage.

[0144] The judgment unit 202 is used to determine whether the acquired multimodal sensing signal is in a state of overall silence or abnormal silence based on the adjusted interpretation priority and judgment reference weight and the modal dominance change in the current stage. When the multimodal sensing signal is in a state of overall silence or abnormal silence, the establishment unit and the judgment unit are triggered respectively.

[0145] The first analysis unit 203 is used to establish a temporal causal chain between the behavior triggering event and the modal response, wherein the temporal causal chain is verified by: continuously retaining multimodal state snapshots before and after the behavior triggering event within a set first time window, and detecting whether the behavior triggering event causes delayed feedback that conforms to the expected logic in other modalities based on the multimodal state snapshots;

[0146] The second analysis unit 204 is used to monitor the cooperative stability among multiple rhythmic signals during the operation. When a rhythm break or imbalance is detected and no effective modal compensation is generated, it is determined to be an unstable critical state.

[0147] The control unit 205 is used to respond to the determination of behavior feedback failure or unstable critical state by verifying the timing causal chain. It sets a second time window as a response observation window and monitors whether the multimodal signal shows a self-recovery trend within the second time window to determine whether to trigger a safety intervention operation. When the determination does not meet the self-recovery condition, it executes response control that matches the current modality dominance.

[0148] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0149] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A safety control method for confined space operations, characterized in that, include: S1. Dynamically identify modal dominance changes during confined space operations based on the operation sequence, and adjust the interpretation priority and judgment reference weight of each modal information according to the changes in the operation stage; S2. Based on the adjusted interpretation priority and judgment reference weight, determine whether the collected multimodal sensing signals are in a state of overall silence or abnormal silence according to the modal dominance change in the current stage. If the multimodal sensing signals are in a state of overall silence or abnormal silence, execute steps S3 and S4 respectively. S3. Establish a temporal causal chain between the behavior triggering event and the modal response, wherein the temporal causal chain is verified by: continuously retaining multimodal state snapshots before and after the behavior triggering event within a set first time window, and detecting whether the behavior triggering event causes delayed feedback that conforms to the expected logic in other modalities based on the multimodal state snapshots; S4. Monitor the coordination stability among multiple rhythmic signals during the operation. When rhythm breakage or imbalance is detected and no effective modal compensation is generated, it is determined to be an unstable critical state. S5. In response to the failure of the behavior feedback verification through the time-series causal chain, or the determination of the unstable critical state, a second time window is set as a response observation window. By monitoring whether the multimodal sensing signal shows a self-recovery trend within the second time window, it is determined whether a safety intervention operation is triggered. When it is determined that the self-recovery condition is not met, response control matching the current modality dominance is executed.

2. The confined space operation safety control method according to claim 1, characterized in that, The method of dynamically identifying modal dominance changes during confined space operations based on the operation sequence includes: A time segment template for modality dominance transition is constructed based on historical operational experience data. The time segment template includes the dominant priority sequence and weight change curve of each modality under different operational stages. The modal distribution characteristics of the current operation stage are matched with the time segment template. When the deviation exceeds the preset dominance transfer deviation threshold, a re-evaluation of modal dominance is triggered. When multiple modalities compete for dominance, the difference in reaction time delay from event triggering to signal response of each modality helps to determine the modal interpretation priority at the current stage.

3. The confined space operation safety control method according to claim 2, characterized in that, The verification process of the temporal causal chain includes: If no expected response features matching the behavior triggering event are detected based on the multimodal state snapshot within three consecutive modal update cycles, the behavior feedback is deemed to have failed. When a modal response related to the behavior triggering event is detected, but the activation time order of each modal response is inconsistent with the preset standard response order, modal logic verification is performed. The modal logic verification is used to determine whether the current response order belongs to an atypical but safe behavior response chain based on the work context and behavior goal.

4. The confined space operation safety control method according to claim 3, characterized in that, The determination of rhythm break includes: Time-frequency domain feature analysis is performed on the periodic rhythmic signal, and based on at least one of the periodic stability, frequency perturbation rate and amplitude continuity obtained from the analysis, it is determined whether an irreversible frequency abrupt change has occurred. When an irreversible frequency abrupt change is detected in at least two different rhythmic signals, and the frequency abrupt change persists for more than one complete rhythmic cycle, it is determined to be a systemic rhythmic collapse.

5. The confined space operation safety control method according to claim 4, characterized in that, The determination of effective modal compensation includes: After determining that the systemic rhythm collapse has occurred, it is detected whether other modes produce alternative responses to the mode whose rhythmic signal has been broken; if the alternative responses are detected, a safety sensitivity difference score is calculated between the mode that produces the alternative responses and the mode whose rhythmic signal has been broken. When the security sensitivity difference score exceeds the set compensation tolerance threshold, it is determined to be invalid modal compensation, and the determination of rhythm collapse is maintained.

6. The confined space operation safety control method according to claim 1, characterized in that, Also includes: Quantify system stability using higher-order perturbation response functions; The expression for the higher-order disturbance response function is: ； In the formula, This is a high-order disturbance response intensity index function of the system at the current moment, used to determine whether it has entered the unrecoverable region; This represents the total number of modes currently participating in the calculation; For modality The risk sensitivity coefficient is set based on historical performance or safety level. For modality In time The signal strength sequence at each moment; For modality The acceleration term of the signal represents the degree of intensification of its changing trend; For modality The rate of change of the signal direction vector represents the speed of disturbance in its trend direction; For modality The maximum amplitude difference within the current time window reflects whether the mode is in an amplified fluctuation state; For modality The trend exponential amplification factor is used to enhance the system's sensitivity to sudden nonlinear amplification; This is to apply nonlinear exponential penalty or amplification to modal amplitude perturbations.

7. The confined space operation safety control method according to claim 6, characterized in that, The control method further includes refined discrimination of modal collapse degree, including: Establish a temporal dependency model between modes based on historical operating condition data; The actual collaborative relationship between each modality and the mutual dependency mismatch rate between the temporal dependency relationship model are calculated in real time. When the mutual dependency mismatch rate exceeds a preset threshold, the response observation window is shortened to enter the intervention determination stage. If some modal trends diverge but key safety modes remain stable, the diverging modal trends are determined to be non-safety-related interference trends and are kept under observation. When a sudden decoupling is detected between two or more key modes and the interdependence mismatch rate exceeds a threshold, the length of the second time window is shortened.

8. The confined space operation safety control method according to claim 7, characterized in that, The monitoring of the self-recovery trend includes: Monitor the rate of change of the modal signal, and determine the trend direction of the modal signal based on the change of the modal signal; When three or more modal signals maintain the same trend direction in three consecutive update cycles, it is marked as a weak recovery state; When the modal signal exhibits high-frequency oscillations with a regular increasing magnitude, it is determined to be nonlinear instability, and the second time window is skipped to directly trigger response control.

9. The confined space operation safety control method according to claim 8, characterized in that, The execution response control includes: Prioritize applying stabilization intervention signals in the opposite direction to the identified fluctuation source modes; If the monitoring system responds to the stabilization intervention signal and the system shows a convergence in the trend direction or a decrease in the intensity of the disturbance, it will switch to the auxiliary recovery path. If the system does not exhibit a structural response or the trend diverges further, mandatory response controls, including audible and visual warnings, forced ventilation, or evacuation orders, should be implemented immediately.

10. A safety control system for confined space operations, characterized in that, include: The identification unit is used to dynamically identify modal dominance changes during confined space operations based on the operation sequence, and adjust the interpretation priority and judgment reference weight of each modal information according to the changes in the operation stage. The judgment unit is used to determine whether the acquired multimodal sensing signal is in a state of overall silence or abnormal silence based on the current stage modal dominance change, according to the adjusted interpretation priority and judgment reference weight. When the multimodal sensing signal is in a state of overall silence or abnormal silence, the establishment unit and the judgment unit are triggered respectively. The first analysis unit is used to establish a temporal causal chain between the behavior triggering event and the modal response, wherein the temporal causal chain is verified by: continuously retaining multimodal state snapshots before and after the behavior triggering event within a set first time window, and detecting whether the behavior triggering event causes delayed feedback that conforms to the expected logic in other modalities based on the multimodal state snapshots; The second analysis unit is used to monitor the coordinated stability among multiple rhythmic signals during the operation. When a rhythm break or imbalance is detected and no effective modal compensation is generated, it is determined to enter an unstable critical state. The control unit is configured to, in response to the determination of behavioral feedback failure through the time-series causal chain verification, or the determination of the unstable critical state, set a second time window as a response observation window, and determine whether to trigger a safety intervention operation by monitoring whether the multimodal signal shows a self-recovery trend within the second time window. When the determination does not meet the self-recovery condition, the control unit executes a response control that matches the current modality dominance.

Images