Multi-modal based hidden danger identification monitoring and early warning method and system

Abstract

The application discloses a hidden danger identification monitoring and early warning method and system based on multi-modal, and belongs to the technical field of risk early warning. The method specifically comprises the following steps: collecting relevant data of a chemical production area, including chemical production related data and personnel related data; judging whether there is a physical inducement causing personnel injury based on a preset process safety critical value; if there is, generating a three-dimensional dynamic danger potential field based on the chemical production related data; generating a personnel motion trajectory based on the personnel related data; mapping the personnel motion trajectory to the three-dimensional dynamic danger potential field; generating a multi-modal knowledge graph; and performing causal reasoning on the path in the multi-modal knowledge graph. When the reasoning result shows that the personnel gathering behavior and the physical inducement exist convergence, an early warning signal is generated, and the early warning signal is output. The application can generate a traceable causal chain when the hidden danger is still in the initial stage based on multi-dimensional information evolution prediction, and significantly improves the accuracy and advance nature of early warning.

Description

Technical Field

[0001] This invention belongs to the field of risk warning technology, specifically a method and system for identifying, monitoring and warning of hidden dangers based on multimodality. Background Technology

[0002] In the actual operation of chemical plants, on-site workers often gather in the plant area or around hazardous sources for reasons such as maintenance, inspection, training, or emergency response. If this gathering is coupled with the state of potential hazards, it can easily amplify the risk of an accident. Therefore, real-time identification of abnormal gatherings and early warning are important technical requirements for chemical safety management.

[0003] Currently, most hazard monitoring focuses on single-dimensional data collection, such as parametric monitoring of equipment operation through sensors like temperature and pressure, or visual recording of personnel behavior through video surveillance. However, these methods are mostly limited to alarms triggered by exceeding static limits, making it difficult to mine the correlations between different modalities of data and reveal potential risk propagation paths under multi-source information fusion. On the other hand, in recent years, technologies such as artificial intelligence and knowledge graphs have been increasingly applied to industrial safety, but existing multi-modal information fusion methods lack dynamic reasoning on causal relationships. Especially in complex interactive scenarios involving hazards, equipment status, environmental parameters, and personnel behavior, there is still a lack of a system framework capable of unified modeling and temporal causal reasoning. Although existing technologies have improved safety management to some extent, they still have certain shortcomings: poor scenario adaptability, lack of dynamic correlation with hazards, leading to delayed early warnings. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention proposes a multimodal method and system for hazard identification, monitoring, and early warning.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] Multimodal hazard identification, monitoring, and early warning methods include:

[0007] Collect relevant data from the chemical production area, including chemical production-related data and personnel-related data;

[0008] Based on the preset process safety threshold, determine whether there are physical causes that could lead to personnel injury. If so, generate a three-dimensional dynamic hazard potential field based on relevant chemical production data.

[0009] The personnel motion trajectory is mapped to a three-dimensional dynamic danger potential field, a multi-modal knowledge graph is generated, and a causal inference is performed on a path in the multi-modal knowledge graph. The causal inference is a joint simulation of the evolution of the three-dimensional dynamic danger potential field and the personnel motion trajectory in a future preset time period. The correlation strength between the personnel gathering behavior and the physical inducement is determined by dynamically updating the risk transmission weight and filtering the convergent path.

[0010] When the inference result indicates that the personnel gathering behavior and the physical inducement exist convergent, a warning signal is generated, and the warning signal is output.

[0011] Specifically, whether a physical inducement leading to personnel injury exists is determined based on a preset process safety critical value. If a physical inducement exists, a three-dimensional dynamic danger potential field is generated based on chemical production related data, including:

[0012] The chemical production related data is time-synchronized processed;

[0013] The processed chemical production related data is compared with the preset process safety critical value to determine whether a physical inducement exists, in which temperature, pressure, concentration or environmental condition abnormalities reach the preset process safety critical value;

[0014] If a physical inducement exists, the position of the danger source, the type of dangerous substance, and the environmental propagation condition corresponding to the physical inducement are obtained and combined with the processed chemical production related data;

[0015] Based on the combined data, a three-dimensional dynamic danger potential field is generated according to the danger source influence factor, the environmental propagation direction, and the personnel accessible path.

[0016] Specifically, the three-dimensional dynamic danger potential field is generated based on the combined data according to the danger source influence factor, the environmental propagation direction, and the personnel accessible path, including:

[0017] Based on the position of the danger source and the type of dangerous substance, the danger source influence factor is determined;

[0018] The danger source influence factor is combined with real-time environmental parameters to obtain a risk intensity coefficient of each danger source under current conditions;

[0019] Based on the risk intensity coefficient, the influence range of the danger source in the three-dimensional space of chemical production is weighted and expanded or contracted to form a weighted influence domain of each danger source;

[0020] The weighted influence domain is superimposed with the actual permitted area and the prohibited area to obtain a set of personnel accessible paths;

[0021] A corresponding risk weight is assigned to each personnel accessible path in the set of personnel accessible paths, and a three-dimensional dynamic danger potential field is generated in the three-dimensional chemical production space.

[0022] Specifically, the personnel motion trajectory is generated based on the personnel-related data, the personnel motion trajectory is mapped to the three-dimensional dynamic danger potential field, a multi-modal knowledge graph is generated, and causal reasoning is performed on the paths in the multi-modal knowledge graph, including:

[0023] The personnel-related data is time-synchronized and spatially aligned to obtain a personnel position sequence in the three-dimensional chemical production space;

[0024] Based on the personnel position sequence, the personnel motion trajectory of each personnel is generated in combination with the personnel-related data collection time interval and the personnel identity identifier, and the timestamp and position accuracy level of each position point in the personnel motion trajectory are labeled;

[0025] The personnel motion trajectory is mapped to the three-dimensional dynamic danger potential field, and the spatial proximity parameter and the time proximity parameter of the weighted influence domain of the personnel motion trajectory and the danger source are calculated during the mapping process;

[0026] A multi-modal knowledge graph containing personnel nodes, chemical equipment nodes, danger source nodes and environment nodes is constructed, and personnel, danger sources, environment parameters and spatial proximity parameters and time proximity parameters are used as attributes of nodes or edges;

[0027] Based on the multi-modal knowledge graph, causal reasoning is performed along the multi-hop path from the danger source node to the personnel node.

[0028] Specifically, the personnel motion trajectory is mapped to the three-dimensional dynamic danger potential field, and the spatial proximity parameter and the time proximity parameter of the weighted influence domain of the personnel motion trajectory and the danger source are calculated during the mapping process, including:

[0029] The position points of the personnel motion trajectory are aligned with the reference coordinate system of the three-dimensional dynamic danger potential field, and the corresponding uncertainty envelope is generated according to the position accuracy level, and the correspondence between the trajectory timestamp and the version of the three-dimensional dynamic danger potential field is established according to the three-dimensional dynamic danger potential field update time sequence;

[0030] The missing positions of the personnel motion trajectory are constrained and completed, and the coverage relationship between the personnel motion trajectory and the weighted influence domain of the danger source is determined, and the entry event, the traversal event and the exit event are marked;

[0031] The personnel motion trajectory is segmented according to the event markers, and the spatial proximity parameter of each mapping segment is calculated, including the distance parameter, the direction consistency parameter and the path fitting degree parameter;

[0032] The time proximity parameter of each mapping segment is calculated in chronological order according to the three-dimensional dynamic danger potential field, and the time proximity parameter includes a duration parameter, a time sequence parameter and a cross-danger-source switching interval;

[0033] In the overlapping area of the weighted influence domain of the danger source, the dominant danger source is determined according to the danger source influence factor and the environmental propagation condition, and the spatial proximity parameter and the time proximity parameter are attributedly marked.

[0034] Specifically, the causal reasoning is performed along a multi-hop path from a danger source node to a personnel node based on the multi-modal knowledge graph, and the causal reasoning includes:

[0035] In the multi-modal knowledge graph, a candidate path set is generated along a multi-hop connection meeting a time sequence constraint based on timestamp information of a personnel node and a trigger time of a danger source node;

[0036] The candidate path set is subjected to node type sequence matching, the danger source node, the environment node, the chemical equipment node and the personnel node are arranged in a propagation chain order, and a path that does not meet an accessibility constraint is eliminated, the accessibility constraint being a personnel accessible path set;

[0037] In combination with a danger source influence factor and an environmental propagation condition, a risk transfer weight is assigned to each edge in the path, and adjustment is made when a path branch, an influence domain overlap or missing data is detected;

[0038] Based on a time window convergence rule, the candidate path set is screened to retain a complete transfer chain path from the danger source node to the personnel node within a preset time period;

[0039] The final risk transfer weight of each edge in the screened path, the node time and the corresponding danger source are stored in the multi-modal knowledge graph, and a causal chain record corresponding to the personnel motion trajectory is generated.

[0040] Specifically, based on the time window convergence rule, the candidate path set is screened to retain a complete transfer chain path from the danger source node to the personnel node within a preset time period, and the screening includes:

[0041] A time window is set with the danger source node trigger time as a starting point, and the timestamps of each node and edge in the candidate path set are aligned to a unified time axis, and nodes and edges not within the time window are eliminated;

[0042] The paths in the candidate path set are subjected to sequence and direction consistency and continuity verification, and a gap node is supplemented at a gap according to the personnel accessible path set and the environmental parameters, and a path that does not meet the verification condition is eliminated;

[0043] Check whether the paths in the candidate path set contain a preset node type combination, eliminate paths missing any key nodes including hazard source nodes, environment nodes, chemical equipment nodes and personnel nodes, and sort the paths in the same time window;

[0044] Merge all the eliminated paths, and retain a complete transmission chain that meets the convergence condition.

[0045] Specifically, when the reasoning result indicates that the personnel gathering behavior and the physical inducement have convergence, a warning signal is generated, comprising:

[0046] Based on the causal chain record in the multi-modal knowledge graph, the path related to personnel gathering is checked for convergence;

[0047] When the convergence check is passed, the risk factors are extracted in chronological order from the causal chain record, and an ordered risk factor sequence is formed;

[0048] A time convergence parameter is generated for each risk factor in the ordered risk factor sequence;

[0049] Based on the ordered risk factor sequence and the time convergence parameter, a warning signal is constructed.

[0050] The multi-modal hazard identification monitoring and early warning system is used to realize the multi-modal hazard identification monitoring and early warning method, comprising a data acquisition module, a three-dimensional field production module, a causal reasoning module and an early warning module;

[0051] The data acquisition module is used to acquire relevant data of the chemical production area, including chemical production related data and personnel related data;

[0052] The three-dimensional field production module is used to determine whether there is a physical inducement that causes personnel injury based on a preset process safety threshold value, and if so, generate a three-dimensional dynamic dangerous potential field based on the chemical production related data;

[0053] The causal reasoning module is used to generate personnel movement trajectories based on personnel related data, map the personnel movement trajectories to the three-dimensional dynamic dangerous potential field, generate a multi-modal knowledge graph, and perform causal reasoning on the paths in the multi-modal knowledge graph;

[0054] The early warning module is used to generate a warning signal when the reasoning result indicates that the personnel gathering behavior and the physical inducement have convergence, and output the warning signal.

[0055] Specifically, the causal reasoning module comprises a parameter calculation unit, a graph construction unit and a causal reasoning unit;

[0056] The parameter calculation unit is configured to calculate a spatial proximity parameter and a time proximity parameter of a personnel motion trajectory and a weighted influence domain of a hazard source in a process in which the personnel motion trajectory is mapped to the three-dimensional dynamic dangerous potential field.

[0057] The graph construction unit is configured to construct a multi-modal knowledge graph comprising personnel nodes, chemical equipment nodes, hazard source nodes and environment nodes.

[0058] The causal reasoning unit is configured to perform causal reasoning along a multi-hop path from a hazard source node to a personnel node based on the multi-modal knowledge graph.

[0059] Compared with the prior art, the present application has the following beneficial effects:

[0060] The present application proposes a multi-modal based hidden danger identification monitoring and early warning method and system, judges physical inducements based on collected multi-modal data and process safety critical values, constructs a three-dimensional dynamic dangerous potential field, then maps a personnel motion trajectory into the potential field, constructs a multi-modal knowledge graph, and performs causal reasoning along a time evolution path, and generates an early warning signal according to the causal reasoning result; the binding of the hazard source and the personnel behavior trajectory is realized, so that the early warning is no longer dependent on a single index, but is based on the dynamic association and evolution prediction of multi-dimensional information, and a traceable causal chain can be generated when the hidden danger is still in the initial stage, thereby significantly improving the accuracy and advance nature of the early warning, reducing the false positive rate and the false negative rate, and providing accurate decision basis and visual risk path for emergency disposal. BRIEF DESCRIPTION OF DRAWINGS

[0061] Figure 1 A multi-modal based hidden danger identification monitoring and early warning method flowchart is provided for the present application.

[0062] Figure 2 A three-dimensional dynamic dangerous potential field schematic diagram is provided for the present application.

[0063] Figure 3 A multi-modal knowledge picture reasoning schematic diagram is provided for the present application.

[0064] Figure 4 A multi-modal based hidden danger identification monitoring and early warning system architecture diagram is provided for the present application. DETAILED DESCRIPTION

[0065] The present application will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any form. It should be noted that, for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made. These all belong to the protection scope of the present application.

[0066] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0067] It should be noted that the various features of the embodiments of the present application can be combined with each other if there is no conflict, and all are within the protection scope of the present application. In addition, although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from the module division in the device or the order in the flowchart. In addition, the "first", "second", "third" and the like used in the present application do not limit the data and execution order, but only distinguish the same items or similar items with basically the same function and effect.

[0068] Unless otherwise defined, all technical and scientific terms used in the present application have the same meaning as understood by a person skilled in the art to which the present application belongs. The terms used in the present application are only for the purpose of describing the specific embodiments of the present application and are not used to limit the present application. The term "and / or" used in the present application includes any and all combinations of one or more related listed items.

[0069] Embodiment 1:

[0070] Please refer to Figure 1 The present application provides an embodiment: a hidden danger identification, monitoring and early warning method based on multi-modal, comprising the following specific steps:

[0071] Step S1: collecting related data of the chemical production area, including chemical production related data and personnel related data.

[0072] In the present embodiment, the chemical production related data includes chemical device running parameters (temperature, pressure, flow, speed, etc.), dangerous material storage state, equipment running condition, gas concentration distribution and change trend, environmental parameters (wind direction, wind speed, temperature and humidity, air pressure) and the like; the personnel related data includes monitoring video, access control record data, wireless communication data, positioning data and the like.

[0073] Step S2: judging whether there is a physical inducement that can cause personnel injury based on the preset process safety critical value, and if there is, generating a three-dimensional dynamic danger potential field based on the chemical production related data.

[0074] The specific steps of step S2 are:

[0075] Step S201: performing time synchronization processing on the chemical production related data.

[0076] Step S202: Compare the processed chemical production related data with the preset process safety threshold value to determine whether there is a temperature, pressure, concentration or environmental condition abnormality reaching the preset process safety threshold value physical inducement.

[0077] In this embodiment, the process safety threshold value can be set according to different device types, material properties and industry safety standards; the determination process can be performed one by one according to the process unit, for example, the temperature parameter is compared with the set maximum safety temperature, and the environmental condition is combined with the wind direction and gas diffusion condition to determine whether the leakage source will directly act on the personnel accessible range under the current wind direction; when the comparison result triggers the physical inducement, the inducement information and the device location, material properties and current environmental condition are recorded together.

[0078] Step S203: If there is a physical inducement, obtain the dangerous source position, dangerous substance type and environmental propagation condition corresponding to the physical inducement, and combine them with the processed chemical production related data.

[0079] In this embodiment, the spatial position data corresponding to the physical inducement is retrieved through the dangerous source database or the dangerous source identifier, including device coordinates, production unit and relative position information with the surrounding area; according to the type and trigger process of the physical inducement, the dangerous substance types involved are queried, which are derived from the process bill of materials (BOM), chemical safety technical specification (MSDS) and process flow diagram, including substance name, main component, dangerous characteristic (flammability, explosiveness, toxicity, corrosiveness) and the like; the environmental propagation condition includes extraction of current and predicted wind direction, wind speed, temperature and humidity, atmospheric stability and other environmental parameters; after the environmental propagation condition is combined with the dangerous source position and the dangerous substance type, a set of data sets representing the risk diffusion trend is formed, which is combined with the processed chemical production related data to obtain the combined data.

[0080] Step S204: Based on the combined data, generate a three-dimensional dynamic dangerous potential field according to the dangerous source influence factor, environmental propagation direction and personnel accessible path.

[0081] As shown in Figure 2 , the specific steps of step S204 are:

[0082] Step S2041: Determine the dangerous source influence factor based on the dangerous source position and the dangerous substance type.

[0083] In the embodiment, according to the position of the hazard source, the production unit where the hazard source is located, the density of surrounding equipment, the spatial relationship with the personnel accessible area, and the relative position with other hazard sources are determined to construct the physical action range of the hazard source from the spatial distribution aspect; by weighting and comprehensively analyzing the physical action range of the hazard source, the hazard substance type characteristics, and the environmental propagation condition, a quantitative index capable of representing the potential influence ability of the hazard source on the surrounding area at the current moment, i.e., the hazard source influence factor, is obtained.

[0084] Step S2042: combining the hazard source influence factor with real-time environmental parameters to obtain the risk intensity coefficient of each hazard source under the current condition.

[0085] In the embodiment, the risk intensity coefficient of each hazard source under the current moment and the current environmental condition is obtained through weighting and fusion based on the physical coupling relationship between each environmental parameter and the hazard source characteristics.

[0086] Step S2043: based on the risk intensity coefficient, the influence range centered on the hazard source in the three-dimensional space of the chemical production is weighted and expanded or contracted to form the weighted influence domain of each hazard source.

[0087] In the embodiment, the basic influence range is weighted and expanded or contracted according to the size and nature of the risk intensity coefficient; the expansion process is suitable for the case where the risk intensity coefficient is high, the influence radius is increased in a specific direction outside the three-dimensional spherical or polyhedral region centered on the hazard source, and the influence domain presents a biased form according to the environmental propagation condition (such as wind direction and wind speed); the contraction process is suitable for the case where the risk intensity coefficient is low, the influence radius in each direction is reduced in proportion, and the extension amount in a specific direction is weakened to obtain the weighted influence domain of each hazard source.

[0088] Step S2044: superimposing the weighted influence domain with the actual permitted area and the prohibited area to obtain the personnel accessible path set.

[0089] In the embodiment, the weighted influence domain is accurately positioned in the three-dimensional space of the chemical production, the intersection relationship between the weighted influence domain and the actual permitted area and the prohibited area is calculated by spatial superposition, the part where the weighted influence domain overlaps with the prohibited area is directly marked as impassable, and the part where the weighted influence domain overlaps with the permitted area is further combined with the plant passage network data and the accessibility graph to extract the accessible path connecting each permitted area and the current position of the personnel to obtain the personnel accessible path set.

[0090] Step S2045: Assign a corresponding risk weight to each personnel accessible path in the set of personnel accessible paths, and generate a three-dimensional dynamic danger potential field in the three-dimensional space of chemical production.

[0091] In this embodiment, for each personnel accessible path, the comprehensive risk value of the path is calculated in combination with factors such as the overlapping degree of the weighted influence domain passed by the path, the total length of the path, the number and type of dangerous sources in the region passed by the path, and real-time environmental conditions; the risk weight is formed by quantifying the influence degree of various risk factors, and then accumulating the weights according to the distribution density and action time in the whole path to reflect the overall risk level of the path.

[0092] Subsequently, the spatial trajectories of all paths and their corresponding risk weights are mapped into the three-dimensional space of chemical production, and the corresponding risk energy distribution is generated in the space with the path as the skeleton to form a three-dimensional dynamic danger potential field.

[0093] Specifically, in Figure 2 , the three-dimensional space coordinate system is established with the dangerous source (plant device) as the center, the weighted influence domain is calculated through the dangerous source influence factor and real-time environmental parameters, such as Figure 2 weighted influence domain 1 in Figure 2 , which dynamically expands or shrinks according to temperature, pressure, wind direction and other environmental conditions, such as Figure 2 weighted influence domain 2 in Figure 2 , the personnel accessible path is obtained according to the actual permitted area (gray area in the figure, color is lighter), prohibited area (external white area) and culling area (area indicated by the straight line), which is used to limit the movable range of personnel in the three-dimensional space, and the personnel nodes numbered 1 to 4 in the figure are distributed along these accessible paths, and the unnumbered personnel node points to other personnel, it should be noted that Figure 2 only part of the path is shown.

[0094] Step S3: generating personnel motion trajectories based on personnel related data, mapping the personnel motion trajectories to the three-dimensional dynamic danger potential field, generating a multi-modal knowledge graph, and performing causal reasoning on the paths in the multi-modal knowledge graph, the causal reasoning is a joint simulation of the evolution of the three-dimensional dynamic danger potential field and the personnel motion trajectories in a future preset time period, the correlation strength between the personnel gathering behavior and the physical inducement is determined by dynamically updating the risk transmission weight, filtering the convergent path.

[0095] The specific steps of step S3 are:

[0096] Step S301: time synchronization and spatial alignment of personnel related data to obtain personnel position sequence in the three-dimensional space of chemical production.

[0097] In the embodiment, the personnel-related data of different sources are time-stamped and uniformly processed, the personnel position data is coordinate-system unified and error-corrected in the spatial dimension, and through the above time synchronization and spatial alignment process, the personnel position sequence in the chemical production three-dimensional space under the unified time reference and unified spatial reference is obtained.

[0098] Step S302: Based on the personnel position sequence, the personnel motion trajectory of each personnel is generated by combining the personnel-related data collection time interval and the personnel identity identifier, and the time stamp and position accuracy level of each position point are marked in the personnel motion trajectory.

[0099] In the embodiment, the personnel identity identifier is used as an index to group the personnel position sequence, so that all position points under the same identity identifier are arranged in chronological order. For the continuously collected position points, the moving order and distance between the position points are determined by calculating the interval between adjacent time stamps, forming a continuous motion trajectory on the time axis. In the process of generating the trajectory, two types of metadata, time stamp and position accuracy level, are attached to each position point. Finally, the motion trajectory of each personnel in the chemical production three-dimensional space is generated.

[0100] Step S303: Map the personnel motion trajectory to the three-dimensional dynamic danger potential field, and calculate the spatial proximity parameter and the time proximity parameter of the weighted influence domain of the personnel motion trajectory and the danger source in the mapping process.

[0101] The specific steps of step S303 are:

[0102] Step S3031: Align the position points of the personnel motion trajectory with the reference coordinate system of the three-dimensional dynamic danger potential field, and generate the corresponding uncertainty envelope according to the position accuracy level, and update the time sequence according to the three-dimensional dynamic danger potential field, to establish the correspondence between the trajectory time stamp and the version of the three-dimensional dynamic danger potential field.

[0103] In the embodiment, through the established coordinate conversion model, the position points of the personnel motion trajectory are projected from their original coordinate system to the unified spatial reference frame of the three-dimensional dynamic danger potential field for position alignment. After the position alignment is completed, the corresponding uncertainty envelope is generated according to the accuracy level of each position point. The uncertainty envelope is realized by constructing a spatial area around the three-dimensional coordinates with the position accuracy level as the parameter. High-precision points correspond to smaller uncertainty areas, and low-precision points correspond to larger envelope areas.

[0104] Finally, based on the time sequence of the three-dimensional dynamic danger potential field, the timestamp of each position point in the personnel motion trajectory is associated with the corresponding time version of the three-dimensional dynamic danger potential field. Specifically, by establishing a time matching index table, each sampling point on the trajectory time axis is bound to the version of the three-dimensional dynamic danger potential field generated within the same time period.

[0105] Step S3032: The missing position of the personnel motion trajectory is constrained and completed, and the coverage relationship between the personnel motion trajectory and the weighted influence domain of the danger source is determined, and the entering event, the crossing event and the leaving event are marked.

[0106] In the embodiment, by analyzing the time interval, spatial distribution and personnel travel speed range of adjacent known position points, the possible position of the missing point is inferred by using the motion continuity constraint and the passing topological constraint, for example, only allowing to pass through the known channel, access control or stair position; during the completion process, the candidate path is screened by combining the personnel historical trajectory pattern and the trajectory data of similar role personnel within the same time period, and the completion result most consistent with the spatial and time constraints is determined.

[0107] After the trajectory completion, the modified personnel motion trajectory is spatially superimposed and analyzed with the weighted influence domain of the danger source, and the coverage relationship between the trajectory point and the weighted influence region of the danger source is determined. Specifically, by scanning the relative position of the trajectory point and the boundary of the weighted influence domain at each time, the spatiotemporal nodes of the personnel position entering the domain from outside the domain (entering event), continuously moving in the domain and crossing different risk intensity intervals (crossing event), and moving from the domain to outside the domain (leaving event) are identified, and the corresponding timestamp and position coordinates are recorded for each type of event.

[0108] Step S3033: The personnel motion trajectory is segmented according to the event markers, and the spatial proximity parameters of each mapping segment are calculated, including the distance parameter, the direction consistency parameter and the path fitting degree parameter.

[0109] In the embodiment, the event markers are used as the dividing points to form a plurality of trajectory sequences arranged in time sequence. After segmentation, the spatial proximity parameters of each mapping segment are calculated. The distance parameter is obtained by calculating the shortest spatial distance sequence of the trajectory point to the boundary or center of the danger source and taking the statistical quantity; the direction consistency parameter is quantified according to the angle change trend between the trajectory motion direction vector and the direction vector pointing to the center of the danger source, reflecting the dynamic characteristics of the personnel motion approaching or moving away from the danger source; the path fitting degree parameter is obtained by comparing the geometric similarity of the trajectory curve and the weighted influence domain boundary or potential field contour of the danger source, quantifying the fitting degree of the trajectory in the spatial form and the risk region profile.

[0110] Step S3034: According to the time sequence of the three-dimensional dynamic dangerous potential field, the time proximity parameters of each mapping segment are calculated, including the duration parameter, the time sequence parameter and the cross-dangerous-source switching interval.

[0111] In this embodiment, the start and end times of the trajectory segment are compared with the effective times of each version of the three-dimensional dynamic dangerous potential field to determine the version interval of the three-dimensional dynamic dangerous potential field corresponding to the segment. After alignment on the time axis, the time proximity parameters of each mapping segment are calculated. The duration parameter can be calculated by the difference between the start and end times of the segment, reflecting the length of time the person stays under a specific potential field condition. The time sequence parameter is determined by analyzing the time sequence of each mapping segment relative to the change nodes of the three-dimensional dynamic dangerous potential field, describing the sequence between the person's movement trajectory and the change of risk situation. The cross-dangerous-source switching interval is calculated according to the dangerous source identifiers involved in different segments and their start and end times, which is the time interval for the person to switch from one dangerous source weighted influence domain to another.

[0112] Step S3035: In the overlapping area of the weighted influence domains of the dangerous sources, the dominant dangerous source is determined according to the dangerous source influence factor and the environmental propagation condition, and the spatial proximity parameter and the time proximity parameter are attributed to the dominant dangerous source.

[0113] In this embodiment, first, the overlapping spatial area of the weighted influence domains of the dangerous sources is identified in the three-dimensional dynamic dangerous potential field. The identification process is completed by analyzing the boundary coordinate relationship of the weighted influence domains of different dangerous sources to determine the range of the area where multiple-source risk overlaps. After determining the overlapping area, the risk contribution value of each dangerous source in the overlapping area is calculated based on the dangerous source influence factor corresponding to each dangerous source and the current environmental propagation condition. The dangerous source influence factor can be composed of parameters such as the physical and chemical properties of dangerous substances and energy release characteristics. The environmental propagation condition can include elements such as wind direction, humidity, temperature, and airflow channel distribution. By comparing the risk contribution values, the dangerous source with the highest comprehensive risk impact is selected as the dominant dangerous source in the area.

[0114] After the dominant dangerous source is determined, the spatial proximity parameter and the time proximity parameter in the area are attributed to the dominant dangerous source, ensuring that in the subsequent causal chain reasoning, the trajectory data related to the area can accurately reflect the main risk source. This attribution process makes the risk analysis in the multi-dangerous-source overlapping scenario unique and interpretable, avoiding reasoning bias caused by ambiguous risk attribution.

[0115] Step S304: A multi-modal knowledge graph containing personnel nodes, chemical equipment nodes, dangerous source nodes and environment nodes is constructed, with personnel, dangerous sources, environmental parameters, spatial proximity parameters and time proximity parameters as attributes of nodes or edges.

[0116] In this embodiment, based on the information of the personnel nodes, chemical equipment nodes, hazard source nodes and environment nodes obtained in the preceding steps, a unified representation form of multi-modal data is established, specifically, data of different sources are classified according to node categories, and a unique identifier is introduced in the node data set; then, the relationship between personnel and hazard sources, environmental parameters and chemical equipment is abstracted as edges in the graph, and attributes such as spatial proximity parameter and time proximity parameter are attached to the edges, the spatial proximity parameter is calculated from the geometric relationship between the personnel motion trajectory and the weighted influence domain of the hazard source, and the time proximity parameter comes from the correspondence between the trajectory timestamp and the hazard event occurrence time.

[0117] Finally, according to the modeling rules of the multi-modal knowledge graph, the above nodes and edges are integrated into a heterogeneous network structure, so that the personnel nodes can be associated with the chemical equipment nodes, hazard source nodes and environment nodes in multiple dimensions, forming a multi-modal knowledge graph that can support subsequent causal reasoning and risk chain analysis.

[0118] Step S305: based on the multi-modal knowledge graph, performing causal reasoning along the multi-hop path from the hazard source node to the personnel node.

[0119] The specific steps of step S305 are:

[0120] Step S3051: in the multi-modal knowledge graph, based on the timestamp information of the personnel node and the trigger time of the hazard source node, a candidate path set is generated along the multi-hop connection that meets the time sequence constraint.

[0121] In this embodiment, first, the time information related to the personnel node and the hazard source node is extracted in the multi-modal knowledge graph, wherein the personnel node contains a timestamp reflecting the time when its position is collected, and the hazard source node contains time information when the corresponding physical cause is determined or triggered; specifically, the timestamps of all personnel nodes are sorted in chronological order, and the trigger time of the hazard source node is mapped to the same time axis to realize the time consistency constraint in the path generation process; then, according to the time sequence relationship, the node connections in the multi-modal knowledge graph are screened, the edges that do not meet the time causal logic are removed, and only the effective connections whose timestamp is earlier than the target event are retained, and then by traversing the multi-hop connection path that meets the time constraint, starting from the personnel node, passing through the intermediate nodes such as chemical equipment nodes and environment nodes, and finally reaching the corresponding hazard source node.

[0122] Finally, the multi-hop paths that meet the above time constraint conditions are recorded as a candidate path set, and a corresponding time sequence identifier is attached to each candidate path to ensure that the order and potential relevance of events can be accurately judged based on the time chain in the causal reasoning process.

[0123] Step S3052: Perform node type sequence matching on the candidate path set, arrange the hazard source nodes, environment nodes, chemical equipment nodes and personnel nodes in the order of the propagation chain, and eliminate paths that do not meet the reachability constraint, the reachability constraint being the personnel reachable path set.

[0124] In this embodiment, for each path in the candidate path set, the hazard source nodes, environment nodes, chemical equipment nodes and personnel nodes are rearranged in the logical order of event propagation according to the type information of the nodes in the multi-modal knowledge graph, so that the path structure can accurately reflect the potential transmission chain from the risk source to the personnel. Specifically, by reading the type label of each node and the position relationship in the path, a node type sequence is generated. After obtaining the node type sequence, the candidate path set is checked for reachability constraints. The reachability constraint is to determine whether the spatial connection between the personnel nodes and other nodes involved in the path is within the scope of the given personnel reachable path set, i.e. only paths that have physical reachability under three-dimensional space and traffic conditions are retained. The spatial position information between path nodes is matched with the personnel reachable path set segment by segment. If there is no reachable connection between any pair of nodes in the path, it is determined that the path does not meet the constraint and is eliminated.

[0125] Step S3053: Assign risk transmission weights to each edge in the path by combining the hazard source influence factor and the environmental propagation condition, and adjust when path branching, influence domain overlap or missing data are detected.

[0126] In this embodiment, first, each edge of the valid path in the multi-modal knowledge graph is associated with the corresponding hazard source influence factor. According to the physical properties, energy level and potential harm degree of the hazard source in the current environment, the initial risk transmission weight of the edge is calculated. After generating the initial risk transmission weight, the path structure is analyzed to detect whether there is path branching, influence domain overlap or data missing. When path branching is detected, the risk transmission weights of the edges from the branching node to the end of each branch are assigned or normalized according to the relative risk contribution proportion of the branching node to each branch. When influence domain overlap is detected, the edge weights involved are enhanced or weakened according to the influence factor of the dominant hazard source in the overlap region. If there is missing data, the missing part is completed by historical records, adjacent time series data or spatial interpolation method, and the risk transmission weight of the corresponding edge is adjusted according to the completion result to ensure the continuity and integrity of the risk propagation calculation of the overall path.

[0127] Step S3054: Based on the time window convergence rule, the candidate path set is screened to retain the complete transmission chain path from the hazard source node to the personnel node within the preset time period.

[0128] The specific steps of step S3054 are:

[0129] Step S30541: Set a time window with the dangerous source node triggering time as the starting point, and align the time stamps of each node and edge in the candidate path set to a unified time axis, and remove nodes and edges that are not within the time window.

[0130] In this embodiment, first, the triggering time of the dangerous source node is taken as a unified reference point to determine the start and end range of the time window, and the span of the time window is set according to the preset event analysis requirement or the time scale of the propagation chain; then, the time stamp data of each node and the edge connected thereto in the candidate path set is calculated according to the relative time offset of the reference point, so that all time information is mapped to a unified time axis; after the time axis alignment is completed, the time stamp of each node and edge is windowed, if the time stamp falls outside the set time window, the corresponding node and its associated edge are removed from the candidate path set; if the node is within the time window but the time interval with the adjacent node exceeds the allowed threshold, the associated edge is marked for processing.

[0131] Step S30542: The paths in the candidate path set are checked for sequence and direction consistency and continuity, and the missing nodes are supplemented at the gaps according to the personnel accessible path set and environmental parameters, and the paths that do not meet the checking conditions are removed.

[0132] In this embodiment, the direction consistency of the node sequence of each path in the candidate path set is checked, that is, according to the chronological relationship of the time stamps and the spatial connectivity direction between the nodes, it is judged whether the arrangement of the path conforms to the logical direction of the propagation from the dangerous source node to the personnel node, and at the same time, it is detected whether the node sequence has abnormal arrangement in reverse or cycle, for continuity checking, the time interval and spatial distance between adjacent nodes are compared to confirm the continuity of the path in time and space, if the interval exceeds the set threshold or a unreachable breakpoint appears, it is marked as having a gap.

[0133] When a gap is detected, the feasible path information in the personnel accessible path set is used, combined with the environmental parameters at the gap position, including temperature, humidity, wind direction, passage state and other factors, to select the nodes that meet the accessibility and environmental condition constraints from the candidate accessible path as the gap completion nodes and insert them into the original path, thereby restoring the integrity of the path; for the gap that cannot be completed by the personnel accessible path set and environmental parameters, or the path that still has inconsistent direction, incorrect sequence or insufficient continuity after completion, the path is directly removed from the candidate path set.

[0134] Step S30543: Check whether the paths in the candidate path set contain the preset node type combination, eliminate paths missing any key nodes including the hazard source node, the environment node, the chemical equipment node, and the personnel node, and sort the paths in the same time window.

[0135] In this embodiment, node type combination checking is performed for each path in the candidate path set, that is, whether the hazard source node, the environment node, the chemical equipment node, and the personnel node are simultaneously contained in the path is verified in sequence according to the preset key node set. The verification process realizes automatic determination of missing nodes by performing type analysis on the path node sequence and set inclusion operation with the key node set. Once any key node is detected to be missing in the path, the path is directly marked as incomplete and eliminated from the candidate path set.

[0136] After completing the node integrity screening, the retained paths are grouped according to the time window, which is determined by the hazard source node triggering time and a preset time length after the triggering time. In the same time window, the paths are sorted in chronological order by comparing the timestamp sequence of the paths, the time interval between the nodes, and the triggering time of the start and end events. When the time parameters are the same, the relative order of the key nodes in the path or the spatial distance between the nodes can be further used for secondary sorting to form an ordered path sequence in the time window.

[0137] Step S30544: Merge all the eliminated paths, and retain one complete transmission chain that satisfies the convergence condition.

[0138] In this embodiment, for multiple paths in the same time window, which have the same start and end node types and the same key node sequence, highly similar paths in spatial position and time sequence are identified by comparing the spatial proximity parameter and the time proximity parameter, and are regarded as potential convergent path groups. In the merging process, the position information and the time information of the non-key nodes in each path are sequentially superimposed to form a merged candidate path, with the key node sequence as the skeleton.

[0139] Step S3055: Store the final risk transmission weight of each edge in the screened path, the node time, and the corresponding hazard source in the multi-modal knowledge graph, and generate a cause-and-effect chain record corresponding to the personnel motion trajectory.

[0140] In this embodiment, according to the dangerous source node, the environment node, the chemical equipment node and the personnel node associated with each edge, the risk transmission weight is bound with the node occurrence time, and the risk transmission data is formed together with the start and end node identification of the edge; when stored in the multi-modal knowledge graph, the above risk transmission data is mounted according to the node and edge attribute rules of the graph database, wherein the attributes of the edge include risk transmission weight, trigger time, associated dangerous source ID, etc.; the attributes of the node include timestamp, spatial coordinates, type label, etc., forming a causal chain consistent with the original path in the multi-modal knowledge graph, which is bound with complete time and space information; finally, combined with the stored personnel motion trajectory data, the corresponding risk transmission path is associated to generate a causal chain record.

[0141] As shown in Figure 3 In the chemical production scene, there are dangerous source nodes, chemical equipment nodes, environment nodes and multiple personnel nodes, and different nodes are connected by directed lines to represent information and risk transmission direction.

[0142] Specifically, the dangerous source node is directly connected with the chemical equipment node and the environment node, representing the diversity of risk influence paths; the environment node is connected to the personnel node through different paths, embodying the multi-hop role of environmental factors in risk propagation; there are multiple paths between the chemical equipment node and the personnel node, and the path is marked with time sequence constraints and risk transmission weight and other information attributes, to express the time sequence relationship and risk intensity quantization method in the risk propagation process.

[0143] Step S4: When the reasoning result shows that the personnel gathering behavior converges with the physical inducement, a warning signal is generated, and the warning signal is output.

[0144] The specific steps of step S4 are:

[0145] Step S401: Based on the causal chain record in the multi-modal knowledge graph, the convergence of the path related to personnel gathering is verified.

[0146] In this embodiment, by setting the personnel quantity threshold, the gathering duration threshold and the spatial radius limit, the causal chain path meeting the conditions is extracted, and the dangerous source node and environment node information involved are associated, so as to obtain a set of potential gathering paths.

[0147] The principle of convergence verification is to judge the convergence degree of multiple paths in time and space. Specifically, the last several key nodes in the path are taken as the aggregation target, the intersection time on the time axis and the coincidence proportion of the spatial position of different paths are analyzed, when multiple paths point to the same or adjacent spatial region within the set time window, and show consistency in the node type sequence, it is determined that these paths converge in behavior trend.

[0148] Step S402: When passing through the convergence verification, the risk factors are extracted in time sequence from the causal chain record, and an ordered risk factor sequence is formed.

[0149] In this embodiment, for the causal chain record that has passed the convergence verification, the risk factors associated with each node are extracted in the time stamp order of the nodes in the link, and the sources of the risk factors include the hazard source influence factor, the environmental transmission condition, the equipment operation state and the personnel behavior characteristics, etc. The time sequence is determined from the earliest node trigger time in the causal chain as the starting point, and is arranged in sequence along the time axis. If multiple risk factors exist at the same time, they are sorted according to the node type priority or the risk weight, forming a non-repeated and ordered risk factor sequence.

[0150] Step S403: Generating a time convergence parameter for each risk factor in the ordered risk factor sequence.

[0151] In this embodiment, the trigger time difference of adjacent risk factors is first extracted, and the time window is set to evaluate the closeness of the risk factors before and after the risk factors in time, i.e. the time convergence parameter.

[0152] Step S404: Constructing an early warning signal based on the ordered risk factor sequence and the time convergence parameter.

[0153] In this embodiment, by traversing the ordered risk factor sequence, the time convergence comprehensive value between adjacent or continuous multiple risk factors is calculated, and the associated edge weight of each risk factor in the multi-modal knowledge graph is combined to judge the probability that the group of risk factors may trigger a linkage effect in a short time. When the comprehensive value exceeds the preset trigger threshold, the group of risk factors is marked as a potential high-risk trigger chain. In the screening process of the potential high-risk trigger chain, the environmental dynamic parameter and the hazard source state change are introduced as auxiliary judgment conditions, and finally a high-risk trigger chain is obtained to form an early warning signal.

[0154] Embodiment 2:

[0155] Please refer to Figure 4 Another embodiment provided by the present application is a multi-modal based hidden danger identification monitoring and early warning system, which comprises a data acquisition module, a three-dimensional field production module, a causal reasoning module and an early warning module.

[0156] The data acquisition module is used for acquiring relevant data of the chemical production area, including chemical production related data and personnel related data.

[0157] The three-dimensional field production module is configured to determine whether a physical inducement that can cause personnel injury exists based on a preset process safety threshold value, and if so, generate a three-dimensional dynamic danger potential field based on chemical production related data.

[0158] The cause-effect reasoning module is configured to generate a personnel motion trajectory based on personnel related data, map the personnel motion trajectory to the three-dimensional dynamic danger potential field, generate a multi-modal knowledge graph, and perform cause-effect reasoning on a path in the multi-modal knowledge graph.

[0159] The warning module is configured to generate a warning signal and output the warning signal when the reasoning result indicates that the personnel gathering behavior converges with the physical inducement.

[0160] The cause-effect reasoning module includes a parameter calculation unit, a graph construction unit, and a cause-effect reasoning unit.

[0161] The parameter calculation unit is configured to calculate a spatial proximity parameter and a time proximity parameter of the personnel motion trajectory and a weighted influence domain of the danger source in a process in which the personnel motion trajectory is mapped to the three-dimensional dynamic danger potential field.

[0162] The graph construction unit is configured to construct a multi-modal knowledge graph including a personnel node, a chemical equipment node, a danger source node, and an environment node.

[0163] The cause-effect reasoning unit is configured to perform cause-effect reasoning along a multi-hop path from the danger source node to the personnel node based on the multi-modal knowledge graph.

[0164] In addition, the part of the above technical solution in the embodiments of the present application that is consistent with the implementation principle of the corresponding technical solution in the prior art is not described in detail to avoid excessive repetition.

[0165] The specific embodiments described above further detail the purposes, technical solutions, and beneficial effects of the present application. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A multimodal hazard identification, monitoring, and early warning method, characterized in that, include: Collect relevant data from the chemical production area, including chemical production-related data and personnel-related data; Based on the preset process safety threshold, determine whether there are physical causes that could lead to personnel injury. If so, generate a three-dimensional dynamic hazard potential field based on relevant chemical production data. Based on personnel-related data, personnel movement trajectories are generated, and these trajectories are mapped to a three-dimensional dynamic hazard potential field to generate a multimodal knowledge graph. Causal reasoning is then performed on the paths in the multimodal knowledge graph. This causal reasoning is a joint simulation of the evolution of the three-dimensional dynamic hazard potential field and personnel movement trajectories within a preset future time period. By dynamically updating risk transmission weights and filtering convergence paths, the correlation strength between personnel gathering behavior and physical triggers is determined. When the reasoning results indicate that the gathering behavior of people is convergent with the physical cause, an early warning signal is generated and output. The process involves determining whether there are physical causes of injury based on preset process safety thresholds. If such causes exist, a three-dimensional dynamic hazard potential field is generated based on relevant chemical production data, including: Time synchronization processing of chemical production-related data; The processed chemical production-related data are compared with the preset process safety threshold to determine whether there are any physical causes that cause abnormal temperature, pressure, concentration, or environmental conditions to reach the preset process safety threshold. If a physical trigger exists, obtain the location of the hazard source, the type of hazardous substance, and the environmental propagation conditions corresponding to the physical trigger, and combine them with the processed chemical production-related data. Based on the combined data, a three-dimensional dynamic hazard potential field is generated according to the hazard source impact factors, environmental propagation direction and personnel access paths. The process of generating a three-dimensional dynamic hazard potential field based on combined data, according to hazard source impact factors, environmental propagation direction, and personnel access paths, includes: Based on the location of the hazard source and the types of hazardous substances, the hazard source impact factors are determined; By combining the hazard source impact factors with real-time environmental parameters, the risk intensity coefficient of each hazard source under the current conditions is obtained; Based on the aforementioned risk intensity coefficient, the influence range centered on the hazard source in the three-dimensional space of chemical production is weighted and expanded or contracted to form the weighted influence domain of each hazard source. The weighted influence domain is overlaid with the actual permitted and prohibited areas to obtain a set of accessible paths for personnel; Within the set of accessible paths for personnel, a corresponding risk weight is assigned to each accessible path, and a three-dimensional dynamic hazard potential field is generated in the three-dimensional space of chemical production. The process of generating personnel movement trajectories based on personnel-related data, mapping these trajectories to a three-dimensional dynamic hazard potential field, generating a multimodal knowledge graph, and performing causal reasoning on the paths in the multimodal knowledge graph includes: By synchronizing and aligning personnel-related data in time and space, a sequence of personnel locations in the three-dimensional space of chemical production is obtained. Based on the personnel location sequence, combined with the personnel-related data collection time interval and personnel identification, the personnel movement trajectory of each person is generated, and the timestamp and position accuracy level of each location point are marked in the personnel movement trajectory. The personnel movement trajectory is mapped to a three-dimensional dynamic hazard potential field, and the spatial proximity parameter and temporal proximity parameter between the personnel movement trajectory and the hazard source are calculated during the mapping process. Construct a multimodal knowledge graph that includes personnel nodes, chemical equipment nodes, hazard source nodes, and environmental nodes, using personnel, hazard source, environmental parameters, spatial proximity parameters, and temporal proximity parameters as attributes of nodes or edges; Based on the multimodal knowledge graph, causal reasoning is performed along a multi-hop path from the hazard source node to the personnel node.

2. The multimodal-based hazard identification, monitoring, and early warning method as described in claim 1, characterized in that, The process of mapping the personnel movement trajectory to a three-dimensional dynamic hazard potential field, and calculating the spatial proximity parameters and temporal proximity parameters between the personnel movement trajectory and the weighted influence domain of the hazard source during the mapping process, includes: Align the location points of the personnel movement trajectory with the reference coordinate system of the three-dimensional dynamic hazard potential field, generate the corresponding uncertainty envelope according to the position accuracy level, and update the time series according to the three-dimensional dynamic hazard potential field to establish the correspondence between the trajectory timestamp and the version of the three-dimensional dynamic hazard potential field. The missing locations of personnel movement trajectories are constrained and filled in, and the coverage relationship between the personnel movement trajectory and the weighted influence domain of the hazard source is determined, and entry events, crossing events and departure events are marked; The movement trajectory of personnel is segmented according to the event markers, and the spatial proximity parameters of each mapped segment are calculated. The spatial proximity parameters include distance parameters, directional consistency parameters, and path fit parameters. According to the update time order of the three-dimensional dynamic hazard potential field, the time proximity parameter of each mapping segment is calculated. The time proximity parameter includes the duration parameter, the time sequence parameter, and the cross-hazard source switching interval. In the overlapping area of ​​the weighted influence domain of the hazard source, the dominant hazard source is determined based on the hazard source influence factor and environmental propagation conditions, and the spatial proximity parameter and temporal proximity parameter are assigned and marked.

3. The multimodal-based hazard identification, monitoring, and early warning method as described in claim 2, characterized in that, The step of performing causal reasoning based on the multimodal knowledge graph along a multi-hop path from the hazard source node to the personnel node includes: In a multimodal knowledge graph, a set of candidate paths is generated based on the timestamp information of personnel nodes and the triggering time of hazard source nodes, along multi-hop connections that meet the time sequence constraints. The candidate path set is subjected to node type sequence matching. Hazard source nodes, environmental nodes, chemical equipment nodes, and personnel nodes are arranged in the order of propagation chain, and paths that do not meet the accessibility constraints are eliminated. The accessibility constraints are the set of personnel accessible paths. By combining the hazard source impact factors and environmental propagation conditions, risk transmission weights are assigned to each edge of the path, and adjustments are made when path branches, overlapping impact domains, or missing data are detected. Based on the convergence rule of time window, the candidate path set is filtered to retain the complete transmission chain path from the hazard source node to the personnel node within the preset time period; The final risk transmission weight, node time, and corresponding hazard source of each edge in the selected path are stored in the multimodal knowledge graph, and a causal chain record corresponding to the personnel movement trajectory is generated.

4. The multimodal-based hazard identification, monitoring, and early warning method as described in claim 3, characterized in that, The time window convergence rule is used to filter the candidate path set, retaining complete transmission chain paths from the hazard source node to the personnel node within a preset time period, including: Starting from the triggering time of the hazard source node, a time window is set, and the timestamps of each node and edge in the candidate path set are aligned to a unified time axis, and nodes and edges that are not within the time window are removed. The paths in the candidate path set are checked for consistency in order and direction and continuity. At the gaps, gap nodes are added based on the set of accessible paths for personnel and environmental parameters. Paths that do not meet the check conditions are eliminated. Check whether the paths in the candidate path set contain a preset combination of node types, remove paths that are missing any key node, and sort the paths within the same time window. The key nodes include hazard source nodes, environmental nodes, chemical equipment nodes, and personnel nodes. All paths that have been removed are merged, and a complete transit chain that satisfies the convergence condition is retained.

5. The multimodal-based hazard identification, monitoring, and early warning method as described in claim 4, characterized in that, When the reasoning results indicate that the gathering behavior and the physical trigger are convergent, an early warning signal is generated, including: Based on the causal chain records in the multimodal knowledge graph, the convergence of paths related to personnel gathering is verified. When passing the convergence check, risk factors are extracted from the causal chain record in chronological order and an ordered risk factor sequence is formed. Generate time convergence parameters for each risk factor in the ordered risk factor sequence; Early warning signals are constructed based on ordered risk factor sequences and time convergence parameters.

6. A multimodal hazard identification, monitoring, and early warning system, used to implement the multimodal hazard identification, monitoring, and early warning method according to any one of claims 1-5, characterized in that, include: Data acquisition module, 3D field production module, causal reasoning module, and early warning module; The data acquisition module is used to collect relevant data from the chemical production area, including chemical production-related data and personnel-related data. The three-dimensional field production module is used to determine whether there are physical causes that could lead to personal injury based on preset process safety thresholds. If so, it generates a three-dimensional dynamic hazard potential field based on relevant chemical production data. The causal reasoning module is used to generate personnel movement trajectories based on personnel-related data, map the personnel movement trajectories to a three-dimensional dynamic danger potential field, generate a multimodal knowledge graph, and perform causal reasoning on the paths in the multimodal knowledge graph. The early warning module is used to generate and output an early warning signal when the reasoning results indicate that the gathering behavior of people and the physical cause are convergent. The causal reasoning module includes: a parameter calculation unit, a graph construction unit, and a causal reasoning unit; The parameter calculation unit is used to calculate the spatial proximity parameter and temporal proximity parameter between the personnel movement trajectory and the hazard source in the process of mapping the personnel movement trajectory to the three-dimensional dynamic hazard potential field. The graph construction unit is used to construct a multimodal knowledge graph that includes personnel nodes, chemical equipment nodes, hazard source nodes, and environmental nodes. The causal reasoning unit performs causal reasoning along a multi-hop path from the hazard source node to the personnel node, based on the multimodal knowledge graph.

Images