Construction safety monitoring method and system based on multi-mode intelligent agent cooperation

Abstract

The invention relates to a construction safety monitoring method and system based on multi-mode intelligent agent cooperation, and belongs to the technical field of building engineering construction. The method comprises the steps of collecting multi-modal data and converting the multi-modal data into a unified data object; carrying out self-adaptive adjustment on the early warning threshold value T'i of the key monitoring index Mi; obtaining a normalized risk value Ri and a risk trend factor lambda i; calculating a comprehensive risk value DPRI, and determining an early warning level; calculating the contribution degree of each risk factor, and identifying a dominant risk factor; and generating an early warning response strategy. According to the invention, a collaborative analysis framework composed of a plurality of intelligent agents with independent functions is constructed, deep fusion and intelligent analysis of multi-source heterogeneous data are realized, a comprehensive risk value is calculated, risk tracing is carried out, potential safety risks are predicted in advance, and an early warning response strategy is generated. And the comprehensiveness of safety monitoring and the accuracy of early warning in a complex construction scene are improved.

Description

Technical Field

[0001] This invention belongs to the field of construction engineering technology, specifically relating to a construction safety monitoring method and system based on multimodal intelligent agent collaboration. Background Technology

[0002] In building construction, the operation of large facilities such as self-climbing formwork and tower cranes in high-rise buildings is dynamic and complex, and is a high-risk link for safety accidents. Improper operation or equipment abnormalities that are not detected in time may cause catastrophic accidents such as structural instability and falls, which seriously threaten the safety of people's lives and property.

[0003] Traditional safety management relies primarily on on-site inspections and experience-based judgment by safety officers, which generally suffers from blind spots in monitoring, delayed risk detection, and strong subjectivity. Currently, some technical solutions exist for monitoring complex construction scenarios. For example, there are IoT-based structural health monitoring systems that use sensors to collect stress, tilt, and vibration data at key locations on formwork or tower cranes. These sensors collect physical parameters in real time and compare them with preset thresholds to trigger alarms. This achieves quantitative monitoring of key indicators to some extent. However, these systems can only collect scattered physical parameters, creating "data silos" that make it difficult to comprehensively analyze complex risks caused by the coupling of multiple factors, and also fail to intuitively perceive the actual on-site work conditions behind the risks. There are also monitoring solutions using computer vision, which automatically identify violations such as personnel not wearing safety helmets or crossing boundaries through cameras and AI algorithms. However, these solutions lack in-depth perception of key physical quantities such as internal structural stress and equipment operating status, and are powerless against progressive and systemic risks.

[0004] The aforementioned existing technologies have improved the informatization level of safety monitoring to a certain extent, but still have the following core shortcomings: 1) Insufficient fusion of multi-source heterogeneous data, failing to effectively analyze complex risks caused by the coupling of multiple factors; 2) Simple risk assessment models, mostly relying on fixed static thresholds, lacking adaptability to dynamic construction conditions; 3) Monitoring mode is mainly "post-event response", lacking the ability to predict the development trend of risks; 4) Early warning information is "black box", unable to provide decision support and risk source explanation for managers; 5) Rigid system architecture, lacking collaborative analysis capabilities between intelligent agents.

[0005] Therefore, there is an urgent need for an intelligent monitoring system and method that can achieve deep fusion of multimodal data, and possess collaborative analysis, predictive assessment, and interpretability, so as to comprehensively improve the initiative and intelligence level of safety management in complex construction scenarios. Summary of the Invention

[0006] This invention provides a construction safety monitoring method and system based on multimodal intelligent agent collaboration, aiming to solve the problems of incomplete monitoring, untimely early warning, and insufficient decision support capabilities in complex construction scenarios in existing technologies.

[0007] To solve the above technical problems, the present invention includes the following technical solutions:

[0008] A construction safety monitoring method based on multimodal intelligent agent collaboration includes:

[0009] Step 1: Collect multimodal data from the construction site in real time and convert the multimodal data into a unified data object;

[0010] Step 2: Use a dynamic threshold adjustment model to adjust the key monitoring indicator M. i The warning threshold T' i Perform adaptive adjustments;

[0011] Step 3: Monitor key indicators M i Nonlinear risk normalization is performed to obtain the normalized risk value R. i And calculate the risk trend factor λ i ;

[0012] Step 4: Based on R i , λ i Calculate the comprehensive risk value DPRI and compare it with the preset multi-level thresholds to determine the warning level;

[0013] Step 5: Decompose the overall risk value (DPRI) into its components, calculate the contribution of each risk factor, and identify the dominant risk factor.

[0014] Step 6: Based on the determined warning level and dominant risk factors, generate a warning response strategy.

[0015] Furthermore, in step two, the dynamic threshold adjustment model is expressed as:

[0016]

[0017] Among them, T i C is the baseline threshold for index i. j For the j-th contextual factor, f j (·) is the adjustment function corresponding to the j-th factor, and its form is (1-k ij ·g j (C j )), where k ij To adjust the coefficient, g j (C j Factor C j The normalized representation of J is the total number of context factors involved in the dynamic threshold adjustment.

[0018] further,

[0019] Among them, M i T' is the real-time measured value of index i. i The dynamic threshold is adjusted in step two, and k is the steepness coefficient;

[0020]

[0021] in, The average rate of change of the indicator within the most recent time window. λ is the average value, s is the trend sensitivity coefficient; i >1 indicates worsening risk, λ i <1 indicates improved risk.

[0022] further,

[0023] Where, ω i ρ represents the base weight of the i-th risk indicator; ij is the coupling coefficient between risk indicators i and j, and N is the total number of key monitoring indicators participating in the comprehensive risk assessment.

[0024] Furthermore, in step three, the key monitoring indicator M... i Nonlinear risk normalization is performed through a multi-agent collaborative analysis module. This module employs a collaborative mechanism based on confidence negotiation and dynamic task grouping, and includes the following sub-steps:

[0025] Define each analytical agent A i In outputting its analysis results Res i At the same time, a confidence score quantifying the certainty of its judgment must be output simultaneously. i Conf i ∈[0,1];

[0026] Set a confidence negotiation threshold θ n When any intelligent agent A i Output confidence Conf i <θ n In this case, the result will not directly enter the risk assessment process, but will instead trigger a "negotiation request," which is broadcast to a predefined set of agents G related to the task. i ;

[0027] set G i After receiving the request, other agents in the system perform cross-validation;

[0028] Whether this risk item is confirmed depends on a fusion confidence level. f This confidence level is calculated by weighting the initial confidence level with the confidence levels of all validation feedbacks. Only when the confidence level is reached... f Exceeding a higher confirmation threshold θ f Only then is the risk item formally adopted.

[0029] Furthermore, in step three, the key monitoring indicator M... i Nonlinear risk normalization is performed through a multi-agent collaborative analysis module; the normalized risk value R is then calculated. i Previously, the multi-agent collaborative analysis module employed time-aligned bidirectional cross-attention for feature-level fusion to enhance information interaction and feature extraction among different modalities, specifically including:

[0030] (1) Feature sequence preparation: The visual analysis agent outputs its intermediate layer visual feature sequence F. v The device monitoring agent outputs its state feature sequence F. s These two sequences represent the visual understanding and the encoding of the physical state of the current scene, respectively.

[0031] (2) Timing alignment: Calculate the time offset Δ between cross-modal timing data within a small time window. t Alignment weight w a ; F s Based on Δ t Resampling or phase shifting to the visual time axis, and using w a As a gating coefficient for aligning time-series data;

[0032] (3) Two-way attention interaction: F v and F s The input is processed using a bidirectional cross-attention model; the forward information flow uses visual features as queries and state features as keys / values ​​to obtain the enhanced visual feature F'. v The reverse information flow uses state features as queries and visual features as keys / values ​​to obtain enhanced state features F'. s ;

[0033] (4) Subsequent risk calculation: Enhanced feature sequence F' v and F' s They are sent back to their respective agents to calculate the normalized risk value R. i However, when the alignment weight w a When the threshold is lower than manually set, it automatically degrades to unidirectional attention or unimodal computing.

[0034] Furthermore, in step four, the calculation of the comprehensive risk value DPRI adopts a risk coupling and propagation model based on dynamic scenario awareness, including the following sub-steps:

[0035] (1) Dynamics of the risk coupling coefficient: The fixed risk coupling coefficient ρ in the original calculation formula is changed. ij Upgraded to a function ρ that dynamically changes based on real-time construction scenarios. ij (C); First, construct a real-time scenario vector C = [S s M w ,L r ,…] T S s M is a component of the work phase. w For weather components, L r For the load component; subsequently, the dynamic coupling coefficient ρ ij (C) Calculation is performed using a pre-trained neural network or a nonlinear mapping function;

[0036] (2) Enhanced predictability of risk propagation chains: In the knowledge base, a directed acyclic graph G of risk propagation is constructed by performing causal inference on historical event data. p =(V,E) p ), where vertex V represents various risk factors, and directed edge E p This represents the causal propagation path between risks; when the system detects a risk R from a parent node upstream in the propagation chain... p Even if the risk value is low, the system will predictively increase the risk R of all its downstream child nodes. c The enhancement improves the sensitivity of the assessment; this is achieved by temporarily adjusting the downstream risk assessment parameters:

[0037] ω' c (t)=ω c ·(1+α·R p (t)) or k' c (t)=k c ·(1+β·R p (t));

[0038] Where, ω' c (t) and k' c (t) represents the temporary weight of the risk of the child node and the steepness coefficient of the Sigmoid function, respectively; α and β are the propagation sensitivity factors.

[0039] Accordingly, the present invention also provides a construction safety monitoring system for multimodal intelligent agent collaboration, comprising:

[0040] The multimodal data sensing module can monitor environmental data, structural status data, equipment status data, personnel status data, and can also collect on-site images or video streams;

[0041] The multi-agent collaborative analysis module includes multiple functionally independent agents that normalize and analyze image data, environmental data, structural status data, equipment status data, and personnel status data, and can calculate risk trend factors for key indicators.

[0042] The adaptive risk threshold module dynamically adjusts the warning thresholds of key monitoring indicators based on current multidimensional data, making risk assessment more closely reflect actual working conditions.

[0043] The dynamic predictive risk assessment module integrates the outputs of various intelligent agents, calculates the comprehensive risk through the built-in dynamic predictive risk index model, and outputs the warning level and risk prediction.

[0044] The AI ​​diagnosis and interaction module can provide risk tracing when the system issues an early warning. Based on the calculation path of the dynamic predictive risk index model, it automatically generates a risk contribution decomposition tree and quantifies the contribution of each risk factor.

[0045] The knowledge base module is responsible for the long-term learning and optimization of the system. It stores each monitoring event in the database and, through learning from historical events, can adaptively fine-tune the weight parameters in the dynamic predictive risk index module, thereby enabling the model's parameters to update themselves.

[0046] Compared with existing technologies, this invention, by adopting the above technical solutions, has the following advantages and positive effects: This invention provides a multimodal intelligent agent collaborative construction safety monitoring method and system. By constructing a collaborative analysis framework composed of multiple functionally independent intelligent agents, it achieves deep fusion and intelligent analysis of multi-source heterogeneous data from construction site cameras, sensors, etc. It combines dynamic predictive risk assessment models and adaptive risk threshold technology, and introduces artificial intelligence models for risk tracing and interactive diagnosis, ultimately achieving early prediction and proactive warning of potential safety risks. This solves the problems of one-sided monitoring and delayed warning caused by data silos and static thresholds in existing technologies, improves the comprehensiveness of safety monitoring and the accuracy of warnings in complex construction scenarios, and provides efficient intelligent decision support for managers. Attached Figure Description

[0047] Figure 1 This is a flowchart of a construction safety monitoring method for multimodal intelligent agent collaboration in one embodiment of the present invention.

[0048] Figure 2 This is a diagram illustrating the overall architecture of a multimodal intelligent agent collaborative construction safety monitoring system according to an embodiment of the present invention.

[0049] Figure 3 This is a flowchart of multi-agent collaborative analysis in one embodiment of the present invention;

[0050] Figure 4 This is a schematic diagram of a dynamic risk threshold adaptive adjustment model in one embodiment of the present invention;

[0051] Figure 5 This is a diagram of a dynamic predictive risk index calculation model in one embodiment of the present invention;

[0052] Figure 6 This is a flowchart illustrating the AI ​​model risk tracing and interaction process in one embodiment of the present invention. Detailed Implementation

[0053] The following detailed description, in conjunction with the accompanying drawings and specific embodiments, provides a more detailed explanation of the multimodal intelligent agent collaborative construction safety monitoring method and system provided by the present invention. The advantages and features of the present invention will become clearer from the following description. It should be noted that the accompanying drawings are all in a very simplified form and use non-precise proportions, and are only used to facilitate and clarify the illustration of the embodiments of the present invention.

[0054] Example 1

[0055] This embodiment provides a construction safety monitoring method based on multimodal intelligent agent collaboration. The following section combines... Figure 1 and Figure 2 As shown, the monitoring method will be further described below. The monitoring method includes the following steps:

[0056] S1. Multimodal Data Acquisition and Structuring. Specifically, this involves real-time acquisition of various monitoring data from the construction site, transforming the multimodal data into a unified data object containing timestamps, work stages, sensor readings, and image data. This step is executed by the multimodal data perception module and the multi-agent collaborative analysis module.

[0057] The multimodal data sensing module is responsible for real-time acquisition of multi-source heterogeneous data from the structured construction site. For example, environmental sensors are used to monitor wind speed, rainfall, and snowfall; structural status sensors are used to monitor platform displacement, structural tilt angle, steel beam stress, and platform vibration; equipment status sensors are used to monitor hydraulic pressure, stroke, and synchronization deviation; and personnel status sensors are used to monitor the number of personnel and verify their qualifications.

[0058] S2. Dynamic risk threshold adaptive adjustment. Combined with... Figure 1 Of Figure 3 As shown, this step is performed through the adaptive risk threshold module, which includes a dynamic threshold adjustment model. Specifically, the dynamic threshold adjustment model is used to adjust the key monitoring indicator M. i The warning threshold T'i Adaptive adjustments are made. These adjustments are based on a real-time scenario vector C, which contains multiple contextual factors influencing the threshold safety boundary. A general dynamic threshold adjustment model can be represented as:

[0059]

[0060] Among them, T i C is the baseline threshold for index i. j For the j-th contextual factor (such as working stage, platform vibration, structural tilt angle, platform load, etc.), f j (·) is the adjustment function corresponding to the j-th factor, and its form can be (1-k) ij ·g j (C j )), where k ij To adjust the coefficient, g j (C j Factor C j The normalized representation of J is the total number of context factors involved in the dynamic threshold adjustment.

[0061] As an example, take the wind speed threshold T' w Taking the adjustment as an example, its specific implementation can be as follows:

[0062]

[0063] Among them, T w V is the baseline wind speed threshold; v and V m These are the real-time vibration value and the baseline threshold of the platform, respectively; S c The risk coefficient for the current work phase; L r For platform load rate; k wv ,k ws ,k wl These are specific adjustment coefficients for the wind speed threshold, representing vibration, operating stage, and load, respectively.

[0064] S3. Multi-dimensional risk factor normalization and trend analysis. For example... Figure 4 As shown, the multi-agent collaborative analysis module includes multiple functionally independent agents that collaborate to perform in-depth analysis of perceived data. This step is executed collaboratively by multiple analytical agents, including those related to the environment, equipment, and personnel. This involves analyzing key monitoring indicators M. i Nonlinear risk normalization is performed to obtain the normalized risk value R. i And calculate the risk trend factor λ i .

[0065] (1) Nonlinear risk normalization: The Sigmoid function can be used to normalize the various monitoring indicators M. iConverted to a normalized risk value R between 0 and 1 i .

[0066]

[0067] Among them, M i T' is the real-time measured value of index i. i is the dynamic threshold adjusted in step S2, and k is the steepness coefficient.

[0068] (2) Calculation of risk trend factor: For indicators with time-series characteristics, calculate their risk trend factor λ. i For example:

[0069]

[0070] in, This represents the average rate of change of this indicator within the most recent time window. λ is the average value, and s is the trend sensitivity coefficient. i >1 indicates worsening risk, λ i <1 indicates improved risk.

[0071] S4. Comprehensive assessment of dynamic predictive risk index. Combined with... Figure 1 Of Figure 5 As shown, specifically: based on R i , λ i The dynamic predictive risk assessment module calculates the comprehensive risk value DPRI and compares it with preset multi-level thresholds to determine the warning level.

[0072] This step is the core of risk quantification, integrating the weighted values, trends, and coupling effects of various risk factors. The basic formula for calculating DPRI is:

[0073]

[0074] Where, ω i R represents the basic weight of the i-th risk indicator. i The normalized risk value calculated in step S3; λ i The trend factor calculated in step S3 (λ for non-time series indicators) i =1); ρ ij The coupling coefficient between risk indicators i and j is stored in a predefined coupling matrix in the basic scheme, such as ρ. wv =0.35; N is the total number of key monitoring indicators participating in the comprehensive risk assessment.

[0075] The calculated comprehensive risk value DPRI is compared with preset multi-level thresholds to determine the warning level.

[0076] S5.AI Risk Origin Tracing. When the comprehensive risk value DPRI is determined and reaches the preset warning level, this module can provide risk origin tracing, interactive diagnosis, and decision support functions. Specifically, it combines... Figure 1 Of Figure 6 As shown, the overall risk value (DPRI) is decomposed into each component, the contribution of each risk factor is calculated, and the dominant risk factor is identified. This step provides decision-makers with a transparent basis for decision-making.

[0077] By backtracking the calculation formula of the comprehensive risk value DPRI in step S4, the total risk is decomposed into each component, which is called risk contribution decomposition.

[0078] Individual Risk Contribution C i =ω i ·R i ·λ i ;

[0079] Coupling risk contribution

[0080] Percentage of each contribution

[0081] Among them, individual risk contribution C i Coupling risk contribution C ij The sum of the various components that make up the overall risk value DPRI, and the percentage of risk contribution P. k This represents the relative proportion of each risk factor in the overall risk. The system ranks each risk factor based on its percentage contribution to identify the dominant risk factor, which serves as a crucial basis for prioritizing risk management and adjusting parameters during subsequent emergency response strategy generation and model parameter self-updating. The system organizes this decomposition result into a "risk contribution tree" and identifies the "dominant risk factor."

[0082] This step is performed through the AI ​​diagnostics and interaction module. Furthermore, the AI ​​diagnostics and interaction module also has an interactive question-and-answer function. When managers ask questions in natural language, the system constructs a Prompt along with structured information such as the risk contribution tree, dominant risk factors, and explanations of coupling effects, and submits it to the Large Language Model (LLM) to generate a logical and easy-to-understand natural language answer, which is then displayed to the user.

[0083] S6. Emergency response strategy generation.

[0084] Based on the warning level and dominant risk factors assessed by DPRI, a warning response strategy is generated. The emergency response agent matches and generates a set of immediate measures from the contingency plan library and prioritizes them according to their risk contribution. Furthermore, the event logging agent archives the event, packaging a complete snapshot of the monitoring, including raw data, dynamic thresholds, analysis results from each agent, DPRI assessment results, XAI diagnostic records, and emergency measures, and stores it in a knowledge base.

[0085] In one specific embodiment, the construction safety monitoring method for multimodal intelligent agent collaboration further includes:

[0086] S7. Automatic Model Parameter Update. This step is performed periodically by the knowledge base module or after receiving human feedback. Specifically, it involves obtaining post-hoc results (such as the manually labeled true severity S). a The event records are used to calculate the error E = S between the predicted risk and the actual result. a -S p S p It is the predicted severity mapped from the warning level; then based on the average error. Fine-tune the model parameters using gradient descent, such as the weights ω for each risk category. k :

[0087]

[0088] Where η is the learning rate, g k It is the gradient adjustment coefficient, ω k It is the weight of the risk category.

[0089] Repeat steps S1 to S7 to form a continuous closed-loop intelligent monitoring process of "perception-analysis-evaluation-diagnosis-response-learning".

[0090] In one specific embodiment, in step S3, the multi-agent collaborative analysis module adopts a collaborative mechanism based on confidence negotiation and dynamic task grouping to replace the original fixed linear workflow, thereby improving the system's ability to handle uncertain information and its emergency response efficiency, including:

[0091] (1) Confidence-based negotiation and verification mechanism: Each analytical agent A is specified to... i In outputting its analysis results Res i At the same time, a confidence score quantifying the certainty of its judgment must be output simultaneously. i ∈[0,1]. Set a confidence negotiation threshold θ. n When any intelligent agent A i Output confidence Conf i <θ nIn this case, the result will not directly enter the risk assessment process, but will instead trigger a "negotiation request." This request is broadcast to a predefined set of agents G related to the task. i set G i Upon receiving the request, other agents in the system perform cross-validation. Ultimately, the confirmation of this risk item depends on a fusion confidence level. f This confidence level is calculated by weighting the initial confidence level with the confidence levels of all validation feedbacks. Only when the confidence level is reached... f Exceeding a higher confirmation threshold θ f Only then is the risk item formally adopted.

[0092] (2) Dynamic task grouping based on situation: The system dynamically groups agents into different working groups based on the global risk status (i.e., warning level), including regular inspection groups and emergency response clusters.

[0093] Routine Inspection Group: Under "normal" conditions, all agents follow a standard linear workflow and operate in a low-power, standard-frequency mode.

[0094] Emergency Response Cluster: Once the warning level reaches "Level 2" or above, the system automatically creates a temporary, high-priority "Emergency Response Cluster". This cluster consists of intelligent agents that handle the core of the crisis. It breaks away from the regular workflow and enters a high-frequency, close-knit interactive loop, concentrating all computing resources to rapidly iteratively diagnose, simulate, and support core risks.

[0095] In one specific embodiment, in step S3, the normalized risk value R is calculated. i Previously, the multi-agent collaborative analysis module employed time-aligned bidirectional cross-attention for feature-level fusion to enhance information interaction and feature extraction among different modalities, specifically including:

[0096] (1) Feature sequence preparation: The visual analysis agent outputs its intermediate layer visual feature sequence F. v The device monitoring agent outputs its state feature sequence F. s These two sequences represent the visual understanding and the encoding of the physical state of the current scene, respectively.

[0097] (2) Timing alignment: Calculate the time offset Δ between cross-modal timing data within a small time window. t (Allowed ± preset range) and alignment weight w a F s Based on Δ t Resampling or phase shifting to the visual time axis, and using w a As a gating coefficient for time-series data alignment.

[0098] (3) Two-way attention interaction: F v and F s The input is processed using a bidirectional cross-attention model. Forward information flow (state → vision): using visual features as the query and state features as the key / value pair, the enhanced visual feature F' is obtained. v Reverse information flow (visual → state): Using state features as the query and visual features as the key / value pair, the enhanced state feature F' is obtained. s .

[0099] (4) Subsequent risk calculation: Enhanced feature sequence F' v and F' s They are sent back to their respective agents to calculate the normalized risk value R. i However, when the alignment weight w a When the threshold is lower than manually set, it automatically degrades to unidirectional attention or unimodal computation to avoid amplifying false coupling caused by erroneous fusion.

[0100] In one specific embodiment, in step S4, the calculation of the Dynamic Predictive Risk Index (DPRI) adopts a risk coupling and propagation model based on dynamic scenario awareness to replace the original static coupling matrix, thereby making the risk assessment closer to the construction site, including the following sub-steps:

[0101] (1) Dynamics of the risk coupling coefficient: The fixed risk coupling coefficient ρ in the original calculation formula is changed. ij Upgraded to a function ρ that dynamically changes based on real-time construction scenarios. ij (C). First, construct a real-time scenario vector C = [S s M w ,L r ,…] T S s M is a component of the work phase. w For weather components, L r This represents the load component. Subsequently, the dynamic coupling coefficient ρ... ij (C) Calculated using a pre-trained neural network or a nonlinear mapping function. For example, the coupling coefficient between wind speed and vibration can be expressed as:

[0102] ρ wv (C)=ρ b ·f s (S s )·f l (L r )

[0103] Where, ρ b Based on the coupling coefficient, f s (·) and f l(·) represents a nonlinear mapping function.

[0104] (2) Enhanced predictability of risk propagation chains: In the knowledge base, a directed acyclic graph G of risk propagation is constructed by performing causal inference on historical event data. p =(V,E) p ), where vertex V represents various risk factors, and directed edge E p This represents the causal propagation path between risks. When the system detects a risk R from a parent node upstream in the propagation chain... p Even if the risk value is low, the system will predictively increase the risk R of all its downstream child nodes. c The sensitivity of the assessment. This enhancement is achieved by temporarily adjusting the downstream risk assessment parameters:

[0105] ω' c (t)=ω c ·(1+α·R p (t)) or k' c (t)=k c ·(1+β·R p (t)); where ω' c (t) and k' c (t) represents the temporary weights of the child node risk and the steepness coefficient of the Sigmoid function, respectively; α and β are the propagation sensitivity factors; ω c This indicates the default importance of the risk factor in the overall risk assessment when the impact of risk propagation is not considered; k c This indicates the system's default sensitivity to changes in the risk factors of this sub-node when the impact of risk propagation is not considered.

[0106] Example 2

[0107] This embodiment provides a multimodal intelligent agent collaborative construction safety monitoring system. It adopts a modular multi-agent collaborative architecture and achieves predictive, explainable, and adaptive intelligent safety monitoring of the construction process (especially self-climbing formwork operations) through dynamic risk assessment, adaptive threshold adjustment, AI diagnosis, and knowledge base optimization. The monitoring system includes a multimodal data perception module, a multi-agent collaborative analysis module, an adaptive risk threshold module, a dynamic predictive risk assessment module, an AI diagnosis and interaction module, and a knowledge base module.

[0108] The multimodal data perception module is responsible for real-time acquisition of multi-source heterogeneous data from the structured construction site. This module integrates various sensors, including but not limited to: (1) environmental sensors for monitoring environmental data, such as those for monitoring wind speed, rainfall, and snowfall; (2) structural status sensors for monitoring structural status data, such as those for monitoring platform displacement, structural tilt angle, steel beam stress, and platform vibration; (3) equipment status sensors for monitoring equipment status data, such as those for monitoring hydraulic pressure, stroke, and synchronization deviation; and (4) personnel status sensors for monitoring personnel status data, such as those for monitoring the number of personnel and qualification verification. In addition, this module also includes a visual information acquisition unit for acquiring on-site images or video streams to provide visual data input for multimodal analysis. The multimodal data perception module can acquire multimodal data from the construction site in real time and convert the multimodal data into a unified data object containing timestamps, work stages, various sensor readings, and image data.

[0109] The multi-agent collaborative analysis module includes multiple functionally independent agents that work collaboratively to perform in-depth analysis on a unified data object transformed from multimodal data and output the analysis results to the dynamic predictive risk assessment module. The multi-agent collaborative method can be a predefined computation graph. As a preferred implementation method, a dynamic collaborative mechanism based on confidence negotiation can also be adopted. The module mainly includes: (1) a visual analysis agent that analyzes the input image data based on the visual understanding ability of a large language model and outputs structured visual analysis results, including personnel count, safety violations, structural anomalies, obstacle descriptions and analysis confidence; (2) an environmental monitoring agent that analyzes environmental factors such as wind, rain, and snow, calculates environmental risk scores, and calculates risk trend factors of key indicators in combination with historical data; (3) an equipment monitoring agent that analyzes equipment status data such as platform displacement, vibration, and stress, calculates equipment risk scores, and calculates risk trend factors of key indicators; (4) a personnel monitoring agent that combines sensor data and visual analysis results to assess personnel qualifications, violations and other risks, and calculates personnel risk scores.

[0110] The adaptive risk threshold module has a built-in dynamic threshold adjustment model that dynamically adjusts the warning thresholds of key monitoring indicators based on the current multidimensional scenario, replacing the traditional fixed thresholds and making risk assessment more closely reflect actual working conditions.

[0111] The Dynamic Predictive Risk Assessment module integrates the outputs of various analytical agents, calculates comprehensive risk through the innovative Dynamic Predictive Risk Index (DPRI) model, and outputs early warning levels and risk predictions.

[0112] The AI-powered diagnostic and interaction module provides risk tracing, interactive diagnosis, and decision support when the system issues an alert. Based on the DPRI model's computational path, it automatically generates a risk contribution decomposition tree, quantifying the contribution of each risk factor. It offers a natural language dialogue interface, allowing managers to ask questions about the reasons for the alert and possible solutions. Based on risk contribution, it intelligently prioritizes and recommends the highest-priority response measures.

[0113] The knowledge base module is responsible for the system's long-term learning and optimization. This module stores each monitoring event in the database. By learning from historical events, the system can adaptively fine-tune the weight parameters in the DPRI model, achieving self-updating of the model's parameters.

[0114] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0115] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A construction safety monitoring method based on multimodal intelligent agent collaboration, characterized in that, include: Step 1: Collect multimodal data from the construction site in real time and convert the multimodal data into a unified data object; Step two, using dynamic threshold adjustment model to adjust the early warning threshold T of key monitoring indicators M i i ​​ Step three, monitor the key monitoring indicators M i Nonlinear risk normalization is performed to obtain a normalized risk value R i , and a risk trend factor λ i is calculated Step 4: Based on R i , λ i Calculate the comprehensive risk value DPRI and compare it with the preset multi-level thresholds to determine the warning level; Step 5: Decompose the overall risk value (DPRI) into its components, calculate the contribution of each risk factor, and identify the dominant risk factor. Step 6: Based on the determined warning level and dominant risk factors, generate a warning response strategy.

2. The construction safety monitoring method for multimodal intelligent agent collaboration as described in claim 1, characterized in that, In step two, the dynamic threshold adjustment model is expressed as: Among them, T i C is the baseline threshold for index i. j For the j-th contextual factor, f j (·) is the adjustment function corresponding to the j-th factor, and its form is (1-k ij ·g j (C j )), where k ij To adjust the coefficient, g j (C j Factor C j The normalized representation of J is the total number of context factors involved in the dynamic threshold adjustment.

3. The construction safety monitoring method for multimodal intelligent agent collaboration as described in claim 1, characterized in that, Among them, M i T' is the real-time measured value of index i. i The dynamic threshold is adjusted in step two, and k is the steepness coefficient; in, This represents the average rate of change of the indicator within the most recent time window. λ is the average value, s is the trend sensitivity coefficient; i >1 indicates worsening risk, λ i <1 indicates improved risk.

4. The construction safety monitoring method for multimodal intelligent agent collaboration as described in claim 1, characterized in that, Where, ω i ρ represents the base weight of the i-th risk indicator; ij is the coupling coefficient between risk indicators i and j, and N is the total number of key monitoring indicators participating in the comprehensive risk assessment.

5. The construction safety monitoring method for multimodal intelligent agent collaboration as described in claim 1, characterized in that, In step three, the key monitoring indicator M is... i Nonlinear risk normalization is performed through a multi-agent collaborative analysis module. This module employs a collaborative mechanism based on confidence negotiation and dynamic task grouping, and includes the following sub-steps: Define each analytical agent A i In outputting its analysis results Res i At the same time, a confidence score quantifying the certainty of its judgment must be output simultaneously. i Conf i ∈[0,1]; Set a confidence negotiation threshold θ n When any intelligent agent A i Output confidence Conf i <θ n In this case, the result will not directly enter the risk assessment process, but will instead trigger a "negotiation request," which will be broadcast to a predefined set of agents G related to the task. i ; set G i After receiving the request, other agents in the system perform cross-validation; Whether this risk item is confirmed depends on a fusion confidence level. f This confidence level is calculated by weighting the initial confidence level with the confidence levels of all validation feedbacks. Only when the confidence level is reached... f Exceeding a higher confirmation threshold θ f Only then was the risk item formally adopted.

6. The construction safety monitoring method for multimodal intelligent agent collaboration as described in claim 1, characterized in that, In step three, the key monitoring indicator M is... i Nonlinear risk normalization is performed through a multi-agent collaborative analysis module. In calculating the normalized risk value R i Previously, the multi-agent collaborative analysis module employed time-aligned bidirectional cross-attention for feature-level fusion to enhance information interaction and feature extraction among agents of different modalities, specifically including: (1) Feature sequence preparation: The visual analysis agent outputs its intermediate layer visual feature sequence F. v The device monitoring agent outputs its state feature sequence F. s These two sequences represent the visual understanding and the encoding of the physical state of the current scene, respectively. (2) Timing alignment: Calculate the time offset Δ between cross-modal timing data within a small time window. t Alignment weight w a ; F s Based on Δ t Resampling or phase shifting to the visual time axis, and using w a As a gating coefficient for aligning time-series data; (3) Two-way attention interaction: F v and F s The input is processed using a bidirectional cross-attention model; the forward information flow uses visual features as queries and state features as keys / values ​​to obtain the enhanced visual feature F'. v The reverse information flow uses state features as queries and visual features as keys / values ​​to obtain enhanced state features F'. s ; (4) Subsequent risk calculation: Enhanced feature sequence F' v and F' s They are sent back to their respective agents to calculate the normalized risk value R. i However, when the alignment weight w a When the threshold is lower than manually set, it automatically degrades to unidirectional attention or unimodal computing.

7. The construction safety monitoring method for multimodal intelligent agent collaboration as described in claim 1, characterized in that, In step four, the calculation of the comprehensive risk value DPRI adopts a risk coupling and propagation model based on dynamic scenario awareness, including the following sub-steps: (1) Dynamics of the risk coupling coefficient: The fixed risk coupling coefficient ρ in the original calculation formula is changed. ij Upgraded to a function ρ that dynamically changes based on real-time construction scenarios. ij (C); First, construct a real-time scenario vector C = [S s M w ,L r ,…] T S s M is a component of the work phase. w For weather components, L r For load components; Subsequently, the dynamic coupling coefficient ρ ij (C) Calculation is performed using a pre-trained neural network or a nonlinear mapping function; (2) Enhanced predictability of risk propagation chains: In the knowledge base, a directed acyclic graph G of risk propagation is constructed by performing causal inference on historical event data. p =(V,E) p ), where vertex V represents various risk factors, and directed edge E p This represents the causal propagation path between risks; when the system detects a risk R from a parent node upstream in the propagation chain... p Even if the risk value is low, the system will predictively increase the risk R of all its downstream child nodes. c The enhancement improves the sensitivity of the assessment; this is achieved by temporarily adjusting the downstream risk assessment parameters: oh c (t)=ω c ·(1+α·R p (t)) c (t)=k c ·(1+β·R p (t)); Where, ω' c (t) and k' c (t) represents the temporary weight of the risk of the child node and the steepness coefficient of the Sigmoid function, respectively; α and β are the propagation sensitivity factors.

8. A construction safety monitoring system based on multimodal intelligent agent collaboration, characterized in that, include: The multimodal data sensing module can monitor environmental data, structural status data, equipment status data, personnel status data, and can also collect on-site images or video streams; The multi-agent collaborative analysis module includes multiple functionally independent agents that normalize and analyze image data, environmental data, structural status data, equipment status data, and personnel status data, and can calculate risk trend factors for key indicators. The adaptive risk threshold module dynamically adjusts the warning thresholds of key monitoring indicators based on current multidimensional data, making risk assessment more closely reflect actual working conditions. The dynamic predictive risk assessment module integrates the outputs of various intelligent agents, calculates the comprehensive risk through the built-in dynamic predictive risk index model, and outputs the warning level and risk prediction. The AI ​​diagnosis and interaction module can provide risk tracing when the system issues an early warning. Based on the calculation path of the dynamic predictive risk index model, it automatically generates a risk contribution decomposition tree and quantifies the contribution of each risk factor. The knowledge base module is responsible for the long-term learning and optimization of the system. The knowledge base module stores each monitoring event in the database. By learning from historical events, it can adaptively fine-tune the weight parameters in the dynamic predictive risk index module, thereby enabling the model's parameters to update themselves.

Images