Cluster construction equipment operation state intelligent identification and safety management and control method and system

Abstract

The application relates to a cluster construction equipment operation state intelligent identification and safety management and control method and system, and belongs to the technical field of building engineering construction. The system comprises a distributed visual collection terminal, a multi-modal sensing device, an edge computing unit and a central processing server. Through multi-camera shooting and time sequence alignment and fusion of multiple source sensors, combined with action semantic modeling and state priori, real-time, accurate and interpretable identification of key states of cluster construction equipment is realized. On this basis, a man-machine system activity index and an adaptive threshold are linked to trigger a hierarchical early warning, the identification robustness and early warning accuracy under complex working conditions are significantly improved, edge-cloud cooperation is supported, false positives and false negatives are reduced, and the timeliness and on-site availability of the system are enhanced.

Description

Technical Field

[0001] This invention belongs to the field of construction engineering technology, specifically relating to a method and system for intelligent identification and safety management of the operational status of cluster construction equipment. Background Technology

[0002] Currently, safety monitoring and status identification at construction sites mostly employ single-modal and fixed-threshold technologies. These include IoT-based strain, displacement, load, and vibration sensor networks, which upload data to the cloud or trigger limit-crossing alarms directly at the edge; or video-based target detection / personnel counting and simple rule matching for localized identification of personnel violations, area intrusions, and equipment start-up / shutdown. These methods can achieve real-time monitoring in material loading, single-point equipment operation, or limited scenarios, but they have the following limitations:

[0003] First, there is a lack of deep integration of multi-source heterogeneous data, making it impossible to form a holistic understanding of the collaborative relationship between "human-machine-environment-process" on site;

[0004] Second, they generally rely on human experience or fixed thresholds, making it difficult to adapt to changes in working conditions, stages, loads and environments, resulting in high false alarm / false alarm rates and poor generalization.

[0005] Third, the semantics and action patterns of the construction phase were not modeled, making it difficult to distinguish the fine-grained key states of cluster construction equipment in the standard layer cycle, thus making it impossible to specifically identify differentiated risks such as abnormal lifting synchronization, amplified pumping vibration, and collision prevention for multiple people working together.

[0006] Fourth, existing video AI mostly focuses on target-level detection, lacking temporal alignment with device sensing and causal correlation analysis, making it difficult to provide explainable risk source decomposition and predictive early warning;

[0007] Fifth, the lack of quantitative assessment and interval determination of the activity level of human-machine systems leads to insufficient coupling between state recognition and risk assessment.

[0008] Therefore, there is an urgent need for a cluster equipment status recognition and safety monitoring technology that integrates vision-language-action models and human-machine system activity theory, and has multimodal deep fusion, temporal consistency constraints, adaptive thresholds and interpretable diagnostic capabilities, so as to improve the recognition accuracy and risk handling efficiency of cluster construction equipment under complex working conditions. Summary of the Invention

[0009] This invention provides a method and system for intelligent identification and safety management of the operating status of cluster construction equipment, which enables real-time, accurate, and interpretable identification and risk warning of four key states of cluster construction equipment, significantly improving the robustness of identification, the timeliness of warning and the interactivity of the site under complex working conditions.

[0010] To solve the above technical problems, the present invention includes the following technical solutions:

[0011] A cluster construction equipment operation status intelligent identification and safety management system includes:

[0012] The distributed vision acquisition terminal includes a multi-level industrial camera array and an image acquisition controller. The multi-level industrial camera array includes a global monitoring camera group, a local operation camera group, and a dedicated equipment camera group for acquiring image data. The image acquisition controller supports simultaneous acquisition from multiple cameras and hardware encoding, transmitting image data to the edge computing processing unit.

[0013] Multimodal sensing devices acquire sensor data from the construction site, including vibration monitoring data, attitude monitoring data, load monitoring data, and displacement monitoring data, and send the sensor data to the edge computing unit;

[0014] The edge computing unit receives image data and sensor data, can preprocess the image data and sensor data, identify the situation of the detection personnel and judge the key working status of the cluster construction equipment through the built-in deep learning object detection model, and extract image features to generate feature vectors.

[0015] The central processing server outputs the key operating status and confidence level of the cluster construction equipment based on the built-in visual language action model; it also fuses or resolves conflicts in the results of the key operating status of the cluster construction equipment judged by the deep learning object detection model and the visual language action model.

[0016] Accordingly, the present invention also provides a method for intelligent identification and safety management of the operating status of cluster construction equipment, comprising the following steps:

[0017] S1. Real-time image data is collected using distributed visual acquisition terminals, and construction site sensor data is collected using multimodal sensing devices;

[0018] S2. The edge computing processing unit receives image data and sensor data, performs data preprocessing, uses a deep learning target detection model to detect personnel status and equipment operating status, and extracts image features to generate feature vectors.

[0019] S3. Calculate the activity level of the human-machine system based on the data output by the deep learning target detection model, and make a preliminary judgment on the key working status of the cluster construction equipment based on the activity level of the human-machine system;

[0020] S4. Determine the key working status of cluster construction equipment based on visual language action models;

[0021] S5. Perform dual-channel fusion decision-making and conflict resolution on the key working states of the cluster construction equipment determined in steps S3 and S4 respectively.

[0022] Furthermore, step S3 calculates the activity level of the human-machine system, specifically including:

[0023] S3-1. Calculate the personnel activity index ;

[0024] ,

[0025] in For the first Risk weights for different types of personnel Obtained adaptively through historical data statistical analysis and machine learning methods; For the first The normalized number of people in this category. , The first one actually detected Number of personnel in this category The first [project] determined based on project scale and construction specifications Maximum number of personnel allowed in this category; For the first The active state function for the class of personnel has a value range of 100. , Total number of personnel categories;

[0026] S3-2. Calculate the device activity index ;

[0027] ,

[0028] in, For the first Risk weights for different types of equipment For the first The number of such devices; For the first The running status function of the device class has the following value: Continuous values; This represents the total number of equipment categories.

[0029] S3-3. Calculate the comprehensive human-computer system activity index. ;

[0030] ,

[0031] in, The human-machine coupling coefficient has a value range of [value missing]. .

[0032] Furthermore, in step S3, the key working status of the cluster construction equipment is initially determined based on the activity level of the human-machine system, specifically including:

[0033] Based on the calculated human-computer system activity index The initial state is determined by the composition ratio of the states, and the interval thresholds for each state are adaptively determined by cluster analysis and statistical learning methods.

[0034] when And the percentage of device activity When the hydraulic system's operating characteristics and the platform's vertical displacement speed exceed the threshold, it is determined to be in an overall climbing state. ; In the formula, They are respectively states The minimum and maximum values ​​of the activity range, This represents the threshold for the percentage of activity levels.

[0035] when And the percentage of active users When multi-task collaborative operation characteristics are detected, it is determined to be a collaborative operation state. , : Respective states The minimum and maximum values ​​of the activity range, This represents the threshold for the percentage of activity levels.

[0036] when Furthermore, when the characteristic frequency of concrete pumping and the operating mode of the concrete placing boom are detected, it is determined to be a pumping pouring state. In the formula, They are respectively states The minimum and maximum values ​​of the activity range;

[0037] when Furthermore, if all activity indicators are below the preset values, the activity is considered to be in a state of static suspension. In the formula, For state The activity threshold.

[0038] Furthermore, in step S4, the key working states of the cluster construction equipment are identified using a visual language action model, specifically as follows:

[0039] Deep convolutional features, including global scene features and local object features, are extracted from the preprocessed image to generate a high-dimensional visual feature vector. , Using a visual language model to generate scene description text, identify key construction instructions and work commands, and construct semantic feature vectors for the scene. , ;in, Visual feature vectors Dimensions semantic feature vector The dimension;

[0040] Based on spatiotemporal graph convolutional network analysis, key construction action patterns are identified from video sequences, generating action feature vectors. , fusion of sensor feature vectors Construct a comprehensive feature vector Input a deep network based on the Transformer architecture, output the probability distribution of four key working states of the cluster construction equipment. And calculate the identification confidence level. and uncertainty measurement ;in, These are the overall climbing statuses. Collaborative work status Pumping and pouring status , Static and inactive state The probability of.

[0041] Furthermore, in step S5, the key operating states of the cluster construction equipment determined in steps S3 and S4 are subject to dual-channel fusion decision-making and conflict resolution, specifically as follows:

[0042] The key working states determined by the human-machine system activity method are denoted as follows: Confidence level is denoted as The key working states determined using the visual language action model method are denoted as follows: Confidence level is denoted as ; Calculate the consistency metric between the two methods ,

[0043] ;

[0044] when When the two methods are highly consistent, the final state is directly output. And add the case to the positive sample library;

[0045] when At that time, the weighted fusion mechanism is activated to calculate the fusion weight. and ;in and The historical reliability scores for the two methods are given respectively, using fusion probability. Determine the final state;

[0046] when The conflict resolution mechanism is triggered on time to extract conflict features. The input is processed by a dedicated conflict resolution network that generates a judgment result. The conflict resolution network generates a judgment result by analyzing historical conflict cases and current context information, and at the same time triggers a security link protection mechanism to temporarily increase the security level and restrict dangerous operations until the conflict is resolved.

[0047] Furthermore, prior to S5, reliability verification is also included, specifically:

[0048] The reliability of the two methods is evaluated by constructing a reliability assessment network. When the reliability of either method falls below a threshold... At that time, a multi-source verification mechanism is initiated, calling the case library for analogical reasoning, activating the backup recognition channel for cross-verification, and using a voting mechanism. By combining the results of multiple methods, the system can maintain stable operation even when a single method fails; among them, For the first The key working state determination results output by this verification method These are candidate critical operating status categories for cluster construction equipment. for , , , any of them, For the first The weighting coefficients of various verification methods reflect the historical reliability or importance of those methods. The index for the verification method indicates the number of participants in the vote. A method or channel for identification.

[0049] Furthermore, the method for intelligent identification and safety management of the operational status of cluster construction equipment also includes:

[0050] S6. Generate differentiated security policies; specifically:

[0051] Constructing a multidimensional risk assessment model ,in This is the current state. For environmental factors vectors, For historical risk event statistics, the risk function parameters are optimized through deep reinforcement learning;

[0052] Establish a tiered response mechanism:

[0053] When the risk level At this time, the system operates normally and records status logs, indicating a safe level.

[0054] when At this time, the monitoring frequency will be increased and emergency response plans will be pre-loaded to address the level of concern.

[0055] when When the time is right, it is a warning level, triggering an audible and visual alarm, restricting certain high-risk operations, and notifying safety management personnel;

[0056] when If the situation is deemed dangerous, immediately implement an emergency shutdown, activate interlock protection, and trigger an automatic alarm.

[0057] Furthermore, between steps S5 and S6, the following is also included:

[0058] The compliance of state transitions is verified based on finite state machine theory, and a complete state transition diagram is defined. ,in For a set of states, For the compliant transformation edge set, each edge Related conversion conditions Minimum duration and maximum duration ;

[0059] Verify the current state transition Does it meet the requirements? And conversion conditions Established, check the duration of the status. Does it meet the requirements? When an illegal conversion is detected, a rollback mechanism is executed. It also records anomaly logs; generates warning information when the duration of a state exceeds a reasonable range; and builds a normal behavior pattern library through state transition history to provide a reference benchmark for anomaly detection.

[0060] in, These are the state transition graph and its set of vertices and edges. This is a state transition condition. For the time constraints of state transitions; For state duration constraints, The duration of the state.

[0061] Furthermore, when calculating the activity level of the human-machine system, a dynamic weight adjustment mechanism is adopted to dynamically adjust the risk weights of various personnel and equipment according to the construction stage.

[0062] Compared with existing technologies, this invention, by adopting the above technical solutions, has the following advantages and positive effects: This invention provides a method and system for intelligent identification and safety management of the operating status of cluster construction equipment. By fusion of multi-camera and multi-source sensor sequences, combined with action semantic modeling and state priors, it achieves real-time, accurate, and interpretable identification of key states of cluster construction equipment. On this basis, it triggers hierarchical early warnings by linking the human-machine system activity index with adaptive thresholds, significantly improving the robustness of identification and the accuracy of early warnings under complex working conditions. At the same time, it supports edge-cloud collaboration, reduces false alarms and missed alarms, and enhances the timeliness and on-site availability of the system. Attached Figure Description

[0063] Figure 1 This is an overall architecture diagram of intelligent identification and safety management of cluster construction equipment operation status in one embodiment of the present invention;

[0064] Figure 2 This is a flowchart of a method for intelligent identification and safety management of the operating status of cluster construction equipment according to an embodiment of the present invention;

[0065] Figure 3 This is a logical relationship diagram of the intelligent identification and safety management method for the operation status of cluster construction equipment in one embodiment of the present invention;

[0066] Figure 4 This is a flowchart of the dual-channel fusion decision-making and conflict resolution mechanism in one embodiment of the present invention. Detailed Implementation

[0067] The following detailed description, in conjunction with the accompanying drawings and specific embodiments, provides a more comprehensive understanding of the multimodal intelligent agent collaborative construction safety monitoring method and system provided by the present invention. The advantages and features of the present invention will become clearer from the following description. It should be noted that the accompanying drawings are all in a very simplified form and use non-precise proportions, and are only used to facilitate and clarify the illustration of the embodiments of the present invention.

[0068] Example 1

[0069] This embodiment provides an intelligent identification and safety management system for the operational status of cluster construction equipment, such as... Figure 1 As shown, the system includes distributed visual acquisition terminals, multimodal sensing devices, edge computing units, and a central processing server. The system adopts an edge-cloud collaborative hardware architecture, utilizing distributed visual acquisition terminals, multimodal sensing devices, edge computing units, and a central processing server, combined with Visual Language Action (VLA) models and Human-Machine System Activity (HMSA) theory, to achieve real-time and accurate identification of the key operating states of the cluster construction equipment. The cluster construction equipment has four key operating states, including overall climbing state. Collaborative work status Pumping and pouring status , Static and inactive state .

[0070] The distributed vision acquisition terminal includes a multi-level industrial camera array and an image acquisition controller. The multi-level industrial camera array comprises a global monitoring camera group covering the entire cluster of construction equipment and its surrounding preset range; local operation camera groups focusing on key areas such as the main platform and work surface; and equipment-specific camera groups fixedly installed on equipment such as tower cranes and concrete placing booms. The global monitoring camera group uses high-resolution wide-angle industrial cameras equipped with motorized pan-tilt units and fixed-focus lenses, achieving large-area monitoring at a low frame rate via PoE power supply. The local operation camera group uses high-speed industrial cameras equipped with zoom lenses and infrared supplementary lights to achieve detailed observation at a higher frame rate. The equipment-specific camera group uses shock-resistant industrial cameras with metal protective covers and reinforced brackets to adapt to equipment vibration environments. The image acquisition controller is based on a high-performance embedded GPU platform, equipped with large-capacity memory and solid-state storage, supporting simultaneous acquisition from multiple cameras and hardware encoding. The image acquisition controller can acquire image data from the distributed vision acquisition terminal and transmit it to the edge computing unit; the image data includes images and videos.

[0071] The multimodal sensing device integrates various sensors, including vibration, attitude, load, and displacement sensors, to acquire physical state information at the construction site. It comprises a vibration monitoring device consisting of multiple triaxial accelerometers fixed to key nodes of the main platform via magnetic bases or bolts; an attitude measurement device consisting of multiple high-precision tilt sensors integrating gyroscopes and magnetometers and connected via an industrial bus; a load monitoring device consisting of multiple strain gauge load sensors installed at key stress locations such as load-bearing pins and equipped with signal amplifiers and high-precision ADC modules; and a displacement measurement device consisting of a combination of laser displacement sensors and wire-type displacement sensors. Each sensor transmits real-time data to the edge computing unit via a wireless transmission module or wired bus.

[0072] The edge computing processing unit is deployed at the construction site for data preprocessing and preliminary analysis. It employs an industrial-grade chassis enclosure and integrates a multi-core high-performance processor, large-capacity ECC memory, multiple professional GPUs, and a high-speed solid-state drive array. Equipped with multiple gigabit Ethernet ports, high-speed USB interfaces, and industrial serial ports, it supports various industrial protocols such as Modbus, OPC UA, and MQTT. It features a wide operating temperature range and a high-protection-level enclosure, and is equipped with a UPS uninterruptible power supply to ensure stable operation in harsh environments. The edge computing processing unit handles computationally intensive tasks such as image preprocessing, target detection, and feature extraction. Furthermore, it performs time synchronization, outlier removal, noise filtering, and feature normalization on the received multimodal sensor data, and correlates and fuses this data with image features at corresponding times to generate unified multi-source feature data for subsequent cluster construction equipment operation status identification and safety management decisions.

[0073] The central processing server is deployed in the project's computer room, running the VLA deep learning model. It consists of multiple GPU servers equipped with high-performance processors, large-capacity memory, and multiple AI accelerator cards, forming computing nodes. It is connected to a distributed storage array that supports redundancy protection and high-speed data transmission, and is interconnected via a high-speed core switch. It is equipped with security protection devices to ensure system security and is responsible for running complex visual language action models and deep learning inference tasks.

[0074] Example 2

[0075] This embodiment provides a method for intelligent identification and safety management of the operational status of cluster construction equipment. The following section will combine... Figures 1 to 4 As shown, the method will be further described below. The method includes the following steps:

[0076] S1. Real-time acquisition of raw image data is achieved using distributed visual acquisition terminals, and raw sensor data is acquired through multimodal sensing devices.

[0077] S2. The edge computing processing unit receives image data and sensor data, performs data preprocessing, uses the built-in deep learning target detection model to detect personnel status and equipment operating status, and extracts image features to generate feature vectors.

[0078] As an example, the GPU acceleration capability of the edge computing unit enables low-latency visual analysis, using parallel computing techniques to perform denoising, distortion correction, and histogram equalization enhancement on batches of images. The deep learning object detection model, based on preprocessed image data, can identify construction workers such as formwork operators, rebar workers, carpenters, and concrete workers. It can also detect the operating status of key equipment such as climbing formwork, tower cranes, concrete placing booms, and construction hoists based on preprocessed sensor data. Hardware accelerators are used to extract image features in parallel, generating feature vectors for subsequent analysis, and supporting multi-scale feature extraction to adapt to targets of different sizes.

[0079] S3. Calculate the human-machine system activity level based on the data output by the deep learning object detection model, and preliminarily determine the key working status of the cluster construction equipment based on the human-machine system activity level. First, calculate the personnel activity index. Through formula Calculation, where For the first Risk weights for different groups of people are adaptively obtained through historical data statistical analysis and machine learning methods. Initial values ​​can be determined using Bayesian optimization or genetic algorithms. For the first The normalized number of personnel in this category is obtained through calculate, The first one actually detected Number of personnel in this category The first [project] determined based on project scale and construction specifications The maximum permissible number of personnel of this type can be dynamically adjusted based on construction organization design and safety management requirements. For the first The active state function for personnel is calculated using fuzzy logic based on multi-dimensional features such as personnel movement speed and work posture, with a value range of [value missing]. , The first step is to calculate the total number of personnel categories; the second step is to calculate the equipment activity index. Through formula Calculation, where The risk weight for the first type of equipment is dynamically calculated by multiplying the equipment failure rate, accident severity, and exposure frequency, and then optimized online using reinforcement learning. For the first The number of such devices For the first The operating state function of this type of equipment is calculated by fusing multi-source information such as vibration spectrum analysis and power monitoring, and its value is... Continuous values ​​reflect the operating intensity of the equipment. The total number of equipment categories; finally, the comprehensive human-machine system activity index is calculated. Through formula Calculation, where The human-machine coupling coefficient is automatically learned through a deep neural network based on historical data and can be dynamically adjusted according to factors such as construction stage and environmental conditions. Its value range is [range missing]. .

[0080] Based on the calculated human-computer system activity index The initial state is determined based on the composition ratio of the states, and the interval thresholds for each state are adaptively determined through cluster analysis and statistical learning methods. And the percentage of device activity When the hydraulic system's operating characteristics and the platform's vertical displacement speed exceed the threshold, it is determined to be in an overall climbing state. ,when And the percentage of active users When multi-task collaborative operation characteristics are detected, it is determined to be a collaborative operation state. ,when Furthermore, when the characteristic frequency of concrete pumping and the operating mode of the concrete placing boom are detected, it is determined to be a pumping pouring state. ,when Furthermore, when all activity indicators are at extremely low levels, the work is deemed to be in a state of stagnation and shutdown. All interval thresholds are continuously optimized through online learning algorithms based on feedback from recognition accuracy.

[0081] S4. Determine the key working status of cluster construction equipment based on a visual language action model. Integrate multimodal features for accurate status recognition, and extract deep convolutional features from preprocessed images, including global scene features and local target features, to generate high-dimensional visual feature vectors. Using a visual language model to generate scene description text, identifying key construction instructions and work commands, and constructing a semantic representation of the scene. Based on spatiotemporal graph convolutional networks, this study analyzes video sequences to identify key construction action patterns such as climbing motion sequences, rebar tying actions, formwork support actions, and concrete pouring actions, generating action features. Fusion of sensor features Constructing a comprehensive feature vector The input is a deep network based on the Transformer architecture, which processes the fused features through a multi-layer self-attention mechanism and outputs the probability distribution of four states. And calculate the recognition confidence level. and uncertainty measurement .

[0082] S5. Perform dual-channel fusion decision-making and conflict resolution on the key working states of the cluster construction equipment determined in steps S3 and S4 respectively.

[0083] Combination Figure 3 and Figure 4 As shown, combining the initial HMSA result and the detailed VLA result, the final state is determined through a complete logical loop. Let the state determined by the HMSA method be... and its confidence level The VLA model identifies the state as follows: and its confidence level First, calculate the consistency metric between the two methods. ,when When the two methods are highly consistent, the final state is directly output. And add this case to the positive sample library; when The weighted fusion mechanism is activated at the specified time to calculate the fusion weight. and ,in and The historical reliability scores for the two methods are given respectively, using fusion probability. Determine the final state; when The conflict resolution mechanism is triggered on time to extract conflict features. The input is processed by a dedicated conflict resolution network that generates a judgment result. This network generates a judgment result by analyzing historical conflict cases and current context information, and at the same time triggers a security link protection mechanism to temporarily increase the security level and restrict dangerous operations until the conflict is resolved.

[0084] In one specific embodiment, the method for intelligent identification and safety management of the operational status of cluster construction equipment further includes:

[0085] S6. Generate differentiated security strategies, specifically by constructing a multi-dimensional risk assessment model. ,in This is the current state. For environmental factors vectors, To analyze historical risk events, we optimize the risk function parameters using deep reinforcement learning; and establish a tiered response mechanism, where the risk level... At the current security level, the system operates normally and records status logs. At this level of concern, monitoring frequency will be increased and emergency response plans will be pre-loaded. At the warning level, an audible and visual alarm is triggered, certain high-risk operations are restricted, and safety management personnel are notified. If the situation is classified as hazardous, immediately execute an emergency shutdown, activate interlock protection, and trigger an automatic alarm; implement predictive maintenance strategies, predicting failure times by analyzing equipment operating trends. ,when It generates maintenance suggestions in real time; optimizes human-machine collaboration efficiency; calculates collision risks through trajectory prediction algorithms; and generates the optimal avoidance path.

[0086] Repeat steps S1 to S6 to form a continuous intelligent identification and safety monitoring closed loop, enabling real-time, accurate, and intelligent identification and control of the working status of cluster construction equipment.

[0087] In one specific embodiment, the calculation of the human-machine system activity level in step S3 adopts a dynamic weight adjustment mechanism, which dynamically adjusts the risk weights of various personnel and equipment according to the construction stage, and appropriately increases the weight of the climbing formwork during critical construction stages such as the first climb. During the concrete pouring stage, the weight of the placing boom should be appropriately increased. Update the weight matrix through historical data analysis .

[0088] In one specific embodiment, a reliability verification step is added between steps S4 and S5. A reliability evaluation network is constructed to assess the reliability of the two methods. If the reliability of either method falls below a threshold... At this time, a multi-source verification mechanism is initiated, calling the case library for analogical reasoning, activating backup recognition channels such as rule-based expert systems for cross-validation, and using a voting mechanism. By combining the results of multiple methods, the system can be ensured to maintain stable operation even if a single method fails. Among these, For the first The key working state determination results output by this verification method These are candidate critical operating status categories for cluster construction equipment. for , , , any of them, For the first The weighting coefficients of various verification methods reflect the historical reliability or importance of those methods. The index for the verification method indicates the number of participants in the vote. A method or channel for identification.

[0089] In one specific embodiment, a state transition compliance verification and timing consistency guarantee step is added between steps S5 and S6. Based on finite state machine theory, the compliance of state transitions is verified, and a complete state transition diagram is defined. ,in For a set of states, For the compliant transformation edge set, each edge Related conversion conditions Minimum duration and maximum duration Verify the current state transition Does it meet the requirements? And conversion conditions Established, check the duration of the status. Does it meet the requirements? A rollback mechanism is executed when an illegal conversion is detected. It also records anomaly logs, generates warning messages when the duration of a state exceeds a reasonable range, and builds a normal behavior pattern library through state transition history to provide a reference benchmark for anomaly detection.

[0090] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0091] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A system for intelligent identification and safety management of the operational status of cluster construction equipment, characterized in that, include: The distributed vision acquisition terminal includes a multi-level industrial camera array and an image acquisition controller. The multi-level industrial camera array includes a global monitoring camera group, a local operation camera group, and a dedicated equipment camera group for acquiring image data. The image acquisition controller supports simultaneous acquisition from multiple cameras and hardware encoding, transmitting image data to the edge computing processing unit. Multimodal sensing devices acquire sensor data from the construction site, including vibration monitoring data, attitude monitoring data, load monitoring data, and displacement monitoring data, and send the sensor data to the edge computing unit; The edge computing unit receives image data and sensor data, can preprocess the image data and sensor data, identify the situation of the detection personnel and judge the key working status of the cluster construction equipment through the built-in deep learning object detection model, and extract image features to generate feature vectors. The central processing server outputs the key operating status and confidence level of the cluster construction equipment based on the built-in visual language action model; it also fuses or resolves conflicts in the results of the key operating status of the cluster construction equipment judged by the deep learning object detection model and the visual language action model.

2. A method for intelligent identification and safety management of the operational status of cluster construction equipment, characterized in that, Includes the following steps: S1. Real-time image data is collected using distributed vision acquisition terminals, and construction site sensor data is collected using multimodal sensing devices; S2. The edge computing processing unit receives image data and sensor data, performs data preprocessing, uses a deep learning target detection model to detect personnel status and equipment operating status, and extracts image features to generate feature vectors. S3. Calculate the activity level of the human-machine system based on the data output by the deep learning target detection model, and make a preliminary judgment on the key working status of the cluster construction equipment based on the activity level of the human-machine system; S4. Determine the key working status of cluster construction equipment based on visual language action models; S5. Perform dual-channel fusion decision-making and conflict resolution on the key working states of the cluster construction equipment determined in steps S3 and S4 respectively.

3. The method for intelligent identification and safety management of the operational status of cluster construction equipment as described in claim 2, characterized in that, Step S3 calculates the activity level of the human-machine system, specifically including: S3-1. Calculate the personnel activity index ; ， in For the first Risk weights for different types of personnel Obtained adaptively through historical data statistical analysis and machine learning methods; For the first The normalized number of people in this category. , The first one actually detected Number of personnel in this category The first [project] determined based on project scale and construction specifications Maximum number of personnel allowed in this category; For the first The active state function for the class of personnel has a value range of 100. , Total number of personnel categories; S3-2. Calculate the device activity index ; ， in, For the first Risk weights for different types of equipment For the first The number of such devices; For the first The running status function of the device class has the following value: Continuous values; This represents the total number of equipment categories. S3-3. Calculate the comprehensive human-computer system activity index. ; ， in, The human-machine coupling coefficient has a value range of [value missing]. .

4. The method for intelligent identification and safety management of the operational status of cluster construction equipment as described in claim 3, characterized in that, Step S3 involves making a preliminary assessment of the key operating status of the cluster construction equipment based on the activity level of the human-machine system, specifically including: Based on the calculated human-computer system activity index The initial state is determined by the composition ratio of the states, and the interval thresholds for each state are adaptively determined by cluster analysis and statistical learning methods. when And the percentage of device activity When the hydraulic system's operating characteristics and the platform's vertical displacement speed exceed the threshold, it is determined to be in an overall climbing state. ; In the formula, They are respectively states The minimum and maximum values ​​of the activity range, This represents the threshold for the percentage of activity levels. when And the percentage of active users When multi-task collaborative operation characteristics are detected, it is determined to be a collaborative operation state. , : Respective states The minimum and maximum values ​​of the activity range, This represents the threshold for the percentage of activity levels. when Furthermore, when the characteristic frequency of concrete pumping and the operating mode of the concrete placing boom are detected, it is determined to be a pumping pouring state. In the formula, They are respectively states The minimum and maximum values ​​of the activity range; when Furthermore, if all activity indicators are below the preset values, the activity is considered to be in a state of static suspension. In the formula, For state The activity threshold.

5. The method for intelligent identification and safety management of the operating status of cluster construction equipment as described in claim 2, characterized in that, In step S4, the key working states of the cluster construction equipment are identified using a visual language action model, specifically as follows: Deep convolutional features, including global scene features and local object features, are extracted from the preprocessed image to generate a high-dimensional visual feature vector. , Using a visual language model to generate scene description text, identify key construction instructions and work commands, and construct semantic feature vectors for the scene. , ;in, Visual feature vectors Dimensions semantic feature vector The dimension; Based on spatiotemporal graph convolutional network analysis, key construction action patterns are identified from video sequences, generating action feature vectors. , fusion of sensor feature vectors Construct a comprehensive feature vector Input a deep network based on the Transformer architecture, output the probability distribution of four key working states of the cluster construction equipment. And calculate the identification confidence level. and uncertainty measurement ;in, These are the overall climbing statuses. Collaborative work status Pumping and pouring status , Static and inactive state The probability of.

6. The method for intelligent identification and safety management of the operational status of cluster construction equipment as described in claim 2, characterized in that, In step S5, the key operating states of the cluster construction equipment determined in steps S3 and S4 are subject to dual-channel fusion decision-making and conflict resolution, specifically as follows: The key working states determined by the human-machine system activity method are denoted as follows: Confidence level is denoted as The key working states determined using the visual language action model method are denoted as follows: Confidence level is denoted as ; Calculate the consistency metric between the two methods , ； when When the two methods are highly consistent, the final state is directly output. And add the case to the positive sample library; when At that time, the weighted fusion mechanism is activated to calculate the fusion weight. and ;in and The historical reliability scores for the two methods are given respectively, using fusion probability. Determine the final state; when The conflict resolution mechanism is triggered on time to extract conflict features. The input is processed by a dedicated conflict resolution network that generates a judgment result. The conflict resolution network generates a judgment result by analyzing historical conflict cases and current context information, and at the same time triggers a security link protection mechanism to temporarily increase the security level and restrict dangerous operations until the conflict is resolved.

7. The method for intelligent identification and safety management of the operating status of cluster construction equipment as described in claim 2, characterized in that, Prior to S5, reliability verification was also performed, specifically: The reliability of the two methods is evaluated by constructing a reliability assessment network. When the reliability of either method falls below a threshold... At that time, a multi-source verification mechanism is initiated, calling the case library for analogical reasoning, activating the backup recognition channel for cross-verification, and using a voting mechanism. By combining the results of multiple methods, the system can maintain stable operation even when a single method fails; among them, For the first The key working state determination results output by this verification method These are candidate critical operating status categories for cluster construction equipment. for , , , any of them, For the first The weighting coefficients of various verification methods reflect the historical reliability or importance of those methods. The index for the verification method indicates the number of participants in the vote. A method or channel for identification.

8. The method for intelligent identification and safety management of the operating status of cluster construction equipment as described in claim 2, characterized in that, Also includes: S6. Generate differentiated security policies; Specifically: Constructing a multidimensional risk assessment model ,in This is the current state. For environmental factors vectors, For historical risk event statistics, the risk function parameters are optimized through deep reinforcement learning; Establish a tiered response mechanism: When the risk level At this time, the system operates normally and records status logs, indicating a safe level. when At this time, the monitoring frequency will be increased and emergency response plans will be pre-loaded to address the level of concern. when When the time is right, it is a warning level, triggering an audible and visual alarm, restricting certain high-risk operations, and notifying safety management personnel; when If the situation is deemed dangerous, immediately implement an emergency shutdown, activate interlock protection, and trigger an automatic alarm.

9. The method for intelligent identification and safety management of the operational status of cluster construction equipment as described in claim 8, characterized in that, Between steps S5 and S6, the following is also included: The compliance of state transitions is verified based on finite state machine theory, and a complete state transition diagram is defined. ,in For a set of states, For the compliant transformation edge set, each edge Related conversion conditions Minimum duration and maximum duration ; Verify the current state transition Does it meet the requirements? And conversion conditions Established, check the duration of the status. Does it meet the requirements? When an illegal conversion is detected, a rollback mechanism is executed. It also records anomaly logs; generates warning information when the duration of a state exceeds a reasonable range; and builds a normal behavior pattern library through state transition history to provide a reference benchmark for anomaly detection. in, These are the state transition graph and its set of vertices and edges. This is a state transition condition. For the time constraints of state transitions; For state duration constraints, The duration of the state.

10. The method for intelligent identification and safety management of the operating status of cluster construction equipment as described in claim 3, characterized in that, When calculating the activity level of the human-machine system, a dynamic weight adjustment mechanism is adopted to dynamically adjust the risk weights of various personnel and equipment according to the construction stage.

Images