Air-ground collaborative construction safety and environmental protection management method based on multi-modal AI driving

Abstract

The application discloses a kind of based on multi-modal AI drive's air-ground collaborative construction safety environmental protection management and control method, it is related to building engineering safety environmental protection comprehensive management technical field, method includes: according to construction site information demarcates inspection area and generates air-ground collaborative inspection scheme, inspection scheme is sent to mobile sensing device;Mobile sensing device receives inspection scheme, and multi-modal data is collected in inspection area by mobile sensing device after receiving inspection scheme, and multi-modal data is transmitted to intelligent decision hub in real time;Control intelligent decision hub carries out multi-modal fusion analysis to received multi-modal data, generates disposal strategy;According to disposal strategy control hardware device, control mobile sensing device and carry out reinspection to rectification area in inspection area.The present application can realize the whole three-dimensional monitoring of construction area, safety environmental protection hidden danger fast identification and disposal, improve management efficiency.

Description

Technical Field

[0001] This invention relates to the field of integrated management technology for safety and environmental protection in construction engineering, and in particular to a method for safety and environmental protection management of air-ground collaborative construction driven by multimodal AI. Background Technology

[0002] In traditional construction sites, safety and environmental protection management primarily relies on manual inspections and fixed monitoring equipment. However, this approach has numerous shortcomings, including incomplete spatiotemporal coverage, delayed risk response, low standardization, low processing efficiency, and high labor costs. These issues result in inefficient safety and environmental management at construction sites, hindering the achievement of refined management and proactive prevention.

[0003] To address the aforementioned technical challenges, existing quadruped robots have achieved breakthroughs in dynamic adaptability to complex construction scenarios. These robots integrate multimodal environmental perception systems and high-precision motion control algorithms, enabling autonomous obstacle avoidance and stable passage in unstructured terrain. Unmanned aerial vehicles (UAVs), leveraging their intelligent flight control and wide-area coverage advantages, construct a high-altitude, three-dimensional monitoring network. Multimodal large-scale models, based on a cross-dimensional data fusion architecture, can uniformly integrate and analyze information such as visual recognition, acoustic information, and environmental parameters, establishing a reasoning framework for safety risks and environmental protection, and simultaneously driving risk warning and response decisions.

[0004] However, current technologies for quadruped robots, drones, and multimodal large-scale models exist in independent applications or simple combinations, lacking systematic and in-depth integration solutions. Existing technologies cannot achieve dynamic networking and task collaboration between quadruped robots and drones, and multimodal data cannot be effectively integrated, resulting in incomplete monitoring, delayed risk response, and an inability to achieve fully automated management from risk perception, intelligent assessment, dynamic handling to experience accumulation. Construction safety and environmental management still relies on manual intervention, and management efficiency and decision-making accuracy need improvement. Summary of the Invention

[0005] The technical problem this invention aims to solve is to address the shortcomings of existing technologies, specifically the problems in traditional construction safety and environmental protection management, such as incomplete spatiotemporal coverage of regional monitoring, delayed risk response, low standardization of management, and low processing efficiency. Specifically, it provides a multimodal AI-driven air-ground collaborative construction safety and environmental protection management method, as detailed below: 1) In a first aspect, the present invention provides a method for safety and environmental protection management of air-ground collaborative construction driven by multimodal AI, the specific technical solution of which is as follows: S1, Delineate the inspection area based on the construction site information and generate an air-ground collaborative inspection plan, and send the inspection plan to the mobile sensing device. S2, after receiving the inspection plan, the mobile sensing device collects multimodal data in the inspection area and transmits the multimodal data to the intelligent decision-making center in real time; S3, control the intelligent decision-making center to perform multimodal fusion analysis on the received multimodal data, and generate a handling strategy; S4, according to the handling strategy, control the hardware device to control the mobile sensing device to perform a review inspection of the rectification area in the inspection area.

[0006] The beneficial effects of the multimodal AI-driven air-ground collaborative construction safety and environmental protection management method provided by this invention are as follows: A collaborative air-ground inspection system is built using mobile sensing devices to achieve full coverage of the construction area and blind-spot-free monitoring of high-risk areas. Through multimodal fusion analysis, traditional manual inspections are transformed into automated data processing workflows, significantly reducing reliance on manpower and costs while improving hazard identification efficiency. An intelligent decision-making center is constructed based on a multimodal large-scale model, enabling millisecond-level response to safety and environmental hazards and improving processing efficiency. Through collaborative air-ground inspection planning, real-time multimodal data acquisition and transmission, intelligent decision-making center fusion analysis to generate response strategies, and the linkage of hardware control and rectification verification processes, standardized management of the entire construction area is achieved.

[0007] Based on the above solution, the present invention can be further improved as follows.

[0008] Furthermore, the mobile sensing device includes at least one quadruped robot and at least one drone; The quadruped robot is equipped with a camera, a gas sensor, an infrared thermal imager, and a radar device. The quadruped robot is used to perform inspection tasks on the ground in the inspection area. The drone is equipped with a camera device and a multispectral imaging system, and is used to perform inspection tasks at high altitude in the inspection area.

[0009] Furthermore, the multimodal fusion analysis is based on a knowledge base of building construction safety standards and an environmental protection standards to conduct risk assessments on unsafe human behaviors, unsafe conditions of objects, and environmental exceedance events, and to generate the aforementioned response strategies.

[0010] Furthermore, controlling the hardware device according to the processing strategy specifically includes: The dust suppression equipment is controlled to execute start and stop commands according to the aforementioned disposal strategy; According to the aforementioned handling strategy, the tower crane limit device is controlled to execute a limit locking command; The audible and visual alarm device is controlled to execute alarm commands according to the aforementioned handling strategy.

[0011] Furthermore, it also includes: Risk event information, handling process data, and review results data are stored in the management database, and risk identification rules are optimized through machine learning algorithms. The risk event information is generated during the multimodal fusion analysis process, the handling process data is generated during the process of controlling hardware devices according to the handling strategy, and the review result data is generated during the review and inspection process of the mobile sensing device. The risk identification rules are used for multimodal fusion analysis.

[0012] 2) In a second aspect, the present invention also provides a multimodal AI-driven air-ground collaborative construction safety and environmental protection management system, the specific technical solution of which is as follows: inspection planning module, perception and transmission module, analysis and decision-making module and closed-loop disposal module; The inspection planning module is used to delineate the inspection area based on the construction site information and generate an air-ground collaborative inspection plan, and send the inspection plan to the mobile sensing device. The sensing and transmission module is used to collect multimodal data in the inspection area through the mobile sensing device after receiving the inspection plan, and to transmit the multimodal data to the intelligent decision-making center in real time. The analysis and decision-making module is used to control the intelligent decision-making center to perform multimodal fusion analysis on the received multimodal data and generate a handling strategy; The closed-loop processing module is used to control the hardware device according to the processing strategy, and to control the mobile sensing device to perform a review and inspection of the rectification area in the inspection area.

[0013] Based on the above solution, the present invention can be further improved as follows.

[0014] Furthermore, the mobile sensing device includes at least one quadruped robot and at least one drone; The quadruped robot is equipped with a camera, a gas sensor, an infrared thermal imager, and a radar device. The quadruped robot is used to perform inspection tasks on the ground in the inspection area. The drone is equipped with a camera device and a multispectral imaging system, and is used to perform inspection tasks at high altitude in the inspection area.

[0015] Furthermore, the multimodal fusion analysis is based on a knowledge base of building construction safety standards and an environmental protection standards to conduct risk assessments on unsafe human behaviors, unsafe conditions of objects, and environmental exceedance events, and to generate the aforementioned response strategies.

[0016] Furthermore, controlling the hardware device according to the processing strategy specifically includes: The dust suppression equipment is controlled to execute start and stop commands according to the aforementioned disposal strategy; According to the aforementioned handling strategy, the tower crane limit device is controlled to execute a limit locking command; The audible and visual alarm device is controlled to execute alarm commands according to the aforementioned handling strategy.

[0017] Furthermore, it also includes: Risk event information, handling process data, and review results data are stored in the management database, and risk identification rules are optimized through machine learning algorithms. The risk event information is generated during the multimodal fusion analysis process, the handling process data is generated during the process of controlling hardware devices according to the handling strategy, and the review result data is generated during the review and inspection process of the mobile sensing device. The risk identification rules are used for multimodal fusion analysis.

[0018] 3) In a third aspect, the present invention also provides a computer device, the computer device including a processor coupled to a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor to enable the computer device to implement any of the above methods.

[0019] 4) In a fourth aspect, the present invention also provides a computer-readable storage medium storing at least one computer program, which is loaded and executed by a processor to enable a computer to implement any of the above methods.

[0020] It should be noted that the beneficial effects of the technical solutions of the second to fourth aspects of the present invention and their corresponding possible implementations can be found in the above description of the technical effects of the first aspect and its corresponding possible implementations, and will not be repeated here. Attached Figure Description

[0021] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart illustrating the steps of a multimodal AI-driven air-ground collaborative construction safety and environmental protection management method according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the visual interactive monitoring interface architecture of a multimodal AI-driven air-ground collaborative construction safety and environmental protection management method according to an embodiment of the present invention. Figure 3 This is a schematic diagram of the large model data processing flow of a multimodal AI-driven air-ground collaborative construction safety and environmental protection management method according to an embodiment of the present invention. Figure 4This is a schematic diagram of the safety and environmental protection collaborative management architecture of a multimodal AI-driven air-ground collaborative construction safety and environmental protection management method according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0023] like Figure 1 As shown in the figure, an embodiment of the present invention provides a method for safety and environmental protection management of air-ground collaborative construction based on multimodal AI, comprising the following steps: S1, delineate the inspection area based on the construction site information and generate an air-ground collaborative inspection plan, and send the inspection plan to the mobile sensing device; S2, after receiving the inspection plan, the mobile sensing device collects multimodal data in the inspection area and transmits the multimodal data to the intelligent decision-making center in real time; S3 controls the intelligent decision-making center to perform multimodal fusion analysis on the received multimodal data and generate a response strategy; S4 controls the hardware equipment according to the handling strategy, and controls the mobile sensing equipment to conduct a review and inspection of the rectification areas in the inspection area.

[0024] The beneficial effects of the multimodal AI-driven air-ground collaborative construction safety and environmental protection management method provided by this invention are as follows: A collaborative air-ground inspection system is built using mobile sensing devices to achieve full coverage of the construction area and blind-spot-free monitoring of high-risk areas. Through multimodal fusion analysis, traditional manual inspections are transformed into automated data processing workflows, significantly reducing reliance on manpower and costs while improving hazard identification efficiency. An intelligent decision-making center is constructed based on a multimodal large-scale model, enabling millisecond-level response to safety and environmental hazards and improving processing efficiency. Through collaborative air-ground inspection planning, real-time multimodal data acquisition and transmission, intelligent decision-making center fusion analysis to generate response strategies, and the linkage of hardware control and rectification verification processes, standardized management of the entire construction area is achieved.

[0025] It should be noted that, for ease of understanding, the technical terms used in this solution will be explained one by one, and will not be repeated hereafter: Construction site information: This refers to the basic data set used to delineate the inspection area, including Building Information Modeling (BIM) model data, real-time construction progress data, historical risk event distribution data, and high-risk work area location information. This solution integrates construction site information through a 3D visualization interactive platform, providing input for intelligent planning of the inspection area.

[0026] Inspection area: refers to a specific spatial area within the construction site, demarcated by safety and environmental management personnel based on factors such as high-altitude hazardous work areas, hazardous chemical storage areas, and the movement trajectory of pollution sources. This plan implements hierarchical management of the inspection area, with different inspection frequencies and deployment densities of mobile sensing equipment corresponding to different risk levels.

[0027] Air-ground collaboration: This refers to a joint working mechanism in which quadruped robots perform inspection tasks on the ground and drones perform inspection tasks at high altitudes, complementing each other's data. This solution uses 5G and the BeiDou satellite navigation system to achieve real-time communication between air and ground equipment, constructing a three-dimensional monitoring network covering the ground and high altitudes to ensure no blind spots in the inspection area.

[0028] Inspection plan: refers to a task planning document that includes the ground inspection path of the quadruped robot cluster, the high-altitude scanning route of the UAV, the data collection type, and the transmission timing requirements. It is sent to the mobile sensing device via 5G and the Beidou satellite navigation system to guide the device to perform collaborative inspection tasks.

[0029] Mobile sensing devices: These refer to data acquisition terminals equipped with multiple types of sensors and capable of autonomous movement, including quadruped robots and drones.

[0030] Multimodal data refers to a collection of multimodal data with different data formats and semantic information collected by mobile sensing devices, including visual image data, acoustic information data, gas concentration data, temperature data, and 3D point cloud data.

[0031] Intelligent decision-making hub: refers to the core computing unit that deploys multimodal large models and runs multimodal fusion analysis algorithms, and is usually composed of edge computing nodes and cloud computing platforms.

[0032] Multimodal fusion analysis refers to the technical process of unifying and integrating visual recognition, acoustic feature extraction, and sensor data analysis results, and establishing cross-dimensional correlation analysis through deep learning networks. This solution's multimodal fusion analysis is based on a knowledge base of building construction safety regulations and an environmental protection regulations knowledge base, jointly assessing unsafe human behaviors, unsafe conditions of objects, and environmental exceedance events.

[0033] Response strategy: refers to the set of response measures formulated for identified safety and environmental risk events, including hardware equipment linkage control instructions, graphic and textual rectification work order content, and navigation path information for responsible work teams.

[0034] Hardware equipment: refers to the construction site facilities that receive and execute physical actions in response to management strategies, including dust suppression equipment, tower crane limit devices, and audible and visual alarm devices. Digital control signals are sent directly to the hardware equipment through the intelligent decision-making center, which can realize the automated handling of risk events.

[0035] Rectification Area: This refers to the specific spatial location where a safety and environmental risk event occurred, which is identified by the intelligent decision-making center through spatiotemporal correlation analysis. In this solution, mobile sensing devices can be controlled to execute an enhanced inspection mode on the rectification area, collecting and verifying data to validate the rectification effect.

[0036] Verification inspection: This refers to the verification inspection of the rectified area by mobile sensing devices after the rectification task is completed. Through high-frequency data collection and multimodal fusion analysis, the differences between the data before and after the rectification are compared to confirm whether the risks have been eliminated.

[0037] Quadruped robots: These are mobile sensing devices with four mechanical legs that can walk autonomously in unstructured terrain. They are equipped with cameras, gas sensors, infrared thermal imagers, and radar equipment to perform tasks such as monitoring the behavior of ground personnel, diagnosing equipment status, and detecting pollutant concentrations.

[0038] Drones: These are unmanned aerial vehicles that use rotors to provide lift and can take off and land vertically. They are equipped with cameras and multispectral imaging systems for high-altitude work safety monitoring, dust diffusion boundary identification, and bare soil coverage detection.

[0039] Camera equipment: refers to sensor components used to capture optical image data, including high-definition cameras and high-precision cameras. In this solution, the camera equipment collects images of personnel behavior, equipment appearance, and panoramic images of the work area, providing input data for the visual recognition algorithm.

[0040] Gas sensors: These are sensing elements used to detect the concentration of pollutants in the air, including particulate matter, volatile organic compounds, and harmful gases. In this solution, the gas sensors monitor environmental parameters at the construction site in real time, triggering environmental event identification when data exceeds standards.

[0041] Infrared thermal imager: A device that generates a temperature distribution image by detecting the intensity of infrared radiation from an object. This solution uses an infrared thermal imager to identify abnormal overheating in equipment and potential heating hazards from damaged cables, assisting in determining unsafe conditions of objects.

[0042] Radar equipment refers to sensors that use electromagnetic waves to detect the distance, speed, and angle of targets, including lidar and millimeter-wave radar. In this solution, lidar constructs a 3D point cloud map of the surrounding environment, while millimeter-wave radar detects the movement of obstacles, providing data support for the quadruped robot's autonomous obstacle avoidance.

[0043] Multispectral imaging system: refers to an imaging device capable of simultaneously acquiring images of multiple specific wavelength ranges. This solution's multispectral imaging system identifies bare soil cover status, dust diffusion boundaries, and vegetation cover, improving the accuracy of identifying environmental pollution events.

[0044] The Construction Safety Code Knowledge Base refers to a structured database that stores mandatory standards, industry norms, and enterprise safety management systems in the construction field. It serves as the basis for multimodal fusion analysis and is used to determine whether unsafe human behaviors and unsafe conditions of objects meet safety requirements.

[0045] Environmental protection regulations knowledge base: refers to a structured database that stores environmental protection regulations, pollutant emission standards, and environmental management requirements for construction sites. It serves as a benchmark for identifying environmental exceedance incidents and provides pollutant concentration thresholds and compliance judgment rules.

[0046] Unsafe acts by humans: refers to behaviors of construction workers that violate construction safety regulations during operations, including not wearing safety helmets, not wearing safety belts, operating hoisting equipment in violation of regulations, and staying in dangerous areas. Images of personnel are collected by video devices, and unsafe acts are automatically identified and risk levels are assessed through multimodal fusion analysis.

[0047] Unsafe conditions of objects: refers to the operating conditions of construction equipment, facilities, and materials that do not meet safety standards, including equipment overheating, cable damage, structural cracks, and missing protective devices. Unsafe conditions of objects are identified by collecting equipment status data through infrared thermal imagers and camera devices and performing multimodal fusion analysis.

[0048] Environmental exceedance events: These refer to events where environmental parameters at the construction site exceed the thresholds specified in the environmental protection standard knowledge base. These include dust concentration exceeding standards, noise pollution exceeding standards, and light pollution exceeding standards. Environmental parameters are monitored using gas sensors and multispectral imaging systems, and environmental exceedance events are determined through multimodal fusion analysis.

[0049] Risk assessment refers to the process of quantitatively analyzing the severity of identified safety and environmental risk events. In this plan, risk assessment is based on a pre-defined L1 to L4 level classification standard, combined with factors such as event type, location, and scope of impact, to output dynamic risk level assessment results.

[0050] Dust suppression equipment: refers to mechanical equipment used to suppress dust pollution at construction sites, including fog cannons and spray systems. The operation of dust suppression equipment can be remotely controlled according to the start and stop instructions in the disposal strategy to achieve rapid dust suppression.

[0051] Tower crane limit device: refers to a safety protection device used to limit the operating range of a tower crane. According to the limit locking command in the disposal strategy, the tower crane with collision risk is locked to prevent accidents.

[0052] Audible and visual alarm devices: These are devices that issue warnings through sound and light signals. According to the alarm instructions in the handling strategy, they are activated when a risk event occurs to remind on-site personnel to evacuate or take protective measures.

[0053] Risk event information: refers to structured data describing safety and environmental risk events, including event type, time of occurrence, location of occurrence, risk level, related equipment and related personnel, which is generated by the intelligent decision-making center during multimodal fusion analysis.

[0054] Process data: refers to dynamic data that records the execution process of risk disposal, including instruction sending time, work order push time, responsible personnel receiving status, hardware equipment response status, and rectification task execution progress. It is automatically generated during the process of controlling hardware equipment and dispatching rectification work orders.

[0055] Verification result data: refers to conclusive data that verifies the effectiveness of rectification measures, including verification time, verification data and verification results. It is collected by mobile sensing devices after rectification is completed and generated after analysis and comparison by the intelligent decision-making center.

[0056] Management Database: This refers to a database system used to store risk event information, handling process data, and review result data. It adopts a hybrid architecture of relational database and time-series database and supports migration of safety management models across projects.

[0057] Machine learning algorithms refer to algorithmic frameworks that train models using historical data and automatically optimize parameters, including deep learning algorithms, association rule mining algorithms, and graph neural network algorithms. In this solution, machine learning algorithms continuously optimize risk identification rules using data from the handling process and review results, thereby improving the accuracy of multimodal fusion analysis.

[0058] Risk identification rules refer to the set of judgment criteria used by the multimodal large model to identify safety and environmental risk events, including feature parameter thresholds, identification logic, and risk level mapping relationships. The risk identification rules in this solution are applied in the multimodal fusion analysis process and are continuously iteratively optimized through machine learning algorithms.

[0059] In another embodiment of this solution, S1 is specifically implemented as follows: Figure 2 This is a schematic diagram of the architecture of the visual interactive monitoring interface, such as... Figure 2 As shown, the BIM model and real-time construction scene data are integrated through a 3D visualization interactive platform. Safety and environmental management personnel can then delineate inspection areas, including high-altitude hazardous work areas, hazardous chemical storage areas, and dynamic pollution sources, on the visualization interactive platform interface.

[0060] Based on pre-defined risk level assessment standards and the current construction phase characteristics of the inspection area, an air-ground collaborative inspection plan is generated, including the inspection scanning route of mobile sensing devices. The mobile sensing devices include at least one quadruped robot and at least one drone. The quadruped robot performs inspection tasks on the ground within the inspection area, while the drone performs inspection tasks from above. The mobile sensing device inspection scanning route includes the quadruped robot's inspection path and the drone's high-altitude scanning flight path.

[0061] The inspection plan transmits encrypted data packets to the mobile sensing device via dual channels: the fifth-generation mobile communication network (5G) and the BeiDou satellite navigation system. 5G refers to wireless communication technology with high speed, low latency, and massive connectivity, while BeiDou satellite navigation system refers to a global satellite navigation system that provides high-precision positioning services.

[0062] In another embodiment of this solution, S2 is specifically implemented as follows: After receiving the inspection plan, the mobile sensing device initiates an autonomous inspection mode according to the inspection scanning route in the plan. Specifically, the quadruped robot moves along the inspection path on the ground of the inspection area, while the drone flies at high altitude in the inspection area according to the high-altitude scanning route.

[0063] The quadruped robot is equipped with cameras, gas sensors, an infrared thermal imager, and radar. The cameras capture images of human behavior and the appearance of the equipment; the gas sensors detect the concentration of particulate matter and volatile organic compounds; the infrared thermal imager identifies overheating anomalies in the equipment and overheating from damaged cables; the radar equipment includes lidar and millimeter-wave radar. LiDAR acquires 3D point cloud data of the surrounding environment and builds local maps, while millimeter-wave radar detects obstacle distances and movement speeds.

[0064] The drone is equipped with a camera device and a multispectral imaging system. The camera device is used to capture panoramic images of the high-altitude work area and the operating status of the hoisting equipment; the multispectral imaging system is used to identify the state of bare soil coverage and the boundary of dust diffusion.

[0065] Multimodal data, including visual images, acoustic information, and environmental parameters, collected by quadruped robots and drones, is transmitted in real time to the intelligent decision-making center via 5G and BeiDou short message service. Timestamps and spatial coordinates are appended during transmission. The multimodal data refers to a heterogeneous collection of information from different sensor types and with different data formats carried by the quadruped robots and drones.

[0066] The advantage is that by building an air-ground collaborative inspection system through quadruped robots and drones, the entire construction area can be covered and high-risk areas can be monitored without blind spots. It also supports the real-time collection of safety and environmental management data in complex scenarios such as near edges and at high altitudes.

[0067] In another embodiment of this solution, S3 is specifically implemented as follows: Figure 3 This is a schematic diagram of the large model data processing flow, such as... Figure 3 As shown, the intelligent decision-making center deploys a multimodal large model to perform multimodal fusion analysis on the received multimodal data. This multimodal large model is based on a knowledge base of building construction safety regulations and an environmental protection regulations knowledge base. It uses deep learning algorithms to extract visual, acoustic, and sensor data features to dynamically assess unsafe human behavior, unsafe conditions of objects, and environmental exceedance events, thereby obtaining risk event information.

[0068] Specifically, unsafe acts by people can be defined as personnel not wearing safety equipment, unsafe conditions of equipment can be defined as abnormal equipment operation, and environmental exceedance events can be defined as events where pollutant concentrations exceed standards. These categories are determined based on the knowledge bases for construction safety regulations and environmental protection regulations.

[0069] The dynamic risk assessment classifies risk levels from L1 to L4, with L1 representing low-risk hazards and L4 representing significant safety risks or severe environmental pollution. The multimodal large-scale model uses spatiotemporal correlation analysis to pinpoint non-compliant work teams or equipment failure points, generating a handling strategy that includes hardware control commands and graphic rectification work orders. Spatiotemporal correlation analysis refers to the logical reasoning process combining the time, location, and trajectory information of mobile sensing devices. Specifically, it involves establishing a motion trajectory model based on the physical location of the mobile sensing devices and multimodal data timestamps, and then filtering out work teams or equipment failure points that are logically correlated in time.

[0070] The advantages lie in integrating multimodal sensors and intelligent analysis algorithms to transform traditional manual inspections into automated data radar equipment and processes. This significantly reduces reliance on manpower and costs while improving hazard identification efficiency, forming a comprehensive regulatory capability encompassing environmental parameter monitoring, equipment status diagnosis, and personnel behavior control. An intelligent decision-making center is built based on a multimodal large-scale model, enabling second-level response and radar equipment intervention for safety and environmental hazards. The multimodal large-scale model is trained using integrated construction safety and environmental protection standards to establish standardized risk identification.

[0071] In another embodiment of this solution, S4 is specifically implemented as follows: Once the disposal strategy is generated, it triggers the linkage control of hardware devices, controlling the hardware devices according to the hardware device control instructions in the disposal strategy. Specifically, this includes: sending start / stop commands to the dust suppression equipment and controlling the dust suppression equipment to execute the start / stop commands; sending limit lock commands to the tower crane limit devices and controlling the tower crane limit devices to execute the limit lock commands; and sending alarm commands to the audible and visual alarm devices and controlling the audible and visual alarm devices to execute the alarm commands.

[0072] Simultaneously, the text and image rectification work order is pushed to the handheld terminal device of the responsible team. The work order content includes a risk description, rectification requirements, and navigation route information. Based on the handling strategy, the hardware device linkage control is triggered, and the text and image rectification work order is pushed to the handheld terminal device to obtain the handling process data.

[0073] After receiving the review instruction, the mobile sensing device activates the enhanced inspection mode. Specifically, the quadruped robot performs high-frequency scanning of the rectification area, while the drone takes multi-angle photos of the rectification area, collecting review data and transmitting it back to the intelligent decision-making center. The enhanced inspection mode refers to a specialized inspection method that increases the frequency and scope of data collection.

[0074] The intelligent decision-making center compares data before and after rectification to verify whether pollutant concentrations meet standards, equipment status has returned to normal, and personnel behavior is compliant, thus obtaining verification results. These verification results, confirmed by comparing data from both periods, represent the conclusions regarding the effectiveness of the rectification efforts. These results can be used as new training samples to input into a multimodal large-scale model, continuously optimizing risk identification rules.

[0075] The advantage is that by automating task assignment, linking intelligent devices, and verifying rectification results, the bottleneck of traditional manual reporting and multi-level approval processes is broken, which improves the response time of the entire process of risk identification, analysis, and disposal to the minute level, and greatly enhances the dynamic management and control efficiency of the construction site.

[0076] Furthermore, the mobile sensing device includes at least one quadruped robot and at least one drone; The quadruped robot is equipped with a camera, gas sensor, infrared thermal imager and radar equipment. The quadruped robot is used to perform inspection tasks on the ground in the inspection area. The drone is equipped with a camera and a multispectral imaging system, and is used to perform inspection tasks at high altitudes in the inspection area.

[0077] Furthermore, multimodal fusion analysis, based on the knowledge base of building construction safety standards and environmental protection standards, conducts risk assessments on unsafe human behaviors, unsafe conditions of objects, and environmental exceedance events, and generates response strategies.

[0078] Furthermore, the hardware devices are controlled according to the handling strategy, specifically including: Control the dust suppression equipment to execute start and stop commands according to the disposal strategy; According to the handling strategy, control the tower crane limit device to execute the limit locking command; The alarm device is controlled to execute alarm commands according to the handling strategy.

[0079] Furthermore, it also includes: Risk event information, handling process data, and review results data are stored in the management database, and risk identification rules are optimized through machine learning algorithms. Risk event information is generated during the multimodal fusion analysis process; handling process data is generated during the process of controlling hardware equipment according to the handling strategy; and review result data is generated during the review and inspection process of mobile sensing equipment. Risk identification rules are used for multimodal fusion analysis. Figure 4 This is a schematic diagram of a collaborative management and control architecture for safety and environmental protection, such as... Figure 4 As shown, it also includes: All risk event information, handling process data, and review result data are stored in the management database. Risk event information is generated through multimodal fusion analysis, including event type, occurrence time, location, risk level, associated equipment, and associated personnel. Handling process data is generated during the control of hardware equipment according to the handling strategy. Review result data is generated during the review and inspection process of mobile sensing devices, including review time, review data, and verification results. The management database uses a relational database to store structured information and a time-series database to store continuous monitoring data. Machine learning algorithms are used to perform multi-dimensional correlation analysis on historical data in the management database. Hazard types, spatiotemporal distribution, and responsible entity dimensions are extracted and used to construct a safety and environmental protection knowledge graph, providing early warnings of the probability of environmental pollution exceeding standards or safety risks within the next 12 hours. The knowledge graph is stored in a graph database format, with nodes representing risk events, equipment, personnel, and location entities, and edges representing causal relationships, temporal relationships, and responsibility relationships. Full-cycle management data is encapsulated into a transferable experience base, supporting the reuse of safety management models across projects and promoting the upgrade of construction safety and environmental management from passive response to proactive prevention. Among them, knowledge graph refers to the semantic network structure automatically extracted from a large amount of construction management data through machine learning algorithms.

[0080] The advantage is that through continuous iterative optimization of machine learning, the impact of differences in management experience on inspection results can be effectively eliminated, ensuring the standardization and consistency of hazard identification, level assessment and rectification strategy output, and comprehensively improving the scientific nature and accuracy of regulatory decisions.

[0081] In the above embodiments, although the steps are numbered S1, S2, etc., they are only specific embodiments given by the present invention. Those skilled in the art can adjust the execution order of S1, S2, etc. according to the actual situation, and these situations are also within the protection scope of the present invention. It can be understood that in some embodiments, some or all of the above embodiments may be included.

[0082] Furthermore, the acquisition process of the data involved in this application follows the principles of legality, legitimacy, and necessity. Based on obtaining the explicit authorization and consent of the user, only the minimum necessary information required to achieve the purpose is collected, and data security protection obligations are fulfilled in accordance with the law.

[0083] This invention also provides a multimodal AI-driven air-ground collaborative construction safety and environmental protection management system, the specific technical solution of which includes: an inspection planning module, a perception and transmission module, an analysis and decision-making module, and a closed-loop disposal module; The inspection planning module is used to delineate the inspection area based on the construction site information and generate an air-ground collaborative inspection plan, which is then sent to the mobile sensing device. The sensing and transmission module is used to collect multimodal data in the inspection area through the mobile sensing device after receiving the inspection plan, and transmit the multimodal data to the intelligent decision-making center in real time. The analysis and decision-making module is used to control the intelligent decision-making center to perform multimodal fusion analysis on the received multimodal data and generate disposal strategies; The closed-loop handling module is used to control hardware devices according to the handling strategy, and to control mobile sensing devices to conduct a review and inspection of the rectification areas in the inspection area.

[0084] It should be noted that the beneficial effects of the multimodal AI-driven air-ground collaborative construction safety and environmental protection management system provided in the above embodiments are the same as those of the multimodal AI-driven air-ground collaborative construction safety and environmental protection management method described above, and will not be repeated here. Furthermore, the system provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to the actual situation to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiments, and will not be repeated here.

[0085] like Figure 5As shown, an embodiment of the present invention provides a computer device 300, which includes a processor 320 coupled to a memory 310. The memory 310 stores at least one computer program 330, which is loaded and executed by the processor 320 to enable the computer device 300 to implement any of the above-described methods. Specifically: The computer device 300 can vary considerably due to differences in configuration or performance. It may include one or more processors 320 (Central Processing Units, CPUs) and one or more memories 310. The memories 310 store at least one computer program 330, which is loaded and executed by the processors 320 to enable the computer device 300 to implement the multimodal AI-driven air-ground collaborative construction safety and environmental protection management method provided in the above embodiment. Of course, the computer device 300 may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. It may also include other components for implementing device functions, which will not be elaborated upon here.

[0086] An embodiment of the present invention provides a computer-readable storage medium storing at least one computer program, which is loaded and executed by a processor to enable a computer to implement any of the above-described methods.

[0087] Alternatively, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, a floppy disk, and an optical data storage device, etc.

[0088] In an exemplary embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the above-described multimodal AI-driven air-ground collaborative construction safety and environmental protection management methods.

[0089] It should be noted that the terms "first," "second," etc., used in the specification of this application are used to distinguish similar objects and represent a limitation on a specific order or sequence. Where appropriate, the order of use for similar objects can be interchanged so that the embodiments of this application described herein can be implemented in an order other than that shown in the figures or description.

[0090] Those skilled in the art will recognize that this invention can be implemented as a system, method, or computer program product. Therefore, this disclosure can be specifically implemented in the following forms: it can be entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, the invention can also be implemented as a computer program product contained in one or more computer-readable media, which includes computer-readable program code.

[0091] Any combination of one or more computer-readable media may be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0092] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for safety and environmental protection management of air-ground collaborative construction driven by multimodal AI, characterized in that, include: S1, Delineate the inspection area based on the construction site information and generate an air-ground collaborative inspection plan, and send the inspection plan to the mobile sensing device. S2, after receiving the inspection plan, the mobile sensing device collects multimodal data in the inspection area and transmits the multimodal data to the intelligent decision-making center in real time; S3, control the intelligent decision-making center to perform multimodal fusion analysis on the received multimodal data, and generate a handling strategy; S4, according to the handling strategy, control the hardware device to control the mobile sensing device to perform a review inspection of the rectification area in the inspection area.

2. The method for safety and environmental protection management of air-ground collaborative construction based on multimodal AI as described in claim 1, characterized in that, The mobile sensing device includes at least one quadruped robot and at least one drone; The quadruped robot is equipped with a camera, a gas sensor, an infrared thermal imager, and a radar device. The quadruped robot is used to perform inspection tasks on the ground in the inspection area. The drone is equipped with a camera device and a multispectral imaging system, and is used to perform inspection tasks at high altitude in the inspection area.

3. The method for safety and environmental protection management of air-ground collaborative construction based on multimodal AI as described in claim 1, characterized in that, The multimodal fusion analysis is based on the knowledge base of building construction safety standards and environmental protection standards. It conducts risk assessments on unsafe human behaviors, unsafe conditions of objects, and environmental exceedance events, and generates the aforementioned disposal strategies.

4. The method for safety and environmental protection management of air-ground collaborative construction based on multimodal AI as described in claim 1, characterized in that, The step of controlling the hardware device according to the processing strategy specifically includes: The dust suppression equipment is controlled to execute start and stop commands according to the aforementioned disposal strategy; According to the aforementioned handling strategy, the tower crane limit device is controlled to execute a limit locking command; The audible and visual alarm device is controlled to execute alarm commands according to the aforementioned handling strategy.

5. The method for safety and environmental protection management of air-ground collaborative construction based on multimodal AI as described in claim 1, characterized in that, Also includes: Risk event information, handling process data, and review results data are stored in the management database, and risk identification rules are optimized through machine learning algorithms. The risk event information is generated during the multimodal fusion analysis process, the handling process data is generated during the process of controlling hardware devices according to the handling strategy, and the review result data is generated during the review and inspection process of the mobile sensing device. The risk identification rules are used for multimodal fusion analysis.

6. A multimodal AI-driven air-ground collaborative construction safety and environmental protection management system, characterized in that, include: The system includes an inspection planning module, a sensing and transmission module, an analysis and decision-making module, and a closed-loop processing module. The inspection planning module is used to delineate the inspection area based on the construction site information and generate an air-ground collaborative inspection plan, and send the inspection plan to the mobile sensing device. The sensing and transmission module is used to collect multimodal data in the inspection area through the mobile sensing device after receiving the inspection plan, and to transmit the multimodal data to the intelligent decision-making center in real time. The analysis and decision-making module is used to control the intelligent decision-making center to perform multimodal fusion analysis on the received multimodal data and generate a handling strategy; The closed-loop processing module is used to control the hardware device according to the processing strategy, and to control the mobile sensing device to perform a review and inspection of the rectification area in the inspection area.

7. A multimodal AI-driven air-ground collaborative construction safety and environmental protection management system according to claim 6, characterized in that, The mobile sensing device includes at least one quadruped robot and at least one drone; The quadruped robot is equipped with a camera, a gas sensor, an infrared thermal imager, and a radar device. The quadruped robot is used to perform inspection tasks on the ground in the inspection area. The drone is equipped with a camera device and a multispectral imaging system, and is used to perform inspection tasks at high altitude in the inspection area.

8. The air-ground collaborative construction safety and environmental protection management system based on multimodal AI as described in claim 6, characterized in that, The multimodal fusion analysis is based on the knowledge base of building construction safety standards and environmental protection standards. It conducts risk assessments on unsafe human behaviors, unsafe conditions of objects, and environmental exceedance events, and generates the aforementioned disposal strategies.

9. A computer device, characterized in that, The computer device includes a processor coupled to a memory, the memory storing at least one computer program, which is loaded and executed by the processor to enable the computer device to implement a multimodal AI-driven air-ground collaborative construction safety and environmental protection management method as described in any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to enable the computer to implement a multimodal AI-driven air-ground collaborative construction safety and environmental protection management method as described in any one of claims 1 to 5.

Images