Construction site safety hazard three-dimensional positioning and intelligent disposal method and system

Abstract

This invention discloses a method and system for three-dimensional positioning and intelligent handling of safety hazards at construction sites. It relates to the technical field of calibrating the intrinsic parameters of monitoring cameras at construction sites and automatically calibrating the extrinsic parameters of cameras using markers with known world coordinates. A simplified geometric model of the scene, including floor plans and edge protection surfaces, is constructed. The monitoring video stream is sampled and thumbnailed for dimensionality reduction. Multiple thumbnails are stitched together into a grid image, which is then input into a large visual model for batch pre-screening of safety risks, extracting only high-resolution original frames containing potential hazards. This invention transforms the two-dimensional hazard information output from the large visual model into three-dimensional real coordinates through camera calibration and ray back-projection geometric calculations, achieving precise spatial location determination of hazards. ArUco QR code marking is used for automatic calibration of camera extrinsic parameters, improving calibration accuracy and supporting automatic recalibration. Voxelized spatial analysis can automatically identify blind spots during inspections, providing a basis for monitoring optimization.

Description

Technical Field

[0001] This invention relates to the field of technology, and in particular to a method and system for three-dimensional positioning and intelligent handling of safety hazards at construction sites. Background Technology

[0002] Construction site safety management is a core challenge in the construction industry. Traditional safety inspections rely primarily on regular manual patrols, which suffer from shortcomings such as incomplete coverage, subjective judgment, and poor timeliness in reporting and handling problems. With the development of intelligent technologies, computer vision-based construction site safety monitoring is gradually being applied.

[0003] Currently, mainstream technologies can be broadly categorized into three types. The first type is safety monitoring based on target detection models, using models such as YOLO and Faster R-CNN to detect fixed categories of targets, such as helmet wearing and reflective vest wearing. This method requires separately labeled training data for each type of hazard, cannot identify non-standardized hazards outside the training set, and has insufficient generalization ability. The second type is open-scene safety analysis based on large-scale visual models. It leverages the semantic understanding capabilities of large-scale visual models to identify hazards in any scene, without being limited by preset categories. However, it can only output hazard descriptions at the two-dimensional image level, unable to obtain the precise location of hazards in real three-dimensional space. In multi-camera scenarios, it cannot distinguish between recurring hazards and blind spots during inspections, nor does it possess spatial hazard statistics and risk analysis capabilities. The third type is 3D localization schemes based on SLAM, mainly used for the self-localization and map reconstruction of mobile cameras. Fixed cameras at construction sites do not require this localization logic, and monocular SLAM suffers from scale ambiguity and high computational complexity, making it unsuitable for fixed monitoring scenarios.

[0004] In summary, existing technologies have failed to effectively integrate the strong semantic understanding capabilities of large visual models with their three-dimensional spatial geometric positioning capabilities, and lack an integrated technical solution that can achieve precise three-dimensional positioning of potential hazards, spatial risk analysis, and intelligent handling throughout the entire process. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a method and system for three-dimensional positioning and intelligent handling of safety hazards at construction sites. The technical solution adopted is as follows: A three-dimensional positioning and intelligent handling method for safety hazards at construction sites includes the following steps: Step 1, the spatial benchmark construction step, includes calibrating the intrinsic parameters of the monitoring cameras at the construction site, automatically calibrating the extrinsic parameters of the cameras by deploying markers with known world coordinates, and constructing a simplified geometric model of the scene that includes floor plans and edge protection surfaces. Step 2: Sample and perform thumbnail dimensionality reduction on the surveillance video stream. Stitch the thumbnails of multiple frames into a grid image and input it into the visual large model for batch security risk pre-screening. Only high-resolution original frames containing potential risks are extracted. Step 3: Input the high-resolution original frame into the visual large model, and guide the output of structured hazard description data through prompts, which includes at least the hazard category, severity level, and normalized pixel coordinates of the core hazard area in the image; Step 4: Using the camera's intrinsic and extrinsic parameters, normalized pixel coordinates are used to construct a three-dimensional spatial ray through distortion correction and inverse projection transformation. The intersection of the three-dimensional spatial ray with the simplified geometric model of the scene is calculated. Based on the spatial constraint logic, the effective intersection point closest to the camera is selected as the real-world three-dimensional coordinates of the hazard, and the floor to which the hazard belongs is determined accordingly. Step 5: Voxelize the construction space and calculate the spatial coverage based on the camera's view frustum to identify uncovered inspection blind spots; at the same time, based on the real-world three-dimensional coordinates and severity level of the hidden dangers, accumulate and generate a spatial risk heat map in the three-dimensional voxel mesh using a time decay function. Step 6: The large language model agent receives the 3D location results and risk score of the hidden danger, generates a disposal decision, and executes autonomous closed-loop actions such as alarm, work order dispatch, blind spot supplementary inspection or evidence consolidation by calling the preset physical tool interface.

[0006] Optionally, the specific method for automatically calibrating camera extrinsic parameters in step 1 is as follows: ArUco QR code markers with unique coded IDs are deployed at fixed locations on the construction site, and the world coordinates of their center points are pre-stored. After the camera acquires images, it automatically detects ArUco markers in the image and obtains their sub-pixel-level image coordinates. The PnP algorithm is used to establish a 2D-3D point pair mapping relationship to calculate the camera extrinsic parameters. The root mean square value of the back projection error is checked, and the marker pixel offset is continuously monitored during system operation. When the offset exceeds the preset threshold, the recalibration process is automatically triggered.

[0007] Optionally, the method for constructing the simplified geometric model of the scene in step 1 is: The system directly extracts floor slab elevations, external wall boundary coordinates, and scaffolding erection parameters from Building Information Modeling (BIM) and automatically generates a set of plane equations for ray intersection and corresponding physical boundary constraint polygons.

[0008] Optionally, the batch security risk pre-screening method in step 2 is: The video stream is sampled at preset time intervals, and the sampled frames are scaled proportionally to fixed low-resolution thumbnails. Multiple consecutive thumbnails are stitched together in a matrix form into a single grid image. The grid image is fed into the visual large model at once, and the model is required to return the location numbers of thumbnails with security risks in a structured list format. Only the original high-resolution video frames corresponding to the thumbnails with the corresponding numbers are sent to step 3 for depth analysis.

[0009] Optionally, the method for solving the real-world 3D coordinates of the hidden danger in step 4 is: The pixel coordinates of potential hazards are corrected based on the camera distortion coefficients. The inverse matrix of the camera intrinsic parameter matrix is ​​used to convert the corrected pixel coordinates into ray directions in the normalized camera coordinate system. Combining the rotation matrix and translation vector in the camera extrinsic parameters, the normalized ray directions are transformed to the world coordinate system, and the world space ray equation is constructed with the camera optical center as the ray origin. The equations of each plane in the simplified geometric model of the scene are traversed to calculate the intersection points and distance parameters of the ray with each plane. Intersection points located outside the geometric boundaries of the planes and intersection points with negative distance parameters are eliminated. The effective intersection point with the smallest distance parameter is selected as the three-dimensional coordinates of the potential hazard to conform to the physical imaging law of near-object occlusion.

[0010] Optionally, the method for generating the spatial risk heatmap by accumulating the time decay function in step 5 is as follows: For each identified hazard, an initial risk weight is assigned based on its severity level; this risk weight is then injected into the spatial voxel corresponding to the hazard's coordinates; the risk values ​​of all voxels are dynamically updated over time according to an exponential decay law, with the update formula being: ; It is the real-time risk value of the voxel at the current time t. It is the initial risk weight when a hazard is identified, determined by the severity level of the hazard. It is a natural constant. This is the risk time decay coefficient, which takes a value greater than 0. The larger the coefficient, the faster the risk decays. t is the current calculation time. This is the moment when the hidden danger is first identified and a risk value is assigned; the updated voxel risk value is normalized and mapped to a color gradient, and a risk heat map is generated by rendering it on the 3D building model.

[0011] Optionally, the method for identifying uncovered inspection blind spots in step 5 is: The three-dimensional space of the construction area is divided into a voxel grid with a preset side length; a three-dimensional view frustum is constructed for each camera based on its intrinsic and extrinsic parameters; the voxel grid is traversed to determine whether the center point of each voxel falls within the view frustum of at least one camera; voxels that do not fall within the view frustum of any camera but are located within the effective space of the building are marked as inspection blind spots.

[0012] Optionally, when a blind spot is identified in step 5, the intelligent agent calls the PTZ camera control tool to automatically adjust the PTZ angle to cover the blind spot for supplementary image acquisition and hazard analysis.

[0013] Optionally, in step 6, for the location of the hidden danger for which a rectification work order has been generated, when the preset rectification period arrives, the intelligent agent automatically schedules the monitoring camera in the corresponding area to re-collect images and calls the visual big model to verify the rectification status, forming a closed-loop verification.

[0014] A 3D Location and Intelligent Handling System for Safety Hazards at Construction Sites, used to realize 3D location and intelligent handling methods for safety hazards at construction sites. The system includes: The camera calibration and geometric modeling module is used to perform camera intrinsic parameter calibration, fully automatic extrinsic parameter calibration and drift self-healing based on ArUco tags, and scene simplification geometric model initialization; The video stream scheduling and preprocessing module is used to access multiple video streams and perform efficient keyframe extraction based on thumbnail grid stitching and visual large model initial screening. The visual large model semantic analysis module is used to perform high-resolution analysis on keyframes and output structured information including semantic descriptions of potential hazards and normalized pixel coordinates. The 3D spatial positioning and calculation module is used to perform ray back projection algorithm based on camera intrinsic and extrinsic parameters, and combine scene geometric model constraints to calculate the 3D world coordinates and floor assignment of potential hazards; The voxel spatial analysis and heatmap module is used to construct voxel grids to calculate the multi-camera field of view coverage to identify blind spots, and generate a three-dimensional risk heatmap based on the time decay mechanism. The intelligent agent decision-making and handling module is used to receive 3D location data of potential hazards and risk data, drive the large language model to perform handling reasoning, and realize alarm, work order dispatch and equipment linkage control through tool call interface.

[0015] In summary, the present invention has at least one of the following beneficial technical effects: This invention provides a method and system for three-dimensional positioning and intelligent handling of safety hazards at construction sites. Through camera calibration and ray back-projection geometric calculations, the two-dimensional hazard information output from the large visual model is transformed into three-dimensional real coordinates, achieving precise spatial location determination of hazards. ArUco QR code marking is used for automatic camera extrinsic parameter calibration, improving calibration accuracy and supporting automatic recalibration. Voxelized spatial analysis can automatically identify blind spots during inspections, providing a basis for monitoring optimization. A three-dimensional risk heat map is generated based on time decay weighting, intuitively presenting the spatial risk distribution. Thumbnail grid stitching and filtering reduces the cost of accessing the large visual model. Relying on intelligent agents, a closed loop of hazard alarm, work order dispatch, evidence consolidation, and review is achieved, improving the automation and intelligence level of construction site safety management. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the method for three-dimensional positioning and intelligent handling of safety hazards at construction sites according to the present invention.

[0017] Figure 2 This is a heat map of spatial risks at the construction site according to a specific embodiment of the present invention; Figure 3 This is a schematic diagram of a work order for rectifying safety hazards at a construction site within a specified time, according to a specific embodiment of the present invention. Detailed Implementation

[0018] The present invention will be further described in detail below with reference to the accompanying drawings.

[0019] This invention discloses a method and system for three-dimensional positioning and intelligent handling of safety hazards at construction sites.

[0020] Reference Figures 1-3 Example 1, a method for three-dimensional location and intelligent handling of safety hazards at construction sites, includes the following steps: Step 1, the spatial benchmark construction step, includes calibrating the intrinsic parameters of the monitoring cameras at the construction site, automatically calibrating the extrinsic parameters of the cameras by deploying markers with known world coordinates, and constructing a simplified geometric model of the scene that includes floor plans and edge protection surfaces. Step 2: Sample and perform thumbnail dimensionality reduction on the surveillance video stream. Stitch the thumbnails of multiple frames into a grid image and input it into the visual large model for batch security risk pre-screening. Only high-resolution original frames containing potential risks are extracted. Step 3: Input the high-resolution original frame into the visual large model, and guide the output of structured hazard description data through prompts, which includes at least the hazard category, severity level, and normalized pixel coordinates of the core hazard area in the image; Step 4: Using the camera's intrinsic and extrinsic parameters, normalized pixel coordinates are used to construct a three-dimensional spatial ray through distortion correction and inverse projection transformation. The intersection of the three-dimensional spatial ray with the simplified geometric model of the scene is calculated. Based on the spatial constraint logic, the effective intersection point closest to the camera is selected as the real-world three-dimensional coordinates of the hazard, and the floor to which the hazard belongs is determined accordingly. Step 5: Voxelize the construction space and calculate the spatial coverage based on the camera's view frustum to identify uncovered inspection blind spots; at the same time, based on the real-world three-dimensional coordinates and severity level of the hidden dangers, accumulate and generate a spatial risk heat map in the three-dimensional voxel mesh using a time decay function. Step 6: The large language model agent receives the 3D location results and risk score of the hidden danger, generates a disposal decision, and executes autonomous closed-loop actions such as alarm, work order dispatch, blind spot supplementary inspection or evidence consolidation by calling the preset physical tool interface.

[0021] By adopting the above technical solutions, a correspondence between camera imaging and real 3D space is established based on spatial benchmark construction, providing a geometric basis for subsequent positioning. Video stream sampling and thumbnail mesh stitching reduce the computational overhead caused by frequent calls to the large visual model. The large visual model is used to complete semantic recognition and pixel-level positioning of potential hazards, and combined with camera intrinsic and extrinsic parameters, a geometric transformation from 2D pixels to 3D space is completed to determine the true location of the hazard. Voxelized spatial analysis enables blind spot identification and risk quantification accumulation. Finally, an intelligent agent completes the fully automated execution of the entire process from hazard discovery to handling, forming a complete safety management closed loop.

[0022] Example 2, the specific method for automatically calibrating camera extrinsic parameters in step 1 is as follows: ArUco QR code markers with unique coded IDs are deployed at fixed locations on the construction site, and the world coordinates of their center points are pre-stored. After the camera acquires images, it automatically detects ArUco markers in the image and obtains their sub-pixel-level image coordinates. The PnP algorithm is used to establish a 2D-3D point pair mapping relationship to calculate the camera extrinsic parameters. The root mean square value of the back projection error is checked, and the marker pixel offset is continuously monitored during system operation. When the offset exceeds the preset threshold, the recalibration process is automatically triggered.

[0023] By adopting the above technical solution, utilizing the unique encoding and stable detection characteristics of ArUco QR codes, three-dimensional coordinates are fixedly deployed and pre-stored at the construction site, establishing a reliable correspondence between two-dimensional image coordinates and three-dimensional world coordinates. Automatic detection of marked corner points and the use of the PnP algorithm for calculation replace manual calibration, improving efficiency and accuracy. Back-projection error verification ensures the reliability of calibration results, and continuous monitoring of marker position changes during operation triggers timely recalibration, maintaining long-term stability of camera extrinsic parameters.

[0024] Example 3, the method for constructing the simplified geometric model of the scene in step 1 is as follows: The system directly extracts floor slab elevations, external wall boundary coordinates, and scaffolding erection parameters from Building Information Modeling (BIM) and automatically generates a set of plane equations for ray intersection and corresponding physical boundary constraint polygons.

[0025] By adopting the above technical solution, relying on the complete structural information of the building contained in the BIM model, floor slab elevations, exterior wall boundaries, and scaffolding parameters can be directly extracted, avoiding on-site manual measurement and repetitive modeling. Complex building structures are simplified into planar equations and boundary polygons, adapting to the requirements of ray intersection calculations. This reduces the complexity of 3D geometric operations while ensuring positioning accuracy, thus improving the computational efficiency of 3D positioning.

[0026] Example 4, the batch security risk pre-screening method in step 2 is: The video stream is sampled at preset time intervals, and the sampled frames are scaled proportionally to fixed low-resolution thumbnails. Multiple consecutive thumbnails are stitched together in a matrix form into a single grid image. The grid image is fed into the visual large model at once, and the model is required to return the location numbers of thumbnails with security risks in a structured list format. Only the original high-resolution video frames corresponding to the thumbnails with the corresponding numbers are sent to step 3 for depth analysis.

[0027] By employing the above technical solution, video streams are sampled at fixed intervals, and multiple frames are scaled to a uniform size and stitched together into a grid image. This enables parallel screening of multiple frames in a single large visual model call. The model only outputs the frame numbers of frames with potential hazards, retaining the corresponding original high-resolution frames for depth analysis, while discarding the remaining frames. This significantly reduces the number of large visual model calls and computational consumption without lowering the hazard detection rate.

[0028] Example 5, the method for solving the real-world three-dimensional coordinates of the hidden danger in step 4 is as follows: The pixel coordinates of potential hazards are corrected based on the camera distortion coefficients. The inverse matrix of the camera intrinsic parameter matrix is ​​used to convert the corrected pixel coordinates into ray directions in the normalized camera coordinate system. Combining the rotation matrix and translation vector in the camera extrinsic parameters, the normalized ray directions are transformed to the world coordinate system, and the world space ray equation is constructed with the camera optical center as the ray origin. The equations of each plane in the simplified geometric model of the scene are traversed to calculate the intersection points and distance parameters of the ray with each plane. Intersection points located outside the geometric boundaries of the planes and intersection points with negative distance parameters are eliminated. The effective intersection point with the smallest distance parameter is selected as the three-dimensional coordinates of the potential hazard to conform to the physical imaging law of near-object occlusion.

[0029] By employing the above technical solution, firstly, pixel coordinate distortion is corrected to eliminate lens errors. Then, an intrinsic parameter inverse transformation is used to convert the pixel points into ray directions in the camera coordinate system. Combined with extrinsic parameters, the ray is transformed into the world coordinate system, and a spatial ray is constructed with the camera's optical center as the starting point. Intersection points are calculated by traversing the scene's geometric planes. After filtering out invalid intersection points, the closest valid intersection point is selected, conforming to the on-site imaging occlusion rules, thus obtaining the true 3D coordinates of the potential hazard and determining its corresponding floor.

[0030] Example 6, the method for generating a spatial risk heatmap by accumulating the time decay function in step 5 is as follows: For each identified hazard, an initial risk weight is assigned based on its severity level; this risk weight is then injected into the spatial voxel corresponding to the hazard's coordinates; the risk values ​​of all voxels are dynamically updated over time according to an exponential decay law, with the update formula being: ; It is the real-time risk value of the voxel at the current time t. It is the initial risk weight when a hazard is identified, determined by the severity level of the hazard. It is a natural constant. This is the risk time decay coefficient, which takes a value greater than 0. The larger the coefficient, the faster the risk decays. t is the current calculation time. This is the moment when the hidden danger is first identified and a risk value is assigned; the updated voxel risk value is normalized and mapped to a color gradient, and a risk heat map is generated by rendering it on the 3D building model.

[0031] By adopting the above technical solution, initial risk weights are set according to the severity level of potential hazards, and these weights are mapped to corresponding three-dimensional voxels to achieve risk spatial loading. An exponential decay function is used to dynamically update the voxel risk values, allowing historical hazard risks to naturally decrease over time, reflecting the real-time change in risk status. After normalization, the risk values ​​are mapped to color gradients, visually presenting the risk distribution on the three-dimensional model and providing a visual basis for safety management.

[0032] Example 7, the method for identifying uncovered inspection blind spots in step 5 is as follows: The three-dimensional space of the construction area is divided into a voxel grid with a preset side length; a three-dimensional view frustum is constructed for each camera based on its intrinsic and extrinsic parameters; the voxel grid is traversed to determine whether the center point of each voxel falls within the view frustum of at least one camera; voxels that do not fall within the view frustum of any camera but are located within the effective space of the building are marked as inspection blind spots.

[0033] By adopting the above technical solution, the construction area is divided into a uniform three-dimensional voxel grid, with the voxel center point as the judgment benchmark. A view frustum spatial range is constructed based on the camera's intrinsic and extrinsic parameters, and each voxel is checked individually to determine whether it is covered by a camera. Voxels not covered by any camera but within the effective area of ​​the building are identified as blind spots for inspection, providing a quantitative basis for monitoring supplementation and inspection scheduling.

[0034] Example 8: When a blind spot is identified in step 5, the intelligent agent calls the PTZ camera control tool to automatically adjust the PTZ angle to cover the blind spot for supplementary image acquisition and hazard analysis.

[0035] By adopting the above technical solution, after identifying blind spots, the intelligent agent directly calls the pan-tilt camera control interface to automatically adjust the pan-tilt attitude according to the spatial location of the blind spot, ensuring that the camera's field of view covers the blind spot area. This completes supplementary image acquisition and hazard analysis, eliminates monitoring gaps, and improves the overall monitoring coverage of the construction site.

[0036] In Example 9, in step 6, for the location of the hidden danger for which a rectification work order has been generated, when the preset rectification period arrives, the intelligent agent automatically schedules the monitoring camera in the corresponding area to re-acquire images and calls the visual big model to verify the rectification status, forming a closed-loop verification.

[0037] By adopting the above technical solution, the intelligent agent records the location and deadline of the hazard corresponding to the rectification work order. After the deadline, it automatically schedules the cameras in the corresponding area to collect the latest images, and then calls the visual big data model to review and judge the area again. This confirms whether the hazard has been rectified, realizing a closed-loop verification of the entire process from dispatch, rectification to review, ensuring that the hazard management is in place.

[0038] Example 10: A three-dimensional positioning and intelligent handling system for safety hazards at construction sites, used to realize a three-dimensional positioning and intelligent handling method for safety hazards at construction sites. The system includes: The camera calibration and geometric modeling module is used to perform camera intrinsic parameter calibration, fully automatic extrinsic parameter calibration and drift self-healing based on ArUco tags, and scene simplification geometric model initialization; The video stream scheduling and preprocessing module is used to access multiple video streams and perform efficient keyframe extraction based on thumbnail grid stitching and visual large model initial screening. The visual large model semantic analysis module is used to perform high-resolution analysis on keyframes and output structured information including semantic descriptions of potential hazards and normalized pixel coordinates. The 3D spatial positioning and calculation module is used to perform ray back projection algorithm based on camera intrinsic and extrinsic parameters, and combine scene geometric model constraints to calculate the 3D world coordinates and floor assignment of potential hazards; The voxel spatial analysis and heatmap module is used to construct voxel grids to calculate the multi-camera field of view coverage to identify blind spots, and generate a three-dimensional risk heatmap based on the time decay mechanism. The intelligent agent decision-making and handling module is used to receive 3D location data of potential hazards and risk data, drive the large language model to perform handling reasoning, and realize alarm, work order dispatch and equipment linkage control through tool call interface.

[0039] By adopting the above technical solutions, the camera calibration and geometric modeling module provides the foundation for spatial positioning, while the video stream scheduling and preprocessing module completes efficient video screening. The visual large model semantic analysis module outputs standardized hazard information, and the 3D spatial positioning solution module completes the coordinate transformation from 2D to 3D. The voxel spatial analysis and heatmap module realizes blind spot identification and risk visualization, and the intelligent agent decision-making and disposal module coordinates reasoning and execution. All modules work together to achieve 3D positioning and intelligent disposal of safety hazards at the construction site.

[0040] The following specific embodiments illustrate the implementation principle of the present invention: This invention was applied to the construction site of an eight-story frame structure residential building. A total of 12 fixed monitoring cameras and 4 pan-tilt cameras were deployed on site. The method and system described in this invention were used to complete the three-dimensional positioning and intelligent handling of safety hazards.

[0041] Spatial benchmark construction: First, the intrinsic parameters of all fixed monitoring cameras on site were calibrated using a checkerboard calibration board to obtain the intrinsic parameter matrix and distortion coefficients for each camera. Thirty-six ArUco QR code markers, each 20 cm on each side, were placed on building columns, exterior walls, and floor edges. The 3D coordinates of the marker center points were measured using a total station, and an ID-coordinate mapping table was established. After the cameras were powered on, they automatically identified at least six markers within the image, obtained sub-pixel corner coordinates, and used the PnP algorithm to calculate the rotation matrix and translation vector, completing the automatic extrinsic parameter calibration. After calibration, the back projection error was calculated; a value less than two pixels was considered valid. During system operation, the marker positions were continuously monitored, and a recalibration process was automatically initiated when the offset exceeded five pixels.

[0042] The floor elevations, exterior wall boundaries, and scaffolding erection parameters are extracted from the project's BIM model to generate a simplified geometric model of the scene. The model includes eight horizontal planes, four vertical planes of the exterior walls, and six scaffolding planes. Each plane is represented by a standard plane equation and includes boundary constraint polygons, providing a geometric basis for subsequent ray intersection calculations.

[0043] Real-time video streams from all 16 cameras were accessed via the RTSP protocol, and all video streams were sampled at 2-second frame intervals. The sampled frames were uniformly scaled to 224x224 pixel thumbnails and stitched together in a 4x4 matrix to form a single mesh image. This mesh image was input into a large visual model, which detected each frame and output the frame numbers of problematic frames as an array. In this case, at a certain moment, the stitched mesh image detected problematic frames 13, 17, and 11. Only the original high-resolution images corresponding to these three frames were retained for depth analysis, while the remaining frames were discarded, significantly reducing computational power consumption.

[0044] Visual large-scale model hazard identification and pixel localization: The selected high-resolution frames were fed into a large-scale visual model, and prompts guided the model to output structured hazard information. Frame 13's analysis indicated that safety belts were not worn during work at height, with a severity level of "major" and normalized pixel coordinates of 0.35 and 0.62. Frame 17's analysis indicated a lack of edge protection, with a severity level of "relatively serious" and normalized pixel coordinates of 0.42 and 0.58. Frame 11's analysis indicated excessive material stacking, with a severity level of "moderate" and normalized pixel coordinates of 0.51 and 0.71. The model simultaneously outputs textual descriptions of the hazards, providing a basis for subsequent location and handling.

[0045] Three-dimensional location of hidden dangers and floor determination: Distortion correction is applied to the three sets of normalized pixel coordinates to eliminate positional deviations caused by lens distortion. The corrected coordinates are then converted to normalized camera coordinates using the inverse of the corresponding camera intrinsic matrix, and a spatial ray in the world coordinate system is constructed by combining it with the camera extrinsic parameters. The ray is then intersected sequentially with the floor plane, exterior wall plane, and scaffolding plane in the simplified geometric model of the scene, and the distance parameters of each intersection point are calculated. Invalid intersection points located outside the plane boundaries and with negative distances are filtered out, and the intersection point with the smallest distance parameter is selected as the true 3D coordinates of the potential hazard.

[0046] Calculations showed that the worker not wearing a safety belt had the following 3D coordinates: X23.6 meters, Y41.2 meters, Z21.7 meters, corresponding to the 8th floor construction floor. The location lacking edge protection had the following 3D coordinates: X19.3 meters, Y38.5 meters, Z18.2 meters, corresponding to the edge of the 7th floor. The location where materials were stacked too high had the following 3D coordinates: X31.4 meters, Y42.9 meters, Z6.3 meters, corresponding to the ground area on the 2nd floor. The system automatically determined the floor assignment and pinpointed the specific location of the hazard.

[0047] Spatial blind spot identification and risk heat map generation: The entire construction site space was divided into a 3D voxel grid with sides of 1 meter. The field of view frustum was constructed based on the intrinsic and extrinsic parameters of each camera. All voxels were traversed to determine if the center point of each voxel was covered by at least one camera. In this case, the northwest corner underground material handling area and the far end of the scaffolding on the east side of the seventh floor were identified as blind spots for inspection; the corresponding voxels were not covered by any camera's field of view.

[0048] The three identified hazards were assigned risk weights: a major hazard with a weight of nine, a relatively large hazard with a weight of three, and a minor hazard with a weight of one. These weights were then assigned to the corresponding three-dimensional voxels, and the risk values ​​of all voxels were dynamically updated using an exponential decay formula. The system normalized the voxel risk values ​​and mapped them to a blue, yellow, and red gradient, generating a three-dimensional risk heatmap and displaying it by floor. The 8th-floor work area was displayed as a high-risk red area, while the 7th and 2nd floors were displayed as medium-risk yellow and low-risk blue areas, respectively.

[0049] Agent processing and closed-loop verification: After receiving the 3D location of the hazard, floor information, and risk score, the intelligent agent generates a response decision through large language model reasoning. For a major hazard on the 8th floor, a work stoppage order is immediately triggered, and an alarm message is sent to the safety officer and project manager in that area, including images of the hazard, 3D coordinates, and floor location. Simultaneously, a rectification work order is generated in the management system, automatically filling in the hazard type, location, level, and rectification deadline. Alarms are sent to hazards on the 7th and 2nd floors respectively, and rectification work orders with deadlines are issued, automatically capturing images of the hazards along with their 3D coordinates, timestamps, and camera numbers, and storing them in the evidence database.

[0050] For the two identified blind spots in the inspection, the intelligent agent automatically directed the corresponding pan-tilt cameras to the blind spots to complete supplementary image acquisition and hazard analysis. After the rectification deadline, the intelligent agent directed the cameras in the corresponding areas to re-acquire images and called the visual big data model to review the rectification status. After review, it was found that safety belts were being worn properly for work on the 8th floor, guardrails had been installed at the edges on the 7th floor, and materials were being stacked properly on the 2nd floor. All three hazards were rectified. The system automatically closed the work order and recorded the review results, completing the intelligent handling of the entire process.

[0051] The above are all preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Therefore, all equivalent changes made in accordance with the structure, shape and principle of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for three-dimensional positioning and intelligent handling of safety hazards at construction sites, characterized by: Includes the following steps: Step 1: Perform intrinsic parameter calibration on the monitoring cameras at the construction site, and automatically calibrate the extrinsic parameters of the cameras by deploying markers with known world coordinates, and construct a simplified geometric model of the scene including floor plans and edge protection surfaces; Step 2: Sample and perform thumbnail dimensionality reduction on the surveillance video stream. Stitch the thumbnails of multiple frames into a grid image and input it into the visual large model for batch security risk pre-screening. Only high-resolution original frames containing potential risks are extracted. Step 3: Input the high-resolution original frame into the visual large model, and guide the output of structured hazard description data through prompts, which includes at least the hazard category, severity level, and normalized pixel coordinates of the core hazard area in the image; Step 4: Using the camera's intrinsic and extrinsic parameters, normalized pixel coordinates are used to construct a three-dimensional spatial ray through distortion correction and inverse projection transformation. The intersection of the three-dimensional spatial ray with the simplified geometric model of the scene is calculated. Based on the spatial constraint logic, the effective intersection point closest to the camera is selected as the real-world three-dimensional coordinates of the hazard, and the floor to which the hazard belongs is determined accordingly. Step 5: Voxelize the construction space and calculate the spatial coverage based on the camera's view frustum to identify uncovered inspection blind spots; Simultaneously, based on the real-world three-dimensional coordinates and severity levels of potential hazards, a spatial risk heatmap is generated in a three-dimensional voxel mesh using a time decay function. Step 6: The large language model agent receives the 3D location results and risk score of the hidden danger, generates a disposal decision, and executes autonomous closed-loop actions such as alarm, work order dispatch, blind spot supplementary inspection or evidence consolidation by calling the preset physical tool interface.

2. The method for three-dimensional positioning and intelligent handling of safety hazards at construction sites according to claim 1, characterized in that: The specific method for automatically calibrating camera extrinsic parameters in step 1 is as follows: ArUco QR code markers with unique coded IDs are deployed at fixed locations on the construction site, and the world coordinates of their center points are pre-stored. After the camera acquires images, it automatically detects ArUco markers in the image and obtains their sub-pixel-level image coordinates. The PnP algorithm is used to establish a 2D-3D point pair mapping relationship to calculate the camera extrinsic parameters. The root mean square value of the back projection error is checked, and the marker pixel offset is continuously monitored during system operation. When the offset exceeds the preset threshold, the recalibration process is automatically triggered.

3. The method for three-dimensional positioning and intelligent handling of safety hazards at construction sites according to claim 2, characterized in that: The method for constructing the simplified geometric model of the scene in step 1 is as follows: The system directly extracts floor slab elevations, external wall boundary coordinates, and scaffolding erection parameters from Building Information Modeling (BIM) and automatically generates a set of plane equations for ray intersection and corresponding physical boundary constraint polygons.

4. The method for three-dimensional positioning and intelligent handling of safety hazards at construction sites according to claim 3, characterized in that: The batch security risk pre-screening method in step 2 is as follows: The video stream is sampled at preset time intervals, and the sampled frames are scaled proportionally to a fixed low-resolution thumbnail. Multiple consecutive thumbnails are stitched together in a matrix to form a single grid image; the grid image is then fed into a large visual model at once, and the model is required to return the location numbers of thumbnails with security risks in a structured list format; only the original high-resolution video frames corresponding to the thumbnails with the corresponding numbers are fed into step 3 for depth analysis.

5. The method for three-dimensional positioning and intelligent handling of safety hazards at construction sites according to claim 4, characterized in that: The method for solving the real-world 3D coordinates of the hidden danger in step 4 is as follows: Correct the pixel coordinates of potential defects based on the camera distortion coefficients; use the inverse matrix of the camera intrinsic parameter matrix to convert the corrected pixel coordinates into ray directions in the normalized camera coordinate system; combine the rotation matrix and translation vector in the camera extrinsic parameters to transform the normalized ray directions to the world coordinate system, and construct the world space ray equation with the camera optical center as the ray origin; traverse the equations of each plane in the simplified geometric model of the scene, calculate the intersection points and distance parameters of the ray with each plane; remove intersection points located outside the geometric boundaries of the planes and intersection points with negative distance parameters; The effective intersection point with the smallest distance parameter is selected as the three-dimensional coordinate of the hidden danger, so as to conform to the physical imaging law of near object occlusion.

6. The method for three-dimensional positioning and intelligent handling of safety hazards at construction sites according to claim 5, characterized in that: The method for generating the spatial risk heatmap by accumulating the time decay function in step 5 is as follows: For each identified hazard, an initial risk weight is assigned based on its severity level; this risk weight is then injected into the spatial voxel corresponding to the hazard's coordinates; the risk values ​​of all voxels are dynamically updated over time according to an exponential decay law, with the update formula being: ; It is the real-time risk value of the voxel at the current time t. It is the initial risk weight when a hazard is identified, determined by the severity level of the hazard. It is a natural constant. This is the risk time decay coefficient, which takes a value greater than 0. The larger the coefficient, the faster the risk decays. t is the current calculation time. This is the moment when the hidden danger is first identified and a risk value is assigned; the updated voxel risk value is normalized and mapped to a color gradient, and a risk heat map is generated by rendering it on the 3D building model.

7. The method for three-dimensional positioning and intelligent handling of safety hazards at construction sites according to claim 6, characterized in that: The method for identifying uncovered inspection blind spots in step 5 is as follows: The three-dimensional space of the construction area is divided into a voxel grid with a preset side length; a three-dimensional view frustum is constructed for each camera based on its intrinsic and extrinsic parameters; the voxel grid is traversed to determine whether the center point of each voxel falls within the view frustum of at least one camera; voxels that do not fall within the view frustum of any camera but are located within the effective space of the building are marked as inspection blind spots.

8. The method for three-dimensional positioning and intelligent handling of safety hazards at construction sites according to claim 7, characterized in that: When a blind spot is identified in step 5, the intelligent agent calls the PTZ camera control tool to automatically adjust the PTZ angle to cover the blind spot for supplementary image acquisition and hazard analysis.

9. The method for three-dimensional positioning and intelligent handling of safety hazards at construction sites according to claim 8, characterized in that: In step 6, for the locations of potential hazards for which rectification work orders have been generated, when the preset rectification period arrives, the intelligent agent automatically schedules the monitoring cameras in the corresponding areas to re-collect images and calls the visual big model to verify the rectification status, forming a closed-loop verification.

10. A three-dimensional positioning and intelligent handling system for safety hazards at construction sites, characterized in that: The system for implementing the three-dimensional positioning and intelligent handling method for construction site safety hazards as described in claim 9 includes: The camera calibration and geometric modeling module is used to perform camera intrinsic parameter calibration, fully automatic extrinsic parameter calibration and drift self-healing based on ArUco tags, and scene simplification geometric model initialization; The video stream scheduling and preprocessing module is used to access multiple video streams and perform efficient keyframe extraction based on thumbnail grid stitching and visual large model initial screening. The visual large model semantic analysis module is used to perform high-resolution analysis on keyframes and output structured information including semantic descriptions of potential hazards and normalized pixel coordinates. The 3D spatial positioning and calculation module is used to perform ray back projection algorithm based on camera intrinsic and extrinsic parameters, and combine scene geometric model constraints to calculate the 3D world coordinates and floor assignment of potential hazards; The voxel spatial analysis and heatmap module is used to construct voxel grids to calculate the multi-camera field of view coverage to identify blind spots, and generate a three-dimensional risk heatmap based on the time decay mechanism. The intelligent agent decision-making and handling module is used to receive 3D location data of potential hazards and risk data, drive the large language model to perform handling reasoning, and realize alarm, work order dispatch and equipment linkage control through tool call interface.

Images