A multi-dimensional perception security penetration management method and system

Abstract

The present application belongs to the technical field of construction safety management, and aims to solve the technical problems of many blind supervision areas, single perception dimension and insufficient management penetration in the safety management of building construction. A multi-dimensional perception safety penetration management method and system are provided, which integrates unmanned aerial vehicles, inspection robots, fixed high-definition cameras and personnel positioning terminals to realize real-time collection of aerial, ground and fixed perspective three-dimensional video streams and personnel position data. Based on GIS geographic information system and CAD construction drawings as double bases, a dynamic three-dimensional scene model of the construction site is constructed, and grid division and visual labeling of the engineering part are realized. Image stitching algorithm, point cloud classification model and illegal behavior identification model are deployed to realize multi-view image fusion, three-dimensional modeling and illegal behavior identification. The penetration management interface is configured as the interface for early warning push, disposal tracking and hierarchical penetration viewing. The present application integrates multi-dimensional perception, grid visualization and penetration control for construction management.

Description

Technical Field

[0001] This invention belongs to the field of construction safety management technology, specifically involving a multi-dimensional perception-based safety penetration management method and system. Background Technology

[0002] Current construction safety management generally suffers from technical pain points such as complex scenarios, numerous regulatory blind spots, limited perception dimensions, and insufficient management penetration. Existing safety management systems mostly rely on single video surveillance or manual inspections, which presents the following problems: First, the identification of engineering parts relies on manual annotation, which is inefficient and easily affected by subjective factors, and cannot achieve automatic positioning through grid visualization. Second, the on-site data collection dimensions are limited, lacking the collaborative perception capabilities of multiple devices such as drones, inspection robots, and fixed cameras, making it difficult to fully cover special construction areas such as high altitudes, deep foundation pits, and confined spaces. Third, the accuracy of unsafe behavior recognition is insufficient, as it can only be based on single-view image analysis and is prone to misjudgment and missed judgment due to occlusion and viewpoint deviation. Fourth, the management system is disconnected from the on-site perception data and lacks the ability to penetrate the interface for linkage, making it difficult to meet the needs of refined management and control of construction safety in the context of intelligent construction.

[0003] To address the aforementioned technical bottlenecks, there is an urgent need to develop an intelligent safety management method and system that integrates multi-dimensional perception, grid visualization, and penetrating control to improve the automation, precision, and full-process level of construction safety management. Summary of the Invention

[0004] In order to solve at least one of the above-mentioned technical problems in the prior art, the present invention provides a multi-dimensional sensing security penetration management method and system.

[0005] This invention employs the following technical solution: a multi-dimensional sensing security penetration management method, comprising the following steps: The process involves acquiring geospatial baseline data and engineering design data of the construction area and performing spatial registration. Based on the registered data, a 3D model of the construction site is generated. Using the 3D digital model, a bounding box tool is used to delineate and define the target area, and the boundary coordinate data of the target area is extracted. Based on a spatial topology algorithm, the boundary coordinate data and the registered engineering design data are spatially correlated to identify all engineering structural objects within the target area. The attribute information of these engineering structural objects is extracted and bound to the corresponding areas in the 3D digital model, generating region-structure-attribute correlation mapping data. Finally, multi-source sensing data is acquired, decoded, stitched together, and spatially calibrated with the 3D model to generate… A dynamic 3D scene model containing dynamic entity locations, engineering component information, and WBS-related information is generated. A violation identification model is constructed to identify safety anomalies, extract the spatial location of the anomalies and map them to the corresponding area of ​​the dynamic 3D scene model, associate them with engineering component information, and generate standardized early warning data to the penetrating management interface. Based on the early warning data and the "geographic coordinates-engineering structure component-WBS decomposition node" association system, penetrating control and closed-loop management are achieved. Combining the WBS decomposition system and RAG technology, a dynamically updated knowledge base system is constructed to optimize model algorithms. Risk patterns are mined and feedback is provided to optimize the model based on the dynamic 3D scene model, WBS decomposition node data, and violation identification model, and the feedback is synchronously provided to the penetrating management interface.

[0006] Preferably, the geospatial basic data is GIS topographic data, and the engineering design data is CAD construction drawings; The steps to generate a 3D model of the construction site include: Acquire GIS topographic data and CAD construction drawings of the construction area, and extract their respective coordinate reference information. In the overlapping area of ​​the GIS topographic data and CAD drawings, select no less than four pairs of corresponding feature points to obtain control point pairs. Based on spatial similarity transformation calculation, perform coordinate calibration on the control point pairs and obtain transformation parameters. Apply the calculated transformation parameters to the coordinates of all CAD drawing elements and register the CAD drawings to the GIS topographic coordinate system. Construct a surface model of the construction site using the calibrated topographic data, and read the calibrated drawing data to stretch or loft key structural elements into three-dimensional entities. Integrate the surface model and structural entities to establish a three-dimensional model of the construction site, and store the mapping relationship between GIS topographic coordinates and CAD part codes in the node attributes of the three-dimensional model.

[0007] Preferably, the step of generating the region-structure-attribute association mapping data includes: Draw an arbitrary polygonal bounding box in a 3D scene or GIS interface as the target area; calculate the vertex sequence of the polygonal bounding box from the real-time captured screen coordinates to the geographic coordinate system, record the vertex sequence of the polygonal bounding box, and close the first and last points to form polygon P; simultaneously project polygon P onto a planar projection coordinate system to obtain planar polygon P′; based on the calibrated spatially located CAD primitives, build an R-tree spatial index according to their minimum bounding rectangle, with each index node storing the CAD primitive ID, MBR boundary, and layer information; based on the R-tree spatial index, quickly query all CAD primitives whose MBR intersects with or contains planar polygon P′, forming... A candidate set C is formed. For each CAD element in the candidate set C, spatial topological relationship judgment is performed based on its geometric type to obtain a set S of all related CAD elements within the polygon selection area. For each CAD element in the set S, attribute extraction and semantic recognition are performed to assign structured attribute information to each CAD element. The grid unit corresponding to the selected polygon P′ is identified and a grid ID is established. A many-to-many relationship between the grid unit and the set S is established, storing information including but not limited to the geographic coordinates of the grid vertex and the set of part attributes, and generating region-structure-attribute association mapping data.

[0008] Preferably, the method for performing topological relationship determination based on the geometric type of each CAD primitive in the candidate set C includes: If the geometric type of the CAD element is a point element, the ray method is used to determine whether the point is inside or on the boundary of the planar polygon P′. If the geometric type of the CAD element is a line element, if any part of the line segment intersects with or is inside the planar polygon P′, it is determined to belong to the grid range. If the geometric type of the CAD element is a surface element, it is determined whether the boundary of the surface element intersects with, contains, or overlaps with the planar polygon P′. If the intersection is not empty, it is included in the engineering structure part of the grid. Finally, the set S of all relevant CAD elements within the polygon selection area is obtained.

[0009] Preferably, the steps for attribute extraction and semantic recognition for each CAD element in set S are as follows: Based on CAD layer naming standards, the layer string is parsed to extract information including but not limited to profession, part type, and subcategory, and automatically mapped to standardized engineering part names. If the CAD element is a block reference with attributes, the attribute tags are read, including but not limited to "WBS code", "part name", and "design strength", to directly obtain the structured part attributes. For CAD elements without clear semantic annotations, their geometric features are extracted and combined with the spatial relationship of adjacent elements, and fuzzy matching is performed with the built-in "engineering part semantic feature library" to determine the most likely engineering part type.

[0010] Preferably, the multi-source sensing data includes, but is not limited to, integrated drone aerial video streams, robot real-time inspection footage, and fixed camera monitoring video streams. The edge computing nodes perform real-time decoding, frame extraction, and coordinate synchronization of three-view video stream data, selecting clear, unobstructed personnel image frames with valid coordinate matching; an image stitching algorithm based on SIFT feature point matching is used to fuse and align personnel image frames from different perspectives; point cloud data is combined for point cloud classification and denoising, and the stitched images, point cloud data, and 3D model are calibrated to generate a dynamic 3D scene model containing personnel location, engineering parts, and WBS association information.

[0011] Preferably, the violation identification model is built on the deep learning YOLOv8 architecture and incorporates construction scene feature transfer training; The violation identification model has a built-in database of unsafe behaviors in construction scenarios and sets differentiated identification thresholds based on different violation types. When an unsafe behavior is detected, the system simultaneously extracts the personnel location coordinates corresponding to that behavior, combines them with the 3D model and coordinate mapping relationship to locate the specific grid cell, and associates it with the corresponding engineering part information to generate standardized early warning data. The early warning data includes, but is not limited to, behavior type, occurrence time, geographic coordinates, corresponding CAD structural part, WBS code, associated image frame, and point cloud screenshot. The penetrating management interface marks the WBS node and engineering part where the unsafe behavior occurred, and automatically associates the historical safety data under that WBS node. At the same time, the system can automatically associate the handling instructions with the corresponding WBS node and engineering part.

[0012] Preferably, the steps for constructing a dynamically updated knowledge base system and optimizing the model algorithm by combining the WBS decomposition system and RAG technology include: Regularly collect core system metrics and user feedback data to form an iteratively optimized dataset; combine the characteristics of construction scenarios and WBS management requirements to build a knowledge base system using a hierarchical storage architecture, establishing a mapping between "risk sources - control measures - legal basis"; configure role definitions, responsibility boundaries, and multi-dimensional constraint instructions for the violation identification model; dynamically update the knowledge base system according to a preset cycle based on the iteratively optimized dataset and externally crawled real-time data sources; adjust the feature point matching strategy of the image stitching algorithm based on scenario experience in the knowledge base system, and add a redundancy detection mechanism; optimize the point cloud classification model, remove interference data through cluster analysis, strengthen the distinction between target entities and the environment, and ensure the association between point cloud data and dynamic 3D scene models and WBS nodes.

[0013] Preferably, the steps for identifying risk distribution patterns based on the 3D scene model, WBS decomposition node data, and violation behavior identification model results, using decision tree and association rule mining algorithms, and combining these with knowledge base system examples to formulate optimization measures include: Using the smallest grid unit of a 3D mesh model as the spatial benchmark and WBS nodes as the link between engineering parts, a risk event dataset integrating five dimensions of features—time, space, behavior, equipment, and environment—is constructed. A CART decision tree is used with "whether an emergency warning is triggered" as the binary classification objective variable to analyze the combined causes of high-risk behaviors. Simultaneously, a weighted FP-Growth algorithm is introduced, incorporating grid risk exposure weights into the traditional support-confidence framework to uncover frequent behavior-environment symbiotic patterns. Based on the above mining results, similar risk combination handling cases are retrieved from the knowledge base using RAG, and case similarity-weighted fusion recommendations are introduced to obtain corresponding optimization measures.

[0014] In a second aspect, the present invention also provides a multi-dimensional sensing security penetration management system, comprising: The data acquisition layer integrates drones, inspection robots, fixed high-definition cameras, and personnel positioning terminals to achieve real-time acquisition of three-dimensional video streams and personnel location data from the air, ground, and fixed perspectives. The model support layer uses GIS geographic information systems and CAD construction drawings as dual foundations to construct a dynamic three-dimensional scene model of the construction site and realize grid division and visual annotation of engineering parts. The intelligent analysis layer deploys image stitching algorithms, point cloud classification models, and violation recognition models to achieve multi-view image fusion, three-dimensional modeling, and violation recognition. The penetrating management interface serves as the interface for early warning push, handling tracking, and hierarchical viewing. The application output layer enables collaboration between mobile and PC terminals and supports early warning push, report generation, and historical review.

[0015] Compared with the prior art, the beneficial effects of the present invention are: 1. By matching the GIS+CAD dual-model base with grid coordinates, the relationship between engineering parts and grids is automatically and quickly established, improving the accuracy of grid management; 2. By integrating data from drones, robots, and cameras, combined with image stitching and point cloud classification technology, blind spots in supervision are eliminated, the accuracy of unsafe behavior identification is improved, and the risk of misjudgment and omission of hidden dangers is reduced; 3. Through a penetrating management interface, hierarchical viewing from overall to local to detailed levels is achieved, linking the entire process of early warning and handling, improving management penetration and handling efficiency; 4. Based on knowledge base retrieval capabilities and model optimization feedback mechanisms, the asset value of project data is enhanced, meeting the needs of automatic identification and early warning handling of unsafe behaviors in projects. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a structural block diagram of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other implementation methods obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] It should be noted that the structures, proportions, sizes, etc., shown in the accompanying drawings of this specification are only for the purpose of assisting those skilled in the art in understanding and reading the content disclosed in the specification, and are not intended to limit the conditions under which the present invention can be implemented. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportional relationships, or adjustments to the size, without affecting the effects and objectives that the present invention can produce, should fall within the scope of the technical content disclosed in the present invention. It should be noted that in this specification, relational terms such as "first" and "second" are only used to distinguish one entity from several other entities, and do not necessarily require or imply any actual relationship or order between these entities.

[0020] This invention provides an embodiment: like Figure 1 As shown, a multi-dimensional perception-based security penetration management method includes the following steps: S1: Obtain geospatial basic data and engineering design data of the construction area, and perform spatial registration processing; based on the registered data, generate a three-dimensional model of the construction site.

[0021] In this embodiment, the geospatial foundation data is high-precision GIS topographic data, and the engineering design data is CAD construction drawings. The coordinate calibration algorithm in S1 is specifically a composite coordinate calibration algorithm based on control points, combining spatial similarity transformation and elevation fitting. Its core is a seven-parameter / four-parameter coordinate transformation and regular grid DEM (Digital Elevation Model) interpolation, combined with CAD entity stretching and texture mapping to achieve 3D model reconstruction. The specific process is as follows: The steps to generate a 3D model of the construction site include: S11: Unify Coordinate Reference: Obtain GIS (Geographic Information System) topographic data of the construction area (such as UAV aerial DEM / DOM (Digital Orthophoto), coordinate system is CGCS2000 projected coordinates) and CAD construction drawings (usually in an independent construction coordinate system), and analyze them to extract their respective coordinate reference information. If the CAD construction drawings lack geographic reference, first establish the correspondence between the construction coordinate system and the projected coordinate system through annotations of at least 3 known construction coordinate control points in the drawings.

[0022] S12: Control point pair extraction: In the overlapping area of ​​GIS terrain and CAD drawings, select no less than four pairs of feature points with the same name (such as building corners, road intersections, and permanent control stakes). The control points should be evenly distributed and not collinear to obtain control point pairs.

[0023] Control point coordinates are marked as follows: GIS coordinates: CAD coordinates: S13: Spatial Similarity Transformation Solution: Based on the spatial similarity transformation solution, coordinate calibration is performed on the control point pairs, and transformation parameters are obtained. Four-parameter calibration with elevation correction is used for coordinate calibration.

[0024] in , Here, m is the translation parameter, θ is the scale factor, and θ is the rotation angle. The parameters are 6 undetermined coefficients for polynomial surface fitting; the parameters are solved globally using the least squares method to minimize the sum of squared residuals at all control points.

[0025] S14: CAD Data Spatial Alignment and Verification: Apply the transformation parameters obtained from the settlement to the coordinate system of all CAD drawings, and register the CAD drawings to the GIS topographic coordinate system; ensure that the calibration error is ≤ the allowable construction error (e.g., ≤ 5cm for horizontal plane and ≤ 10cm for vertical plane) by checking the residuals of the remaining control points and the matching accuracy of randomly selected points in different areas. If the requirements are not met, add control points or adjust the order of the fitting model.

[0026] S15: 3D Model Reconstruction: Construct a regular triangular mesh (TIN) or grid model using calibrated high-precision DEM / point cloud data to generate a realistic construction site terrain surface; read calibrated drawing data (CAD layers, line types, block references, and elevation attributes) and stretch or loft key structural elements (building foundations, pits, supports) into 3D entities; for complex structures, BIM entities can be generated by combining cross-section lofting; merge the terrain model and structural entities in a unified coordinate system to establish a 1:1 3D model of the construction site including terrain, existing buildings, proposed structures, and temporary facilities, and save the mapping relationship between GIS terrain coordinates and CAD part codes in the node attributes of the 3D model.

[0027] S2: Based on the 3D digital model, the target area is defined by using a selection tool, and the boundary coordinate data of the target area is extracted. Based on the spatial topology relationship algorithm, the boundary coordinate data and the registered engineering design data are spatially correlated to identify all engineering structural objects contained in the target area, extract the attribute information of the engineering structural objects, and bind the attribute information to the corresponding area in the 3D digital model to generate the region-structure-attribute correlation mapping data.

[0028] In this embodiment, the spatial topology relationship algorithm in S2 specifically employs a "ray-based spatial inclusion detection" + "spatial index-accelerated topology query" algorithm, combined with CAD layer / block attribute parsing and semantic dictionary matching of engineering parts, to complete part coordinate filtering and attribute identification. The steps for generating region-structure-attribute association mapping data include: S21: Extracting geographic coordinates of selected polygon vertices: Draw an arbitrary polygonal selection area in the 3D scene or GIS interface as the target area; calculate the geographic coordinates based on the real-time captured screen coordinates, record the vertex sequence V={(lon1,lat1),(lon2,lat2),...,(lonn,latn)} of the polygonal selection area, and close the first and last points to form polygon P; at the same time, project polygon P onto the planar projection coordinate system to obtain the planar polygon P′; S22: Construction of CAD primitive spatial index: Based on the calibrated spatial location of all CAD engineering primitives (such as pile foundation outline, pile cap edge line, column and beam axis, etc.), an R-tree spatial index is built according to its minimum bounding rectangle (MBR), and each index node stores the CAD primitive ID, MBR boundary and the layer information to support fast range retrieval.

[0029] S23: Initial screening of candidate primitives (spatial topology coarse screening): Based on the R-tree spatial index, quickly query all CAD primitives that intersect or contain the MBR with the planar polygon P′ to form a candidate set C; this step avoids the need to make precise judgments on whether points are inside the polygons in the full CAD data, thus improving efficiency.

[0030] S24: Precise spatial inclusion determination: For each CAD element in the candidate set C, perform spatial topological relationship determination based on its geometric type to obtain the set S of all related CAD elements within the polygon selection area.

[0031] Specifically, the method for performing topological relation determination based on the geometric type of each CAD primitive in the candidate set C includes: S241: If the geometric type of the CAD drawing element is a point element (such as an independent pile location or an embedded part), use the ray method to determine whether the point is inside or on the boundary of the planar polygon P′. S242: If the geometry type of the CAD primitive is a linear primitive (such as beam axis, pipe centerline), if any part of the line segment intersects with the planar polygon P′ or is inside the polygon, it is determined to belong to the grid range; the Cohen-Sutherland line segment clipping algorithm can be used to perform clipping test first.

[0032] S243: If the geometric type of the CAD drawing element is a planar element (such as the bottom of the foundation pit or the construction area block), determine whether the boundary of the planar element intersects, contains, or overlaps with the planar polygon P′. If the intersection is not empty, it is an engineering structure part included in the grid. Finally, obtain the set S of all relevant CAD elements within the polygonal selection area.

[0033] S25: For each CAD element in set S, perform attribute extraction and semantic recognition, and assign structured attribute information to each CAD element.

[0034] The steps for attribute extraction and semantic recognition for each CAD primitive in set S are as follows: S251: Based on the CAD layer naming standard, parse the layer string, extract information including but not limited to profession, part type, and subcategory, and automatically map it to standardized engineering part names; S252: If the CAD element is a block reference with attributes, read the attribute labels, which include but are not limited to "WBS code", "part name" and "design strength", and directly obtain the structured part attributes; S253: For CAD elements without explicit semantic annotations, extract their geometric features (such as area, perimeter, and convex hull shape) and combine them with the spatial relationships of adjacent elements. Perform fuzzy matching with the built-in "Engineering Part Semantic Feature Library" to determine the most likely engineering part type (such as strip base, isolated base, waterproof membrane, etc.).

[0035] After the above information is compiled, each part is assigned a structured attribute record containing "CAD structural part name, related discipline, WBS association code, coordinate range" and other information.

[0036] S26: Identify the grid cells corresponding to the selected polygon P′ (which can be automatically subdivided by fixed side length or directly treated as a single large grid) and establish a grid ID; establish a many-to-many association between the grid cells and set S in the database, storing information including but not limited to the geographic coordinates of the grid vertices and the set of location attributes, and generate region-structure-attribute association mapping data. Subsequently, all subsequent queries, risk labeling, and early warning location for the grid can be directly retrieved quickly using the grid ID to find the corresponding CAD structural parts and their attributes, supporting end-to-end, transparent management.

[0037] S3: Acquire multi-source sensing data, decode and stitch the multi-source sensing data, and perform spatial calibration with the 3D model to generate a dynamic 3D scene model containing dynamic entity positions, engineering part information, and WBS association information.

[0038] In this embodiment, the multi-source sensing data includes, but is not limited to, integrated drone aerial video streams, robot real-time inspection footage, and fixed camera monitoring video streams. The edge computing nodes perform real-time decoding, frame extraction, and coordinate synchronization of three-view video stream data, selecting personnel image frames with a resolution ≥1080P, no occlusion, and valid coordinate matching; an image stitching algorithm based on SIFT (Scale Invariant Feature Transform) feature point matching is used to fuse and align personnel image frames from different perspectives; point cloud data is combined for point cloud classification (distinguishing between personnel, equipment, construction materials, and structural components) and noise reduction processing, and the stitched images, point cloud data, and 3D models are coordinate-calibrated to generate a dynamic 3D scene model containing personnel location, engineering parts, and WBS (Work Breakdown Structure) related information.

[0039] S4: Construct a violation identification model, identify safety anomalies, extract the spatial location of the anomalies and map them to the corresponding area of ​​the dynamic 3D scene model, associate engineering part information, and generate standardized early warning data to the penetrating management interface.

[0040] The violation identification model is built on the deep learning YOLOv8 architecture and incorporates construction scene feature transfer training.

[0041] The violation identification model has a built-in database of unsafe behavior features in construction scenarios and sets differentiated identification thresholds based on different violation types. Specifically, a safety helmet feature matching degree of ≥85% for personnel's head area is considered compliant, while <85% triggers an alarm; a safety belt feature matching degree of ≥80% for personnel is considered compliant, while <80% triggers an alarm; and a guardrail feature matching degree of ≥95% for personnel is considered compliant, while <95% triggers an alarm.

[0042] When an unsafe act is detected, the location coordinates of the personnel corresponding to the act are extracted simultaneously. Combined with the 3D model and coordinate mapping relationship, the location is pinpointed to a specific grid cell, and associated with the corresponding engineering part information to generate standardized early warning data. The early warning data includes, but is not limited to, the type of act, the time of occurrence, the geographical coordinates, the corresponding CAD structural part, the WBS code, the associated image frame, and the point cloud screenshot. The penetrating management interface marks the WBS node and engineering part where the unsafe act occurred, and automatically associates the historical safety data under the WBS node to provide data support for subsequent early warning and handling. At the same time, the handling instructions can be automatically associated with the corresponding WBS node and engineering part.

[0043] S5: Based on the association system of early warning data and "geographic coordinates-engineering structural parts-WBS decomposition nodes", it realizes penetrating control and closed-loop management.

[0044] Specifically, after receiving the warning information, the penetrating management interface immediately triggers a tiered warning mechanism. Based on the risk level of the unsafe behavior (general warning: personal protective equipment violation; important warning: area intrusion, unauthorized operation; emergency warning: objects thrown from heights, potential fall from height), the warnings are displayed in blue, yellow, and red pop-up windows, accompanied by differentiated sound prompts. The warnings are also pushed to the corresponding managers via mobile SMS and WeChat / DingTalk work notifications. The warning information clearly indicates the type of behavior, the time of occurrence, the precise location, the corresponding CAD structural part, the WBS code, and the associated screenshot.

[0045] Secondly, it supports multi-level penetration viewing: managers can start from the overall 3D model of the site, click on the warning pop-up to directly penetrate to the corresponding grid unit, and display the CAD structural drawing details, WBS decomposition node information and division of responsibilities of the unit; further penetrate to the real-time video stream of the corresponding perspective, track the dynamic behavior of personnel throughout the process, and at the same time, review the past safety warnings and handling records under the WBS node to provide decision-making reference for on-site handling.

[0046] Managers can issue targeted instructions through the interface. The instructions are automatically associated with the corresponding WBS nodes and project locations, specifying the rectification requirements, responsible persons, and completion deadlines (e.g., for the issue of personnel not wearing safety helmets in the WBS node #3 pile foundation area, the safety officer is required to conduct on-site verification and rectification within 15 minutes). The instructions are also pushed to the mobile devices of the on-site responsible persons, who can upload photos, videos, and rectification completion reports through their mobile devices.

[0047] S6: Combining the WBS decomposition system and RAG (retrieval-enhanced generation) technology, a dynamically updated knowledge base system is constructed to optimize the model algorithm.

[0048] The specific steps include: S61: Regularly collect core system data, classify and statistically analyze indicators such as unsafe behavior identification accuracy, early warning response time, closed-loop handling efficiency, and misjudgment / missed judgment cases for each engineering part according to WBS nodes, and collect feedback from on-site management personnel and construction personnel to form an iterative optimization dataset.

[0049] S62: Combining the characteristics of construction scenarios and the needs of WBS management, a hierarchical storage architecture is adopted to build a knowledge base system (using a vector database (Milvus) to store unstructured data and a relational database (MySQL) to store structured data, with core layers including: basic compliance layer, hidden danger feature layer, case handling layer, and WBS management layer), and establishing an association mapping of "risk source - control measures - legal basis"; S63: Configure the role definition, responsibility boundaries, and multi-dimensional constraint instructions for the violation identification model; define the role of the safety hazard identification agent as "construction safety special analyst" and clarify its analytical perspective and responsibility boundaries; constraint instructions include accuracy constraints (high-altitude operation identification accuracy ≥90%), scene adaptation constraints (e.g., point cloud denoising algorithm is preferred for deep foundation pit scenarios), output specification constraints (sorted by hazard level, listing WBS locations and relevant basis), and logical constraints (e.g., fire protection facility configuration must be checked for hot work operations), to guide the agent to accurately execute identification tasks.

[0050] S64: Based on iteratively optimized datasets and externally captured real-time data sources, the knowledge base system is dynamically updated according to a preset cycle. On the one hand, the dataset is automatically synchronized and iterated, and new misjudgment cases, feedback hazards, and the latest specifications are entered into the knowledge base to update the mapping relationship. On the other hand, cutting-edge industry data is captured in real time through RAG technology to supplement new hazard characteristics and ensure that the knowledge base is upgraded in sync with construction scenarios and compliance requirements.

[0051] S65: For complex scenarios involving WBS nodes with insufficient recognition accuracy and high false positive rates, the RAG retrieval engine is used to retrieve similar scenario cases, feature parameters, and optimization schemes from the knowledge base. Positive samples (compliant behavior images / point clouds) and negative samples (unsafe behaviors and false positive scenarios) are extracted and incrementally trained on the YOLOv8 architecture model. During training, the focus is on optimizing scene adaptation parameters. For example, in deep foundation pit occlusion scenarios, the attention mechanism is adjusted to enhance personnel contour differentiation; in high-altitude operation scenarios, white balance parameters are optimized to improve light adaptability. Simultaneously, differentiated recognition thresholds are set for each WBS node (high-altitude operation safety helmet recognition threshold ≥90%, ground operation ≥85%). After training, the results are validated using a test set (30% of the iterative data is extracted) to ensure that the recognition accuracy of the corresponding WBS nodes improves by more than 10%, and the false positive rate is ≤3%.

[0052] S66: Adjust the feature point matching strategy of the image stitching algorithm by combining scene experience in the knowledge base system, adjust the SIFT feature point matching strategy of the image stitching algorithm, and add a redundancy detection mechanism; optimize the point cloud classification model, introduce the DBSCAN (density-based spatial clustering with noise) density clustering algorithm to remove interference point clouds such as construction materials and equipment shadows, enhance the distinction between personnel, components and environment, and ensure the association between point cloud data and dynamic 3D scene model and WBS nodes.

[0053] S7: Relying on dynamic 3D scene models, WBS decomposition node data, and violation behavior identification models, risk patterns are discovered and feedback is provided for optimization, and the feedback is simultaneously sent to the penetrating management interface.

[0054] Specifically, based on the 3D scene model, WBS decomposition node data, and the identification results of the violation behavior identification model, the steps of using decision tree and association rule mining algorithms to uncover risk distribution patterns and formulating optimization measures in conjunction with knowledge base system cases include: S71: Using the smallest grid unit divided by the 3D mesh model as the spatial reference and WBS nodes as the link between engineering parts, a risk event dataset integrating five-dimensional features of time, space, behavior, equipment, and environment is constructed. Each record corresponds to an early warning or an unsafe behavior identified by the agent. Fields include: grid ID, WBS code, engineering part type (deep foundation pit / high-altitude / ground / tunnel, etc.), behavior type, risk level, time of occurrence (day shift / night shift, and corresponding to the specific construction stage such as earthwork, main structure, decoration), personnel density (number of people in the grid extracted in real time from the point cloud), camera occlusion index (the proportion of occluded area in the monitored image of the grid), illuminance (calculated from the average brightness of the camera image), etc.

[0055] S72: Using CART decision trees with "whether an emergency warning (red card) is triggered" as the binary target variable, analyze the combined triggers of high-risk behaviors.

[0056] Tree node splitting is based on the Gini coefficient minimization criterion: in Let A be the proportion of category k in D, and let A be the candidate feature (such as the grid space height category). Gini coefficient; The Gini coefficient is the eigenvalue A. For dataset; For dataset 1; For dataset 2; select to make The system automatically generates interpretable rules based on the minimum features and cut-off points, such as: IF Spatial Height Category = "High Altitude" AND Construction Stage = "Main Structure" AND Personnel Density ≥ 3 people / grid AND Camera Obstruction Index > 0.2 THEN Emergency Warning Probability = 87%. These rules are directly mapped to WBS nodes, revealing that "severe camera obstruction and dense personnel during high-altitude main structure operations" is a high-risk scenario combination, rather than a single factor, achieving deep integration with the scenario.

[0057] Meanwhile, the weighted FP-Growth algorithm (frequent pattern growth algorithm) is introduced to incorporate grid risk exposure weights into the traditional support-confidence framework to explore frequent behavior-environment symbiotic patterns. Define the weights of grid j Based on its historical accident frequency, the criticality of WBS nodes, and the amount of personnel exposure: In the formula, The frequency of unsafe behaviors in this grid over the past 30 days. For the average daily exposure time of personnel, The criticality index of the WBS node (mapped by the fatal risk level associated with the node in the knowledge base, 0~1); α, β, γ are weight coefficients, each defaulting to 1 / 3, which can be adjusted by AHP from knowledge base cases; , , These represent the maximum values ​​of unsafe behavior frequency over the past 30 days, average daily personnel exposure hours, and WBS node criticality index across all grids.

[0058] Weighted support with weighted confidence Defined as: in Y represents the set of all grid transactions containing itemset X, and Y represents the consequent of the association rule, i.e., the result item or accompanying feature that frequently co-occurs with the precondition X. The system sets a minimum weighted support threshold (e.g., 0.05) and a minimum weighted confidence threshold (e.g., 0.6) to mine strong association rules, such as: {Spatial: High-altitude edge, Behavior: Not wearing a seatbelt} ⇒ {Time period: Afternoon shift} (WS=0.12, WC=0.78), indicating that the behavior of not wearing a seatbelt while working at a high-altitude edge is highly co-occurring during the afternoon shift and is significantly affected by the time period factor. These association rules are aggregated by WBS nodes to locate high-frequency co-occurring risk patterns in specific locations and output to the "Risk Induced - Pattern Library".

[0059] S73: Based on the above mining results, similar risk combination disposal cases are retrieved from the knowledge base using RAG, and a case similarity-weighted fusion recommendation is introduced: in The overall similarity between the new case and existing cases in the knowledge base. For WBS node similarity, For the similarity of unsafe behavior types, To assess the similarity of WBS construction environment characteristics, Similarity is calculated for construction phases and time patterns. Each sub-item's similarity is calculated using a cosine similarity coefficient, with the weight of λ dynamically adjusted based on the validity feedback of cases in the knowledge base, satisfying... λ 1+ λ 2+ λ 3+ λ 4=1, the initial value can be set by domain experts. Select the top-3 similarity case measures, and generate specific instructions based on current project constraints (such as personnel configuration and equipment resources). For example: "For WBS node 3# high-altitude welding grid, refer to case K-217, increase the number of on-site monitoring during the period of 14:00-16:00, and at the same time adjust the downward angle of the corresponding camera from 50° to 30° to reduce obstruction and missed detection. The measures are directly pushed to the safety director to support the issuance of instructions and process control."

[0060] like Figure 2 As shown, in a second aspect, the present invention also provides a multi-dimensional sensing security penetration management system, comprising: The data acquisition layer integrates drones, inspection robots, fixed high-definition cameras, and personnel positioning terminals to achieve real-time acquisition of three-dimensional video streams and personnel location data from the air, ground, and fixed perspectives, and supports synchronous transmission and caching of data from multiple devices. The model support layer uses GIS geographic information system and CAD construction drawings as dual foundations to construct a dynamic three-dimensional scene model of the construction site, and realizes grid division and visual annotation of engineering parts, supporting the accurate integration of multi-source data and three-dimensional model; The intelligent analysis layer deploys image stitching algorithms, point cloud classification models, and violation recognition models to achieve multi-view image fusion, 3D modeling, and violation recognition. A transparent management interface serves as the interface for early warning push notifications, handling tracking, and hierarchical viewing; The application output layer enables collaboration between mobile and PC terminals and supports alert push notifications, report generation, and historical data review.

[0061] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A multi-dimensional sensing-based security penetration management method, characterized in that, Includes the following steps: Obtain geospatial basic data and engineering design data of the construction area, and perform spatial registration processing; Based on the registered data, a three-dimensional model of the construction site is generated; Based on the 3D digital model, the target area is defined by using a selection tool, and the boundary coordinate data of the target area is extracted. Based on the spatial topology algorithm, the boundary coordinate data and the registered engineering design data are spatially correlated to identify all engineering structural objects contained in the target area, extract the attribute information of the engineering structural objects, and bind the attribute information to the corresponding area in the 3D digital model to generate region-structure-attribute correlation mapping data. Acquire multi-source sensing data, decode and stitch the multi-source sensing data, and perform spatial calibration with the 3D model to generate a dynamic 3D scene model containing dynamic entity positions, engineering part information, and WBS association information. Construct a violation identification model to identify safety anomalies, extract the spatial location of the anomalies and map them to the corresponding area of ​​the dynamic 3D scene model, associate engineering part information, and generate standardized early warning data to the penetrating management interface; Based on the association system of early warning data and "geographic coordinates-engineering structural parts-WBS decomposition nodes", we can achieve penetrating control and closed-loop management; By combining the WBS decomposition system and RAG technology, a dynamically updated knowledge base system is constructed, and the model algorithm is optimized. By leveraging dynamic 3D scene models, WBS decomposition node data, and violation identification models, risk patterns are identified and feedback is provided for optimization, with the feedback simultaneously transmitted to the penetrating management interface.

2. The multi-dimensional sensing security penetration management method according to claim 1, characterized in that: The geospatial basic data is GIS topographic data, and the engineering design data is CAD construction drawings; The steps to generate a 3D model of the construction site include: Obtain GIS topographic data and CAD construction drawings of the construction area, and analyze and extract their respective coordinate reference information; Within the overlapping area of ​​GIS topographic and CAD drawings, select no fewer than four pairs of feature points with the same name to obtain control point pairs; Based on the spatial similarity transformation solution, the coordinates of the control point pairs are calibrated, and the transformation parameters are obtained; The transformation parameters obtained from the settlement are applied to the element coordinates of all CAD drawings, and the CAD drawings are registered to the GIS terrain coordinate system. A surface model of the construction site is constructed using the corrected terrain data. The key structural elements are stretched or lofted into three-dimensional entities by reading the calibrated drawing data. The surface model and the structural entities are integrated to establish a three-dimensional model of the construction site. The mapping relationship between GIS terrain coordinates and CAD part codes is stored in the node attributes of the three-dimensional model.

3. The multi-dimensional sensing security penetration management method according to claim 1, characterized in that: The steps for generating region-structure-attribute association mapping data include: Draw an arbitrary polygonal bounding box in a 3D scene or GIS interface as the target area; calculate the geographic coordinate system based on the real-time captured screen coordinates, record the vertex sequence of the polygonal bounding box area, and close the first and last points to form polygon P; at the same time, project polygon P onto the planar projection coordinate system to obtain the planar polygon P′. Based on the calibrated spatial location of CAD primitives, an R-tree spatial index is built according to their minimum bounding rectangle, and each index node stores the CAD primitive ID, MBR boundary and the layer information. Based on the R-tree spatial index, all CAD primitives whose MBR intersects or contains the planar polygon P′ are quickly queried to form a candidate set C. For each CAD element in the candidate set C, spatial topological relationship judgment is performed based on its geometric type to obtain the set S of all related CAD elements within the polygon selection area; for each CAD element in the set S, attribute extraction and semantic recognition are performed to assign structured attribute information to each CAD element. Identify the grid cells corresponding to the selected polygon P′ and establish a grid ID; establish a many-to-many relationship between the grid cells and the set S, store information including but not limited to the geographic coordinates of the grid vertex and the set of location attributes, and generate region-structure-attribute association mapping data.

4. The multi-dimensional sensing security penetration management method according to claim 3, characterized in that: Methods for performing topological relation determination based on the geometric type of each CAD primitive in candidate set C include: If the geometric type of the CAD drawing element is a point element, use the ray method to determine whether the point is inside or on the boundary of the planar polygon P′. If the geometry type of a CAD primitive is a linear primitive, and any part of the line segment intersects with or is inside a planar polygon P′, it is determined to belong to the grid range. If the geometric type of the CAD primitive is a planar primitive, determine whether the boundary of the planar primitive intersects, contains, or overlaps with the planar polygon P′. If the intersection is not empty, the engineering structure part is included in the grid. Finally, obtain the set S of all relevant CAD elements within the polygonal selection area.

5. The multi-dimensional sensing security penetration management method according to claim 4, characterized in that: The steps for attribute extraction and semantic recognition for each CAD primitive in set S are as follows: Based on the CAD layer naming standard, the layer string is parsed and information including but not limited to professional, part type and subcategory is extracted and automatically mapped to standardized engineering part names. If the CAD element is a block reference with attributes, read the attribute labels, which include but are not limited to "WBS code", "part name" and "design strength", and directly obtain the structured part attributes; For CAD elements without explicit semantic annotations, their geometric features are extracted and combined with the spatial relationships of adjacent elements. Fuzzy matching is then performed with the built-in "Engineering Part Semantic Feature Library" to determine the most likely engineering part type.

6. The multi-dimensional sensing security penetration management method according to claim 1, characterized in that: Multi-source sensing data includes, but is not limited to, integrated drone aerial video streams, robot real-time inspection footage, and fixed camera monitoring video streams. The edge computing nodes perform real-time decoding, frame extraction, and coordinate synchronization of three-view video stream data, selecting clear, unobstructed personnel image frames with valid coordinate matching; an image stitching algorithm based on SIFT feature point matching is used to fuse and align personnel image frames from different perspectives; point cloud data is combined for point cloud classification and denoising, and the stitched images, point cloud data, and 3D model are calibrated to generate a dynamic 3D scene model containing personnel location, engineering parts, and WBS association information.

7. The multi-dimensional sensing security penetration management method according to claim 1, characterized in that: The violation identification model is built on the deep learning YOLOv8 architecture and incorporates construction scenario feature transfer training. The violation identification model has a built-in database of unsafe behaviors in construction scenarios and sets differentiated identification thresholds based on different violation types. When an unsafe behavior is detected, the system simultaneously extracts the personnel location coordinates corresponding to that behavior, combines them with the 3D model and coordinate mapping relationship to locate the specific grid cell, and associates it with the corresponding engineering part information to generate standardized early warning data. The early warning data includes, but is not limited to, behavior type, occurrence time, geographic coordinates, corresponding CAD structural part, WBS code, associated image frame, and point cloud screenshot. The penetrating management interface marks the WBS node and engineering part where the unsafe behavior occurred, and automatically associates the historical safety data under that WBS node. At the same time, the system can automatically associate the handling instructions with the corresponding WBS node and engineering part.

8. The multi-dimensional sensing security penetration management method according to claim 1, characterized in that: The steps for constructing a dynamically updated knowledge base system and optimizing the model algorithm by combining the WBS decomposition system and RAG technology include: Regularly collect core system metrics data and user feedback data to create an iterative optimization dataset; Combining the characteristics of construction scenarios with the needs of WBS management, a hierarchical storage architecture is adopted to build a knowledge base system and establish an association mapping of "risk source - control measures - legal basis"; Configure the role definition, responsibility boundaries, and multi-dimensional constraint instructions for the violation identification model; The knowledge base system is dynamically updated according to a preset cycle based on iteratively optimized datasets and externally crawled data sources. By combining scene experience from the knowledge base system, the feature point matching strategy of the image stitching algorithm is adjusted, and a redundancy detection mechanism is added; the point cloud classification model is optimized, and interference data is removed through cluster analysis to enhance the distinction between target entities and the environment, ensuring the association between point cloud data and dynamic 3D scene model and WBS nodes.

9. The multi-dimensional sensing security penetration management method according to claim 1, characterized in that: Based on the 3D scene model, WBS decomposition node data, and the identification results of the violation behavior recognition model, the steps of using decision tree and association rule mining algorithms to uncover risk distribution patterns and formulating optimization measures in conjunction with knowledge base system cases include: Using the smallest grid unit of the three-dimensional mesh model as the spatial reference and WBS nodes as the link between engineering parts, a risk event dataset integrating five-dimensional features of time, space, behavior, equipment and environment is constructed. We used CART decision trees with "whether an emergency warning is triggered" as the binary target variable to analyze the combined causes of high-risk behaviors. At the same time, we introduced the weighted FP-Growth algorithm to incorporate grid risk exposure weights into the traditional support-confidence framework to explore frequent behavior-environment symbiotic patterns. Based on the above mining results, similar risk combination disposal cases were retrieved from the knowledge base using RAG, and case similarity-weighted fusion recommendations were introduced to obtain corresponding optimization measures.

10. A multi-dimensional sensing security penetration management system, based on the multi-dimensional sensing security penetration management method as described in any one of claims 1 to 9, characterized in that, include: The data acquisition layer integrates drones, inspection robots, fixed high-definition cameras, and personnel positioning terminals to achieve real-time acquisition of three-dimensional video streams and personnel location data from the air, ground, and fixed perspectives. The model support layer uses GIS geographic information system and CAD construction drawings as dual bases to construct a dynamic three-dimensional scene model of the construction site and realize the grid division and visual annotation of engineering parts. The intelligent analysis layer deploys image stitching algorithms, point cloud classification models, and violation recognition models to achieve multi-view image fusion, 3D modeling, and violation recognition. A transparent management interface serves as the interface for early warning push notifications, handling tracking, and hierarchical viewing; The application output layer enables collaboration between mobile and PC terminals and supports alert push notifications, report generation, and historical data review.

Images