Traffic facility risk state real-time monitoring method based on point cloud detection

Abstract

The invention relates to the field of traffic facility monitoring, in particular to a traffic facility risk state real-time monitoring method based on point cloud detection, and the method comprises the steps: collecting a three-dimensional point cloud and a two-dimensional image through a laser radar and a camera in a time-space synchronization manner, and building a multi-modal data set; calculating a dumping angle based on template matching, ICP registration, PCA normal extraction and voxelization integrity verification; the inclination angle is compared with a multi-level threshold value, and when the inclination angle exceeds the threshold value, asynchronous early warning of interface window popup, acousto-optic prompt and remote pushing is triggered; after image preprocessing and feature extraction, a convolutional neural network is adopted to identify abrasion, corrosion and crack diseases and quantify the damage degree; a risk threshold model is constructed, space attitude and surface state features are fused, confidence assessment is performed through dynamic weight weighting and time sequence accumulation analysis in combination with lightweight LSTM prediction, and finally a security level is determined and graded early warning is triggered. According to the invention, closed-loop monitoring from discrete acquisition to continuous intelligent perception is realized, and the risk identification precision and response speed are improved.

Description

Technical Field

[0001] This invention relates to the field of traffic facility monitoring, and in particular to a method for real-time monitoring of the risk status of traffic facilities based on point cloud detection. Background Technology

[0002] In recent years, the focus of highway construction has gradually shifted from large-scale new construction to the reconstruction and expansion of existing lines. These projects require expansion under strict constraints while maintaining normal traffic flow on existing roads, resulting in extremely complex construction environments. They must contend with undulating terrain such as mountains and hills, and operate in critical structural areas such as tunnel entrances, bridge approach bridges, high-fill and deep-cut roadbeds, and high slopes, exhibiting typical characteristics of "dynamic interaction and overlapping risks." During construction, large, tall equipment (such as bridge erecting machines and hoisting machinery) frequently performs high-level or cross-line operations. Due to their high center of gravity and sensitive structural stability, coupled with limited operating space and continuous disturbance from surrounding traffic flow, any instability accidents such as overturning or sliding can not only cause equipment damage and project delays but may also directly encroach on adjacent operating lanes, posing a fatal threat to high-speed vehicles (especially motor vehicles traveling at 80-120 km / h), creating a chain of safety risks in both the construction and operation spaces.

[0003] Currently, the industry's risk management for the construction of tall structures in renovation and expansion projects still relies primarily on the traditional "human-based" model: mainly depending on on-site safety officers' regular inspections and visual observations, or on local data collection through simple sensors. This approach has significant technical limitations: on the one hand, manual monitoring is limited by blind spots in observation angles, nighttime or inclement weather conditions, differences in experience, and distractions, making it difficult to achieve accurate all-weather perception of key parameters such as the overall posture and structural integrity of the equipment; on the other hand, point sensors can only acquire discrete local data, failing to reflect changes in the overall spatial state of the equipment, and the phenomenon of data silos is serious, making it difficult to identify the risk accumulation process in a timely manner through multi-parameter correlation analysis. Crucially, traditional monitoring methods are almost nonexistent for flexible protective structures such as protective nets and temporary support bamboo rafts—although these facilities do not directly participate in construction operations, they serve as core safety barriers to intercept falling rocks and earthwork collapses; their collapse, displacement, or damage will directly sever the "risk interception chain." However, due to its soft material and non-rigid structure, conventional contact sensors are difficult to attach effectively; non-contact monitoring is also easily affected by environmental factors such as construction dust and low light at night, resulting in its operation being in a long-term "unknowable and uncontrollable" management blind spot.

[0004] As transportation infrastructure develops towards intelligence and safety, there is an urgent need to break through the technical bottlenecks of traditional "human-based monitoring + point-based monitoring." This invention integrates next-generation information technology and engineering safety theory to develop a non-contact real-time monitoring system capable of penetrating complex environments, specifically designed for the unique scenarios of reconstruction and expansion projects. This system dynamically captures the overall spatial posture of tall structures and flexible protective structures, combining real-time data analysis and intelligent threshold early warning to achieve early and accurate identification and proactive intervention of risks such as tipping, slippage, and damage. It effectively compensates for the shortcomings of existing monitoring technologies in terms of coverage, sensing accuracy, and environmental adaptability, comprehensively enhancing the safety resilience of both the "construction-operation" spaces in reconstruction and expansion projects, and providing key technical support for the safe construction of highway reconstruction and expansion projects. Summary of the Invention

[0005] Therefore, the present invention provides a method for real-time monitoring of the risk status of traffic facilities based on point cloud detection, in order to solve the aforementioned problems existing in the prior art.

[0006] To achieve the above objectives, the present invention provides a method for real-time monitoring of the risk status of traffic facilities based on point cloud detection, comprising:

[0007] Step S1: Real-time acquisition of 3D point cloud data and 2D image data of traffic facilities to establish a multimodal dataset;

[0008] Step S2: Based on the multimodal data set, the current state of the traffic facilities is identified in real time to obtain the identification result, and the tilting angle is calculated based on the identification result;

[0009] Step S3: Compare the tilting angle with the preset risk threshold. If the threshold is exceeded, trigger the early warning branch and output the risk alarm through the multi-channel asynchronous notification mechanism. Otherwise, enter the normal monitoring branch and record the status information to obtain the spatial attitude analysis result.

[0010] Step S4: Based on the two-dimensional image data, perform image processing and pattern recognition to analyze the surface condition of the traffic facilities and quantify the degree of damage to obtain the surface condition analysis results.

[0011] Step S5: Based on the risk threshold model, a comprehensive health status assessment is performed on the spatial attitude analysis results and the surface state analysis results to determine the safety level, and a corresponding graded early warning mechanism is triggered.

[0012] Furthermore, the process of step S2 includes:

[0013] Based on the 3D point cloud data in the multimodal dataset, a subset of the target region point cloud is extracted by template region of interest matching;

[0014] An iterative nearest-point algorithm is used to iteratively optimize and align the region template point cloud and the region scene point cloud;

[0015] The registered point cloud is downsampled and principal component analysis is performed to extract the normal direction. At the same time, the matched scene point cloud is subjected to voxelized space filling feature analysis to determine the structural integrity.

[0016] Only when the structural integrity judgment is passed, the angle between the vertical projection vector of the template point cloud and the scene point cloud is calculated based on the normal direction and used as the tilt angle.

[0017] Furthermore, the process of downsampling the registered point cloud and performing principal component analysis to extract the normal direction, while simultaneously performing voxelized space-filling feature analysis on the matched scene point cloud to determine structural integrity, includes:

[0018] The point cloud is divided into a voxel grid of predetermined size using a voxel grid filtering method, and the center point of each grid is retained.

[0019] The normal direction is obtained by calculating the point cloud covariance matrix and extracting the eigenvector corresponding to the minimum eigenvalue.

[0020] The voxelization space filling feature analysis divides the matched scene point cloud into a voxel grid of predetermined size along the three axes, counts the point cloud thickness within each voxel along the vertical direction and calculates the thickness variance, and determines the structure to be complete when the variance is less than a preset threshold.

[0021] Furthermore, the process of calculating the point cloud covariance matrix and extracting the eigenvector corresponding to the minimum eigenvalue as the normal direction includes:

[0022] ,

[0023] in, Let M be the centroid of the point cloud, and M be the total number of points in the point cloud after downsampling. Represents the outer product of vectors. , For the downsampled point cloud data, the covariance matrix is ​​solved. eigenvalues and corresponding feature vectors , , The eigenvector corresponding to the smallest eigenvalue That is, the normal direction of the local plane of the point cloud. .

[0024] Furthermore, the process of dividing the matched scene point cloud into a voxel grid of predetermined size along the three axes and calculating the point cloud thickness within each voxel and the thickness variance along the vertical direction includes:

[0025] The matched scene point cloud is divided into voxel meshes along the X, Y, and Z axes;

[0026] For each voxel, the vertical distance between the highest and lowest points of its internal point cloud is calculated along the vertical direction as the thickness of the voxel.

[0027] The thickness of all effective voxel units is statistically analyzed and their variance is calculated.

[0028] Furthermore, the process of step S3 includes:

[0029] The tilting angle is calculated using the vector dot product formula and then converted to a degree system.

[0030] The tilting angle is compared with a preset risk threshold, wherein...

[0031] If the tilt angle is greater than or equal to the risk threshold, it is determined to be a tipping risk state. The tipping angle, timestamp and matching quality parameters are recorded to the log, and the risk state is displayed in real time on the user interface. The asynchronous early warning mechanism of interface pop-up, sound prompt and remote message push is triggered synchronously.

[0032] If the tilt angle is less than the risk threshold, it is determined to be a safe state, the status information is recorded to the log, and the monitoring status is displayed normally on the user interface; the spatial attitude analysis result includes the tilt angle, risk judgment status, timestamp, and point cloud matching quality parameters.

[0033] Furthermore, the process of step S4 includes:

[0034] Preprocessing operations such as denoising, enhancement, and color correction are performed on the two-dimensional image data to obtain the processing result;

[0035] Based on the processing results, surface features such as texture, color distribution, and crack shape are extracted to obtain the diseased area;

[0036] The convolutional neural network or semantic segmentation model is used to detect and classify the diseased areas to identify disease types including at least wear, corrosion, cracks and dirt.

[0037] The area proportion of each type of diseased area is calculated to quantify the degree of damage, and the surface health status level is assessed to obtain the surface condition analysis results.

[0038] Furthermore, the process of calculating the area proportion of various diseased areas to quantify the degree of damage and assessing the surface health status level to obtain the surface condition analysis results includes:

[0039] For each type of disease area detected, the number of pixels is counted and divided by the total number of pixels in the image to obtain the area percentage.

[0040] The comprehensive damage index is calculated by setting weighting coefficients based on the area proportion and the severity of the disease type.

[0041] The comprehensive damage index is compared with a preset health status threshold to classify the surface health status level;

[0042] The surface condition analysis results include the type of disease, the area ratio of each type of disease, the comprehensive damage index, and the surface health status level.

[0043] Furthermore, the process of step S5 includes:

[0044] Based on the dynamic configuration of a multi-dimensional threshold vector including tilting angle threshold, surface damage index threshold, environmental wind speed threshold and vibration amplitude threshold in the current construction stage, the spatial attitude analysis result and the surface state analysis result are compared with each threshold component respectively. When any component exceeds the corresponding threshold, a risk candidate is generated and the second layer of judgment is activated.

[0045] Calculate the cumulative excess of tilt angle in multiple consecutive frames of historical data and compare it with a cumulative threshold related to the structure type. When the cumulative excess exceeds the cumulative threshold, trigger a trend warning and activate the third layer of judgment.

[0046] A lightweight time-series prediction model is used to predict the tilt angle in the future time period based on historical frame sequences, and the risk exceedance probability of the prediction confidence interval is calculated. When the risk exceedance probability is greater than a preset probability threshold, a pre-warning is triggered and the fourth layer of judgment is activated.

[0047] A multi-factor coupled risk function integrating spatial attitude, surface defects, environmental vibration and structural dynamic characteristics is constructed. Based on the judgment results of each previous level, the weight allocation in the coupled risk function is dynamically adjusted and risk adjudication is performed to obtain the adjudication result. The adjudication result is mapped to a safety level and a graded early warning mechanism is triggered.

[0048] Furthermore, the process of constructing a multi-factor coupled risk function that integrates spatial attitude, surface defects, environmental vibration, and structural dynamic characteristics, and dynamically adjusting the weight allocation in the coupled risk function based on the judgment results of each previous level to obtain the judgment result includes:

[0049] The multi-factor coupled risk function integrates normalized spatial attitude features, surface disease features, environmental vibration features, and structural dynamic characteristics. Each feature is weighted and fused through learnable dynamic weights to obtain a risk probability value. The dynamic weights are adaptively adjusted according to the highest warning level in the judgment results from the first to the third layer.

[0050] The risk probability value is compared with a dynamic convergence threshold, which is based on a nonlinear mapping between the multi-dimensional threshold vector and the warning level coefficient. The higher the warning level, the higher the threshold sensitivity.

[0051] When the risk probability value is greater than the dynamic convergence threshold and continues for a preset number of frames, a final risk alarm is triggered.

[0052] Compared with existing technologies, the advantages of this invention are as follows: This invention achieves spatiotemporal fusion of LiDAR and camera, utilizing the characteristic of laser penetrating dust to acquire three-dimensional geometric shapes, and combining this with image texture to form complementary perception, thus solving the problem of missing information in complex construction scenarios. After point cloud is template-matched and ICP-registered, principal component analysis is used to extract normal vectors as attitude references, and simultaneous voxelization of thickness variance verifies structural continuity; this dual verification avoids noise-induced misjudgments. The overturning angle exceeding the limit triggers an early warning, conforming to the torque balance principle under gravity. Surface defect identification involves quantifying and weighting defects such as cracks and corrosion, with weights allocated according to degradation laws such as cracks weakening the section modulus and corrosion reducing material strength, making the damage index negatively correlated with load-bearing capacity. The risk threshold model transforms instantaneous anomalies into a continuous risk measure by incorporating historical dip angles through a second-layer sliding integral and introducing registration quality weights, reflecting the time dependence of damage accumulation. The third-layer LSTM utilizes sequence autocorrelation to predict future states and calculates the risk exceedance probability through confidence intervals, achieving risk extrapolation. The fourth-layer coupling function employs a hybrid of linear weighting and nonlinear product terms, exponentially amplifying risk when multiple factors are simultaneously elevated, simulating the positive feedback domino effect in engineering accidents. Dynamic weights enforce the dominance of key risks based on the warning level, while the adaptive threshold decreases sensitivity as the risk level increases. The final decision requires the risk probability to continuously exceed the threshold, and the monitoring frequency is dynamically adjusted in conjunction with the safety level, forming a closed loop of data collection, analysis, prediction, decision-making, and response, improving the accuracy and speed of risk identification. Attached Figure Description

[0053] Figure 1 A flowchart illustrating the real-time monitoring method for the risk status of traffic facilities based on point cloud detection provided by this invention;

[0054] Figure 2 This is a flowchart illustrating step S2 in the point cloud-based real-time monitoring method for traffic facility risk status provided by the present invention.

[0055] Figure 3 This is a flowchart illustrating step S4 in the point cloud-based real-time monitoring method for traffic facility risk status provided by the present invention.

[0056] Figure 4 This is a flowchart illustrating step S5 in the point cloud-based real-time monitoring method for traffic facility risk status provided by the present invention. Detailed Implementation

[0057] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0058] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0059] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0060] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0061] Please see Figure 1 As shown, this invention provides a method for real-time monitoring of the risk status of traffic facilities based on point cloud detection, including:

[0062] Step S1: Real-time acquisition of 3D point cloud data and 2D image data of traffic facilities to establish a multimodal dataset;

[0063] Specifically, a lidar and camera collaborative acquisition device is set up at the monitoring site. The lidar is responsible for emitting laser beams and receiving reflected signals to acquire high-density three-dimensional point cloud data of the target traffic facilities (protective nets / bamboo rafts). The point cloud data includes the geometric shape, spatial location, and distance information of the facility's surface. The image acquisition component (visible light camera or infrared camera) simultaneously captures two-dimensional image data of the facility, supplementing texture, color, and detailed features. Both are fixed to the same support platform (such as a tripod or monitoring tower) through a rigid connection structure to ensure approximately consistent viewing angles. The system adopts a unified time control strategy to achieve multi-sensor timing coordination: the lidar and camera are simultaneously triggered by an external synchronization signal (such as a PPS pulse) to ensure that the data acquisition time difference is less than 10 milliseconds; software timestamp alignment: a unified time identifier based on GPS or NTP protocol synchronization is embedded in the data packet to record the precise acquisition time of each frame of point cloud and image; a timestamp index buffer is established on the industrial control computer to automatically match the nearest neighbor point cloud frame and image frame according to the time window (such as ±50ms) to form data pairs that are strictly corresponding in the time dimension. The matched 3D point cloud data and 2D image data are encapsulated into a multimodal data set. Each point cloud frame in the set is associated with an image frame at the corresponding time, and a point cloud-pixel mapping relationship is established in the data structure (such as through calibrated camera intrinsic and extrinsic parameters), providing a foundation for subsequent point cloud coloring, feature fusion, and surface state analysis.

[0064] Step S2: Based on the multimodal data set, the current state of the traffic facilities is identified in real time to obtain the identification result, and the tilting angle is calculated based on the identification result;

[0065] Specifically, such as Figure 2 As shown, the process of step S2 includes:

[0066] Step S21: Based on the 3D point cloud data in the multimodal data set, extract a subset of the target region point cloud by template region of interest matching;

[0067] Specifically, the system first loads pre-stored normal state point cloud template data, which is obtained through offline acquisition of the standard pose of traffic facilities (protective nets / bamboo rafts) during the initial installation phase. The minimum and maximum x, y, and z coordinates of this template point cloud in 3D space are calculated to construct an initial 3D bounding box. Based on actual construction and installation deviations and sensor acquisition errors, the boundary range of this bounding box in key directions (especially the height direction) is artificially expanded to form a Region of Interest (ROI). The system then filters out subsets of point clouds located within this ROI from both the template point cloud and the current frame scene point cloud, generating regional template point clouds and regional scene point clouds. This process extracts relevant point clouds of the target facility from complex construction backgrounds and removes irrelevant background interference.

[0068] Step S22: The iterative nearest point algorithm is used to iteratively optimize and align the region template point cloud and the region scene point cloud.

[0069] Specifically, the ICP (Iterative Closest Point) algorithm is used to iteratively optimize and align the regional template point cloud and the regional scene point cloud. The system constructs a KD-Tree data structure to accelerate nearest neighbor search. For each point in the template point cloud, its nearest neighbor matching point is searched in the scene point cloud, and corresponding point pairs with a distance less than a set threshold are retained. By minimizing the distance error function between point pairs, the rotation matrix and translation vector are iteratively calculated to gradually align the regional template point cloud to the pose of the regional scene point cloud. The process terminates when the registration error difference between two consecutive iterations is less than a preset tolerance or the maximum number of iterations is reached, and the registration matrix is ​​output to achieve accurate spatial positioning and pose alignment of the target traffic facility in the scene.

[0070] Step S23: Downsample the registered point cloud and perform principal component analysis to extract the normal direction. At the same time, perform voxelized space filling feature analysis on the matched scene point cloud to determine the structural integrity.

[0071] Specifically, the process of downsampling the registered point cloud and performing principal component analysis to extract the normal direction, while simultaneously performing voxelized space-filling feature analysis on the matched scene point cloud to determine structural integrity, includes:

[0072] The point cloud is divided into a voxel grid of predetermined size using a voxel grid filtering method, and the center point of each grid is retained.

[0073] Specifically, to reduce data redundancy, improve computational efficiency, and suppress noise interference, the system performs downsampling processing on both the registered template point cloud and the matched scene point cloud. Using the VoxelGrid filtering method, the voxel grid side length parameters are pre-set (0.02~0.1 meters based on the size of the protective net / bamboo raft and the point cloud density), dividing the point cloud space into several cubic voxel units. For each voxel unit, all original points contained within it are counted, and the geometric center or mean coordinates are calculated as the representative point of that voxel. The representative point is retained, and the remaining points are discarded. This process, while preserving the macroscopic geometry of the facility, compresses the amount of point cloud data in a single frame to 10%~20% of the original size, significantly reducing the computational load of subsequent PCA calculations and voxelization analysis. Simultaneously, the averaging operation effectively suppresses random measurement noise and environmental floating dust points in the original lidar point cloud, improving data quality.

[0074] The normal direction is obtained by calculating the point cloud covariance matrix and extracting the eigenvector corresponding to the minimum eigenvalue.

[0075] Specifically, the process of calculating the point cloud covariance matrix and extracting the eigenvector corresponding to the minimum eigenvalue as the normal direction includes:

[0076] ,

[0077] in, Let M be the centroid of the point cloud, and M be the total number of points in the point cloud after downsampling. Represents the outer product of vectors. , For the downsampled point cloud data, the covariance matrix is ​​solved. eigenvalues and corresponding feature vectors , , The eigenvector corresponding to the smallest eigenvalue That is, the normal direction of the local plane of the point cloud. .

[0078] Specifically, for downsampled point cloud data ,in( Let M be the 3D coordinate vector of the l-th point (M is the total number of points after downsampling). The system first calculates the centroid of the point cloud, which represents the geometric center of the point cloud in 3D space. This centroid is obtained by adding the coordinate vectors of all points and dividing by the total number of points. The centroid is used for subsequent decentroiding, eliminating the influence of overall point cloud translation on geometric feature extraction and ensuring that the covariance matrix only reflects the shape and orientation distribution characteristics of the point cloud. Based on the centroid, the system performs a decentroiding operation on each point to obtain the decentroided coordinate vector. Then, the outer product of each decentroid vector and itself is calculated. The outer product is a 3×3 matrix that encodes the directional dispersion of a point relative to the centroid. Summing the outer product matrices over all points and taking the average yields the covariance matrix of the point cloud. The covariance matrix C is essentially a symmetric positive definite matrix, where each element represents the correlation between different coordinate dimensions, and the overall matrix describes the distribution shape and principal extension direction of the point cloud in three-dimensional space. Eigenvalue decomposition is performed on the covariance matrix C to solve for its eigenvalues. and the corresponding feature vectors , , According to the PCA principle, the largest eigenvalue... Corresponding feature vector It indicates the direction of the most discrete point cloud distribution (i.e., the main extension direction), while the minimum eigenvalue... Corresponding feature vector This indicates the direction in which the point cloud distribution is most concentrated, and this direction is perpendicular to the optimal plane for local fitting of the point cloud. For approximately planar structures such as protective nets / bamboo rafts, the eigenvector corresponding to the minimum eigenvalue is... Let n be the normal direction of the local point cloud plane (i.e., n = This method extracts normals from the global distribution through statistical averaging, making it more robust to noise and point density variations compared to local surface fitting methods.

[0079] The voxelization space filling feature analysis divides the matched scene point cloud into a voxel grid of predetermined size along the three axes, counts the point cloud thickness within each voxel along the vertical direction and calculates the thickness variance, and determines the structure to be complete when the variance is less than a preset threshold.

[0080] Specifically, the voxelization space-filling feature analysis divides the matched scene point cloud into a voxel grid of predetermined size along three axes, and calculates the point cloud thickness within each voxel and the thickness variance along the vertical direction, including:

[0081] The matched scene point cloud is divided into voxel meshes along the X, Y, and Z axes;

[0082] Specifically, the scene point cloud (i.e., the target point cloud after registration and alignment in step S22, i.e., the point cloud whose spatial position is matched with the template point cloud) is uniformly divided into several cubic voxel units along the X, Y, and Z axes. The side length u of each voxel unit is preset according to the physical size of the protective net / bamboo raft and the point cloud scanning resolution (typically ranging from 0.05 to 0.15 meters). This division method transforms the disordered point cloud set into a regular three-dimensional grid structure, which facilitates statistical feature analysis. The system traverses each point cloud and calculates the index (i, j, k) of the voxel based on its three-dimensional coordinates (x, y, z), realizing the spatial discretization and grid classification of the point cloud, laying the foundation for subsequent voxel-by-voxel statistics.

[0083] For each voxel, the vertical distance between the highest and lowest points of its internal point cloud is calculated along the vertical direction as the thickness of the voxel.

[0084] Specifically, for each voxel unit (index q), the system statistically analyzes the distribution characteristics of its internal point cloud along the vertical direction (Z-axis). Specifically, it retrieves the z-coordinate values ​​of all point clouds within that voxel and extracts the maximum value. and minimum value The difference between the two is calculated as the vertical thickness of the voxel unit. This thickness value characterizes the physical depth of the protective net / bamboo raft within the local space area. For a structurally complete and tightly installed bamboo raft, its thickness distribution should be approximately equal to the diameter of the bamboo raft or the weaving thickness of the protective net (usually 5-15 cm), and remain relatively stable across different voxels.

[0085] The thickness of all effective voxel units is statistically analyzed and their variance is calculated.

[0086] Specifically, this involves removing blank voxels that do not contain any point cloud and thickness. For abnormally small (e.g., less than 2 cm) invalid voxels, only voxel units containing valid structural point clouds are retained, and the total number of valid voxels is denoted as Q. The mean thickness of all valid voxels is calculated. Then, the thickness variance is calculated:

[0087]

[0088] in, For the first Z-axis thickness of individual units The total number of effective voxel units. This represents the average thickness along the Z-axis.

[0089] Specifically, when the thickness variance Less than the set threshold (e.g.) Typical value This corresponds to a thickness fluctuation range of 10cm in real-world scenarios, i.e. If the point cloud is relatively uniformly distributed in the vertical direction, and has the geometric continuity and thickness consistency that a protective net or bamboo raft structure should have, then the object is determined to be a valid target structure and retained for subsequent tilting state analysis; otherwise, it is determined to be invalid and removed.

[0090] Step S24: Only when the structural integrity judgment is passed, the angle between the vertical projection vector of the template point cloud and the scene point cloud is calculated based on the normal direction as the tilt angle.

[0091] Specifically, only when the structural integrity assessment passes, the system calculates the projection vectors of the template point cloud and the scene point cloud in the vertical direction (or gravity direction) based on the normal direction n. The angle between the two projection vectors is calculated using the vector dot product formula and converted into a numerical value in degrees as the tilt angle. This angle quantifies the degree of tilt of the current scene point cloud relative to the normal state template point cloud, achieving a precise numerical representation of the tilting posture of the traffic facility.

[0092] Step S3: Compare the tilting angle with the preset risk threshold. If the threshold is exceeded, trigger the early warning branch and output the risk alarm through the multi-channel asynchronous notification mechanism. Otherwise, enter the normal monitoring branch and record the status information to obtain the spatial attitude analysis result.

[0093] Specifically, step S3 includes the following process:

[0094] The tilting angle is calculated using the vector dot product formula and then converted to a degree system.

[0095] Specifically, based on the template point cloud normal direction extracted in step S24 Scene point cloud normal direction The system calculates the angle between the projection vectors of the two normals onto the vertical direction using the dot product formula. Let the approximate vector of the gravity direction be... Then the vertical projection vector is and After obtaining the cosine value through dot product, the inverse cosine function is used to calculate the angle in radians, which is then converted to degrees to obtain the tilt angle θ (unit: degrees). The conversion formula is as follows: This calculation process ensures that the angle value accurately quantifies the degree of tilt of the current facility relative to its normal installation posture, with a numerical accuracy of 0.1°, providing a reliable basis for subsequent threshold comparisons.

[0096] The tilting angle is compared with a preset risk threshold, wherein...

[0097] If the tilt angle is greater than or equal to the risk threshold, it is determined to be a tipping risk state. The tipping angle, timestamp and matching quality parameters are recorded to the log, and the risk state is displayed in real time on the user interface. The asynchronous early warning mechanism of interface pop-up, sound prompt and remote message push is triggered synchronously.

[0098] If the tilt angle is less than the risk threshold, it is determined to be a safe state, the status information is recorded to the log, and the monitoring status is displayed normally on the user interface; the spatial attitude analysis result includes the tilt angle, risk judgment status, timestamp, and point cloud matching quality parameters.

[0099] Specifically, the system performs a nonlinear comparison between the calculated tilt angle θ and preset multi-level risk thresholds. The threshold system is set at three levels based on engineering safety specifications and facility type, with typical values ​​as follows:

[0100] Level 1 Warning (Attention): θ∈[5°,8°), indicating slight tilt requiring close observation;

[0101] Level 2 warning (alert): θ∈[8°,12°), indicating moderate tilt requiring intervention;

[0102] Level 3 alarm (danger): θ≥12°, indicating severe tilting requiring immediate action.

[0103] If θ is less than the minimum threshold (5°), it is determined to be a safe state, and the process enters the normal monitoring branch; if θ is greater than or equal to any threshold, it is determined to be a corresponding dumping risk state according to the threshold level exceeded, and the process enters the early warning branch.

[0104] Specifically, when a dumping risk is identified, the system simultaneously performs three recording and notification operations:

[0105] Data storage layer: The tilt angle θ, risk level, current timestamp (accurate to milliseconds), and point cloud matching quality parameters (such as ICP registration error and number of valid matching point pairs) are written into a separate risk log file in a structured format, and risk marker bits are set to facilitate subsequent data traceability and accident review analysis.

[0106] User interface layer: The current tilt angle, risk level and facility location information are displayed in real time in a red highlighted pop-up window in the user interface, while triggering periodic sound alarms (such as a buzzer sound at 2-second intervals) to achieve on-site real-time early warning.

[0107] Network transmission layer: Through the built-in 4G / 5G communication module, it pushes structured alarm messages containing time, location, tilt angle and risk level to the remote monitoring platform or mobile APP. It supports multiple channels such as SMS, WeChat and in-platform notifications to ensure that managers can obtain risk information as soon as possible even if they are not on site.

[0108] If the system determines the status to be safe, it performs only lightweight recording and display operations: recording the tilt angle, timestamp, and point cloud matching quality parameters to a regular log file (distinct from the risk log) for daily status statistics and health trend analysis. The current monitoring status is displayed normally in the user interface with a green indicator, without triggering any alarms, keeping the monitoring interface simple and avoiding information overload for operators.

[0109] Step S4: Based on the two-dimensional image data, perform image processing and pattern recognition to analyze the surface condition of the traffic facilities and quantify the degree of damage to obtain the surface condition analysis results.

[0110] Specifically, such as Figure 3 As shown, the process of step S4 includes:

[0111] Step S41: Perform denoising, enhancement, and color correction preprocessing operations on the two-dimensional image data to obtain the processing result;

[0112] Specifically, bilateral filtering or nonlocal mean filtering (NLM) algorithms are used to remove image speckles and salt-and-pepper noise caused by dust, moisture, and sensor noise in the construction environment. Bilateral filtering smooths noise while preserving edge details, avoiding blurring of disease edges; NLM uses image patch similarity for weighted averaging, resulting in better noise suppression in areas with repetitive textures (such as protective mesh). Adaptive histogram equalization (CLAHE) is applied to enhance image contrast and solve the problem of local underexposure / overexposure caused by low light at night or direct strong light. CLAHE divides the image into several sub-regions, performs histogram equalization on each, and sets contrast limit parameters (clip limit = 2.0~3.0) to prevent noise amplification, making the grayscale differences of disease features (such as fine cracks and rust spots) more significant. Automatic white balance (AWB) and color space conversion are performed to eliminate color cast problems under different lighting conditions (such as cloudy days, dusk, and artificial light sources), converting the image from RGB space to HSV or Lab color space. The hue (H) component of the HSV space is more distinguishable in terms of the yellowish-brown of corroded areas and the dark black of cracks; the ab component of the Lab space is more sensitive to the red of rust and the gray of dirt, providing a stable color benchmark for subsequent color feature extraction.

[0113] Step S42: Based on the processing results, extract surface features such as texture, color distribution, and crack shape to obtain the diseased area;

[0114] Specifically, texture roughness and uniformity are extracted using Gray-Level Co-occurrence Matrix (GLCM) or Local Binary Pattern (LBP) operators. GLCM calculates the contrast, entropy, and correlation of the image in the 0°, 45°, 90°, and 135° directions. Corroded areas exhibit high entropy and low correlation, while contaminated areas exhibit low contrast and high uniformity. LBP encodes local pixel gray-level difference patterns; the LBP histogram of crack areas shows specific directional peaks, significantly different from normal mesh textures. Histogram distributions of hue, saturation, and brightness are statistically analyzed in the HSV color space, and color moments (mean, variance, and skewness) are calculated. Wear areas show increased brightness variance due to surface material detachment; the hue mean of corroded areas is biased towards the orange-red range (H∈[10°,30°]); contaminated areas show decreased saturation and concentrated brightness distribution. Linear structures are extracted using Canny edge detection or Hessian matrix eigenvalue analysis. The Canny detector captures crack edge pixels using dual thresholds (e.g., low threshold = 50, high threshold = 150). The Hessian matrix analyzes the principal curvature direction of each pixel, marking regions with high linear response intensity as candidate cracks. Morphological closing operations are then used to connect the fracture edges, and connected component analysis is used to extract the minimum bounding rectangle and skeleton of the crack. The crack length, width, and number of branches are calculated to achieve a geometric parameterized description of the crack.

[0115] Step S43: Use a convolutional neural network or semantic segmentation model to detect and classify the diseased area to identify at least the disease types including wear, corrosion, cracks and dirt.

[0116] Specifically, the convolutional neural network crops the candidate regions extracted in step S42 into fixed-size image patches (e.g., 224×224) and inputs them into a pre-trained ResNet-50 or VGG-16 network. The network extracts deep features through multiple convolutional layers and outputs four-class probabilities (wear, corrosion, cracks, and dirt) at fully connected layers, suitable for fine-grained classification after the diseased areas have been located. The semantic segmentation model performs end-to-end pixel-level segmentation on the preprocessed image, using a U-Net or DeepLabv3+ architecture. The encoder extracts multi-scale features through the ResNet backbone network, and the decoder restores spatial resolution through upsampling and skip connections, outputting a disease category probability map of the same size as the input image, with each pixel labeled with its disease type or normal background. This method does not require pre-extraction of candidate regions and has a higher detection rate for minor diseases (such as early rust spots). The model is trained offline using a pre-labeled dataset of disease images. This dataset contains at least 5,000 samples covering different lighting conditions, angles, and disease severity. Data augmentation (rotation, flipping, and brightness adjustment) is employed to improve generalization ability. During online monitoring, the model weights are loaded onto the GPU or NPU acceleration unit of an industrial control computer, and the single-frame inference time is controlled within 100-200 milliseconds to meet real-time processing requirements.

[0117] Step S44: Calculate the area proportion of various diseased areas to quantify the degree of damage, and assess the surface health status level to obtain the surface status analysis results.

[0118] Specifically, the process of calculating the area proportion of various types of diseased areas to quantify the degree of damage and assessing the surface health status level to obtain the surface condition analysis results includes:

[0119] For each type of disease area detected, the number of pixels is counted and divided by the total number of pixels in the image to obtain the area percentage.

[0120] Specifically, the pixel set Si is defined for the i-th type of defect (i∈{wear, corrosion, crack, dirt}), and its pixel count Ni is obtained by traversing all pixels labeled as category i in the image and accumulating them. To improve statistical accuracy, the system adopts a dual-threshold strategy: for the probability map output by the semantic segmentation model, only pixels with a confidence score greater than 0.7 are included in Si, and low-confidence noise points are removed; for candidate regions detected by CNN, only lesions with a connected pixel count greater than 50 are counted, avoiding misclassification of isolated noise points as defects. The total effective pixel count Ntotal belonging to the traffic facility area in the image is calculated, that is, background areas such as the sky, ground, and construction machinery are excluded through masking operations. The mask is generated by projecting the ROI point cloud from step S21 onto the image plane, ensuring that the statistical range is strictly limited to the surface of the target facility. The area percentage Ai of the i-th type of disease is obtained by the normalization formula: Ai = Ntotal / Ni × 100%. This indicator objectively reflects the coverage of each type of disease on the facility surface in the form of a percentage. For example, A corrosion = 15% means that the corroded area accounts for 15% of the total area of ​​the facility.

[0121] The comprehensive damage index is calculated by setting weighting coefficients based on the area proportion and the severity of the disease type.

[0122] Specifically, based on traffic engineering specifications and historical accident data, cracks have the highest weight (w_crack = 0.4) because they directly weaken the structural bearing capacity and are prone to expansion; corrosion leads to material performance degradation and has the second highest weight (w_corrosion = 0.3); wear affects appearance and durability but does not endanger structural safety in the short term, so it has a lower weight (w_wear = 0.2); and contamination mainly has a visual impact and has the lowest weight (w_contamination = 0.1). The sum of the weight coefficients ∑wi = 1 to ensure the standardization of the comprehensive index. In special scenarios, the weights can be adaptively adjusted. For example, when cracks are detected concentrated at the edge of the facility or at connection nodes (high-risk areas), w_crack automatically increases by 0.05; when signs of peeling appear in the corrosion area, w_corrosion automatically increases by 0.03, increasing the sensitivity of high-risk defects. The comprehensive damage index D is obtained by weighting and summing the area proportions of each type of defect with their corresponding weights. D is a value between 0 and 100, representing the combined effects of multiple types of defects. For example, D=28.5 might correspond to 10% cracks, 15% corrosion, 3.5% wear, and 0% fouling, directly reflecting the overall degree of facility degradation. Compared to a single area indicator, this index more scientifically characterizes the comprehensive risk of multiple concurrent defects.

[0123] The comprehensive damage index is compared with a preset health status threshold to classify the surface health status level;

[0124] Specifically, the comprehensive damage index D is compared with preset multi-level health status thresholds to classify surface health status levels, achieving a standardized mapping from qualitative to quantitative analysis: Based on the characteristics of facility materials, service life, and maintenance standards, three threshold levels are set: Good (D<5%): No obvious defects or only minor stains on the surface, no intervention required; Attention (5%≤D<15%): Slight wear or stains exist, recommended to be included in the regular inspection plan; Warning (15%≤D<30%): Moderate corrosion or cracks, maintenance should be arranged within 1 month; Dangerous (D≥30%): Severe cracks or multiple defects occur simultaneously, posing a risk of structural failure, requiring immediate replacement or reinforcement.

[0125] The surface condition analysis results include the type of disease, the area ratio of each type of disease, the comprehensive damage index, and the surface health status level.

[0126] Step S5: Based on the risk threshold model, a comprehensive health status assessment is performed on the spatial attitude analysis results and the surface state analysis results to determine the safety level, and a corresponding graded early warning mechanism is triggered.

[0127] Specifically, the risk threshold model achieves a complete decision-making loop from instantaneous anomaly detection to trend evolution prediction and comprehensive risk quantification through the organic synergy of multi-dimensional threshold vector configuration, time-series cumulative analysis, prediction confidence assessment and multi-factor coupled adjudication.

[0128] Specifically, such as Figure 4 As shown, the process of step S5 includes:

[0129] Step S51: Based on the current construction stage, dynamically configure a multi-dimensional threshold vector including tilt angle threshold, surface damage index threshold, environmental wind speed threshold and vibration amplitude threshold. Compare the spatial attitude analysis result and the surface state analysis result with each threshold component. When any component exceeds the corresponding threshold, generate a risk candidate and activate the second layer of judgment.

[0130] Specifically, the threshold configuration mode is automatically switched by parsing the BIM construction logs or manually entered progress nodes in the project progress management system. The threshold vector T = [θ_th, D_th, V_th, A_th, T_th] contains five core components:

[0131] θ_th (tilting angle threshold): dynamically adjusted according to the structure type. Rigid protective netting θ_th = 5° (initial stage) → 6° (mid-stage) → 7.5° (late stage); flexible bamboo rafts θ_th = 8° → 10° → 12°, reflecting the strategy of increasing safety margin as construction disturbance decreases with the construction period.

[0132] D_th (Surface Damage Index Threshold): Set based on material degradation tolerance. Metal mesh D_th=25%; bamboo structure D_th=20%, reflecting the differences in sensitivity of different materials to corrosion / cracking.

[0133] V_th (Ambient Wind Speed ​​Threshold): Connects to real-time data from the meteorological station at the construction site and corrects for equipment height. At a height of 10 meters, V_th = 12 m / s; at a height of 20 meters, V_th = 10 m / s, and the wind pressure gradient correction formula V_th(z) = V_th(10) × (z / 10)^(-0.12) is applied.

[0134] A_th (Vibration Amplitude Threshold): Integrates accelerometer spectrum data to distinguish construction vibration sources. During mechanical operations, A_th = 3g; during transportation, A_th = 1.5g. The vibration type is identified through the frequency band energy distribution.

[0135] T_th (Temperature Gradient Threshold): To address the effects of thermal expansion and contraction, a daily temperature difference ΔT_th = 25℃ or an hourly temperature change rate T_rate_th = 5℃ / h is set to prevent misjudgment of material fatigue. The threshold vector is automatically updated hourly based on the construction stage index and stored in Flash non-volatile memory to prevent loss in case of power failure. The spatial attitude analysis results (including tipping angle θ and point cloud matching quality parameter σ_ICP) and the surface condition analysis results (including comprehensive damage index D and surface health status level H_health) are normalized and compared component by component to generate risk candidates and encode them as risk status words: The priority level of tipping risk is calculated according to the following formula:

[0136]

[0137] Where θ is the tilting angle, and θ_th is the tilting angle threshold corresponding to the current construction stage. This integer priority reflects the degree of exceeding the limit, increasing by 1 level for every 20% exceeding the threshold.

[0138] For the comprehensive damage index D, perform multi-disease synergistic enhancement calculation:

[0139]

[0140] Where D is the comprehensive damage index, D_th is the surface damage index threshold, and N_disease_types is the number of detected concurrent disease types (the union of wear, corrosion, cracks, and fouling). This formula reflects the risk superposition effect when multiple diseases coexist.

[0141] When the wind speed V_wind>V_th or the vibration amplitude A_vib>A_th, the risk amplification mode is activated, and P_θ and P_D are uniformly upgraded by 1 level to reflect the deteriorating effect of the harsh environment on structural safety.

[0142] When any component exceeds the threshold, the second-level judgment is activated, and the candidate set and its priority are encoded into an 8-bit binary risk status word (RSW) and broadcast to the edge computing node via the CAN bus.

[0143] Step S52: Calculate the cumulative excess tilt angle of historical data for multiple consecutive frames and compare it with the cumulative threshold related to the structure type. When the cumulative excess exceeds the cumulative threshold, trigger a trend warning and activate the third layer judgment.

[0144] Specifically, a sliding time window buffer B_θ[t-N_frame+1,...,t] of length N_frame=180 frames (corresponding to 3 minutes @ 1Hz sampling) is established to store the historical tilt angle sequence. The accumulated excess (AE) is defined as the positive integral of the tilt angle sequence with respect to a threshold:

[0145]

[0146] Where θ_k is the tilt angle of the k-th frame, and θ_th is the tilt angle threshold corresponding to the current construction stage. This integral considers both the tilt angle amplitude and duration to avoid false triggering due to momentary jitter. To enhance robustness, a point cloud matching quality parameter σ_ICP (ICP registration error output in step S3) is introduced for dynamic weighting. The formula for the weighted cumulative excess is:

[0147]

[0148] Among them, the dynamic weighting coefficient w_ICP,k is determined by the registration error:

[0149]

[0150] σ_ICP,k represents the ICP registration error of the k-th frame, and σ_0 represents the baseline error. This weight ensures that the contribution of frames with poor registration quality is automatically attenuated, avoiding the accumulation of noise data that could contaminate the final result.

[0151] Set the differential cumulative threshold AE_limit based on the structural dynamics characteristics:

[0152] Rigid frame structure: AE_limit = N_frame × 2°, allowing small periodic oscillations.

[0153] Flexible suspension structure: AE_limit = N_frame × 4°, tolerating wind-induced low-frequency oscillations.

[0154] The system performs a comparison and judgment: if AE_weighted(t) > AE_limit, a trend warning is triggered, the third-level judgment is activated, and the cumulative excess rate AE_rate = AE_weighted(t) / N_frame is used as the input feature of the third-level layer. The sign sequence of tilt angle change direction Sgn(Δθ_k) within the time window is calculated. If the number of alternations N_reverse > 5 (e.g., frequent oscillations), it is determined to be an environmental disturbance rather than a true tilt trend, and the trend warning is automatically suppressed. Simultaneously, it requires AE to exceed the limit for three consecutive time windows before final confirmation to prevent occasional excess noise.

[0155] Step S53: A lightweight time-series prediction model is used to predict the tilt angle in the future time period based on the historical frame sequence, and the risk exceedance probability of the prediction confidence interval is calculated. When the risk exceedance probability is greater than the preset probability threshold, a pre-warning is triggered and the fourth layer of judgment is activated.

[0156] Specifically, a lightweight prediction model based on LSTM is adopted, with the following network structure: input layer (5-dimensional) → LSTM layer (64 hidden units, dropout=0.2) → fully connected layer (32 units, ReLU) → output layer (2 units, predicting μ_pred and σ_pred respectively). The input feature vector X_t=[θ_t, AE_rate, V_wind, A_vib, T_temp] contains the current tilt angle, cumulative rate, and environmental parameters to achieve physical information embedding. In the offline phase, 1000 segments of historical construction monitoring data (each segment ≥500 frames) are used for pre-training, with the loss function being negative log-likelihood.

[0157]

[0158] Where μ_pred,i and σ_pred,i are the predicted mean and standard deviation of the i-th sample, respectively, and θ_true,i is the actual observed tilt angle. During the online phase, the parameters of the last two layers are fine-tuned weekly using data from the most recent 7 days, with a learning rate η = 1e-4 to prevent model drift.

[0159] Perform Monte Carlo sampling (M=200 times) on the predicted output to generate the tilt angle distribution θ_future ~ N(μ_pred, σ_pred^2) for the next Δt=30 seconds. Calculate the probability of risk exceeding the threshold (PoE):

[0160]

[0161] Where Φ(·) is the standard normal cumulative distribution function, θ_th is the tilt angle threshold, and μ_pred and σ_pred are the mean and standard deviation of the prediction model output.

[0162] If PoE > PoE_limit = 15%, a predictive warning is triggered, activating the fourth level of judgment. Simultaneously, Conditional Value at Risk (CVaR) is calculated as an indicator of extreme risk intensity.

[0163]

[0164] in, The standard normal probability density function is used. CVaR quantifies the expected tilt level once the risk occurs and is used for weight adjustment in the fourth layer.

[0165] Calculate the first-order autocorrelation coefficient ρ_1 of the continuous prediction residual ε_t = θ_true_t - μ_pred_t. If |ρ_1| > 0.3, it indicates that there is a systematic error in the model, and a model retraining request is automatically triggered. At the same time, monitor the calibration of σ_pred_t: compare the coverage of the prediction interval [μ_pred ± 1.96σ_pred] with 95%. If the deviation is > 5%, adjust the bias term of the σ_pred output layer for online calibration.

[0166] Step S54: Construct a multi-factor coupled risk function that integrates spatial attitude, surface defects, environmental vibration and structural dynamic characteristics. Based on the judgment results of each previous level, dynamically adjust the weight allocation in the coupled risk function and make a risk judgment to obtain the judgment result. Map the judgment result to a safety level and trigger a graded early warning mechanism.

[0167] Specifically, the process of constructing a multi-factor coupled risk function that integrates spatial attitude, surface defects, environmental vibration, and structural dynamic characteristics, and dynamically adjusting the weight allocation in the coupled risk function based on the judgment results of each previous level to obtain the judgment result includes:

[0168] The multi-factor coupled risk function integrates normalized spatial attitude features, surface disease features, environmental vibration features, and structural dynamic characteristics. Each feature is weighted and fused through learnable dynamic weights to obtain a risk probability value. The dynamic weights are adaptively adjusted according to the highest warning level in the judgment results from the first to the third layer.

[0169] Specifically, a four-dimensional feature vector F = [f_pose, f_surface, f_vib, f_dynamic] is constructed, and each component is normalized before being input into the coupling function.

[0170] Spatial attitude characteristics: f_pose = min(θ / θ_max, 1) × (1 + 0.2×P_θ), where θ_max=20° is the physical limit tilt angle, and P_θ is the tilt angle exceeding priority calculated in the first layer.

[0171] Surface damage characteristics: f_surface = min(D / D_max, 1) × (1 + 0.1×H_health), where D_max=50% is the maximum permissible damage index, and H_health∈{0,1,2,3} is the surface health status level (good, attention, warning, danger).

[0172] Environmental vibration characteristics: f_vib = min(A_vib_rms / A_limit, 1) × S_vib, where A_vib_rms is the effective value of vibration, A_limit is the vibration amplitude threshold, and S_vib∈[0,1] is the spectral kurtosis, which characterizes the impact.

[0173] Structural dynamic characteristics: f_vib =min× (1 + ΔT / ΔT_critical), where ΔT is the temperature gradient and ΔT_critical = 50℃ is the critical temperature difference.

[0174] The weight vector W = [w_pose, w_surface, w_vib, w_dynamic] is dynamically generated through a gating network, and its adjustment logic strictly follows the "highest warning level among the judgment results of the first to third layers".

[0175] A miniature multilayer perceptron (MLP) is used, with the input layer receiving a 5-dimensional vector:

[0176] X_gate = [RSW, L_2, L_3, ζ_structure, S_vib]

[0177] Wherein, RSW is an 8-bit binary risk status word, encoding the priority P_θ and P_D of the candidate generated in the first layer; L_2 is the second layer trend warning activation flag (0 / 1), set to 1 when AE_weighted > AE_limit; L_3 is the third layer prediction warning activation flag (0 / 1), set to 1 when PoE > PoE_limit; ζ_structure is the structural integrity flag (0=fracture, 1=intact) output by the voxelization analysis in step S2; S_vib is the environmental vibration kurtosis.

[0178] The MLP hidden layer has 8 neurons, using the ReLU activation function; the output layer has 4 neurons whose weights are generated using Softmax normalization.

[0179]

[0180] The weight adjustment rules follow the safety-first principle: if the third layer is activated (L_3=1), w_pose is forced to be ≥0.5, highlighting the dominance of predictive risk; if the structural integrity flag ζ_structure=0 (structural fracture), w_pose=0.9 and w_surface=0.1 are set, with structural fracture taking precedence; if the environmental vibration S_vib>0.6 (strong impact), w_vib is automatically increased by 30%, but the upper limit does not exceed 0.25. The weight vector is updated every frame and smoothed using EMA: W_smooth = 0.9×W_smooth + 0.1×W_current, to prevent weight oscillation.

[0181] The coupling risk function comprises two terms: a linear weighted term and a nonlinear coupling term, which are mapped to the probability space via the Sigmoid function.

[0182] Where σ(·) is the Sigmoid activation function, α = 0.7 is the linear weighted principal coefficient, and β = 0.3 is the nonlinear coupling coefficient;

[0183] γ_i ∈ [0.5, 1.5] is the feature sensitivity index, determined through offline Bayesian optimization: using historical accident tracing data as the optimization objective, the goal is to maximize the detection rate of the risk probability R for real safety events, and the value of γ_i is fixed after 100 iterations and convergence. When any feature f_i is close to 1 (high risk), the product term (1+f_i)^{γ_i} exponentially amplifies the risk; when multiple features are simultaneously at high values, the nonlinear coupling term dominates.

[0184] The risk probability value is compared with a dynamic convergence threshold, which is based on a nonlinear mapping between the multi-dimensional threshold vector and the warning level coefficient. The higher the warning level, the higher the threshold sensitivity.

[0185] Specifically, the dynamic threshold R_th is generated based on the multi-dimensional threshold vector T = [θ_th, D_th, V_th, A_th, T_th] and the warning level coefficient K, thereby achieving adaptive adjustment of the threshold sensitivity.

[0186]

[0187] Where R_base = 0.6 is the baseline threshold under no-warning conditions; K = max(P_θ, P_D, L_2, L_3) is the highest warning level among the previous layer judgment results (range 1~4); T_norm is the normalization index for environmental parameter exceedance, calculated as follows:

[0188]

[0189] When the wind speed V_wind or vibration A_vib exceeds its threshold, T_norm→1, and the threshold R_th decreases by about 5%, improving the response sensitivity. The higher the warning level, the lower the threshold R_th (e.g., R_th drops to 0.36 when K=4), and the system's ability to detect weak risk signs increases non-linearly.

[0190] When the risk probability value is greater than the dynamic convergence threshold and continues for a preset number of frames, a final risk alarm is triggered.

[0191] Specifically, the system maintains a continuous over-limit counter C_R for the risk probability value R. When R > R_th, the counter increments: C_R ← C_R + 1; otherwise, it resets: C_R ← 0. A final risk alarm is triggered when the following condition is met:

[0192]

[0193] That is, at a sampling rate of 1Hz, the risk probability must continuously exceed the dynamic threshold for at least 5 seconds to avoid false alarms due to instantaneous disturbances. The risk probability value R is mapped to a four-level security level to trigger a differentiated early warning mechanism.

[0194] First safety level (safe): R ≤ 0.3, the facility is considered to be in a stable state, the monitoring frequency is reduced to f_sample=0.2Hz, and only routine logs are recorded.

[0195] Second security level (attention): 0.3 < R ≤ 0.5, indicating a potential risk trend, monitoring frequency is increased to f_sample=0.5Hz, and a risk briefing is pushed to the engineering management platform at 18:00 every day.

[0196] Third safety level (warning): 0.5 < R ≤ 0.7, indicating significant risk accumulation. The monitoring frequency is encrypted to f_sample=1Hz, and messages are pushed to the mobile terminal of the on-site engineer in real time. The user interface flashes a yellow warning.

[0197] Fourth safety level (hazardous): R > 0.7, indicating the structure is on the verge of failure. The highest monitoring frequency is f_sample=2Hz. Immediately activate the on-site audible and visual alarm (110dB buzzer + red and blue strobe) and trigger a three-channel remote notification in parallel.

[0198] SMS channel: Send a text containing the tilt angle θ, damage index D, and risk probability R to the project manager;

[0199] Voice call channel: Automatically dials preset emergency numbers and repeatedly broadcasts risk levels and facility numbers;

[0200] Platform push channel: Pushes structured alarm data packets to the smart construction site platform to trigger emergency response plans.

[0201] Specifically, this invention achieves spatiotemporal fusion of LiDAR and camera, utilizing the ability of lasers to penetrate dust to acquire three-dimensional geometric shapes. This, combined with image texture, forms complementary perception, addressing the problem of missing information in complex construction scenarios. After template matching and ICP registration, point clouds are processed by principal component analysis to extract normal vectors as attitude references. Simultaneous voxelization of thickness variance verifies structural continuity, and this dual verification avoids noise-induced misjudgments. Over-limit tilting angle triggers an early warning system consistent with the torque balance principle under gravity. Surface defect identification quantifies and weights defects such as cracks and corrosion, allocating weights based on degradation patterns such as cracks weakening the cross-sectional modulus and corrosion reducing material strength, resulting in a negative correlation between the damage index and bearing capacity. The risk threshold model, through a second layer that integrates historical tilt angles and introduces registration quality weights, transforms instantaneous anomalies into continuous risk measures, reflecting the time dependence of damage accumulation. The third layer, LSTM, uses sequence autocorrelation to predict future states, calculating the risk exceedance probability through confidence intervals to extrapolate risk. The fourth layer coupling function employs a mixture of linear weighting and nonlinear product terms, exponentially amplifying risk when multiple factors are simultaneously elevated, simulating a positive feedback domino effect in engineering accidents. Dynamic weights enforce the dominance of key risks based on the warning level, while adaptive thresholds reduce sensitivity as the risk level increases. The final decision requires the risk probability to consistently exceed the threshold. Monitoring frequency is dynamically adjusted based on the safety level, forming a closed loop of data collection, analysis, prediction, decision-making, and response, thus improving the accuracy and speed of risk identification.

[0202] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

[0203] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for real-time monitoring of the risk status of traffic facilities based on point cloud detection, characterized in that, include: Step S1: Real-time acquisition of 3D point cloud data and 2D image data of traffic facilities to establish a multimodal dataset; Step S2: Based on the multimodal data set, the current state of the traffic facilities is identified in real time to obtain the identification result, and the tilting angle is calculated based on the identification result; Step S3: Compare the tilting angle with the preset risk threshold. If the threshold is exceeded, trigger the early warning branch and output the risk alarm through the multi-channel asynchronous notification mechanism. Otherwise, enter the normal monitoring branch and record the status information to obtain the spatial attitude analysis result. Step S4: Based on the two-dimensional image data, perform image processing and pattern recognition to analyze the surface condition of the traffic facilities and quantify the degree of damage to obtain the surface condition analysis results. Step S5: Based on the risk threshold model, a comprehensive health status assessment is performed on the spatial attitude analysis results and the surface state analysis results to determine the safety level, and a corresponding graded early warning mechanism is triggered.

2. The method for real-time monitoring of traffic facility risk status based on point cloud detection according to claim 1, characterized in that, The process of step S2 includes: Based on the 3D point cloud data in the multimodal dataset, a subset of the target region point cloud is extracted by template region of interest matching; An iterative nearest-point algorithm is used to iteratively optimize and align the region template point cloud and the region scene point cloud; The registered point cloud is downsampled and principal component analysis is performed to extract the normal direction. At the same time, the matched scene point cloud is subjected to voxelized space filling feature analysis to determine the structural integrity. Only when the structural integrity judgment is passed, the angle between the vertical projection vector of the template point cloud and the scene point cloud is calculated based on the normal direction and used as the tilt angle.

3. The method for real-time monitoring of traffic facility risk status based on point cloud detection according to claim 2, characterized in that, The process of downsampling the registered point cloud and performing principal component analysis to extract the normal direction, while simultaneously performing voxelized space-filling feature analysis on the matched scene point cloud to determine structural integrity, includes: The point cloud is divided into a voxel grid of predetermined size using a voxel grid filtering method, and the center point of each grid is retained. The normal direction is obtained by calculating the point cloud covariance matrix and extracting the eigenvector corresponding to the minimum eigenvalue. The voxelization space filling feature analysis divides the matched scene point cloud into a voxel grid of predetermined size along the three axes, counts the point cloud thickness within each voxel along the vertical direction and calculates the thickness variance, and determines the structure to be complete when the variance is less than a preset threshold.

4. The method for real-time monitoring of traffic facility risk status based on point cloud detection according to claim 3, characterized in that, The process of calculating the point cloud covariance matrix and extracting the eigenvector corresponding to the minimum eigenvalue as the normal direction includes: ， in, Let M be the centroid of the point cloud, and M be the total number of points in the point cloud after downsampling. Represents the outer product of vectors. , For the downsampled point cloud data, the covariance matrix is ​​solved. eigenvalues and corresponding feature vectors The eigenvector corresponding to the smallest eigenvalue That is, the normal direction of the local plane of the point cloud. .

5. The method for real-time monitoring of traffic facility risk status based on point cloud detection according to claim 4, characterized in that, The voxelization space-filling feature analysis divides the matched scene point cloud into a voxel grid of predetermined size along three axes, and calculates the point cloud thickness within each voxel and the thickness variance along the vertical direction, including: The matched scene point cloud is divided into voxel meshes along the X, Y, and Z axes; For each voxel, the vertical distance between the highest and lowest points of its internal point cloud is calculated along the vertical direction as the thickness of the voxel. The thickness of all effective voxel units is statistically analyzed and their variance is calculated.

6. The method for real-time monitoring of traffic facility risk status based on point cloud detection according to claim 5, characterized in that, The process of step S3 includes: The tilting angle is calculated using the vector dot product formula and then converted to a degree system. The tilting angle is compared with a preset risk threshold, wherein... If the tilt angle is greater than or equal to the risk threshold, it is determined to be a tipping risk state. The tipping angle, timestamp and matching quality parameters are recorded to the log, and the risk state is displayed in real time on the user interface. The asynchronous early warning mechanism of interface pop-up, sound prompt and remote message push is triggered synchronously. If the tilt angle is less than the risk threshold, it is determined to be a safe state, the status information is recorded to the log, and the monitoring status is displayed normally on the user interface; the spatial attitude analysis result includes the tilt angle, risk judgment status, timestamp, and point cloud matching quality parameters.

7. The method for real-time monitoring of traffic facility risk status based on point cloud detection according to claim 6, characterized in that, The process of step S4 includes: Preprocessing operations such as denoising, enhancement, and color correction are performed on the two-dimensional image data to obtain the processing result; Based on the processing results, surface features such as texture, color distribution, and crack shape are extracted to obtain the diseased area; The convolutional neural network or semantic segmentation model is used to detect and classify the diseased areas to identify disease types including at least wear, corrosion, cracks and dirt. The area proportion of each type of diseased area is calculated to quantify the degree of damage, and the surface health status level is assessed to obtain the surface condition analysis results.

8. The method for real-time monitoring of traffic facility risk status based on point cloud detection according to claim 7, characterized in that, The process of calculating the area proportion of various diseased areas to quantify the degree of damage and assessing the surface health status level to obtain the surface condition analysis results includes: For each type of disease area detected, the number of pixels is counted and divided by the total number of pixels in the image to obtain the area percentage. The comprehensive damage index is calculated by setting weighting coefficients based on the area proportion and the severity of the disease type. The comprehensive damage index is compared with a preset health status threshold to classify the surface health status level; The surface condition analysis results include the type of disease, the area ratio of each type of disease, the comprehensive damage index, and the surface health status level.

9. The method for real-time monitoring of traffic facility risk status based on point cloud detection according to claim 8, characterized in that, The process of step S5 includes: Based on the dynamic configuration of a multi-dimensional threshold vector including tilting angle threshold, surface damage index threshold, environmental wind speed threshold and vibration amplitude threshold in the current construction stage, the spatial attitude analysis result and the surface state analysis result are compared with each threshold component respectively. When any component exceeds the corresponding threshold, a risk candidate is generated and the second layer of judgment is activated. Calculate the cumulative excess of tilt angle in multiple consecutive frames of historical data and compare it with a cumulative threshold related to the structure type. When the cumulative excess exceeds the cumulative threshold, trigger a trend warning and activate the third layer of judgment. A lightweight time-series prediction model is used to predict the tilt angle in the future time period based on historical frame sequences, and the risk exceedance probability of the prediction confidence interval is calculated. When the risk exceedance probability is greater than a preset probability threshold, a pre-warning is triggered and the fourth layer of judgment is activated. A multi-factor coupled risk function integrating spatial attitude, surface defects, environmental vibration and structural dynamic characteristics is constructed. Based on the judgment results of each previous level, the weight allocation in the coupled risk function is dynamically adjusted and risk adjudication is performed to obtain the adjudication result. The adjudication result is mapped to a safety level and a graded early warning mechanism is triggered.

10. The method for real-time monitoring of traffic facility risk status based on point cloud detection according to claim 9, characterized in that, The process of constructing a multi-factor coupled risk function that integrates spatial attitude, surface defects, environmental vibration, and structural dynamic characteristics, and dynamically adjusting the weight allocation in the coupled risk function based on the judgment results of each previous level to obtain the judgment result includes: The multi-factor coupled risk function integrates normalized spatial attitude features, surface disease features, environmental vibration features, and structural dynamic characteristics. Each feature is weighted and fused through learnable dynamic weights to obtain a risk probability value. The dynamic weights are adaptively adjusted according to the highest warning level in the judgment results from the first to the third layer. The risk probability value is compared with a dynamic convergence threshold, which is based on a nonlinear mapping between the multi-dimensional threshold vector and the warning level coefficient. The higher the warning level, the higher the threshold sensitivity. When the risk probability value is greater than the dynamic convergence threshold and continues for a preset number of frames, a final risk alarm is triggered.

Images