Engineering safety and quality intelligent evaluation method and system based on machine vision

Abstract

The present application belongs to the cross technical field of engineering intelligent management and computer vision, and specifically relates to an engineering safety and quality intelligent evaluation method and system based on machine vision. The method generates fusion perception data by spatiotemporal registration of visual data collected by unmanned aerial vehicles and fixed monitoring, extracts scene semantic feature vectors of the same work area under different perspectives, and calculates cross-perspective semantic consistency measurement values. After detecting consistency abnormality attenuation, the method respectively maps and generates three-dimensional evaluation values of safety, quality and progress through multi-channel decomposition, and couples and fuses the evaluation values into unified construction health evaluation indexes. The method injects the indexes into a digital twin model for deviation prediction and closed-loop adaptive adjustment of perception strategies. The present application reverses cross-perspective semantic differences from registration noise into state mutation detection signals, and realizes multi-dimensional coupled evaluation and closed-loop management and control of safety, quality and progress.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of intelligent engineering management and computer vision, specifically relating to a machine vision-based intelligent evaluation method and system for engineering safety and quality. Background Technology

[0002] Construction sites involve multiple elements such as personnel, machinery, environment, and engineering entities. Their safety management, quality control, and progress tracking have long relied on manual inspections and experience-based judgment, resulting in inherent flaws such as incomplete perception coverage, delayed response, and strong subjectivity. In recent years, the rapid development of drone technology and video surveillance technology has provided new technical means for intelligent perception at construction sites; however, existing solutions still have significant shortcomings in terms of data fusion depth and coupling of control dimensions.

[0003] Chinese invention patent CN114219687B discloses an intelligent method for identifying construction safety hazards that integrates human-machine vision. This method uses an eye tracker to obtain a target salience map by tracking eye movements, combines this with a convolutional neural network to classify and identify hazard locations, and uses a semantic knowledge base for similarity matching to achieve automatic hazard identification. However, this solution only focuses on the single task dimension of construction safety hazard identification, without addressing construction quality deviation detection and schedule deviation analysis. Furthermore, its perception data comes from a single fixed camera, lacking the collaborative perception capabilities of drones and fixed monitoring, and failing to utilize cross-perspective information from multi-source heterogeneous visual data to improve the comprehensiveness of management and control.

[0004] Chinese patent application CN120495954A discloses a construction site safety monitoring method and system based on large-scale models and RAG (Real-Time Image Processing). This method uses a large visual model to identify safety violations in construction site images, and combines retrieval-enhanced generation technology for regulatory matching and automatic generation of safety logs. However, this solution is also limited to the single functional dimension of safety monitoring and heavily relies on the reasoning capabilities of large-scale language models. It fails to establish a coupled analytical framework between safety, quality, and schedule, and does not provide a closed-loop feedback control mechanism based on digital twins.

[0005] Chinese invention patent CN111006646A discloses a method for monitoring construction progress based on UAV oblique photography technology. This method uses UAVs to acquire oblique images, constructs a true 3D model, and compares and analyzes it with the planned progress. However, this solution only achieves the single function of progress monitoring and lacks the ability to identify safety hazards and detect quality defects. Furthermore, its data acquisition method is intermittent UAV oblique photography, which lacks the continuity and complementarity with fixed video surveillance.

[0006] A comprehensive analysis of existing technologies reveals that the core bottleneck in the current construction management field lies in the fact that drone inspection data and fixed video surveillance data are processed as two independent sensing systems. The spatiotemporal differences and perspective differences between the two are all regarded as registration errors that need to be eliminated. This leads to the discarding of abrupt changes in the state of construction entities contained in cross-perspective information as noise. The cross-correlation of the three types of management events—safety, quality, and progress—is severed, making it impossible to achieve multi-dimensional coupled evaluation and closed-loop management within a unified feature space. Summary of the Invention

[0007] To address the technical bottlenecks in existing engineering construction visual perception technologies, such as the independent processing of UAV inspection data and fixed monitoring data, and the discarding of construction state change signals contained in cross-view information as registration noise, which prevents the realization of multi-dimensional coupled evaluation of safety, quality, and schedule, this invention provides a machine vision-based intelligent evaluation method and system for engineering safety and quality. By reversing the cross-view semantic consistency measurement from registration noise to state change detection signals and constructing a unified health evaluation index system, this invention achieves coupled detection and closed-loop evaluation of three types of control events—safety, quality, and schedule—within a unified feature space, based on the principle of scene semantic representation space consistency decay, under the premise of the coexistence of two heterogeneous visual data sources: intermittent UAV patrols and continuous fixed monitoring observations.

[0008] The technical solution of this invention is as follows:

[0009] The machine vision-based intelligent evaluation method for engineering safety and quality includes the following steps:

[0010] Step S1: Divide the construction site into multiple work areas, perform spatiotemporal registration on the first visual data collected by the UAV at the construction site and the second visual data collected by the fixed monitoring equipment, and generate fused perception data based on scene geometric constraints and timestamp alignment;

[0011] Step S2: Based on the fused perception data, extract the scene semantic feature vectors of the same work area from the perspective of the UAV and the perspective of the fixed monitoring. The UAV perspective and the fixed monitoring perspective together constitute two perspectives. Calculate the cross-perspective semantic consistency metric value between the scene semantic feature vectors under the two perspectives. Construct a reference baseline for the cross-perspective semantic consistency metric value of the work area under normal working conditions and detect the abnormal decay of the cross-perspective semantic consistency metric value relative to the reference baseline.

[0012] Step S3: Based on the spatiotemporal distribution pattern of the abnormal attenuation, safety risk evaluation value, quality deviation evaluation value, and schedule deviation evaluation value are generated respectively. The safety risk evaluation value, quality deviation evaluation value, and schedule deviation evaluation value are then fused through coupling weights to generate a unified construction health evaluation index.

[0013] Step S4: The construction health evaluation index is injected into the digital twin model of the construction site to predict the deviation and obtain the predicted deviation. The predicted deviation is then fed back to adaptively adjust the inspection strategy of the UAV and the area of ​​interest of the fixed monitoring equipment. The inspection strategy includes the flight path of the UAV and the area of ​​interest is the monitoring range of the fixed monitoring equipment.

[0014] This invention also provides a machine vision-based intelligent evaluation system for engineering safety and quality, comprising a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to perform the functions of the following modules:

[0015] The air-ground collaborative perception fusion module is configured to divide the construction site into multiple work zones, perform spatiotemporal registration of the first visual data collected by the UAV at the construction site and the second visual data collected by the fixed monitoring equipment, and generate fused perception data based on scene geometric constraints and timestamp alignment.

[0016] The cross-view semantic consistency analysis module is configured to extract scene semantic feature vectors of the same work area from the perspective of a drone and the perspective of a fixed monitoring based on the fused perception data. The drone perspective and the fixed monitoring perspective together constitute two perspectives. The module calculates the cross-view semantic consistency metric between the scene semantic feature vectors under the two perspectives, constructs a reference baseline for the cross-view semantic consistency metric under normal working conditions of the work area, and detects abnormal decay of the cross-view semantic consistency metric relative to the reference baseline.

[0017] The multidimensional health assessment module is configured to generate safety risk assessment values, quality deviation assessment values, and schedule deviation assessment values ​​by mapping the spatiotemporal distribution pattern of the abnormal attenuation, and to generate a unified construction health assessment index by coupling weights to fuse the safety risk assessment values, the quality deviation assessment values, and the schedule deviation assessment values.

[0018] The digital twin closed-loop control module is configured to inject the construction health evaluation index into the digital twin model of the construction site to predict the deviation and obtain the predicted deviation, and to feed back the predicted deviation to adaptively adjust the inspection strategy of the UAV and the area of ​​interest of the fixed monitoring equipment. The inspection strategy includes the flight path of the UAV and the area of ​​interest is the monitoring range of the fixed monitoring equipment.

[0019] The beneficial effects of this invention are as follows:

[0020] First, by spatiotemporally registering the first-vision data collected by UAVs and the second-vision data collected by fixed monitoring equipment and generating fused perception data in a unified coordinate system, air-ground collaborative full-domain perception coverage of the construction site is achieved. The mechanism lies in the fact that the UAV perspective provides high-resolution global observation from a top-down or oblique view, while the fixed monitoring perspective provides continuous local observation from a level view. The two complement each other in terms of spatial coverage and temporal continuity, and the fused perception data possesses both globality and continuity. Compared to solutions that rely solely on a fixed camera's single perspective for safety hazard identification, this invention reduces blind spots in the construction site's observations, providing a complete data foundation for subsequent cross-perspective analysis.

[0021] Secondly, by extracting scene semantic feature vectors of the same work area from different perspectives and calculating cross-perspective semantic consistency metrics and detecting their abnormal attenuation, an innovative utilization of perspective differences, traditionally considered registration noise, is achieved, transforming them into state change detection signals. The mechanism is as follows: Under normal operating conditions, the semantic representation of the same construction area maintains a stable and consistent mapping relationship across different perspectives. When a physical state change occurs in the work area, such as safety hazards, quality defects, or schedule deviations, the resulting visual appearance changes disrupt this consistent mapping, leading to an abnormal attenuation of the semantic consistency metric. Therefore, attenuation itself is an indicator signal of state change. This mechanism allows safety, quality, and schedule control events to share the same detection front-end, generating a synergistic effect far exceeding the sum of the effects of three independent detection systems. Compared to the single-task mode where each system operates independently in existing technologies, this solution requires only one cross-perspective semantic analysis pipeline to simultaneously cover the three types of control needs, achieving true multi-dimensional coupled evaluation.

[0022] Third, by injecting construction health evaluation indicators into a digital twin model for deviation prediction and feeding back the predicted deviations to adjust the UAV inspection strategy and monitoring focus areas, a complete closed loop of perception-analysis-evaluation-control is achieved. The mechanism is as follows: when the health evaluation indicators of a certain work area continue to deteriorate, the deviation trend predicted by the digital twin model drives the UAV to increase the frequency of inspections of that work area and reduce its flight altitude to obtain higher resolution observation data. Simultaneously, it guides fixed monitoring equipment to adjust its focus and concentrate its monitoring on that work area. This feedback adjustment causes system resources to automatically concentrate on high-risk areas, forming a self-reinforcing control closed loop. Compared to solutions that only provide open-loop progress comparisons, the closed-loop feedback mechanism of this invention enables the system to proactively adapt to the dynamic changes in the construction environment, achieving continuous optimization of the perception strategy without manual intervention.

[0023] Fourth, the three-dimensional coupled evaluation of safety, quality, and schedule generates a significant nonlinear synergistic effect. The mechanism lies in the fact that, in actual construction, rushing to meet deadlines is often accompanied by increased safety risks and decreased quality standards; these three factors are inherently causally related. This solution, by simultaneously detecting and evaluating these three types of control events within a unified semantic consistency metric space, can automatically capture this causal chain and quantify it through an abnormal coupling index. This allows for early warnings even in the early stages of events, transforming the control model from reactive response to proactive prevention. This cross-dimensional coupled analysis capability is unavailable from any existing single-function system. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating the intelligent evaluation method for engineering safety and quality based on machine vision provided by the present invention.

[0025] Figure 2 This is a schematic diagram of the architecture of the intelligent evaluation system for engineering safety and quality based on machine vision provided by the present invention. Detailed Implementation

[0026] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are illustrated using a dam construction scenario in a water conservancy project as an example, but the scope of protection of the present invention is not limited thereto.

[0027] like Figure 1 As shown, the machine vision-based intelligent evaluation method for engineering safety and quality of the present invention includes the following steps: Step S1: Air-Ground Collaborative Perception Fusion. The core task of this step is to perform spatiotemporal registration of the first visual data collected by the UAV at the construction site and the second visual data collected by the fixed monitoring equipment, and generate fused perception data based on scene geometric constraints and timestamp alignment.

[0028] In this embodiment, multiple rotary-wing UAVs and multiple fixed monitoring cameras are deployed at the construction site. The UAVs fly periodically along a preset inspection path, collecting top-down or oblique-view image sequences at altitudes of 30 to 200 meters. The flight cycle is set to once every 2 to 8 hours, depending on the project scale, with each inspection lasting 15 to 45 minutes. The overlap rate of adjacent frames in the collected image sequences is no less than 60%. Each frame includes corresponding GPS coordinates and inertial measurement unit attitude data. The GPS positioning accuracy is better than 0.02 meters (with RTK differential correction), and the attitude data includes three degrees of freedom: pitch, roll, and yaw. The fixed monitoring cameras are installed at fixed locations on the construction site, such as tower cranes, fences, and temporary buildings, collecting continuous video streams with a resolution of no less than 1920 x 1080 pixels and a frame rate of no less than 25 frames per second. The video stream includes a timestamp and the camera number.

[0029] The specific process of spatiotemporal registration is as follows: First, the timestamps of UAV images and fixed monitoring video frames are aligned to unify the two types of data under a GPS time reference, with a timestamp alignment accuracy better than 0.1s. Second, using the pre-set ground control points at the construction site as geometric registration references, images from different sources are projected onto a unified geographic coordinate system through affine transformation. The number of ground control points is no less than four per work area, and these control points are made of high-reflectivity materials to facilitate simultaneous identification by both UAVs and fixed monitoring. For UAV images, initial estimates of camera extrinsic parameters are established using the GPS coordinates and IMU attitude data carried by the UAV, followed by refined calibration using the ground control points. For fixed monitoring video frames, projection transformation is performed directly using the camera extrinsic and intrinsic parameters calibrated during installation. The registered fused perception data is organized by work area, with each work area corresponding to a spatiotemporal data block containing all visual data and metadata for that work area at different times and from different perspectives.

[0030] In practical engineering applications, construction sites are typically divided into several work zones, each corresponding to a specific construction location or process. The division of work zones is determined based on the construction organization design. A typical hydraulic engineering dam construction scenario can be divided into a foundation pit excavation zone, a concrete pouring zone, a formwork construction zone, a steel reinforcement processing zone, and a material storage zone. The organizational structure of the fused sensing data is as follows: using the work zone number as the first index dimension, the timestamp as the second index dimension, and the viewpoint type as the third index dimension, forming a three-dimensional indexed data cube.

[0031] It is worth emphasizing that the spatiotemporal registration in this step is not ultimately aimed at eliminating viewpoint differences, but rather at providing a unified spatial reference framework for extracting semantic feature vectors of the same work area from different viewpoints in the subsequent step S2. After registration, although the images of the same work area from the UAV's top-down view and the fixed monitoring head-up view are aligned to the same geographic coordinate area, the visual appearance observed from the two viewpoints still retains a natural difference—this difference is the core signal source utilized in step S2 of this invention, rather than a residual error that needs to be further eliminated.

[0032] In addition, during data preprocessing, distortion correction and automatic exposure compensation are performed on the drone images to eliminate lens radial distortion and brightness differences caused by varying flight altitudes. Keyframe extraction and motion blur detection are performed on the fixed monitoring video stream to remove low-quality frames caused by camera shake or rapid target movement. The quality of the preprocessed data is quantitatively evaluated using an image sharpness score; frames with a sharpness score below a preset threshold are marked as invalid and excluded from subsequent analysis.

[0033] Step S2: Cross-view semantic consistency measurement and abnormal decay detection. The core task of this step is to extract scene semantic feature vectors of the same work area from the perspectives of UAVs and fixed monitoring based on fused perception data, calculate the cross-view semantic consistency measurement value, and detect its abnormal decay.

[0034] A multi-scale feature extraction network based on an attention mechanism is used to extract scene semantic feature vectors from the input visual data. This network uses a pre-trained deep convolutional neural network as its backbone to extract semantic features at multiple spatial resolution scales. Specifically, the feature maps output from stages 2, 3, 4, and 5 of the backbone network correspond to scales of 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the original input resolution, respectively. Features at each scale are fused from top to bottom using a feature pyramid structure. During the fusion process, a channel attention mechanism is introduced to adaptively weight feature channels at different scales, enabling the network to automatically emphasize the feature channels most discriminative to the current scene based on the content of the input image. The multi-scale fused feature map is compressed into a fixed-dimensional feature vector using global average pooling, serving as the scene semantic feature vector for the work area from that perspective. In this embodiment, the scene semantic feature vector is set to 512 dimensions.

[0035] For the first visual data from the drone's perspective, since it is an intermittently acquired image sequence, the frame with the highest clarity is selected from multiple frames of images of the work area during each inspection as a representative frame for feature extraction, resulting in a semantic feature vector from the drone's perspective. For the second visual data from a fixed monitoring perspective, since it is a continuous video stream, the video frame closest to the drone's inspection timestamp is used as a representative frame for feature extraction, resulting in a semantic feature vector from the fixed monitoring perspective.

[0036] After extracting the scene semantic feature vectors of the same work area from two perspectives, a cross-perspective semantic consistency metric is calculated. This metric is defined as the normalized correlation between the semantic feature vectors of the two perspectives in a high-dimensional feature space, and its calculation formula is as follows:

[0037] ,

[0038] in: For the first The work area is in the first The cross-perspective semantic consistency metric at each time point is a scalar, and its value range is [value range missing]. , dimensionless, is calculated by this formula, and represents the degree of consistency of semantic representation of the same work area under two perspectives. The closer the value is to 1, the higher the consistency. For the first The work area is in the first The semantic feature vectors of the UAV perspective at each time point are 512-dimensional column vectors, dimensionless, and are obtained by processing the UAV image using the aforementioned multi-scale feature extraction network. For the first The work area is in the first The semantic feature vectors of fixed monitoring perspectives at each time point are 512-dimensional column vectors, dimensionless, obtained by processing the monitoring video frames using the aforementioned multi-scale feature extraction network; superscript... This represents the vector transpose operation; This represents the L2 norm operation, i.e. ,in For vector dimensions, For vector number 1 Each component.

[0039] Under normal operating conditions, the semantic consistency metric values ​​of the same work area fluctuate around a stable baseline value from both perspectives. The reference baseline is constructed using a sliding window statistical method: for each work area... Take the nearest Consistency metric sequence at each sampling time point Calculate their mean as the reference baseline value. , where window size The sampling period is set to 10 to 30, with the following criteria: if the period is too small, the baseline will be unstable and prone to false alarms; if the period is too large, it will be insensitive to slowly changing anomalies.

[0040] The formula for calculating the attenuation gradient is:

[0041] ,

[0042] in: For the first The work area is in the first The consistent decay gradient at each time point is a scalar with a value range of . , dimensionless, is calculated by this formula. A positive value indicates that the current consistency is lower than the baseline, i.e., a decay has occurred, and a negative value indicates that the consistency is higher than the baseline. The reference baseline value is , which is a scalar and has a range of values. , dimensionless, is calculated from the mean of the sliding window, and represents the consistency level of the work area under normal working conditions.

[0043] When the gradient decays Exceeding the preset threshold At that time, it is determined that a candidate abnormal event has occurred in the work area at that specific time point. Threshold The value range is from 0.05 to 0.30, with 0.15 being preferred. A threshold that is too small can cause normal random fluctuations to be misjudged as anomalies, increasing the false alarm rate; a threshold that is too large can cause genuine anomalies to be missed, reducing detection sensitivity. In engineering practice, the threshold can be adjusted within the above range according to the environmental stability and management needs of the construction site.

[0044] Separating candidate abnormal events by event type is a key step in achieving a three-dimensional coupled evaluation of safety, quality, and schedule. In actual construction site conditions, different types of control events exhibit different time-frequency characteristics in the semantic feature space: safety events typically exhibit a high-frequency abrupt change pattern, such as workers not wearing safety helmets or illegally entering dangerous areas, which are instantaneous behavioral anomalies, and their decay signals show a sharp peak-like abrupt change on the time axis; quality events typically exhibit a low-frequency gradual change pattern, such as cracks appearing on the concrete pouring surface or accumulated deviations in rebar spacing, which are slowly evolving structural anomalies, and their decay signals show a smooth but continuous trend change on the time axis; schedule events typically exhibit a periodic fluctuation pattern, such as intermittent work stoppages and resumptions caused by process delays, and their decay signals show periodic fluctuations on the time axis related to construction shifts or process cycles.

[0045] Utilizing the aforementioned time-frequency characteristic differences, the decay gradient sequence Multi-channel decomposition is performed. The decomposition method uses a bandpass filter bank with adjustable quality factor to separate the attenuation gradient signal into high-frequency components. Low-frequency components and bandpass components These correspond to the safety decay signal, quality decay signal, and progress decay signal, respectively. The decomposition formula is:

[0046] ,

[0047] ,

[0048] ,

[0049] in: For the first The work area is in the first The safety attenuation signal at each time point is a scalar, dimensionless quantity, obtained by convolution operation, and represents the abnormal intensity of the safety dimension of the work area. This is a quality degradation signal, with a similar meaning; This is a progress decay signal, with a similar meaning; The high-pass filter for the security channel Each coefficient is a scalar, dimensionless quantity, determined by the filter design, and its cutoff frequency is set to 0.3 to 0.5 times the sampling frequency (i.e., allowing signal components that are 30% higher than the sampling frequency to pass through). The low-pass filter coefficients for the quality channel are set, with the cutoff frequency set to 0.05 to 0.15 times the sampling frequency; The passband filter coefficients for the progress channel are set to a range of 0.15 to 0.30 times the sampling frequency. is the filter order, which is a positive integer ranging from 8 to 32, with 16 being preferred. Higher order filters offer better frequency selectivity but also result in greater computational delay.

[0050] After completing the three-channel decomposition, the co-occurrence intensity of the three types of attenuated signals is calculated, and the abnormal coupling index is defined:

[0051] ,

[0052] in: For the first The work area is in the first The anomalous coupling index at each time point is a scalar with a value range of [value missing]. , dimensionless, is calculated by this formula, and characterizes the coupling strength of the three types of control events occurring simultaneously. The larger the value, the higher the systemic management risk. To prevent extremely small positive numbers from being divided by zero, the value is taken as... The formula is dimensionless. The design logic of this formula is that the product of the first three terms reaches its maximum value when all three types of signals are positive (i.e., all three types of anomalies occur simultaneously). The normalization factor at the end makes the coupling index the highest when the intensity of the three types of signals is evenly distributed, and the coupling index is low when the anomaly is concentrated in a single dimension.

[0053] In actual construction sites, the differences in construction procedures across different work areas lead to spatial non-stationarity of the prior parameters for the aforementioned frequency domain decomposition. Taking the construction of a hydraulic dam as an example, the frequency characteristics of safety events in the excavation area differ significantly from those in the concrete pouring area: safety events in the excavation area are mainly mechanical collisions and slope instability, with a time scale on the order of minutes; while safety events in the pouring area are mainly personnel slipping and formwork displacement, with a time scale on the order of seconds. If uniform frequency domain decomposition parameters are used for all work areas, a cross-work area misclassification problem will occur, where normal operating conditions in one type of work area are misjudged as abnormal signals in another type of work area.

[0054] To address this issue, context-conditional adaptive frequency domain prior calibration is introduced. Current process state information for each work area is obtained from the digital twin model and encoded into a process state vector. This serves as the conditional input to the frequency domain prior calibration model, dynamically generating frequency domain separation parameters specific to this work area. The calibration formula is:

[0055] ,

[0056] in: For the first The calibrated filter coefficient vector for each work area has a dimension of... (3 channels each) (Multiple coefficients spliced ​​together), dimensionless, calculated by this formula, replacing the original fixed coefficients for multi-channel decomposition of this work area; To calibrate the mapping matrix, the dimension is Dimensionless, learned from training data, where For intermediate layer dimensions, the value ranges from 32 to 128, with 64 being preferred; It is the sigmoid activation function, i.e. Map the input to Within the range; This is a process status encoding matrix with dimension 1. Dimensionless, learned from training data, where The dimension of the process state vector; For the first The process state vector of each work area has a dimension of . Dimensionless, obtained from the digital twin model, it encodes information such as the current process type, process progress percentage, and main equipment type of the work area; Let be the bias vector, with dimension . , dimensionless, learned from training data; The basic filter coefficient vector (i.e., the uncalibrated default parameters) has dimensions of... It is dimensionless and determined by classical filter design methods.

[0057] During the iterative operation of context-adaptive calibration, the calibration parameters of each work area... These parameters change dynamically as construction progresses. By monitoring the temporal evolution of these parameters, a significant statistical correlation was found between the parameter drift rate and the construction quality stability of the work area. The formula for calculating the parameter drift rate is:

[0058] ,

[0059] in: For the first The work area is in the first The parameter drift rate at each time point is a scalar, and its value range is... , dimensionless, is calculated by this formula, and characterizes the average drastic change of the frequency domain separation parameter of the work area within the time window. The larger the value, the more unstable the construction state of the work area is, and indirectly indicates the greater the fluctuation of construction quality. The time window size for calculating the drift rate is a positive integer, ranging from 5 to 20 sampling periods, preferably 10. If the window is too small, the drift rate will fluctuate greatly; if the window is too large, it will be insensitive to short-term quality fluctuations. For the first The work area is in the first The calibration parameter vector at each time point is defined as before; It is the square of the L2 norm.

[0060] The parameter drift rate, as an indirect measure of construction quality stability, has been incorporated into the subsequent unified health evaluation index system. Its physical interpretation is as follows: when the construction process of a work area is stable and the quality is controllable, the visual scene of that work area exhibits regular changes, the optimal parameters of the frequency domain decomposition remain relatively stable, and the parameter drift rate is low. Conversely, when the construction process fluctuates and the quality is uncontrollable, the visual scene of that work area exhibits irregular changes, the optimal parameters of the frequency domain decomposition are constantly adjusted to adapt to the new data distribution, and the parameter drift rate increases.

[0061] Step S3: Generation of Unified Construction Health Evaluation Indicators. The core task of this step is to generate safety risk evaluation values, quality deviation evaluation values, and schedule deviation evaluation values ​​based on the spatiotemporal distribution pattern of abnormal attenuation, and then fuse them into a unified construction health evaluation index through coupling weights.

[0062] Safety risk assessment value Based on safety attenuation signal A mapping process is performed. This process comprehensively considers three dimensions of the safety attenuation signal: amplitude, duration, and spatial diffusion range. The first dimension is the instantaneous amplitude of the safety attenuation signal, reflecting the severity of the safety anomaly at the current moment. The second dimension is the duration for which the safety attenuation signal exceeds the safety threshold within a continuous sampling period, reflecting the persistence of the safety anomaly; a longer duration indicates that the safety hazard is not an occasional disturbance but a persistent risk source. The third dimension is the spatial diffusion range of the safety attenuation signal, i.e., the number of adjacent work areas simultaneously exhibiting safety attenuation signals at the same time point, reflecting the spatial propagation characteristics of the safety risk.

[0063] Quality Deviation Evaluation Value Based on mass decay signal and parameter drift rate A joint mapping is performed. The quality decay signal reflects the currently observable quality deviation, while the parameter drift rate reflects the quality trend change that is not yet explicit but has already manifested at the parameter level. The joint mapping of the two enables quality assessment to have the dual capabilities of assessing the current state and predicting future trends. In the joint mapping, the contribution weight of the parameter drift rate gradually increases over time, reflecting the concept of quality management shifting from post-event detection to pre-event prevention.

[0064] Schedule Deviation Evaluation Value Based on progress decay signal The schedule is compared and mapped with the planned schedule nodes in the digital twin model. The periodic fluctuation pattern of the schedule decay signal is compared with the work process handover cycle in the construction plan. When the actual fluctuation cycle deviates from the planned cycle by more than the preset tolerance, it is judged as a schedule deviation. The schedule deviation evaluation value considers both the deviation direction and the deviation magnitude. Deviations that lag behind the planned schedule are assigned a positive value to indicate increased risk, while deviations that are ahead of the planned schedule are assigned a lower value to indicate schedule margin after excluding the simultaneous increase in safety risks.

[0065] The value ranges of all three evaluation values ​​are normalized to [value range]. , where 0 indicates that the dimension is completely normal, and 1 indicates that the dimension is extremely high risk.

[0066] A unified construction health evaluation index is obtained by coupling weights and fusing three types of evaluation values:

[0067] ,

[0068] in: For the first The work area is in the first A unified construction health evaluation index at each time point is a scalar quantity with a value range of [value range missing]. , dimensionless, is calculated by this formula, and the larger the value, the higher the comprehensive management and control risk of the work area; Here, is the weighting coefficient for the safety risk assessment value, and is a scalar with a value range of . Dimensionless, with a preferred value of 0.40. A higher weight indicates a higher proportion of the safety dimension in the overall evaluation. The weighting coefficient for the quality deviation evaluation value is preferably 0.35; The weighting coefficient for the schedule deviation evaluation value is preferably 0.25; the sum of the three weighting coefficients is 1.0. The weighting coefficients of the abnormal coupling index are denoted as , which are scalars and have a range of values ​​of . , dimensionless, preferably 0.5, this coefficient controls the contribution of multidimensional coupling effect to the comprehensive evaluation; This is the abnormal coupling index, defined as before.

[0069] Construction health evaluation indicators are divided into four levels based on their numerical range: Normal level To focus on the level, At the warning level, Alarm levels are indicated by green, yellow, orange, and red colors in the 3D scene of the digital twin model, representing the alarm levels for the corresponding work areas. When an alarm or alert level is triggered, the system automatically generates an alarm report, which includes the event type, event location, occurrence time, anomaly coupling index, and suggested handling measures.

[0070] Step S4: Digital Twin Closed-Loop Control. The core task of this step is to inject construction health evaluation indicators into the digital twin model of the construction site for deviation prediction, and to use the predicted deviations as feedback to adaptively adjust the drone inspection strategy and the area of ​​interest of fixed monitoring equipment.

[0071] A digital twin model is a virtual mapping of a construction site, comprising a 3D terrain model, a building information model, a construction equipment configuration model, and a construction schedule timeline model. The 3D terrain model, based on a digital elevation model (DEM), overlays orthophotos and oblique photogrammetry to accurately represent the site's topography and completed structures. The building information model, built upon design drawings, includes the geometric dimensions, material properties, and construction process parameters of structural components. The construction equipment configuration model records the current location, operating status, and maintenance plans for various types of machinery. The construction schedule timeline model, based on the construction organization design and construction progress plan, uses Gantt charts and the critical path method to express the logical relationships and time constraints between different work processes. After construction health evaluation indicators are injected into the digital twin model, the model employs a timeline prediction method based on long short-term memory networks to predict health evaluation indicators for each work area at several future time steps. The prediction time span is set to 1 to 7 days based on control requirements. The prediction results are compared with preset control thresholds to obtain the prediction deviation.

[0072] The adaptive adjustment process of feeding back prediction bias to the perception strategy is achieved through the following formula:

[0073] ,

[0074] in: For the first The adjusted drone inspection frequency for each work area is a scalar quantity, measured in times per hour (times / h), with a range of values ​​of [value range missing]. The value calculated by this formula represents the inspection density of the work area by the UAV. The basic inspection frequency is a scalar quantity, with units of times / hour, ranging from 0.125 to 0.5 times / hour (i.e., one inspection every 2 to 8 hours), and is determined by the construction organization design. Here, is the feedback gain coefficient, and is a scalar with a value range of . Dimensionless, preferably 1.5, controls the sensitivity of feedback adjustment. If the gain is too large, the frequency adjustment will be too drastic, resulting in insufficient drone battery life. If the gain is too small, the adjustment will not be timely, resulting in insufficient inspection of high-risk areas. The first prediction for the digital twin model In the future, each work area The health evaluation indicators after time are scalar, dimensionless, and output by the time series prediction model. The control threshold is a scalar, dimensionless quantity, with a value of 0.50 (corresponding to the lower limit of the warning level). The minimum inspection frequency is set at 0.0625 times / hour (meaning at least one inspection every 16 hours). The highest inspection frequency is set to 2.0 times / hour (i.e., one inspection every 30 minutes).

[0075] Simultaneously, the focus areas of fixed monitoring equipment are adaptively adjusted. When the predicted health assessment indicators of a certain work area exceed the control threshold, the fixed monitoring cameras associated with that work area automatically adjust their pan-tilt angles and focal lengths to focus on that work area. The adjustment strategy is as follows: the top three work areas with the largest prediction deviations are set as priority focus areas and assigned to the nearest adjustable cameras for directional monitoring.

[0076] The closed-loop feedback mechanism enables the system to have self-reinforcing capabilities: when the risk of a certain work area increases, the increased inspection frequency and the concentrated monitoring focus lead to an increase in the data collection density of that work area, thereby improving the temporal resolution and detection sensitivity of cross-perspective semantic consistency analysis, and thus generating more accurate health evaluation indicators, forming a positive feedback closed loop of perception-analysis-evaluation-control.

[0077] To further enhance the interpretability of the evaluation results, this embodiment also introduces a causal graph structure learning method based on construction physical constraints. Parameter drift rate, anomaly coupling index, and environmental sensor data (including temperature, humidity, wind speed, rainfall, etc.) are used as nodes of the causal graph, and the physical constraints of the construction process are used to limit the search space of the causal graph edges.

[0078] The physical constraints include: a positive causal relationship between construction safety events and personnel density and machinery operating status; a conditional causal relationship between quality defects and sudden temperature changes and abnormal humidity; a direct causal relationship between schedule deviations and material supply interruptions; and a bidirectional causal relationship between increased safety events and expedited schedules. Under these physical constraints, a conditional independence test method is used to learn the edges of the causal graph and identify the causal relationship structure between each node.

[0079] The learned causal graph enables the system to upgrade from statistical correlation to causal attribution: when an alarm is triggered in a certain work area, the system not only reports that the health evaluation indicators of that work area are abnormal, but also uses the causal graph to backtrack and analyze the root cause of the abnormality, providing targeted handling suggestions for managers.

[0080] like Figure 2As shown, this embodiment provides a machine vision-based intelligent evaluation system for engineering safety and quality. Its hardware architecture includes a processor, memory, communication interfaces, and a display device. The processor uses a GPU-accelerated computing platform, with a graphics processing unit configured with at least 8GB of video memory for accelerating inference of deep learning models. The memory includes both running memory and data storage. At least 32GB of running memory is used for real-time data processing, and the data storage adopts a distributed storage architecture. The total capacity is determined based on the construction cycle and data acquisition density; a typical water conservancy dam construction project requires at least 10TB of storage space. The communication interfaces include a wireless communication interface with a UAV ground station and a wired network interface with a fixed monitoring system. The display device uses a large-screen LCD for visualizing the digital twin 3D scene.

[0081] The system's software architecture includes the following four functional modules:

[0082] The air-to-ground collaborative perception fusion module receives first visual data relayed from the UAV ground station and second visual data from the fixed monitoring system, and executes the spatiotemporal registration and fused perception data generation functions described in step S1. This module internally includes a timestamp synchronization submodule, a geometric registration submodule, and a data indexing submodule. The timestamp synchronization submodule is responsible for unifying data from different sources to a GPS time reference. The geometric registration submodule uses ground control points to achieve coordinate system unification. The data indexing submodule organizes the fused perception data using work area number, timestamp, and viewpoint type as a three-dimensional index. The output data of this module is transmitted to the cross-viewpoint semantic consistency analysis module via an internal data bus.

[0083] The cross-view semantic consistency analysis module performs the functions described in step S2, including scene semantic feature vector extraction, cross-view semantic consistency metric calculation, abnormal attenuation detection, multi-channel attenuation mode decomposition, and context-adaptive frequency domain prior calibration. Internally, this module deploys an inference engine based on an attention-based multi-scale feature extraction network, supporting real-time feature extraction from UAV images and surveillance video frames. The module also includes a filter parameter memory, storing calibration parameters for each work area and updating them in each analysis cycle. The module's output includes three-channel attenuation signals, abnormal coupling index, and parameter drift rate, which are transmitted to the multi-dimensional health evaluation module via a data bus.

[0084] The multi-dimensional health assessment module performs the three-dimensional evaluation value mapping generation and coupled weight fusion functions described in step S3. This module receives three-channel attenuation signals and parameter drift rates as input, calculates safety risk assessment values, quality deviation assessment values, and schedule deviation assessment values ​​respectively, and generates a unified construction health assessment index through coupled weight fusion. The module also includes a warning level classification submodule and a warning report generation submodule. The warning level classification submodule classifies each work area into four levels—normal, attention, warning, and alarm—based on the numerical range of the health assessment index. The warning report generation submodule automatically generates a structured report containing the event type, location, time, and suggested handling measures when a warning or alarm level is triggered. The output of this module is transmitted to the digital twin closed-loop control module and display device.

[0085] The digital twin closed-loop control module executes the deviation prediction and adaptive adjustment functions described in step S4. This module maintains digital twin model instances of the construction site, including a 3D terrain model, a building information model, and a construction schedule time series model. A time series prediction engine based on a long short-term memory network is deployed within the module to perform rolling predictions of health evaluation indicators for each work area at several future time steps. The prediction results are transmitted to the gimbal controllers of the UAV ground station and the fixed monitoring system via a feedback control interface, enabling adjustments to the inspection frequency and the area of ​​interest. Simultaneously, this module includes a causal graph structure learning submodule, which uses construction physical constraints to identify causal relationships, providing interpretable root cause analysis for abnormal alarms.

[0086] The four modules described above are connected sequentially according to the data flow direction from steps S1 to S4, forming a serial processing pipeline. Simultaneously, the feedback output of the digital twin closed-loop control module is connected back to the configuration interface of the air-ground collaborative sensing fusion module, forming a complete closed-loop control circuit.

[0087] To verify the technical effectiveness of this invention, a comparative experiment was conducted for three months at the construction site of a dam in a water conservancy project. The experimental setup was as follows:

[0088] The experimental group used the method and system provided by this invention, deploying two rotary-wing UAVs and 12 fixed monitoring cameras to cover all eight work areas of the dam construction. Control group A used a traditional video monitoring scheme with only fixed monitoring cameras, control group B used a scheme with only UAV inspections, and control group C used a scheme in which UAVs and fixed monitoring cameras operated independently without cross-view fusion analysis.

[0089] The experimental results are as follows:

[0090] Regarding the detection rate of safety hazards, the detection rate of safety hazards in the experimental group was 94.3%, while that in control group A was 71.2%, control group B was 65.8%, and control group C was 82.5%. The experimental group improved by 23.1 percentage points compared to control group A and by 11.8 percentage points compared to control group C (simple parallel non-fusion). The latter improvement came from the synergistic effect of cross-perspective semantic consistency analysis.

[0091] Regarding the lead time for quality defect warnings, the experimental group was able to provide an average warning signal 2.3 days before quality defects became visible, while control groups A and B had no such capability, and control group C's warning lead time was only 0.5 days. The 2.3-day lead time in the experimental group comes from the indirect measurement mechanism of parameter drift rate on quality stability.

[0092] Regarding false alarm rates, the experimental group had a false alarm rate of 8.7%, control group A had a rate of 15.3%, and control group C had a rate of 22.1%. The false alarm rate of control group C was actually higher than that of control group A because it did not introduce context-adaptive frequency domain prior calibration, leading to significant cross-regional misclassification due to signal aliasing between different work zones. The experimental group effectively suppressed such false alarms through context calibration.

[0093] In terms of overall system efficiency, the experimental group saved approximately 1200 man-hours of manual inspection during the 3-month experimental period, and the overall control response time was reduced from an average of 4.2 hours in the traditional scheme to 0.35 hours. The closed-loop feedback mechanism automatically adjusted the drone inspection strategy 47 times during the experiment. In 38 of these adjustments, the target work areas indeed found control events requiring handling in the next inspection cycle, achieving an effectiveness rate of 80.9%. Furthermore, during continuous operation, the system identified 12 cross-dimensional causal relationship chains through causal graph structure learning. The most typical chain is as follows: continuous high temperatures led to deterioration of concrete curing conditions, causing an increase in quality degradation signals; the construction team increased nighttime work intensity to catch up with the schedule delays caused by high temperatures; insufficient nighttime lighting further increased safety degradation signals; and the coupling of these three types of signals ultimately triggered a systemic alarm. This case fully verifies the practical value of the three-dimensional coupling evaluation mechanism of this invention.

[0094] The experimental results above demonstrate that the method and system provided by this invention significantly outperform existing technologies in terms of safety hazard detection rate, early warning time for quality defects, false alarm rate, and overall efficiency. Furthermore, the synergistic effect of cross-perspective semantic consistency analysis and the dealiasing effect of context-adaptive calibration have been quantitatively verified. This invention achieves unified perception and coupled evaluation of safety, quality, and schedule, providing an effective technical solution for intelligent management of engineering construction.

[0095] The embodiments of the present invention are not limited to the specific embodiments described above. Those skilled in the art can make various equivalent changes or substitutions based on the technical solutions of the present invention, and all such changes or substitutions should be included within the protection scope of the present invention.

Claims

1. A method for intelligent evaluation of engineering safety and quality based on machine vision, characterized in that, Includes the following steps: The construction site is divided into multiple work areas. Spatiotemporal registration is performed on the first visual data collected by drones and the second visual data collected by fixed monitoring equipment at the construction site. Fusion perception data is generated based on scene geometric constraints and timestamp alignment. Based on the fused perception data, scene semantic feature vectors of the same work area are extracted from the perspective of the UAV and the perspective of the fixed monitoring. The UAV perspective and the fixed monitoring perspective together constitute two perspectives. The cross-perspective semantic consistency metric value between the scene semantic feature vectors under the two perspectives is calculated. A reference baseline for the cross-perspective semantic consistency metric value of the work area under normal working conditions is constructed, and abnormal decay of the cross-perspective semantic consistency metric value relative to the reference baseline is detected. Based on the spatiotemporal distribution pattern of the abnormal attenuation, safety risk evaluation value, quality deviation evaluation value, and schedule deviation evaluation value are generated respectively. The safety risk evaluation value, quality deviation evaluation value, and schedule deviation evaluation value are then fused through coupling weights to generate a unified construction health evaluation index. The construction health evaluation indicators are injected into the digital twin model of the construction site to predict deviations and obtain predicted deviations. The predicted deviations are then fed back to adaptively adjust the inspection strategy of the UAV and the area of ​​interest of the fixed monitoring equipment. The inspection strategy includes the flight path of the UAV, and the area of ​​interest is the monitoring range of the fixed monitoring equipment.

2. The method according to claim 1, characterized in that, The step of detecting abnormal decay of the cross-perspective semantic consistency metric value relative to the reference baseline includes: calculating the decay gradient of the cross-perspective semantic consistency metric value relative to the reference baseline, and determining it as a candidate abnormal event when the decay gradient exceeds a preset threshold.

3. The method according to claim 2, characterized in that, The method further includes separating the candidate abnormal events by event type, utilizing the differences that safety events exhibit high-frequency abrupt change patterns, quality events exhibit low-frequency gradual change patterns, and progress events exhibit periodic fluctuation patterns in the semantic feature space, performing multi-channel pattern decomposition on the attenuation gradient, and outputting safety attenuation signals, quality attenuation signals, and progress attenuation signals respectively; and calculating the anomaly coupling index based on the simultaneous co-occurrence strength of the three types of attenuation signals, the anomaly coupling index being used to indicate the level of systemic management risk.

4. The method according to claim 3, characterized in that, In the multi-channel mode decomposition, the frequency domain separation parameters of each channel are adaptively calibrated according to the context of the work area, including: obtaining the current process status information of each work area from the digital twin model, inputting the process status information as a context condition into the frequency domain prior calibration model, and generating corresponding frequency domain separation parameters for each work area.

5. The method according to claim 4, characterized in that, The method further includes using the time evolution trajectory of the frequency domain separation parameters of each work area as an indirect measure of construction quality stability, including: calculating the parameter drift rate of the frequency domain separation parameters of each work area within a preset time window, modeling the correlation between the parameter drift rate and the construction quality level of the work area, and incorporating the parameter drift rate into the construction health evaluation index system to achieve trend prediction of construction quality.

6. The method according to claim 5, characterized in that, The association modeling adopts a causal graph structure learning method based on construction physical constraints. The parameter drift rate, the abnormal coupling index, and environmental sensor data are used as nodes of the causal graph. The physical constraints of the construction process are used to limit the search space of the edges of the causal graph, and the causal relationship structure between the nodes is learned.

7. The method according to claim 1, characterized in that, The acquisition of the first visual data collected by the UAV includes: periodically flying according to a preset inspection path, collecting a sequence of top-down or oblique view images at an altitude of 30 to 200m, wherein the overlap rate of adjacent frames in the image sequence is not less than 60%, and each frame image is accompanied by corresponding GPS coordinates and inertial measurement unit attitude data.

8. The method according to claim 1, characterized in that, The extraction of the scene semantic feature vector adopts a multi-scale feature extraction network based on the attention mechanism. Semantic features are extracted from the input visual data at multiple spatial resolution scales, and the features at different scales are adaptively weighted and fused through the channel attention mechanism to generate a scene semantic feature vector of a unified dimension.

9. The method according to claim 1, characterized in that, The method also includes classifying and issuing early warnings for the construction health evaluation indicators. The indicators are divided into four levels: normal, attention, early warning, and alarm, based on their numerical range. The early warning level of the corresponding work area is marked with different colors in the three-dimensional scene of the digital twin model. When an early warning or alarm level is triggered, an early warning report containing the event type, location, time, and suggested handling measures is automatically generated.

10. A machine vision-based intelligent evaluation system for engineering safety and quality, used to implement the method described in any one of claims 1-9, characterized in that, It includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program to perform the functions of the following modules: The air-ground collaborative perception fusion module is configured to divide the construction site into multiple work zones, perform spatiotemporal registration of the first visual data collected by the UAV at the construction site and the second visual data collected by the fixed monitoring equipment, and generate fused perception data based on scene geometric constraints and timestamp alignment. The cross-view semantic consistency analysis module is configured to extract scene semantic feature vectors of the same work area from the perspective of a drone and the perspective of a fixed monitoring based on the fused perception data. The drone perspective and the fixed monitoring perspective together constitute two perspectives. The module calculates the cross-view semantic consistency metric between the scene semantic feature vectors under the two perspectives, constructs a reference baseline for the cross-view semantic consistency metric under normal working conditions of the work area, and detects abnormal decay of the cross-view semantic consistency metric relative to the reference baseline. The multidimensional health assessment module is configured to generate safety risk assessment values, quality deviation assessment values, and schedule deviation assessment values ​​by mapping the spatiotemporal distribution pattern of the abnormal attenuation, and to generate a unified construction health assessment index by coupling weights to fuse the safety risk assessment values, the quality deviation assessment values, and the schedule deviation assessment values. The digital twin closed-loop control module is configured to inject the construction health evaluation index into the digital twin model of the construction site to predict the deviation and obtain the predicted deviation, and to feed back the predicted deviation to adaptively adjust the inspection strategy of the UAV and the area of ​​interest of the fixed monitoring equipment. The inspection strategy includes the flight path of the UAV and the area of ​​interest is the monitoring range of the fixed monitoring equipment.

Images