A smart construction site construction safety early warning method

Abstract

The present application belongs to the technical field of construction safety, and particularly relates to a kind of wisdom construction site construction safety early warning method.This method first carries out spatiotemporal joint self-adaptive denoising to monitoring video image, then extracts frequency domain multiscale features through two-dimensional discrete wavelet transform, combines with optical flow method to extract spatial deformation field features, and fuses to obtain multi-modal feature vector.Normal construction stage samples are collected, and a normal operation feature memory bank is built using density clustering.The real-time image features are compared with the memory bank to calculate the abnormality score, the early warning threshold is updated adaptively based on Bayesian inference, and the initial early warning result is continuously verified in the time dimension, and the early warning signal is output after meeting the standard.The present application does not require a large number of accident labeled samples, fuses static and dynamic features, has strong environmental interference resistance, can effectively reduce false positives and false negatives, and is suitable for complex and variable construction sites.

Description

Technical Field

[0001] This invention belongs to the field of construction safety technology, and in particular relates to a smart construction site construction safety early warning method. Background Technology

[0002] Construction sites are typical high-risk work environments, posing various safety hazards such as falls from heights, falling objects, mechanical injuries, and collapses. Traditional construction safety monitoring mainly relies on manual inspections, which suffers from limited monitoring range, poor real-time performance, high labor costs, and susceptibility to human factors. With the development of computer vision technology, intelligent safety early warning systems based on video surveillance are gradually being applied.

[0003] In existing technologies, most video-based construction safety early warning methods employ supervised learning, requiring the pre-labeling of a large amount of sample data containing various types of safety accidents. However, construction safety accidents are low-probability events, making it difficult to obtain a sufficient number of labeled samples, resulting in insufficient generalization ability of supervised learning models. Furthermore, construction site environments are complex and variable; factors such as lighting conditions, construction procedures, and equipment locations can change at any time. Fixed supervised learning models struggle to adapt to dynamically changing scenarios, easily generating numerous false alarms and missed alarms. Additionally, existing technologies typically focus only on the static visual features of images, ignoring motion information and spatiotemporal correlation features in video sequences, failing to effectively identify slowly developing safety hazards and abnormal behavior patterns. For example, slow deformation of scaffolding, abnormal vibration of tower cranes, and violations of regulations by construction workers may not be obvious in a single frame, but they have significant characteristics over time. Therefore, the industry urgently needs a smart construction site safety early warning method that does not require a large number of labeled samples, can adapt to complex and dynamic construction environments, and integrates both static visual features and dynamic motion features to improve the accuracy and real-time performance of early warnings and reduce the incidence of safety accidents. Summary of the Invention

[0004] In view of the technical problems existing in the above-mentioned background art, the present invention proposes a smart construction site construction safety early warning method.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] S1. Acquire monitoring video of the construction site, perform two-dimensional discrete wavelet transform on each frame of the monitoring video to obtain the low-frequency subband coefficient matrix and high-frequency subband coefficient matrix of each frame in multiple frequency bands, and construct a frequency domain multi-scale feature description of each frame.

[0007] S2. Calculate the optical flow field between adjacent frames in the surveillance video based on the optical flow method, obtain the motion vector of each pixel in each frame, and extract the spatial deformation field features composed of the motion vector gradient.

[0008] S3. The frequency domain multi-scale feature description is fused with the spatial deformation field feature to obtain the multimodal feature vector of each frame image;

[0009] S4. During the preset collection period of the normal operation phase of building construction, the multimodal feature vectors are collected to form a normal sample set. The normal sample set is clustered using a density clustering algorithm to construct a feature memory library for normal operation. The feature memory library contains multiple feature cluster centers and their corresponding feature neighborhood radii and sample densities.

[0010] S5. Calculate the current multimodal feature vector for each frame of image acquired in real time, compare the current multimodal feature vector with the feature memory, and obtain the anomaly score of the current frame.

[0011] S6. Compare the anomaly score with the early warning judgment threshold, and record the images that exceed the early warning judgment threshold as the initial early warning images;

[0012] S7. Perform time-dimensional persistence verification on the initial warning image. When the duration of the initial warning image meets the warning conditions, output a safety warning signal.

[0013] Preferably, before performing two-dimensional discrete wavelet transform on each frame of the image in step S1, the image also needs to undergo spatiotemporal joint adaptive denoising processing, specifically including:

[0014] Centered on the current frame image, acquire the N adjacent frame images before and after it to form a spatiotemporal image window containing 2N+1 frames;

[0015] The Sobel operator is used to calculate the horizontal spatial gradient of the current frame image. and vertical spatial gradient Calculate the temporal gradient between adjacent frames within the spatiotemporal image window. ;

[0016] Based on the spatial gradient , and time gradient A local weighted average is constructed for each pixel using a Gaussian kernel function. Spacetime structure tensor;

[0017] The spatiotemporal structure tensor is decomposed into eigenvalues ​​to obtain three eigenvalues ​​arranged in descending order. The temporal consistency metric of each pixel is calculated based on the eigenvalues.

[0018] Based on the temporal consistency metric and the spatial components of the spatiotemporal structure tensor, adaptive anisotropic diffusion filtering is applied to the current frame image to obtain a denoised image.

[0019] Preferably, the method of performing adaptive anisotropic diffusion filtering on the current frame image based on the temporal consistency metric and the spatial components of the spatiotemporal structure tensor to obtain the denoised image is as follows:

[0020] Extract the top left corner of the spatiotemporal structure tensor Spatial subtensor; Eigenvalue decomposition is performed on the spatial subtensor to obtain two spatial eigenvalues ​​arranged in descending order and their corresponding orthogonal eigenvectors;

[0021] An anisotropic diffusion tensor is constructed based on orthogonal eigenvectors. The current frame image is then subjected to anisotropic diffusion filtering for M iterations using the diffusion tensor. After M iterations, the denoised image is obtained.

[0022] Preferably, in step S1, a two-dimensional discrete wavelet transform is performed on each frame of the surveillance video to obtain the low-frequency subband coefficient matrix and the high-frequency subband coefficient matrix of each frame in multiple frequency bands, thereby constructing a frequency domain multi-scale feature description of each frame, specifically including:

[0023] For each frame of the denoised image, an L-level two-dimensional discrete wavelet transform is performed using a wavelet basis to obtain the L-level two-dimensional discrete wavelet transform of each frame. Low-frequency subband coefficient matrix of level and high-frequency subband coefficient matrix , , ,in , It is a high-frequency component in the horizontal direction. These are high-frequency components in the vertical direction. These are high-frequency components in the diagonal direction;

[0024] For each sub-band coefficient matrix, sub-band energy features, sub-band coefficient statistical features, and sub-band texture features are calculated to constitute the frequency domain multi-scale feature description. The sub-band energy feature is the square mean of all coefficients in each sub-band coefficient matrix. The sub-band coefficient statistical features include the mean, variance, skewness, and kurtosis of each sub-band coefficient matrix. The sub-band texture features include the contrast, correlation, and entropy of the gray-level co-occurrence matrix of each sub-band coefficient matrix.

[0025] Preferably, step S2 involves calculating the optical flow field between adjacent frames in the surveillance video based on the optical flow method, obtaining the motion vector of each pixel in each frame, and extracting the spatial deformation field features composed of the motion vector gradients. Specifically, this includes:

[0026] The optical flow field is constructed by calculating the two-dimensional motion vector of each pixel between adjacent frames using the optical flow method.

[0027] The horizontal and vertical gradients of the optical flow field are calculated to obtain the motion vector gradient field.

[0028] The divergence and curl characteristics, as well as the two principal strain values ​​of the strain tensor, are calculated based on the gradient field of the motion vector to form the spatial deformation field characteristics.

[0029] Preferably, step S3 involves fusing the frequency domain multi-scale feature description with the spatial deformation field features to obtain a multimodal feature vector for each frame of the image, specifically including:

[0030] The sub-band features of each level and direction in the frequency domain multi-scale feature description are concatenated to obtain the initial frequency domain feature vector.

[0031] The initial frequency domain feature vector is reduced in dimension by using principal component analysis to obtain a frequency domain feature vector with dimension K.

[0032] The divergence, curl, and two principal strain values ​​in the spatial deformation field characteristics are divided into P×Q non-overlapping sub-blocks in space. The mean and standard deviation of the characteristic values ​​in each sub-block are calculated to obtain a spatial deformation statistical feature vector with a dimension of 4×P×Q.

[0033] The frequency domain dimensionality-reduced feature vector is concatenated with the spatial deformation statistical feature vector to obtain the multimodal feature vector.

[0034] Preferably, the specific implementation of constructing the normal operation state feature memory in step S4 includes:

[0035] During the normal operation phase of building construction, the multimodal feature vector of one frame of image is collected at every preset sampling interval, and a normal sample set containing T samples is collected within the preset collection period.

[0036] Calculate the cluster center, feature neighborhood radius, and sample density for each core feature cluster. The feature neighborhood radius is the 95th percentile of the distance from all samples in the cluster to the cluster center, and the sample density is the ratio of the number of samples in the cluster to the volume of the feature neighborhood.

[0037] The cluster centers, feature neighborhood radii, and sample densities of the core feature clusters are stored as the normal operating state feature memory.

[0038] Preferably, the specific implementation of comparing the current multimodal feature vector with the feature memory in step S5 to obtain the anomaly score of the current frame is as follows:

[0039] Calculate the Mahalanobis distance between the multimodal feature vector of the current frame and the i-th cluster center in the feature memory;

[0040] Take the minimum value among all Mahalanobis distances, divide the minimum value by the feature neighborhood radius of the corresponding cluster, and normalize to obtain the global anomaly score of the current frame.

[0041] Preferably, the warning determination threshold in step S6 is adaptively updated through Bayesian inference, specifically including:

[0042] A probability density distribution model is established for the anomaly score sequence of historical W frames, and the prior probability distribution of the anomaly score is obtained by using the Gaussian kernel density estimation method.

[0043] After obtaining a new anomaly score, the prior distribution is updated to a posterior distribution based on Bayes' theorem.

[0044] The initial threshold is obtained by segmenting the histogram of anomaly scores using the maximum inter-class variance method.

[0045] The initial threshold is calibrated using the posterior distribution to obtain an adaptive early warning judgment threshold.

[0046] Compared with existing technologies, the advantages and positive effects of this invention are as follows: by integrating wavelet frequency domain multi-scale static features with optical flow spatial deformation dynamic features, and taking into account both image structure and motion deformation information, it can effectively identify various hidden safety hazards. Relying on density clustering to construct a normal feature memory library eliminates the need for a large number of accident-labeled samples, solving the problems of scarce construction safety accident samples and weak generalization ability of supervised learning. Pre-processing with spatiotemporal joint adaptive denoising can resist interference from complex noise and light fluctuations at construction sites. Simultaneously, employing Bayesian adaptive thresholding combined with temporal persistence verification adapts to dynamically changing construction scenarios, significantly reducing the probability of false alarms and missed alarms, and significantly improving the accuracy, real-time performance, and environmental adaptability of construction safety early warning. Attached Figure Description

[0047] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart illustrating a smart construction site safety early warning method. Detailed Implementation

[0049] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described below in conjunction with the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0050] Numerous specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways than those described herein, and therefore the invention is not limited to the specific embodiments disclosed in the following specification.

[0051] In this embodiment, considering that most existing video-based construction safety early warning methods employ supervised learning, a large amount of pre-labeled sample data containing various types of safety accidents is required. However, construction safety accidents are low-probability events, making it difficult to obtain a sufficient number of labeled samples, resulting in insufficient generalization ability of supervised learning models. Furthermore, construction site environments are complex and variable; factors such as lighting conditions, construction procedures, and equipment locations can change at any time. Fixed supervised learning models struggle to adapt to dynamically changing scenarios, easily generating numerous false alarms and missed alarms. Therefore, this invention proposes a smart construction site safety early warning method.

[0052] The intelligent construction site safety early warning method provided by this invention is deployed in edge computing nodes or cloud servers. The system hardware architecture includes: high-definition network cameras deployed at various key locations on the construction site, an edge computing gateway, a cloud server, and an audible and visual alarm device. The high-definition network cameras are used to collect real-time monitoring video of the construction site; the edge computing gateway is used to preprocess the video data and perform preliminary feature extraction, reducing network transmission bandwidth; the cloud server is used to run the core anomaly detection algorithm, build and update the feature memory database, and issue early warning commands based on the anomaly detection results; the audible and visual alarm device is used to issue audible and visual alarms when safety hazards are detected, alerting on-site construction personnel and management personnel. Specific implementation methods are as follows: Figure 1 As shown.

[0053] First, surveillance video of the construction site is acquired. A two-dimensional discrete wavelet transform is performed on each frame of the video to obtain the low-frequency sub-band coefficient matrix and high-frequency sub-band coefficient matrix for each frame across multiple frequency bands, constructing a frequency domain multi-scale feature description for each frame. Considering that direct feature extraction in the spatial domain is easily affected by factors such as illumination changes and noise interference, frequency domain analysis can decompose the image into sub-bands of different frequencies. The low-frequency sub-band contains the overall outline and main content of the image and is insensitive to illumination changes; the high-frequency sub-band contains the image's edges, textures, and details, capable of capturing minute anomalies. Multi-scale wavelet transform can extract image features from different resolution levels, enabling the system to identify safety hazards at different scales, such as the overall displacement of large equipment and minute cracks in small components. Therefore, before performing a two-dimensional discrete wavelet transform on each frame, spatiotemporal joint adaptive denoising processing is required. This is because the construction site environment is complex, and surveillance videos inevitably contain various noises, such as electronic noise, noise caused by illumination changes, and motion blur. These noises can severely affect the accuracy of subsequent feature extraction and anomaly detection.

[0054] The spatiotemporal joint adaptive denoising process specifically includes: taking the current frame image as the center, acquiring the N adjacent frames before and after it, forming a spatiotemporal image window containing 2N+1 frames. In this embodiment, N is set to 4, meaning the spatiotemporal image window contains 9 frames. A 9-frame window is chosen because it provides sufficient temporal context information to distinguish noise from real motion without introducing excessive computational overhead, ensuring the system's real-time performance. Then, the Sobel operator is used to calculate the horizontal spatial gradient of the current frame image. and vertical spatial gradient The Sobel operator is a commonly used edge detection operator. It calculates the gradient by performing a 3×3 convolution operation on the image, and has the advantages of simple computation and some noise suppression. It calculates the temporal gradient between adjacent frames within a spatiotemporal image window. The temporal gradient reflects the change of pixel values ​​over time and can effectively distinguish between static noise and dynamic objects. Based on the spatial and temporal gradients, a Gaussian kernel function is used for local weighted averaging to construct a 3×3 spatiotemporal structure tensor at each pixel. The spatiotemporal structure tensor can simultaneously describe the structural information of an image in both spatial and temporal dimensions. Its mathematical expression is: ,in, For Gaussian kernel function, This is a convolution operation. For the aforementioned spatiotemporal structure tensor... Eigenvalue decomposition yields three eigenvalues ​​arranged in descending order. and its corresponding orthogonal feature vectors; calculate the temporal consistency measure of each pixel based on the feature values. The time-domain consistency measure The expression is: ,in, To prevent division by zero of extremely small positive numbers; the temporal consistency metric belongs to [0,1]. The closer its value is to 1, the more consistent the features at that pixel are in the time dimension, and the more likely they are to be true structural features; the closer its value is to 0, the more random the temporal dimension changes at that pixel, and the more likely they are to be noise.

[0055] Next, based on the temporal consistency metric and the spatial components of the spatiotemporal structure tensor, adaptive anisotropic diffusion filtering is applied to the current frame image to obtain the denoised image. Specifically, the spatiotemporal structure tensor is extracted. The top-left 2×2 space subtensor The spatial subtensor The horizontal line at the top represents the Gaussian weighted average; eigenvalue decomposition is performed on the spatial subtensor to obtain two spatial eigenvalues ​​arranged in descending order. and the corresponding orthogonal eigenvectors ,in, The direction of the edge tangent. The edge normal direction is used; a noise estimation algorithm based on temporal median residuals is used to adaptively estimate the noise standard deviation of the current frame image. The noise standard deviation The expression is: ,in, The median residual in the time domain is given, and MAD represents the median absolute deviation, based on the above. Set diffusion parameters: Where c is a preset scaling constant, ranging from 1 to 3. Anisotropic diffusion tensor D is constructed, and the expression for the anisotropic diffusion tensor D is: ,in, The eigenvector matrix, the first diffusion coefficient Second diffusion coefficient: ,when When it approaches 0 The diffusion tensor D approaches 1. This leads to strong denoising with isotropic properties. tending towards 1 and Much larger hour, The diffusion tensor tends towards 0, exhibiting strong anisotropy, diffusing only along the edge tangent direction, while diffusion across the edge normal direction is completely suppressed; the current frame image is subjected to M iterations of anisotropic diffusion filtering using the diffusion tensor D, with the expression for each iteration being: Where n is the iteration number index, greater than or equal to 1 and less than or equal to M. For time step, The gradient operator is used; after M iterations, the denoised image is obtained.

[0056] The core of this implementation lies in using the spatial components of the spatiotemporal structure tensor to guide the diffusion direction, while automatically adjusting the diffusion intensity using a temporal consistency metric, achieving adaptive filtering that varies "depending on the region." Specifically, a 2×2 spatial sub-tensor is first extracted from the 3D spatiotemporal structure tensor. Eigenvalue decomposition is then performed on this sub-tensor to obtain two spatial eigenvalues ​​and their corresponding eigenvectors. The eigenvector corresponding to the larger eigenvalue points to the normal direction of the local image edge, while the eigenvector corresponding to the smaller eigenvalue points to the tangent direction of the edge. These two directions constitute the principal axes of the diffusion filter. Subsequently, a diffusion tensor is constructed, which always allows diffusion along the edge tangent direction, while the diffusion intensity along the edge normal direction is controlled by a second diffusion coefficient. The design of the second diffusion coefficient is a key innovation of this scheme. Its expression uses temporal consistency as the core modulation factor, forming an adaptive mechanism: when the temporal consistency of a pixel is low, it indicates that the change at that point is random noise. In this case, the diffusion tensor tends to be isotropic, spreading uniformly in all directions to strongly suppress noise. When the temporal consistency is high and the spatial feature values ​​differ significantly, it indicates that there is a stable true edge at that point. In this case, the diffusion tensor exhibits strong anisotropy, spreading only along the edge direction and hardly spreading across the edge direction, thus completely preserving edge information. Finally, the anisotropic diffusion equation is solved multiple times using this diffusion tensor on the current frame image. Each iteration calculates the divergence based on the diffusion tensor and updates the pixel value. After a preset number of iterations, the denoised image is obtained. The advantage of this approach is that it eliminates the need for manual differentiation between noise and edge regions. The discrimination and diffusion mode switching are automatically completed by data-driven processes. While effectively removing interference such as light flicker and sensor noise commonly found in construction monitoring videos, it perfectly preserves spatial structural information crucial for safety monitoring, such as scaffold outlines and equipment edges, providing high-quality input images for subsequent wavelet transforms and feature extraction.

[0057] After image denoising, each frame of the denoised image undergoes an L-level two-dimensional discrete wavelet transform using a wavelet basis. In this embodiment, the db4 wavelet is chosen as the wavelet basis because it has good regularity and tight support, and can achieve good localization characteristics in both the frequency and time domains; L is set to 3, i.e., a 3-level wavelet decomposition is performed. The 3-level decomposition achieves a good balance between computational complexity and feature representation capability. Specifically, each frame of the denoised image undergoes an L-level two-dimensional discrete wavelet transform using a wavelet basis, obtaining the L-level two-dimensional discrete wavelet transform of each frame. Low-frequency subband coefficient matrix of level and high-frequency subband coefficient matrix , , ,in , It is a high-frequency component in the horizontal direction. These are high-frequency components in the vertical direction. The high-frequency components are located diagonally. For each sub-band coefficient matrix, sub-band energy features, sub-band coefficient statistical features, and sub-band texture features are calculated to form the frequency domain multi-scale feature description. The sub-band energy feature is the square mean of all coefficients in each sub-band coefficient matrix. The sub-band coefficient statistical features include the mean, variance, skewness, and kurtosis of each sub-band coefficient matrix. The sub-band texture features include the contrast, correlation, and entropy of the gray-level co-occurrence matrix of each sub-band coefficient matrix.

[0058] Then, considering that most safety hazards in construction sites are related to the movement and deformation of objects, such as unauthorized climbing by construction workers, abnormal swinging of tower cranes, and deformation of scaffolding, optical flow can accurately describe the motion of each pixel in an image, while the spatial deformation field features composed of motion vector gradients can further capture the non-uniformity of motion and deformation characteristics. Compared with directly using motion vectors, spatial deformation field features are insensitive to changes in illumination and background motion, and can more accurately identify abnormal motion patterns. Therefore, the optical flow field between adjacent frames in a surveillance video is calculated based on the optical flow method to obtain the motion vector of each pixel in each frame, and the spatial deformation field features composed of motion vector gradients are extracted. This embodiment uses the Lucas-Kanade optical flow method to calculate the optical flow field between adjacent frames. The Lucas-Kanade optical flow method is an optical flow calculation method based on a local window, which assumes that the motion vectors of all pixels are the same within a small spatial window. The motion vector is solved by minimizing the squared difference of gray levels of pixels within the window, which has the advantages of fast calculation speed and robustness to noise. After establishing the optical flow field, the horizontal gradient is calculated separately. and vertical gradient The motion vector gradient field is obtained, where u and v are the motion vector components of the pixel in the x and y directions, respectively. Based on the motion vector gradient field, divergence features are calculated, describing the degree of divergence or convergence of the motion vector field, which can identify the expansion or contraction of objects. The curl characteristic describes the degree of rotation of the motion vector field and can identify the rotational motion of an object. The two principal strain values ​​of the strain tensor, along with the strain tensor itself, constitute the spatial deformation field characteristics. The strain tensor describes the degree of deformation of the object, and its expression is: The strain tensor is decomposed into eigenvalues ​​to obtain two principal strain values.

[0059] Next, the frequency domain multi-scale feature description and the spatial deformation field feature are fused to obtain the multimodal feature vector of each frame. This step of fusing the frequency domain multi-scale feature description and the spatial deformation field feature is mainly due to the fact that construction monitoring videos contain rich texture structure information and complex motion dynamic information. A single-dimensional feature description is difficult to fully depict the real state of the construction site, while the two features have natural complementarity in terms of information dimension. After fusion, the expressive and discriminative power of the feature vector can be significantly improved. From a technical point of view, the frequency domain multi-scale feature description decomposes the image into different frequency sub-bands through wavelet transform, capturing static structural information such as texture details, edge contours, and spatial distribution of various objects at the construction site, such as the geometric arrangement of scaffolding and the morphological characteristics of material stacking, focusing on answering the question "What is in the scene?". On the other hand, the spatial deformation field feature calculates divergence, curl, and principal strain values ​​based on optical flow gradient, depicting the local deformation mode of the pixel-level motion field, such as the convergence and divergence trend of objects, rotational motion characteristics, and stretching and compression states, focusing on answering the question "What changes are happening in the scene?". Relying solely on texture features cannot distinguish between static obstacles and temporary safety hazards, while relying solely on motion features cannot identify structural anomalies caused by slow deformation. Specifically, the sub-band features of each level and direction in the frequency domain multi-scale feature description are concatenated to obtain an initial frequency domain feature vector. Principal component analysis is used to reduce the dimensionality of this initial frequency domain feature vector, resulting in a frequency domain dimensionality-reduced feature vector of dimension K. The divergence, curl, and two principal strain values ​​in the spatial deformation field features are spatially divided into P×Q non-overlapping sub-blocks. The mean and standard deviation of the feature values ​​within each sub-block are calculated to obtain a spatial deformation statistical feature vector of dimension 4×P×Q. Finally, the frequency domain dimensionality-reduced feature vector and the spatial deformation statistical feature vector are concatenated end-to-end to obtain the multimodal feature vector. This fusion strategy's multimodal complementarity enhances the robustness of the features; when sudden changes in illumination affect texture features, motion features can still be stably represented, and vice versa. In addition, the high-dimensional joint feature space can more accurately define the boundary between normal and abnormal states, providing a more reliable basis for subsequent memory comparison and anomaly scoring.

[0060] Then, within a preset collection period during the normal operation phase of building construction, the multimodal feature vectors are collected to form a normal sample set. Density clustering algorithm is used to perform cluster analysis on the normal sample set to construct a feature memory library for normal operation. The feature memory library contains multiple feature cluster centers and their corresponding feature neighborhood radii and sample densities.

[0061] Furthermore, considering that the normal operation of a construction site is not a single, unchanging fixed scene, but rather presents various normal state modes as construction progresses, equipment is rotated, and personnel shifts occur, a simple background image or a single feature template cannot be used to represent "normal." Instead, a memory database capable of covering multiple normal modes is needed. This solution continuously collects multimodal feature vectors as normal samples within a preset acquisition period during the normal operation phase of construction. This design takes into account the multi-condition characteristics of construction sites. For example, the work content, equipment type, and personnel density differ significantly between the foundation construction phase and the main structure construction phase, resulting in completely different texture structures and motion patterns in the corresponding video images. These constitute different clustering modes for the normal operation state. Density clustering algorithms are used instead of partitioning clustering such as K-means because density clustering does not require a preset number of clusters, can automatically discover cluster structures of arbitrary shapes in the sample space, and naturally possesses the ability to identify outlier noise points. This is particularly important for occasional abnormal frame samples in construction monitoring, preventing noise points from being forcibly assigned to a certain cluster center and thus contaminating the representation of normal modes. This embodiment uses the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm to perform clustering analysis on a normal sample set. DBSCAN is a density-based clustering algorithm that defines a cluster as the largest set of density-connected points. It can discover clusters of arbitrary shapes and automatically identify noise points. The calculation of core feature clustering parameters is performed in three steps: First, the cluster center is calculated. For each core cluster output by DBSCAN, the arithmetic mean of the multimodal feature vectors of all samples within the cluster is calculated by dimension to obtain the center vector representing that normal mode. Second, the feature neighborhood radius is calculated. The Mahalanobis distance from each sample within the cluster to the center vector is calculated sequentially. All distance values ​​are arranged in ascending order, and the 95th percentile value is taken as the neighborhood radius. This can cover more than 95% of the normal samples and exclude the very few marginal outliers mixed in during the collection process. The third step calculates the sample density. Based on the dimension of the feature vector and the neighborhood radius, the volume of the corresponding high-dimensional hypersphere is calculated. Then, the total number of samples in the cluster is divided by this volume to obtain the sample density value reflecting the frequency of the normal pattern. Finally, the center vector, neighborhood radius, sample density, number of samples in the cluster, and corresponding environmental label of each core cluster are packaged into a cluster entry. All entries are sorted by sample density from high to low and stored in the feature memory. This structured storage method supports subsequent fast retrieval and comparison, prioritizing the inspection of high-frequency normal patterns, and can significantly improve the computational efficiency of real-time anomaly detection.

[0062] Then, for each frame of the image acquired in real time, the current multimodal feature vector is calculated, and the current multimodal feature vector is compared with the feature memory to obtain the anomaly score of the current frame. Specifically, the multimodal feature vector x of the current frame is calculated and compared with the i-th cluster center in the feature memory. Mahalanobis distance between ,in, Let be the covariance matrix of the i-th cluster. Take the minimum value among all Mahalanobis distances. The minimum value Normalization is performed by dividing by the feature neighborhood radius r of the corresponding cluster to obtain the global anomaly score of the current frame. The larger the value, the further the image features of the current frame deviate from the normal operating state.

[0063] Next, the anomaly score is compared with the early warning threshold. Images exceeding the early warning threshold are recorded as initial early warning images. Furthermore, considering the core challenges of dynamic and changing construction site environments, gradual evolution of the construction process, and the poor robustness of fixed thresholds, the following considerations are taken into account. In construction scenarios, light intensity fluctuates drastically between day and night and with varying weather conditions; severe weather such as dust, rain, and snow interferes with image quality; background features differ significantly at different construction stages; and personnel and equipment density dynamically changes with the work rhythm. These factors can all lead to a systematic shift in the feature distribution of normal images. If a fixed early warning threshold is used, either a threshold that is too high will miss real safety hazards, or a threshold that is too low will generate a large number of invalid alarms, leading to early warning fatigue among on-duty personnel. Therefore, this embodiment uses Bayesian inference to adaptively update the threshold. Specifically, a probability density distribution model is established for the anomaly score sequence of historical W frames. A Gaussian kernel density estimation method is used to obtain the prior probability distribution of the anomaly scores. When a new anomaly score is obtained, the prior distribution is updated to a posterior distribution based on the Bayesian formula. The maximum inter-class variance method is used to perform initial threshold segmentation on the histogram of the anomaly scores to obtain an initial threshold. The initial threshold is then calibrated using the posterior distribution to obtain an adaptive warning judgment threshold. Based on the probability distribution of historical anomaly scores, prior knowledge is continuously iterated, quantifying changes in the environment and construction phases into a posterior probability distribution, and automatically calibrating the threshold to match the normal feature distribution of the current scene. This method can stably control the false alarm rate within a preset range while maintaining high sensitivity to real anomalies. It eliminates the need for frequent manual parameter tuning and forms a closed loop with the online incremental update of the feature memory, ensuring long-term stable operation of the system throughout the entire construction cycle.

[0064] Considering the typical characteristics of construction sites—numerous instantaneous environmental disturbances and frequent accidental anomalies—various factors in construction scenarios, such as birds flying across the frame, light flickering due to sunlight obstruction, dust particles drifting across the image, reflections from equipment surfaces, and motion blur caused by rapid personnel movement, can all cause the anomaly score of a single frame image to briefly exceed the threshold. However, these are all instantaneous disturbances posing no safety risk. Without time-dimensional verification, the system would generate a massive number of invalid alarms, wasting significant manpower in troubleshooting and causing severe warning fatigue among safety management personnel, ultimately leading to the neglect of genuine safety hazards. Finally, the initial warning image undergoes time-dimensional persistence verification. When the duration of the initial warning image meets the warning conditions, a safety warning signal is output.

[0065] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments for application in other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A smart construction site safety early warning method, characterized in that, Includes the following steps: S1. Acquire monitoring video of the construction site, perform two-dimensional discrete wavelet transform on each frame of the monitoring video to obtain the low-frequency subband coefficient matrix and high-frequency subband coefficient matrix of each frame in multiple frequency bands, and construct a frequency domain multi-scale feature description of each frame. S2. Calculate the optical flow field between adjacent frames in the surveillance video based on the optical flow method, obtain the motion vector of each pixel in each frame, and extract the spatial deformation field features composed of the motion vector gradient. S3. The frequency domain multi-scale feature description is fused with the spatial deformation field feature to obtain the multimodal feature vector of each frame image; S4. During the preset collection period of the normal operation phase of building construction, the multimodal feature vectors are collected to form a normal sample set. The normal sample set is clustered using a density clustering algorithm to construct a feature memory library for normal operation. The feature memory library contains multiple feature cluster centers and their corresponding feature neighborhood radii and sample densities. S5. Calculate the current multimodal feature vector for each frame of image acquired in real time, compare the current multimodal feature vector with the feature memory, and obtain the anomaly score of the current frame. S6. Compare the anomaly score with the early warning judgment threshold, and record the images that exceed the early warning judgment threshold as the initial early warning images; S7. Perform time-dimensional persistence verification on the initial warning image. When the duration of the initial warning image meets the warning conditions, output a safety warning signal.

2. The intelligent construction site safety early warning method according to claim 1, characterized in that, Before performing two-dimensional discrete wavelet transform on each frame of the image in step S1, spatiotemporal joint adaptive denoising processing is also required, specifically including: Centered on the current frame image, acquire the N adjacent frame images before and after it to form a spatiotemporal image window containing 2N+1 frames; The Sobel operator is used to calculate the horizontal spatial gradient of the current frame image. and vertical spatial gradient Calculate the temporal gradient between adjacent frames within the spatiotemporal image window. ; Based on the spatial gradient , and time gradient A local weighted average is constructed for each pixel using a Gaussian kernel function. Spacetime structure tensor; The spatiotemporal structure tensor is decomposed into eigenvalues ​​to obtain three eigenvalues ​​arranged in descending order. The temporal consistency metric of each pixel is calculated based on the eigenvalues. Based on the temporal consistency metric and the spatial components of the spatiotemporal structure tensor, adaptive anisotropic diffusion filtering is applied to the current frame image to obtain a denoised image.

3. The intelligent construction site safety early warning method according to claim 2, characterized in that, The implementation of adaptive anisotropic diffusion filtering on the current frame image based on the temporal consistency metric and the spatial components of the spatiotemporal structure tensor to obtain the denoised image is as follows: Extract the top left corner of the spatiotemporal structure tensor Spatial subtensor; Eigenvalue decomposition is performed on the spatial subtensor to obtain two spatial eigenvalues ​​arranged in descending order and their corresponding orthogonal eigenvectors; An anisotropic diffusion tensor is constructed based on orthogonal eigenvectors. The current frame image is then subjected to anisotropic diffusion filtering for M iterations using the diffusion tensor. After M iterations, the denoised image is obtained.

4. The intelligent construction site safety early warning method according to claim 1, characterized in that, In step S1, a two-dimensional discrete wavelet transform is performed on each frame of the surveillance video to obtain the low-frequency subband coefficient matrix and the high-frequency subband coefficient matrix of each frame in multiple frequency bands, thereby constructing a frequency domain multi-scale feature description of each frame, specifically including: For each frame of the denoised image, an L-level two-dimensional discrete wavelet transform is performed using a wavelet basis to obtain the L-level two-dimensional discrete wavelet transform of each frame. Low-frequency subband coefficient matrix of level and high-frequency subband coefficient matrix , , ,in , It is a high-frequency component in the horizontal direction. These are high-frequency components in the vertical direction. These are high-frequency components in the diagonal direction; For each sub-band coefficient matrix, sub-band energy features, sub-band coefficient statistical features, and sub-band texture features are calculated to constitute the frequency domain multi-scale feature description. The sub-band energy feature is the square mean of all coefficients in each sub-band coefficient matrix. The sub-band coefficient statistical features include the mean, variance, skewness, and kurtosis of each sub-band coefficient matrix. The sub-band texture features include the contrast, correlation, and entropy of the gray-level co-occurrence matrix of each sub-band coefficient matrix.

5. The intelligent construction site safety early warning method according to claim 1, characterized in that, In step S2, the optical flow field between adjacent frames in the surveillance video is calculated based on the optical flow method to obtain the motion vector of each pixel in each frame. The spatial deformation field features composed of the motion vector gradient are then extracted, specifically including: The optical flow field is constructed by calculating the two-dimensional motion vector of each pixel between adjacent frames using the optical flow method. The horizontal and vertical gradients of the optical flow field are calculated to obtain the motion vector gradient field. The divergence and curl characteristics, as well as the two principal strain values ​​of the strain tensor, are calculated based on the gradient field of the motion vector to form the spatial deformation field characteristics.

6. The intelligent construction site safety early warning method according to claim 1, characterized in that, In step S3, the frequency domain multi-scale feature description is fused with the spatial deformation field features to obtain a multimodal feature vector for each frame of the image, specifically including: The sub-band features of each level and direction in the frequency domain multi-scale feature description are concatenated to obtain the initial frequency domain feature vector. The initial frequency domain feature vector is reduced in dimension by using principal component analysis to obtain a frequency domain feature vector with dimension K. The divergence, curl, and two principal strain values ​​in the spatial deformation field characteristics are divided into P×Q non-overlapping sub-blocks in space. The mean and standard deviation of the characteristic values ​​in each sub-block are calculated to obtain a spatial deformation statistical feature vector with a dimension of 4×P×Q. The frequency domain dimensionality-reduced feature vector is concatenated with the spatial deformation statistical feature vector to obtain the multimodal feature vector.

7. The intelligent construction site safety early warning method according to claim 1, characterized in that, The specific implementation of constructing the normal operation state feature memory in step S4 includes: During the normal operation phase of building construction, the multimodal feature vector of one frame of image is collected at every preset sampling interval, and a normal sample set containing T samples is collected within the preset collection period. Calculate the cluster center, feature neighborhood radius, and sample density for each core feature cluster. The feature neighborhood radius is the 95th percentile of the distance from all samples in the cluster to the cluster center, and the sample density is the ratio of the number of samples in the cluster to the volume of the feature neighborhood. The cluster centers, feature neighborhood radii, and sample densities of the core feature clusters are stored as the normal operating state feature memory.

8. The intelligent construction site safety early warning method according to claim 1, characterized in that, The specific implementation of comparing the current multimodal feature vector with the feature memory in step S5 to obtain the anomaly score of the current frame is as follows: Calculate the Mahalanobis distance between the multimodal feature vector of the current frame and the i-th cluster center in the feature memory; Take the minimum value among all Mahalanobis distances, divide the minimum value by the feature neighborhood radius of the corresponding cluster, and normalize to obtain the global anomaly score of the current frame.

9. A smart construction site safety early warning method according to claim 1, characterized in that, In step S6, the early warning judgment threshold is adaptively updated through Bayesian inference, specifically including: A probability density distribution model is established for the anomaly score sequence of historical W frames, and the prior probability distribution of the anomaly score is obtained by using the Gaussian kernel density estimation method. After obtaining a new anomaly score, the prior distribution is updated to a posterior distribution based on Bayes' theorem. The initial threshold is obtained by segmenting the histogram of anomaly scores using the maximum inter-class variance method. The initial threshold is calibrated using the posterior distribution to obtain an adaptive early warning judgment threshold.

Images