Intelligent construction site management method and system based on photoelectric and video fences

Abstract

The invention discloses an intelligent construction site management method and system based on photoelectricity and a video fence, the system is realized based on the intelligent construction site management system, the system comprises a control module and an information acquisition device connected with the control module, and the information acquisition device comprises a camera device, a cable monitoring device, an environment state parameter sensor and a construction equipment intelligent switch; and the cable monitoring devices are arranged along the extending direction of the cable at intervals of preset lengths. The method comprises the following steps: regularly acquiring and preprocessing real-time data of a construction site; analyzing the video data to identify a dynamic area; cable abnormity and equipment state are evaluated; generating a motion intensity graph and a key frame candidate set; tracking a target, generating a track, and determining a responsibility subject. According to the method, comprehensive monitoring and anomaly detection of the construction site are realized through multi-source data fusion and intelligent analysis, and the management efficiency and safety are improved.

Description

Technical Field

[0001] The present invention relates to construction site management, including cable fault detection and management, construction site safety management, and in particular to an intelligent construction site management method and system based on photoelectric and video fences. Background Art

[0002] In the complex and ever-changing construction environment, traditional manual supervision methods can no longer meet the growing needs for safety and efficiency. In construction sites and industrial and mining enterprises, various wires and cables connecting construction machinery and power tools face severe safety challenges. Due to the harsh operating environment and frequent movement, these cables are more susceptible to external force damage such as pulling, crushing, smashing, frequent bending, etc., resulting in broken core failures. However, due to the outer sheath, the broken core failure point is often not visible from the surface, which makes it difficult to find the fault point quickly and accurately. For this reason, the Chinese patent with publication number CN102590714 provides a cable fault detector, but does not disclose the data processing method of the detector. The monitoring lacks systematicity, and there may be negligence and errors in recording, and there is the possibility of omissions, which poses risks and has little guiding significance for subsequent management.

[0003] To this end, the Chinese patent with publication number CN116243072B provides a maintenance management method for a systematic maintenance management system of electrical equipment suitable for construction sites, which discloses the information collection and processing of electrical equipment, but does not conduct traceability and responsibility tracking.

[0004] In short, in the existing management system, most systems still operate independently and lack effective data fusion and collaborative analysis capabilities. In complex construction site environments, existing video analysis technologies often have difficulty coping with problems such as lighting changes and occlusions, resulting in low recognition accuracy. Monitoring of important facilities such as cables and pile foundations, as well as construction areas and dangerous areas is still relatively weak, especially in the detection of hidden problems such as broken core failures. In addition, most existing systems focus on post-analysis and lack effective prediction and prevention mechanisms.

[0005] Therefore, research and innovation are needed. Summary of the invention

[0006] The purpose of the invention is to provide an intelligent construction site management method and system based on photoelectric and video fences, in order to solve the above-mentioned problems existing in the prior art.

[0007] Technical solution, an intelligent construction site management method based on photoelectric and video fences, implemented based on an intelligent construction site management system, including a control module, and an information collection device connected to the control module, the information collection device including a camera device, a cable monitoring device, an environmental state parameter sensor, and an intelligent switch for construction equipment; the cable monitoring device is set at predetermined lengths along the direction in which the cable extends; The method comprises the following steps: Step S1, acquiring real-time data of the construction site at every predetermined period, and preprocessing it to form a preprocessed data set, including enhanced video data, cable operation data, environmental state parameters and equipment switch status; Step S2, read enhanced video data, identify and obtain the current construction site status, obtain and mark the dynamic area, and form a dynamic area data set; Step S3, read the cable operation data and the equipment switch status, extract the cable abnormal parameters, evaluate the severity of the abnormality, output the abnormality level index and compare it with the threshold value, if it exceeds the threshold value, record the timestamp of the abnormality occurrence time and the corresponding detection area coordinates; Step S4, according to the timestamp of the abnormality occurrence time, retrieve a video segment of a predetermined length from the dynamic area data set, determine the key frame position range and form a key frame candidate set; Step S5: Identify and track the target based on the key frame and match it across frames to generate a target trajectory, and obtain a list of responsible entities based on the target trajectory.

[0008] According to another aspect of the present application, there is also provided an intelligent construction site management system based on photoelectric and video fences, comprising a control module, and an information collection device connected to the control module, the information collection device comprising a camera device, a cable monitoring device, an environmental state parameter sensor, and an intelligent switch for construction equipment; the cable monitoring device is arranged at predetermined lengths along the direction in which the cable extends; wherein the control module comprises: at least one processor; and, a memory communicatively connected to at least one of the processors; wherein, The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the intelligent construction site management method based on photoelectric and video fences as described in any of the above technical solutions.

[0009] Beneficial effects: Through multimodal data fusion technology, combined with cable parameter monitoring and video analysis, it can effectively detect and locate hidden faults such as broken cores; improve the accuracy and timeliness of fault diagnosis; through target tracking and trajectory analysis, the system can accurately identify the responsible party; and improve the efficiency and accuracy of construction site supervision. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1It is a flow chart of the present invention.

[0011] Figure 2 It is a flow chart of step S1 of the present invention.

[0012] Figure 3 It is a flow chart of step S2 of the present invention.

[0013] Figure 4 It is a flow chart of step S3 of the present invention.

[0014] Figure 5 It is a flow chart of step S4 of the present invention.

[0015] Figure 6 It is a flow chart of step S5 of the present invention. DETAILED DESCRIPTION

[0016] According to one aspect of the present application, there is provided an intelligent construction site management method based on photoelectric and video fences, which is implemented based on an intelligent construction site management system, including a control module, and an information collection device connected to the control module, the information collection device including a camera device, a cable monitoring device, an environmental state parameter sensor, and an intelligent switch for construction equipment; the cable monitoring device is arranged at predetermined lengths along the direction in which the cable extends; The method comprises the following steps: Step S1, every predetermined period, the real-time data of the construction site is acquired through the information acquisition device, and pre-processed to form a pre-processed data set; the real-time data of the construction site includes enhanced video data of the construction site, cable operation data, environmental state parameters and equipment switch status; Step S2, reading the construction site video data in the preprocessing data set, identifying and acquiring the current construction site status, obtaining and marking the dynamic area, and forming a dynamic area data set; Step S3, read the cable operation data and equipment switch status in the preprocessing data set, and call the preconfigured anomaly detection module to extract the cable abnormality parameters; integrate the cable abnormality parameters, equipment switch status and environmental status parameters, call the fuzzy reasoning system to evaluate the severity of the abnormality, output the abnormality level index and compare it with the threshold value, if it exceeds the threshold value, record the timestamp of the abnormality occurrence time and the corresponding detection area coordinates; Step S4: according to the timestamp, retrieve a video segment of a predetermined length from the construction site video data, calculate the motion vector of each frame, and generate a motion intensity map; determine the key frame position range and form a key frame candidate set; Step S5: Identify the tracking target based on the key frame candidate set, perform cross-frame matching on the tracking target, generate the target trajectory, and obtain the list of responsible entities according to the target trajectory; the tracking target includes personnel and equipment.

[0017] In this embodiment, through the coordinated work of the camera device, the cable monitoring device, the environmental state parameter sensor and the intelligent switch of the construction equipment, the system can collect real-time data of the construction site in all directions. Through multi-source data collection, the comprehensiveness and accuracy of monitoring are significantly improved. The environmental state parameters improve the accuracy and reliability of anomaly detection. By generating motion intensity graphs and key frame recognition, the system can quickly locate key events and improve the response speed of abnormal events. Through target tracking and trajectory analysis, the system can accurately identify the responsible party. In short, the efficiency and accuracy of construction site supervision are improved.

[0018] In this embodiment, the hardware system is as follows: As the central processing unit of the system, the control module is responsible for coordinating and managing the operation of the entire system, establishing direct data connections with various information collection devices, and receiving and processing information from these devices.

[0019] Camera device Multiple high-definition cameras are installed along the perimeter of the construction site and in key areas. The cameras are connected to the control module via a wired or wireless network. Real-time video streams are transmitted to the control module for processing and analysis.

[0020] The cable monitoring device is similar to a cable fault detector. A monitoring point is set at a predetermined length (such as every 50 meters) along the extension direction of the cable.

[0021] Each monitoring point contains the following components: a wire and cable fault tester, which is directly connected to the cable under test. A fault sensing probe, which is connected to the audio signal input of the tester. A positioning signal sensor and a path signal sensor, which are connected to the cable under test and transmit the signal to the positioning signal receiver. A high-energy impact signal generator and a path signal generator, which are connected to the cable under test and are used to generate test signals. The data from all these components are summarized by a dedicated data acquisition unit and then transmitted to the control module.

[0022] Environmental status parameter sensors include various environmental sensors installed at key locations on the construction site, such as temperature, humidity, dust, noise sensors, etc. These sensors transmit data to the control module in real time via wireless networks (such as LoRa or NB-IoT).

[0023] Install smart switches on major construction equipment. These switches can monitor the equipment's switch status, operating hours, energy consumption and other information. The data is transmitted to the control module in real time via a wireless network.

[0024] In another embodiment of the present application, a photoelectric sensor array for perimeter protection is also included.

[0025] According to one aspect of the present application, step S1 specifically comprises: Step S11, every predetermined period, obtaining the original video data collected by the construction site camera device and performing enhancement processing to obtain enhanced video data; Step S12, obtaining cable parameters through a cable monitoring device to form an original cable data set, a normal mode of the cable, an abnormal mode of the cable and cable parameter values; Step S13, obtaining the initial environmental state parameters collected by each environmental state parameter sensor, and performing fusion processing to obtain fused environmental state parameters; Step S14: collecting working parameters of the intelligent switch of the construction equipment, constructing original equipment data, and performing preprocessing to form the equipment switch state; Step S15: Based on the output data of steps S11 to S14, a multimodal data set is constructed, and time alignment is performed through a multidimensional dynamic time warping algorithm to form a preprocessed data set, i.e., a construction site basic data set, and stored.

[0026] By enhancing the raw video data, the video quality is improved, which is helpful for subsequent video analysis. The cable parameters are obtained through the cable monitoring device, and the normal and abnormal modes are distinguished, which helps to quickly identify potential cable problems. Environmental status parameters According to one aspect of the present application, step S2 is specifically: Step S21, read enhanced video data, extract video frames, and for each video frame, use superpixel segmentation to divide the region, obtain several superpixel regions and extract feature vectors, cluster the feature vectors through hierarchical clustering method, and obtain the initial scene segmentation result; compared with traditional pixel-level processing, it is more efficient and can retain the edge information of the image. Through clustering, it can better adapt to complex construction site scenes.

[0027] Step S22: Optimize the initial scene segmentation result using a spatiotemporal Markov random field module and minimize the energy function to obtain a spatiotemporal consistent scene segmentation result, thereby improving the consistency and accuracy of the scene segmentation.

[0028] Step S23: Based on the spatiotemporal consistent scene segmentation result, an improved optical flow estimation method is used to calculate the motion vector field between consecutive frames, the dynamic area is identified according to the motion vector field, and the motion significant area is segmented by calculating the amplitude and direction histogram of the motion vector to obtain the motion analysis result; each motion of the motion significant area is tracked to obtain the dynamic area tracking result, that is, the dynamic area data set; Step S24: Annotate and hierarchically represent the dynamic area tracking results, annotate and integrate the cable operation data and environmental data, construct and output a comprehensive dynamic area description data set. This is to establish a mapping relationship between image, cable, and environmental state parameters. Use the spatiotemporal range of the dynamic area as a constraint to filter the corresponding cable and environment information to reduce the total amount of information. Collect data that may have problems to reduce the scope and workload of subsequent screening.

[0029] The system's ability to perceive dynamic conditions on construction sites has been significantly improved, enabling it to detect and track abnormal activities in a timely manner, thus enhancing the effectiveness of construction site safety supervision.

[0030] According to one aspect of the present application, step S3 is specifically: Step S31, obtain a comprehensive dynamic area description data set, perform multivariate anomaly detection and dimensionality reduction, and obtain a processed multi-source synchronous data set, including cable operation data, environmental state model, and dynamic area data set; effectively reduce data redundancy and improve the efficiency of subsequent analysis.

[0031] Step S32, read the cable operation data and perform modal decomposition and construct a multi-dimensional phase space, and use a clustering algorithm to preliminarily detect whether there is an abnormal pattern; at the same time, based on the processed multi-source synchronous data set, call the pre-configured anomaly detection module, the module uses a sliding window singular spectrum analysis method and a local anomaly factor method to calculate and output the cable anomaly detection result, that is, to obtain a cable anomaly parameter set; the cable anomaly parameter set includes: anomaly type, anomaly score and corresponding timestamp; improves the accuracy and reliability of anomaly detection.

[0032] Step S33, fusing the cable abnormal parameter set with the equipment switch state and environmental state parameters in the multi-source synchronous data set, and outputting the fused abnormal feature data set, i.e., the abnormality detection result; Step S34, calling a preconfigured fuzzy inference system to evaluate each data point in the abnormal feature data set (abnormal detection result), obtaining an abnormal severity evaluation result, and comparing it with the abnormal level threshold to obtain an abnormal level determination result; Step S35, record the timestamp and coordinate information of the abnormal severity assessment result, and output the spatiotemporal positioning result.

[0033] Environmental status parameters improve the system's ability to identify and respond to abnormal conditions on the construction site, providing technical support for timely discovery and handling of potential safety hazards, thereby improving the overall safety level of the construction site.

[0034] According to one aspect of the present application, step S4 is specifically: Step S41, reading the dynamic area data set and the anomaly detection result, and extracting the corresponding video clip from the dynamic area data set; Step S42: using an adaptive contrast enhancement algorithm to further enhance and stabilize the video clips, and outputting an optimized set of video clips; Step S43, for each optimized video clip set, obtain and calculate the motion saliency based on the corresponding motion analysis results, obtain the motion intensity map, calculate the visual saliency, generate a multi-feature fusion saliency map, and output the saliency sequence of each video clip; generating a multi-feature fusion saliency map can more comprehensively capture the important information in the video.

[0035] Step S44: Based on the optimized video clip set, calculate the temporal saliency, obtain the potential key frame position range, determine the potential key frame, and output the key frame candidate set. Calculating the temporal saliency to determine the potential key frame position range can effectively locate the key moment when the abnormal event occurs.

[0036] In some embodiments, if it is detected that the dynamic area data set cannot cover all key video clips, then based on the temporal and spatial continuity analysis, some video frames are searched again in the enhanced video to supplement the key video frames.

[0037] It not only improves the accuracy of key frame extraction, but also reduces the amount of video data that needs to be manually reviewed. Video analysis and key information extraction improve the response speed and processing efficiency of abnormal events.

[0038] According to one aspect of the present application, step S5 is specifically: Step S51, read the key frame data set, dynamic area data set, anomaly detection results and pre-configured registration database; use the YOLOvx target detection algorithm for identification and tracking; obtain the target list; generally use YOLOv5 or above version.

[0039] Step S52, performing cross-frame matching on the target results one by one; generating target trajectory data to form a target trajectory; being able to effectively track the moving target and generate continuous target trajectory data.

[0040] Step S53: Combine the target trajectory and use the spatiotemporal correlation analysis algorithm to determine the target closest to the detection area at the time of the abnormality, and output a list of potential responsible entities. This can effectively associate abnormal events with specific targets (such as workers or equipment).

[0041] In most cases, the responsible party information can be obtained from the key video frame. If the information is missing, such as when the cross-frame matching is broken or disappears, it is searched in the dynamic area dataset. If it is still not found, it is searched in the enhanced video. Overall, the search speed is fast and the efficiency is high.

[0042] It not only improves the efficiency of accident investigation, but also provides an objective and reliable basis for construction site safety management. It improves the accuracy and intelligence level of construction site supervision, and provides a technical solution for timely discovery of safety hazards and rapid identification of responsible parties.

[0043] According to one aspect of the present application, in step S32, the cable operation data is read and modal decomposition is performed and a multidimensional phase space is constructed, and a clustering algorithm is used to preliminarily detect whether there is an abnormal pattern, specifically: Step S321, read each time series in the cable operation data, identify all local extreme points in the time series, generate upper and lower envelopes by cubic spline interpolation method and calculate the mean envelope, calculate the difference between each time series and the mean envelope, and determine whether the difference meets the eigenfunction modal condition. If so, output the eigenmode function, calculate the residual between the time series and the eigenmode function, and use the residual as a new time series until the residual becomes a monotonic function; Step S322, read each intrinsic mode function, perform Hilbert transform on it, and obtain an analytical signal; based on the analytical signal, calculate the instantaneous amplitude and instantaneous phase; use the instantaneous phase to calculate the instantaneous frequency; output the instantaneous frequency and amplitude data of each intrinsic mode function; Step S323, read the instantaneous frequency and amplitude data of each cable parameter, select a specific frequency component as a dimension, and construct a multidimensional phase space; map the data at each time point to the space to form a point set; and output the constructed multidimensional phase space data; Step S324, read the multidimensional phase space data, set the clustering parameter ε and the minimum point number threshold; for each point in the space, calculate the number of points in its ε-neighborhood; if the number of points is greater than or equal to the minimum point number threshold, mark the point as a core point; for each core point, recursively add the points in its ε-neighborhood to the same cluster; mark the points that are not assigned to any cluster as outliers; output the clustering results and outlier point marks; Step S325, read the clustering results and abnormal point marks, analyze the distribution characteristics of the abnormal points; extract the specific parameter values ​​corresponding to the abnormal points in combination with the original cable parameter data; identify and classify the detected abnormal patterns according to the preset abnormal pattern judgment criteria; output the abnormal pattern description and the corresponding cable parameter values.

[0044] The accuracy and reliability of cable anomaly detection are improved, and potential cable problems can be discovered early, thereby preventing possible safety accidents. At the same time, this method also provides important data support for cable maintenance and life prediction, which helps to optimize cable management strategies and improve the safety level of electricity use on construction sites.

[0045] According to one aspect of the present application, in step S32, based on the processed multi-source synchronous data set, a pre-configured anomaly detection module is called, and the module uses a sliding window singular spectrum analysis method and a local anomaly factor method to calculate and output the cable anomaly detection result, specifically: Step S326, read the multi-source synchronous data set, extract the cable related parameters, and construct a multi-dimensional time series, where each dimension represents a cable parameter and each point on the time axis corresponds to a sampling time; Step S327, constructing a trajectory matrix based on the multidimensional time series and determining the size of the sliding window; performing singular value decomposition on the estimation matrix, obtaining singular values ​​and sorting them, reconstructing the signal based on the first N singular values ​​and calculating the reconstruction error; Step S328, read the reconstruction error, calculate the moving average and standard deviation of the reconstruction error, calculate the threshold value according to the moving average, standard deviation and pre-configured adjustable parameters, and if the reconstruction error is greater than the threshold value, mark it as a potential outlier to form a potential outlier set; Step S329: For each potential outlier point, calculate the distance and reachable distance from each point to the kth nearest neighbor, calculate the local density of the outlier point based on the reachable distance, and calculate the LOF score by the average of the ratio of the local density of each outlier point's neighbor and the local density of the outlier point itself; determine whether the LOF score is greater than a threshold, if so, confirm it as an outlier, and form an outlier set; N and k are natural numbers greater than 0.

[0046] The sensitivity and reliability of anomaly detection are improved, and potential cable problems can be discovered early, thereby effectively preventing possible safety accidents. At the same time, the adaptability of this method also enables it to better cope with different types of cables and different construction site environments, improving the versatility and practicality of the system.

[0047] According to one aspect of the present application, step S32 further includes: Step S3210: randomly select a number of outliers as cluster centers, assign each of the remaining outliers to the nearest cluster center, and update the cluster center to the mean of all outliers of this class until convergence; Step S3211: calling a preconfigured decision tree module, performing feature importance analysis on each cluster, and generating a descriptive label for each cluster based on the feature importance; Step S3212: Use the dynamic time warping algorithm to calculate the distance between each cluster and the normal situation, and give the abnormality analysis result.

[0048] It not only improves the accuracy of anomaly detection, but also provides more detailed and meaningful anomaly information. Through automatic classification and severity assessment, the system reduces the workload of manual analysis and improves the efficiency of anomaly handling. By identifying different types of anomalies and their severity, managers can formulate maintenance strategies and safety measures more targetedly.

[0049] According to one aspect of the present application, in step S11, the process of enhancing the original video data of the construction site is specifically as follows: Step S111, obtaining original video data of the construction site, extracting video frame images, and decomposing each video frame image by discrete wavelet transform to obtain four frequency sub-bands; Step S112: for each frequency sub-band, calculate the local mean and standard deviation of the frequency sub-band; construct an enhancement function based on the local mean, standard deviation and pre-stored adjustable parameters; apply the enhancement function to each pixel in the frequency sub-band; Step S113: Reconstruct the enhanced frequency subbands using a weighted summation method to obtain an enhanced image, and splice the enhanced images to form an enhanced video stream, that is, enhanced video data.

[0050] By adjusting the adjustable parameters, the system can flexibly balance detail enhancement and noise suppression to adapt to different construction site environments and monitoring needs. The weighted summation method is used to reconstruct the enhanced frequency subband, which can effectively fuse the information of different frequency components and generate enhanced images with better visual effects. It not only improves the clarity and contrast of the image, but also effectively suppresses noise, which is particularly suitable for processing video data in complex and changeable environments such as construction sites.

[0051] According to one aspect of the present application, step S21 is specifically: Step S211, reading enhanced video data and extracting video frames; Step S212: for each frame of the enhanced video data, divide it into several initial regions, set the region center, iteratively calculate the distance between each pixel and the superpixel center, update the pixel attribution and the superpixel center, until convergence or reaching the maximum number of iterations, and obtain the superpixel segmentation result; Step S213, for each superpixel region in the superpixel segmentation result, calculate the color histogram, use the improved SURF algorithm to extract texture features, calculate the shape descriptor, and combine these features into a comprehensive feature vector; Step S214: cluster all comprehensive feature vectors using a hierarchical clustering algorithm, calculate the distance matrix between feature vectors, construct a hierarchical tree, cut the feature hierarchical tree according to a preset threshold, and obtain an initial scene segmentation result.

[0052] In this embodiment, the superpixel segmentation method is used to perform preliminary segmentation of the video frame, which is more efficient than traditional pixel-level processing and can effectively retain the edge information and local consistency of the image. By iteratively optimizing the superpixel center and pixel attribution, the system can generate preliminary segmentation results that conform to the image structure.

[0053] According to one aspect of the present application, step S22 is specifically: Step S221, reading the initial scene segmentation results of several consecutive frames, constructing a spatiotemporal Markov random field model, and constructing an energy function including a data term and a smoothing term; Step S222: using a Gaussian mixture model to estimate the probability that a pixel belongs to each label, and calculating the data item in the energy function; Step S223, calculating weights based on color similarity and spatial distance of adjacent pixels to obtain a smoothing term in the energy function; Step S224: Apply the graph cut algorithm to minimize the energy function to obtain an optimized scene segmentation label, and generate and output a scene segmentation result that is consistent in time and space based on the scene segmentation label.

[0054] In this embodiment, an energy function including data terms and smoothing terms is constructed, which can effectively balance local observation and global consistency. The Gaussian mixture model is used to estimate the probability that a pixel belongs to each label, which can better handle complex data distribution and improve the accuracy of segmentation. The weight calculation method based on color similarity and spatial distance is introduced so that the smoothing term can better maintain the structural information of the image. The graph cut algorithm is applied to minimize the energy function, which can efficiently solve large-scale optimization problems and is suitable for processing high-resolution construction site monitoring videos. By considering the temporal consistency between consecutive frames, the temporal stability of the scene segmentation results is improved, and the segmentation jitter caused by factors such as illumination changes and occlusion is effectively reduced. The optimization method based on spatiotemporal Markov random fields not only improves the accuracy of single-frame segmentation, but also ensures consistency across frames, which is particularly suitable for processing dynamic and complex construction site environments. By generating spatiotemporal consistent scene segmentation results, this method provides a reliable foundation for subsequent target tracking, behavior analysis and other tasks, thereby improving the performance and stability of the entire construction site monitoring system. It significantly enhances the system's ability to understand the construction site environment and provides more accurate and coherent visual information for applications such as safety management and anomaly detection, thereby improving the intelligence level of construction site management and decision-making support capabilities.

[0055] According to one aspect of the present application, step S23 is specifically: Step S231, reading the scene segmentation result that is consistent in time and space, and applying the improved optical flow estimation algorithm to the consecutive frames; Step S232: for each pixel, construct an energy function including a brightness constant term and a motion smoothing term between two adjacent frames; Step S233, using a variational method and a multi-resolution strategy to solve the optimization problem and obtain a pixel-level motion vector field; Step S234, calculating the amplitude and direction histogram of the motion vector field, and segmenting the motion-significant area using an adaptive threshold method; Step S235, applying morphological operations, including opening and closing operations, to the segmented motion significant regions to refine the region boundaries and obtain candidate dynamic regions; Step S236: for each candidate dynamic region, construct a cyclic shift sample and train a kernel correlation filter as a target tracker; Step S237, for subsequent frame images, using the trained kernel correlation filter to calculate the response map, and locate the maximum response point as the new target position; Step S238: Update the tracker model according to the new position and output the dynamic area tracking result.

[0056] By constructing an energy function containing constant brightness terms and smooth motion terms, and solving it using the variational method and multi-resolution strategy, the system can effectively handle large-scale motion and occlusion problems. By calculating the amplitude and direction histogram of the motion vector field and using the adaptive threshold method to segment the motion-significant area, the real dynamic target can be effectively identified and the interference of background motion can be reduced. The kernel correlation filter is used for target tracking, which can adapt to changes in the target's appearance while maintaining efficient calculation, and is particularly suitable for the situation where the target frequently enters and exits and is occluded in the construction site environment. By constructing cyclic shift samples and online update models, the system can adjust the tracking strategy in real time and improve the stability of long-term tracking. The method combining optical flow estimation and kernel correlation filtering not only improves the accuracy of dynamic target recognition and tracking, but also improves the processing efficiency, which is particularly suitable for real-time processing of large-scale construction site monitoring video data.

[0057] According to one aspect of the present application, step S44 is specifically: Step S441, reading each enhanced video segment and the corresponding saliency sequence; calculating the cumulative saliency curve of each video segment, the curve representing the cumulative change of saliency over time; analyzing the inflection point of the cumulative saliency curve, and identifying preliminary key frame candidate positions; Step S442: Use the pre-trained residual network to extract deep features of each frame in the video; based on the extracted deep features, calculate the visual similarity between adjacent frames; construct a frame similarity graph, where each node represents a frame and the weight of the edge represents the similarity between frames; apply a spectral clustering algorithm to the frame similarity graph to segment the video into several scenes; in each scene, select the frame with the highest significance as the representative frame of the scene; Step S443: merge the candidate frames obtained based on the cumulative significance curve and the representative frames obtained based on the scene segmentation; and output the merged comprehensive initial key frame candidate set.

[0058] In this embodiment, the cumulative significance curve is calculated and its inflection point is analyzed, which can effectively identify the moment when the visual information in the video changes significantly, and provides an important basis for the preliminary positioning of key frames. The introduction of the pre-trained residual network to extract deep features can capture high-level semantic information that is difficult to identify with traditional methods, and significantly improve the accuracy of key frame selection. By calculating the visual similarity between frames and constructing a frame similarity graph, the system can fully understand the temporal structure of the video. The application of the spectral clustering algorithm for scene segmentation can adaptively discover the semantic boundaries in the video, so as to better organize and understand the video content. The most significant frame is selected as a representative in each scene, ensuring that the extracted key frame can fully cover the main content of the video. By merging the candidate frames obtained based on cumulative significance and scene segmentation, the system can generate a comprehensive key frame set that takes into account both local significance and global structure.

[0059] According to one aspect of the present application, critical events on a construction site often occur in operations that are not visually significant but have higher risks; conventional significance analysis cannot effectively identify violations of construction regulations; and critical safety information may be obscured by dust, obstructions, or a cluttered background on the construction site. Step S44 may also be: Receive the video clip and the construction area safety level map, calculate the significance sequence of the video clip, perform regional weighting on the significance sequence according to the safety level map, and generate a safety risk weighted significance sequence; Apply the construction site operation specification recognition model to analyze the video clips, identify the operation type and its specification compliance score, and calculate the operation risk weighting factor; Generate a safety-oriented cumulative significance curve based on the safety risk weighted significance sequence and the operational risk weighting factor; According to the safety-oriented cumulative significance curve, a multi-level threshold detection algorithm is used to identify key frames and output key frame candidate sets.

[0060] Among them, the steps of filtering the key frame candidate set based on the construction site knowledge graph are: calculating the construction semantic similarity between candidate frames; removing redundant frames with highly similar construction semantics; retaining key frames that are representative in construction semantics to form the final key frame candidate set.

[0061] Specifically, step S44a, read each enhanced video clip and the corresponding significance sequence, as well as the construction area safety level map and the operation risk database; based on the construction area safety level map, perform regional weighting on the significance sequence to form a safety risk weighted significance sequence S_r(t); Step S44b, identifying standard processes and non-standard operations in the video: Extract the deep features f_t of each frame; Use the pre-trained construction site operation specification recognition model M_op to classify the features and obtain the operation type O_t and specification compliance score C_t; Mark frames with a compliance score lower than a threshold T_c as potential risk frames; Calculate the operational risk weighting factor R_op(t) = β·(1-C_t)·I(O_t), where I(O_t) is the inherent risk index of the operation type and β is an adjustable parameter; Step S44c, generating a safety-oriented cumulative significance curve CSC_safety(t) = ∑_{i=1}^t [α·S_r(i) + (1-α)·R_op(i)]; wherein α is a balance parameter between significance and operational risk, and is dynamically adjusted according to the engineering stage; Step S44d, using a multi-level threshold detection algorithm to identify key frames: Apply adaptive three-layer threshold {T_low, T_med, T_high} on the CSC_safety curve; For high-risk operation areas (critical parts of the project), the T_low threshold is used; For medium-risk areas, the T_med threshold is used; For low-risk areas, the T_high threshold is used; Identify inflection points that meet corresponding thresholds as key frame candidates; Step S44e: using a redundant frame filtering method based on the construction site knowledge graph, calculating the construction semantic similarity between candidate frames, retaining key frames that are representative in construction semantics, and outputting a final key frame candidate set; Weight the significance based on the safety level of different areas of the site to ensure that high-risk areas receive adequate attention even if they are not visually significant. Build a database of standard operating specifications for the site to evaluate the compliance of operating behaviors in real time and identify possible violations, even if these operations are minor. Dynamically adjust thresholds based on regional risk levels and adopt more sensitive detection strategies for key processes and high-risk areas.

[0062] According to one aspect of the present application, dust causes low image contrast and loss of details; alternating strong light and shadows causes uneven exposure; insufficient lighting during nighttime construction causes missing information. Traditional image enhancement methods often use global or fixed parameter processing strategies, which cannot adapt to the complex changes in the construction site environment, resulting in unsatisfactory enhancement effects. Video enhancement processing can also be: Step S11a: Obtain the original video data and simultaneously read the environmental state parameter data at the corresponding moment, including the light intensity L_env, the dust concentration D_env, the humidity H_env, and the weather condition W_env; Step S11b: Construct an adaptive enhancement parameter set based on the environmental state parameters: Calculate the environmental complexity index ECI = f(L_env, D_env, H_env, W_env); Determine the wavelet decomposition level N_level = min(4, 2 + ⌊ECI / 0.3⌋); Calculate the enhancement intensity factor γ = 1.0 + 0.5·D_env + 0.3·(1 - L_env / L_max); Set the defogging intensity δ = min(0.8, D_env·1.2); Step S11c: For each video frame I(x,y), apply environment - adaptive multi - scale enhancement: Apply N_level layer wavelet decomposition: [LL_N, {LH_i, HL_i, HH_i}_{i = 1}^N] = WaveletDecomp(I, N_level); Apply an adaptive enhancement strategy to different frequency sub - bands: Low - frequency sub - band (LL_N): Apply adaptive contrast stretching, F_LL(x,y) = g(ECI)·LL_N(x,y)+(1 - g(ECI))·μ(x,y); Where g(ECI) = 1.2 + 0.3·ECI, and μ(x,y) is the local mean; Medium - frequency sub - bands (LH_{N - 1}, HL_{N - 1}): Apply dust - adaptive enhancement, F_MF(x,y) = MF(x,y)·(1 + α(D_env)·(σ(x,y) / σ_max)); Where α(D_env) = 1.5 + D_env·0.8; High - frequency sub - bands (HH_{i}, i < N - 1): Apply detail - preserving enhancement, F_HF(x,y) = HF(x,y)·(1 + β(L_env)·(1 - exp(-σ(x,y)² / k))); Where β(L_env) is an adaptive parameter based on the light condition; Step S11d: For the detected shadow region S, apply shadow - adaptive brightening: Identify the shaded region S = {(x,y) | I(x,y) <T_shadow(L_env)}; Apply adaptive gamma correction to shadow areas: I_S(x,y) = I(x,y)^(1 / γ_S); Where γ_S = 1.0 + 0.8·(1-L_local / L_max), L_local is the local illumination; Step S11e: For dusty environments, apply the deep learning dehazing network DeHazeNet: Estimated dust transmission map t(x,y) = DeHazeNet(I, D_env); Apply adaptive dehazing strength: I_clear(x,y) = (I(x,y) A·(1-δ·t(x,y))) / max(t(x,y), t_min); where A is the atmospheric light estimate and t_min is the minimum transmittance threshold; Step S11f, fuse the enhancement results of each layer and use weighted reconstruction: I_enhanced = WaveletRecon([F_LL, {w_i·F_LH_i, w_i·F_HL_i, w_i·F_HH_i}]); The weight w_i is adaptively adjusted based on the environmental state parameters and frequency characteristics; Step S11g: applying global tone mapping to the reconstructed image to ensure visual consistency, and splicing the enhanced frames to form an enhanced video stream.

[0063] Dynamically adjust enhancement parameters and strategies based on real-time environmental status parameters (light, dust concentration, humidity, etc.). Automatically determine the number of wavelet decomposition layers according to environmental complexity, and apply targeted enhancement to different frequency sub-bands. Perform specific enhancements on shadow areas, dust areas, etc. to address challenges in construction site environments.

[0064] According to another aspect of the present application, there is also provided an intelligent construction site management system based on photoelectric and video fences, comprising a control module, and an information collection device connected to the control module, the information collection device comprising a camera device, a cable monitoring device, an environmental state parameter sensor, and an intelligent switch for construction equipment; the cable monitoring device is arranged at predetermined lengths along the direction in which the cable extends; wherein the control module comprises: at least one processor; and, a memory communicatively connected to at least one of the processors; wherein, The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the intelligent construction site management method based on photoelectric and video fences as described in any of the above technical solutions.

[0065] Implementation case: The intelligent construction site management system includes a control module and an information collection device connected thereto. The control module is composed of an industrial-grade server, equipped with an Intel Xeon E5-2680 v4 processor, 128GB RAM and 8TB SSD storage, and runs a Linux operating system. The information collection device includes: 16 high-definition network cameras (model: Hikvision DS-2CD5126G0-IZS) were installed around the perimeter and key areas of the construction site, with a resolution of 2688×1520, a frame rate of 30fps, a 120dB wide dynamic range, and support for H.265 encoding. The cameras are connected to the control module via Gigabit Ethernet to achieve real-time transmission of video data.

[0066] Along the direction of cable extension, a group of cable monitoring devices is set up every 50 meters, totaling 32 groups, each group includes: wire and cable fault tester (model: ETCR9500C), accuracy ±2%, measurement range 0-600V; fault sensing probe (model: FD-380A), sensitivity ≤0.1mA; positioning signal sensor (model: DS-500), positioning accuracy ±0.5m; path signal sensor (model: RS-200), maximum detection depth 3m; high energy impact signal generator (model: HSG-800), maximum output voltage 8kV; path signal generator (model: PSG-300), output frequency 512Hz / 1kHz / 8kHz / 33kHz optional. These devices are connected to the data acquisition unit (model: DAQ-2000) through the RS-485 bus, which communicates with the control module through a 4G industrial-grade router to achieve real-time data transmission.

[0067] Various environmental sensors were deployed at key locations on the construction site: 12 sets of temperature and humidity sensors (model: THD-100), with accuracy of temperature ±0.5°C and humidity ±3%RH; 8 sets of dust concentration sensors (model: PM-2000), with a detection range of 0-1000μg / m 3 , accuracy ±10μg / m 3 ; Noise detector (model: SLM-200), 6 groups, detection range 30-130dB, accuracy ±1.5dB; Gas detector (model: MGD-400), 4 groups, can detect CO, H 2 S.O. 2 , combustible gases.

[0068] Each sensor transmits data to the LoRa gateway deployed on the construction site through the LoRaWAN network (working frequency 470MHz), and then transmits it to the control module via the wired network.

[0069] A total of 24 sets of smart switches (model: IS-800) have been installed on major construction equipment (including tower cranes, concrete pump trucks, excavators, etc.). Each set of smart switches has the following functions: equipment start and stop status monitoring; running time statistics; power monitoring (range 0-100kW, accuracy ±0.5%); overload protection; leakage detection (sensitivity 10mA); the smart switches realize real-time data transmission through the NB-IoT network, and the communication frequency is once every 30 seconds.

[0070] This system performs data collection and preprocessing operations every predetermined period T (T is set to 10 seconds), and the specific implementation is as follows: After obtaining the original video data from the camera device, discrete wavelet transform is used for enhancement processing. The specific steps are as follows: Apply a two-dimensional discrete wavelet transform (DWT) to each video frame I(x,y) to obtain four frequency subbands: LL (low frequency), LH (horizontal high frequency), HL (vertical high frequency) and HH (diagonal high frequency): [LL, LH, HL, HH] = DWT(I(x,y)); Calculate local statistical characteristics for each frequency subband: Use a 5×5 sliding window W to calculate the local mean μ and standard deviation σ: μ(i,j) = (1 / 25)∑(x,y)∈WI(x,y); σ(i,j) = sqrt((1 / 25)∑(x,y)∈W (I(x,y) μ(i,j)) 2 ); Construct an adaptive enhancement function F and adopt different enhancement strategies for different sub-bands: Apply contrast stretching to the LL subband: F_LL(x,y) = g·LL(x,y) + (1-g)·μ(x,y); where g is the contrast gain factor, set to 1.2; Apply nonlinear enhancement to LH, HL and HH subbands F_HF(x,y) = HF(x,y)·(1 + α·(σ(x,y) / σ M ax); Where HF represents the high frequency subband (LH, HL or HH), α is the enhancement factor (set to 1.5), σ Max is the maximum standard deviation of the entire subband; The enhanced image is reconstructed using the weighted summation method: I_enhanced(x,y) = IDWT(w_LL·F_LL, w_LH·F_LH, w_HL·F_HL, w_HH·F_HH); where IDWT is the inverse discrete wavelet transform, w_LL=0.6, w_LH=1.2, w_HL=1.2, w_HH=1.0 are the weights of each subband.

[0071] Different from traditional global histogram equalization, by considering the local characteristics and frequency domain information of the image, it can suppress noise amplification while improving contrast, which is particularly suitable for scenes with large lighting changes and much interference in construction sites. By adaptively adjusting the enhancement level of each frequency band, the system can improve overall clarity while retaining details, providing high-quality visual input for subsequent scene analysis.

[0072] The raw cable data obtained by the cable monitoring device include: voltage V(t), current I(t), resistance R(t), temperature T(t) and power factor PF(t). The preprocessing steps are as follows: Data smoothing and denoising: Apply Savitzky-Golay filter to each parameter to remove high-frequency noise: X_smooth(t) = SG_filter(X(t), window_size=11,polynomial_order=3); where X represents any cable parameter.

[0073] Outlier detection and processing: The modified Z-score method is used to identify outliers: Z(t) = |X_smooth(t)median(X_smooth)| / MAD(X_smooth); where MAD is the median absolute deviation. If Z(t)>3.5, X_smooth(t) is replaced by the local median.

[0074] Feature Extraction: Calculates statistical characteristics of each cable parameter, including mean, standard deviation, maximum, minimum, crest factor, and skewness.

[0075] Parameter correlation analysis: Calculate the correlation coefficient matrix C between each parameter, where C ij Represents the Pearson correlation coefficient between parameter i and parameter j.

[0076] Combining statistical feature extraction and parameter correlation analysis, it not only removes noise but also retains the weak signal characteristics of potential faults. Compared with the traditional method that only relies on threshold detection, it can more comprehensively capture the intrinsic relationship between cable parameters.

[0077] The initial environmental state parameters obtained from the environmental state parameter sensor include: temperature Te(t), humidity H(t), PM2.5 concentration P(t), noise intensity N(t) and gas concentration G(t). The preprocessing steps are as follows: Standardization: Convert each environmental state parameter into a unified scale X_norm(t) = (X(t) X M in) / (X M ax M in); Weight calculation: According to the influence of the parameters on construction safety, the weight vector W = [w_Te, w_H, w_P, w_N, w_G] is assigned, where w_Te=0.15, w_H=0.10, w_P=0.20, w_N=0.15, w_G=0.40; Environmental risk index calculation ERI(t) = ∑(w i × X_norm,i(t)); Environmental status model construction: Based on the environmental risk index ERI(t), the construction site environmental status is divided into three categories: safety (S), warning (W) and danger (D), which is implemented through the Bayesian classifier: P(S|ERI) = p(ERI|S)·p(S) / p(ERI); P(W|ERI) = p(ERI|W)·p(W) / p(ERI); P(D|ERI) = p(ERI|D)·p(D) / p(ERI); Environmental status = argmax{P(S|ERI), P(W|ERI), P(D|ERI)}.

[0078] The introduction of a risk-based environmental status assessment model comprehensively considers the impact of multiple environmental factors on construction site safety. Compared with the traditional method of independently processing each environmental status parameter, it can provide a more holistic environmental risk assessment and is closely integrated with the subsequent anomaly detection module, significantly improving the system's ability to identify anomalies caused by environmental factors.

[0079] The working parameters collected from the intelligent switch of construction equipment include: switch state St(t), running time RT(t), power P(t), load ratio LR(t) and leakage current LC(t). The preprocessing steps are as follows: outlier detection: the local outlier factor (LOF) method is used to detect outliers; equipment usage pattern recognition: the k-means clustering algorithm is applied to divide the equipment usage pattern into four categories: no load, normal operation, full load and overload; equipment status risk assessment: combined with the switch parameters and equipment usage pattern, the equipment status risk coefficient DSR(t) = α·St(t) + β·f(RT(t)) + γ·g(P(t)) + δ·h(LR(t)) +ε·j(LC(t)) is calculated; where α=0.1, β=0.2, γ=0.25, δ=0.25, ε=0.2 are weight coefficients, and f, g, h, j are risk mapping functions of each parameter.

[0080] Combining equipment parameter monitoring and usage pattern recognition, the equipment status risk can be assessed more accurately. By establishing the equipment status risk factor DSR, the system can quantify the degree of abnormality of the equipment, providing an important basis for subsequent abnormality detection.

[0081] In order to solve the problem of inconsistent sampling frequency of multi-source data, the multi-dimensional dynamic time warping (MD-DTW) algorithm is used for time alignment: each data source is represented as a time series matrix: X = [x 1 , x 2 , ..., x n ], where x n is the data vector at time point n; construct a multidimensional distance matrix: D(i,j) = ||X i Y j || 2 , where X and Y are two data sources to be aligned; calculate the optimal alignment path: use dynamic programming to solve the minimum cumulative distance matrix; perform time resampling based on the optimal path to align each data source to a unified time standard.

[0082] Compared with traditional linear interpolation methods, it can better handle the nonlinear time deformation problem of multi-source data, and is particularly suitable for processing multi-modal data of construction sites with different sampling frequencies and delays. Through precise time alignment, the system can accurately capture cross-modal abnormal patterns, laying the foundation for subsequent multi-source data fusion analysis.

[0083] The superpixel segmentation of the preprocessed enhanced video is implemented as follows: The SLIC (Simple Linear Iterative Clustering) algorithm is used for initial superpixel segmentation: the image is divided into grids of approximately s×s (s=20 pixels), and the center of each grid is used as the superpixel center; the distance metric D = sqrt((l_p-l_k)) between each pixel p and its surrounding superpixel center k is calculated. 2 + (a_p-a_k) 2 + (b_p-b_k) 2 ) / m 2 + sqrt((x_p-x_k) 2 + (y_p-y_k) 2 ) / s 2 ; Where (l, a, b) is the CIELAB color space coordinate, (x, y) is the pixel position, and m is the balance parameter between color distance and spatial distance (set to 10); Iterate and update the superpixel center and pixel attribution until convergence (set the maximum number of iterations to 10).

[0084] Extract features for each superpixel region: Color features: Calculate the histogram H_color (16-dimensional vector) of the Lab color space; Texture features: Using SURF (Speeded-Up Robust Features) algorithm; Calculate the Haar wavelet responses dx and dy (scales are set to 1.2, 2.4, 3.6, 4.8); construct the directional gradient histogram H_grad (8-dimensional vector); calculate the gradient variance σ_dx, σ_dy and covariance σ_dxdy; Shape features: Calculate the shape descriptor F_shape, including area ratio, contour complexity, directionality, compactness, etc. (6-dimensional vector); Construct a comprehensive feature vector F = [w_c·H_color, w_t·H_grad, w_t·[σ_dx, σ_dy, σ_dxdy], w_s·F_shape]; where w_c=0.4, w_t=0.4, w_s=0.2 are the weights of each feature; Compared with the original SURF, the analysis of multi-scale Haar wavelet response is added, which can more effectively capture the texture features in the construction site environment. The comprehensive feature vector F integrates color, texture and shape information, provides a more comprehensive regional description, and provides rich visual features for subsequent scene segmentation.

[0085] Hierarchical clustering based on superpixel feature vectors to achieve preliminary scene segmentation: Calculate the distance matrix between feature vectors M(i,j) = d(F i , F j ) = sqrt(∑(w_k·(F i ,k F j ,k) 2 )); where w_k is the weight of the k-th dimension feature; The Ward minimum variance method is used to construct a hierarchical tree: Initialization: Each superpixel is treated as an independent cluster; Iteration: Each time the two clusters with the smallest variance are merged until the preset number of clusters or variance threshold is reached; The hierarchical tree is cut according to the preset threshold τ_cluster (set to 0.3) to obtain the initial scene segmentation result.

[0086] Compared with traditional k-means, it does not require the number of clusters to be specified in advance, can adaptively discover the hierarchical structure of data, and is more suitable for dealing with multi-category area segmentation problems in complex construction site scenes. By adjusting the cutting threshold τ_cluster, the granularity of segmentation can be flexibly controlled to meet different analysis needs.

[0087] In order to improve the spatiotemporal consistency of scene segmentation, the spatiotemporal Markov random field (ST-MRF) model is used for optimization: Construct energy function E(L) = ∑E_data(L i) + λ_s·∑E_smooth(L i , L j ) + λ_t·∑E_temp(L i ,t, L i ,t-1); where L is the label configuration, E_data is the data term, E_smooth is the spatial smoothing term, E_temp is the temporal consistency term, λ_s=0.6 and λ_t=0.4 are weight coefficients; Data item calculation: Gaussian mixture model (GMM) is used to estimate the probability E_data(L i ) = -log(P(I i |L i )); where P(I i |L i ) is given by GMM, using 3 Gaussian components for each label; Spatial smoothing term calculation: E_smooth(L i , L j ) = exp(-β·||I i I j || 2 )·δ(L i ≠ L j ), where β is the color difference coefficient (set to 0.1), and δ(·) is the indicative function; Time consistency term calculation: E_temp(L i ,t, L i ,t-1) = w_t·δ(L i ,t ≠ L i ,t-1); where w_t is the time penalty weight, which is adaptively adjusted according to the difference between frames.

[0088] The α-expansion graph cut algorithm is used to minimize the energy function and obtain the optimized scene segmentation result.

[0089] The spatial consistency and temporal consistency constraints are unified into an energy optimization framework, overcoming the shortcomings of the traditional scene segmentation method, which is temporal instability. By considering the spatial and temporal relationship, this method can effectively reduce the segmentation jitter caused by factors such as illumination changes and camera shake, improve the stability and consistency of the segmentation results, and provide a reliable basis for subsequent dynamic region recognition.

[0090] Based on the scene segmentation results that are consistent in time and space, an improved optical flow estimation method is used to identify dynamic areas: The Farneback algorithm is used to calculate the pixel-level dense optical flow field: for each pixel (x, y), the displacement vector (u, v) is estimated between adjacent frames; the energy function E(u, v) = ∑(I_1(x, y) I_2(x+u, y+v)) is constructed. 2 + λ·(||grad u|| 2 + ||grad v|| 2 ), where I_1 and I_2 are adjacent frames, and λ is the weight of the smoothing term (set to 0.05); The energy function is solved by using the variational method and multi-resolution strategy to obtain the optical flow field (u, v); Calculate the optical flow magnitude and direction histogram: Optical flow magnitude M(x,y) = sqrt(u(x,y) 2 + v(x,y) 2 ); Optical flow direction θ(x,y) = arctan(v(x,y) / u(x,y)); Construct 8-direction histogram H_flow; Adaptive threshold segmentation of motion salient areas: Calculate the global mean μ of the optical flow amplitude M and standard deviation σ M ; Determine the adaptive threshold T Motion = μ M + k·σ M , where k is a coefficient (set to 2.0); if M(x,y)>T Motion , it is marked as a motion salient point; Morphological processing is used to optimize dynamic regions: an opening operation is applied to eliminate noise (the structural element is a 3×3 rectangle); a closing operation is applied to fill holes (the structural element is a 5×5 rectangle); and connected domain analysis is performed to remove regions with an area smaller than the threshold T_area (100 pixels).

[0091] Based on the traditional Farneback algorithm, a multi-resolution strategy and variational optimization framework are added to more accurately handle large displacements and deformations. The adaptive threshold mechanism dynamically adjusts the segmentation parameters according to the statistical characteristics of optical flow to adapt to the changes in motion patterns in different construction site scenes. Morphological post-processing further optimizes the boundaries and connectivity of dynamic areas, improves the stability of recognition results, and provides high-quality initial areas for subsequent target tracking.

[0092] For the identified dynamic area, the kernel correlation filter (KCF) algorithm is used for target tracking: Target model initialization: extract HOG features x of dynamic areas; construct ideal response output y (Gaussian shape, 1 at the center and 0 attenuation around); Compute the filter w = F in the Fourier domain -¹(F(y) OF(x)* / (F(x) OF(x)* + λ)) where F is the Fourier transform, O is the element-wise product, * is the conjugate, and λ is the regularization parameter (set to 0.001). Target position update: extract HOG feature z from the search area of ​​subsequent frames; Calculate the response graph r = F -1 (F(w) OF(z)); find the maximum response point as the new target position; Model update: extract HOG features x at the new position new ; Update the filter w using the learning rate η new = (1-η)·w +η·w new ; where η is set to 0.02, w new Based on x new The new filter is calculated; Assign a unique ID to each tracked target and record its trajectory information; It has the characteristics of high computational efficiency and strong robustness, and is particularly suitable for the needs of real-time tracking of multiple targets in construction sites. By performing fast calculations in the Fourier domain, it can effectively handle changes in target appearance and partial occlusion, providing technical support for long-term stable tracking. The cyclic shift sample strategy and online model update mechanism enable the tracker to adapt to gradual changes in target appearance, improving the tracking performance of the system in complex construction sites.

[0093] Perform empirical mode decomposition (EMD) on the cable operation data to decompose the intrinsic mode function (IMF): identify all local extreme points in the cable time series X(t); generate the upper envelope curve e by cubic spline interpolation method Max (t) and the lower envelope e Min (t); calculate the mean envelope m(t) = (e Max (t) + e Min (t)) / 2; calculate the difference h(t) = X(t) m(t); check whether h(t) satisfies the IMF condition: the difference between the number of extreme points and the number of zero crossings does not exceed 1; the local mean is approximately zero; if so, h(t) is an IMF component; calculate the residual signal r(t) = X(t) h(t), and repeat steps 1-5 with r(t) as the new signal until r(t) becomes a monotonic function; empirical mode decomposition can separate the different frequency components in X(t) X(t) = ∑c i (t) +r_n(t); where c i (t) is the IMF component, r_n(t) is the residual trend; Different from traditional Fourier analysis, it can handle nonlinear and non-stationary signals, and is particularly suitable for analyzing complex time-varying signals such as cable parameters. By adaptively decomposing the signal into multiple intrinsic mode functions, this method can effectively extract cable characteristics on different time scales, providing fine-grained frequency information for abnormal pattern recognition.

[0094] Perform Hilbert transform on each IMF component and analyze its instantaneous characteristics: For each IMF component c i (t) Calculate the Hilbert transform H[c i (t)] = (1 / π)∫(c i (τ) / (t-τ))dτ; Construct the analytical signal z i (t) = c i (t) + j·H[c i (t)] = a i (t)·exp(j·φ i (t)); where a i (t) is the instantaneous amplitude, φ i (t) is the instantaneous phase; Calculate the instantaneous frequency ω i (t) = dφ i (t) / dt; construct the Hilbert spectrum HS(ω,t) = ∑a i (t)·δ(ω-ω i (t)); where δ is the Dirac function.

[0095] Combining EMD decomposition and Hilbert transform, it can provide a time-frequency distribution diagram of the signal, intuitively showing the frequency changes of cable parameters over time. By calculating the instantaneous frequency and amplitude, this method can accurately capture the transient changes in cable parameters, which is particularly suitable for detecting transient faults such as broken cores, and significantly improves the system's ability to identify potential faults at an early stage.

[0096] Based on the Hilbert spectrum analysis results, a multidimensional phase space is constructed and density clustering is applied: Select characteristic frequency components for each cable parameter (voltage, current, resistance, etc.), generally select the instantaneous frequency and amplitude corresponding to the first three IMF components; construct a multidimensional phase space P, where each dimension represents a characteristic component: P = [ω_V1, a_V1, ω_V2, a_V2, ω_V3, a_V3, ω i1 , a i1 , ..., ω_R3, a_R3]; Where ω_Vn and a_Vn are the instantaneous frequency and amplitude of the nth IMF component of the voltage, respectively. Perform DBSCAN density clustering in P space: set parameters: ε=0.15 (neighborhood radius), MinPts=5 (minimum number of points); for each point p, calculate the number of points in its ε-neighborhood N_ε(p); if |N_ε(p)| ≥ MinPts, then p is a core point; recursively add all density-reachable points to the same cluster; unassigned points are marked as noise points (potential outliers); Analyze the characteristics of abnormal points: calculate the Mahalanobis distance between the abnormal points and each normal cluster; extract the original cable parameter values ​​corresponding to the abnormal points; match them according to the preset abnormal pattern template library to determine the abnormal type.

[0097] The time-frequency characteristics of cable parameters are mapped into high-dimensional space, so that normal operation modes form dense clusters, while abnormal modes appear as outliers. Compared with traditional single-variable threshold detection, it can capture the complex correlation between parameters and significantly improve the sensitivity and accuracy of anomaly detection. Especially for hidden faults such as broken cores that are difficult to detect in traditional monitoring, this method can issue early warnings by analyzing the abnormal trajectories of cable parameters in phase space, providing important guarantees for construction site safety management.

[0098] In order to further improve the stability of anomaly detection, the sliding window singular spectrum analysis (SSA) method is used: Construct a multidimensional cable parameter time series matrix X, where each row represents a parameter and each column represents a time point; For a sliding window W of length L, construct the trajectory matrix A = [X t , X t+1 , ..., X t+K- 1]; where K = W-L+1, X t is the parameter vector at time t; Perform singular value decomposition (SVD) on the trajectory matrix A: A = UΣV^T; where Σ is a singular value diagonal matrix, U and V are orthogonal matrices; select the components corresponding to the first N largest singular values ​​to reconstruct the signal: A*= ∑ i=1 N σ i u i v i T ; where σ i is the i-th singular value, u i and v i are the corresponding left and right singular vectors; calculate the reconstruction error e_t = ||X_t – A*_t|| 2; Adaptive threshold calculation T_t = μ_e + γ·σ_e; where μ_e and σ_e are the moving average and standard deviation of the reconstruction error, respectively, and γ is the sensitivity parameter (set to 3.0); if e_t>T_t, it is marked as a potential outlier.

[0099] It can capture the main patterns and structural changes in multivariate time series. By extracting the main components of the signal and monitoring the reconstruction error, it can effectively detect abnormal changes in cable parameters and maintain high sensitivity even in the presence of noise and interference. The adaptive threshold mechanism dynamically adjusts the judgment criteria according to the statistical characteristics of the signal, allowing the system to adapt to the cable operation status under different working conditions, reducing the false alarm rate while improving the detection rate.

[0100] The potential outliers detected by the SSA method are further verified by applying the local outlier factor (LOF) analysis: For each potential outlier point p, calculate its k nearest neighbor distance d_k(p), where k is set to 5; Calculate the reachable distance reach-dist_k(p,o) = max{d_k(o), dist(p,o)}; where o is the neighbor point of p, dist(p,o) is the Euclidean distance between the two points; calculate the local reachable density lrd_k(p) = 1 / (∑_{o∈N_k(p)}reach-dist_k(p,o) / |N_k(p)|); where N_k(p) is the set of k nearest neighbors of p.

[0101] Calculate the local outlier factor LOF_k(p) = ∑_{o∈N_k(p)} (lrd_k(o) / lrd_k(p)) / |N_k(p)|; if LOF_k(p)>T_LOF (the threshold is set to 1.5), it is confirmed as an outlier point; By comparing the local density of a data point with its neighbors, local outliers in different density areas can be effectively detected. Compared with global anomaly detection methods, LOF is particularly suitable for processing cable parameter data with multi-mode distribution, and can identify abnormal patterns that deviate significantly in local areas. By combining LOF with the SSA method, the system implements a two-level verification mechanism, significantly reducing the false alarm rate while maintaining sensitivity to minor anomalies, providing reliable protection for cable safety monitoring on construction sites.

[0102] In order to further understand the cause of the anomaly, cluster analysis is performed on the detected anomalies: The improved k-means++ algorithm is used to cluster the outliers: k initial center points are selected and a distance-weighted probability selection strategy is used; standard k-means iterative optimization is applied; the silhouette coefficient is used to automatically determine the optimal number of clusters; Use the random forest decision tree model to analyze feature importance: construct a training set: normal samples (label 0) and various abnormal samples (label 1, 2, ...); train a random forest model (100 trees, maximum depth 12); calculate the feature importance score based on Gini impurity.

[0103] Generate descriptive labels for each anomaly cluster based on feature importance, for example: "voltage transient anomaly - mainly affects IMF2 frequency component"; "current-temperature coupling anomaly - related to ambient temperature"; The dynamic time warping (DTW) algorithm is used to calculate the similarity between abnormal and normal patterns: DTW(X,Y) = min{∑w i d(x_{i1},y_{i2})}; where w i is the path weight, d is the distance function; based on the DTW distance, a severity level (1-5) is assigned to each abnormal cluster.

[0104] It can not only detect anomalies, but also automatically classify and explain the causes of anomalies. Through random forest feature importance analysis, the system can identify the key parameters and frequency components that cause anomalies, providing intuitive and understandable anomaly descriptions for construction site managers. The dynamic time warping algorithm can quantify the degree of difference between anomalies and normal patterns, providing an objective basis for anomaly severity assessment. This intelligent anomaly interpretation mechanism significantly improves the interpretability and practicality of the system, allowing non-professionals to understand complex cable anomaly patterns, and providing decision support for timely and targeted maintenance measures.

[0105] Fusion of cable anomaly detection results with equipment status and environmental status parameters: Construct an abnormal feature vector F_anomaly = [E_cable, DSR, ERI, Loc, T]; where E_cable is the cable abnormality parameter, DSR is the equipment status risk, ERI is the environmental risk index, Loc is the location information, and T is the timestamp; Calculate the correlation matrix C of each anomaly source using the Spearman rank correlation coefficient: C ij = ρ(F i , F j ) =cov(rg i , rg j ) / (σ_{rg i}·σ_{rg j}); where rg i is the hierarchical sequence of feature i. Anomaly correlation analysis is performed based on the correlation matrix to identify potential causal relationships.

[0106] A comprehensive anomaly feature vector is constructed by combining cable parameters, equipment status, and environmental factors. By analyzing the correlation and causal relationship between anomaly features, the system can more accurately understand the root cause and propagation path of the anomaly, providing a multi-dimensional information basis for anomaly severity assessment. For example, the system can identify the difference between cable parameter fluctuations caused by ambient temperature changes and real cable faults, significantly reducing the false alarm rate.

[0107] Use fuzzy inference system to evaluate the severity of abnormalities: Define input fuzzy variables: Cable abnormality degree (E_cable): {low, medium, high}; Device status risk (DSR): {safe, warning, dangerous}; Environmental risk index (ERI): {normal, warning, severe}; Duration (Duration): {short, medium, long}; Define output fuzzy variables: Abnormal severity (Severity): {minor, general, serious, critical, catastrophic}; A fuzzy rule base was established, including 25 rules, such as: if (E_cable=high) and (DSR=dangerous) and (ERI=bad) and (Duration=long), then (Severity=disaster); if (E_cable=medium) and (DSR=warning) and (ERI=normal) and (Duration=short), then (Severity=normal); Apply Mamdani fuzzy reasoning: Map inputs to fuzzy sets; Apply fuzzy operators (min for AND, max for OR); Combine the outputs of all rules; Calculate output values ​​using the centroid method; Compare the defuzzified result with the threshold to determine the anomaly level.

[0108] It can handle the uncertainty and ambiguity in anomaly assessment, which is more in line with the decision-making thinking of human experts. Unlike traditional hard threshold judgment, fuzzy reasoning can comprehensively consider the combined impact of multiple factors on the severity of anomalies by simulating the human reasoning process, and make a more reasonable assessment. Especially in complex and changeable environments such as construction sites, fuzzy reasoning systems show better adaptability and robustness, providing more accurate priority judgments for anomaly handling, helping managers to reasonably allocate resources and give priority to high-risk anomalies.

[0109] Accurately locate the abnormalities that are evaluated as serious in time and space: record the exact timestamp T of the abnormality anomaly ; According to the location information of the cable monitoring device, determine the preliminary spatial coordinates (x, y, z) where the abnormality occurs initial ; Precise positioning process: high-precision scanning within ±5 meters around the preliminary coordinates; applying the principle of triangulation and combining multi-point signal strength to optimize the position; calculating the final abnormal coordinates (x, y, z) final , the accuracy is improved to ±0.5 meters; the abnormal coordinates are mapped to the 3D model of the construction site to generate visual abnormal points.

[0110] Combining timestamp recording and spatial coordinate positioning, it is possible to accurately determine the time and location of anomalies. Through high-precision scanning and multi-point triangulation optimization, the positioning accuracy is significantly improved, which is particularly suitable for locating hidden fault points in cables. The visualization function of mapping abnormal points to the 3D model of the construction site intuitively displays the spatial distribution of abnormalities, providing strong support for maintenance personnel to quickly locate faults and significantly improving troubleshooting efficiency.

[0111] Based on the anomaly timestamp, relevant segments are extracted from the stored video data: Extraction time range: [T anomaly Δt before , T anomaly + Δt after ]; where Δt before =30 seconds, Δt after = 60 seconds; Adaptively enhancing the contrast of the extracted video clips, including: calculating a brightness histogram of each frame of the video clips; The CLAHE (Contrast Limited Adaptive Histogram Equalization) algorithm was applied to the image, which included: dividing the image into 8×8 grids; performing histogram equalization on each grid independently with a contrast limit threshold of 3.0; merging grid boundaries using bilinear interpolation; Applying video stabilization algorithms to reduce jitter, including: calculating feature point matches between consecutive frames; estimating the global motion model (affine transformation); applying exponential smoothing filters to smooth the transformation parameters; re-transforming each frame based on the smoothed parameters; Compared with traditional global histogram equalization, it can maintain local details and edge information while suppressing noise amplification, which is particularly suitable for processing videos with uneven lighting in construction sites. The video stabilization algorithm improves video quality and the accuracy of subsequent analysis by smoothing camera jitter. This video enhancement method provides high-quality input data for key frame extraction and target recognition, significantly improving the system's visual analysis capabilities in harsh environments.

[0112] Based on the enhanced video clips, calculate the motion intensity map and analyze the visual saliency: Calculate the inter-frame motion vector field: Use the Farneback optical flow algorithm to calculate the pixel-level motion vector (u, v); calculate the motion amplitude matrix M: M(x, y) = sqrt(u(x, y)2 + v(x,y) 2 ); Generate motion intensity map MI: For each pixel position (x, y), calculate the temporal cumulative motion intensity MI(x, y) = ∑_t w_t·M_t(x, y); where w_t is the temporal weight, with recent frames having a higher weight; Computing Visual Saliency Maps VS: Calculate multi-feature saliency using the Itti-Koch model: extract brightness, color, and orientation features; construct a 9-scale Gaussian pyramid; calculate center-periphery differences; combine and normalize across scales; By fusing motion intensity and visual saliency, we obtain the multi-feature fusion saliency map FS = α·norm(MI) + (1-α)·norm(VS); where α=0.7 is the motion saliency weight and norm is the normalization function.

[0113] It combines dynamic information and static visual features to fully capture important content in the video. Compared with traditional single-feature saliency analysis, it can more accurately identify key targets and events in the construction site environment, and maintain high performance even under complex backgrounds and changing lighting conditions. By generating high-quality saliency maps, the system provides a reliable basis for subsequent key frame selection, improving the efficiency and accuracy of video analysis.

[0114] Based on the saliency sequence, calculate the temporal saliency and select keyframes: Calculate the cumulative significance curve CSC(t) = ∑ i=1 t mean(FS i ); where mean(FS i ) is the average value of the saliency map of the i-th frame.

[0115] Analyze the inflection point of the CSC curve and use the second-order derivative method to detect significant change points: d 2 CSC(t) / dt 2 = (CSC(t+1)2·CSC(t) + CSC(t-1)); if |d 2 CSC(t) / dt 2 |>T i nflection (the threshold is set to 0.05), then t is the inflection point and the corresponding frame is the preliminary candidate frame.

[0116] Deep feature extraction and scene segmentation: Use the pre-trained ResNet-50 network to extract the deep features of each frame f_t; calculate the inter-frame similarity matrix S: S(i,j) = cos_sim(f i , f j); Apply spectral clustering algorithm to segment the video into K scenes (K is automatically determined by feature vector gap method); Select the most significant frame as the representative frame in each scene; Merge the candidate frames obtained based on inflection point and scene segmentation, and output the comprehensive key frame candidate set.

[0117] Taking into account the temporal structure and semantic content of the video, the most representative keyframes can be selected. Cumulative saliency curve analysis captures significant change points in the video, while scene segmentation based on deep features identifies semantic boundaries. The combination of the two ensures that the keyframes contain both important events and cover the main content of the video. In particular, the introduction of a pre-trained ResNet-50 network to extract deep features enables the system to understand the high-level semantics of the construction site scene, improves the quality and representativeness of keyframe selection, and provides refined visual data for subsequent target recognition and tracking.

[0118] Based on the key frame candidate set, the improved YOLOv5 algorithm is used for target detection: YOLOv5 model configuration: the backbone network is CSPDarknet53, the input resolution is 640×640; the feature pyramid is the PANet structure, and multi-scale feature fusion; the anchor box size is 9 prior boxes obtained by adaptive clustering; the loss function is CIoU loss for bounding box regression, and BCE loss for classification.

[0119] Special optimization of construction site target detection: adding simulation of common environmental interferences on construction sites such as low light, rain, fog, and dust; using Focal Loss to alleviate the problem of category imbalance; adding additional small-scale detection heads; Target recognition process: Apply the YOLOv5 model to each keyframe to obtain detection results (category, confidence, bounding box); apply the non-maximum suppression (NMS) algorithm to merge overlapping boxes, and set the IoU threshold to 0.5; filter low-confidence detection results (threshold is 0.4); output the target list, including common objects on the construction site such as personnel, vehicles, and equipment; Cross-frame target matching and trajectory generation: Use SORT (Simple Online and Realtime Tracking) algorithm for target tracking; use Kalman filter to predict target position; use Hungarian algorithm for data association based on IoU distance; handle target appearance and disappearance (maximum disappearance frame number is set to 30); generate complete trajectory data for each target; Special optimizations have been made for construction site environments, including innovative improvements such as data enhancement, category balance, and small target enhancement, which significantly improve the target detection performance in complex construction site environments. In particular, the addition of a data enhancement strategy that simulates common environmental interference on construction sites makes the model more robust and able to maintain good performance under harsh conditions such as low light and dust. The SORT tracking algorithm combines Kalman filter prediction and Hungarian algorithm data association to maintain stable tracking effects in situations such as short-term occlusion and rapid movement of the target, providing reliable trajectory data for the identification of responsible entities.

[0120] Combine the target trajectory and abnormal information to determine the potential responsible party: For each tracking target i, calculate its position p at the abnormal time t_anomaly i (t_anomaly); Calculate the spatial distance d between the target position and the anomaly position i = ||p i (t_anomaly) p_anomaly||; Calculate the minimum distance d between the target and the abnormal area in the time window [t_anomaly-Δt, t_anomaly] M in,i; Behavior analysis: extract the target’s motion characteristics (speed, acceleration, direction change) before the anomaly; calculate the behavior anomaly score s_behavior,i, based on the deviation from the normal behavior pattern; analyze the interaction pattern of the target with the device / cable; Calculate the responsibility score i = w d exp(-d Min,i / σ d ) + w b s_behavior,i + w i ·s identity,i ; where w d =0.5, w b =0.3, w i =0.2 is the weight coefficient, s identity,i It is a priori score based on the target identity; the targets are sorted according to the responsibility score to generate a list of potential responsible entities.

[0121] Combining spatiotemporal correlation analysis and behavioral analysis, it can comprehensively consider the spatial proximity, temporal correlation, and behavioral abnormality between the target and the anomaly, and achieve more accurate responsibility judgment. In particular, the behavioral anomaly score is introduced. By analyzing the target's movement pattern before the anomaly and the way it interacts with the equipment / cable, the system can identify abnormal behaviors that may cause anomalies. The responsibility score calculation formula combines multiple factors and reasonably models the influence of spatial distance through an exponential decay function, making the responsibility judgment more objective and reasonable. This intelligent responsibility tracking mechanism provides site managers with an objective basis for accident responsibility analysis, which helps to improve the pertinence and effectiveness of safety management.

[0122] After the system was actually deployed at a high-rise construction site, a three-month comparative test was conducted with the traditional monitoring and cable fault detection system. The system performance indicators are as follows: Cable core breakage fault detection: detection rate is 95.8% (traditional system: 72.3%); false alarm rate is 4.2% (traditional system: 15.7%); average advance warning time is 4.6 hours (traditional system: no warning capability); positioning accuracy is ±0.65 meters (traditional system: ±3.2 meters); Object detection and tracking: Person detection accuracy is 94.2% (traditional system: 85.1%); Equipment detection accuracy is 92.7% (traditional system: 79.8%); Tracking stability (MOTA) is 83.5% (traditional system: 61.2%); Cross-camera tracking accuracy is 78.9% (traditional system: 45.3%); Abnormal event response: The average event detection time is 1.8 seconds (traditional system: 15.6 seconds); the accuracy rate of responsible party identification is 89.3% (traditional system: no this function); the overall system reliability is 99.2% (operating time ratio).

[0123] Through the actual application of this system, the accident rate on construction sites has been reduced by 47.8%, the equipment maintenance cost has been reduced by 32.5%, and the workers' safety awareness and sense of responsibility have been significantly improved. In particular, the system's responsible subject identification function makes the tracking of construction site responsibilities more accurate and fair, greatly reducing liability disputes and improving management efficiency.

[0124] By integrating cable monitoring with video analysis technology, and through the collaborative processing of multi-source data, timely detection and precise positioning of hidden faults such as broken cores are achieved. The cable anomaly detection method based on empirical mode decomposition and Hilbert spectrum analysis can capture small changes in cable parameters and significantly improve the fault warning capability. The abnormal pattern recognition method combining multi-dimensional phase space and density clustering can effectively distinguish different types of cable anomalies and provide targeted guidance for maintenance. The dynamic area recognition algorithm combining spatiotemporal Markov random fields and improved optical flow estimation significantly improves the target recognition and tracking performance in complex construction site environments. The introduction of multi-feature fusion significance analysis and deep feature scene segmentation technology realizes high-quality key frame extraction, providing support for the rapid positioning and analysis of abnormal events. The responsible subject identification mechanism based on spatiotemporal association and behavior analysis realizes the automatic tracking of the responsibility of abnormal events, providing an objective basis for construction site safety management.

[0125] It should be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present invention will not further describe various possible combinations.

Claims

1. An intelligent construction site management method based on photoelectric and video fences, characterized in that: The steps include: Step S1, acquiring real-time data of the construction site at every predetermined period, and preprocessing it to form a preprocessed data set, including enhanced video data, cable operation data, environmental state parameters and equipment switch status; Step S2, read enhanced video data, identify and obtain the current construction site status, obtain and mark the dynamic area, and form a dynamic area data set; Step S3, read the cable operation data and the equipment switch status, extract the cable abnormal parameters, evaluate the severity of the abnormality, output the abnormality level index and compare it with the threshold value, if it exceeds the threshold value, record the timestamp of the abnormality occurrence time and the corresponding detection area coordinates; Step S4, according to the timestamp of the abnormality occurrence time, retrieve a video segment of a predetermined length from the dynamic area data set, determine the key frame position range and form a key frame candidate set; Step S5: Identify the tracking target based on the key frame candidate set and match it across frames to generate a target trajectory, and obtain a list of responsible entities based on the target trajectory.

2. The method according to claim 1, characterized in that Step S1 is specifically as follows: Step S11, every predetermined period, obtaining the original video data of the construction site and performing enhancement processing to obtain enhanced video data; Step S12, acquiring cable monitoring data and performing modal decomposition to form original cable operation data, including normal cable mode, abnormal cable mode and cable parameter values; Step S13, obtaining the initial environmental state parameters of the environmental state parameter sensor, and performing fusion processing to obtain the environmental state parameters; Step S14: collecting working parameters of the intelligent switch of the construction equipment, constructing original equipment data, and performing preprocessing to form the equipment switch state; Step S15: Based on the output data of steps S11 to S14, a multimodal data set is constructed, and time alignment is performed through a multidimensional dynamic time warping algorithm to form a preprocessed data set.

3. The method according to claim 2, characterized in that Step S2 is specifically as follows: Step S21, read enhanced video data, extract video frames, perform region division, extract feature vectors and clustering for each video frame, and obtain an initial scene segmentation result; Step S22: Optimize the initial scene segmentation result using a spatiotemporal Markov random field module and minimize the energy function to obtain a spatiotemporal consistent scene segmentation result; Step S23, based on the spatiotemporal consistent scene segmentation result, calculate the motion vector field, identify the dynamic area, and segment the motion significant area; track each motion of the motion significant area to obtain the dynamic area tracking result, that is, the dynamic area data set; Step S24: annotate and hierarchically represent the dynamic region tracking results, construct and output a comprehensive dynamic region description data set.

4. The method according to claim 3, characterized in that Step S3 is specifically as follows: Step S31, obtaining a comprehensive dynamic region description data set, performing multivariate anomaly detection and dimensionality reduction, and obtaining a processed multi-source synchronous data set; Step S32, reading the cable operation data, and preliminarily detecting whether there is an abnormal pattern; based on the processed multi-source synchronous data set, calculating and outputting a cable abnormal parameter set; Step S33, fusing the cable abnormal parameter set with the equipment switch state and environmental state parameters in the multi-source synchronous data set, and outputting the fused abnormal feature data set, i.e., the abnormality detection result; Step S34, calling a preconfigured fuzzy inference system to evaluate each data point of the abnormality detection result to obtain an abnormality severity evaluation result; Step S35, record the timestamp and coordinate information of the abnormality severity assessment result, and output the spatiotemporal positioning result.

5. The method according to claim 4, characterized in that Step S4 is specifically as follows: Step S41, reading the dynamic area data set and the anomaly detection result, and extracting the corresponding video clip from the dynamic area data set; Step S42, optimizing the video segments again and outputting an optimized video segment set; Step S43: for each optimized video clip set, obtain and calculate motion saliency and visual saliency according to the corresponding motion analysis results, and output a saliency sequence for each video clip; Step S44: Based on the optimized video clip set, calculate the temporal saliency, obtain the potential key frame position range, determine the potential key frame, and output the key frame candidate set.

6. The method according to claim 5, characterized in that Step S5 is specifically as follows: Step S51, read the key frame data set, the dynamic area data set, the anomaly detection result and the pre-configured registration database; use the YOLOvx target detection algorithm for identification and tracking; obtain the target list; Step S52, performing cross-frame matching on the target results one by one; generating target trajectory data to form a target trajectory; Step S53: Combine the target trajectory and use the spatiotemporal correlation analysis algorithm to determine the target closest to the detection area at the time when the anomaly occurs, and output a list of potential responsible entities.

7. The method according to claim 2, characterized in that In step S11, the process of enhancing the original video data of the construction site is specifically as follows: Step S111, obtaining original video data of the construction site, extracting video frame images, and decomposing each video frame image by discrete wavelet transform to obtain four frequency sub-bands; Step S112: for each frequency sub-band, calculate the local mean and standard deviation of the frequency sub-band; construct an enhancement function based on the local mean, standard deviation and pre-stored adjustable parameters; For each pixel in the frequency subband, an enhancement function is applied; Step S113: Reconstruct the enhanced frequency subbands using a weighted summation method to obtain enhanced images, and splice the enhanced images to form enhanced video data.

8. The method according to claim 3, characterized in that Step S21 is specifically as follows: Step S211, read and divide each frame of the enhanced video data into several initial regions, set the region center, iteratively calculate the distance between each pixel and the superpixel center, update the pixel attribution and superpixel center, until convergence or reaching the maximum number of iterations, and obtain the superpixel segmentation result; Step S212: for each superpixel region in the superpixel segmentation result, calculate the color histogram, use the improved SURF algorithm to extract texture features, calculate the shape descriptor, and combine them into a comprehensive feature vector; Step S213: cluster all comprehensive feature vectors, calculate the distance matrix between feature vectors, construct a hierarchical tree, cut the feature hierarchical tree according to a preset threshold, and obtain an initial scene segmentation result.

9. The method according to claim 3, characterized in that Step S22 is specifically as follows: Step S221, reading the initial scene segmentation results of several consecutive frames, constructing a spatiotemporal Markov random field model, including an energy function of data terms and smoothing terms; Step S222: using a Gaussian mixture model to estimate the probability that a pixel belongs to each label, and calculating the data item in the energy function; Step S223, calculating weights based on the color similarity and spatial distance of adjacent pixels to obtain a smoothing term in the energy function; Step S224: Apply the graph cut algorithm to minimize the energy function to obtain an optimized scene segmentation label, and generate and output a spatiotemporally consistent scene segmentation result based on the scene segmentation label.

10. The method according to claim 5, characterized in that Step S44 is specifically as follows: Step S441, reading each optimized video segment and the corresponding saliency sequence; calculating the cumulative saliency curve of each video segment; analyzing the inflection point of the cumulative saliency curve, and identifying preliminary key frame candidate positions; Step S442: using a pre-trained residual network, extracting deep features of each frame in the video; calculating the visual similarity between adjacent frames based on the extracted deep features; constructing a frame similarity graph; applying a spectral clustering algorithm to the frame similarity graph to segment the video into several scenes; in each scene, selecting the frame with the highest significance as the representative frame of the scene; Step S443: merge the candidate frames obtained based on the cumulative significance curve and the representative frames obtained based on the scene segmentation; and output the merged comprehensive initial key frame candidate set.

11. An intelligent construction site management system based on photoelectric and video fences, characterized in that: It includes a control module, and an information collection device connected to the control module, the information collection device includes a camera device, a cable monitoring device, an environmental state parameter sensor, and an intelligent switch for construction equipment; The cable monitoring device is arranged at predetermined lengths along the direction in which the cable extends; wherein the control module comprises: at least one processor; and, a memory communicatively connected to at least one of the processors; wherein, The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the intelligent construction site management method based on photoelectric and video fences as described in any one of claims 1 to 10.

Images