Pipe network periphery construction dynamic monitoring method based on multi-target visual identification

Abstract

The application discloses a pipe network periphery construction dynamic monitoring method based on multi-target visual identification, relates to the technical field of intelligent monitoring intersection, and comprises the following steps: integrating and analyzing the environment of a construction site and a future moving path, identifying potential construction damage risks, and generating a potential risk list through classified processing; classifying and sorting the potential risk list according to severity and urgency, and generating a pipe network monitoring report through an integrated data analysis method. Convolution, feature transformation and time sliding are used to extract visual features, terrain features and temperature distribution features and to realize weighted fusion, so that unified representation of multi-source heterogeneous data is realized, and the comprehensive perception result can comprehensively reflect the static structure and dynamic change of a construction scene.

Description

Technical Field

[0001] This invention relates to the field of intelligent monitoring technology, and in particular to a method for dynamic monitoring of construction around pipeline networks based on multi-target visual recognition. Background Technology

[0002] In the field of urban underground pipeline network operation and maintenance, dynamic monitoring of construction activities is a crucial link in ensuring infrastructure safety. Current mainstream methods primarily rely on visual sensors to collect data from the construction area, combined with target detection algorithms (such as YOLO and Faster R-CNN) to identify machinery and personnel targets, and supplemented by point cloud registration techniques (such as the ICP algorithm) to construct a 3D scene. These techniques achieve environmental perception through multi-view image fusion and utilize Kalman filtering or particle filtering for moving target tracking. With the development of deep learning, feature extraction and multimodal data fusion methods based on neural network models have become a research hotspot in the field, enabling automated identification and analysis of elements such as construction machinery types and personnel distribution.

[0003] However, existing methods still have limitations in complex scenarios: First, the spatiotemporal alignment accuracy of multi-source heterogeneous data (such as RGB images, depth maps, and heat maps) is affected by sensor clock synchronization errors and coordinate system differences, resulting in insufficient spatial consistency of fused features; Second, traditional motion trajectory prediction relies heavily on linear models (such as uniform speed assumptions), which makes it difficult to accurately fit the nonlinear motion patterns of construction machinery (such as steering and speed change), affecting the timeliness of risk warnings. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a method for dynamic monitoring of pipeline construction around the pipeline network based on multi-target visual recognition to solve the problems of poor consistency of multi-source data fusion and inaccurate fitting of complex motion patterns of construction machinery.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides a method for dynamic monitoring of construction around pipeline networks based on multi-target visual recognition, comprising,

[0008] Collect and preprocess view data of the construction area;

[0009] Multimodal view features are extracted from view data and deep fusion analysis is performed to generate comprehensive perception results;

[0010] The comprehensive perception results are transformed into a 3D point cloud dataset, and the 3D point cloud datasets from multiple perspectives are spatiotemporally aligned to construct a 3D model of the construction area.

[0011] Based on the 3D model of the construction area, the target entities are marked, and the movement trajectory of the target entities is tracked to predict their future movement paths.

[0012] The construction site environment and future movement paths are integrated and analyzed to identify potential construction damage risks, which are then classified and processed to generate a potential risk list.

[0013] Potential risks are categorized and sorted by severity and urgency, and pipeline monitoring reports are generated using integrated data analysis.

[0014] As a preferred embodiment of the pipeline network perimeter construction dynamic monitoring method based on multi-target visual recognition described in this invention, the view data of the construction area includes the type, quantity, dwell time, and number of personnel wearing safety helmets of large construction machinery.

[0015] The preprocessing includes image denoising, illumination correction, color enhancement, lens distortion correction, and image resolution unification of the view data to eliminate environmental interference;

[0016] As a preferred embodiment of the pipeline network perimeter construction dynamic monitoring method based on multi-target visual recognition described in this invention, the specific steps for extracting multimodal view features from view data are as follows:

[0017] Select a deep learning model based on task requirements, perform convolution, activation and pooling on the view data to extract visual features, perform feature transformation on the view data to extract terrain features, and extract temperature distribution features through time sliding.

[0018] Visual features, terrain features, and temperature distribution features are stitched together and weighted in the embedding space to generate multimodal view features.

[0019] As a preferred embodiment of the multi-target visual recognition-based method for dynamic monitoring of pipeline perimeter construction as described in this invention, the specific steps for generating the comprehensive sensing results are as follows:

[0020] The multimodal view features are normalized and nonlinear transformations are performed to map the multimodal view features to the semantic space, generating high-level semantic feature vectors.

[0021] The high-level semantic feature vector is reduced in dimensionality to generate a low-dimensional high-level semantic feature vector, and a sigmoid classifier is used to predict the category, identify the category label and location coordinates.

[0022] By integrating category labels and location coordinates through deep fusion analysis, a comprehensive perception result is generated.

[0023] As a preferred embodiment of the multi-target visual recognition-based method for dynamic monitoring of pipeline perimeter construction as described in this invention, the specific steps for converting the comprehensive perception results into a three-dimensional point cloud dataset and spatiotemporally aligning the three-dimensional point cloud datasets from multiple perspectives to construct a three-dimensional model of the construction area are as follows.

[0024] Based on the comprehensive perception results, the depth value of each target entity in the three-dimensional space is estimated and integrated to generate a multi-source depth map;

[0025] Multi-source features are extracted from multi-source depth maps by time-segmented sliding windows, and the spatial coordinates of the feature points corresponding to the multi-source features are obtained by analyzing the deformation, thereby generating an intrinsic parameter matrix.

[0026] Based on the intrinsic parameter matrix, each pixel in the multi-source depth map is converted into point coordinates in three-dimensional space, and the calculation is repeated for all valid pixels in the multi-source depth map to generate a three-dimensional point cloud dataset.

[0027] By fitting a 3D point cloud dataset with a neural network model, an implicit surface function is constructed and optimized to generate a mesh model.

[0028] Based on the mesh model, time-sliding segmentation is performed on the 3D point cloud dataset from multiple perspectives to generate mesh point coordinates, and spatiotemporal alignment is performed to construct a 3D model of the construction area.

[0029] As a preferred embodiment of the multi-target visual recognition-based method for dynamic monitoring of pipeline perimeter construction as described in this invention, the steps of marking work entities based on a three-dimensional model of the construction area, tracking the movement trajectory of the work entities, and predicting their future movement paths are as follows.

[0030] The 3D model is projected back onto the 2D image plane to identify and locate the target work entities within the construction area;

[0031] Based on the initial state of the target entity, the initial conditions of the Kalman filter are set, and the motion trajectory of the target entity is tracked to generate a motion trajectory map of the target entity.

[0032] Based on the trajectory map of the target object, a comparative analysis is conducted with typical motion patterns to identify the motion pattern of the target object. Then, second-order polynomial regression fitting is performed to predict the future movement path of the target object.

[0033] As a preferred embodiment of the multi-target visual recognition-based method for dynamic monitoring of pipeline perimeter construction as described in this invention, the steps of integrating and analyzing the construction site environment and future movement paths to identify potential construction damage risks, classifying them, and generating a potential risk list are as follows.

[0034] Different risk levels are defined based on the distance and speed between the work target and the hazardous area;

[0035] Based on the risk level, the future movement path is combined with the construction site environment to calculate the distance change between the moving operation target and the danger zone on the future movement path, and to identify potential construction damage risks.

[0036] By combining historical construction activity records and potential construction damage risks, a deep learning-based risk classification algorithm is used to classify each construction damage risk point to a corresponding risk level and generate a list of potential risks.

[0037] As a preferred embodiment of the multi-target visual recognition-based method for dynamic monitoring of pipeline perimeter construction as described in this invention, the steps of classifying and sorting the potential risk list according to severity and urgency, and generating a pipeline monitoring report through integrated data analysis are as follows.

[0038] Based on the list of potential risks, a predefined set of rules is used to compare and analyze the on-site construction environment conditions to generate the severity of damage to the pipeline network caused by each risk.

[0039] The urgency of a collision is determined by the time difference between the intersection of the target work entity and the danger point obtained from the future movement path;

[0040] Based on the severity and urgency of the risks, a comprehensive risk value is calculated, and all risk items are classified into different levels.

[0041] Within the same level, they are further sorted according to the magnitude of the comprehensive risk value, and a pipeline monitoring report is generated through integrated data analysis.

[0042] In a second aspect, the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, wherein when the computer program is executed by the processor, it implements any step of the method for dynamic monitoring of pipeline perimeter construction based on multi-target visual recognition as described in the first aspect of the present invention.

[0043] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the method for dynamic monitoring of pipeline perimeter construction based on multi-target visual recognition as described in the first aspect of the present invention.

[0044] The beneficial effects of this invention are as follows: By extracting visual features, terrain features, and temperature distribution features through convolution, feature transformation, and time sliding, and then performing weighted fusion, a unified representation of multi-source heterogeneous data is achieved. This enables the comprehensive perception results to fully reflect the static structure and dynamic changes of the construction scene, providing high-precision semantic input for subsequent 3D modeling. By fitting a 3D point cloud using a neural network model and optimizing implicit surface functions to generate a mesh model, the limitations of traditional point cloud reconstruction are overcome. The use of implicit functions to continuously express complex surfaces improves the geometric accuracy and topological integrity of the 3D model, achieving full-dimensional dynamic perception and accurate risk warning of the construction scene around the pipeline network, ensuring the scientific rigor and timeliness of the monitoring report. Attached Figure Description

[0045] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a flowchart of a method for dynamic monitoring of pipeline perimeter construction based on multi-target visual recognition.

[0047] Figure 2 This is a flowchart of the multimodal view feature extraction process.

[0048] Figure 3 Flowchart for building a 3D point cloud model.

[0049] Figure 4 Generate a flowchart for risk warning. Detailed Implementation

[0050] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0051] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0052] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0053] Reference Figures 1-4 This is one embodiment of the present invention, which provides a method for dynamic monitoring of pipeline construction around a pipeline network based on multi-target visual recognition, including the following steps:

[0054] S1. Collect and preprocess the view data of the construction area;

[0055] The view data of the construction area includes the type, quantity, dwell time, and number of personnel wearing safety helmets of large construction machinery;

[0056] It should be noted that the view data of the construction area is collected through multi-source sensors deployed on the construction site, including high-resolution surveillance cameras, infrared thermal imagers, and depth cameras. The cameras capture RGB images at a fixed frequency (e.g., 10fps) to record the type, quantity, and location information of construction machinery (e.g., excavators, cranes). Target detection algorithms (e.g., YOLOv5) are used to statistically analyze the machinery's dwell time in real time, and its activity trajectory is analyzed in conjunction with timestamps. Simultaneously, a head detection model (e.g., ResNet-50) is used to identify the helmet-wearing status of personnel, and a multi-target tracking algorithm (e.g., DeepSORT) is used to associate target IDs between frames to achieve dynamic statistics on the number of personnel.

[0057] Preprocessing includes image denoising, illumination correction, color enhancement, lens distortion correction, and image resolution unification of the view data, as well as eliminating environmental interference.

[0058] It should be noted that, firstly, image noise is eliminated through non-local mean denoising (NL-Means) or wavelet transform, and illumination correction is performed based on Retinex theory to compensate for uneven illumination; secondly, adaptive histogram equalization (CLAHE) is applied to enhance color contrast, while Zhang Zhengyou calibration method is used to calculate lens distortion parameters, and geometric correction is completed through inverse mapping; finally, bicubic interpolation is used to uniformly scale all images to the target resolution (e.g., 1920×1080) to ensure the scale consistency of multi-source data.

[0059] The preprocessed view data is calibrated using a time and space synchronization algorithm to correct time differences between different data sources, generating time-synchronized and spatially aligned view data.

[0060] It should be noted that, firstly, a time synchronization operation is performed on the preprocessed view data. By unifying the time base of each data source, clock offsets and acquisition delays between different sensor acquisition times are compensated, ensuring that data sources of different modalities are aligned in the time dimension, resulting in time-synchronized multi-source view data. Next, a spatial alignment operation is performed to unify data sources from different viewpoints or coordinate systems into the same spatial coordinate system, outputting time-synchronized and spatially aligned view data.

[0061] S2. Extract multimodal view features from the view data and perform deep fusion analysis to generate comprehensive perception results;

[0062] Select a deep learning model based on task requirements, perform convolution, activation and pooling on the view data to extract visual features, perform feature transformation on the view data to extract terrain features, and extract temperature distribution features through time sliding.

[0063] The task requirements include target identification, scene understanding, and environmental monitoring.

[0064] It should be noted that, firstly, a deep learning model (CNN neural network architecture) suitable for multimodal view feature extraction is selected based on task requirements. Next, within the CNN neural network, convolution, activation, and pooling operations are sequentially performed on the input view data to complete feature extraction, outputting visual features. A multilayer perceptron (MLP) is used to perform feature transformation on each point in the input view data. Subsequently, global max pooling is performed on the transformed points to aggregate the transformed features, constructing a deep feature extraction network. Spatial structure features are extracted using this deep feature extraction network, outputting terrain features. A time-sliding method is employed to segment the view data with a fixed window length to extract temperature distribution features.

[0065] The training process of a multilayer perceptron is as follows: First, the weights and bias parameters of the multilayer perceptron are initialized; then, the preprocessed view data is input into the multilayer perceptron, and nonlinear feature transformation is performed on each point through multiple fully connected layers. Finally, global max pooling is performed to complete the training process.

[0066] Visual features refer to high-level image representations with semantic information extracted by performing multi-level convolution, activation, and pooling operations on input view data (such as two-dimensional images or video frames) through neural network models.

[0067] Terrain features refer to the abstract representation of terrain undulations and surface morphology generated by performing multilayer perceptron (MLP) feature transformation and global pooling operations on the input view data through a deep feature extraction network.

[0068] Temperature distribution characteristics refer to the characteristic representation that reflects the temperature distribution pattern and dynamic changes within a specific time period by using the time sliding window method to calculate the statistical characteristics (such as mean, variance, etc.) and changing trends of temperature values ​​within each window.

[0069] The training process of a deep learning model is as follows: First, prepare and preprocess the training dataset to ensure that it contains sufficient labeled information to support supervised learning. Next, initialize the model parameters and define the loss function and optimizer. The loss function is used to quantify the difference between the model's predicted values ​​and the true labels, while the optimizer is responsible for updating the model parameters to minimize the loss. Then, input the training dataset into the deep learning model in batches, perform forward propagation to calculate the prediction results, and evaluate the prediction error through the loss function. After that, backpropagation is performed based on the prediction error to adjust the network weights, and the model parameters are gradually optimized using gradient descent or its variants (such as Adam) to complete the training process.

[0070] Visual features, terrain features, and temperature distribution features are stitched together and weighted in the embedding space to generate multimodal view features.

[0071] It should be noted that, firstly, a high-level image representation with semantic information is obtained from visual features, an abstract representation describing terrain undulations and surface morphology is obtained from terrain features, and a time-dynamic representation reflecting the thermal environment state is obtained from temperature distribution features. Secondly, a multi-source fusion feature algorithm is used to perform unified representation learning on the three representations. Next, alignment and concatenation operations are performed in the embedding space to concatenate the three features learned from the unified representation learning along the feature dimension, forming a high-dimensional joint feature vector. Subsequently, the concatenated high-dimensional joint feature vector is weighted and fused to generate multimodal view features.

[0072] Unified representation learning includes feature normalization, dimension alignment, and learnable weight assignment.

[0073] The multimodal view features are normalized and nonlinear transformations are performed to map the multimodal view features to the semantic space, generating high-level semantic feature vectors.

[0074] It should be noted that, firstly, the multimodal view features are normalized, outputting normalized multimodal view features. Then, through layer-by-layer linear transformations and nonlinear mapping operations, the normalized multimodal view features are projected into the semantic space. Finally, in the semantic space, a nonlinear transformation is performed on the normalized multimodal view features using an MLP, outputting a high-level semantic feature vector.

[0075] The high-level semantic feature vector is reduced in dimensionality to generate a low-dimensional high-level semantic feature vector, and a sigmoid classifier is used to predict the category, identify the category label and location coordinates.

[0076] It should be noted that principal component analysis (PCA) is used to calculate the covariance matrix of the high-level semantic feature vectors. The eigenvalues ​​and eigenvectors of the covariance matrix are then solved, and the eigenvalues ​​are sorted from largest to smallest. The eigenvectors corresponding to the k largest eigenvalues ​​are selected to form a projection matrix. In this projection matrix, the original high-level semantic feature vectors are projected onto a new low-dimensional subspace to achieve dimensionality reduction, resulting in low-dimensional high-level semantic feature vectors. These low-dimensional feature vectors are then input into a Sigmoid classifier. Through linear combination and mapping with the Sigmoid function, the classifier outputs probability results between 0 and 1, determining the confidence level of the target entity belonging to a certain category, thereby identifying the category label. Simultaneously, a regression branch is set in the CNN network structure to identify the position coordinates of the target entity.

[0077] By integrating category labels and location coordinates through deep fusion analysis, a comprehensive perception result is generated.

[0078] Furthermore, firstly, category labels and location coordinates are used as inputs to the deep fusion analysis method. Category labels are the probability outputs of binary or multiple categories obtained by a Sigmoid classifier predicting the category of a low-dimensional high-level semantic feature vector. Location coordinates are the spatial location information of the target entity predicted by a regression branch based on the same low-dimensional high-level semantic feature vector. Then, the deep fusion analysis method is used to jointly model the category labels and location coordinates. Specifically, this involves concatenating and weighting the category labels and location coordinates at the feature level to enhance the semantic correlation between category recognition and spatial localization. For example, in a specific implementation, joint optimization of the target entity can be set, allowing the category prediction result and the location offset to participate in gradient updates during training. Finally, a unified representation containing category recognition and location information is output, generating a comprehensive perception result.

[0079] Semantic space refers to a high-dimensional vector space in which data points are represented as feature vectors. These feature vectors can capture and express the semantic information or meaning of the original data.

[0080] S3. The comprehensive perception results are converted into a 3D point cloud dataset, and the 3D point cloud datasets from multiple perspectives are spatiotemporally aligned to construct a 3D model of the construction area.

[0081] Based on the comprehensive perception results, the depth value of each target entity in the three-dimensional space is estimated and integrated to generate a multi-source depth map;

[0082] It should be noted that the comprehensive perception results are used as the input to the stereo matching algorithm. These comprehensive perception results include category labels and location coordinate information integrated through depth fusion analysis. Then, a depth estimation algorithm is used to estimate the depth value of each target entity in three-dimensional space through a neural network model based on the location coordinate information in the comprehensive perception results. For example, in a specific implementation, a monocular depth estimation network can be used to combine two-dimensional image information with location coordinates to derive pixel-level depth prediction. Then, the estimated depth value is fused with the depth data of the viewpoint (directly acquired by the sensor) to finally output a multi-source depth map with a unified spatial reference.

[0083] The training process of the neural network model first involves inputting preprocessed 3D point cloud coordinate data into the model via forward propagation. Feature extraction and transformation are then performed through multiple convolutional and fully connected layers. The LeakyReLU activation function (α=0.01) and residual connection structure are used to enhance feature representation capabilities, while batch normalization layers accelerate convergence. During the backpropagation phase, the gradient of the combined loss function (L1 loss weight 0.7 + SSIM loss weight 0.3) with respect to the parameters of each layer is calculated using automatic differentiation. The Adam optimizer (initial learning rate 0.001) is used in conjunction with a cosine annealing strategy to dynamically adjust the learning rate, and L2 weight decay (coefficient 1e) is introduced. -4 Regularization is performed using dropout; loss changes are recorded every 50 iterations during training, and PSNR and IoU metrics are evaluated on the validation set for each training iteration.

[0084] Multi-source features are extracted from multi-source depth maps by time-segmented sliding windows, and the spatial coordinates of the feature points corresponding to the multi-source features are obtained by analyzing the deformation, thereby generating an intrinsic parameter matrix.

[0085] It should be noted that, firstly, a fixed-size time-segmented sliding window is defined, for example, the window size can be set to 5 frames, to traverse the entire multi-source depth map sequence; then, at each window position, the multi-source depth information within the corresponding time period is extracted as multi-source features, which include, but are not limited to, depth change rate, displacement between adjacent frames, etc.; then, the deformation of the multi-source features is analyzed using structured light scanning method to obtain the spatial coordinates of the feature points corresponding to the multi-source features; and a calibration board is used to help determine the relative positional relationship between the camera and the projector, and then the intrinsic parameter matrix is ​​derived based on geometric relationships (such as the principle of triangulation).

[0086] Based on the intrinsic parameter matrix, each pixel in the multi-source depth map is converted into point coordinates in three-dimensional space, and the calculation is repeated for all valid pixels in the multi-source depth map to generate a three-dimensional point cloud dataset.

[0087] It should be noted that, firstly, the depth value of each pixel in the multi-source depth map is read. Combined with the focal length and principal point coordinates in the intrinsic parameter matrix, the pixel coordinates (u,v) are converted to coordinates in the normalized camera coordinate system using the perspective projection formula. Next, the depth value z is used to convert the coordinates in the normalized camera coordinate system to point coordinates in the 3D camera coordinate system. This calculation is repeated for all valid pixels in the multi-source depth map to obtain an initial 3D point set. Then, a stereo matching algorithm is used to calculate the disparity between adjacent viewpoints, and the spatial position of the 3D point set coordinates is optimized using the triangulation principle to eliminate redundant points with inconsistent depths. Finally, the optimized 3D point coordinate set is fused with RGB color information to generate a 3D point cloud dataset with color attributes.

[0088] By fitting a 3D point cloud dataset with a neural network model, an implicit surface function is constructed and optimized to generate a mesh model.

[0089] It should be noted that, firstly, a 3D point cloud dataset is used, which contains spatial coordinate information, such as the x, y, and z coordinates of each point. A neural network model is used to fit the 3D point cloud dataset. The neural network model receives the coordinates of the 3D point cloud dataset as input and outputs the corresponding implicit surface function values. The implicit surface function is defined as a continuous function in 3D space that maps surface points to zero values ​​and interior points to positive values. The weight parameters of the neural network model are optimized by minimizing the error between the implicit surface function values ​​predicted by the neural network model and the true implicit surface function values, such as the mean square error. After optimization, equally spaced sampling points are taken in 3D space, and the trained neural network model is used to predict the implicit surface function values ​​of the sampling points. Based on the zero isosurface of the implicit surface function, the moving cube algorithm is used to extract the triangular mesh model from the implicit surface function values ​​of the sampling points to generate the mesh model.

[0090] Based on the mesh model, time-sliding segmentation is performed on the 3D point cloud dataset from multiple perspectives to generate mesh point coordinates, and spatiotemporal alignment is performed to construct a 3D model of the construction area.

[0091] Furthermore, firstly, vertex coordinates are extracted based on the mesh model, and the 3D point cloud datasets from multiple perspectives are divided into several segments according to the time series. Euclidean clustering is performed on the 3D point cloud datasets within each time window to separate independent objects or regions. Subsequently, within each independent object or region, the iterative nearest-point algorithm is used for point cloud registration, calculating the rigid transformation matrix between 3D point cloud datasets from adjacent time windows, and achieving spatiotemporal alignment by minimizing the distance between corresponding points. The aligned 3D point cloud datasets are then used to generate continuous surfaces through Poisson reconstruction. Finally, all time window data are merged to construct a complete 3D model of the construction area.

[0092] S4. Based on the 3D model of the construction area, mark the target objects of the work, track the movement trajectory of the target objects, and predict the future movement path;

[0093] The 3D model is projected back onto the 2D image plane to identify and locate the target work entities within the construction area;

[0094] The target of the operation refers to the type of large construction machinery and the personnel wearing safety helmets.

[0095] It should be noted that, firstly, the projection parameters from the 3D model of the complete construction area to the 2D image plane are determined, including the camera intrinsic and extrinsic parameters (e.g., rotation matrix R and translation vector t). Perspective projection is used to convert the vertex coordinates of the 3D model into pixel coordinates of the 2D image, generating a depth map and a color texture map. Subsequently, an object detection algorithm is used to process the network structure of the depth map and color texture map, outputting bounding box coordinates and class probabilities. All bounding boxes are sorted according to their class probabilities, and the bounding box with the highest confidence is retained as the current optimal detection result. Then, the intersection-over-union (IoU) ratio of the co-boundary boxes is calculated sequentially. If it exceeds a set IoU threshold (range: 0~1), it is considered a duplicate detection and is discarded. Finally, the set of retained bounding boxes is output, achieving accurate identification and localization of the work entity targets within the construction area.

[0096] The intersection-union ratio (IU) threshold is usually set according to the specific task requirements. For example, 0.5 is often used as the threshold in object detection tasks.

[0097] Based on the initial state of the target entity, the initial conditions of the Kalman filter are set, and the motion trajectory of the target entity is tracked to generate a motion trajectory map of the target entity.

[0098] It should be noted that, firstly, in the initial state of the target entity, the initial state vector of the target entity is extracted through sliding segmentation, and the initial state covariance matrix of the Kalman filter is set. The state transition matrix and process noise covariance matrix of the linear motion model are then established. In the prediction phase, the Kalman filter calculates the prior state estimate; in the update phase, the observations provided by the target detection algorithm are fused to calculate the posterior state estimate. The prediction-update cycle is repeated at each time step, outputting a smoothed sequence of target entity states. Finally, the position estimates from each time step are concatenated to generate the target entity motion trajectory map.

[0099] The process of establishing the state transition matrix for a linear motion model is as follows: First, determine the composition of the initial state vector and establish the physical relationship between displacement and velocity based on Newton's laws of motion. For a uniform motion model, the velocity component remains constant. The physical relationship between displacement and velocity is established using Newton's laws of motion and transformed into matrix form, generating a 4×4 state transition matrix. The diagonal elements are set to 1 to maintain the original state, and displacement is calculated at the intersection of the position row and the velocity column. Finally, the complete state transition matrix is ​​obtained.

[0100] The process of establishing the process noise covariance matrix is ​​as follows: First, the sources of uncertainty are analyzed, including factors such as velocity changes and external force disturbances. For the uniform motion model, it is assumed that position noise is generated by the integral of velocity noise. Noise propagation is derived based on physical relationships, and the noise variance of each state component is calculated. The noise variance of each state component is then filled into the diagonal to construct a 4×4 process noise covariance matrix.

[0101] Based on the trajectory map of the target object, a comparative analysis is conducted with typical motion patterns to identify the motion pattern of the target object. Then, second-order polynomial regression fitting is performed to predict the future movement path of the target object.

[0102] Furthermore, the motion trajectory map of the target entity is first plane-formatted using coordinates. The coordinate sequence is extracted from the trajectory map, and a dynamic time warping algorithm is used to calculate the similarity between the target entity's motion trajectory and typical motion patterns (e.g., linear, circular, and reciprocating motion), identifying the motion pattern with the highest matching degree. Then, under the motion pattern with the highest matching degree, a second-order polynomial regression is performed on the target entity's motion trajectory coordinates, and the polynomial coefficients are solved using the least squares method. Based on the fitted polynomial equation, predicted coordinates are calculated using future time points, and the predicted coordinates are connected to generate a continuous trajectory line for the target entity's future movement path.

[0103] S5. Integrate and analyze the environment of the construction site and future movement paths, identify potential construction damage risks, classify them, and generate a potential risk list.

[0104] Different risk levels are defined based on the distance and speed between the work target and the hazardous area;

[0105] It should be noted that, firstly, the shortest Euclidean distance between the current location of the work entity and the boundary of the hazardous area is calculated. A distance threshold is then defined. For example, if the shortest Euclidean distance is greater than or equal to the distance threshold, it is classified as a safe level; if the shortest Euclidean distance is less than the distance threshold, it is classified as a hazardous level.

[0106] The boundaries of hazardous areas are determined by analyzing historical accident data, environmental characteristics, and the distribution of movement trajectories.

[0107] Based on the risk level, the future movement path is combined with the construction site environment to calculate the distance change between the target work object and the danger zone on the future movement path, and to identify potential construction damage risks.

[0108] It should be noted that, firstly, the future movement path of the work entity is discretized into multiple prediction points to obtain the geometric boundary coordinates of the hazardous area in the 3D model of the construction site. For each prediction point, the shortest Euclidean distance to the boundary of the hazardous area is calculated. Based on the distance threshold (1~3m) corresponding to the risk level, prediction points with a shortest Euclidean distance lower than the distance threshold are marked, identifying areas in the path segment that may cause construction damage.

[0109] By combining historical construction activity records and potential construction damage risks, a deep learning-based risk classification algorithm is used to classify each construction damage risk point to a corresponding risk level and generate a list of potential risks.

[0110] Furthermore, the construction damage risks in historical data are first normalized to match the currently identified potential construction damage risks. Then, after calculations by convolutional and fully connected layers, the risk level probability distribution is output using the Softmax function. The risk level probability distribution corresponding to the highest risk probability is selected and mapped to each construction damage risk point. Finally, a list of potential risks, including location coordinates, prediction time, and risk level, is generated in chronological order.

[0111] S6. Classify and sort the potential risks according to their severity and urgency, and generate a pipeline monitoring report using integrated data analysis.

[0112] Based on the list of potential risks, a predefined set of rules is used to compare and analyze the on-site construction environment conditions to generate the severity of damage to the pipeline network caused by each risk.

[0113] It should be noted that, firstly, based on the risk list, the risk point parameters in the potential risk list are read, and the protection level and structural parameters of the corresponding pipeline area in the predefined rule set are retrieved. The risk point parameters are matched with the entries in the predefined rule set one by one to identify the risk level, and the impact force is calculated using Hertzian contact theory to output the damage severity classification (e.g., level 3 damage corresponds to "structural cracking").

[0114] The rule set predefinition process adopts a multi-stage collaborative construction method: First, the data mining algorithm (Apriori association rule mining) is used to analyze more than 3,000 risk event records in historical monitoring reports to extract high-frequency feature combination patterns; then, the Analytic Hierarchy Process (AHP) is used to assign weights to the preliminary rules to establish the rule set.

[0115] Preliminary rules refer to the basic judgment conditions that are pre-selected through data analysis and used to evaluate targets (such as danger zone delineation or motion pattern matching), such as unweighted original rule entries like distance thresholds, speed limits, and historical accident frequency.

[0116] The urgency of a collision is determined by the time difference between the intersection of the target work entity and the danger point obtained from the future movement path;

[0117] It should be noted that, firstly, the sequence of predicted points along the future movement path of the target entity is extracted, and the Euclidean distance between each predicted point and the danger point is calculated. When a predicted point with an Euclidean distance less than the safety threshold is detected, the timestamp corresponding to the predicted point is recorded. Based on the target entity's current speed and remaining distance, the collision occurrence time is linearly extrapolated. The time difference Δt is obtained by subtracting the current time from the collision prediction time. The collision urgency is then output based on a predefined urgency level classification standard.

[0118] It should be noted that the safety threshold is set based on the on-site working environment and industry standards, and the value range is 3~5m (the safety threshold range is a reasonable range derived from comprehensive on-site physical characteristics, industry standards, historical data analysis and engineering practice).

[0119] The predefined process for grading urgency levels involves collecting collision time differences from a historical accident database. The correlation index between the statistical distribution data and the degree of pipeline damage is predefined.

[0120] Based on the severity and urgency of the risks, a comprehensive risk value is calculated, and all risk items are classified into different levels.

[0121] The specific expression for calculating the comprehensive risk value is:

[0122] ;

[0123] in, This represents the overall risk value; This represents the distance attenuation coefficient (typically ranging from 2 to 6). Indicates distance from the first The distance between each predicted point; Indicates maximum speed; Indicates the time difference between collisions; Indicates the smoothing factor; Indicates the total number of predicted points; Indicates the prediction point Risk level;

[0124] It should be noted that, firstly, the severity and urgency of the risk are normalized and mapped to a uniform scoring range. Based on the risk level, a weighted summation formula is used to calculate the comprehensive risk value. According to a predefined comprehensive risk level classification standard, the comprehensive risk value is mapped to the corresponding risk level.

[0125] The pre-definition process of the comprehensive risk level classification standard: The pre-definition process of the comprehensive risk level classification standard strictly follows engineering practice specifications and establishes a scientific classification system through multi-dimensional data fusion and dynamic optimization mechanisms;

[0126] Within the same level, they are further sorted according to the magnitude of the comprehensive risk value, and a pipeline monitoring report is generated through integrated data analysis.

[0127] Furthermore, all risk items at the same risk level are first sorted in descending order of their comprehensive risk value. An integrated data analysis method then correlates these sorted risk items with pipeline topology data, environmental monitoring data, and construction progress data. Through multi-dimensional data fusion, pipeline monitoring report entries are generated, ultimately forming a complete pipeline monitoring report containing risk ranking, associated pipe segment information, and environmental parameters.

[0128] This embodiment also provides a computer device applicable to the dynamic monitoring method for pipeline perimeter construction based on multi-target visual recognition, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to realize the dynamic monitoring method for pipeline perimeter construction based on multi-target visual recognition proposed in the above embodiment.

[0129] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0130] This embodiment also provides a storage medium storing a computer program. When executed by a processor, the program implements the method for dynamic monitoring of pipeline perimeter construction based on multi-target visual recognition as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0131] In summary, this invention achieves unified representation of multi-source heterogeneous data by extracting visual features, terrain features, and temperature distribution features through convolution, feature transformation, and temporal sliding, followed by weighted fusion. This enables the comprehensive perception results to fully reflect the static structure and dynamic changes of the construction scene, providing high-precision semantic input for subsequent 3D modeling. By fitting a 3D point cloud using a neural network model and optimizing implicit surface functions to generate a mesh model, it overcomes the limitations of traditional point cloud reconstruction. Utilizing implicit functions to continuously express complex surfaces improves the geometric accuracy and topological integrity of the 3D model, achieving full-dimensional dynamic perception and accurate risk warning of the construction scene around the pipeline network, ensuring the scientific rigor and timeliness of the monitoring report.

[0132] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for dynamic monitoring of pipeline network perimeter construction based on multi-target visual recognition, characterized in that: The application relates to a method for monitoring a construction site, and belongs to the field of construction safety monitoring. View data of a construction area is collected and preprocessed; Multi-modal view features are extracted from the view data, and deep fusion analysis is performed to generate comprehensive perception results; The comprehensive perception results are converted into a three-dimensional point cloud dataset, and three-dimensional point cloud datasets of multiple perspectives are spatio-temporally aligned to construct a three-dimensional model of the construction area; Based on the three-dimensional model of the construction area, work entity targets are labeled, the motion trajectories of the work entity targets are tracked, and future movement paths are predicted, specifically as follows, The three-dimensional model is projected back to a two-dimensional image plane, and work entity targets in the construction area are identified and located; According to the initial state of the work entity targets, the initial conditions of the Kalman filter are set, the motion trajectories of the work entity targets are tracked, and a work entity target motion trajectory graph is generated; According to the work entity target motion trajectory graph, a comparison analysis is performed with typical motion modes, the motion mode of the work entity target is identified, and a second-order polynomial regression fitting is performed to predict the future movement path of the work entity target; The environment of the construction site and the future movement path are integrated and analyzed, potential construction damage risks are identified, and classification processing is performed to generate a potential risk list; The potential risk list is classified and sorted according to severity and urgency, and a pipe network monitoring report is generated through integrated data analysis.

2. The multi-target visual recognition based pipeline perimeter construction dynamic monitoring method of claim 1, wherein: The view data of the construction area includes the type, quantity, stay time of large construction machinery, and the number of personnel wearing safety helmets; The preprocessing includes image denoising, illumination correction, color enhancement, lens distortion correction, and image resolution unification processing of the view data, and environmental interference is eliminated. 3.The method of claim 2, wherein the method further comprises: determining a distance between the target object and the camera based on the distance between the target object and the camera and the distance between the camera and the reference object. The multi-modal view features are extracted from the view data, specifically as follows, According to task requirements, a deep learning model is selected, the view data is convolved, activated, and pooled, visual features are extracted, terrain features are extracted by performing feature transformation on the view data, and temperature distribution features are extracted through time sliding; The visual features, terrain features, and temperature distribution features are spliced and weighted in an embedding space to generate multi-modal view features.

4. The multi-target visual recognition based pipeline perimeter construction dynamic monitoring method of claim 3, wherein: The comprehensive perception results are generated, specifically as follows, The multi-modal view features are normalized, and a non-linear transformation is performed to map the multi-modal view features to a semantic space to generate high-level semantic feature vectors; The high-level semantic feature vectors are reduced in dimension to generate low-dimensional high-level semantic feature vectors, and a Sigmoid classifier is used for class prediction to identify class labels and position coordinates; The class labels and position coordinates are integrated through a deep fusion analysis method to generate comprehensive perception results.

5. The multi-target visual recognition based pipeline perimeter construction dynamic monitoring method of claim 4, wherein: The comprehensive perception results are converted into a three-dimensional point cloud dataset, and three-dimensional point cloud datasets of multiple perspectives are spatio-temporally aligned to construct a three-dimensional model of the construction area, specifically as follows, According to the comprehensive perception results, the depth values of each work entity target in three-dimensional space are estimated and integrated to generate a multi-source depth map; Multi-source state features are extracted in the multi-source depth map through time segmentation sliding windows, and the spatial coordinates of feature points corresponding to the multi-source state features are obtained by analyzing deformation to generate an intrinsic matrix; According to the intrinsic matrix, each pixel in the multi-source depth map is converted into point coordinates in three-dimensional space, and the calculation is repeated for all valid pixels in the multi-source depth map to generate a three-dimensional point cloud dataset; Through the neural network model, the three-dimensional point cloud dataset is fitted, an implicit surface function is constructed and optimized, and a mesh model is generated; Based on the mesh model, the three-dimensional point cloud datasets of multiple perspectives are time-sliding segmented to generate mesh point coordinates, and spatio-temporal alignment is performed to construct a three-dimensional model of the construction area. 6.The multi-target visual recognition based pipeline perimeter construction dynamic monitoring method of claim 1, wherein: The environment of the construction site and the future moving path are integrated and analyzed to identify potential construction damage risks and classify them to generate a potential risk list, and the specific steps are as follows, According to the distance and speed factors of the work entity target and the dangerous area, different risk levels are defined; According to the risk level, the future moving path is combined with the environment of the construction site to calculate the distance change of the work entity target on the future moving path and the dangerous area, and potential construction damage risks are identified; Combined with historical construction activity record data and potential construction damage risks, a deep learning risk classification algorithm is used for classification, each construction damage risk point is mapped to the corresponding risk level, and a potential risk list is generated.

7. The multi-target visual recognition based pipeline perimeter construction dynamic monitoring method of claim 6, wherein: The potential risk list is classified and sorted according to severity and urgency, and an integrated data analysis method is used to generate a pipe network monitoring report, and the specific steps are as follows, According to the potential risk list, a predefined rule set is compared and analyzed with the construction environment conditions on site to generate the severity of each risk to the pipe network damage; According to the future moving path, the time difference between the work entity target and the dangerous point intersection is obtained to obtain the urgency of the collision; According to the severity and urgency of the risk, a comprehensive risk value is calculated, and all risk items are divided into different levels; In the same level, according to the size of the comprehensive risk value, the pipe network monitoring report is generated through the integrated data analysis method.

8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that: The processor executes the computer program to realize the steps of the pipe network surrounding construction dynamic monitoring method based on multi-target visual recognition according to any one of claims 1-7.

9. A computer readable storage medium having stored thereon a computer program, characterized in that: The computer program is executed by the processor to realize the steps of the pipe network surrounding construction dynamic monitoring method based on multi-target visual recognition according to any one of claims 1-7.

Images