BIM-based pipeline construction progress visual management system

Abstract

The application discloses a pipeline construction progress visualization management system based on BIM and belongs to the technical field of building engineering management. The system obtains image data of a construction site containing pipeline components through an image acquisition device, a data storage unit stores BIM three-dimensional geometric data, BIM attribute data, planned installation time data and actual installation state data, a semantic segmentation network identifies the types and installation states of the pipeline components from the image data, a graph construction module constructs graph structure data reflecting spatial constraint relationships according to the BIM three-dimensional geometric data, a graph neural network module calculates the spatial constraint propagation paths of uninstalled pipeline components, a progress re-planning module generates updated installation sequence data, and a visualization rendering module differentiates the pipeline components in a three-dimensional scene by color and transparency. The application realizes automatic collection of construction progress data, automatic deduction of spatial constraint relationships and automatic adjustment of installation sequences, and improves the intuitiveness and response efficiency of progress management.

Description

Technical Field

[0001] This invention relates to the field of building engineering management technology, and in particular to a BIM-based visualization management system for pipeline construction progress. Background Technology

[0002] Pipeline construction is a crucial link in the installation of pipeline systems within building construction projects. It is widely used in various new and expansion projects, including industrial plants, commercial complexes, municipal utility tunnels, subway stations, petrochemical plants, and long-distance pipelines. At pipeline construction sites, there are numerous and diverse pipeline components. These components are connected via flanges, welds, or threads, forming a complex spatial topology network. There are also tight assembly constraints between construction procedures. Effective management of the construction schedule directly impacts the overall project's timeline, cost control, and quality and safety assurance.

[0003] In current pipeline construction progress management practices, construction managers typically conduct regular site inspections and manually record the types and quantities of installed pipeline components, then manually compare the records with a pre-prepared construction schedule. This method, relying on manual inspections and paper or spreadsheet records, suffers from significant delays and is prone to errors. Firstly, the complex environment and dispersed work areas at construction sites make it difficult for managers to comprehensively grasp the progress of all work areas at any given time, resulting in progress data often lagging behind the actual construction status by hours or even days. Secondly, the manual recording process is affected by the experience level and subjective judgment of the recorders, easily leading to misrecording of pipeline component types or omissions in installation status, making it difficult to guarantee the accuracy of progress statistics. When construction conditions change due to design changes, material delivery delays, equipment failures, or human resource allocation, the original schedule needs to be adjusted accordingly. In existing technologies, schedule adjustments mainly rely on construction managers judging the scope of pipeline components affected by changes based on their personal experience and manually modifying the time parameters of each process in Gantt charts or network diagrams. However, the spatial assembly constraints between pipe components differ from the floor construction logic in traditional civil engineering. The constraint transmission paths and influence ranges are often implicit in the geometric topology of the three-dimensional pipe network, and cannot be fully and accurately reflected solely by the task prerequisite relationships in a two-dimensional Gantt chart. When construction managers manually adjust plans, they may easily overlook certain pipe components with spatial interference relationships, leading to idle work during actual execution due to the inability to carry out subsequent procedures because the prerequisite components have not been installed.

[0004] Some engineering projects have attempted to introduce BIM technology for construction progress simulation and visualization. Specifically, before construction, piping components in the BIM model are manually associated with task items in the schedule. During construction, the progress status of each piping component is manually updated to drive changes in the component colors in the 3D model. However, this approach still doesn't solve the core problem of low automation in progress data acquisition; progress status updates still rely on manual input, and the real-time performance and accuracy of the data are not fundamentally improved. Furthermore, the association between the BIM model and the schedule remains static during construction. When design changes lead to the addition, deletion, or spatial relocation of piping components in the BIM model, the original component-task associations need to be manually re-established and re-bound, and the maintenance workload increases dramatically with the frequency of changes. In addition, existing BIM progress visualization systems can only present a binary state of completed or incomplete piping components, unable to automatically deduce the optimal installation sequence of subsequently installed components when construction conditions change, and unable to provide actionable decision support for resource allocation and work sequence arrangement on the construction site.

[0005] In summary, existing technologies in pipeline construction have significant shortcomings in three aspects: automated progress data acquisition, automatic analysis of spatial constraints, and automatic adjustment of the schedule after changes in construction conditions. A technical solution is needed to effectively meet key requirements such as recognizing real-time images of the construction site, analyzing BIM spatial topology, and intelligently planning the construction schedule. Summary of the Invention

[0006] This invention enables the automatic collection of construction progress data, the automatic deduction of spatial constraint relationships, and the automatic adjustment of installation sequence, thereby improving the intuitiveness and responsiveness of progress management.

[0007] To achieve the above objectives, the present invention adopts the following solution: A BIM-based pipeline construction progress visualization management system, which includes: An image acquisition device that acquires image data of pipe components at the construction site; The data storage unit stores the BIM 3D geometric data, BIM attribute data, planned installation time data, and actual installation status data of the pipeline components. The semantic segmentation network receives image data output by the image acquisition device, identifies the type and installation status of pipe components in the image data, retrieves the corresponding pipe component in the data storage unit according to the identified pipe component type, writes the identified pipe component installation status as the actual installation status data into the data storage unit and associates it with the corresponding pipe component. The graph construction module obtains the BIM three-dimensional geometric data of each pipe component from the data storage unit, and constructs graph structure data reflecting the spatial constraint relationship between the pipe components based on the spatial connection relationship between each pipe component in the BIM three-dimensional geometric data. The graph neural network module receives graph structure data, obtains the actual installation status data of each pipe component from the data storage unit, inputs the graph structure data with the node status of the installed pipe component as the constraint condition, calculates the node status propagation result of the uninstalled pipe component under the spatial constraint relationship affected by the change of construction conditions, and outputs the spatial constraint propagation path data of the uninstalled pipe component. The schedule replanning module receives and adjusts the installation sequence of the uninstalled pipe components according to the spatial constraint propagation path data, generates updated installation sequence data, and writes it into the data storage unit. The visualization rendering module reads the actual installation status data and updated installation sequence data corresponding to each pipe component from the data storage unit, and performs differentiated rendering of each pipe component in the 3D scene.

[0008] Preferably, the semantic segmentation network adopts an encoder-decoder architecture, wherein the encoder includes multiple downsampling convolutional layers for extracting multi-scale feature maps of pipe components in the image data; the decoder includes multiple upsampling convolutional layers for restoring the multi-scale feature maps to the resolution of the input image and outputting pixel-by-pixel classification results. During the training phase, the semantic segmentation network is jointly trained using a synthetic image dataset generated by rendering BIM 3D geometric data and a real image dataset from the construction site. The synthetic image dataset is generated by simulating 2D rendered images of pipe components under different lighting conditions, different degrees of occlusion, and different construction progress stages in a 3D virtual environment. The real image dataset is obtained by acquiring images from the construction site using an image acquisition device and manually annotating them. During the training process, a domain-adaptive loss function is used to reduce the difference in feature distribution between the synthetic image dataset and the real image dataset.

[0009] Preferably, the domain adaptive loss function includes a feature extractor, a classifier, and a domain discriminator. The feature extractor extracts multi-scale feature maps from the input image data, the classifier outputs pixel-level prediction results of pipe component type and installation status based on the multi-scale feature maps, and the domain discriminator determines the domain label prediction results of whether the input image data comes from a synthetic image dataset or a real image dataset based on the multi-scale feature maps and outputs them. During training, the domain adaptive loss function is composed of a weighted sum of a classification loss term and a domain adversarial loss term. The classification loss term is used to calculate the cross-entropy between the predicted result output by the classifier and the labeled ground truth. The domain adversarial loss term is used to calculate the cross-entropy between the domain label predicted result output by the domain discriminator and the domain label ground truth. During backpropagation, the gradient inversion layer applies a reverse gradient to the feature extractor, so that the multi-scale feature maps learned by the feature extractor tend to be consistent in distribution between the synthetic image dataset and the real image dataset.

[0010] Preferably, the graph neural network module includes a graph attention convolutional layer and a node state propagation layer; The graph attention convolutional layer receives graph structure data output by the graph construction module. In the graph structure data, nodes represent pipe components, and edges represent the spatial connection relationships between pipe components. The graph attention convolutional layer calculates the attention weight of each edge based on the node features at both ends of the edge. The specific attention weight is associated with the connection type corresponding to the spatial connection relationship. The connection types include flange connection, welded connection, and threaded connection. The node state propagation layer uses the node state corresponding to the installed pipe components as the initial constraint condition, uses the attention weight as the transmission strength coefficient of spatial constraint propagation, and propagates the node state layer by layer along the edges in the graph structure data, outputting the spatial constraint propagation path data corresponding to the uninstalled pipe components.

[0011] As a preferred embodiment, the graph attention convolutional layer introduces a regularization term when calculating the attention weight of each edge. The regularization term assigns a prior weight of connection type to each edge according to the connection type corresponding to the spatial connection relationship. The prior weight of connection type corresponding to flange connection is a first numerical range, the prior weight of connection type corresponding to welded connection is a second numerical range, and the prior weight of connection type corresponding to threaded connection is a third numerical range. The lower limit of the first numerical range is higher than the upper limit of the second numerical range, and the lower limit of the second numerical range is higher than the upper limit of the third numerical range. The graph attention convolutional layer performs a weighted sum of the calculated attention weights and the corresponding prior weights of the connection types, and then outputs the normalized attention weights through an activation function. The normalized attention weights are used as the propagation strength coefficients of spatial constraint propagation and input to the node state propagation layer.

[0012] As a preferred option, the schedule replanning module has a built-in resource constraint table, which records the number of types of machinery and work teams required for the installation of various types of pipe components. The schedule replanning module retrieves the current number of available machines and the current number of available work teams from the data storage unit. It then performs a topology sorting and iterative filtering on all uninstalled pipe components involved in the spatial constraint propagation path data. Each round of iterative filtering includes: Extract the set of components with an in-degree of zero from the spatial constraint propagation path from the remaining uninstalled pipe components. Remove pipe components from the set of components with an in-degree of zero that require more types of machinery than the current number of available machinery or more types of work teams than the current number of available work teams. Mark the remaining pipe components as members of the next batch of installation sequence, update the current number of available machinery and work teams, and remove the marked pipe components from the remaining uninstalled pipe components before entering the next round of iteration and screening, until all uninstalled pipe components are marked to the corresponding installation batch. The schedule replanning module generates updated installation sequence data based on each installation batch.

[0013] As a preferred option, the visualization rendering module has a built-in rendering rule mapping table. In the rendering rule mapping table, the actual installation status data is defined, with the first color value corresponding to the installation completed status, the second color value corresponding to the installation delayed status, and the third color value corresponding to the non-installed status. The actual installation status of each pipe component is mapped to the corresponding color value according to the rendering rule mapping table. The visualization rendering module also extracts the installation batch number of each pipe component from the updated installation sequence data, calculates the transparency value of each pipe component based on the installation batch number, and the earlier the installation batch number, the lower the corresponding transparency value. The color value and transparency value are superimposed and applied to the material properties of the corresponding pipe component in the 3D scene. In the 3D scene, the actual installation status is distinguished by color and the updated installation sequence is distinguished by transparency.

[0014] As a preferred embodiment, the image acquisition device includes a visible light camera mounted on a drone and multiple network cameras deployed at fixed locations on the construction site. The visible light camera mounted on the drone acquires first-view image data of the pipeline components according to a preset inspection route, and the multiple network cameras acquire second-view image data of the pipeline components at fixed shooting angles. The image acquisition device also includes an image registration unit. The image registration unit performs feature point matching on the first-view image data and the second-view image data acquired at the same time and calculates the projection transformation matrix. Based on the projection transformation matrix, the first-view image data and the second-view image data are mapped to a unified image coordinate system and then image stitching is performed to generate a full view image data of the construction site without any viewpoint occlusion blind spots. The full view image data of the construction site is then sent to the semantic segmentation network as the image data output by the image acquisition device.

[0015] Preferably, the data storage unit includes a BIM basic data partition, a progress snapshot partition, and a change log partition. The BIM basic data partition stores the initial version of the BIM 3D geometric data and BIM attribute data of the piping components. The progress snapshot partition stores the planned installation time data of each piping component and the actual installation status data of each time node. The change log partition records the timestamp, the value before modification, and the value after modification of each modification operation performed on the BIM 3D geometric data or BIM attribute data of the piping components in the BIM basic data partition. After receiving externally input BIM design change data, the data storage unit first copies the original data of the modified pipe components in the current BIM basic data partition to the change log partition, and then applies the modification operation to the BIM basic data partition. After each modification operation in the BIM basic data partition of the data storage unit, the graph construction module re-acquires the BIM three-dimensional geometric data of the pipe components affected by the modification operation and locally updates the corresponding nodes and edges in the graph structure data.

[0016] Preferably, the graph construction module obtains the BIM three-dimensional geometric data and BIM attribute data of each pipe component from the data storage unit. The BIM attribute data includes the coordinates of the connection ports, the orientation vector of the connection ports, and the specification parameters of the connection ports of each pipe component. The graph construction module calculates the three-dimensional spatial distance between any two pipe component connection ports based on the connection port coordinates. Two connection ports with a three-dimensional spatial distance less than a preset distance threshold are marked as candidate connection pairs. For each candidate connection pair, the module checks whether the orientation vectors of the connection ports meet the condition of being collinear in opposite directions and whether the connection port specifications match. Only candidate connection pairs that simultaneously meet the conditions of having a three-dimensional spatial distance less than the preset distance threshold, having orientation vectors of the connection ports being collinear in opposite directions, and having matching connection port specifications are determined as valid spatial connection relationships. Based on the valid spatial connection relationships, edges are established between the nodes of the corresponding pipe components in the graph structure data.

[0017] The present invention includes at least the following beneficial effects: (1) Automatic identification of image data collected at the construction site through semantic segmentation network to obtain the type and installation status of pipeline components, replacing the traditional manual inspection and manual recording method, significantly reducing human error and information lag in the progress data collection process, enabling construction management personnel to grasp the overall picture of the actual progress at the site with higher frequency and lower delay; (2) Automatic construction of graph structure data reflecting the spatial constraint relationship of pipeline components based on the matching judgment of the connection port coordinates, orientation vectors and specification parameters in the BIM three-dimensional geometric data through graph construction module, avoiding the tedious operation and potential omission of manually specifying the connection relationship one by one in the BIM model, and improving the integrity and accuracy of the construction of the spatial constraint topology; (3) Automatic acquisition of the propagation path and influence intensity of the impact of construction condition changes on the uninstalled pipeline components by performing node state propagation calculation on the graph structure data with the installed pipeline components as the initial constraint conditions through graph neural network module. The sorting overcomes the technical defects of existing technologies that cannot reflect the three-dimensional spatial assembly constraints of pipeline components, and provides a decision basis that is more in line with the actual physical constraints for schedule adjustment; (4) By introducing the restrictions on the number of mechanical types and the number of work groups in the resource constraint table on the basis of the spatial constraint propagation path through the schedule replanning module, the updated installation sequence data that takes into account both spatial constraints and current available resource restrictions is generated by the topological sorting iterative screening mechanism, which effectively avoids the problem of the plan being unexecutable due to the neglect of on-site resource supply capacity when generating the plan based solely on spatial constraints in the existing method; (5) By using the visualization rendering module to distinguish the actual installation status of each pipeline component in the same three-dimensional scene with color coding and to distinguish the updated installation sequence with transparency coding, the construction management personnel can intuitively grasp the current progress status and subsequent construction priority ranking without switching views or consulting additional data tables, reducing the cognitive load of multi-source information integration and improving the response speed of schedule management decisions. Attached Figure Description

[0018] Figure 1 This is a system principle block diagram of the present invention. Detailed Implementation

[0019] The present invention will now be described in further detail with reference to the accompanying drawings, so that those skilled in the art can implement it based on the description.

[0020] like Figure 1 As shown, the BIM-based pipeline construction progress visualization management system provided by this invention includes: An image acquisition device that acquires image data of pipe components at the construction site; The data storage unit stores the BIM 3D geometric data, BIM attribute data, planned installation time data, and actual installation status data of the pipeline components. The semantic segmentation network receives image data output by the image acquisition device, identifies the type and installation status of pipe components in the image data, retrieves the corresponding pipe component in the data storage unit according to the identified pipe component type, writes the identified pipe component installation status as the actual installation status data into the data storage unit and associates it with the corresponding pipe component. The graph construction module obtains the BIM three-dimensional geometric data of each pipe component from the data storage unit, and constructs graph structure data reflecting the spatial constraint relationship between the pipe components based on the spatial connection relationship between each pipe component in the BIM three-dimensional geometric data. The graph neural network module receives graph structure data, obtains the actual installation status data of each pipe component from the data storage unit, inputs the graph structure data with the node status of the installed pipe component as the constraint condition, calculates the node status propagation result of the uninstalled pipe component under the spatial constraint relationship affected by the change of construction conditions, and outputs the spatial constraint propagation path data of the uninstalled pipe component. The schedule replanning module receives and adjusts the installation sequence of the uninstalled pipe components according to the spatial constraint propagation path data, generates updated installation sequence data, and writes it into the data storage unit. The visualization rendering module reads the actual installation status data and updated installation sequence data corresponding to each pipe component from the data storage unit, and performs differentiated rendering of each pipe component in the 3D scene.

[0021] The image acquisition device is used to acquire image data of pipe components at the construction site. It can use a visible light camera mounted on a drone to periodically acquire image data along a preset inspection route, or it can use multiple network cameras deployed at fixed locations at the construction site to continuously acquire image data at fixed shooting angles. The image acquisition device outputs the acquired image data to the subsequent processing module at a preset resolution and frame rate. The data storage unit stores the BIM 3D geometric data, BIM attribute data, planned installation time data, and actual installation status data of the pipe components. The BIM 3D geometric data includes the 3D spatial coordinates, geometric dimensions, and spatial orientation information of the pipe components. The BIM attribute data includes the type identifier, specifications, and connection port information of the pipe components. The planned installation time data records the preset installation date or preset installation time period for each pipe component in the construction schedule. The actual installation status data records the current installation progress status of each pipe component as identified and confirmed during construction, including three values: installation completed, installation delayed, and not installed.

[0022] The semantic segmentation network is built upon a convolutional neural network and employs an encoder-decoder architecture. The encoder extracts multi-scale feature maps of pipe components from the image data through multiple downsampling convolutional layers. The decoder restores the multi-scale feature maps to the original resolution of the input image data through multiple upsampling convolutional layers and outputs pixel-by-pixel classification results. The pixel-by-pixel classification results output by the semantic segmentation network include the pipe component type and installation status of each pixel. Pipe component types include straight pipe sections, elbows, tees, flanges, valves, etc., and installation status includes installed, not installed, and partially installed. Based on the identified pipe component type, the semantic segmentation network retrieves the corresponding pipe component in the data storage unit. The retrieval process is achieved by matching the identified pipe component type with the type identifier contained in the BIM attribute data in the data storage unit. After a successful match, the semantic segmentation network writes the identified pipe component installation status as the actual installation status data into the data storage unit and associates it with the corresponding pipe component.

[0023] The graph construction module obtains the BIM 3D geometric data of each pipe component from the data storage unit. Based on the spatial connection relationship between each pipe component in the BIM 3D geometric data, it constructs graph structure data reflecting the spatial constraint relationship between the pipe components. In the graph structure data, each node corresponds to one pipe component, and each edge corresponds to a valid spatial connection relationship between two pipe components. The determination condition for a valid spatial connection relationship is that the 3D spatial distance between the connection ports of the two pipe components is less than a preset distance threshold, the orientation vectors of the connection ports satisfy the condition of being collinear in opposite directions, and the specifications of the connection ports match each other. The preferred value range of the preset distance threshold is 50-200mm. The graph construction module stores the constructed graph structure data in the data storage unit in the form of an adjacency matrix or an adjacency list.

[0024] The graph neural network module receives the graph structure data output by the graph construction module, obtains the actual installation status data corresponding to each pipe component from the data storage unit, and inputs the graph structure data with the node status corresponding to the installed pipe component as a constraint condition. The node status corresponding to the installed pipe component is set to the active state when the actual installation status data shows that the installation is completed. The graph neural network module propagates the active state to the adjacent inactive node along each edge in the graph structure data. During the propagation process, the graph neural network module calculates the propagation weight based on the spatial constraint strength between nodes. The spatial constraint strength is related to the connection type, which includes flange connection, welded connection, and threaded connection. Different connection types correspond to different propagation weight value ranges. The graph neural network module calculates the node status propagation result of the uninstalled pipe component under the spatial constraint relationship affected by the change in construction conditions, and outputs the spatial constraint propagation path data corresponding to the uninstalled pipe component. The spatial constraint propagation path data indicates the order and degree of influence of the change in construction conditions on the uninstalled pipe component.

[0025] The schedule replanning module receives the spatial constraint propagation path data output by the graph neural network module. Based on the spatial constraint propagation path data, it adjusts the installation order of the uninstalled pipe components. The adjustment process includes performing topological sorting and iterative filtering on all uninstalled pipe components involved in the spatial constraint propagation path data. In each round of iterative filtering, the set of components with an in-degree of zero in the spatial constraint propagation path is extracted from the remaining uninstalled pipe components. The updated installation order data is generated and written to the data storage unit. The updated installation order data includes the installation batch number or installation priority sorting result corresponding to each uninstalled pipe component.

[0026] The visualization rendering module reads the actual installation status data and updated installation sequence data corresponding to each pipe component from the data storage unit. It performs differentiated rendering on each pipe component in the 3D scene. Differentiated rendering includes assigning different color values ​​to pipe components based on the installation completion status, installation delay status, and non-installation status in the actual installation status data. For example, the optional color value corresponding to the installation completion status is green, the optional color value corresponding to the installation delay status is red, and the optional color value corresponding to the non-installation status is gray. The transparency value is calculated based on the installation batch number to which each pipe component belongs in the updated installation sequence data. The earlier the installation batch number, the lower the transparency value, so that the pipe components installed first stand out more in the 3D scene. The later the installation batch number, the higher the transparency value. The visualization rendering module overlays the color value and transparency value and applies them to the material properties of the corresponding pipe components in the 3D scene and refreshes the display in real time.

[0027] Compared to existing pipeline construction progress management systems, this technical solution automatically identifies the installation status of pipeline components from construction site image data using a semantic segmentation network, replacing manual data entry and significantly reducing human error and time delays in the progress data collection process. The graph construction module and graph neural network module automatically analyze the spatial constraint relationships between pipeline components and calculate the propagation path of the impact of changes in construction conditions on uninstalled pipeline components, replacing decision-making methods relying on the experience of construction management personnel and effectively reducing errors in construction sequence arrangement caused by omissions in spatial constraint judgment. The schedule replanning module automatically generates updated installation sequence data based on the spatial constraint propagation path, replacing manual adjustments to the construction schedule plan and significantly shortening the response time for schedule replanning after changes in construction conditions. The visualization rendering module simultaneously displays the actual installation status and updated installation sequence of each pipeline component in a 3D scene, enabling construction management personnel to more intuitively grasp the overall construction progress and subsequent construction priorities, further improving the precision and efficiency of construction progress management.

[0028] In another technical solution, the semantic segmentation network adopts an encoder-decoder architecture, where the encoder includes multiple downsampling convolutional layers to extract multi-scale feature maps of pipe components in the image data; the decoder includes multiple upsampling convolutional layers to restore the multi-scale feature maps to the input image resolution and output pixel-by-pixel classification results. During the training phase, the semantic segmentation network is jointly trained using a synthetic image dataset generated by rendering BIM 3D geometric data and a real image dataset from the construction site. The synthetic image dataset is generated by simulating 2D rendered images of pipe components under different lighting conditions, different degrees of occlusion, and different construction progress stages in a 3D virtual environment. The real image dataset is obtained by acquiring images from the construction site using an image acquisition device and manually annotating them. During the training process, a domain-adaptive loss function is used to reduce the difference in feature distribution between the synthetic image dataset and the real image dataset.

[0029] The semantic segmentation network employs an encoder-decoder architecture. The encoder contains multiple downsampling convolutional layers. These layers progressively reduce the spatial resolution of the input image data and increase the number of feature channels through convolution operations with a stride of two or pooling operations. The encoder extracts multi-scale feature maps of pipe components from the image data. The feature maps extracted by shallow downsampling convolutional layers contain the edge contours and local texture information of the pipe components, while the feature maps extracted by deep downsampling convolutional layers contain the semantic category and global context information of the pipe components. The decoder contains multiple upsampling convolutional layers. These layers progressively restore the spatial resolution of the feature maps to the original size of the input image data through transpose convolution operations or bilinear interpolation combined with convolution operations. The decoder inputs the restored feature maps into the classification layer and outputs pixel-by-pixel classification results. In the pixel-by-pixel classification results, each pixel position corresponds to a pipe component type label and a pipe component installation status label.

[0030] During the training phase, the semantic segmentation network is jointly trained using a synthetic image dataset generated from BIM 3D geometric data rendering and a dataset of real images from the construction site. The synthetic image dataset is generated by simulating 2D rendered images of pipe components under different lighting conditions, occlusion levels, and construction progress stages in a 3D virtual environment. For example, optional simulation parameters for lighting conditions include a magnification range of point light source intensity from 0.5 to 2.0 and an intensity range of ambient light intensity from 0.2 to 0.8. Optional simulation parameters for occlusion levels include the number of randomly generated occluding objects from 0 to 5 and the proportion of occluding objects in the image from 5% to 40%. Optional simulation parameters for construction progress stages include the percentage of pipe component installation completion increasing from 0 to 100% in 25% increments. The real image dataset is obtained by acquiring construction site images using an image acquisition device and manually annotating them. The manual annotation process includes using annotation tools to draw polygonal regions along the pipe component outline in each construction site image and assigning pipe component type labels and pipe component installation status labels to each polygonal region. During training, a domain-adaptive loss function is employed to reduce the feature distribution differences between the synthetic image dataset and the real image dataset. This domain-adaptive loss function consists of a weighted sum of a classification loss term and a domain-adversarial loss term. The classification loss term calculates the cross-entropy loss between the pixel-by-pixel classification result output by the semantic segmentation network and the labeled ground truth value. The domain-adversarial loss term uses a domain discriminator to determine whether the feature map originates from the synthetic or real image dataset and calculates the domain classification cross-entropy loss. During backpropagation, the domain-adversarial loss term applies a backward gradient to the feature extractor through a gradient inversion layer to confuse the domain discriminator. By jointly training the semantic segmentation network using both synthetic and real image datasets, the problem of insufficient labeled samples from real construction site images, which limits the generalization ability of the semantic segmentation network, is effectively solved. The introduction of the domain-adaptive loss function allows the pipe component recognition ability learned by the semantic segmentation network on the synthetic image dataset to be more smoothly transferred to real construction site scenarios, significantly reducing the negative impact of domain offset between synthetic and real images on recognition accuracy. The synthetic image dataset is automatically generated by simulating diverse lighting conditions, occlusion levels, and construction progress stages in a 3D virtual environment. This significantly reduces the time and manpower costs required for manual data collection and annotation, while also improving the adaptability and robustness of the semantic segmentation network to the complex and ever-changing environment of the construction site.

[0031] In another technical solution, the domain adaptive loss function includes a feature extractor, a classifier, and a domain discriminator. The feature extractor extracts multi-scale feature maps from the input image data. The classifier outputs pixel-level prediction results of pipe component type and installation status based on the multi-scale feature maps. The domain discriminator determines the domain label prediction results of whether the input image data comes from a synthetic image dataset or a real image dataset based on the multi-scale feature maps and outputs them. During training, the domain adaptive loss function is composed of a weighted sum of a classification loss term and a domain adversarial loss term. The classification loss term is used to calculate the cross-entropy between the predicted result output by the classifier and the labeled ground truth. The domain adversarial loss term is used to calculate the cross-entropy between the domain label predicted result output by the domain discriminator and the domain label ground truth. During backpropagation, the gradient inversion layer applies a reverse gradient to the feature extractor, so that the multi-scale feature maps learned by the feature extractor tend to be consistent in distribution between the synthetic image dataset and the real image dataset.

[0032] The domain-adaptive loss function comprises three sub-network structures: a feature extractor, a classifier, and a domain discriminator. The feature extractor extracts multi-scale feature maps from the input image data. In its implementation, the feature extractor shares network parameters and computational processes with the encoder part of the semantic segmentation network. The feature extractor receives panoramic images of the construction site from an image acquisition device or rendered image data from a synthetic image dataset. After cascading processing by multiple downsampling convolutional layers and nonlinear activation layers, it outputs multi-scale feature maps containing different spatial resolutions and semantic abstraction levels. The classifier outputs pixel-level predictions of pipe component types and installation states based on the multi-scale feature maps output by the feature extractor. In its implementation, the classifier shares network parameters and computational processes with the decoder part of the semantic segmentation network. The classifier uses multiple upsampling convolutional layers to restore the multi-scale feature maps to the original spatial resolution of the input image data and outputs a multi-dimensional probability vector at each pixel location. The dimension of the multi-dimensional probability vector is equal to the product of the number of pipe component type categories and the number of pipe component installation state categories.

[0033] The domain discriminator outputs the domain label prediction results of the input image data, which originates from either a synthetic or real image dataset, based on the multi-scale feature map output by the feature extractor. In its implementation, the domain discriminator is a binary classification network composed of several fully connected layers and non-linear activation layers. After receiving the multi-scale feature map from the feature extractor and compressing it into a one-dimensional feature vector through global average pooling, the domain discriminator outputs a binary probability value representing an estimate of the probability that the input image data comes from a real image dataset. During training, the domain adaptive loss function is composed of a weighted sum of a classification loss term and a domain adversarial loss term. The preferred value range for the weighting coefficient between the classification loss term and the domain adversarial loss term is 0.5-2.0 to adjust the training emphasis on balancing semantic segmentation accuracy and domain invariance.

[0034] The classification loss term calculates the cross-entropy between the pixel-level predictions of pipe component types and installation states output by the classifier and the ground truth labels. For synthetic image datasets, the ground truth labels are automatically generated pixel-by-pixel semantic labels during the rendering of images from a 3D virtual environment; for real image datasets, they are polygon region labels drawn during manual annotation. The domain adversarial loss term calculates the cross-entropy between the domain label predictions output by the domain discriminator and the ground truth domain labels. During backpropagation, this cross-entropy is applied to the feature extractor via a gradient reversal layer. The gradient reversal layer performs an identity mapping operation during forward propagation to maintain input-output consistency, and during backpropagation, it multiplies the input gradient by a scaling factor of -1 to reverse the gradient direction. The backpropagation mechanism of the domain adversarial loss term causes the parameter update direction of the feature extractor to shift towards increasing the classification error of the domain discriminator while decreasing the classification loss term. This results in a more consistent distribution of the multi-scale feature maps learned by the feature extractor between the synthetic and real image datasets.

[0035] This approach employs a gradient reversal layer mechanism within the domain-adaptive loss function to impose domain adversarial training constraints on the feature extractor during the training process of the semantic segmentation network. This effectively addresses the domain shift problem, where the model's generalization ability on real construction site images is severely reduced when training the semantic segmentation network using only synthetic image datasets. Compared to traditional transfer learning methods that pre-train on synthetic image datasets and then fine-tune on a small number of real images, this approach, through end-to-end joint training of the domain adversarial loss term, enables the feature extractor to actively suppress domain-related information during the feature representation learning stage while retaining semantic information related to pipe component type and installation status. This significantly improves the recognition accuracy and robustness of the semantic segmentation network on real construction site images, while drastically reducing the dependence on the number of labeled real image samples.

[0036] In another technical solution, the graph neural network module includes a graph attention convolutional layer and a node state propagation layer; The graph attention convolutional layer receives graph structure data output by the graph construction module. In the graph structure data, nodes represent pipe components, and edges represent the spatial connection relationships between pipe components. The graph attention convolutional layer calculates the attention weight of each edge based on the node features at both ends of the edge. The specific attention weight is associated with the connection type corresponding to the spatial connection relationship. The connection types include flange connection, welded connection, and threaded connection. The node state propagation layer uses the node state corresponding to the installed pipe components as the initial constraint condition, uses the attention weight as the transmission strength coefficient of spatial constraint propagation, and propagates the node state layer by layer along the edges in the graph structure data, outputting the spatial constraint propagation path data corresponding to the uninstalled pipe components.

[0037] The graph attention convolutional layer receives graph structure data output from the graph construction module. In this data, nodes represent pipe components, each with a type feature vector and a geometric feature vector. The type feature vector is generated by one-hot encoding or embedding encoding of the pipe component type. The geometric feature vector is formed by concatenating the pipe component's 3D spatial coordinates with the orientation vector of its connection port. Edges represent the spatial connections between pipe components, each with a connection type identifier and a matching degree value for the connection port specification. The graph attention convolutional layer calculates the attention weight for each edge based on the node features at both ends. The attention weight reflects the strength of the mutual influence between the nodes at both ends of the edge during spatial constraint propagation. The calculation process includes: concatenating the node features at both ends of the edge and inputting them into a single-layer fully connected network to output the original attention score; normalizing the original attention score using an activation function to obtain the attention weight, which ranges from 0 to 1. Attention weights are associated with the connection types corresponding to spatial connectivity. Connection types include flange connections, welded connections, and threaded connections. Flange connections provide a detachable rigid constraint between two pipe components and have a high bending moment transfer capacity. Welded connections provide a non-detachable continuous rigid constraint between two pipe components and have the highest bending moment and shear force transfer capacity. Threaded connections provide a detachable axial constraint between two pipe components and have a low bending moment transfer capacity. Different connection types correspond to different initial attention bias values ​​in the graph attention convolutional layer. The node state propagation layer uses the node states corresponding to the installed pipe components as initial constraints. The nodes of the installed pipe components are marked as completed in the actual installation state data, and their node state values ​​are set to 1. The node state values ​​of the other nodes without installed pipe components are set to 0. The node state propagation layer uses the attention weights calculated by the graph attention convolution layer as the propagation strength coefficients of spatial constraints. It propagates the node states layer by layer along the edges in the graph structure data. In each layer of propagation, the node state value is multiplied by the corresponding attention weight in the outgoing edge direction and then accumulated to the target node. The propagation process is iteratively executed until the node state values ​​converge or the preset propagation round is reached. The suggested value range for the preset propagation round is 3-8 times. The node state propagation layer outputs the spatial constraint propagation path data corresponding to the uninstalled pipe components.

[0038] This solution automatically learns the importance weights of spatial connections between pipe components through graph attention convolutional layers, replacing the traditional method that relies on manual experience to preset fixed propagation coefficients. This allows the spatial constraint propagation process to more accurately reflect the differences in physical constraints of different types of pipe connections during actual construction. The node state propagation layer iteratively propagates along the graph structure using the actual state of the installed pipe components as the initial condition. It can automatically deduce the propagation path and intensity ranking of the impact of changes in construction conditions on complex pipe networks. This effectively overcomes the limitations of existing technologies that rely solely on Gantt chart logical relationships and cannot reflect the three-dimensional spatial assembly constraints of pipe components. It provides the schedule replanning module with a decision-making basis that is more consistent with the actual physical constraints of construction.

[0039] In another technical solution, the graph attention convolutional layer introduces a regularization term when calculating the attention weight of each edge. The regularization term assigns a prior weight of connection type to each edge according to the connection type corresponding to the spatial connection relationship. The prior weight of connection type corresponding to flange connection is a first numerical range, the prior weight of connection type corresponding to welded connection is a second numerical range, and the prior weight of connection type corresponding to threaded connection is a third numerical range. The lower limit of the first numerical range is higher than the upper limit of the second numerical range, and the lower limit of the second numerical range is higher than the upper limit of the third numerical range. The graph attention convolutional layer performs a weighted sum of the calculated attention weights and the corresponding prior weights of the connection types, and then outputs the normalized attention weights through an activation function. The normalized attention weights are used as the propagation strength coefficients of spatial constraint propagation and input to the node state propagation layer.

[0040] The regularization term assigns a prior weight to each edge based on the connection type corresponding to the spatial connection relationship. This prior weight reflects the inherent differences in the degree of influence of different pipe connection methods on the propagation of spatial constraints during construction. Flange connections achieve a detachable rigid connection between two pipe components by bolting the flange and have a high bending moment transmission capacity. The prior weight for flange connections falls within the first numerical range, with selectable values ​​between 0.7 and 0.9. Welded connections form a non-detachable continuous rigid connection between two pipe components through metal melting and solidification, and have the highest bending moment and shear force transmission capacity. The prior weight for welded connections falls within the second numerical range, with selectable values ​​between 0.4 and 0.6. Threaded connections achieve a detachable axial connection between two pipe components through thread engagement and have a lower bending moment transmission capacity. The prior weight for threaded connections falls within the third numerical range, with selectable values ​​between 0.1 and 0.3. The lower limit of the first numerical range is higher than the upper limit of the second numerical range, and the lower limit of the second numerical range is higher than the upper limit of the third numerical range, so that the basic influence of different connection types in the process of spatial constraint propagation forms a decreasing hierarchical gradient.

[0041] The graph attention convolutional layer performs a weighted summation of the calculated attention weights and the corresponding prior weights for connection types. The contribution ratio of each weight during the weighted summation is controlled by a hyperparameter balancing coefficient, which can be selected between 0.3 and 0.7 to achieve a balance between data-driven attention learning results and prior engineering knowledge. The weighted summation result is then processed by an activation function to output normalized attention weights. The activation function can use a flexible maximum transfer function to map the weighted summation result to a probability distribution space between zero and one. The normalized attention weights are then used as the propagation strength coefficients for spatial constraint propagation and input to the node state propagation layer. A larger propagation strength coefficient indicates a more significant impact of the edge on adjacent nodes during node state propagation. By introducing a connection type-based regularization term and connection type prior weights into the attention weight calculation of the graph attention convolutional layer, the problem of existing graph attention networks exhibiting discrepancies between attention weight learning results and actual physical constraints in pipeline component spatial constraint propagation scenarios due to insufficient training data or uneven data distribution is effectively solved. The introduction of regularization terms enables the graph attention convolutional layer to adaptively adjust attention weights by utilizing node feature information from the graph structure data, while also being guided and constrained by the inherent constraint strengths of different connection types in engineering experience. This significantly improves the discriminative power and rationality of attention weights among flange, welded, and threaded connections. By incorporating prior domain knowledge, the calculation results of spatial constraint propagation paths better reflect the actual differences in assembly sequence constraints imposed by different connection methods in pipeline construction engineering practice.

[0042] In another technical solution, the schedule replanning module has a built-in resource constraint table, which records the number of types of machinery and work teams required for the installation of various types of pipe components. The schedule replanning module retrieves the current number of available machines and the current number of available work teams from the data storage unit. It then performs a topology sorting and iterative filtering on all uninstalled pipe components involved in the spatial constraint propagation path data. Each round of iterative filtering includes: Extract the set of components with an in-degree of zero from the spatial constraint propagation path from the remaining uninstalled pipe components. Remove pipe components from the set of components with an in-degree of zero that require more types of machinery than the current number of available machinery or more types of work teams than the current number of available work teams. Mark the remaining pipe components as members of the next batch of installation sequence, update the current number of available machinery and work teams, and remove the marked pipe components from the remaining uninstalled pipe components before entering the next round of iteration and screening, until all uninstalled pipe components are marked to the corresponding installation batch. The schedule replanning module generates updated installation sequence data based on each installation batch.

[0043] The schedule replanning module receives spatial constraint propagation path data output by the graph neural network module. This data includes the influence propagation hierarchy and node state propagation results of each uninstalled pipe component under spatial constraints. The schedule replanning module has a built-in resource constraint table, which records the number of machine types and work teams required for the installation of each type of pipe component in a structured key-value pair format. The number of machine types includes one or more of the following: the number of cranes, welding machines, and cutting machines. The number of work teams includes one or more of the following: the number of welders, pipe fitters, and crane operators. Each type of pipe component in the resource constraint table corresponds to one resource requirement record, which is pre-written during the system initialization phase based on the resource allocation plan in the construction organization design document. The schedule replanning module obtains the current available machinery quantity data and the current available work team quantity data from the data storage unit. The current available machinery quantity data records the number of available machines of various types that can be called up on the construction day in the form of a list of machinery equipment numbers. The current available work team quantity data records the number of available personnel of various types of work teams that can be put into construction on the construction day in the form of a list of work team numbers. The current available machinery quantity data and the current available work team quantity data are synchronized to the data storage unit in real time by the construction site resource management system through the data interface.

[0044] The schedule replanning module performs topological sorting and iterative filtering on all uninstalled pipe components involved in the spatial constraint propagation path data. This iterative filtering is performed in rounds until all uninstalled pipe components are marked to their corresponding installation batches. Each round of iterative filtering first extracts the set of components with an in-degree of zero from the remaining uninstalled pipe components in the spatial constraint propagation path. An in-degree of zero indicates that no preceding uninstalled pipe component in the spatial constraint propagation path poses a spatial constraint obstacle to this pipe component; that is, all preceding dependent components of this pipe component have met the installation conditions or have been marked as installed in terms of spatial constraint relationships. In the component set with zero in-degree, the schedule replanning module checks each pipe component to see if the required number of machine types and work teams exceeds the corresponding available quantities in the current available machine and work team data. The check process involves retrieving the corresponding resource requirement record for each pipe component type from the resource constraint table and comparing the required quantity with the current available quantity. If the required quantity of any machine type exceeds the current available machine quantity or the required quantity of any work team type exceeds the current available work team quantity, the pipe component is removed from the component set with zero in-degree. The remaining pipe components are marked as members of the next batch of installation sequences. The marking operation includes assigning an installation batch number to each marked pipe component and writing the installation batch number to the data storage unit. The installation batch number starts from 1 and increments. All marked pipe components in the same iteration share the same installation batch number. The schedule replanning module updates the current available machine and work team data. The update operation deducts the machine and work team quantities consumed by the marked pipe components in this batch from the current available quantities to reflect resource occupancy. The schedule replanning module removes the marked pipe components from the remaining uninstalled pipe components and then proceeds to the next round of iterative filtering. The removal operation deletes the corresponding nodes of the marked pipe components from the node set of the spatial constraint propagation path and updates the in-degree count of the remaining nodes. Iterative filtering continues until all uninstalled pipe components are marked to their corresponding installation batches. The schedule replanning module generates updated installation sequence data based on the marking order of pipe components in each installation batch. The updated installation sequence data is written to the data storage unit in batch sequence form for the visualization rendering module to read.

[0045] This solution introduces a resource constraint table to perform a secondary screening of the installation sequence based on the spatial constraint propagation path through a schedule replanning module. This effectively solves the problem of unexecutable schedules caused by existing technologies that generate installation sequences solely based on spatial constraints while ignoring the actual resource supply capacity at the construction site. The topology sorting iterative screening mechanism ensures that the generation of the installation sequence follows both the spatial assembly constraints between pipe components and the actual supply capacity of currently available machinery and work teams, guaranteeing the practical operability of the updated installation sequence data under the resource conditions at the construction site. Compared to existing methods that rely on manual adjustments to the schedule on Gantt charts, this solution can automatically generate an installation sequence that takes into account both spatial and resource constraints when construction conditions change or resource supply changes. This significantly reduces the manual intervention time required for schedule replanning and lowers the risk of construction delays and idle work caused by resource allocation conflicts.

[0046] In another technical solution, the visualization rendering module has a built-in rendering rule mapping table. In the rendering rule mapping table, the actual installation status data is defined. The installation completed status corresponds to the first color value, the installation delayed status corresponds to the second color value, and the installation not installed status corresponds to the third color value. According to the rendering rule mapping table, the actual installation status of each pipe component is mapped to the corresponding color value. The visualization rendering module also extracts the installation batch number of each pipe component from the updated installation sequence data, calculates the transparency value of each pipe component based on the installation batch number, and the earlier the installation batch number, the lower the corresponding transparency value. The color value and transparency value are superimposed and applied to the material properties of the corresponding pipe component in the 3D scene. In the 3D scene, the actual installation status is distinguished by color and the updated installation sequence is distinguished by transparency.

[0047] The visualization rendering module reads the actual installation status data and updated installation sequence data corresponding to each pipeline component from the data storage unit. In the actual installation status data, the installation completion status indicates that the pipeline component has completed all installation procedures according to the design requirements and has passed the quality acceptance. The installation delay status indicates that the pipeline component has not been installed after the preset installation date corresponding to the planned installation time data and there is a deviation between the current construction progress and the planned progress. The non-installed status indicates that the pipeline component has not yet entered the installation procedure. The visualization rendering module has a built-in rendering rule mapping table. The rendering rule mapping table defines the first color value corresponding to the installation completed state, the second color value corresponding to the installation delayed state, and the third color value corresponding to the installation not installed state in the actual installation status data. For example, the first color value can be selected as pure green with a green channel value of 255 and red and blue channel values ​​of 0 in the RGB color space; the second color value can be selected as pure red with a red channel value of 255 and green and blue channel values ​​of 0 in the RGB color space; and the third color value can be selected as gray with red, green, and blue channel values ​​of 128 in the RGB color space. The rendering rule mapping table stores the mapping relationship between the three status identifiers and the corresponding color values ​​in the form of key-value pairs and is specified by the configuration parameters during the system initialization phase.

[0048] The visualization rendering module maps the actual installation status of each pipe component to its corresponding color value based on the rendering rule mapping table. The mapping process iterates through the unique identifiers of all pipe components in the 3D scene, queries the data storage unit for the actual installation status data corresponding to that pipe component, and overwrites the diffuse color component in the pipe component's material properties with the color value corresponding to that status from the rendering rule mapping table. The visualization rendering module extracts the installation batch number of each pipe component from the updated installation sequence data. This batch number is assigned sequentially to each batch of pipe components by the schedule replanning module during the topology sorting and iterative filtering process and written into the data storage unit. A smaller batch number indicates that the pipe component will be scheduled for construction earlier in the updated installation sequence. The visualization rendering module calculates the transparency value of each pipe component based on the installation batch number. The transparency value ranges from 0 to 1 and shows a monotonically increasing mapping relationship with the installation batch number. The earlier the installation batch number, the lower the corresponding transparency value, so that the pipe components installed first are more opaque and visually more prominent in the 3D scene. The later the installation batch number, the higher the corresponding transparency value, so that the pipe components installed later are more semi-transparent and visually weaker in the 3D scene.

[0049] The visualization rendering module overlays color and transparency values ​​onto the material properties of corresponding pipe components in the 3D scene. During the overlay process, the mapped color value is written to the diffuse color channel of the material properties, and the calculated transparency value is written to the opacity channel. The 3D scene rendering pipeline calculates the final pixel output color based on the diffuse color and opacity values ​​in the material properties during the per-piece metashading stage. In the 3D scene, color distinguishes the actual installation status, and transparency distinguishes the updated installation order. Construction managers can intuitively see that a green pipe component has been installed, a red pipe component is lagging behind and requires priority, and a gray pipe component has not yet been installed. For pipe components of the same color, higher opacity indicates a earlier installation order, while lower opacity indicates a later installation order. By overlaying color-coded and transparency-coded dual-channel information in the same 3D scene, construction managers can simultaneously grasp the actual installation progress status and subsequent construction priority of each pipe component without switching views or consulting additional data tables, significantly improving the intuitiveness and efficiency of construction progress information communication. The dual-channel visualization rendering method used in this solution integrates spatial location information, progress status information, and sequence priority information into the same visual screen, effectively reducing the cognitive load on managers to mentally integrate information from multiple sources and reducing delays in progress judgment and decision-making caused by information dispersion.

[0050] In another technical solution, the image acquisition device includes a visible light camera mounted on a drone and multiple network cameras deployed at fixed locations on the construction site. The visible light camera mounted on the drone acquires first-view image data of the pipeline components according to a preset inspection route, and the multiple network cameras acquire second-view image data of the pipeline components at fixed shooting angles. The image acquisition device also includes an image registration unit. The image registration unit performs feature point matching on the first-view image data and the second-view image data acquired at the same time and calculates the projection transformation matrix. Based on the projection transformation matrix, the first-view image data and the second-view image data are mapped to a unified image coordinate system and then image stitching is performed to generate a full view image data of the construction site without any viewpoint occlusion blind spots. The full view image data of the construction site is then sent to the semantic segmentation network as the image data output by the image acquisition device.

[0051] A visible light camera mounted on a drone collects first-view images of the pipeline components along a preset inspection route. This route is pre-planned and generated by the drone's ground station software based on the three-dimensional spatial range of the construction site and the distribution density of the pipeline components. During the drone's flight along the route, the visible light camera is triggered to capture images at preset time intervals, ranging from 5 to 30 seconds. The acquired first-view images include the overall layout information of the pipeline components from a top-down perspective and their spatial position from a high-altitude perspective. Multiple network cameras collect second-view images of the pipeline components at fixed shooting angles. These cameras are fixedly mounted on the top of tower cranes, building corners, or temporary supports at the construction site. Each network camera is set to a fixed shooting angle by adjusting the gimbal's pitch and horizontal rotation angles to cover a specific construction area of ​​the pipeline components. The second-view images include local details of the pipeline components from a close-up horizontal perspective and supplementary perspective information for obstructed areas.

[0052] The image registration unit performs feature point matching on first-view and second-view image data acquired at the same time. Feature point matching employs a scale-invariant feature transformation algorithm or an accelerated robust feature algorithm to extract key point positions and feature description vectors in each image. Matching feature point pairs are found by calculating the Euclidean distance between the feature description vectors in the two images, and a random sampling consensus algorithm is used to eliminate mismatched point pairs to improve matching accuracy. The image registration unit calculates a projection transformation matrix based on the matched feature point pairs. This projection transformation matrix is ​​a 3x3 homography matrix used to describe the two-dimensional planar mapping relationship between the pixel coordinate systems of the first-view and second-view image data. The image registration unit maps the first-view and second-view image data to a unified image coordinate system based on the projection transformation matrix. This unified image coordinate system can be selected as a coordinate system with the webcam image as a reference or a coordinate system with the drone aerial image as a reference. During the mapping process, a perspective transformation operation is performed on the first-view or second-view image data to eliminate geometric distortions caused by perspective differences.

[0053] After completing coordinate system one, image stitching is performed. The image stitching process includes pixel-level fusion processing of the two mapped images in the overlapping area. The fusion processing can use multi-band fusion algorithms or feathering fusion algorithms to eliminate brightness abrupt changes and color differences at the stitching seam. The image registration unit generates a panoramic image of the construction site without any viewing angle occlusion blind spots. The panoramic image data integrates the global view from the drone's high-altitude perspective with the close-up local details from the network camera, and eliminates the observation blind spots caused by the obstruction of pipe components or temporary facilities from a single perspective through multi-view complementarity. The panoramic image data of the construction site is sent as image data output by the image acquisition device to the semantic segmentation network. After receiving the panoramic image, the semantic segmentation network can perform pixel-level recognition and installation status judgment based on the complete and unobstructed visual information of the pipe components. By using a visible light camera mounted on a drone and multiple fixed network cameras to collaboratively acquire multi-view images and generate panoramic image data of the construction site through image registration and stitching, the problem of some pipe components not being effectively observed due to the limited viewing angle of a single image acquisition device or on-site obstruction is effectively solved, resulting in omissions in the semantic segmentation network recognition. Compared with existing technologies that rely solely on single-point monitoring cameras or single drone aerial images, this solution significantly expands the coverage of a single image acquisition through multi-view complementarity and image stitching technology, and greatly reduces the proportion of pipe components missed due to occlusion, thereby improving the completeness and reliability of automatic identification of the installation status of pipe components.

[0054] In another technical solution, the data storage unit includes a BIM basic data partition, a progress snapshot partition, and a change log partition. The BIM basic data partition stores the initial version of the BIM 3D geometric data and BIM attribute data of the piping components. The progress snapshot partition stores the planned installation time data of each piping component and the actual installation status data of each time node. The change log partition records the timestamp, the value before modification, and the value after modification of each modification operation performed on the BIM 3D geometric data or BIM attribute data of the piping components in the BIM basic data partition. After receiving externally input BIM design change data, the data storage unit first copies the original data of the modified pipe components in the current BIM basic data partition to the change log partition, and then applies the modification operation to the BIM basic data partition. After each modification operation in the BIM basic data partition of the data storage unit, the graph construction module re-acquires the BIM three-dimensional geometric data of the pipe components affected by the modification operation and locally updates the corresponding nodes and edges in the graph structure data.

[0055] The data storage unit includes a BIM basic data partition, a progress snapshot partition, and a change log partition. The BIM basic data partition stores the initial version of the BIM 3D geometric data and BIM attribute data of the piping components. The initial version is the first written version after the BIM model data is exported and confirmed before the start of construction. The BIM 3D geometric data uses boundary representation or constructed solid geometry to describe the 3D shape information of the piping components. The BIM attribute data uses structured fields to store the type identifier, specifications, and connection port information of the piping components. The progress snapshot partition stores the planned installation time data and the actual installation status data of each piping component at each time node. The planned installation time data stores the date and timestamp value of the preset installation date, indexed by the unique identifier of the piping component. The actual installation status data stores the status enumeration value and update timestamp of each status change in time series format, indexed by the unique identifier of the piping component. The change log partition records each modification operation performed on the BIM 3D geometric data or BIM attribute data of the pipe components in the BIM base data partition. Each change log record includes the timestamp of the modification operation, the unique identifier of the modified pipe component, the name of the modified field, the value before modification, and the value after modification. The change log partition adopts an append-only data management mode to ensure the complete traceability of the change history.

[0056] The data storage unit receives externally input BIM design change data, which originates from design change documents output by BIM modeling software or change notification messages pushed by the design collaboration platform. This data includes a list of unique identifiers for the modified piping components and the newly added or modified BIM 3D geometric data and BIM attribute data for each component. Upon receiving the BIM design change data, the data storage unit first copies the original data of the modified piping components from the current BIM base data partition to the change log partition. This copying operation is performed under database transaction protection to ensure the complete preservation of the original data in its pre-change state. The data storage unit then applies the modification operations to the BIM base data partition. These operations include updating / replacing or adding / inserting records for the corresponding piping components in the BIM base data partition. Update / replacing operations overwrite the content of existing BIM 3D geometric data fields or BIM attribute data fields, while adding / inserting operations create new data records for the new piping components.

[0057] The graph construction module reacquires the BIM 3D geometric data of the affected pipe components after each modification operation to the BIM basic data partition of the data storage unit. The module detects BIM basic data partition change events by subscribing to the data storage unit's change notification mechanism or periodically polling the latest modification timestamp in the change log partition. The module locates the affected pipe components and their spatially adjacent sets of pipe components. It only recalculates the 3D spatial distances and connection conditions between connection ports for pipe components within the affected area and locally updates the corresponding nodes and edges in the graph structure data, rather than performing a full reconstruction of the graph structure data. The local update operation includes removing and re-adding the affected nodes from the graph structure data and recalculating the edge connections between the node and its neighbors. The feature vectors of the nodes corresponding to the modified pipe components are synchronously updated to reflect their latest BIM 3D geometric data and BIM attribute data.

[0058] This solution achieves separate storage and change tracking of BIM static data and dynamic construction progress data by setting up BIM basic data partitions, progress snapshot partitions, and change log partitions in the data storage unit. The append-only write mechanism of the change log partition ensures the complete recording of the BIM design change history and the ability to restore the data state at any point in time. The graph construction module only performs incremental updates to the graph structure data of local nodes and edges affected by BIM design changes, rather than full reconstruction. This effectively solves the problem of wasted computing resources and response delays caused by the need to rebuild the entire graph structure data after a design change in the existing technology. It significantly shortens the update response time of the graph structure data after a design change and improves the system's adaptability to frequent design change scenarios.

[0059] In another technical solution, the graph construction module obtains the BIM three-dimensional geometric data and BIM attribute data of each pipe component from the data storage unit. The BIM attribute data includes the coordinates of the connection ports, the orientation vector of the connection ports, and the specification parameters of the connection ports of each pipe component. The graph construction module calculates the three-dimensional spatial distance between any two pipe component connection ports based on the connection port coordinates. Two connection ports with a three-dimensional spatial distance less than a preset distance threshold are marked as candidate connection pairs. For each candidate connection pair, the module checks whether the orientation vectors of the connection ports meet the condition of being collinear in opposite directions and whether the connection port specifications match. Only candidate connection pairs that simultaneously meet the conditions of having a three-dimensional spatial distance less than the preset distance threshold, having orientation vectors of the connection ports being collinear in opposite directions, and having matching connection port specifications are determined as valid spatial connection relationships. Based on the valid spatial connection relationships, edges are established between the nodes of the corresponding pipe components in the graph structure data.

[0060] BIM 3D geometric data defines the spatial shape and orientation of pipe components in the form of a 3D spatial coordinate sequence or parametric geometric description. BIM attribute data includes the connection port coordinates, connection port orientation vector, and connection port specification parameters of each pipe component. The connection port coordinates represent the spatial position of the center point of the pipe component interface end face using the X-axis, Y-axis, and Z-axis values ​​in a 3D Cartesian coordinate system. The connection port orientation vector represents the direction of the normal to the pipe component interface end face using a 3D unit vector. The connection port specification parameters include the nominal diameter of the pipe component, pressure rating, and standard code of the connection flange.

[0061] The graph construction module calculates the 3D spatial distance between any two pipe component connection ports based on their coordinates. The 3D spatial distance is calculated using Euclidean distance metric to determine the straight-line distance between the coordinates of the two connection ports. The module marks two connection ports with a 3D spatial distance less than a preset distance threshold as candidate connection pairs. The preferred range for the preset distance threshold is 50-200 mm. This range considers the tolerance for assembly gaps during pipe component installation and the modeling accuracy deviation of the BIM 3D geometric data. In practice, 100 mm can be selected as the default threshold to accommodate the actual assembly accuracy requirements of most pipe engineering projects. For each candidate connection pair, the graph construction module checks whether the orientation vectors of the connection ports meet the anti-collinearity condition. The anti-collinearity condition is determined by calculating the dot product of the orientation vectors of the two connection ports. If the dot product is less than -0.98, the two orientation vectors are determined to be basically anti-collinear, meaning the two connection ports are in a face-to-face position during installation. For candidate connection pairs that meet the condition of reverse collinearity, the specifications of the connection ports are further checked for matching. The specification matching check includes: 1) comparing the nominal diameter values ​​of the two connection ports. If the nominal diameter values ​​are equal, the nominal diameter matching check is passed; 2) comparing the pressure rating identifiers of the two connection ports. If the pressure rating identifiers are the same, the pressure rating matching check is passed; 3) comparing the standard codes of the connecting flanges of the two connection ports. If the standard codes of the connecting flanges are the same or belong to compatible matching standards, the flange standard matching check is passed. The graph construction module only determines candidate connection pairs that simultaneously meet the conditions of three-dimensional spatial distance less than a preset distance threshold, reverse collinearity of connection port orientation vectors, and matching of connection port specifications as valid spatial connection relationships. Two pipe components determined to have valid spatial connection relationships can be physically connected and installed during construction and assembly. Based on the valid spatial connection relationship, edges are established between the nodes of the corresponding pipe components in the graph structure data. Each edge stores the connection type identifier and the matching degree value of the connection port specifications corresponding to the valid spatial connection relationship.

[0062] This solution automatically identifies valid spatial connections between pipe components using a graph construction module based on three criteria: distance calculation of connection port coordinates, anti-collinearity check of connection port orientation vectors, and matching check of connection port specification parameters. This effectively solves the problem of omissions or incorrect connections caused by manual specification of pipe connections in the BIM model in existing technologies. Compared to connection relationship determination methods based solely on spatial distance, this solution introduces orientation vector verification and specification parameter matching verification, significantly reducing the rate of misclassifying spatially adjacent but inconsistently oriented or incompatible pipe components as valid connections. This improves the accuracy and reliability of spatial constraint relationships expressed in graph structure data, providing a more precise topological foundation for the spatial constraints of pipe components in the graph neural network module.

[0063] It should be noted that although the steps are described in a specific order above, this does not mean that they must be performed in that order. In fact, some of these steps can be executed concurrently, or even in a different order, as long as the required functionality is achieved. The number of devices and processing scale described herein are for simplification of the invention; applications, modifications, and variations of this invention will be readily apparent to those skilled in the art.

[0064] Although embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. For those skilled in the art, other modifications can be easily made. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and illustrations shown and described herein.

Claims

1. A BIM-based pipeline construction progress visualization management system, characterized in that, include: An image acquisition device that acquires image data of the construction site, including pipe components; The data storage unit stores the BIM 3D geometric data, BIM attribute data, planned installation time data, and actual installation status data of the pipeline components. The semantic segmentation network receives image data output by the image acquisition device, identifies the type and installation status of pipe components in the image data, retrieves the corresponding pipe component in the data storage unit according to the identified pipe component type, writes the identified pipe component installation status as the actual installation status data into the data storage unit and associates it with the corresponding pipe component. The graph construction module obtains the BIM three-dimensional geometric data of each pipe component from the data storage unit, and constructs graph structure data reflecting the spatial constraint relationship between the pipe components based on the spatial connection relationship between each pipe component in the BIM three-dimensional geometric data. The graph neural network module receives graph structure data, obtains the actual installation status data of each pipe component from the data storage unit, inputs the graph structure data with the node status of the installed pipe component as the constraint condition, calculates the node status propagation result of the uninstalled pipe component under the spatial constraint relationship affected by the change of construction conditions, and outputs the spatial constraint propagation path data of the uninstalled pipe component. The schedule replanning module receives and adjusts the installation sequence of the uninstalled pipe components according to the spatial constraint propagation path data, generates updated installation sequence data, and writes it into the data storage unit. The visualization rendering module reads the actual installation status data and updated installation sequence data corresponding to each pipe component from the data storage unit, and performs differentiated rendering of each pipe component in the 3D scene.

2. The BIM-based pipeline construction progress visualization management system according to claim 1, characterized in that, The semantic segmentation network adopts an encoder-decoder architecture, where the encoder includes multiple downsampling convolutional layers to extract multi-scale feature maps of pipe components in the image data; the decoder includes multiple upsampling convolutional layers to restore the multi-scale feature maps to the resolution of the input image and output pixel-by-pixel classification results. During the training phase, the semantic segmentation network is jointly trained using a synthetic image dataset generated by rendering BIM 3D geometric data and a real image dataset from the construction site. The synthetic image dataset is generated by simulating 2D rendered images of pipe components under different lighting conditions, different degrees of occlusion, and different construction progress stages in a 3D virtual environment. The real image dataset is obtained by acquiring images from the construction site using an image acquisition device and manually annotating them. During the training process, a domain-adaptive loss function is used to reduce the difference in feature distribution between the synthetic image dataset and the real image dataset.

3. The BIM-based pipeline construction progress visualization management system according to claim 2, characterized in that, The domain adaptive loss function includes a feature extractor, a classifier, and a domain discriminator. The feature extractor extracts multi-scale feature maps from the input image data. The classifier outputs pixel-level prediction results of pipe component type and installation status based on the multi-scale feature maps. The domain discriminator determines whether the input image data comes from a synthetic image dataset or a real image dataset based on the multi-scale feature maps and outputs the domain label prediction results. During training, the domain adaptive loss function is composed of a weighted sum of a classification loss term and a domain adversarial loss term. The classification loss term is used to calculate the cross-entropy between the predicted result output by the classifier and the labeled ground truth. The domain adversarial loss term is used to calculate the cross-entropy between the domain label predicted result output by the domain discriminator and the domain label ground truth. During backpropagation, the gradient inversion layer applies a reverse gradient to the feature extractor, so that the multi-scale feature maps learned by the feature extractor tend to be consistent in distribution between the synthetic image dataset and the real image dataset.

4. The BIM-based pipeline construction progress visualization management system according to claim 1, characterized in that, The graph neural network module includes a graph attention convolutional layer and a node state propagation layer; The graph attention convolutional layer receives graph structure data output by the graph construction module. In the graph structure data, nodes represent pipe components, and edges represent the spatial connection relationships between pipe components. The graph attention convolutional layer calculates the attention weight of each edge based on the node features at both ends of the edge. The specific attention weight is associated with the connection type corresponding to the spatial connection relationship. The connection types include flange connection, welded connection, and threaded connection. The node state propagation layer uses the node state corresponding to the installed pipe components as the initial constraint condition, uses the attention weight as the transmission strength coefficient of spatial constraint propagation, and propagates the node state layer by layer along the edges in the graph structure data, outputting the spatial constraint propagation path data corresponding to the uninstalled pipe components.

5. The BIM-based pipeline construction progress visualization management system according to claim 4, characterized in that, The graph attention convolutional layer introduces a regularization term when calculating the attention weight of each edge. The regularization term assigns a prior weight to each edge based on the connection type corresponding to the spatial connection relationship. The prior weight of the connection type corresponding to the flange connection is in the first numerical range, the prior weight of the connection type corresponding to the welded connection is in the second numerical range, and the prior weight of the connection type corresponding to the threaded connection is in the third numerical range. The lower limit of the first numerical range is higher than the upper limit of the second numerical range, and the lower limit of the second numerical range is higher than the upper limit of the third numerical range. The graph attention convolutional layer performs a weighted sum of the calculated attention weights and the corresponding prior weights of the connection types, and then outputs the normalized attention weights through an activation function. The normalized attention weights are used as the propagation strength coefficients of spatial constraint propagation and input to the node state propagation layer.

6. The BIM-based pipeline construction progress visualization management system according to claim 1, characterized in that, The schedule replanning module has a built-in resource constraint table, which records the number of different types of machinery and work teams required for the installation of various types of pipe components. The schedule replanning module retrieves the current number of available machines and the current number of available work teams from the data storage unit. It then performs a topology sorting and iterative filtering on all uninstalled pipe components involved in the spatial constraint propagation path data. Each round of iterative filtering includes: Extract the set of components with an in-degree of zero from the spatial constraint propagation path from the remaining uninstalled pipe components. Remove pipe components from the set of components with an in-degree of zero that require more types of machinery than the current number of available machinery or more types of work teams than the current number of available work teams. Mark the remaining pipe components as members of the next batch of installation sequence, update the current number of available machinery and work teams, and remove the marked pipe components from the remaining uninstalled pipe components before entering the next round of iteration and screening, until all uninstalled pipe components are marked to the corresponding installation batch. The schedule replanning module generates updated installation sequence data based on each installation batch.

7. The BIM-based pipeline construction progress visualization management system according to claim 1, characterized in that, The visualization rendering module has a built-in rendering rule mapping table. In the rendering rule mapping table, the actual installation status data is defined. The first color value corresponds to the installation completed status, the second color value corresponds to the installation delayed status, and the third color value corresponds to the installation not installed status. The actual installation status of each pipe component is mapped to the corresponding color value according to the rendering rule mapping table. The visualization rendering module also extracts the installation batch number of each pipe component from the updated installation sequence data, calculates the transparency value of each pipe component based on the installation batch number, and the earlier the installation batch number, the lower the corresponding transparency value. The color value and transparency value are superimposed and applied to the material properties of the corresponding pipe component in the 3D scene. In the 3D scene, the actual installation status is distinguished by color and the updated installation sequence is distinguished by transparency.

8. The BIM-based pipeline construction progress visualization management system according to claim 1, characterized in that, The image acquisition device includes a visible light camera mounted on a drone and multiple network cameras deployed at fixed locations on the construction site. The visible light camera mounted on the drone acquires first-view image data of the pipeline components according to a preset inspection route, and the multiple network cameras acquire second-view image data of the pipeline components at fixed shooting angles. The image acquisition device also includes an image registration unit. The image registration unit performs feature point matching on the first-view image data and the second-view image data acquired at the same time and calculates the projection transformation matrix. Based on the projection transformation matrix, the first-view image data and the second-view image data are mapped to a unified image coordinate system and then image stitching is performed to generate a full view image data of the construction site without any viewpoint occlusion blind spots. The full view image data of the construction site is then sent to the semantic segmentation network as the image data output by the image acquisition device.

9. The BIM-based pipeline construction progress visualization management system according to claim 1, characterized in that, The data storage unit includes a BIM basic data partition, a progress snapshot partition, and a change log partition. The BIM basic data partition stores the initial versions of the BIM 3D geometric data and BIM attribute data of the piping components. The progress snapshot partition stores the planned installation time data and the actual installation status data of each time node for each piping component. The change log partition records the timestamp, the value before modification, and the value after modification for each modification operation performed on the BIM 3D geometric data or BIM attribute data of the piping components in the BIM basic data partition. After receiving externally input BIM design change data, the data storage unit first copies the original data of the modified pipe components in the current BIM basic data partition to the change log partition, and then applies the modification operation to the BIM basic data partition. After each modification operation in the BIM basic data partition of the data storage unit, the graph construction module re-acquires the BIM three-dimensional geometric data of the pipe components affected by the modification operation and locally updates the corresponding nodes and edges in the graph structure data.

10. The BIM-based pipeline construction progress visualization management system according to claim 1, characterized in that, The graph construction module obtains the BIM 3D geometric data and BIM attribute data of each pipe component from the data storage unit. The BIM attribute data includes the coordinates of the connection ports, the orientation vector of the connection ports, and the specification parameters of the connection ports for each pipe component. The graph construction module calculates the three-dimensional spatial distance between any two pipe component connection ports based on the connection port coordinates. Two connection ports with a three-dimensional spatial distance less than a preset distance threshold are marked as candidate connection pairs. For each candidate connection pair, the module checks whether the orientation vectors of the connection ports meet the condition of being collinear in opposite directions and whether the connection port specifications match. Only candidate connection pairs that simultaneously meet the conditions of having a three-dimensional spatial distance less than the preset distance threshold, having orientation vectors of the connection ports being collinear in opposite directions, and having matching connection port specifications are determined as valid spatial connection relationships. Based on the valid spatial connection relationships, edges are established between the nodes of the corresponding pipe components in the graph structure data.

Images