Intelligent building facility-oriented multi-party collaborative progress visualization monitoring method

Abstract

The application provides a multi-party collaborative progress visualization monitoring method for intelligent building facilities, relates to the technical field of intelligent building construction monitoring, and comprises the following steps: collecting data to establish a three-dimensional model; establishing a causal relationship identifier based on time sequence and correlation characteristics, obtaining a causal graph, calculating an abnormal propagation probability, and marking a risk area; performing information mapping to generate monitoring data flow and extract features; calculating feature correlation and mapping into a scene semantic segmentation graph; and rendering to obtain a visual three-dimensional dynamic construction progress in combination with risk area and progress information. The application realizes risk early warning and progress visualization monitoring of the construction process.

Description

Technical Field

[0001] This invention relates to the field of intelligent building construction monitoring technology, and in particular to a multi-party collaborative progress visualization monitoring method for intelligent building facilities. Background Technology

[0002] With the rapid development of the construction industry, the construction management of intelligent building facilities faces increasingly higher requirements. Traditional construction progress monitoring mainly relies on manual inspections and document recording, which is inefficient and unable to reflect the dynamic changes on the construction site in real time. In recent years, with the widespread application of digital technology in the construction industry, construction monitoring methods based on three-dimensional digital models have received widespread attention and application. These methods, by collecting multi-dimensional data from the construction site and combining computer vision and data analysis technologies, can more intuitively and accurately display the construction progress and quality status.

[0003] Existing construction monitoring systems lack in-depth analysis of the causal relationships between construction elements, failing to effectively identify potential risk points and their propagation paths, resulting in untimely and inaccurate risk warnings. Current technologies are insufficient in integrating multi-source heterogeneous data, making it difficult to effectively map the physical state of the construction scene to digital models, leading to fragmented monitoring information and impacting decision-making efficiency. Furthermore, existing construction monitoring visualization methods are limited, lacking semantic understanding and adaptive rendering capabilities, failing to provide personalized visualizations based on different users' needs, thus reducing the effectiveness and intuitiveness of information delivery. Summary of the Invention

[0004] This invention provides a multi-party collaborative progress visualization monitoring method for intelligent building facilities, which can solve the problems in the prior art.

[0005] A first aspect of this invention provides a multi-party collaborative progress visualization monitoring method for intelligent building facilities, comprising:

[0006] Collect historical construction data and multi-dimensional data information from intelligent building facilities, and establish a three-dimensional digital model based on the multi-dimensional data information;

[0007] A causal relationship identifier is established based on the temporal and correlation features in the historical construction data. The multi-dimensional data information is input into the causal relationship identifier to obtain a causal strength matrix. A causal graph is established based on the causal strength matrix. Based on the causal graph, the probability of abnormal propagation of construction is calculated using the Bayesian probability propagation algorithm. When the probability of abnormal propagation of a certain node is greater than the construction risk warning threshold, it is marked as a risk area in the three-dimensional digital model.

[0008] The causal graph is mapped to the three-dimensional digital model to generate a construction scene monitoring data stream. Feature extraction is performed through MobileNet convolutional neural network. The extracted feature information is then collaboratively analyzed through a self-organizing network to obtain the state feature matrix of the construction scene.

[0009] The feature correlation of the state feature matrix is ​​calculated using the Pearson correlation coefficient, and the feature correlation is mapped to a scene semantic segmentation map using a multi-layer deconvolution structure. Based on the scene semantic segmentation map and the risk area and project progress information, scene rendering is performed to obtain a visualized three-dimensional dynamic construction progress.

[0010] A causal relationship identifier is established based on the temporal and correlation characteristics of the historical construction data. The multi-dimensional data information is input into the causal relationship identifier to obtain a causal strength matrix. A causal graph is then established based on the causal strength matrix, including:

[0011] The process involves collecting time series and parameter series from historical construction data, calculating similarity features between time series using dynamic time warping, calculating trend intensity features of time series using a sliding time window, concatenating the similarity features and trend intensity features to obtain time series features, calculating correlation coefficients between parameter series and constructing a correlation coefficient matrix, performing feature decomposition on the correlation coefficient matrix, and selecting the feature vectors corresponding to the largest preset number of feature values ​​as association features.

[0012] The time-series features and the association features are combined and subjected to conditional independence and time-delay causality tests to obtain a causal relationship identifier. The conditional independence test obtains the instantaneous causal strength matrix by calculating the difference between the mutual information and conditional mutual information between the parameter sequences. The time-delay causality test obtains the delayed sequence by performing time delay processing on the parameter sequences, and calculates the ratio of the mutual information entropy between the delayed sequence and the current sequence to obtain the time-delay causality strength matrix.

[0013] The multi-dimensional data information is input into the causal relationship identifier, the instantaneous causal strength matrix and the time-delayed causal strength matrix are calculated and weighted to obtain the causal strength matrix;

[0014] Based on the causal strength matrix, causal relationships with causal strength greater than a preset association threshold are taken as directed edges, and the corresponding construction parameters are taken as nodes. The maximum spanning tree algorithm is used to select the edge with the largest causal strength to ensure the connectivity of the graph, thus obtaining a causal graph.

[0015] Based on the causal graph, the probability of anomaly propagation during construction is calculated using the Bayesian probability propagation algorithm. When the probability of anomaly propagation at a certain node exceeds the construction risk warning threshold, the area marked as a risk region in the three-dimensional digital model includes:

[0016] The construction parameter nodes and their causal relationship edges in the causal graph are constructed into a Bayesian network. The frequency of co-occurrence between each node and its parent node is counted, and the conditional probability of the node is calculated based on the ratio of the co-occurrence frequency to the frequency of the parent node. For each node in the Bayesian network, the deviation between the current parameter value of the node and the historical mean is calculated. When the deviation exceeds a preset multiple of the standard deviation, the node is determined to be in an abnormal state; otherwise, it is in a normal state.

[0017] Based on the Bayesian network, the probability propagation value between nodes is calculated. The probability propagation value is the product of the conditional probability of the parent node and the conditional probabilities of all neighboring nodes received by the parent node. The probability propagation value passed from the parent node is multiplied by the probability propagation value passed from the child node and normalized to obtain the belief value of the node.

[0018] When a node is in the abnormal state, the node's belief value is used as the probability of the abnormal state. When a node is in the normal state, 1 is subtracted from the node's belief value to obtain the probability of the abnormal state. The probability of the abnormal state is weighted and accumulated with the conditional probabilities of all its child nodes to obtain the probability of abnormal propagation.

[0019] When the probability of abnormal propagation of a node is greater than the construction risk warning threshold, the spatial coordinates of the node in the three-dimensional digital model are determined. Taking the spatial coordinates as the center, the ratio of the probability of abnormal propagation to the construction risk warning threshold is calculated and multiplied by the baseline influence radius to obtain the risk influence radius. The risk area is marked within the risk influence radius.

[0020] The causal graph is mapped to the three-dimensional digital model to generate a construction scene monitoring data stream. Feature extraction is performed using a MobileNet convolutional neural network. The extracted feature information is then collaboratively analyzed using a self-organizing network to obtain the state feature matrix of the construction scene, which includes:

[0021] Obtain the physical spatial coordinates of nodes in the causal graph, establish a mapping between the physical spatial coordinates and the spatial position coordinates of the corresponding engineering components in the three-dimensional digital model, and generate node position mapping information; collect multimodal data at the physical spatial coordinates indicated by the node position mapping information and integrate them into a construction scene monitoring data stream in chronological order;

[0022] The construction scene monitoring data stream is input into the MobileNet convolutional neural network. Features are extracted from each data channel through a deep convolutional layer, and then the features extracted from each data channel are combined into a multimodal feature vector through a point-state convolutional layer.

[0023] The multimodal feature vector is input into the self-organizing neural network. The Euclidean distance between the multimodal feature vector and the weights of the network neurons is calculated. The neuron with the smallest Euclidean distance is selected as the winning neuron. The learning rate decay coefficient and the range of neighboring neurons of the winning neuron are determined. The difference between the multimodal feature vector and the current weight of the winning neuron is multiplied by the learning rate decay coefficient to obtain the weight adjustment amount of the winning neuron. The topological distance between the neighboring neurons and the winning neuron is calculated. The weight adjustment coefficient of the neighboring neurons is determined based on the topological distance. The weight adjustment coefficient is multiplied by the weight adjustment amount of the winning neuron to obtain the weight adjustment amount of the neighboring neurons. The weight adjustment amounts of the winning neuron and its neighboring neurons are respectively superimposed on the corresponding current weights and reorganized into a state feature matrix according to the feature dimension.

[0024] The feature correlation of the state feature matrix is ​​calculated using the Pearson correlation coefficient, and the feature correlation is mapped to a scene semantic segmentation map using a multi-layer deconvolution structure, including:

[0025] Data samples are obtained for every two feature dimensions in the state feature matrix. The covariance and standard deviation of the data samples are calculated. The covariance is divided by the product of the standard deviations of the two feature dimensions to obtain the Pearson correlation coefficient between the feature dimensions. A feature correlation matrix is ​​generated based on the Pearson correlation coefficient.

[0026] The feature correlation matrix is ​​decomposed into a linear combination of orthogonal components. The feature correlation is mapped to the three-dimensional spatial domain by weighted superposition of orthogonal components of different scales and then input into a multi-layer deconvolution structure. The multi-layer deconvolution structure performs upsampling and nonlinear transformation in sequence. The upsampling fills the deconvolution kernel with multiples of the upsampling coefficients and inserts zero values ​​between adjacent elements of the input features. The deconvolution kernel is used to calculate and obtain an enlarged feature map. The nonlinear transformation sets the negative values ​​in the enlarged feature map to zero. This process is passed layer by layer in the multi-layer deconvolution structure until a scene feature map with the same size as the scene is generated.

[0027] In the scene feature map, the Euclidean distance between each pixel and each semantic category feature template is calculated to obtain the cost value of assigning the pixel to different semantic categories. At the same time, the L2 norm of the difference between the feature vectors of each pixel and its four neighboring pixels is calculated to obtain the semantic category difference degree of the adjacent pixels. The weighted sum of the cost value and the semantic category difference degree is used as the energy value of the pixel. The semantic category of each pixel is updated iteratively through gradient descent until the energy values ​​of all pixels converge to obtain the scene semantic segmentation map.

[0028] Based on the scene semantic segmentation map and the risk area and project progress information, scene rendering is performed to obtain a visualized 3D dynamic construction progress, including:

[0029] The semantic label value of each pixel is determined according to the engineering component type of the corresponding region of the scene semantic segmentation map. The risk level value of the corresponding position is determined based on the risk assessment result of the risk region. The semantic label value and the risk level value are weighted and fused to obtain the fusion mapping value.

[0030] Based on the three-dimensional digital model, the depth information of the construction scene is obtained. The coordinate transformation is used to map the two-dimensional pixel coordinates in the scene semantic segmentation map to the corresponding positions in the three-dimensional digital model. The spatial height value is determined based on the depth information in the three-dimensional digital model. A corresponding three-dimensional construction space coordinate system is generated according to the two-dimensional pixel coordinates and the spatial height value.

[0031] In the three-dimensional construction space coordinate system, the construction status identifier of the engineering component is set according to the fusion mapping value, and the engineering components with different construction statuses are mapped to different display styles. The construction status identifier of the engineering component is dynamically updated based on the project progress information, and the completion index of the engineering component is converted into the corresponding fill ratio for visualization, thereby generating a three-dimensional construction scene with dynamic display effect of construction progress.

[0032] A second aspect of the present invention provides a multi-party collaborative progress visualization monitoring system for intelligent building facilities, comprising:

[0033] The first unit is used to collect historical construction data and multi-dimensional data information in intelligent building facilities, and to establish a three-dimensional digital model based on the multi-dimensional data information.

[0034] The second unit is used to establish a causal relationship identifier based on the temporal and correlation features in the historical construction data. The multi-dimensional data information is input into the causal relationship identifier to obtain a causal strength matrix. A causal graph is established based on the causal strength matrix. Based on the causal graph, the probability of abnormal propagation of construction is calculated using the Bayesian probability propagation algorithm. When the probability of abnormal propagation of a certain node is greater than the construction risk warning threshold, it is marked as a risk area in the three-dimensional digital model.

[0035] The third unit is used to map the causal graph to the three-dimensional digital model, generate a construction scene monitoring data stream, extract features through a MobileNet convolutional neural network, and perform collaborative analysis of the extracted feature information through a self-organizing network to obtain the state feature matrix of the construction scene.

[0036] The fourth unit is used to calculate the feature correlation of the state feature matrix using the Pearson correlation coefficient, and to map the feature correlation into a scene semantic segmentation map using a multi-layer deconvolution structure; based on the scene semantic segmentation map and the risk area and project progress information, scene rendering is performed to obtain a visualized three-dimensional dynamic construction progress.

[0037] A third aspect of the present invention,

[0038] An electronic device is provided, comprising:

[0039] processor;

[0040] Memory used to store processor-executable instructions;

[0041] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0042] Fourth aspect of the embodiments of the present invention,

[0043] A computer-readable storage medium is provided, having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0044] The beneficial effects of this application are as follows:

[0045] The multi-party collaborative progress visualization monitoring method provided by this invention can accurately identify potential risks in the construction process by establishing a causal relationship identifier based on historical construction data, thereby achieving early warning and prevention of construction risks and effectively reducing safety hazards and quality risks in the construction process.

[0046] This invention maps causal graphs to three-dimensional digital models and analyzes and processes construction scenarios using deep learning technology, enabling real-time and precise monitoring of construction progress. This improves the efficiency and accuracy of construction management and provides intuitive and comprehensive decision support for project managers.

[0047] This invention employs a multi-layer deconvolution structure to map feature correlations into a scene semantic segmentation map, and combines risk area and project progress information for scene rendering, thereby realizing a visual display of construction progress. This enables all parties involved to have a clearer understanding of the project's progress, promotes multi-party collaborative management, and improves the overall construction quality and efficiency of intelligent building facilities. Attached Figure Description

[0048] Figure 1 This is a flowchart illustrating a multi-party collaborative progress visualization monitoring method for intelligent building facilities according to an embodiment of the present invention.

[0049] Figure 2 This is a schematic diagram of the process for calculating the probability of anomaly propagation in a Bayesian network. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0052] Figure 1 This is a flowchart illustrating a multi-party collaborative progress visualization monitoring method for intelligent building facilities according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0053] Collect historical construction data and multi-dimensional data information from intelligent building facilities, and establish a three-dimensional digital model based on the multi-dimensional data information;

[0054] A causal relationship identifier is established based on the temporal and correlation features in the historical construction data. The multi-dimensional data information is input into the causal relationship identifier to obtain a causal strength matrix. A causal graph is established based on the causal strength matrix. Based on the causal graph, the probability of abnormal propagation of construction is calculated using the Bayesian probability propagation algorithm. When the probability of abnormal propagation of a certain node is greater than the construction risk warning threshold, it is marked as a risk area in the three-dimensional digital model.

[0055] The causal graph is mapped to the three-dimensional digital model to generate a construction scene monitoring data stream. Feature extraction is performed through MobileNet convolutional neural network. The extracted feature information is then collaboratively analyzed through a self-organizing network to obtain the state feature matrix of the construction scene.

[0056] The feature correlation of the state feature matrix is ​​calculated using the Pearson correlation coefficient, and the feature correlation is mapped to a scene semantic segmentation map using a multi-layer deconvolution structure. Based on the scene semantic segmentation map and the risk area and project progress information, scene rendering is performed to obtain a visualized three-dimensional dynamic construction progress.

[0057] In one optional implementation, a causal relationship identifier is established based on the temporal and correlation characteristics of the historical construction data. The multi-dimensional data information is input into the causal relationship identifier to obtain a causal strength matrix. Establishing a causal graph based on the causal strength matrix includes:

[0058] The process involves collecting time series and parameter series from historical construction data, calculating similarity features between time series using dynamic time warping, calculating trend intensity features of time series using a sliding time window, concatenating the similarity features and trend intensity features to obtain time series features, calculating correlation coefficients between parameter series and constructing a correlation coefficient matrix, performing feature decomposition on the correlation coefficient matrix, and selecting the feature vectors corresponding to the largest preset number of feature values ​​as association features.

[0059] The time-series features and the association features are combined and subjected to conditional independence and time-delay causality tests to obtain a causal relationship identifier. The conditional independence test obtains the instantaneous causal strength matrix by calculating the difference between the mutual information and conditional mutual information between the parameter sequences. The time-delay causality test obtains the delayed sequence by performing time delay processing on the parameter sequences, and calculates the ratio of the mutual information entropy between the delayed sequence and the current sequence to obtain the time-delay causality strength matrix.

[0060] The multi-dimensional data information is input into the causal relationship identifier, the instantaneous causal strength matrix and the time-delayed causal strength matrix are calculated and weighted to obtain the causal strength matrix;

[0061] Based on the causal strength matrix, causal relationships with causal strength greater than a preset association threshold are taken as directed edges, and the corresponding construction parameters are taken as nodes. The maximum spanning tree algorithm is used to select the edge with the largest causal strength to ensure the connectivity of the graph, thus obtaining a causal graph.

[0062] When collecting time series and parameter sequences from historical construction data, various construction parameters, including equipment operating parameters, environmental parameters, and engineering quality parameters, can be collected through a sensor network at the construction site. For example, for the concrete pouring process, the collected parameter sequences include data from multiple dimensions such as concrete temperature, ambient temperature, pouring rate, and vibration time, forming a multi-dimensional parameter sequence indexed by timestamps. The collection frequency can be set to once every 10 minutes, continuously collecting for 72 hours, generating a parameter sequence of 432 sampling points.

[0063] For the collected time-series data, similarity features between sequences are calculated using dynamic time warping. Specifically, two time series to be compared are selected, such as a concrete temperature sequence T and a pouring rate sequence S, and a distance matrix is ​​established between the two sequences. For each time point ti in sequence T and each time point sj in sequence S, the square of the difference in parameter values ​​is calculated and filled into the corresponding position in the distance matrix. Then, the path with the minimum distance is found using a backtracking method; the length of this path is the similarity value between the two sequences. For the concrete construction case mentioned above, the similarity calculation result between the temperature sequence and the pouring rate sequence is 0.78, indicating that they have high temporal similarity.

[0064] The trend intensity of a time series is calculated using a sliding time window. The sliding window size is set to 30 minutes, and the window sliding step is 5 minutes. For each data point within the window, the slope of the linear regression is calculated; the absolute value of the slope represents the trend intensity. For example, if the trend intensity of a concrete temperature series within a certain time window is 0.15°C / minute, it indicates a significant upward trend in temperature within that window.

[0065] By concatenating similarity features and trend intensity features, a comprehensive temporal feature is obtained. Taking concrete construction as an example, the similarity feature 0.78 between the temperature sequence and the pouring rate sequence is concatenated with the trend intensity feature 0.15 of the temperature sequence to form a two-dimensional feature vector [0.78, 0.15], which serves as the feature describing the temporal relationship between these two parameters.

[0066] The correlation between parameter sequences was analyzed, and the correlation coefficients between the parameter sequences were calculated and a correlation coefficient matrix was constructed. Taking five key construction parameters as an example, including concrete temperature, ambient temperature, pouring rate, vibration time, and concrete slump, the correlation coefficient matrix obtained showed that the correlation coefficient between concrete temperature and pouring rate was 0.82, indicating that the two were highly correlated.

[0067] The correlation coefficient matrix is ​​subjected to eigenvalue decomposition, and the eigenvectors corresponding to the three largest eigenvalues ​​are selected as the correlation features. For example, the first three eigenvalues ​​obtained after eigenvalue decomposition are 2.56, 1.89, and 0.93, and the corresponding eigenvectors are used as features describing the correlation structure between parameters.

[0068] The temporal and correlation features are combined and a conditional independence test is performed. For any two parameter sequences A and B, their mutual information value is calculated, and then the conditional mutual information value is calculated under the condition of a third parameter sequence C. If the difference between the two is close to zero, it indicates that A and B are conditionally independent under the condition of C, and there is no direct causal relationship. For example, the mutual information value between concrete temperature and slump is 0.65, while under the condition of a given ambient temperature, the conditional mutual information value is 0.12, a difference of 0.53, indicating that there is a certain direct causal relationship. By testing all parameter pairs, an instantaneous causal strength matrix is ​​constructed.

[0069] When performing time-delay causality testing, the parameter sequence is time-delayed. Taking the concrete temperature sequence as an example, a 1-hour delayed sequence is created, and the mutual information entropy between this delayed sequence and the current pouring rate sequence is calculated. This is then compared with the mutual information entropy of the original sequence. A ratio of 1.45, greater than 1, indicates the existence of a time-delay causal relationship from concrete temperature to pouring rate. By testing multiple delay times between different parameters, a time-delay causality strength matrix is ​​constructed.

[0070] Multi-dimensional data is input into the causal relationship identifier to calculate the instantaneous causal strength matrix and the time-delayed causal strength matrix. These matrices are then weighted with weights of 0.6 and 0.4, respectively, to obtain the final causal strength matrix. In this matrix, the causal strength from ambient temperature to concrete temperature is 0.82, and the causal strength from pouring rate to concrete slump is 0.68, both indicating significant causal relationships.

[0071] Based on the causal strength matrix, an association threshold of 0.5 is set. Causal relationships with a strength greater than this threshold are designated as directed edges, and the corresponding construction parameters are designated as nodes. For example, edges such as ambient temperature → concrete temperature and pouring rate → concrete slump are retained. To ensure graph connectivity, a maximum spanning tree algorithm is used to select the edge with the highest causal strength, ultimately forming a complete causal graph. In this graph, ambient temperature is identified as a key root cause affecting concrete construction quality, influencing concrete temperature and pouring process, thereby affecting the final concrete performance.

[0072] The causal graph constructed using the above methods can intuitively display the causal relationships between construction parameters, providing decision support for construction management, helping to identify key control parameters, optimize construction processes, and improve project quality.

[0073] In one optional implementation, based on the causal graph, the probability of anomaly propagation during construction is calculated using a Bayesian probability propagation algorithm. When the probability of anomaly propagation at a certain node exceeds a construction risk warning threshold, the area marked as a risk region in the three-dimensional digital model includes:

[0074] The construction parameter nodes and their causal relationship edges in the causal graph are constructed into a Bayesian network. The frequency of co-occurrence between each node and its parent node is counted, and the conditional probability of the node is calculated based on the ratio of the co-occurrence frequency to the frequency of the parent node. For each node in the Bayesian network, the deviation between the current parameter value of the node and the historical mean is calculated. When the deviation exceeds a preset multiple of the standard deviation, the node is determined to be in an abnormal state; otherwise, it is in a normal state.

[0075] Based on the Bayesian network, the probability propagation value between nodes is calculated. The probability propagation value is the product of the conditional probability of the parent node and the conditional probabilities of all neighboring nodes received by the parent node. The probability propagation value passed from the parent node is multiplied by the probability propagation value passed from the child node and normalized to obtain the belief value of the node.

[0076] When a node is in the abnormal state, the node's belief value is used as the probability of the abnormal state. When a node is in the normal state, 1 is subtracted from the node's belief value to obtain the probability of the abnormal state. The probability of the abnormal state is weighted and accumulated with the conditional probabilities of all its child nodes to obtain the probability of abnormal propagation.

[0077] When the probability of abnormal propagation of a node is greater than the construction risk warning threshold, the spatial coordinates of the node in the three-dimensional digital model are determined. Taking the spatial coordinates as the center, the ratio of the probability of abnormal propagation to the construction risk warning threshold is calculated and multiplied by the baseline influence radius to obtain the risk influence radius. The risk area is marked within the risk influence radius.

[0078] like Figure 2 As shown, the method includes:

[0079] This technical solution first constructs a Bayesian network of construction parameter nodes. The directed connections between construction parameter nodes and their causal edges in the causal graph serve as the basic structure of the Bayesian network. For example, in a subway tunnel construction scenario, construction parameters such as "earth pressure," "tunnel boring machine thrust," "tunnel boring machine torque," "grouting volume," and "tunneling speed" can be used as nodes, and a Bayesian network can be constructed based on the causal relationships between them. When calculating conditional probabilities, frequency statistics are performed on each pair of node relationships by analyzing historical construction data. For example, regarding the relationship between the "earth pressure" node (A) and its parent node "soil condition" (B), if the historical data shows that the earth pressure anomaly occurred 15 times when the stratum was sand, and the total number of times the stratum was sandy is 50, then the conditional probability P(A anomaly | B is sandy) = 15 / 50 = 0.3 can be calculated. Similarly, a conditional probability table of all nodes and their parent nodes can be calculated.

[0080] When determining the status of a node, the construction parameter values ​​of each node are collected in real time and compared with the historical average of that parameter. Assuming the historical average of the "tunnel boring machine thrust" at a certain node is 2000 kN, the standard deviation is 200 kN, the preset multiplier is 2.5, and the currently collected thrust value is 2600 kN, the calculated deviation is |2600-2000|=600 kN. This deviation exceeds 2.5 times the standard deviation (200×2.5=500 kN), therefore the node is determined to be in an abnormal state. For the "tunneling speed" node, if the historical average is 30 mm / min, the standard deviation is 5 mm / min, and the current value is 36 mm / min, the deviation is |36-30|=6 mm / min, which is less than 2.5 times the standard deviation (5×2.5=12.5 mm / min), then the node is determined to be in a normal state.

[0081] Based on the constructed Bayesian network, the probability propagation values ​​between nodes are calculated. The probability propagation process of a node is divided into two directions: top-down and bottom-up. For node "grouting volume" (C), its parent nodes are "earth pressure" (A) and "tunnel boring machine thrust" (B), with conditional probabilities P(C|A) = 0.4 and P(C|B) = 0.6, respectively. If the conditional probability received by A from all its neighboring nodes is 0.5, and that received by B is 0.7, then the probability propagation value from A to C is 0.4 × 0.5 = 0.2, and the probability propagation value from B to C is 0.6 × 0.7 = 0.42. Node C also receives backpropagation values ​​from its child nodes, assumed to be 0.3. Multiplying these probability propagation values ​​and normalizing them yields the belief value of node C. The normalization process is: 0.2 × 0.42 × 0.3 = 0.0252. Assuming the sum of the products of all states is 0.05, the normalized belief value is 0.0252 / 0.05 = 0.504.

[0082] When determining the probability of an abnormal state of a node, different processing is applied based on the node's current state. For the node "Shield Thrust" in an abnormal state, its belief value is 0.6, so its probability of an abnormal state is directly taken as 0.6. For the node "Tunneling Speed" in a normal state, its belief value is 0.7, so its probability of an abnormal state is 1-0.7=0.3. To calculate the probability of anomaly propagation of a node, the probability of an abnormal state of the node is weighted and accumulated with the conditional probabilities of all its child nodes. For example, the probability of an abnormal state of node "Shield Thrust" (B) is 0.6. It has two child nodes "Grouting Volume" (C) and "Surface Settlement" (D), with conditional probabilities P(C|B)=0.6 and P(D|B)=0.4, respectively. Then the probability of anomaly propagation of B is 0.6×(0.6+0.4)=0.6.

[0083] When the probability of abnormal propagation of a node exceeds the preset construction risk warning threshold, a risk area will be marked in the 3D digital model. Assuming the probability of abnormal propagation for node "surface subsidence" is 0.75 and the construction risk warning threshold is 0.7, the spatial coordinates of this node in the 3D model are determined, for example (X=500, Y=200, Z=30). The ratio of the abnormal propagation probability to the warning threshold is calculated as 0.75 / 0.7≈1.07. If the baseline influence radius is set to 50 meters, the risk influence radius is 50 × 1.07 = 53.5 meters. Within a radius of 53.5 meters centered on these coordinates, the risk area is marked on the 3D digital model. This area can be visualized using a red semi-transparent circular or spherical region, displaying the risk level and related parameter information, such as "Surface subsidence risk area, abnormal propagation probability 0.75, risk influence radius 53.5 meters".

[0084] The above method enables real-time monitoring of abnormal states of various parameters during construction, and calculates the probability and impact range of anomaly propagation based on Bayesian networks. The risk areas are then displayed intuitively in a three-dimensional digital model, providing construction managers with accurate risk warnings so that appropriate risk control measures can be taken in a timely manner to ensure construction safety.

[0085] In one optional implementation, the causal graph is mapped to the three-dimensional digital model to generate a construction scene monitoring data stream. Feature extraction is then performed using a MobileNet convolutional neural network. The extracted feature information is then collaboratively analyzed using a self-organizing network to obtain a state feature matrix of the construction scene, including:

[0086] Obtain the physical spatial coordinates of nodes in the causal graph, establish a mapping between the physical spatial coordinates and the spatial position coordinates of the corresponding engineering components in the three-dimensional digital model, and generate node position mapping information; collect multimodal data at the physical spatial coordinates indicated by the node position mapping information and integrate them into a construction scene monitoring data stream in chronological order;

[0087] The construction scene monitoring data stream is input into the MobileNet convolutional neural network. Features are extracted from each data channel through a deep convolutional layer, and then the features extracted from each data channel are combined into a multimodal feature vector through a point-state convolutional layer.

[0088] The multimodal feature vector is input into the self-organizing neural network. The Euclidean distance between the multimodal feature vector and the weights of the network neurons is calculated. The neuron with the smallest Euclidean distance is selected as the winning neuron. The learning rate decay coefficient and the range of neighboring neurons of the winning neuron are determined. The difference between the multimodal feature vector and the current weight of the winning neuron is multiplied by the learning rate decay coefficient to obtain the weight adjustment amount of the winning neuron. The topological distance between the neighboring neurons and the winning neuron is calculated. The weight adjustment coefficient of the neighboring neurons is determined based on the topological distance. The weight adjustment coefficient is multiplied by the weight adjustment amount of the winning neuron to obtain the weight adjustment amount of the neighboring neurons. The weight adjustment amounts of the winning neuron and its neighboring neurons are respectively superimposed on the corresponding current weights and reorganized into a state feature matrix according to the feature dimension.

[0089] In this embodiment, the physical spatial coordinates of the nodes in the causal graph are first obtained. For example, for the steel structure node P1 at the construction site, its physical spatial coordinates are (25.6, 37.2, 12.8) meters; for the concrete pouring area node P2, its physical spatial coordinates are (42.3, 18.5, 0.5) meters; and for the hoisting equipment node P3, its physical spatial coordinates are (30.1, 22.7, 15.3) meters. A mapping relationship is established between these physical spatial coordinates and the spatial position coordinates of the corresponding engineering components in the 3D digital model. For example, the steel structure node P1 corresponds to the steel beam component with ID S2048 in the 3D model, and the concrete pouring area node P2 corresponds to the floor slab component with ID C1053 in the model. Through a spatial coordinate transformation matrix, a transformation relationship from physical space to model space is established, generating a node position mapping information table, which includes information such as node ID, physical coordinates, model component ID, and model coordinates.

[0090] After determining the node location mapping information, multimodal data is collected at the indicated physical space coordinates. Specifically, vibration and temperature sensors are installed at steel structure node P1 to collect vibration data at a frequency of 100Hz and temperature data every 5 minutes; humidity and temperature sensors are installed at concrete pouring area node P2 to collect data every 10 minutes; and a camera is installed at hoisting equipment node P3 to collect video data at 1080p resolution and 30 frames per second. These different types of data with different collection frequencies are integrated in chronological order to form a construction scene monitoring data stream containing timestamps, node IDs, data types, and data values. For example, between timestamps 20230515080000 and 20230515080100, 6000 vibration data points and 1 temperature data point from node P1, 1 humidity data point and 1 temperature data point from node P2, and 3000 frames of video data from node P3 are integrated.

[0091] The integrated construction scene monitoring data stream is input into a MobileNet convolutional neural network for feature extraction. The input layer of the MobileNet network is configured to adapt to the format of multimodal data, including a 32×32 pixel image channel, a 64-dimensional temporal data channel, and a 16-dimensional discrete state data channel. The network uses deep convolutional layers to extract features for each data channel separately: a 3×3 convolutional kernel with a stride of 2 and a depth multiplier of 1 for the image channel; a 1×5 convolutional kernel with a stride of 1 and a depth multiplier of 1 for the temporal data channel; and a 1×3 convolutional kernel with a stride of 1 and a depth multiplier of 1 for the discrete state data channel. After deep convolution, 128-dimensional features are extracted from the image channel, 32-dimensional features from the temporal data channel, and 8-dimensional features from the discrete state data channel. Subsequently, a 1×1 point-state convolutional layer is used to combine the features extracted from each data channel into a 168-dimensional multimodal feature vector. For example, for data with timestamp 20230515080030, the first 5 elements of the multimodal feature vector obtained after processing by the MobileNet network are [0.756, 0.231, -0.482, 0.119, 0.624].

[0092] The extracted multimodal feature vectors are input into a self-organizing neural network for collaborative analysis. The self-organizing network is configured as a 10×10 two-dimensional grid structure, with each neuron having a 168-dimensional weight vector. The initial weights are randomly distributed in the interval [0,1]. The Euclidean distance between the input multimodal feature vector and the weights of each neuron in the network is calculated. For example, the Euclidean distance between the feature vector [0.756, 0.231, -0.482, 0.119, 0.624, ...] and the weight vector [0.421, 0.365, -0.258, 0.193, 0.701, ...] of neuron (3,5) is calculated to be 2.731. By comparing all distance values, the neuron (4,6) with the smallest Euclidean distance is selected as the winning neuron, with a distance value of 1.254. The learning rate decay coefficient of the winning neuron is determined, with an initial learning rate of 0.1 and an exponential decay function. At the current iteration count of 1500, the learning rate decays to 0.053. Simultaneously, the range of neighboring neurons is determined, with an initial neighborhood radius of 5 and a current neighborhood radius decaying to 2.1. Multiply the difference between the multimodal feature vector [0.756, 0.231, -0.482, 0.119, 0.624, ...] and the current weights of the winning neuron (4,6) [0.682, 0.307, -0.395, 0.201, 0.589, ...] by the learning rate decay coefficient 0.053 to obtain the weight adjustment of the winning neuron [0.0039, -0.0040, -0.0046, -0.0043, 0.0019, ...].

[0093] Calculate the topological distance between neighboring neurons and the winning neuron. For example, the topological distance between neuron (3,5) and the winning neuron (4,6) is (1...). 2 +1 2 ) 1 / 2=1.414. The weight adjustment coefficients of neighboring neurons are determined based on topological distance. Using the Gaussian neighborhood function, the adjustment coefficient for neighboring neuron (3,5) is 0.684. This adjustment coefficient is multiplied by the weight adjustment of the winning neuron to obtain the weight adjustment of neighboring neuron (3,5) [0.0027, -0.0027, -0.0031, -0.0029, 0.0013, ...]. The weights of the winning neuron and all its neighboring neurons are updated by adding their respective weight adjustments to the current weights. After the update, the weights of the winning neuron (4,6) become [0.6859, 0.3030, -0.3996, 0.1967, 0.5909, ...], and the weights of neighboring neuron (3,5) become [0.4237, 0.3623, -0.2611, 0.1901, 0.7023, ...]. The updated neuron weights are reorganized according to the feature dimensions to obtain a 10×10×168 state feature matrix, which is used to represent the state features of the current construction scenario.

[0094] In one optional implementation, the feature correlation of the state feature matrix is ​​calculated using the Pearson correlation coefficient, and the feature correlation is mapped to a scene semantic segmentation map using a multi-layer deconvolution structure, including:

[0095] Data samples are obtained for every two feature dimensions in the state feature matrix. The covariance and standard deviation of the data samples are calculated. The covariance is divided by the product of the standard deviations of the two feature dimensions to obtain the Pearson correlation coefficient between the feature dimensions. A feature correlation matrix is ​​generated based on the Pearson correlation coefficient.

[0096] The feature correlation matrix is ​​decomposed into a linear combination of orthogonal components. The feature correlation is mapped to the three-dimensional spatial domain by weighted superposition of orthogonal components of different scales and then input into a multi-layer deconvolution structure. The multi-layer deconvolution structure performs upsampling and nonlinear transformation in sequence. The upsampling fills the deconvolution kernel with multiples of the upsampling coefficients and inserts zero values ​​between adjacent elements of the input features. The deconvolution kernel is used to calculate and obtain an enlarged feature map. The nonlinear transformation sets the negative values ​​in the enlarged feature map to zero. This process is passed layer by layer in the multi-layer deconvolution structure until a scene feature map with the same size as the scene is generated.

[0097] In the scene feature map, the Euclidean distance between each pixel and each semantic category feature template is calculated to obtain the cost value of assigning the pixel to different semantic categories. At the same time, the L2 norm of the difference between the feature vectors of each pixel and its four neighboring pixels is calculated to obtain the semantic category difference degree of the adjacent pixels. The weighted sum of the cost value and the semantic category difference degree is used as the energy value of the pixel. The semantic category of each pixel is updated iteratively through gradient descent until the energy values ​​of all pixels converge to obtain the scene semantic segmentation map.

[0098] In the step of calculating feature correlation, the state feature matrix is ​​first obtained, which contains data samples across multiple feature dimensions. For each pair of feature dimensions in the matrix, the corresponding data samples are extracted and processed. For example, for feature dimensions i and j, their data sample sets Xi and Xj are extracted. The covariance Cov(Xi,Xj) of these two sample sets is calculated, along with their respective standard deviations σi and σj. The Pearson correlation coefficient rij is obtained by dividing the covariance by the product of the two standard deviations, i.e., rij = Cov(Xi,Xj) / (σi×σj). In a practical case, assuming the feature matrix has 5 dimensions, a 5×5 correlation coefficient matrix R is calculated, where each element rij represents the degree of correlation between the i-th feature and the j-th feature, with values ​​ranging from [-1,1]. For example, if r12 = 0.85, it indicates that feature 1 and feature 2 are highly positively correlated; if r34 = -0.72, it indicates that feature 3 and feature 4 are highly negatively correlated.

[0099] In the feature mapping step, the feature correlation matrix R obtained in the previous step is decomposed into a linear combination of orthogonal components. Specifically, R is decomposed into R = QΛQ through eigenvalue decomposition. TΛ is a diagonal eigenvalue matrix, where Q is the eigenvector matrix and Λ is the eigenvalue matrix. The top k principal orthogonal components are selected based on the magnitude of the eigenvalues; for example, k=3 indicates selecting the orthogonal components corresponding to the three largest eigenvalues. These orthogonal components are weighted and superimposed to map the feature correlation to a three-dimensional spatial domain. The weighting coefficients can be the square roots of the corresponding eigenvalues, ensuring that important correlation patterns receive higher weights. The mapped feature representation is input into a multi-layer deconvolution structure. This structure contains multiple deconvolution layers, each performing upsampling and nonlinear transformation operations. During upsampling, the deconvolution kernel is expanded; for example, if the upsampling coefficient is 2, a zero value is inserted between adjacent elements of the deconvolution kernel; if the input feature size is 32×32, it becomes 64×64 after a 2x upsampling. Simultaneously, zero values ​​are also inserted between adjacent elements of the input feature map to expand its size. The expanded deconvolution kernel is then used to convolve the expanded input features to obtain an enlarged feature map. The nonlinear transformation uses the ReLU activation function, setting negative values ​​in the enlarged feature map to zero while retaining positive values. This process is passed layer by layer in the multi-layer deconvolution structure until a scene feature map with the same size as the original scene is generated. For example, if the original scene size is 256×256 and the initial feature map size is 16×16, then four 2x upsampling operations are required to finally obtain a 256×256 scene feature map.

[0100] In the semantic segmentation step, the scene feature map generated in the previous step is processed. For each pixel p in the feature map, its Euclidean distance to each semantic category feature template is calculated. Assuming there are m semantic categories, corresponding to m feature templates T1, T2, ..., Tm, for the feature vector Fp of pixel p, m distance values ​​d1, d2, ..., dm are calculated, where di represents the Euclidean distance between Fp and Ti. These distance values ​​serve as the cost for assigning pixel p to each semantic category. Simultaneously, the L2 norm of the difference between the feature vectors of pixel p and its four neighboring pixels (top, bottom, left, and right) is calculated to obtain the semantic category dissimilarity between adjacent pixels. For example, for pixel p and its upper neighbor pixel q, ||Fp-Fq||2 is calculated as the dissimilarity between them. The weighted sum of the cost value and the semantic category difference is used as the energy value E(p) of a pixel, i.e., E(p) = λ1·min(d1,d2,...,dm) + λ2·∑||Fp-Fq||2, where λ1 and λ2 are weight coefficients, which can be set to λ1=0.7 and λ2=0.3. The semantic category of each pixel is iteratively updated using the gradient descent method, selecting the category that reduces the energy value in each iteration. The iteration stops when the energy value change of all pixels is less than a preset threshold (e.g., 0.001) or the maximum number of iterations (e.g., 100) is reached, resulting in a scene semantic segmentation map. In the segmentation result, each pixel is assigned to a specific semantic category, such as road, building, vehicle, pedestrian, etc., forming a complete scene semantic understanding.

[0101] In one optional implementation, scene rendering is performed based on the scene semantic segmentation map combined with the risk area and project progress information to obtain a visualized three-dimensional dynamic construction progress, including:

[0102] The semantic label value of each pixel is determined according to the engineering component type of the corresponding region of the scene semantic segmentation map. The risk level value of the corresponding position is determined based on the risk assessment result of the risk region. The semantic label value and the risk level value are weighted and fused to obtain the fusion mapping value.

[0103] Based on the three-dimensional digital model, the depth information of the construction scene is obtained. The coordinate transformation is used to map the two-dimensional pixel coordinates in the scene semantic segmentation map to the corresponding positions in the three-dimensional digital model. The spatial height value is determined based on the depth information in the three-dimensional digital model. A corresponding three-dimensional construction space coordinate system is generated according to the two-dimensional pixel coordinates and the spatial height value.

[0104] In the three-dimensional construction space coordinate system, the construction status identifier of the engineering component is set according to the fusion mapping value, and the engineering components with different construction statuses are mapped to different display styles. The construction status identifier of the engineering component is dynamically updated based on the project progress information, and the completion index of the engineering component is converted into the corresponding fill ratio for visualization, thereby generating a three-dimensional construction scene with dynamic display effect of construction progress.

[0105] The semantic label value of each pixel is determined based on the type of engineering component in the corresponding region of the scene semantic segmentation map. During implementation, after the camera acquires images of the construction site, a deep learning model performs semantic segmentation on the images, classifying each pixel into different types of engineering components, such as concrete components, steel components, and mechanical equipment. For each pixel (x, y) in the image, a semantic label value L(x, y) is assigned according to its type of engineering component. Specifically, the semantic label value can be set to 1 for concrete components, 2 for steel components, 3 for mechanical equipment, 4 for workers, and so on. For example, for a construction site image with a resolution of 1920×1080, after semantic segmentation, a label matrix of the same size can be obtained, where the pixel at coordinates (500, 600) is identified as a concrete component, then L(500, 600) = 1.

[0106] The risk level value for a corresponding location is determined based on the risk assessment results of the risk area. The risk assessment is based on multiple factors, including risks associated with working at heights, machinery operation, and material storage. For each pixel (x, y) in the image, a risk level value R(x, y) is assigned according to the risk level of its surrounding area. For example, the risk level is divided into three levels: low risk, medium risk, and high risk, corresponding to risk level values ​​of 0.2, 0.5, and 0.8, respectively. In practical applications, if a concrete component at coordinates (500, 600) is located in a high-altitude work area, its risk level value R(500, 600) = 0.8.

[0107] The semantic label value and the risk level value are weighted and fused to obtain the fusion mapping value. For each pixel (x,y) in the image, its fusion mapping value F(x,y) is calculated. In this embodiment, a weighted average method is used for fusion, i.e., F(x,y) = α×L(x,y) + β×R(x,y), where α and β are weight coefficients, satisfying α+β=1. Specifically, α=0.7 and β=0.3 can be set, indicating that the semantic label dominates in the fusion mapping, while considering the influence of risk factors. For example, for the pixel at coordinates (500,600), its fusion mapping value F(500,600) = 0.7×1 + 0.3×0.8 = 0.94.

[0108] Depth information of a construction scene is obtained based on a 3D digital model. The 3D digital model can be a BIM model or a point cloud model, containing the 3D coordinates of each point in the construction scene. Using devices such as depth cameras or LiDAR, the depth information D(x,y) of each point in the construction scene can be obtained, representing the distance from the camera to the point in the scene. For example, for a pixel at coordinates (500, 600), its depth value D(500, 600) = 15.6 meters.

[0109] Coordinate transformation is used to map the 2D pixel coordinates in the scene semantic segmentation image to their corresponding positions in the 3D digital model. Let the camera's intrinsic parameter matrix be K, and its extrinsic parameter matrix be [R|t], where R is the rotation matrix and t is the translation vector. For a 2D pixel coordinate (x, y) and its depth value D(x, y) in the image, its 3D coordinates (X, Y, Z) can be calculated. Specifically, the normalized camera coordinates are first calculated, and then the world coordinates are obtained through extrinsic parameter matrix transformation. For example, for a pixel at coordinates (500, 600), its 3D coordinates are (25.3, 17.8, 15.6) meters.

[0110] Spatial height values ​​are determined based on depth information from a 3D digital model. For a point (X, Y, Z) in 3D space, its spatial height value H is the Z coordinate value. In practical applications, terrain undulations need to be considered, in which case the relative height can be obtained by subtracting the ground height from the Z coordinate value. For example, for a point with coordinates (25.3, 17.8, 15.6), if the ground height is 0.5 meters, then its spatial height value H = 15.1 meters.

[0111] A corresponding three-dimensional construction space coordinate system is generated based on the two-dimensional pixel coordinates and spatial height values. The three-dimensional construction space coordinate system includes horizontal plane coordinates (X,Y) and height coordinates Z, forming a complete three-dimensional coordinate system (X,Y,Z). For ease of construction management, the world coordinate system can be converted into a construction coordinate system with a certain reference point on the construction site as the origin. For example, if the southwest corner of the construction site is set as the origin (0,0,0), then the point (25.3,17.8,15.6) is represented in the construction coordinate system as (25.3-0,17.8-0,15.6-0.5)=(25.3,17.8,15.1).

[0112] In the three-dimensional construction space coordinate system, the construction status identifier of the engineering component is set according to the fusion mapping value. Different construction status identifiers S(X,Y,Z) can be assigned to points (X,Y,Z) in the three-dimensional space based on the fusion mapping value F(x,y). For example, fusion mapping values ​​of 0-0.3 can be set to correspond to the "not started" state, 0.3-0.6 to the "in progress" state, and 0.6-1.0 to the "completed" state. For a point with a fusion mapping value F(500,600) = 0.94, its construction status identifier is "completed".

[0113] Different display styles are mapped to engineering components in different construction states. Different display styles, including color, transparency, and texture, can be set for different construction state indicators. For example, components in the "Not Started" state are displayed as gray semi-transparent (RGB:128,128,128, transparency:50%), components in the "In Progress" state are displayed as yellow opaque (RGB:255,255,0, transparency:0%), and components in the "Completed" state are displayed as green opaque (RGB:0,255,0, transparency:0%). Furthermore, a red warning effect (RGB:255,0,0, transparency:30%) can be overlaid on high-risk areas.

[0114] The construction status indicators of engineering components are dynamically updated based on project progress information. Project progress information includes planned start time, planned finish time, actual start time, actual finish time, and percentage of completion. As construction progresses, this information is continuously updated, and correspondingly, the construction status indicators of engineering components also need to be updated dynamically. For example, if the planned construction period for a concrete wall is 10 days, and 7 days have been completed, representing 70% completion, then its construction status indicator will be "In Progress," displayed as a yellow opaque color.

[0115] The completion indicators of engineering components are converted into corresponding fill ratios for visual representation. For engineering components in the "in progress" stage, their completion level can be intuitively represented by the fill ratio. For example, for a concrete wall with a completion level of 70%, it can be filled to 70% of its height from bottom to top in the 3D model. The filled part is displayed in green (completed part), and the unfilled part is displayed as gray semi-transparent (unfinished part). In this way, by observing the fill status in the 3D model, managers can intuitively understand the completion status of each engineering component.

[0116] Generate a 3D construction scene with dynamic display of construction progress. Through the above steps, a 3D visualization scene containing construction progress information and risk area information can be generated. This scene supports dynamic updates, can display the construction status at different points in time, and allows for interactive operations such as panning, zooming, and rotating, helping construction managers to fully understand project progress and risk situations.

[0117] This invention relates to a multi-party collaborative progress visualization monitoring system for intelligent building facilities, the system comprising:

[0118] The first unit is used to collect historical construction data and multi-dimensional data information in intelligent building facilities, and to establish a three-dimensional digital model based on the multi-dimensional data information.

[0119] The second unit is used to establish a causal relationship identifier based on the temporal and correlation features in the historical construction data. The multi-dimensional data information is input into the causal relationship identifier to obtain a causal strength matrix. A causal graph is established based on the causal strength matrix. Based on the causal graph, the probability of abnormal propagation of construction is calculated using the Bayesian probability propagation algorithm. When the probability of abnormal propagation of a certain node is greater than the construction risk warning threshold, it is marked as a risk area in the three-dimensional digital model.

[0120] The third unit is used to map the causal graph to the three-dimensional digital model, generate a construction scene monitoring data stream, extract features through a MobileNet convolutional neural network, and perform collaborative analysis of the extracted feature information through a self-organizing network to obtain the state feature matrix of the construction scene.

[0121] The fourth unit is used to calculate the feature correlation of the state feature matrix using the Pearson correlation coefficient, and to map the feature correlation into a scene semantic segmentation map using a multi-layer deconvolution structure; based on the scene semantic segmentation map and the risk area and project progress information, scene rendering is performed to obtain a visualized three-dimensional dynamic construction progress.

[0122] A third aspect of the present invention provides an electronic device, comprising:

[0123] processor;

[0124] Memory used to store processor-executable instructions;

[0125] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0126] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0127] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0128] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-party collaborative progress visualization monitoring method for intelligent building facilities, characterized in that, include: Collect historical construction data and multi-dimensional data information from intelligent building facilities, and establish a three-dimensional digital model based on the multi-dimensional data information; A causal relationship identifier is established based on the temporal and correlation characteristics of the historical construction data. The multi-dimensional data information is input into the causal relationship identifier to obtain a causal strength matrix. A causal graph is then established based on the causal strength matrix, including: Collect time series and parameter series from the historical construction data, calculate the similarity features between the time series through dynamic time warping, calculate the trend intensity features of the time series through a sliding time window, and concatenate the similarity features and the trend intensity features to obtain the time series features; Calculate the correlation coefficient between the parameter sequences and construct a correlation coefficient matrix. After performing feature decomposition on the correlation coefficient matrix, select the feature vectors corresponding to the largest feature values ​​of a preset number of features as the associated features. The time-series features and the association features are combined and subjected to conditional independence and time-delay causality tests to obtain a causal relationship identifier. The conditional independence test obtains the instantaneous causal strength matrix by calculating the difference between the mutual information and conditional mutual information between the parameter sequences. The time-delay causality test obtains the delayed sequence by performing time delay processing on the parameter sequences, and calculates the ratio of the mutual information entropy between the delayed sequence and the current sequence to obtain the time-delay causality strength matrix. The multi-dimensional data information is input into the causal relationship identifier, the instantaneous causal strength matrix and the time-delayed causal strength matrix are calculated and weighted to obtain the causal strength matrix; Based on the causal strength matrix, causal relationships with causal strength greater than a preset association threshold are taken as directed edges, and the corresponding construction parameters are taken as nodes. The maximum spanning tree algorithm is used to select the edge with the largest causal strength to ensure the connectivity of the graph, thus obtaining a causal graph. Based on the causal graph, the probability of abnormal propagation during construction is calculated using the Bayesian probability propagation algorithm. When the probability of abnormal propagation at a certain node is greater than the construction risk warning threshold, it is marked as a risk area in the three-dimensional digital model. The causal graph is mapped to the three-dimensional digital model to generate a construction scene monitoring data stream. Feature extraction is performed through MobileNet convolutional neural network. The extracted feature information is then collaboratively analyzed through a self-organizing network to obtain the state feature matrix of the construction scene. The feature correlation of the state feature matrix is ​​calculated by Pearson correlation coefficient, and the feature correlation is mapped to a scene semantic segmentation map by a multi-layer deconvolution structure. Based on the scene semantic segmentation map and the risk area and project progress information, scene rendering is performed to obtain a visualized three-dimensional dynamic construction progress.

2. The method according to claim 1, characterized in that, Based on the causal graph, the probability of anomaly propagation during construction is calculated using the Bayesian probability propagation algorithm. When the probability of anomaly propagation at a certain node exceeds the construction risk warning threshold, the area marked as a risk region in the three-dimensional digital model includes: The directed connections between the construction parameter nodes and their causal relationship edges in the causal graph are constructed into a Bayesian network. The frequency of co-occurrence between each node and its parent node is counted, and the conditional probability of the node is calculated based on the ratio of the co-occurrence frequency to the frequency of the parent node. For each node in the Bayesian network, the deviation between the node's current parameter value and its historical mean is calculated. When the deviation exceeds a preset multiple of the standard deviation, the node is determined to be in an abnormal state; otherwise, it is in a normal state. Based on the Bayesian network, the probability propagation value between nodes is calculated. The probability propagation value is the product of the conditional probability of the parent node and the conditional probabilities of all neighboring nodes received by the parent node. The probability propagation value passed from the parent node is multiplied by the probability propagation value passed from the child node and normalized to obtain the belief value of the node. When a node is in the abnormal state, the node's belief value is used as the probability of the abnormal state; when a node is in the normal state, 1 minus the node's belief value is used as the probability of the abnormal state. The abnormal state probability is obtained by weighted summing the conditional probabilities of all its child nodes; When the probability of abnormal propagation of a node is greater than the construction risk warning threshold, the spatial coordinates of the node in the three-dimensional digital model are determined. Taking the spatial coordinates as the center, the ratio of the probability of abnormal propagation to the construction risk warning threshold is calculated and multiplied by the baseline influence radius to obtain the risk influence radius. The risk area is marked within the risk influence radius.

3. The method according to claim 1, characterized in that, The causal graph is mapped to the three-dimensional digital model to generate a construction scene monitoring data stream. Feature extraction is performed using a MobileNet convolutional neural network. The extracted feature information is then collaboratively analyzed using a self-organizing network to obtain the state feature matrix of the construction scene, which includes: Obtain the physical spatial coordinates of the nodes in the causal graph, and establish a mapping between the physical spatial coordinates and the spatial position coordinates of the corresponding engineering components in the three-dimensional digital model to generate node position mapping information; Multimodal data is collected at the physical space coordinates indicated by the node location mapping information and integrated into a construction scene monitoring data stream in chronological order; The construction scene monitoring data stream is input into the MobileNet convolutional neural network. Features are extracted from each data channel through a deep convolutional layer, and then the features extracted from each data channel are combined into a multimodal feature vector through a point-state convolutional layer. The multimodal feature vector is input into the self-organizing neural network. The Euclidean distance between the multimodal feature vector and the weights of the network neurons is calculated. The neuron with the smallest Euclidean distance is selected as the winning neuron. The learning rate decay coefficient and the range of neighboring neurons of the winning neuron are determined. The difference between the multimodal feature vector and the current weights of the winning neuron is multiplied by the learning rate decay coefficient to obtain the weight adjustment amount of the winning neuron. Calculate the topological distance between the neighboring neurons and the winning neuron, determine the weight adjustment coefficient of the neighboring neurons based on the topological distance, multiply the weight adjustment coefficient by the weight adjustment amount of the winning neuron to obtain the weight adjustment amount of the neighboring neurons, and superimpose the weight adjustment amounts of the winning neuron and its neighboring neurons onto the corresponding current weights and reorganize them into a state feature matrix according to the feature dimension.

4. The method according to claim 1, characterized in that, The feature correlation of the state feature matrix is ​​calculated using the Pearson correlation coefficient, and the feature correlation is mapped to a scene semantic segmentation map using a multi-layer deconvolution structure, including: Data samples are obtained for every two feature dimensions in the state feature matrix. The covariance and standard deviation of the data samples are calculated. The covariance is divided by the product of the standard deviations of the two feature dimensions to obtain the Pearson correlation coefficient between the feature dimensions. A feature correlation matrix is ​​generated based on the Pearson correlation coefficient. The feature correlation matrix is ​​decomposed into a linear combination of orthogonal components. The feature correlation is mapped to the three-dimensional spatial domain by weighted superposition of orthogonal components of different scales and then input into a multi-layer deconvolution structure. The multi-layer deconvolution structure performs upsampling and nonlinear transformation in sequence. The upsampling fills the deconvolution kernel with multiples of the upsampling coefficients and inserts zero values ​​between adjacent elements of the input features. The deconvolution kernel is used to calculate and obtain an enlarged feature map. The nonlinear transformation sets the negative values ​​in the enlarged feature map to zero. This process is passed layer by layer in the multi-layer deconvolution structure until a scene feature map with the same size as the scene is generated. In the scene feature map, the Euclidean distance between each pixel and each semantic category feature template is calculated to obtain the cost value of the pixel being assigned to different semantic categories. At the same time, the L2 norm of the difference between the feature vectors of each pixel and its four neighboring pixels is calculated to obtain the semantic category difference degree of the adjacent pixels. The weighted sum of the cost value and the semantic category difference degree is used as the energy value of the pixel. The semantic category of each pixel is updated iteratively by gradient descent until the energy values ​​of all pixels converge, thus obtaining the scene semantic segmentation map.

5. The method according to claim 1, characterized in that, Based on the scene semantic segmentation map and the risk area and project progress information, scene rendering is performed to obtain a visualized 3D dynamic construction progress, including: The semantic label value of each pixel is determined according to the engineering component type of the corresponding region of the scene semantic segmentation map. The risk level value of the corresponding position is determined based on the risk assessment result of the risk region. The semantic label value and the risk level value are weighted and fused to obtain the fusion mapping value. Based on the three-dimensional digital model, the depth information of the construction scene is obtained. The coordinate transformation is used to map the two-dimensional pixel coordinates in the scene semantic segmentation map to the corresponding positions in the three-dimensional digital model. The spatial height value is determined based on the depth information in the three-dimensional digital model. A corresponding three-dimensional construction space coordinate system is generated according to the two-dimensional pixel coordinates and the spatial height value. In the three-dimensional construction space coordinate system, the construction status identifier of the engineering component is set according to the fusion mapping value, and the engineering components with different construction statuses are mapped to different display styles. The construction status identifier of the engineering component is dynamically updated based on the project progress information, and the completion index of the engineering component is converted into the corresponding fill ratio for visualization, thereby generating a three-dimensional construction scene with dynamic display effect of construction progress.

6. A multi-party collaborative progress visualization monitoring system for intelligent building facilities, used to implement the method as described in any one of claims 1-5, characterized in that, include: The first unit is used to collect historical construction data and multi-dimensional data information in intelligent building facilities, and to establish a three-dimensional digital model based on the multi-dimensional data information. The second unit is used to establish a causal relationship identifier based on the temporal and correlation features in the historical construction data, input the multi-dimensional data information into the causal relationship identifier to obtain a causal intensity matrix, and establish a causal graph based on the causal intensity matrix. Based on the causal graph, the probability of abnormal propagation during construction is calculated using the Bayesian probability propagation algorithm. When the probability of abnormal propagation at a certain node is greater than the construction risk warning threshold, it is marked as a risk area in the three-dimensional digital model. The third unit is used to map the causal graph to the three-dimensional digital model, generate a construction scene monitoring data stream, extract features through a MobileNet convolutional neural network, and perform collaborative analysis of the extracted feature information through a self-organizing network to obtain the state feature matrix of the construction scene. The fourth unit is used to calculate the feature correlation of the state feature matrix through the Pearson correlation coefficient, and to map the feature correlation into a scene semantic segmentation map using a multi-layer deconvolution structure. Based on the scene semantic segmentation map and the risk area and project progress information, scene rendering is performed to obtain a visualized three-dimensional dynamic construction progress.

7. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 5.

Images