Multi-source data fusion method and system based on intelligent construction management cloud platform

Abstract

The present application relates to a kind of multi-source data fusion method and system based on wisdom construction management cloud platform, multi-source data fusion method includes obtaining the multimodal operating condition data uploaded to cloud platform, for different types of multimodal operating condition data, respectively extract operating condition characteristic data;Form a unified construction state feature vector based on operating condition characteristic data;Construction state feature vector is input to weight distribution model in cloud platform, to obtain dynamic weight vector, wherein, weight distribution model is configured to dynamically determine the weight condition of data source when data weighted fusion according to the construction condition reflected by current construction state feature, each weight in dynamic weight vector corresponds to one or a kind of data source;Using dynamic weight vector to carry out weighted fusion to multi-source data stream, form the operating condition description after fusion.Can make that the fusion weight of data source is fully applicable to the actual characteristics of construction.

Description

Technical Field

[0001] This invention relates to the field of smart construction management, and in particular to a multi-source data fusion method and system based on a smart construction management cloud platform. Background Technology

[0002] The intelligent construction management cloud platform is a construction management platform based on Internet and IoT technologies, enabling construction companies to achieve collaborative control over aspects such as progress, cost, and quality. The intelligent construction management cloud platform can access and integrate multi-source data, including real-time sensor and operation and maintenance data, and perform cloud-native real-time processing of heterogeneous data such as equipment sensor and environmental data. However, when accessing multiple data sources such as video surveillance, sensor networks, inspection terminals, and business databases, the reliability and importance of data sources are difficult to determine due to the complexity and variability of the construction environment, making traditional data weighting methods unsuitable for the construction field. Summary of the Invention

[0003] The purpose of this application is to propose a multi-source data fusion method and system based on a smart construction management cloud platform to solve the problems of complex and ever-changing construction environments, difficulty in determining the reliability and importance of data sources, and the inapplicability of traditional data weight determination methods to the construction field.

[0004] The multi-source data fusion method based on the smart construction management cloud platform in this application includes:

[0005] Acquire multimodal operating condition data uploaded to the cloud platform, and extract operating condition feature data for different types of multimodal operating condition data respectively;

[0006] Based on the aforementioned working condition feature data, a unified construction state feature vector is formed;

[0007] The construction status feature vector is input into the weight allocation model in the cloud platform to obtain a dynamic weight vector. The weight allocation model is configured to dynamically determine the weights for data weighting and fusion based on the construction status reflected by the current construction status features. Each weight in the dynamic weight vector corresponds to one or a type of data source.

[0008] The dynamic weight vector is used to perform weighted fusion of multi-source data streams to form a fused operating condition description.

[0009] Optionally, the weight allocation model includes a feature interaction layer for explicitly learning the joint influence of different state features on weight decisions. The feature interaction layer is a bilinear interactive pooling layer or a pairwise interaction layer, used to calculate the second-order combined features among the features of each dimension of the construction state feature vector.

[0010] Optionally, the training process of the weight allocation model includes:

[0011] Construct a training sample set, where each sample includes working condition feature data generated from historical multimodal working condition data, and reference fusion weight labels;

[0012] Construct an initial neural network that includes the feature interaction layer;

[0013] The initial neural network is trained using a loss function that includes a constraint term, which is used to penalize excessively low values ​​of the prediction weights for safety-critical data sources.

[0014] When the training meets the convergence condition, the trained weight allocation model is obtained.

[0015] Optionally, the loss function is L = MSE(W_pred, W_label) + λ * Σ ReLU(δ - W_security), where L is the total loss, MSE is the mean squared error loss, W_pred is the weight assigned by the weight allocation model, W_label is the reference fusion weight label, λ is the regularization coefficient, ReLU is the linear rectified function, δ is the weight safety threshold for the safety-critical data source, and W_security is the weight predicted by the weight allocation model for the safety-critical data source.

[0016] Optionally, forming a unified construction state feature vector based on the working condition feature data includes concatenating various working condition feature data vectors and forming a unified construction state feature vector through a normalization layer.

[0017] The extraction of operating condition feature data for different types of multimodal operating condition data includes at least one of the following:

[0018] Extract current schedule feature data based on the description in the schedule file from the building information model;

[0019] Extract visual scene feature data from on-site images;

[0020] Semantic feature data is extracted from construction log text;

[0021] Real-time risk characteristic data is extracted based on on-site environmental sensors and equipment sensors.

[0022] Optionally, using the dynamic weight vector to perform weighted fusion of multi-source data streams includes:

[0023] Based on real-time working data from the construction site, one or more active work areas are identified using a spatial clustering algorithm;

[0024] Based on the affiliation between the data source and the active work area, the data sources at the construction site are divided into regional groups;

[0025] Obtain a construction event knowledge graph, which contains the association between predetermined events and data sources;

[0026] Based on the construction event knowledge graph, semantic association analysis is performed on the data sources, and data sources that may logically participate in the same predetermined event are associated to form semantic association groups.

[0027] Based on the regional grouping and the semantic association grouping, a data source grouping mapping table is formed to guide the fusion strategy of multi-source data streams.

[0028] Optionally, after forming the data source grouping mapping table based on the region grouping and the semantic association group, the method further includes:

[0029] Based on the grouping of the data source grouping mapping table and the weights guided by the dynamic weight vector, the original data streams of the data sources within the same group are fused to form the intra-group feature fusion result.

[0030] The fusion results of the intra-group features are then matched with a pre-constructed construction event knowledge graph to determine event patterns.

[0031] When a specific construction event is matched, the dynamic fusion weights corresponding to the data sources related to the event are recalibrated and improved based on the matched construction event, and the confidence of the work condition description is adjusted.

[0032] A working condition description report is generated based on the adjusted results.

[0033] Optionally, the recalibration and enhancement of the dynamic fusion weights corresponding to the data sources related to the matched construction events includes:

[0034] Identify the key data source for the construction event based on the matched construction events;

[0035] Based on the importance of the construction event in the knowledge graph, the dynamic fusion weight of the key data source in subsequent fusion is increased proportionally.

[0036] Optionally, adjusting the confidence level of the operating condition description includes:

[0037] Based on the historical frequency of the matched construction events, the completeness of the current event pattern matching, and the contextual consistency of the key data source data, the adjusted confidence level is obtained through Bayesian update or weighted synthesis.

[0038] On the other hand, this application also provides a multi-source data fusion system based on a smart construction management cloud platform, including:

[0039] The feature extraction unit is configured to acquire multimodal working condition data uploaded to the cloud platform and extract working condition feature data for different types of multimodal working condition data.

[0040] The feature merging unit is configured to form a unified construction state feature vector based on the working condition feature data;

[0041] A dynamic weighting unit is configured to input the construction state feature vector into the weight allocation model in the cloud platform to obtain a dynamic weighting vector. The weight allocation model is configured to dynamically determine the weights for data weighting and fusion based on the construction status reflected by the current construction state features. Each weight in the dynamic weighting vector corresponds to one or a type of data source.

[0042] The weighted fusion unit is configured to use the dynamic weight vector to perform weighted fusion of multi-source data streams to form a fused operating condition description.

[0043] The multi-source data fusion method based on a smart construction management cloud platform provided in this application determines the weight allocation of data sources based on construction status characteristics using a dedicated weight allocation model. These construction status characteristics are extracted from multimodal working condition data and reflect the current actual construction progress and environment. This allows the fusion weights of data sources to be dynamically determined as construction progresses, effectively reflecting the importance of each data source at different project stages and under different construction environments. This ensures that the fusion weights of the data sources are fully applicable to the actual characteristics of construction. As the underlying data fusion processing part, the multi-source data fusion method in this embodiment provides cleaner and more reliable data support for subsequent analysis and decision-making processes. Attached Figure Description

[0044] Figure 1 A basic flowchart illustrating the multi-source data fusion method based on a smart construction management cloud platform provided in this application embodiment;

[0045] Figure 2 This is a schematic diagram of the structure of a multi-source data fusion system based on a smart construction management cloud platform provided in an embodiment of this application. Detailed Implementation

[0046] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0047] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0048] Example 1:

[0049] This embodiment provides a multi-source data fusion method based on a smart construction management cloud platform. Please refer to [link to relevant documentation]. Figure 1 This method includes, but is not limited to, the following steps:

[0050] S101. Obtain multimodal operating condition data uploaded to the cloud platform, and extract operating condition feature data for different types of multimodal operating condition data respectively;

[0051] In this embodiment, the smart construction management cloud platform can acquire data files related to construction planning, including but not limited to progress schedules. Simultaneously, sensor data and monitoring data from construction equipment at the construction site can also be uploaded to the smart construction management cloud platform. The multimodal working condition data in this embodiment is a collective term for this data that reflects the planned construction situation or the current construction situation. For example, multimodal working condition data can be various types of data from different sources, including but not limited to BIM (Building Information Modeling) model files, text data, and image data.

[0052] S102. Based on the working condition feature data, a unified construction state feature vector is formed;

[0053] In some implementations, forming a unified construction state feature vector based on working condition feature data includes concatenating various working condition feature data vectors and forming a unified construction state feature vector through a normalization layer.

[0054] For different types of multimodal operating condition data, operating condition feature data are extracted, including at least one of the following:

[0055] Extract current schedule feature data based on the description in the schedule file from the building information model;

[0056] Extract visual scene feature data from on-site images;

[0057] Semantic feature data is extracted from construction log text.

[0058] Schedule files from Building Information Modeling (BIM) include, but are not limited to, .ifc or .xml formats. For example, a schedule file can be parsed to extract a list of construction components currently in the "in progress" state. These components are then coded and normalized based on their type (e.g., "foundation," "column," "wall," etc.), their respective sub-projects (e.g., "main structure," "decoration"), and planned completion percentage. This step can generate a fixed-dimensional schedule feature data vector.

[0059] The on-site images can be taken by drones or fixed cameras. In practical applications, these images can be real-time or phased images taken at predetermined intervals such as daily or weekly. In this embodiment, a lightweight convolutional neural network (such as MobileNetV3) can be used to extract visual scene features. In this embodiment, the lightweight convolutional neural network is pre-trained on a self-built construction phase image dataset, which contains a large number of construction site images labeled with phases (such as "foundation pit excavation", "steel structure hoisting", "curtain wall installation", etc.). The on-site images are input into the lightweight convolutional neural network, its feature map before the global average pooling layer is extracted, and dimensionality reduction is performed through a fully connected layer to finally obtain a visual scene feature data vector.

[0060] Construction log text includes, but is not limited to, plain text data such as supervision logs and daily construction reports. Semantic features can be extracted using a pre-trained natural language processing (NLP) model. The NLP model in this embodiment is trained on construction-related professional corpus and is capable of understanding construction-related terminology. This process yields a semantic feature data vector.

[0061] Real-time risk characteristic data vectors are formed by aggregating information from environmental sensors (such as wind speed and rainfall) and equipment status alarms (such as overload and offline).

[0062] The aforementioned feature data, after being spliced ​​and batch normalized, forms a construction status feature vector, which will serve as the input to the weight allocation model.

[0063] S103. Input the construction status feature vector into the weight allocation model in the cloud platform to obtain the dynamic weight vector;

[0064] The weighting model is configured to dynamically determine the weights for data fusion based on the construction conditions reflected by the current construction status characteristics. Each weight in the dynamic weight vector corresponds to one or a class of data sources.

[0065] S104. Use dynamic weight vectors to weight and fuse multi-source data streams to form a fused working condition description;

[0066] The multi-source data fusion method in this embodiment determines the weight allocation of data sources based on construction status characteristics using a dedicated weight allocation model. These construction status characteristics, extracted from multimodal working condition data, reflect the current actual construction progress and environment, allowing the fusion weights of data sources to be dynamically determined as construction progresses. This effectively reflects the importance of each data source at different project stages and under different construction environments, ensuring that the fusion weights are fully applicable to the actual characteristics of construction. As the underlying data fusion processing part, this multi-source data fusion method provides cleaner and more reliable data support for subsequent analysis and decision-making processes (such as anomaly diagnosis).

[0067] In some implementations, the weighting model includes a feature interaction layer for explicitly learning the joint influence of different state features on weight decisions. The feature interaction layer is a bilinear interactive pooling layer or a pairwise interactive layer, used to calculate the second-order combined features among the features of each dimension of the construction state feature vector.

[0068] In multi-source data fusion decision-making at construction sites, the effect of a single feature is often coupled with other features. For example, the "high wind speed" feature obtained from a wind speed sensor may only slightly increase the weight of construction equipment, but when it appears simultaneously with the "tower crane hoisting" feature described in construction phases or site images, it should have a strong synergistic boosting effect on the weights of tower crane monitoring videos and wind speed sensor data, while potentially suppressing the weights of ground inspection videos. The feature interaction layer aims to explicitly and efficiently model and extract the second-order combination relationships between these feature pairs, injecting them as higher-order features into the network, enabling the weight allocation model to understand the complex relationships within the construction scenario.

[0069] The following examples illustrate two implementation methods for the feature interaction layer.

[0070] In one example, the feature interaction layer employs a bilinear interactive pooling layer. For the input construction state feature vector, two different linear transformations are performed to obtain two feature maps: V_i = S · W_i + b_i, i ∈ {1,2}, where W_i is the trainable weight matrix and b_i is the bias. The outer product of the two feature maps V_1 and V_2 on each sample is calculated to obtain the interaction matrix. The elements in the interaction matrix capture the interaction strength between the j-th and k-th dimensions of the original features. The interaction matrix is ​​flattened into a long vector, which represents the second-order combination information between all feature pairs. This vector is concatenated with the original construction state feature vector to form an enhanced feature vector. This step ensures that the weight allocation model retains both the original semantic information and injects the pattern information of the interactions between features.

[0071] In one example, the feature interaction layer employs a pairwise interaction layer, a simplified and efficient implementation of bilinear interactive pooling, which directly computes the combination of pairwise multiplications of each element of the feature vector. The vector F_interaction output by the pairwise interaction layer can be obtained by flattening the lower triangular part (excluding the diagonal) of the formula F_interaction = (S * S^T), which avoids introducing an additional weight matrix and makes the computation more efficient. Similarly, this vector is passed to the next layer and concatenated with the original construction state feature vector.

[0072] The feature interaction layer in this embodiment learns complex interaction patterns between features automatically through a data-driven approach, without the need for manual definition of specific interaction rules. It is highly adaptable and can discover unknown effective combinations.

[0073] In some implementations, the training process of the weight allocation model includes:

[0074] Construct a training sample set, where each sample includes working condition feature data generated from historical multimodal working condition data, and reference fusion weight labels;

[0075] Construct an initial neural network that includes a feature interaction layer;

[0076] The initial neural network is trained using a loss function that includes a constraint term, which penalizes excessively low prediction weights for safety-critical data sources.

[0077] When the training meets the convergence condition, the trained weight allocation model is obtained.

[0078] In this embodiment, the training sample set includes time-aligned multimodal raw data, including but not limited to building information model progress snapshots, on-site images / videos, supervision log text, and sensor time-series data. Special events that have occurred (e.g., abnormal events) are labeled, recording their occurrence time, type, and scope of impact. Based on each historical time point in the training sample set, a construction status feature vector is formed according to the method described above in this embodiment. Using time point t as a baseline, a time window is looked back to check if any labeled special events have occurred within this time window. If no event has occurred, the situation at task time point t is stable, and the optimal weight labels should tend to make the fusion result most stable and energy-efficient. If a special event has occurred, a backtracking analysis is performed: experts or a simulation review system can evaluate how to adjust the fusion weights of each data source at time t to make the fusion result generated based on the data at time t most clearly and timely reveal the signs that the special event is about to occur. For example, if the special event is "tower crane tilting," backtracking may find that increasing the weights of "tower crane monitoring video" and "tilt sensor" and decreasing the weights of irrelevant sources at time t can trigger an effective warning in the fusion result earliest. The weights derived from this backtracking are the optimal weight labels at time t. The construction state feature vector S(t) at time t and the corresponding optimal weight labels constitute a training sample. In practical applications, a large number of training samples are needed to form training, validation, and test sets.

[0079] In this embodiment, an initial neural network is constructed, which includes a feature interaction layer for explicitly learning the joint influence of different state features on weight decisions. Constructing the initial neural network includes determining the input layer dimension, determining the specific type of the feature interaction layer, determining the number of layers, neurons, and activation functions of the subsequent multilayer perceptron, and determining the output layer dimension.

[0080] In this embodiment, a loss function with constraints is used for training. The constraints are used to penalize the excessively low predicted weights of safety-critical data sources. This ensures that while pursuing the overall accuracy of weight allocation, the weight allocation model also forcibly embeds prior knowledge in the construction safety domain, avoiding the dangerous situation where the weight allocation model suppresses the weights of safety-critical data sources too much in order to minimize the main loss.

[0081] In some implementations, the loss function is L = MSE(W_pred, W_label) + λ * Σ ReLU(δ -W_security), where L is the total loss, MSE is the mean squared error loss, W_pred is the weight assigned by the weight allocation model, W_label is the reference fusion weight label, λ is the regularization coefficient, ReLU is the linear rectified function, δ is the weight safety threshold for the safety-critical data source, and W_security is the weight predicted by the weight allocation model for the safety-critical data source.

[0082] The loss function in this embodiment comprises two parts: a main loss term and a constraint term, to simultaneously optimize the accuracy and security of the weight allocation model, effectively addressing the actual needs of construction supervision scenarios. MSE(W_pred, W_label) is the main loss term, driving the model to learn historical optimal decisions. λ * Σ ReLU(δ - W_security) serves as a security regularizer, forcibly embedding construction safety regulations as prior knowledge into the weight allocation model, preventing the model from ignoring critical safety information during optimization. In some implementations, δ is set based on domain knowledge and engineering consensus, and can be determined by analyzing the lowest effective weight of effective early warning data sources in historical accident cases during fusion. In some implementations, λ can be determined through cross-validation, trying different λ values ​​(e.g., [0.1, 0.3, 0.5, 1.0]) on the validation set, selecting the value that ensures almost all weights of safety-critical data sources are higher than δ while maintaining a low main loss term.

[0083] To achieve accurate and efficient data fusion, some implementations employ a two-layer fusion strategy. Using dynamic weight vectors to weightedly fuse multi-source data streams includes:

[0084] Based on real-time working data from the construction site, one or more active work areas are identified using a spatial clustering algorithm;

[0085] Based on the affiliation between the data source and the active work area, the data sources at the construction site are divided into regional groups;

[0086] Obtain a construction event knowledge graph, which contains the relationship between pre-defined events and data sources;

[0087] Based on the construction event knowledge graph, semantic association analysis is performed on the data sources, and data sources that may logically participate in the same predetermined event are associated to form semantic association groups.

[0088] Based on regional grouping and semantic association groups, a data source grouping mapping table is formed to guide the fusion strategy of multi-source data streams.

[0089] In this embodiment, real-time operational data includes the real-time location of equipment or other physical objects, such as electronic tags, Bluetooth, GPS, and other information devices installed on construction equipment, work vehicles, and workers' safety helmets. By acquiring the location points of all objects and employing density-based clustering algorithms such as DBSCAN (Density-Based Spatial Clustering of Applications with Noise) or OPTICS (Ordering Points To Identify the Clustering Structure), dense areas of location points within the construction area are discovered. During this process, the neighborhood radius and minimum number of points can be set according to the actual site scale. This allows for the identification of the core location, approximate range, and equipment or other physical objects within the active work area of ​​the construction site. This embodiment divides the construction area and associated objects through clustering identification, adaptively determining active work areas as the actual construction progresses, providing real-time and accurate references for data correlation.

[0090] Furthermore, this embodiment also performs semantic association analysis on data sources based on the construction event knowledge graph, associating data sources that are logically likely to participate in the same predetermined event to form semantic association groups. For example, the precursors to the "foundation pit collapse" event recorded in the knowledge graph include "settlement within the foundation pit" and "vibration of the surrounding soil." When the system identifies a "foundation pit settlement monitoring point," even if the "surrounding vibration sensor" is set in another area (such as the subway protection zone outside the construction site boundary), the system will dynamically associate these two data sources into a semantic association group through graph reasoning.

[0091] It is understood that in this embodiment, data sources that are in the same active region or are associated in the knowledge graph are grouped together. The same data source may belong to both a grouping based on active operation regions and a semantic association group formed based on construction event knowledge graph associations.

[0092] After forming the data source grouping mapping table based on region grouping and semantic association groups, it also includes:

[0093] Based on the grouping of the data source grouping mapping table and the weights guided by the dynamic weight vector, the original data streams of the data sources within the same group are fused to form the intra-group feature fusion result.

[0094] The feature fusion results within the group are matched with the pre-built construction event knowledge graph to perform event pattern matching;

[0095] When a specific construction event is matched, the dynamic fusion weights corresponding to the data sources related to the event are recalibrated and improved based on the matched construction event, and the confidence of the work condition description is adjusted.

[0096] A working condition description report is generated based on the adjusted results.

[0097] When performing data fusion, the data sources in the groups are first partially fused, which significantly reduces the amount of data in a single calculation. Then, event pattern matching is performed on the partially fused data formed within the group, which can increase the fusion weight of relevant data sources based on historical events. The resulting work condition description report can better reflect the events that deserve attention, and helps to highlight data related to cross-modal anomalies during construction, providing more reliable data support for subsequent anomaly decision-making.

[0098] In some implementations, recalibrating and improving the dynamic fusion weights corresponding to the data sources related to the matched construction events includes:

[0099] Identify the key data source for the construction event based on the matched construction events;

[0100] Based on the importance of construction events in the knowledge graph, the dynamic fusion weight of key data sources in subsequent fusion is increased proportionally.

[0101] In this embodiment, when a specific construction event is matched, it is determined which data sources provide key information. For example, when matching a construction event of "foundation pit collapse," sensor A providing "settlement" data and sensor B providing "stress" data provide key information. Therefore, the dynamic fusion weights of sensor A and sensor B are recalibrated and increased. The specific increase amount is determined based on the importance of the matched event; the more important / serious the event, the greater the increase. For example, if the importance is "extremely important," the weight of the corresponding data source is increased by 30%; if the importance is "moderate," the weight is increased by 10%. It is understood that after increasing the weights of these key data sources, normalization is required to ensure that the total weight sum is 1. In event matching, the higher the matching degree, the higher the confidence level of the work condition description.

[0102] In some embodiments, adjusting the confidence level of the operating condition description includes:

[0103] Based on the historical frequency of the matched construction events, the completeness of the current event pattern matching, and the data quality of the key data sources, the adjusted confidence level is obtained through Bayesian update or weighted synthesis.

[0104] In practical applications, the confidence level of the working condition description can be set with a base value, and the final confidence level can be determined based on the historical frequency of the matched events, the matching degree, and the data quality of the data source.

[0105] This embodiment also provides a multi-source data fusion system 100 based on a smart construction management cloud platform, see [link to documentation]. Figure 2 As shown, it includes, but is not limited to: feature extraction unit 101, feature merging unit 102, dynamic weighting unit 103, and weighted fusion unit 104.

[0106] The feature extraction unit 101 is configured to acquire multimodal operating condition data uploaded to the cloud platform and extract operating condition feature data for different types of multimodal operating condition data.

[0107] The feature merging unit 102 is configured to form a unified construction state feature vector based on the working condition feature data.

[0108] The dynamic weighting unit 103 is configured to input the construction status feature vector into the weight allocation model in the cloud platform to obtain a dynamic weight vector. The weight allocation model is configured to dynamically determine the weights for data weighting and fusion based on the construction conditions reflected by the current construction status features. Each weight in the dynamic weight vector corresponds to one type or category of data source.

[0109] The weighted fusion unit 104 is configured to use a dynamic weight vector to perform weighted fusion of multi-source data streams to form a fused operating condition description.

[0110] The specific execution process of each unit in the multi-source data fusion system 100 based on the smart construction management cloud platform can also refer to the steps of the multi-source data fusion method based on the smart construction management cloud platform provided in this embodiment, which will not be repeated in this embodiment.

[0111] Furthermore, although exemplary embodiments have been described herein, their scope includes any and all embodiments based on this disclosure that have equivalents, modifications, omissions, combinations (e.g., schemes involving intersections of various embodiments), adaptations, or changes. They are not limited to the examples described in this specification or during the implementation of this application, and such examples are to be interpreted as non-exclusive.

[0112] The above description is intended to be illustrative and not restrictive. For example, the above examples (or one or more of them) can be used in combination with each other. Other embodiments can be used by those skilled in the art when reading the above description.

[0113] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multi-source data fusion method based on a smart construction management cloud platform, characterized in that, include: Acquire multimodal operating condition data uploaded to the cloud platform, and extract operating condition feature data for different types of multimodal operating condition data respectively; Based on the aforementioned working condition feature data, a unified construction state feature vector is formed; The construction status feature vector is input into the weight allocation model in the cloud platform to obtain a dynamic weight vector. The weight allocation model is configured to dynamically determine the weights for data weighting and fusion based on the construction status reflected by the current construction status features. Each weight in the dynamic weight vector corresponds to one or a type of data source. The dynamic weight vector is used to perform weighted fusion of multi-source data streams to form a fused operating condition description.

2. The multi-source data fusion method based on a smart construction management cloud platform as described in claim 1, characterized in that, The weight allocation model includes a feature interaction layer for explicitly learning the joint influence of different state features on weight decisions. The feature interaction layer is a bilinear interactive pooling layer or a pairwise interactive layer, used to calculate the second-order combined features among the features of each dimension of the construction state feature vector.

3. The multi-source data fusion method based on a smart construction management cloud platform as described in claim 2, characterized in that, The training process of the weight allocation model includes: Construct a training sample set, where each sample includes working condition feature data generated from historical multimodal working condition data, and reference fusion weight labels; Construct an initial neural network that includes the feature interaction layer; The initial neural network is trained using a loss function that includes a constraint term, which is used to penalize excessively low values ​​of the prediction weights for safety-critical data sources. When the training meets the convergence condition, the trained weight allocation model is obtained.

4. The multi-source data fusion method based on a smart construction management cloud platform as described in claim 3, characterized in that, The loss function is L = MSE(W_pred, W_label) + λ * Σ ReLU(δ - W_security), where L is the total loss, MSE is the mean squared error loss, W_pred is the weight assigned by the weight allocation model, W_label is the reference fusion weight label, λ is the regularization coefficient, ReLU is the linear rectified function, δ is the weight safety threshold for the safety-critical data source, and W_security is the weight predicted by the weight allocation model for the safety-critical data source.

5. The multi-source data fusion method based on a smart construction management cloud platform as described in claim 1, characterized in that, The process of forming a unified construction state feature vector based on the working condition feature data includes concatenating various working condition feature data vectors and forming a unified construction state feature vector through a normalization layer. The extraction of operating condition feature data for different types of multimodal operating condition data includes at least one of the following: Extract current schedule feature data based on the description in the schedule file from the building information model; Extract visual scene feature data from on-site images; Semantic feature data is extracted from construction log text; Real-time risk characteristic data is extracted based on on-site environmental sensors and equipment sensors.

6. The multi-source data fusion method based on a smart construction management cloud platform as described in any one of claims 1-5, characterized in that, Using the dynamic weight vector to perform weighted fusion of multi-source data streams includes: Based on real-time working data from the construction site, one or more active work areas are identified using a spatial clustering algorithm; Based on the affiliation between the data source and the active work area, the data sources at the construction site are divided into regional groups; Obtain a construction event knowledge graph, which contains the association between predetermined events and data sources; Based on the construction event knowledge graph, semantic association analysis is performed on the data sources, and data sources that may logically participate in the same predetermined event are associated to form semantic association groups. Based on the regional grouping and the semantic association grouping, a data source grouping mapping table is formed to guide the fusion strategy of multi-source data streams.

7. The multi-source data fusion method based on a smart construction management cloud platform as described in claim 6, characterized in that, After forming the data source grouping mapping table based on the region grouping and the semantic association group, the method further includes: Based on the grouping of the data source grouping mapping table and the weights guided by the dynamic weight vector, the original data streams of the data sources within the same group are fused to form the intra-group feature fusion result. The fusion results of the intra-group features are then matched with a pre-constructed construction event knowledge graph to determine event patterns. When a specific construction event is matched, the dynamic fusion weights corresponding to the data sources related to the event are recalibrated and improved based on the matched construction event, and the confidence of the work condition description is adjusted. A working condition description report is generated based on the adjusted results.

8. The multi-source data fusion method based on a smart construction management cloud platform as described in claim 7, characterized in that, The recalibration and enhancement of the dynamic fusion weights corresponding to the data sources related to the matched construction events includes: Identify the key data source for the construction event based on the matched construction events; Based on the importance of the construction event in the knowledge graph, the dynamic fusion weight of the key data source in subsequent fusion is increased proportionally.

9. The multi-source data fusion method based on a smart construction management cloud platform as described in claim 8, characterized in that, The adjustment of the confidence level of the operating condition description includes: Based on the historical frequency of the matched construction events, the completeness of the current event pattern matching, and the contextual consistency of the key data source data, the adjusted confidence level is obtained through Bayesian update or weighted synthesis.

10. A multi-source data fusion system based on a smart construction management cloud platform, characterized in that, include: The feature extraction unit is configured to acquire multimodal operating condition data uploaded to the cloud platform, and extract operating condition feature data for different types of multimodal operating condition data respectively; The feature merging unit is configured to form a unified construction state feature vector based on the working condition feature data; A dynamic weighting unit is configured to input the construction state feature vector into the weight allocation model in the cloud platform to obtain a dynamic weighting vector. The weight allocation model is configured to dynamically determine the weights for data weighting and fusion based on the construction status reflected by the current construction state features. Each weight in the dynamic weighting vector corresponds to one or a type of data source. The weighted fusion unit is configured to use the dynamic weight vector to perform weighted fusion of multi-source data streams to form a fused operating condition description.

Images