A construction structure safety monitoring method and system based on sensing data

Abstract

The application relates to a construction structure safety monitoring method and system based on sensing data. The method adopts a domain learning mechanism, selects engineering structures consistent with construction structures in various construction specifications as standard structures, and then transmits the data distribution rules of the multi-source sensing data of the standard structures to the construction structures, so as to mine the invariant features and basic structures between different structures belonging to the same structure category. Moreover, the method constructs two training branches. The first branch network extracts local features of the multi-source sensing data through CNN, the second branch network extracts global features of the multi-source sensing data through Transformer, and the respective first graph feature vectors and second graph feature vectors are generated based on the graph neural network. Then, the sensing data of the same category are aggregated and fused, so as to realize the fusion of the local sensing features and the global sensing features, improve the training effect of the safety monitoring model, and finally improve the prediction accuracy of the safety state of the construction structure based on the safety monitoring model.

Description

Technical Field

[0001] This application relates to the field of construction structure safety monitoring technology, and in particular to a construction structure safety monitoring method and system based on sensor data. Background Technology

[0002] Currently, intelligent construction (such as large-span prefabricated monolithic steel frame-cast-in-place floor slab structures) has been incorporated into important measures for national economic and social development in the new era. During the structural construction process of intelligent construction, monitoring the safety status of the structure is a crucial measure to ensure the safety of construction personnel and the progress of construction.

[0003] Traditional monitoring methods relied on manual analysis of sensor data. However, with the development of neural networks, deep learning-based methods have gradually been adopted for analyzing the safety status of construction structures. However, deep learning methods still have limitations. Specifically, deep learning-based safety monitoring models require extensive training data, but construction structures are often unfinished, resulting in limited and incomplete data collection. This makes it impossible to collect complete data on the entire completed structure, leading to certain errors in deep learning-based safety monitoring of construction structures. Summary of the Invention

[0004] The following is an overview of the subject matter described in detail herein. This overview is not intended to limit the scope of the claims.

[0005] The main objective of this disclosure is to propose a construction structure safety monitoring method and system based on sensor data, which can improve the accuracy of predicting the safety status of construction structures based on safety monitoring models.

[0006] A first aspect of this application proposes a method for monitoring the safety of construction structures based on sensor data, the method comprising:

[0007] Acquire first multi-source sensor data of a standard structure and second multi-source sensor data of a construction structure, wherein the standard structure and the construction structure are engineering structures of the same type;

[0008] The first multi-source sensing data and the second multi-source sensing data are used as training data. The training data are respectively input into the first branch network and the second branch network of the preset network model to obtain the first graph feature vector extracted by the first branch network and the second graph feature vector extracted by the second branch network. The first branch network includes a convolutional neural network and a graph neural network, and the second branch network includes a Transformer and a graph neural network.

[0009] The sensor data of the same category in the first feature vector are aggregated to obtain the third feature vector, and the sensor data of the same category in the second feature vector are aggregated to obtain the fourth feature vector; then, the sensor data of the same category in the third feature vector and the fourth feature vector are fused to obtain the fused feature vector.

[0010] Based on the fused feature vectors, the safety status of the construction structure is predicted using the network model.

[0011] A loss function is constructed, and based on the loss function, the network model is backpropagated until the trained network model is used as a safety monitoring model; wherein, the loss function includes at least: a first loss that measures the distribution error between the feature vectors of the first multi-source sensor data corresponding to the map and the feature vectors of the second multi-source sensor data corresponding to the map, a second loss that measures the domain classification error between the first multi-source sensor data and the second multi-source sensor data, and a third loss that measures the error between the safety status predicted by the classifier and the actual safety status of the construction structure;

[0012] The multi-source sensor data of the construction structure at the current moment is input into the safety monitoring model to obtain the current safety status of the construction structure predicted by the safety monitoring model.

[0013] The construction structure safety monitoring method based on sensor data provided in this disclosure has at least the following beneficial effects:

[0014] (1) This method adopts a domain learning mechanism, selects an engineering structure that is consistent with the construction structure in various construction specifications as a standard structure (large-span prefabricated integral steel frame-cast-in-place floor deck structure), and then collects multi-source sensor data of the standard structure as training data for the safety monitoring model for the construction structure. That is, the data distribution pattern of the multi-source sensor data of the standard structure is transferred to the multi-source sensor data of the construction structure, thereby mining the invariant features and basic structure between different but the same type of engineering structures, thereby improving the training effect of the safety monitoring model, and finally improving the accuracy of the prediction of the safety status of the construction structure based on the safety monitoring model.

[0015] (2) This method constructs two training branches, namely the first branch network and the second branch network. The first branch network focuses on extracting local features of multi-source sensor data based on CNN, while the second branch network focuses on extracting global features of multi-source sensor data through Transformer. Then, based on graph neural networks, the first graph feature vector and the second graph feature vector are generated respectively. Then, sensor data of the same category are aggregated, and sensor data of corresponding categories are fused to achieve the fusion of local and global sensor features, so as to improve the training effect of the safety monitoring model and ultimately improve the prediction accuracy of the safety status of construction structures based on the safety monitoring model.

[0016] In some embodiments, the first branch network includes a cascaded convolutional neural network, a first projection head, and a graph neural network;

[0017] The calculation process of the feature vector of the first graph output by the first branch network includes:

[0018] The first intermediate feature is extracted from the first multi-source sensing data using the convolutional neural network.

[0019] The first intermediate feature is projected into a first projection feature by the first projection head, and the first projection feature is transposed into a second projection feature. The second projection feature is multiplied by the first projection feature to obtain a third projection feature, and the third projection feature is normalized to obtain a second intermediate feature.

[0020] The graph neural network extracts a first graph feature vector from the second intermediate features; wherein the computation function of the graph neural network is:

[0021]

[0022] Where A is the second intermediate feature, F is the mapping function of the computation function of the graph neural network, and T1 to T n Let t1 be the size of the receptive field from the 1st to the nth, and t2 be the size of the receptive field from the 1st to the nth. n Let be the order of the polynomial. to These are the weight values ​​corresponding to each polynomial. to It is a diagonal matrix representing the eigenvalues ​​of each polynomial.

[0023] In some embodiments, the second branch network includes a cascaded embedding layer, a Transformer, a second projection head, and a graph neural network;

[0024] The calculation process of the feature vector of the second graph output by the second branch network includes:

[0025] The first intermediate feature is extracted from the second multi-source sensing data according to the embedding layer;

[0026] The second intermediate feature is extracted from the first intermediate feature using the Transformer.

[0027] The second intermediate feature is projected into a first projection feature by the second projection head, the first projection feature is transposed into a second projection feature, the second projection feature is multiplied by the first projection feature to obtain a third projection feature, and the third projection feature is normalized to obtain a third intermediate feature.

[0028] The graph neural network extracts a second graph feature vector from the third intermediate feature; wherein the computation function of the graph neural network is:

[0029]

[0030] Where A is the third intermediate feature, F is the mapping function of the computation function of the graph neural network, and T1 to T n Let t1 be the size of the receptive field from the 1st to the nth, and t2 be the size of the receptive field from the 1st to the nth. n Let ω be the order of the polynomial. t1 To ω tn These are the weight values ​​corresponding to each polynomial. to It is a diagonal matrix representing the eigenvalues ​​of each polynomial.

[0031] In some embodiments, the loss function further includes a first contrastive loss and a second contrastive loss;

[0032] Before predicting the safety status of the construction structure based on the network model according to the fused feature vectors, the method further includes:

[0033] A first positive and negative sample pair is constructed based on the fused feature vector and the first graph feature vector; the first positive sample pair in the first positive and negative sample pair is the graph feature vector corresponding to the same category of sensor data, and the first negative sample pair is the graph feature vector corresponding to different categories of sensor data.

[0034] Based on the first positive and negative sample pair, the first contrast loss is calculated using the following formula:

[0035]

[0036] in, For the first contrastive loss, exp is the natural exponential function, and sim(h) a1, h b1 ) represents the cosine similarity of the first positive sample pair, sim(h) a1, hi ) represents the cosine similarity of the first negative sample pair, and τ is the temperature parameter;

[0037] A second positive and negative sample pair is constructed based on the fused feature vector and the second graph feature vector; the second positive sample pair is the graph feature vector corresponding to the same category of sensor data, and the second negative sample pair is the graph feature vector corresponding to different categories of sensor data.

[0038] The second contrastive loss is calculated based on the second positive and negative sample pair using the following formula:

[0039]

[0040] in, For the second contrast loss, sim(h) a2, h b2 ) represents the cosine similarity of the second positive sample pair, sim(h) a2, h j ) represents the cosine similarity of the second negative sample pair.

[0041] In some embodiments, the third loss is the cross-entropy loss;

[0042] Furthermore, the second loss is the binary cross-entropy loss;

[0043] And, and the first loss is the maximum mean difference loss.

[0044] In some embodiments, the network model further includes: a prediction network;

[0045] The prediction network is a fully convolutional neural network used to predict the safety status of the construction structure.

[0046] In some embodiments, the sample size of sensor data of any category in the training data is the same;

[0047] The calculation formula for fusing sensor data belonging to the same category in the feature vectors of the third and fourth images includes:

[0048]

[0049] Among them, v i Let v be the feature vector of the third graph corresponding to the i-th category of sensor data in the first multi-source sensor data. j v is the feature vector of the third map corresponding to the j-th category of the second multi-source sensing data, where the i-th category and the j-th category are the same category; i,1 to v i,Dv represents the feature vector of the third image corresponding to the first sensor data point to the feature vector of the third image corresponding to the Dth sensor data point. j,1 to v j,D M represents the feature vectors of the fourth image corresponding to the first sensing data point to the feature vectors of the fourth image corresponding to the Dth sensing data point; ij For v i and v j The interaction matrix, W ij It is M ij Preset parameter matrix, P ij For v i and v j The fusion result.

[0050] A second aspect of this application provides a construction structure safety monitoring system based on sensor data, the system comprising:

[0051] The model training module is used to acquire first multi-source sensor data of the standard structure and second multi-source sensor data of the construction structure, wherein the standard structure and the construction structure are engineering structures of the same type.

[0052] The first multi-source sensing data and the second multi-source sensing data are used as training data. The training data are respectively input into the first branch network and the second branch network of the preset network model to obtain the first graph feature vector extracted by the first branch network and the second graph feature vector extracted by the second branch network. The first branch network includes a convolutional neural network and a graph neural network, and the second branch network includes a Transformer and a graph neural network.

[0053] The sensor data of the same category in the first feature vector are aggregated to obtain the third feature vector, and the sensor data of the same category in the second feature vector are aggregated to obtain the fourth feature vector; then, the sensor data of the same category in the third feature vector and the fourth feature vector are fused to obtain the fused feature vector.

[0054] Based on the fused feature vectors, the safety status of the construction structure is predicted using the network model.

[0055] A loss function is constructed, and based on the loss function, the network model is backpropagated until the trained network model is used as a safety monitoring model; wherein, the loss function includes at least: a first loss that measures the distribution error between the feature vectors of the first multi-source sensor data corresponding to the map and the feature vectors of the second multi-source sensor data corresponding to the map, a second loss that measures the domain classification error between the first multi-source sensor data and the second multi-source sensor data, and a third loss that measures the error between the safety status predicted by the classifier and the actual safety status of the construction structure;

[0056] The model uses a module to input multi-source sensor data of the construction structure at the current moment into the safety monitoring model, so as to obtain the current safety status of the construction structure predicted by the safety monitoring model.

[0057] A third aspect of this application provides an electronic device, including at least one controller and a memory for communicatively connecting to the at least one controller; the memory stores instructions executable by the at least one controller, which, when executed by the controller, cause the controller to perform the construction structure safety monitoring method based on sensor data as described in the first aspect.

[0058] A fourth aspect of this application provides a computer-readable storage medium storing computer-executable instructions for causing a computer to perform the construction structure safety monitoring method based on sensor data as described above.

[0059] It is understood that the beneficial effects of the second to fourth aspects compared with the related technologies are the same as the beneficial effects of the first aspect compared with the related technologies. Please refer to the relevant description in the first aspect above, which will not be repeated here. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 This is a flowchart of an embodiment of a construction structure safety monitoring method based on sensor data provided in this application;

[0062] Figure 2 This is a schematic diagram of the structure of the safety monitoring model provided in this application;

[0063] Figure 3 This is a structural diagram of an embodiment of a construction structure safety monitoring system based on sensor data provided in this application;

[0064] Figure 4 This is a structural diagram of an embodiment of the electronic device provided in this application. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0066] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0067] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0068] Currently, intelligent construction (such as large-span prefabricated monolithic steel frame-cast-in-place floor slab structures) has been incorporated into important measures for national economic and social development in the new era. During the structural construction process of intelligent construction, monitoring the safety status of the structure is a crucial measure to ensure the safety of construction personnel and the progress of construction.

[0069] Traditional monitoring methods relied on manual analysis of sensor data. However, with the development of neural networks, deep learning-based methods have gradually been adopted for analyzing the safety status of construction structures. However, deep learning methods still have limitations. Specifically, deep learning-based safety monitoring models require extensive training data, but construction structures are often unfinished, resulting in limited and incomplete data collection. This makes it impossible to collect complete data on the entire completed structure, leading to certain errors in deep learning-based safety monitoring of construction structures.

[0070] To address the aforementioned technical shortcomings, such as Figure 1 One embodiment of this application provides a method for monitoring the safety of construction structures based on sensor data, the method comprising the following steps S110 to S160:

[0071] Step S110: Obtain the first multi-source sensor data of the standard structure and the second multi-source sensor data of the construction structure.

[0072] In step S110, the standard structure and the construction structure are the same type of engineering structure, such as a large-span prefabricated monolithic steel frame-cast-in-place floor slab structure.

[0073] The construction structure can be an unfinished engineering structure, while the standard structure is a completed engineering structure. Here, an engineering structure that is consistent with the construction structure in various construction specifications can be selected as the standard structure. The standard structure is used to provide data as training data for the safety monitoring model. That is, by using the principle of transfer learning, the data distribution pattern of the multi-source sensor data of the standard structure is transferred to the construction structure, thereby mining the invariant features and basic structure between different engineering structures that belong to the same structural type.

[0074] Multi-source sensor data refers to data collected from different sensors, such as vibration sensors, temperature sensors, pressure sensors, and torque sensors. Therefore, the first and second multi-source sensor data mentioned above constitute a data matrix corresponding to the sensor data. It is important to note that the same number and type of sensors are used for data collection for both standard and construction structures, and the sensor installation locations are also identical to minimize errors. Furthermore, the sensor data here has undergone preprocessing, including but not limited to noise reduction and dimensionality unification.

[0075] Step S120: Use the first multi-source sensing data and the second multi-source sensing data as training data, and input the training data into the first branch network and the second branch network of the preset network model respectively to obtain the first graph feature vector extracted by the first branch network and the second graph feature vector extracted by the second branch network respectively.

[0076] In step S120, the first branch network includes a convolutional neural network and a graph neural network, and the second branch network includes a Transformer and a graph neural network. The convolutional neural network and Transformer are used to extract features, while the graph neural network generates an undirected graph to facilitate the extraction of interactions within the graph.

[0077] CNN and Transformer mainly extract features from the first or second multi-source sensor data and generate the corresponding feature matrix.

[0078] Due to the translation invariance of CNNs, they are robust to changes in local feature locations, making them suitable for extracting local features from sensor data. Meanwhile, Transformers, with their self-attention mechanism, can capture long-range spatiotemporal dependencies in sensor data, making them suitable for extracting global features from sensor data.

[0079] Graph neural networks are a type of convolutional neural network architecture specifically designed for processing graph-structured data. Graph-structured data can use edges to describe the relationships between nodes. By constructing graph feature vectors corresponding to the first multi-source sensing data and the second multi-source sensing data, it is easy to capture the interaction relationships between the feature information of the first and second multi-source sensing data.

[0080] Step S130: Aggregate the sensor data of the same category in the feature vector of the first image to obtain the feature vector of the third image; aggregate the sensor data of the same category in the feature vector of the second image to obtain the feature vector of the fourth image; and fuse the sensor data of the same category in the feature vector of the third image and the feature vector of the fourth image to obtain the fused feature vector.

[0081] In step S130, since the first and second multi-source sensing data contain sensing data collected by multiple sensors of different categories, in order to reduce the difficulty of data processing and improve computational efficiency, the sensing data of different types are clustered to facilitate subsequent data processing. Then, the graph feature vectors extracted by the two networks for the same type of sensing data are fused to fully combine the advantages of CNN and Transformer, fusing local and global sensing features of the sensing data to improve the performance of subsequent model training.

[0082] Step S140: Based on the fused feature vectors, predict the safety status of the construction structure using the network model.

[0083] In step S140, a network model can be used to predict the security status. For example, a fully connected layer or a fully convolutional neural network can be used for prediction.

[0084] It is important to note that during the construction of multi-source sensor data, labels can be pre-assigned, with each label representing a different security status. The specific structure of the data will result in different security statuses, which are not limited here.

[0085] Step S150: Construct a loss function. Based on the loss function, perform backpropagation on the network model until the trained network model is used as a security monitoring model.

[0086] In step S150, at least three types of losses are used to train the model, wherein:

[0087] The first loss measures the distribution error between the first and second multi-source sensor data. Since the first multi-source sensor data of the standard structure can be used as source domain data and the second multi-source sensor data as target domain data, the maximum mean difference loss can be used to align the feature information between the source and target domain data. The second loss measures the domain classification error between the first and second multi-source sensor data. To determine whether the extracted graph feature vector comes from the first or second multi-source sensor data and to train the network model's feature extraction stage to "fool" the network model's domain classification stage, binary cross-entropy can be used. The third loss measures the error between the safety state predicted by the network model and the actual safety state of the construction structure. This is to improve the network model's classification prediction results; cross-entropy can be used for this term. These three losses can significantly improve the model training effect.

[0088] Steps S110 to S150 constitute the training process of the model.

[0089] Step S160: Input the multi-source sensor data of the construction structure at the current moment into the safety monitoring model to obtain the current safety status of the construction structure predicted by the safety monitoring model.

[0090] The construction structure safety monitoring method based on sensor data provided in this application has at least the following beneficial effects:

[0091] (1) This method adopts a domain learning mechanism, selects an engineering structure that is consistent with the construction structure in various construction specifications as a standard structure (large-span prefabricated integral steel frame-cast-in-place floor deck structure), and then collects multi-source sensor data of the standard structure as training data for the safety monitoring model for the construction structure. That is, the data distribution pattern of the multi-source sensor data of the standard structure is transferred to the multi-source sensor data of the construction structure, thereby mining the invariant features and basic structure between different but the same type of engineering structures, thereby improving the training effect of the safety monitoring model, and finally improving the accuracy of the prediction of the safety status of the construction structure based on the safety monitoring model.

[0092] (2) This method constructs two training branches, namely the first branch network and the second branch network. The first branch network focuses on extracting local features of multi-source sensor data based on CNN, while the second branch network focuses on extracting global features of multi-source sensor data through Transformer. Then, based on graph neural networks, the first graph feature vector and the second graph feature vector are generated respectively. Then, sensor data of the same category are aggregated, and sensor data of corresponding categories are fused to achieve the fusion of local and global sensor features, so as to improve the training effect of the safety monitoring model and ultimately improve the prediction accuracy of the safety status of construction structures based on the safety monitoring model.

[0093] Furthermore, the first branch network includes a convolutional neural network, a first projection head, and a graph neural network;

[0094] The calculation process of the feature vector of the first graph output by the first branch network includes the following steps S210 to S230:

[0095] Step S210: Extract the first intermediate feature from the first multi-source sensing data using a convolutional neural network.

[0096] Step S220: Project the first intermediate feature into a first projected feature using the first projection head, transpose the first projected feature into a second projected feature, multiply the second projected feature with the first projected feature to obtain a third projected feature, and normalize the third projected feature to obtain the second intermediate feature. The projection head can be a multilayer perceptron. By converting the features extracted by the CNN into projected features through the projection head, subsequent computational processing can be facilitated.

[0097] Step S230: Extract the first graph feature vector from the second intermediate features using the graph neural network; wherein the computation function of the graph neural network is:

[0098]

[0099] Where A is the second intermediate feature, F is the mapping function of the computation function of the graph neural network, and T1 to T n Let t1 be the size of the receptive field from the 1st to the nth, and t2 be the size of the receptive field from the 1st to the nth. n Let be the order of the polynomial. to These are the weight values ​​corresponding to each polynomial. to It is a diagonal matrix representing the eigenvalues ​​of each polynomial.

[0100] Furthermore, the second branch network includes an embedding layer, a Transformer, a second projection head, and a graph neural network;

[0101] The calculation process of the feature vector of the second graph output by the second branch network includes the following steps S310 to S340:

[0102] Step S310: Extract the first intermediate feature from the second multi-source sensing data based on the embedding layer.

[0103] Step S320: Extract the second intermediate feature from the first intermediate feature using the Transformer.

[0104] Step S330: Project the second intermediate feature into a first projection feature using the second projection head, transpose the first projection feature into a second projection feature, multiply the second projection feature with the first projection feature to obtain a third projection feature, and normalize the third projection feature to obtain a third intermediate feature.

[0105] Step S340: Extract the second graph feature vector from the third intermediate feature using the graph neural network; where the computation function of the graph neural network is:

[0106]

[0107] Where A is the third intermediate feature, F is the mapping function of the computation function of the graph neural network, and T1 to T n Let t1 be the size of the receptive field from the 1st to the nth, and t2 be the size of the receptive field from the 1st to the nth. n Let be the order of the polynomial. to These are the weight values ​​corresponding to each polynomial. to It is a diagonal matrix representing the eigenvalues ​​of each polynomial.

[0108] In the process described above, compared to traditional graph neural networks that only involve fixed receptive fields, the graph neural network provided by this method adds n receptive fields, thereby improving the quality of the acquired graph feature vectors.

[0109] Furthermore, the loss function also includes a first contrastive loss and a second contrastive loss;

[0110] Before predicting the safety status of the construction structure based on the fused feature vectors in step S150, the method further includes steps S410 to S440:

[0111] Step S410: Construct a first positive and negative sample pair based on the fused feature vector and the first graph feature vector; the first positive sample pair in the first positive and negative sample pair is the graph feature vector corresponding to the same category of sensor data, and the first negative sample pair is the graph feature vector corresponding to different categories of sensor data.

[0112] Step S420: Calculate the first contrastive loss based on the first positive and negative sample pair using the following formula:

[0113]

[0114] in, For the first contrastive loss, exp is the natural exponential function, and sim(h) a1, h b1 ) represents the cosine similarity of the first positive sample pair, sim(h) a1, h i ) represents the cosine similarity of the first negative sample pair, and τ is the temperature parameter;

[0115] Step S430: Construct a second positive and negative sample pair based on the fused feature vector and the second graph feature vector; the second positive sample pair is the graph feature vector corresponding to the same category of sensor data, and the second negative sample pair is the graph feature vector corresponding to different categories of sensor data.

[0116] Step S440: Based on the second positive and negative sample pair, calculate the second contrastive loss using the following formula:

[0117]

[0118] in, For the second contrast loss, sim(h) a2, h b2 ) represents the cosine similarity of the second positive sample pair, sim(h) a2, h j ) represents the cosine similarity of the second negative sample pair.

[0119] In this method, by constructing positive and negative samples, the feature information of the same type of sensor data is made closer in the feature space, and the feature information of different types of sensor data is more distinguishable in the feature space. This enables the safety monitoring model to extract more class-discriminative fused feature vectors, and ultimately improves the accuracy of predicting the safety status of construction structures based on the safety monitoring model.

[0120] Furthermore, the sample size of sensor data of any category in the training data is the same;

[0121] The calculation formula for fusing the sensor data of different categories in the feature vector of the third image and the corresponding category in the feature vector of the fourth image in step S140 includes the following:

[0122]

[0123] Among them, v i Let v be the feature vector of the third graph corresponding to the i-th category of sensor data in the first multi-source sensor data. jv is the feature vector of the third map corresponding to the j-th category of the second multi-source sensing data, where the i-th category and the j-th category are the same category; i,1 to v i,D v represents the feature vector of the third image corresponding to the first sensor data point to the feature vector of the third image corresponding to the Dth sensor data point. j,1 to v j,D M represents the feature vectors of the fourth image corresponding to the first sensing data point to the feature vectors of the fourth image corresponding to the Dth sensing data point; ij For v i and v j The interaction matrix, W ij It is M ij Preset parameter matrix, P ij For v i and v j The fusion result.

[0124] This step is mainly to fuse the feature vectors of different engineering structures that collect sensing data from the same type of sensors, so as to fully integrate local sensing features and global sensing features.

[0125] In conventional methods, the fusion of sensor data of the same type corresponding to different structures is usually achieved using fully connected layers or vector multiplication. However, if only fully connected layers or eigenvector multiplication are used, the resulting eigenvector value will lose too much information contained in the sensor data, thus failing to effectively capture the deep-level interaction features between different sensor features. This method forms a matrix M. ij Each element in the matrix represents the interaction result between the same type of sensor data in different domains of the feature vector, which has a deeper level of interaction. It contains interaction information between different features in different dimensions, and introduces a parameter matrix to learn the weights of feature interaction between the source domain and the target domain, adaptively selects effective interaction features, and improves the fusion effect.

[0126] like Figure 2 One embodiment of this application provides a construction structure safety monitoring method based on sensor data, which includes the following steps:

[0127] 1) First, construct a security monitoring model;

[0128] The security monitoring model includes a generation phase and a discrimination phase;

[0129] The generator includes a first branch network and a second branch network. The first branch network includes a CNN, a projection head, and a graph neural network. The second branch network includes an embedding layer, a Transformer, a projection head, and a graph neural network.

[0130] The discrimination stage includes a predictor, which further includes a contrastive learning mechanism, a domain classification discrimination mechanism, and a distribution difference discrimination mechanism.

[0131] 2) Secondly, the safety monitoring model is trained;

[0132] The training data includes source domain data and target domain data (both with labels), namely, the first multi-source sensor data of the standard structure and the second multi-source sensor data of the construction structure. The multi-source sensors can include, but are not limited to, temperature, pressure, torque, and vibration sensors. The multi-source sensor data constructs a data matrix. The standard structure is a completed, large-span prefabricated monolithic steel frame-cast-in-place floor slab structure, while the construction structure is the same structure under construction.

[0133] The purpose of this embodiment is to predict the safety status of the construction structure through a safety monitoring model.

[0134] The first multi-source sensor data (matrix form) of the standard structure and the second multi-source sensor data (matrix form) of the construction structure are both input into the first branch network to obtain the first graph feature vector output by the first branch network.

[0135] The first multi-source sensor data of the standard structure and the second multi-source sensor data of the construction structure are both input into the second branch network to obtain the second graph feature vector output by the second branch network.

[0136] In the first and second branch networks, compared to traditional graph neural networks that only involve fixed receptive fields, this method provides graph convolutional networks with multiple receptive fields, which can improve the quality of the acquired graph feature vectors.

[0137] Then, based on the relevant clustering algorithm, the sensor data of the same category in the feature vector of the first image are aggregated to obtain the feature vector of the third image, and the sensor data of the same category in the feature vector of the second image are aggregated to obtain the feature vector of the fourth image.

[0138] Secondly, the sensor data belonging to the same category in the feature vectors of the third and fourth images are fused to obtain a fused feature vector; the fusion method uses the following formula:

[0139]

[0140] It can enhance the level of interaction.

[0141] Then, construct the loss function;

[0142] The first loss (measuring the maximum mean difference in distribution error between the feature vectors of the first and second multi-source sensor data maps) is calculated. The second loss (the binary cross-entropy of the domain classification error between the first and second multi-source sensor data maps, based on the discrimination mechanism) is calculated. The third loss (the third loss cross-entropy of the error between the predicted safety state and the actual safety state of the construction structure, based on the prediction mechanism) is calculated. The first contrast loss and the second contrast loss are also calculated. Finally, the above losses are weighted to obtain the total loss function.

[0143] Backpropagation;

[0144] 3) Finally, the construction structure is predicted based on the safety monitoring model.

[0145] This method has at least the following beneficial effects:

[0146] (1) This method adopts a domain learning mechanism, selects an engineering structure that is consistent with the construction structure in various construction specifications as a standard structure, and then collects multi-source sensor data of the standard structure as training data for the safety monitoring model for the construction structure. That is, the data distribution pattern of the multi-source sensor data of the standard structure is transferred to the multi-source sensor data of the construction structure, so as to explore the invariant features and basic structure between different but the same type of engineering structures, thereby improving the training effect of the safety monitoring model and ultimately improving the accuracy of the prediction of the safety status of the construction structure based on the safety monitoring model.

[0147] (2) This method constructs two training branches, namely the first branch network and the second branch network. The first branch network focuses on extracting local features of multi-source sensor data based on CNN, while the second branch network focuses on extracting global features of multi-source sensor data through Transformer. Then, based on graph neural networks, the first graph feature vector and the second graph feature vector are generated respectively. Then, sensor data of the same category are aggregated, and sensor data of corresponding categories are fused to achieve the fusion of local and global sensor features, so as to improve the training effect of the safety monitoring model and ultimately improve the prediction accuracy of the safety status of construction structures based on the safety monitoring model.

[0148] (3) This method improves the graph convolutional neural network in the first branch network and the second branch network by increasing the number of receptive fields, thereby improving the quality of the obtained graph feature vectors.

[0149] (4) This method adds a contrastive learning mechanism. By constructing positive and negative samples, the feature information of the same type of sensor data is closer in the feature space, and the feature information of different types of sensor data is more distinguishable in the feature space. This enables the safety monitoring model to extract more type-distinguishing fused feature vectors, and ultimately improves the accuracy of predicting the safety status of construction structures based on the safety monitoring model.

[0150] (5) In the process of fusing sensor data of the same category in the feature vectors of the third and fourth figures, compared with fully connected layers or feature vector multiplication, this method forms a matrix, and each element in the matrix represents the interaction result between sensor data of the same type in different domains in the feature vectors. It has a deeper degree of interaction, which includes interaction information between different features in different dimensions. Furthermore, it introduces a parameter matrix to learn the weights of feature interaction between the source and target domains, adaptively selects effective interaction features, and improves the fusion effect.

[0151] like Figure 3 One embodiment of this application provides a construction structure safety monitoring system based on sensor data, the system comprising:

[0152] The model training module 1000 is used to acquire the first multi-source sensor data of the standard structure and the second multi-source sensor data of the construction structure. The standard structure and the construction structure are the same type of engineering structure.

[0153] The first multi-source sensing data and the second multi-source sensing data are used as training data. The training data are input into the first branch network and the second branch network of the preset network model, respectively, to obtain the first graph feature vector extracted by the first branch network and the second graph feature vector extracted by the second branch network. The first branch network includes a convolutional neural network and a graph neural network, and the second branch network includes a Transformer and a graph neural network.

[0154] The sensor data of the same category in the feature vector of the first image are aggregated to obtain the feature vector of the third image. The sensor data of the same category in the feature vector of the second image are aggregated to obtain the feature vector of the fourth image. Then, the sensor data of the same category in the feature vector of the third image and the feature vector of the fourth image are fused to obtain the fused feature vector.

[0155] Based on the fused feature vectors, the safety status of the construction structure is predicted using a network model.

[0156] A loss function is constructed, and the network model is backpropagated based on the loss function until the trained network model is used as a safety monitoring model. The loss function includes at least: a first loss that measures the distribution error between the feature vectors of the first multi-source sensor data map and the feature vectors of the second multi-source sensor data map; a second loss that measures the domain classification error between the first multi-source sensor data and the second multi-source sensor data; and a third loss that measures the error between the safety status predicted by the classifier and the actual safety status of the construction structure.

[0157] The model uses module 2000 to input multi-source sensor data of the construction structure at the current moment into the safety monitoring model, and obtain the current safety status of the construction structure predicted by the safety monitoring model.

[0158] It should be noted that since the construction structure safety monitoring system based on sensor data in this embodiment is based on the same inventive concept as the construction structure safety monitoring method based on sensor data described above, the corresponding content in the construction structure safety monitoring method embodiment based on sensor data is also applicable to the construction structure safety monitoring system embodiment based on sensor data, and will not be described in detail here.

[0159] This system has at least the following beneficial effects:

[0160] (1) This system adopts a domain learning mechanism, selects an engineering structure that is consistent with the construction structure in various construction specifications as a standard structure (large-span prefabricated integral steel frame-cast-in-place floor deck structure), and then collects multi-source sensor data of the standard structure as training data for the safety monitoring model for the construction structure. That is, the data distribution pattern of the multi-source sensor data of the standard structure is transferred to the multi-source sensor data of the construction structure, so as to explore the invariant features and basic structure between different but the same type of engineering structures, thereby improving the training effect of the safety monitoring model and ultimately improving the accuracy of the prediction of the safety status of the construction structure based on the safety monitoring model.

[0161] (2) This system constructs two training branches, namely the first branch network and the second branch network. The first branch network focuses on extracting local features of multi-source sensor data based on CNN, while the second branch network focuses on extracting global features of multi-source sensor data through Transformer. Then, based on graph neural networks, the first graph feature vector and the second graph feature vector are generated respectively. Then, sensor data of the same category are aggregated, and sensor data of corresponding categories are fused to achieve the fusion of local sensor features and global sensor features, so as to improve the training effect of the safety monitoring model and ultimately improve the prediction accuracy of the safety status of construction structures based on the safety monitoring model.

[0162] In some embodiments, the model training module 1000 is further configured to:

[0163] The first branch network includes a cascaded convolutional neural network, a first projection head, and a graph neural network;

[0164] The calculation process of the feature vector of the first graph output by the first branch network includes:

[0165] The first intermediate feature is extracted from the first multi-source sensing data using a convolutional neural network.

[0166] The first intermediate feature is projected into a first projection feature by the first projection head, and the first projection feature is transposed into a second projection feature. The second projection feature is multiplied by the first projection feature to obtain a third projection feature. The third projection feature is then normalized to obtain a second intermediate feature.

[0167] The graph neural network extracts the first graph feature vector from the second intermediate feature; the computation function of the graph neural network is:

[0168]

[0169] Where A is the second intermediate feature, F is the mapping function of the computation function of the graph neural network, and T1 to T n Let t1 be the size of the receptive field from the 1st to the nth, and t2 be the size of the receptive field from the 1st to the nth. n Let be the order of the polynomial. to These are the weight values ​​corresponding to each polynomial. to It is a diagonal matrix representing the eigenvalues ​​of each polynomial.

[0170] In some embodiments, the model training module 1000 is further configured to:

[0171] The second branch network consists of cascaded embedding layers, a Transformer, a second projection head, and a graph neural network;

[0172] The calculation process of the feature vector of the second graph output by the second branch network includes:

[0173] The first intermediate feature is extracted from the second multi-source sensing data based on the embedding layer;

[0174] The second intermediate feature is extracted from the first intermediate feature using the Transformer.

[0175] The second intermediate feature is projected into the first projection feature by the second projection head, the first projection feature is transposed into the second projection feature, the second projection feature is multiplied by the first projection feature to obtain the third projection feature, and the third projection feature is normalized to obtain the third intermediate feature.

[0176] The graph neural network extracts the second graph feature vector from the third intermediate feature; the computation function of the graph neural network is:

[0177]

[0178] Where A is the third intermediate feature, F is the mapping function of the computation function of the graph neural network, and T1 to T n Let t1 be the size of the receptive field from the 1st to the nth, and t2 be the size of the receptive field from the 1st to the nth. n Let be the order of the polynomial. to These are the weight values ​​corresponding to each polynomial. to It is a diagonal matrix representing the eigenvalues ​​of each polynomial.

[0179] In some embodiments, the model training module 1000 is further configured to:

[0180] The loss function also includes a first contrastive loss and a second contrastive loss;

[0181] Before predicting the safety status of the construction structure based on the fused feature vectors and the network model, the method also includes:

[0182] The first positive and negative sample pairs are constructed based on the fused feature vector and the first graph feature vector; the first positive sample pair in the first positive and negative sample pairs is the graph feature vector corresponding to the same category of sensor data, and the first negative sample pair is the graph feature vector corresponding to different categories of sensor data.

[0183] The first contrastive loss is calculated based on the first positive and negative sample pair using the following formula:

[0184]

[0185] in, For the first contrastive loss, exp is the natural exponential function, and sim(h) a1 ,h b1 ) represents the cosine similarity of the first positive sample pair, sim(h) a1 ,h i ) represents the cosine similarity of the first negative sample pair, and τ is the temperature parameter;

[0186] The second positive and negative sample pairs are constructed based on the fused feature vector and the second graph feature vector; the second positive sample pair is the graph feature vector corresponding to the same category of sensor data, and the second negative sample pair is the graph feature vector corresponding to different categories of sensor data.

[0187] The second contrastive loss is calculated based on the second positive-negative sample pair using the following formula:

[0188]

[0189] in, For the second contrast loss, sim(h) a2 ,h b2 ) represents the cosine similarity of the second positive sample pair, sim(h) a2 ,h j ) represents the cosine similarity of the second negative sample pair.

[0190] In some embodiments, the model training module 1000 is further configured to:

[0191] Network models also include: predictive networks;

[0192] The prediction network is a fully convolutional neural network used to predict the safety status of the construction structure.

[0193] In some embodiments, the model training module 1000 is further configured to:

[0194] The sample size of sensor data of any category in the training data is the same;

[0195] The calculation formula for fusing sensor data belonging to the same category from the feature vectors of the third and fourth images includes:

[0196]

[0197] Among them, v i Let v be the feature vector of the third graph corresponding to the i-th category of sensor data in the first multi-source sensor data. j v is the feature vector of the third map corresponding to the j-th category of the second multi-source sensing data, where the i-th category and the j-th category are the same category; i,1 to v i,D v represents the feature vector of the third image corresponding to the first sensor data point to the feature vector of the third image corresponding to the Dth sensor data point. j,1 to v j,D M represents the feature vectors of the fourth image corresponding to the first sensing data point to the feature vectors of the fourth image corresponding to the Dth sensing data point; ij For v i and v j The interaction matrix, W ij It is M ij Preset parameter matrix, P ij For v i and v j The fusion result.

[0198] like Figure 4 As shown in the illustration, this application also provides an electronic device, which includes:

[0199] At least one memory;

[0200] At least one processor;

[0201] At least one program;

[0202] The program is stored in memory, and the processor executes at least one program to implement the above-described construction structure safety monitoring method based on sensor data.

[0203] This electronic device can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.

[0204] The electronic devices according to embodiments of this application will now be described in detail.

[0205] The processor 1600 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0206] The memory 1700 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1700 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1700 and is called and executed by the processor 1600 to execute the construction structure safety monitoring method based on sensor data according to the embodiments of this application.

[0207] The input / output interface 1800 is used to implement information input and output.

[0208] The communication interface 1900 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0209] Bus 2000 transmits information between various components of the device (e.g., processor 1600, memory 1700, input / output interface 1800, and communication interface 1900);

[0210] The processor 1600, memory 1700, input / output interface 1800 and communication interface 1900 are connected to each other within the device via bus 2000.

[0211] This application embodiment also provides a storage medium, which is a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the above-described construction structure safety monitoring method based on sensor data.

[0212] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device.

[0213] In some implementations, the memory may optionally include memory remotely located relative to the processor, which can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0214] The embodiments described in this application are intended to more clearly illustrate the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will know that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0215] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0216] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0217] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0218] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0219] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0220] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0221] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0222] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0223] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0224] The above is a detailed description of the preferred embodiments of this application. However, the embodiments of this application are not limited to the above-described implementation methods. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the embodiments of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of the embodiments of this application.

Claims

1. A method for monitoring the safety of construction structures based on sensor data, characterized in that, The method includes: Acquire first multi-source sensor data of a standard structure and second multi-source sensor data of a construction structure, wherein the standard structure and the construction structure are engineering structures of the same type; Using the first and second multi-source sensing data as training data, the training data is input into the first and second branch networks of a preset network model, respectively, to obtain the first graph feature vector extracted by the first branch network and the second graph feature vector extracted by the second branch network. The first branch network includes a cascaded convolutional neural network, a first projection head, and a graph neural network; the second branch network includes a cascaded embedding layer, a Transformer, a second projection head, and a graph neural network. The calculation process of the first graph feature vector output by the first branch network includes: The first intermediate feature is extracted from the first multi-source sensing data using the convolutional neural network. The first intermediate feature is projected into a first projection feature by the first projection head, and the first projection feature is transposed into a second projection feature. The second projection feature is multiplied by the first projection feature to obtain a third projection feature, and the third projection feature is normalized to obtain a second intermediate feature. The graph neural network extracts a first graph feature vector from the second intermediate features; wherein the computation function of the graph neural network is: ； in, As the second intermediate feature, The mapping function is the computation function of the graph neural network. to From the 1st to the 2nd The size of the receptive field to Let be the order of the polynomial. to These are the weight values ​​corresponding to each polynomial. to This is the diagonal matrix representing the eigenvalues ​​of each polynomial; The calculation process of the feature vector of the second graph output by the second branch network includes: The first intermediate feature is extracted from the second multi-source sensing data according to the embedding layer; The second intermediate feature is extracted from the first intermediate feature using the Transformer. The second intermediate feature is projected into a first projection feature by the second projection head, the first projection feature is transposed into a second projection feature, the second projection feature is multiplied by the first projection feature to obtain a third projection feature, and the third projection feature is normalized to obtain a third intermediate feature. The second graph feature vector is extracted from the third intermediate feature using the graph neural network. The sensor data of the same category in the first feature vector are aggregated to obtain the third feature vector. The sensor data of the same category in the second feature vector are aggregated to obtain the fourth feature vector. The sensor data of the same category in the third feature vector and the fourth feature vector are fused to obtain the fused feature vector. Based on the fused feature vectors, the safety status of the construction structure is predicted using the network model. A loss function is constructed, and based on the loss function, the network model is backpropagated until the trained network model is used as a safety monitoring model; wherein, the loss function includes at least: a first loss that measures the distribution error between the feature vectors of the first multi-source sensor data corresponding to the map and the feature vectors of the second multi-source sensor data corresponding to the map, a second loss that measures the domain classification error between the first multi-source sensor data and the second multi-source sensor data, and a third loss that measures the error between the safety status predicted by the classifier and the actual safety status of the construction structure; The multi-source sensor data of the construction structure at the current moment is input into the safety monitoring model to obtain the current safety status of the construction structure predicted by the safety monitoring model; the sample size of sensor data of any category in the training data is the same; The calculation formula for fusing sensor data belonging to the same category in the feature vectors of the third and fourth images includes: ； ； in, The first multi-source sensing data The third image feature vector corresponding to each category of sensor data. The second multi-source sensing data The third image feature vector corresponding to the sensor data of each category, and the... The category and the All categories belong to the same category; to The feature vectors of the third image corresponding to the first sensor data point are the feature vectors of the third image corresponding to the Dth sensor data point. to The feature vectors of the fourth image corresponding to the first sensing data point to the feature vectors of the fourth image corresponding to the Dth sensing data point; for and The interaction matrix, yes Preset parameter matrix, for and The fusion result; The standard structure is a completed engineering structure, while the construction structure is an unfinished engineering structure.

2. The construction structure safety monitoring method based on sensor data according to claim 1, characterized in that, The loss function further includes a first contrastive loss and a second contrastive loss; Before predicting the safety status of the construction structure based on the network model according to the fused feature vectors, the method further includes: A first positive and negative sample pair is constructed based on the fused feature vector and the first graph feature vector; the first positive sample pair in the first positive and negative sample pair is the graph feature vector corresponding to the same category of sensor data, and the first negative sample pair is the graph feature vector corresponding to different categories of sensor data. Based on the first positive and negative sample pair, the first contrast loss is calculated using the following formula: ； in, As the first comparison loss, It is a natural exponential function. The cosine similarity of the first positive sample pair. The cosine similarity of the first negative sample pair. For temperature parameters; A second positive and negative sample pair is constructed based on the fused feature vector and the second graph feature vector; the second positive sample pair is the graph feature vector corresponding to the same category of sensor data, and the second negative sample pair is the graph feature vector corresponding to different categories of sensor data. The second contrastive loss is calculated based on the second positive and negative sample pair using the following formula: ； in, For the second comparison loss, The cosine similarity of the second positive sample pair. The cosine similarity of the second negative sample pair.

3. The construction structure safety monitoring method based on sensor data according to claim 1, characterized in that, The network model also includes: a prediction network; The prediction network is a fully convolutional neural network used to predict the safety status of the construction structure.

4. A construction structure safety monitoring system based on sensor data, characterized in that, The system includes: The model training module is used to acquire first multi-source sensor data of the standard structure and second multi-source sensor data of the construction structure, wherein the standard structure and the construction structure are engineering structures of the same type. Using the first and second multi-source sensing data as training data, the training data is input into the first and second branch networks of a preset network model, respectively, to obtain the first graph feature vector extracted by the first branch network and the second graph feature vector extracted by the second branch network. The first branch network includes a cascaded convolutional neural network, a first projection head, and a graph neural network; the second branch network includes a cascaded embedding layer, a Transformer, a second projection head, and a graph neural network. The calculation process of the first graph feature vector output by the first branch network includes: The first intermediate feature is extracted from the first multi-source sensing data using the convolutional neural network. The first intermediate feature is projected into a first projection feature by the first projection head, and the first projection feature is transposed into a second projection feature. The second projection feature is multiplied by the first projection feature to obtain a third projection feature, and the third projection feature is normalized to obtain a second intermediate feature. The graph neural network extracts a first graph feature vector from the second intermediate features; wherein the computation function of the graph neural network is: ； in, As the second intermediate feature, The mapping function is the computation function of the graph neural network. to From the 1st to the 2nd The size of the receptive field to Let be the order of the polynomial. to These are the weight values ​​corresponding to each polynomial. to This is the diagonal matrix representing the eigenvalues ​​of each polynomial; The calculation process of the feature vector of the second graph output by the second branch network includes: The first intermediate feature is extracted from the second multi-source sensing data according to the embedding layer; The second intermediate feature is extracted from the first intermediate feature using the Transformer. The second intermediate feature is projected into a first projection feature by the second projection head, the first projection feature is transposed into a second projection feature, the second projection feature is multiplied by the first projection feature to obtain a third projection feature, and the third projection feature is normalized to obtain a third intermediate feature. The second graph feature vector is extracted from the third intermediate feature using the graph neural network. The sensor data of the same category in the first feature vector are aggregated to obtain the third feature vector. The sensor data of the same category in the second feature vector are aggregated to obtain the fourth feature vector. The sensor data of the same category in the third feature vector and the fourth feature vector are fused to obtain the fused feature vector. Based on the fused feature vectors, the safety status of the construction structure is predicted using the network model. A loss function is constructed, and based on the loss function, the network model is backpropagated until the trained network model is used as a safety monitoring model; wherein, the loss function includes at least: a first loss that measures the distribution error between the feature vectors of the first multi-source sensor data corresponding to the map and the feature vectors of the second multi-source sensor data corresponding to the map, a second loss that measures the domain classification error between the first multi-source sensor data and the second multi-source sensor data, and a third loss that measures the error between the safety status predicted by the classifier and the actual safety status of the construction structure; The model uses a module to input multi-source sensor data of the construction structure at the current moment into the safety monitoring model, so as to obtain the current safety status of the construction structure predicted by the safety monitoring model.

5. An electronic device, characterized in that, include: At least one controller and a memory for communicatively connecting with said at least one controller; The memory stores instructions that can be executed by the at least one controller, which are executed by the controller to cause the controller to perform the construction structure safety monitoring method based on sensor data as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the construction structure safety monitoring method based on sensor data as described in any one of claims 1 to 3.

Images