Multi-source data fused large steel structure digital twin model construction method and system

Abstract

The invention belongs to the field of steel structure health monitoring, and provides a multi-source data fused large steel structure digital twin model construction method and system, and the method comprises the steps: constructing a geometric model of an in-service steel structure based on multi-view point cloud data and a design drawing; obtaining multi-source heterogeneous data of the steel structure; performing cross-modal fusion on the multi-source heterogeneous data to obtain cross-modal fusion features; constructing a digital twinborn model corresponding to the steel structure geometric model by combining the steel structure geometric model and the cross-modal fusion features; and performing dynamic prediction correction on the constructed digital twinborn model to obtain a corrected digital twinborn model. The current steel structure damage condition is comprehensively predicted by fusing multi-parameter signals including strain, temperature, vibration and the like, and the damage state prediction is more accurate.

Description

Technical Field

[0001] The present invention belongs to the field of steel structure health monitoring, and in particular relates to a method and system for constructing a large-scale steel structure digital twin model integrating multi-source data. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] The traditional steel structure health monitoring system has shortcomings such as opaque global status, inaccurate fault tracing, unclear situation evolution, and unclear disposal decisions, which bring one-sided, static, passive and blind operation and maintenance challenges to complex large-scale steel structures.

[0004] For example, the existing steel structure health monitoring method uses digital twin technology to build a digital twin model corresponding to the steel structure entity, such as the publication number CN 111881495A, the invention name is a prestressed steel structure safety assessment method based on digital twin, which uses BIM and three-dimensional scanning technology to build a digital twin model of the steel structure, relying on the force sensor arranged on the surface of the steel cable to obtain real-time monitoring signals, and establishes a structural prediction model based on experimental and historical big data, and predicts the safety risk level of the structure based on real-time data. The defects of this method are: the health status of the steel structure is judged only by the tension of the steel cable, and the evaluation index is single. Another example is the model building method, system, equipment and storage medium based on digital twins with publication number CN 116227225 A. It first obtains theoretical basic data and experimental data, builds a theoretical simulation model according to the theoretical basic data and model building rules, and then collects data based on strain gauges installed at multiple positions of the steel structure, corrects the theoretical simulation model in combination with the model correction rules, and uses the corrected model as the target simulation model. The defects are: the sensor sensing points are sparsely arranged, and it is difficult to obtain sensor data between adjacent installation points, resulting in the loss of key perception information of the steel structure. Summary of the invention

[0005] In order to solve at least one technical problem existing in the above-mentioned background technology, the present invention provides a method and system for constructing a large-scale digital twin model of steel structure by integrating multi-source data. It obtains cross-modal fusion features by cross-modal fusion of multi-source heterogeneous data, and integrates multi-parameter signals such as strain, temperature, and vibration to comprehensively predict the current damage of the steel structure, making the damage status prediction more accurate.

[0006] In order to achieve the above object, the present invention adopts the following technical solution: A first aspect of the present invention provides a method for constructing a large-scale steel structure digital twin model by integrating multi-source data, comprising the following steps: The geometric model of the in-service steel structure is constructed based on multi-view point cloud data and design drawings; Acquire multi-source heterogeneous data reflecting the health status of steel structures; Cross-modal fusion of multi-source heterogeneous data is performed to obtain cross-modal fusion features, including: Based on the multi-source heterogeneous data of steel structure, strain and temperature characteristic mapping and vibration characteristic mapping are obtained; Calculate the representative value of the information of the strain and temperature characteristic map at any position as a univariate function; The correlation between the information at any position in the strain and temperature feature map and the information at any position in the vibration feature map is calculated as an embedded Gaussian function as a binary function; Cross-modal fusion is performed based on univariate functions and binary functions to obtain cross-modal fusion features; The digital twin model corresponding to the steel structure geometric model is constructed by combining the steel structure geometric model data and the cross-modal fusion feature data; The constructed digital twin model is dynamically predicted and corrected to obtain a corrected digital twin model.

[0007] Furthermore, the geometric model of the in-service steel structure is constructed based on the acquired multi-view point cloud data and the design drawings, including: A three-dimensional model of the in-service steel structure is constructed based on the acquired multi-view point cloud data; Build a BIM design model of the in-service steel structure based on the design drawings; The 3D model of the in-service steel structure is compared with the BIM design model, and the characteristic point matrix of the 3D point cloud model of the steel component is matched to obtain the geometric 3D model of the steel structure.

[0008] Furthermore, after obtaining multi-source heterogeneous data reflecting the health status of the steel structure, data preprocessing is performed, including: De-noising multi-source heterogeneous data based on the constructed denoising autoencoder; Normalize the multi-source heterogeneous data after denoising.

[0009] Furthermore, the formula for obtaining the cross-modal fusion feature by cross-modal fusion based on the unary function and the binary function is: , Among them, the one-variable function Used to calculate feature maps In Location Information Characteristic value, binary function Compute feature maps as embedded Gaussian functions Middle position Information and feature maps Middle position Information of relevance.

[0010] Furthermore, multi-source heterogeneous data reflecting the health status of the steel structure are acquired based on the constructed multi-source heterogeneous data acquisition system, which includes a pulse light generator, a circulator, a wavelength division multiplexer, two photodetectors, a sensing optical fiber and a computer; the computer sends a control signal to the pulse light generator and the photodetector and starts to collect signals through three channels at the same time, the pulse light generator emits pulse light, which reaches the sensing optical fiber arranged at a set position on the surface of the steel structure through the circulator and the wavelength division multiplexer, the backward Rayleigh reflected light is collected by the first photodetector through the wavelength division multiplexer and the circulator, and the backward Brillouin reflected light including Stokes light and anti-Stokes light is collected by the second photodetector through the wavelength division multiplexer respectively, and the collected reflected light signal is converted into an electrical signal and transmitted to the main control computer for analysis, so as to obtain the strain, temperature or vibration distribution of the steel structure surface at the current moment.

[0011] Furthermore, the digital twin model corresponding to the steel structure geometric model is constructed by combining the steel structure geometric model data and the cross-modal fusion feature data, including: Align the steel structure geometry model data and the cross-modal fusion feature data in the time dimension and the space dimension; The deduction model is trained based on the aligned steel structure geometric model data and the fused feature data before and after the steel structure parameter changes to obtain a trained deduction model; Combining the real-time monitoring of multi-source heterogeneous data reflecting the health status of the steel structure and the results of finite element simulation analysis, the trained deduction model is used to map the monitoring data to the three-dimensional model, thus completing the construction of the digital twin model.

[0012] Furthermore, the constructed digital twin model is dynamically predicted and corrected to obtain a corrected digital twin model. The basis for the correction is to select different weights for multi-source data fusion according to the differences in the impact of multi-source data on different nodes, so as to achieve the goal of minimizing the sum of squared Euclidean distances between the fusion result and each data.

[0013] A second aspect of the present invention provides a large-scale steel structure digital twin model construction system integrating multi-source data, comprising: A geometric model building module, which is used to build a geometric model of the in-service steel structure based on multi-view point cloud data and design drawings; A data acquisition module, which is used to acquire multi-source heterogeneous data reflecting the health status of the steel structure; The cross-modal fusion module is used to perform cross-modal fusion on multi-source heterogeneous data to obtain cross-modal fusion features, including: Based on the multi-source heterogeneous data of steel structure, strain and temperature characteristic mapping and vibration characteristic mapping are obtained; Calculate the representative value of the information of the strain and temperature characteristic map at any position as a univariate function; The correlation between the information at any position in the strain and temperature feature map and the information at any position in the vibration feature map is calculated as an embedded Gaussian function as a binary function; Cross-modal fusion is performed based on univariate functions and binary functions to obtain cross-modal fusion features; A twin model construction module, which is used to combine the steel structure geometry model data and the cross-modal fusion feature data to construct a digital twin model corresponding to the steel structure geometry model; The constructed digital twin model is dynamically predicted and corrected to obtain a corrected digital twin model.

[0014] Furthermore, multi-source heterogeneous data are acquired based on the constructed multi-source heterogeneous data acquisition system, which includes a pulse light generator, a circulator, a wavelength division multiplexer, two photodetectors, a sensing optical fiber and a computer; the computer sends a control signal to the pulse light generator and the photodetector and starts to collect signals through three channels at the same time, the pulse light generator emits pulse light, which reaches the sensing optical fiber arranged at a set position on the surface of the steel structure through the circulator and the wavelength division multiplexer, the backward Rayleigh reflected light is collected by the first photodetector through the wavelength division multiplexer and the circulator, the backward Brillouin reflected light including Stokes light and anti-Stokes light is collected by the second photodetector through the wavelength division multiplexer respectively, the collected reflected light signal is converted into an electrical signal and transmitted to the main control computer for analysis, so as to obtain the strain, temperature or vibration distribution of the steel structure surface at the current moment.

[0015] Furthermore, in the twin model construction module, the constructed digital twin model is dynamically predicted and corrected to obtain a corrected digital twin model. The basis for the correction is to select different weights for multi-source data fusion according to the differences in the impact of multi-source data on different nodes, so as to achieve the goal of minimizing the sum of squared Euclidean distances between the fusion results and each data.

[0016] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention obtains cross-modal fusion features by cross-modal fusion of multi-source heterogeneous data. During fusion, multi-parameter signals including strain, temperature, vibration, etc. are used to comprehensively predict the current damage of the steel structure. The damage state prediction is more accurate, which solves the problems of single existing evaluation indicators and lack of key perception information of steel structures.

[0017] 2. The present invention applies distributed optical fiber monitoring technology to the health monitoring of large steel structures. Specifically, distributed optical fibers are laid at key nodes / parts of the steel structure to obtain multi-parameter information such as strain, temperature, vibration, etc. at the key nodes of the steel structure to avoid information omission problems.

[0018] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0020] Figure 1 It is a flow chart of a method for constructing a large-scale steel structure digital twin model by integrating multi-source data provided by an embodiment of the present invention; Figure 2 It is a point cloud registration neural network architecture provided by an embodiment of the present invention; Figure 3 It is the steel structure key parameter monitoring system acquisition provided by the embodiment of the present invention; Figure 4 is a schematic diagram of a denoising autoencoder network provided by an embodiment of the present invention; Figure 5 is a schematic diagram of cross-modal fusion features provided by an embodiment of the present invention; Figure 6 It is a schematic diagram of the structure of a generative decoder provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0022] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0023] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0024] In view of the problems of traditional steel structure health monitoring system mentioned in the background technology of this application, such as opaque global status, inaccurate fault tracing, unclear situation evolution, and unclear disposal decision-making, the present invention is based on multi-location and multi-type distributed optical fiber monitoring data to establish a multi-scale, multi-dimensional, highly reliable full-cycle digital twin model that integrates virtual and real, collaborates with virtual and real, and links virtual and real to support intelligent perception, judgment, decision-making and execution.

[0025] Specifically, it includes the following aspects: First, research on the refined 3D model reconstruction technology based on BIM and 3D laser panoramic scanning, applicable to different life stages of steel structures, to achieve the synchronization of the twin and physical entity’s appearance; Second, we will study the global state update mechanism that combines massive multi-source heterogeneous perception data fusion with large-scale structural finite element analysis to achieve state synchronization between the twin and the physical entity. Third, study the method of simulating the development of the performance trend of large steel structures under multiple scenarios, multiple working conditions and multiple risk factors to achieve the synchronization of the behavior of the twin and the physical entity.

[0026] Ultimately, digital twins will be used to transform “passive maintenance” into “active maintenance” and further develop a new generation of intelligent and safe operation and maintenance system for large steel structures.

[0027] Embodiment 1 like Figure 1 As shown, this embodiment provides a method for constructing a large-scale steel structure digital twin model by integrating multi-source data, including the following steps: Step 1: Construct a geometric model of the in-service steel structure based on the acquired multi-view point cloud data and design drawings; The specific steps include: Step 101: construct a three-dimensional model of the in-service steel structure based on the acquired multi-view point cloud data; The specific steps include: Step 1011: Acquire multi-view point cloud data; A terrestrial laser scanner (TLS) is used to scan the steel structure to obtain its point cloud data. In terms of multi-view point cloud 3D reconstruction, since a single laser scanner cannot obtain all the information about the steel structure, a specific plan needs to be formulated based on the steel structure. Multiple viewpoints are scanned to obtain different spatial information of the steel structure, and further all the information of the steel structure is obtained through registration to complete the reconstruction of a high-precision 3D model of the steel structure.

[0028] Step 1012: performing preprocessing such as noise reduction and voxel downsampling on the acquired point cloud data; Step 1013: aligning the point cloud data based on the attention mechanism to achieve rough alignment of the point cloud; Multi-view point cloud data registration is complex and of low precision, which seriously affects the three-dimensional reconstruction results of the model. The key to point cloud registration is feature extraction, and deep learning methods can extract deeper and more expressive features.

[0029] Based on this, the present invention proposes a point cloud registration method based on the attention mechanism, and builds Figure 2 The neural network architecture of point cloud registration is shown in Figure 1. First, the geometric feature extraction network (FCGF) is used to extract the features of the two point clouds, then the dynamic convolutional network (DGCNN) is used to strengthen the global features of the two point clouds, and finally the attention mechanism of Transformer is used to integrate the mutual information between the two point clouds to establish a connection.

[0030] Step 1014: construct multiple point cloud relationship pose graphs based on a graph optimization method, so that all point clouds are registered into a globally consistent representation; In order to reduce the computational burden and improve modeling efficiency, the sparse sampling method is used to reduce the density or quantity of point cloud data, thereby reducing the difficulty and time of data processing while ensuring modeling accuracy.

[0031] Step 1015: Finally, perform surface reconstruction to obtain a three-dimensional reconstructed model of the steel structure; Step 102: Obtain design drawings of newly constructed and constructed large-scale steel structures and construct a BIM design model; For newly built and constructed large-scale steel structures, a BIM design model is constructed through design drawings. During the construction process, the three-dimensional reconstruction method described in step 101 is used to reconstruct the three-dimensional model of the steel structure in the construction phase. The steel structure BIM design model is updated and corrected based on the three-dimensional model in the construction phase to achieve consistency between the three-dimensional model and the on-site data.

[0032] Step 103: Compare the three-dimensional model of the in-service steel structure with the BIM design model to obtain a three-dimensional geometric model of the steel structure; During on-site construction, in order to address problems such as geometric deviations and shape differences between the BIM design model and the on-site steel components, the quality of the steel components can be monitored by comparing the BIM design model with the 3D reconstructed model of the steel structure scanned on-site.

[0033] In view of key issues such as high precision requirements for steel component connections and complex error assessment of steel component assembly, different target feature points are designed according to the component type. The high-precision three-dimensional point cloud data of steel components obtained by three-dimensional laser scanning is semantically segmented to achieve the extraction of measured target feature points.

[0034] In this embodiment, the similarity between feature point sets is compared through the Proctor analysis algorithm, and the matching of the feature point matrix of the three-dimensional point cloud model of the steel component is achieved through continuous iterative optimization. Based on the obtained similarity transformation parameters, the point sets of adjacent steel components are accurately aligned, and the assembly order of the steel components is determined to complete the accurate construction of the three-dimensional point cloud model of the steel structure.

[0035] Step 2: Obtain multi-source heterogeneous data reflecting the health of the steel structure and perform preprocessing; Specifically include: Step 201: Building a multi-source heterogeneous data acquisition system for steel structures; In this embodiment, Figure 3 As shown in the figure, the steel structure key parameter monitoring system based on distributed optical fiber sensing is used to collect real-time monitoring data. The system includes a pulse light generator, a circulator, a wavelength division multiplexer, two photodetectors, a sensing optical fiber and a main control computer; When the system is working normally, the main control computer sends control signals to the pulse light generator and the photodetector and starts to collect signals through three channels at the same time. The pulse light generator emits infrared pulse light with a central wavelength of C band, which reaches the sensor fiber arranged at a set position on the surface of the steel structure through the circulator and the wavelength division multiplexer. Due to changes in external stress, temperature or vibration, the scattering information of the pulse light in the sensor fiber changes. This system mainly collects backward Rayleigh scattered light and two-way backward Brillouin scattered light. The backward Rayleigh reflected light is collected by the photodetector APD1 through the wavelength division multiplexer and the circulator. The backward Brillouin reflected light includes Stokes light and anti-Stokes light. Due to different wavelengths, they are collected by the photodetector APD2 through the wavelength division multiplexer. After the photodetector collects the reflected light signal, it converts it into an electrical signal and transmits it to the main control computer. After the main control computer performs signal preprocessing such as denoising and decoupling on the collected signal, it generates the corresponding strain, temperature or vibration curve diagram to obtain the strain, temperature or vibration distribution of the steel structure surface at the current moment.

[0036] When the system detects abnormal strain, temperature or vibration at a certain location, the location of the steel structure damage can be determined through signal decoupling analysis. Secondly, the intensity of the scattered signal can be analyzed to determine whether the optical fiber currently laid on the surface of the steel structure is broken or bent, thereby ensuring the reliability of the acquired data.

[0037] Step 202: Acquire multi-source heterogeneous data of the steel structure, including: environmental strain, temperature, vibration signal and other data reflecting the health of the steel structure, recorded as y ; Step 203: preprocessing the acquired multi-source heterogeneous data of steel structures; Multi-source heterogeneous data is structured data that is mainly dynamic time series data. By performing data cleaning work on multi-source data by deleting duplicate data and completing missing data, dynamic correction of existing erroneous data can be completed.

[0038] Aiming at the problem that the signal acquired by the system is accompanied by noise interference and the signal confidence is reduced, wavelet denoising, adaptive filtering and other methods are used to design a denoising autoencoder network to perform denoising preprocessing on the collected data, such as Figure 4 As shown, Figure 4 middle y is the original input signal, Y is the input signal after adding noise, h is the hidden layer neuron, y’ is the signal output by the neurons in the output layer.

[0039] The input layer neurons give the original input signal y Add random noise and discard some neurons with a certain probability to generate a noisy input signal Y. After the input layer neurons calculate, Y is mapped to the hidden layer, as shown in the following formula: , in is the initial training weight of the network, is the network bias term.

[0040] In order to improve the training effect of the network, the random discard method is used to optimize the network during the training process. The neurons in the hidden layer are discarded with a certain probability like the input signal. Therefore, the structure of the neural network is different in each iterative update, which can effectively improve the generalization ability of the model.

[0041] Hidden layer feature vector after randomly discarding neurons h Can reverse decode to get an output signal of the same size as the original input signal y' , the formula is as follows: , in Generate network training weights for random dropout, Generate network bias terms for random dropout.

[0042] The main purpose of network training is to minimize the input signal and reconstructed signal The error between is the loss function, and the formula is as follows: , in is the number of neurons.

[0043] It is worth noting that the denoising autoencoder of the present invention is improved on the basis of the autoencoder. The main idea is to first train an automatic encoder, add random noise to the original training data, and reconstruct the input data in the output layer so that the learned data has better generalization ability, thereby realizing denoising processing of the original data.

[0044] In order to solve the problem of differences in data representation, a data standardization and synchronization processing strategy is adopted. The data is normalized through the Min-Max and Z-Score methods to eliminate the differences in dimensions and dimensions between the data.

[0045] Step 3: Perform cross-modal fusion on multi-source heterogeneous data to obtain cross-modal fusion features; The multi-source heterogeneous data collected during the operation and maintenance of large steel structures contains a large amount of information. How to extract key features from heterogeneous multi-source information, filter redundant information, and achieve deep integration of multi-source data and three-dimensional models is a difficult problem that needs to be urgently solved in the current multi-source heterogeneous data fusion digital twin model.

[0046] The specific steps include: Step 301: Input the distributed optical fiber strain, temperature and vibration time series data collected based on the multi-source heterogeneous data monitoring platform into the pre-trained model to obtain the strain and temperature feature mapping and vibration feature mapping ; In this embodiment, the pre-trained model can use a deep neural network (such as a convolutional neural network, CNN, or a recurrent neural network, RNN) as the basic architecture, and effectively extract features from the input data by training the features learned on a large amount of labeled data.

[0047] A feature extraction method based on a pre-trained model is adopted to make the network pay more attention to the useful feature information of distributed optical fiber dynamic monitoring data and improve its ability to suppress redundant information.

[0048] Step 302: Then, the data of different modalities are comprehensively processed through feature fusion technology, and a cross-modal attention mechanism is introduced to further perform feature interaction between multi-source heterogeneous data, thereby improving the data fusion effect and providing more comprehensive, accurate and reliable information.

[0049] Figure 5 The structure diagram of the cross-modal attention module structure, the two inputs of this module are strain and temperature feature maps and vibration feature mapping , combined with strain and temperature feature mapping , vibration feature mapping And the constructed cross-modal cross-attention module obtains cross-modal fusion features ; The cross-modal attention module can be defined as: , in, Yes Features and positions at all positions on The result of normalized weighted aggregation of features; yes The index of a position in yes The index of a position in Unary Function Used to calculate feature maps In Location Information The characterization value is usually calculated using a linear weighted approach, as shown below: ,in As the weight parameters are optimized during the model training process, a 1×1 convolution operation can be used to implement it in actual operations.

[0050] In addition, in this embodiment, the binary function Compute feature maps as embedded Gaussian functions Middle position Information and feature maps Middle position Information Correlation, a binary function The expression is: , in, and As two embedded terms, the calculation process is as follows: , , Among them, the weight matrix and A 1×1 convolution operation is used to achieve optimization.

[0051] Finally, the cross-modal fusion features of the output of the cross-modal attention module By feature map and Adding them element by element gives: , in, As the weight matrix that needs to be trained, it is also implemented using a 1×1 convolution operation.

[0052] Step 4: Combine the steel structure geometry model data and cross-modal fusion features to construct a digital twin model corresponding to the steel structure geometry model; The specific steps include: Step 401: First, to ensure accurate data mapping and consistency, synchronous alignment of multi-source heterogeneous data and three-dimensional models in time and space dimensions is completed by using methods such as resampling and time series alignment; Step 402: training the deduction model based on the multi-view point cloud data and the multi-source heterogeneous data fusion features reflecting the changes of the steel structure health data before and after, to obtain a trained neural network deduction model; It is expressed as: , , in Represents the reconstructed geometric model of the steel structure, Indicates the data fusion characteristics before the key parameters of the steel structure change. It represents the data fusion characteristics after the key parameters of the steel structure change at time t. express t The mapping relationship between the multi-source data monitoring field and the reconstructed model deformation field at all times, represents the neural network deduction model, express t Monitoring data at all times; Step 403: Utilize the distributed fiber optic monitoring technology to collect real-time information on multiple physical quantities, such as temperature, stress, and vibration, and combine it with the results of finite element simulation analysis. Use the trained deduction model to map and deduce the data to achieve accurate mapping of the monitoring data to the three-dimensional model, thereby completing the construction of the digital twin model.

[0053] Step 404: Compare the monitoring results under the current monitoring state with the mapping results, and verify the accuracy and credibility of the digital twin deduction model by monitoring the difference between the measured data and the mapping results: , in represents the average calibration result, Represents the measured data, Represents the model mapping result.

[0054] Step 5: Perform dynamic prediction and correction on the constructed digital twin model to obtain a corrected digital twin model; In order to further improve the accuracy of the digital twin model, a dynamic prediction and correction method for the digital twin model of multi-source data fusion is proposed. According to the difference in the impact of multi-source data on different nodes, different weights are selected for multi-source data fusion to achieve the goal of minimizing the sum of squares of the Euclidean distances between the fusion results and each data. The calculation formula is: , in, For the Data sources in The spatial context information at the moment, For the The weight of each data source is set according to its impact on the steel structure. For the The bias item of the data source is used to adjust the fusion result and can be used as a part of the network optimization parameters.

[0055] Step 6: Modify the twin model by combining the fused data with the simulation data of the finite element model; like Figure 6 As shown in the figure, a generative decoder is used to predict the distributed optical fiber monitoring data at the current moment, and the multi-step prediction results are directly output to ensure the high restoration degree of the digital twin model. Specifically, the following steps are included: Step 601, input monitoring signal: the collected steel structure monitoring data is used as the "start character", and after all the predicted data are set to 0, it is input into the decoder as the decoder input data. The trained decoder predicts the decoder input data and puts the prediction result into the prediction content for output; Step 602: The decoding process of the entire decoder abandons the dynamic decoding process and uses a single forward process to decode and obtain the entire output sequence.

[0056] In addition, MSE is selected as the loss function during training, and the loss is calculated for the entire prediction content to obtain the final prediction result. The dimension of the fully connected layer output depends on the dimension of the variable to be predicted.

[0057] It should be noted that the original prediction method is point-by-point dynamic decoding, that is, the hidden layer state of the previous step and the output of the previous step are input to calculate the hidden layer state of the current step, and then the output data of the next step is predicted. This method has the disadvantages of a sharp drop in prediction speed and a sharp increase in loss as the monitoring time becomes longer. The use of a generative decoder can speed up the prediction speed of the network and reduce the cumulative error propagation during inference.

[0058] Through the dynamic prediction mechanism, it is ensured that the digital twin model reflects the state changes of the steel structure entity, and ultimately the construction of a digital twin model that collaborates with the steel structure entity in real time is realized.

[0059] Embodiment 2 This embodiment provides a large-scale steel structure digital twin model construction system integrating multi-source data, including: A geometric model building module, which is used to build a geometric model of the in-service steel structure based on multi-view point cloud data and design drawings; A data acquisition module, which is used to acquire multi-source heterogeneous data of steel structures; The cross-modal fusion module is used to perform cross-modal fusion on multi-source heterogeneous data to obtain cross-modal fusion features; specifically, it includes: Based on the multi-source heterogeneous data of steel structure, strain and temperature characteristic mapping and vibration characteristic mapping are obtained; Calculate the representative value of the information of the strain and temperature characteristic map at any position as a univariate function; The correlation between the information at any position in the strain and temperature feature map and the information at any position in the vibration feature map is calculated as an embedded Gaussian function as a binary function; Cross-modal fusion is performed based on univariate functions and binary functions to obtain cross-modal fusion features; A twin model construction module, which is used to combine the steel structure geometry model and the cross-modal fusion features to construct a digital twin model corresponding to the steel structure geometry model; The constructed digital twin model is dynamically predicted and corrected to obtain a corrected digital twin model.

[0060] The specific implementation process of each of the above modules is consistent with the content of Example 1, and please refer to the implementation process of Example 1 for details.

[0061] Embodiment 3 This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps in the method for constructing a large-scale steel structure digital twin model by integrating multi-source data as described above are implemented.

[0062] Embodiment 4 This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the method for constructing a large-scale steel structure digital twin model by integrating multi-source data as described above are implemented.

[0063] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for constructing a digital twin model of a large steel structure by integrating multi-source data, characterized in that: The steps include: The geometric model of the in-service steel structure is constructed based on multi-view point cloud data and design drawings; Acquire multi-source heterogeneous data reflecting the health status of steel structures; Cross-modal fusion of multi-source heterogeneous data is performed to obtain cross-modal fusion features, including: Based on the multi-source heterogeneous data of steel structure, strain and temperature characteristic mapping and vibration characteristic mapping are obtained; Calculate the representative value of the information of the strain and temperature characteristic map at any position as a univariate function; The correlation between the information at any position in the strain and temperature feature map and the information at any position in the vibration feature map is calculated as an embedded Gaussian function as a binary function; Cross-modal fusion is performed based on univariate functions and binary functions to obtain cross-modal fusion features; The digital twin model corresponding to the steel structure geometric model is constructed by combining the steel structure geometric model data and the cross-modal fusion feature data; The constructed digital twin model is dynamically predicted and corrected to obtain a corrected digital twin model.

2. The method for constructing a large-scale steel structure digital twin model integrating multi-source data according to claim 1, characterized in that: The geometric model of the in-service steel structure is constructed based on the acquired multi-view point cloud data and design drawings, including: A three-dimensional model of the in-service steel structure is constructed based on the acquired multi-view point cloud data; Build a BIM design model of the in-service steel structure based on the design drawings; The 3D model of the in-service steel structure is compared with the BIM design model, and the characteristic point matrix of the 3D point cloud model of the steel component is matched to obtain the geometric 3D model of the steel structure.

3. The method for constructing a large-scale steel structure digital twin model integrating multi-source data according to claim 1, characterized in that: After obtaining multi-source heterogeneous data reflecting the health status of the steel structure, data preprocessing is performed, including: De-noising multi-source heterogeneous data based on the constructed denoising autoencoder; Normalize the multi-source heterogeneous data after denoising.

4. The method for constructing a large-scale steel structure digital twin model integrating multi-source data according to claim 1, characterized in that: The formula for obtaining the cross-modal fusion feature by cross-modal fusion based on unary function and binary function is: ， Among them, the one-variable function Used to calculate feature maps In Location Information Characteristic value, binary function Compute feature maps as embedded Gaussian functions Middle position Information and feature maps Middle position Information of relevance.

5. The method for constructing a large-scale steel structure digital twin model integrating multi-source data according to claim 1, characterized in that: The multi-source heterogeneous data reflecting the health status of the steel structure are obtained based on the constructed multi-source heterogeneous data acquisition system, which includes a pulse light generator, a circulator, a wavelength division multiplexer, two photodetectors, a sensing optical fiber and a computer; the computer sends a control signal to the pulse light generator and the photodetector and starts to collect signals through three channels at the same time, the pulse light generator emits pulse light, which reaches the sensing optical fiber arranged at a set position on the surface of the steel structure through the circulator and the wavelength division multiplexer, the backward Rayleigh reflected light is collected by the first photodetector through the wavelength division multiplexer and the circulator, and the backward Brillouin reflected light includes Stokes light and anti-Stokes light, which are collected by the second photodetector through the wavelength division multiplexer respectively, and the collected reflected light signal is converted into an electrical signal and transmitted to the main control computer for analysis to obtain the strain, temperature or vibration distribution of the steel structure surface at the current moment.

6. The method for constructing a large-scale steel structure digital twin model integrating multi-source data according to claim 1, characterized in that: The digital twin model corresponding to the steel structure geometric model is constructed by combining the steel structure geometric model data and the cross-modal fusion feature data, including: Align the steel structure geometry model data and the cross-modal fusion feature data in the time dimension and the space dimension; The deduction model is trained based on the aligned steel structure geometric model data and the fused feature data before and after the steel structure parameter changes to obtain a trained deduction model; Combining the real-time monitoring of multi-source heterogeneous data reflecting the health status of the steel structure and the results of finite element simulation analysis, the trained deduction model is used to map the monitoring data to the three-dimensional model, thus completing the construction of the digital twin model.

7. The method for constructing a large-scale steel structure digital twin model integrating multi-source data according to claim 1, characterized in that: The constructed digital twin model is dynamically predicted and corrected to obtain a corrected digital twin model. The correction is based on the difference in the impact of multi-source data on different nodes, and different weights are selected to fuse multi-source data to achieve the goal of minimizing the sum of squared Euclidean distances between the fusion result and each data.

8. A large-scale steel structure digital twin model construction system integrating multi-source data, characterized by: include: A geometric model building module, which is used to build a geometric model of the in-service steel structure based on multi-view point cloud data and design drawings; A data acquisition module, which is used to acquire multi-source heterogeneous data reflecting the health status of the steel structure; The cross-modal fusion module is used to perform cross-modal fusion on multi-source heterogeneous data to obtain cross-modal fusion features, including: Based on the multi-source heterogeneous data of steel structure, strain and temperature characteristic mapping and vibration characteristic mapping are obtained; Calculate the representative value of the information of the strain and temperature characteristic map at any position as a univariate function; The correlation between the information at any position in the strain and temperature feature map and the information at any position in the vibration feature map is calculated as an embedded Gaussian function as a binary function; Cross-modal fusion is performed based on univariate functions and binary functions to obtain cross-modal fusion features; A twin model construction module, which is used to combine the steel structure geometry model data and the cross-modal fusion feature data to construct a digital twin model corresponding to the steel structure geometry model; The constructed digital twin model is dynamically predicted and corrected to obtain a corrected digital twin model.

9. The large-scale steel structure digital twin model construction system integrating multi-source data according to claim 8, characterized in that: Multi-source heterogeneous data are acquired based on the constructed multi-source heterogeneous data acquisition system, which includes a pulse light generator, a circulator, a wavelength division multiplexer, two photodetectors, a sensing optical fiber and a computer; the computer sends a control signal to the pulse light generator and the photodetector and starts to collect signals through three channels at the same time, the pulse light generator emits pulse light, which reaches the sensing optical fiber arranged at a set position on the surface of the steel structure through the circulator and the wavelength division multiplexer, the backward Rayleigh reflected light is collected by the first photodetector through the wavelength division multiplexer and the circulator, and the backward Brillouin reflected light includes Stokes light and anti-Stokes light, which are collected by the second photodetector through the wavelength division multiplexer respectively, and the collected reflected light signal is converted into an electrical signal and transmitted to the main control computer for analysis, so as to obtain the strain, temperature or vibration distribution of the steel structure surface at the current moment.

10. The large-scale steel structure digital twin model construction system integrating multi-source data according to claim 8, characterized in that: In the twin model construction module, the constructed digital twin model is dynamically predicted and corrected to obtain a corrected digital twin model. The basis for the correction is to select different weights for multi-source data fusion according to the differences in the impact of multi-source data on different nodes, so as to achieve the goal of minimizing the sum of squared Euclidean distances between the fusion results and each data.

Images