Building material monitoring system based on Internet of Things data

Abstract

The invention discloses a building material monitoring system based on data of the Internet of Things. The building material monitoring system comprises a multi-modal data acquisition module which is used for deploying a multi-modal sensor to acquire multi-modal building data; the data state extraction module is used for integrating different modal data according to an optimal alignment relationship by utilizing an optimized phase synchronization loss function based on the multi-modal building data to obtain optimized comprehensive feature data; and the state monitoring acquisition module is used for performing feature screening through an improved meta-learning framework to obtain the importance of comprehensive state data on building material state evaluation, performing reconstruction optimization based on the importance to obtain optimal state data, and obtaining a building material monitoring result based on the optimal state data. The system can automatically adapt to different building materials and monitoring scenes, the attention degree of each feature is automatically adjusted according to the material characteristics, and the self-adaptability and the intelligent level of the monitoring system are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a building material monitoring system based on Internet of Things data. Background Art

[0002] At present, with the continuous expansion of the building scale and the increase in the complexity of the building structure, the quality and performance of building materials play a decisive role in the safety and durability of the building structure. Accurately and real-time monitoring of the state of building materials has become a key link in ensuring the quality of construction projects. Traditional means of monitoring building materials have many limitations. And the detection frequency is limited, it is difficult to capture the real-time changes of the material state, and it is easy to miss key abnormal situations.

[0003] In the prior art, the data collected by different types of sensors often have problems of time asynchronization and inconsistent formats. For the collected data, traditional analysis methods are mostly based on simple threshold judgment or empirical models, and it is difficult to explore the complex internal relationships and change laws behind the data, and it is impossible to accurately evaluate the actual state of building materials. For example, when monitoring concrete materials, traditional methods are difficult to comprehensively consider the influence of the coupling effects of multiple factors such as stress, temperature and humidity, and composition changes on the durability of concrete. Therefore, a building material monitoring system based on Internet of Things data is proposed herein. Summary of the Invention

[0004] In order to overcome the above-mentioned defects of the prior art and to achieve the above object, the present invention proposes the following technical solutions: A building material monitoring system based on Internet of Things data, comprising: A multi-modal data acquisition module: deploying multi-modal sensors to collect multi-modal building data; A data state extraction module: based on the multi-modal building data, using an optimized phase synchronization loss function to integrate different modal data according to the best alignment relationship to obtain optimized comprehensive feature data; A state monitoring and acquisition module: obtaining the importance of the comprehensive state data for the evaluation of the building material state through a feature screening by an improved meta-learning framework, reconstructing and optimizing based on the importance to obtain the optimal state data, and obtaining the building material monitoring result based on the optimal state data; The optimized phase synchronization loss function is obtained by adding an optimal time delay parameter and a feature difference optimization factor to the basic phase synchronization loss function; The improved meta-learning framework is based on a spatio-temporal separated Transformer architecture and combines a dual attention mechanism to obtain the building material monitoring result from the comprehensive state data.

[0005] The multi-modal building data includes stress material deformation data, temperature and humidity influence data, composition change data, and material traceability data; The multi-modal building data is obtained by collecting and preprocessing stress material deformation data, temperature and humidity influence data, composition change data, and material traceability data at a specific frequency and integrating them according to the timestamp and sensor type.

[0006] The process of obtaining the optimal time delay parameter is as follows: Define the original phase synchronization loss function as an objective function; Adjust the original time delay parameter within the parameter space based on the objective function and update the original time delay parameter according to the gradient direction until the objective function converges; After the objective function converges, find the original time delay parameter that minimizes the objective function, calculate the gradient of the loss function based on the current time delay parameter, and obtain the optimal time delay parameter when the change in the loss function gradient is less than a specific threshold.

[0007] The process of obtaining the feature difference optimization factor is as follows: Obtain the change rates of two different modal building data and traverse the entire data sequence to calculate the square of the change rate at each time point; Process the results of all time points through a recurrent neural network to input the sequence data, receive and update the internal state at each time step, and obtain the feature difference optimization factor.

[0008] The dual attention mechanism is obtained by adding a global attention module on the basis of the self-attention mechanism in a spatio-temporally separated Transformer.

[0009] The process of obtaining the importance of the comprehensive state data for the evaluation of the building material state is as follows: Separate and process the time and space dimensions through a spatio-temporally separated Transformer architecture to obtain the feature representation in the time dimension and the feature representation in the space dimension; Perform feature fusion based on the feature representation in the time dimension and the feature representation in the space dimension to obtain the spatio-temporal feature representation; Obtain the spatio-temporal feature representation based on the dual attention mechanism and calculate the global importance score of each feature; Combine the global importance score and the spatio-temporal feature representation in a contrastive manner, and then map out the final importance score of each feature through the fully connected layer of the spatio-temporally separated Transformer architecture, which is the importance for the evaluation of the building material state.

[0010] The process of reconstructing and optimizing to obtain the optimal state data based on the importance is as follows: Divide the important features and auxiliary features based on the importance, and construct a feature hierarchical structure based on the important feature group and the auxiliary feature group; Optimize and reconstruct the feature hierarchy through a variant of the generative adversarial network to obtain the optimal state data.

[0011] The process of the reconstruction optimization is implemented based on a variant of the generative adversarial network; The variant of the generative adversarial network is obtained by adding a guide on the basis of the traditional generative adversarial network; The variant of the generative adversarial network includes a generator, a discriminator, and a guide; The guide adjusts the input of the generator, enabling the generator to prioritize features with high importance and perform feature generation and fusion according to feature associations for optimization and reconstruction.

[0012] The present invention has the following beneficial effects: In the present invention, first, different modality data are integrated by optimizing the phase synchronization loss function, and the function adds the optimal time delay parameter and the feature difference optimization factor. Not only is the synchronization of different modality data in the time dimension optimized through the time delay parameter but also the differences in feature dimensions are considered with the help of the feature difference optimization factor to achieve more comprehensive alignment of the data in the time and feature spaces, further reflecting the comprehensive internal correlation between the multi-modal data of building materials and effectively solving the problems of asynchronous time and inaccurate feature alignment of different modality data; Secondly, an improved meta-learning framework is based on the spatio-temporal separated Transformer architecture and combined with the dual attention mechanism. The spatio-temporal separated Transformer architecture reduces the computational cost and better captures the spatio-temporal relationship. The dual attention mechanism focuses on the overall importance of different time steps and modality features, and adjusts the feature relationship through contrastive learning, enabling the system to automatically adapt to different building materials and monitoring scenarios and automatically adjust the attention to each feature according to the material characteristics; Finally, the feature groups are divided based on importance and a hierarchy is constructed, and then optimized and reconstructed through a variant of the generative adversarial network. The guide in the variant of the generative adversarial network guides the generator to generate more representative feature representations according to the feature importance and association relationship, enabling the model to pay more attention to important features and better monitor the state of building materials. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a system block diagram of a building material monitoring system based on Internet of Things data proposed by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0014] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0015] Embodiment: As Figure 1 shown, a building material monitoring system based on Internet of Things data proposed by the present invention includes: Multimodal data acquisition module: Deploy multimodal sensors to collect stress material deformation data, temperature and humidity influence data, composition change data, and material traceability data to form multimodal building data; The process of collecting stress material deformation data is as follows: Collect stress material deformation data through a stress-strain sensor , within the elastic limit, stress and strain The relationship is , where is the elastic modulus. Through the stress-strain sensor, the deformation of building materials under stress can be obtained in real time. For example, in a concrete structure, the sensor can sense the tiny deformation of concrete caused by external loads; The process of collecting temperature and humidity influence data is as follows: Collect temperature and humidity influence data through a temperature and humidity sensor , the influence of temperature data and humidity data on building materials includes affecting their strength and dimensional stability. For example, temperature changes will cause the thermal expansion and contraction of materials, generating internal stress. The temperature and humidity sensor can obtain the influence of the ambient temperature and relative humidity on building materials; The process of collecting composition change data is as follows: Obtain composition change data through a composition analysis sensor , for some composite materials or building materials with active chemical properties, the composition analysis sensor can detect the changes in their chemical components. For example, in steel, detect whether the carbon content, alloy element content, etc. have changed; The process of collecting material traceability data is as follows: Collect material traceability data through a material traceability sensor (RFID) , by attaching RFID tags to building materials, record the production date and material source of the materials, which is convenient for tracking the source of building materials and quality traceability; Each sensor collects data at a specific sampling frequency. For example, for stress-strain sensors in a building structure subjected to dynamic loads, the instantaneous changes in stress and strain are captured through a relatively high sampling frequency (100 times per second). For temperature and humidity sensors, data is collected at a relatively low sampling frequency (once every 5 minutes). After the collected data undergoes preliminary preprocessing to remove noise and outliers, it is integrated according to the timestamp and sensor type. The integrated multi-modal building data is represented as , represents the multi-modal building data, is the timestamp; Specifically, the process of integrating according to the timestamp and sensor type is as follows: Arrange the preprocessed data in chronological order. Since different sensors have different sampling frequencies and the granularity of timestamps also varies, these data are unified and aligned in the time dimension so that each time point can correspond to the data collected by each sensor at that moment or the moment closest to it. Then, based on the aligned timestamps, the data of different sensor types at the same time point are integrated and grouped according to sensor type to form multi-modal building data .

[0016] Data status extraction module: Based on the multi-modal building data, use the optimized phase synchronization loss function to integrate different modal data according to the best alignment relationship to obtain optimized comprehensive feature data; Based on the multi-modal building data , use the optimized phase synchronization loss function to align any two different modal building data. Let the two different modal building data and . By adding the best time delay parameter and feature difference optimization factor to the basic phase synchronization loss function, the optimized phase synchronization loss function is obtained, which is expressed as:

[0017] where, is the data sequence length, represents the best time delay parameter, represents the weight coefficient, is the feature difference optimization factor; The process of obtaining the best time delay parameter is as follows: The basic phase synchronization loss function contains the original time delay parameter. Define the basic phase synchronization loss function as an objective function , within a certain parameter space, continuously adjust the value of the original time delay parameter, calculate the objective function value, and update the original time delay parameter according to the gradient direction until the objective function converges, and find the one that makes the objective function minimum . In each iteration, calculate the gradient of the loss function according to the current value, and repeat this process until the convergence condition is met (the change in the loss function value is less than the threshold ) to obtain the optimal time delay parameter ; Specifically, the iteration formula is expressed as:

[0018] where is the learning rate, is the previous time delay parameter, represents the gradient of the previous time delay parameter; The process of obtaining the parameter space is as follows: The setting of the range is based on prior knowledge of the problem and the actual situation. In the monitoring of building materials, it is known that the time delay between stress changes and temperature and humidity changes will not exceed a certain specific value (for example, it is judged not to exceed 10 sampling periods according to physical principles and actual experience). Then, the parameter space of the time delay parameter can be set to [0, 10] (the sampling period is a fixed value); Specifically, in each iteration, according to the current time delay parameter , substitute it into the phase synchronization loss function , calculate the function value at this time. This value reflects the difference degree between the two data sequences under the current time delay parameter . Then, calculate the gradient of the phase synchronization loss function with respect to the time delay parameter . The gradient is a vector, and its direction indicates the direction in which the function value increases fastest. Update the time delay parameter according to the formula of , where the learning rate controls the step size of each update . If the learning rate is too large, it will cause the time delay parameter to skip the optimal value during the update process and fail to converge. If the learning rate is too small, the optimization process will become slow and more iteration times are required to converge. By continuously repeating the process of calculating the objective function value and updating the time delay parameter , gradually make the time delay parameter approach the optimal value; Feature difference optimization factor The acquisition process is as follows: The feature difference optimization factor is obtained based on the difference measurement of two different modalities of building data in the feature dimension; For example, for stress material deformation data and composition change data, while paying attention to synchronization in time, consider the difference between the stress change rate and the composition change rate (which can be the difference between any two different modalities of data); Obtain multi-modal building data The change rates of any two different modalities of building data in the multi-modal building data are obtained. Here, the stress change rate and the composition change rate are taken as examples; Obtain the change rate of the stress material deformation data and the change rate of the composition change data , traverse the entire data sequence, from the time stamp to ( is the length of the data sequence), calculate the squares of the stress change rate and the composition change rate at each time point in turn (the difference between the change rates of any two different modalities of building data. Here, the stress change rate and the composition change rate are taken as examples), and then process the result of all time points through a recurrent neural network to input the sequence data, receive at each time step and update the internal state. Finally, the output or the transformed result is obtained to get the value of the feature difference optimization factor , which is expressed by the formula:

[0019] where represents the recurrent neural network operation; Specifically, the state of building materials is not only related to the data value, but also includes the influence of dynamic feature changes (such as the difference between stress mutation and composition change). By calculating the square of the difference between change rates to capture feature differences, and then using RNN to process sequence data, it can effectively integrate the feature difference information in the time series, comprehensively reflect the alignment requirements of the feature dimension, innovatively introduce the feature change rate difference into the loss function, and use RNN to process this sequence, making full use of its memory and dynamic feature capture capabilities for time series, so that the acquisition of feature differences not only includes simple accumulation, but also includes sequence correlation information, improving the depth of feature space alignment; For each modality of building data in the multi-modal building data , apply the optimized phase synchronization loss function for processing respectively; For example, for the stress material deformation data and the temperature and humidity influence data (any two different modalities of building data. Here, and are taken as examples), by adjusting the time delay parameter , such that is minimized, and the optimal time alignment relationship between the two is found through the feature difference optimization factor ; Similarly, such operations are also performed on other modal data, and the comprehensive state data after integration of all different modal data according to the optimal alignment relationship is obtained ; Specifically, the core of the phase synchronization loss function belongs to the cross-modal feature alignment mechanism. The essential purpose is to align different modal data as much as possible in the time and feature spaces. However, the original function only performs alignment based on the data values themselves and does not deeply consider deeper feature differences such as the feature change rate; For example, the data values of two materials are close, but one changes violently (such as sudden stress impact) and the other changes gently (such as slow natural oxidation of components). The original function cannot effectively distinguish this difference, resulting in inaccurate alignment in the feature space; Through the optimized phase synchronization loss function, not only the synchronization of different modal data in the time dimension is optimized through the time delay parameter (such as adjusting the acquisition time deviation between stress data and component data), but also the differences in the feature dimensions are considered with the help of the feature difference optimization factor (such as the difference between the stress change rate and the component change rate), enabling more comprehensive alignment of the data in the time and feature spaces and further reflecting the comprehensive internal correlation between the multi-modal data of building materials.

[0020] State monitoring acquisition module: The importance of the comprehensive state data for the state assessment of building materials is obtained through feature screening by an improved meta-learning framework, and the optimal state data is obtained through reconstruction and optimization based on the importance, and the monitoring results of building materials are obtained based on the optimal state data; The improved meta-learning framework is based on the spatio-temporal separated Transformer architecture and combines the dual attention mechanism to obtain the monitoring results of building materials from the comprehensive state data; The process of obtaining the importance of the comprehensive state data for the state assessment of building materials is as follows: The time and space dimensions are separated and processed through the spatio-temporal separated Transformer architecture; In the time dimension, the comprehensive state data is unfolded by time steps and then input into the Transformer layer in time series, and the self-attention in the time dimension is calculated through linear transformation to obtain the feature representation in the time dimension ; In the space dimension, for the comprehensive state data The different modal features in the image are mapped to multiple subspaces through linear transformation, and the separate self-attention is calculated in each subspace for fusion to obtain the feature representation in the spatial dimension. ; Represent the features in the time dimension and feature representation in spatial dimensions Perform feature fusion to obtain spatiotemporal feature representation ; Specifically, the traditional Transformer has high computational complexity when processing long sequences, while the time-space separation Transformer architecture separates the time and space dimensions, which can effectively reduce the computational cost and better capture the time-space relationship; A dual attention mechanism is introduced, which adds a global attention module based on the self-attention mechanism in the spatiotemporal separation Transformer to focus on the overall importance of different time steps and modal features for the state assessment of building materials; The global attention module represents the spatial and temporal features. Calculate the global importance score for each feature , the formula is:

[0021] in, represents a mathematical function, Represents the global importance score The weight matrix of Represents the global importance score The amount of bias term; The spatiotemporal features obtained by self-attention are represented and the global importance score In the process of combining, the contrastive learning method is used to bring the features with high correlation with the state of building materials closer and push the features with low correlation farther away, and then map the final importance score of each feature through the fully connected layer of the spatiotemporal separation Transformer , is the total number of features, and this final importance score represents the importance of the assessment of the building material condition; Specifically, the implementation process of contrastive learning is as follows: during the calculation process, for feature representations with high correlation of building material states, the temporally separated Transformers adjust parameters to bring them closer to each other in the feature space, that is, the distance between their feature vectors becomes smaller, while for feature representations with low correlation, they are moved away from each other; For example, the stress change characteristics and material deformation characteristics are highly relevant to the evaluation of the structural stability of building materials. Contrastive learning will make the representations of these two characteristics in the feature space closer, while for some secondary environmental factor characteristics that have little relation to structural stability, contrastive learning will increase the distance between their representations and the key characteristics; The process of reconstructing and optimizing based on importance to obtain the optimal state data is as follows: According to the final importance scores , divide them into an important feature group and an auxiliary feature group, and set a threshold (set to 0.8), features with importance scores greater than are classified into the important feature group , and those less than are the auxiliary feature group ; Based on the important feature group and the auxiliary feature group , construct a feature hierarchical structure. The important feature group is at the highest level, representing factors that play a key role in the state evaluation of building materials, such as features like stress and strain that directly affect structural safety. The auxiliary feature group is at the bottom level, covering features related to some secondary environmental factors or accidental factors; Optimize and reconstruct the feature hierarchical structure through a variant of the generative adversarial network (GAN). The traditional GAN consists of a generator and a discriminator. On this basis, add a guidance generator . The guidance generator provides guidance information for the generator according to the feature hierarchical structure and the result of association mining. Specifically: In the improved generative adversarial network (GAN) variant, the generator generates a feature representation that is similar to the original feature distribution but more compact and representative according to the importance scores; The discriminator judges the difference between the generated feature representation and the real feature representation; The guidance generator enables the generator to focus on features with high importance preferentially by adjusting the input of the generator, and perform reasonable feature generation and fusion according to feature associations, finally obtaining the optimized and reconstructed optimal state data ; Specifically, the guidance generator plays a role by adjusting the input and parameter update direction of the generator. When updating the generator parameters, the guidance generator influences the weight update of the generator according to the importance scores of the features , adjusts the loss function of the generator, and the adjusted loss function of the generator is:

[0022] Among them, Indicates the importance score of the nth feature, Indicates the part of the feature representation generated by the generator that is related to the important feature group in the generated feature representation; Indicates the part of the feature representation generated by the generator that is related to the auxiliary feature group in the generated feature representation; Based on the optimal state data, the monitoring results of building materials are as follows: Assume that after meta-learning feature screening and reconstruction optimization, the optimal state data contains the following key features (dimensionless values): Optimal stress state: reflecting the real-time stress borne by the concrete, importance score 0.92 (important feature); Optimal strain state: the degree of deformation of the concrete, importance score 0.88 (important feature); Optimal temperature state: the influence of environmental temperature on the concrete, importance score 0.75 (auxiliary feature); Optimal humidity state: the influence of environmental humidity on the durability of the concrete, importance score 0.70 (auxiliary feature); According to the concrete material standard and historical failure data, the state thresholds of important features and auxiliary features are set as follows:

[0023] When any feature is monitored to exceed the warning threshold, the corresponding feature type is first displayed, and then the corresponding single warning is triggered; When it is monitored that both the stress and strain exceed the warning threshold, or the humidity exceeds the failure threshold and the temperature > 30°C, it is determined as a composite feature failure in the failure state.

[0024] In the application, several formulas involved are calculated by taking their dimensionless values. The formulas are established by collecting a large amount of data for software simulation to obtain a formula that is closest to the real situation. Some coefficients or weights in the formulas are set by those skilled in the art according to the actual situation, so no more details will be given here.

Claims

1. A building material monitoring system based on Internet of Things data, characterized in that: include: Multimodal data acquisition module: deploy multimodal sensors to collect multimodal building data; Data state extraction module: Based on multimodal building data, the optimized phase synchronization loss function is used to integrate different modal data according to the best alignment relationship to obtain optimized comprehensive feature data; Condition monitoring acquisition module: Through an improved meta-learning framework, feature screening is performed to obtain the importance of comprehensive condition data for building material condition assessment, and reconstruction optimization is performed based on the importance to obtain the optimal condition data, and building material monitoring results are obtained based on the optimal condition data; The optimized phase synchronization loss function is obtained by adding an optimal time delay parameter and a feature difference optimization factor to a basic phase synchronization loss function; The improved meta-learning framework is based on the spatiotemporal separation Transformer architecture and combines the dual attention mechanism to obtain building material monitoring results from comprehensive state data.

2. A building material monitoring system based on Internet of Things data according to claim 1, characterized in that: The multimodal building data includes stress material deformation data, temperature and humidity impact data, composition change data and material traceability data; The multimodal building data is obtained by collecting and preprocessing stress material deformation data, temperature and humidity impact data, composition change data and material traceability data at a specific frequency, and integrating them according to timestamps and sensor types.

3. A building material monitoring system based on Internet of Things data according to claim 1, characterized in that: The optimal time delay parameter acquisition process is as follows: The original phase synchronization loss function is defined as an objective function; Adjusting the original time delay parameters in the parameter space based on the objective function and updating the original time delay parameters according to the gradient direction until the objective function converges; After the objective function converges, the original time delay parameter that minimizes the objective function is found, and the loss function gradient is calculated according to the current time delay parameter. When the loss function gradient change is less than a specific threshold, the optimal time delay parameter is obtained.

4. The building material monitoring system based on Internet of Things data according to claim 1 is characterized in that: The process of obtaining the feature difference optimization factor is as follows: Obtain the change rate of building data of two different modes, and traverse the entire data series to calculate the square of the change rate at each time point; The results of all time points are processed through a recurrent neural network to process the input sequence data, receive and update the internal state at each time step, and obtain the feature difference optimization factor.

5. The building material monitoring system based on Internet of Things data according to claim 1 is characterized in that: The dual attention mechanism is obtained by adding a global attention module based on the self-attention mechanism in the spatiotemporal separation Transformer.

6. A building material monitoring system based on Internet of Things data according to claim 5, characterized in that: The process of obtaining the importance of comprehensive status data for building material status assessment is as follows: The time and space dimensions are separated through the time-space separation Transformer architecture to obtain feature representation in the time dimension and feature representation in the space dimension; Based on the feature representation in the time dimension and the feature representation in the space dimension, feature fusion is performed to obtain the spatiotemporal feature representation; Obtain spatiotemporal feature representation based on the dual attention mechanism and calculate the global importance score of each feature; The global importance score is combined with the spatiotemporal feature representation in a contrastive way, and then the final importance score of each feature, i.e., its importance to the assessment of the building material status, is mapped out through the fully connected layers of the spatiotemporal-separated Transformer architecture.

7. The building material monitoring system based on Internet of Things data according to claim 1 is characterized in that: The process of reconstructing and optimizing based on importance to obtain the optimal state data is as follows: Divide the important feature group and the auxiliary feature group based on importance, and build a feature hierarchy structure based on the important feature group and the auxiliary feature group; The feature hierarchy is optimized and reconstructed through a generative adversarial network variant to obtain the optimal state data.

8. The building material monitoring system based on Internet of Things data according to claim 7 is characterized in that: The reconstruction optimization process is implemented based on a generative adversarial network variant; The generative adversarial network variant is obtained by adding a guide on the basis of the traditional generative adversarial network; The generative adversarial network variant includes a generator, a discriminator and a guide; The guide adjusts the input of the generator so that the generator can give priority to features with high importance, and generates and fuses features according to feature associations for optimized reconstruction.

Images