Building crack monitoring system and method based on machine learning

Abstract

The invention discloses a building crack monitoring system and method based on machine learning, and the system comprises a lower computer, a cloud server and an upper computer, and the interior of the upper computer is provided with a machine learning model, a multi-modal feature fusion module, an intelligent decision module, and a service management module. Data denoising and enhancement are performed on input data through a machine learning model, the definition of a crack image is ensured, features are automatically learned and extracted from the image through structures of a convolutional layer, a pooling layer and a full-connection layer, whether a crack exists in the image is judged, the crack is classified and identified, a neural network is established, and the image quality is improved. A learning model for building crack behavior recognition is established, image data of different modes and environment sensing data in the same period are fused in cooperation with multi-mode feature fusion, complementary information between the modes is captured, the building crack recognition precision is improved, crack image data collected regularly and the environment sensing data form time sequence data, and the time sequence data are stored in a database. And predicting the crack propagation speed.

Description

Technical Field

[0001] The present invention relates to the technical field of building crack monitoring, and specifically to a building crack monitoring system and method based on machine learning. Background Art

[0002] Cracks usually appear in structural buildings, especially in the northern regions where the climate is dry, the annual temperature difference is large, and the daily temperature difference in summer is also relatively large. During or after the construction of a house, cracks will occur in the wall under the influence of environmental climate, service life, and other aspects. The forms of cracks include diagonal cracks, vertical cracks, horizontal cracks, inverted V-shaped cracks, etc. They affect the function and aesthetics of the building, and in severe cases, they may even reduce the safety of the building structure and its seismic performance. Therefore, the crack detection of building structures has always been a very important task.

[0003] At present, most building crack monitoring systems rely on single sensing monitoring devices, which greatly limits the monitoring effect and cannot effectively achieve multi-dimensional data integration and real-time monitoring. In addition, in terms of crack image processing, the data quality of traditional sensors is insufficient, with a lot of noise resulting in low image clarity, and it is difficult to fuse multi-source data, thus affecting the accuracy of crack image recognition.

[0004] Therefore, the research direction of the present invention is how to develop a building crack monitoring system and method based on machine learning to solve the above-mentioned problems. Summary of the Invention

[0005] To solve the above-mentioned technical problems, the present invention provides a building crack monitoring system and method based on machine learning.

[0006] To achieve the above object, the present invention provides the following technical solutions: First, a building crack monitoring system based on machine learning is provided, including a lower computer, a cloud server, and an upper computer. The lower computer is a crack monitoring device, and it is internally provided with a micro-distance camera, a positioning sensor, an environmental sensor, a microprocessor, and a wireless transmission module; An image sensor is provided inside the micro-distance camera; A storage module is provided in the cloud server; The upper computer is an intelligent device with the function of sending operation instructions, and it is internally provided with a machine learning model, a multi-modal feature fusion, and a service management module; Among them, the wireless transmission module is connected to the cloud server through a base station to realize data exchange and communication between the lower computer and the upper computer.

[0007] Preferably, the positioning sensor is used for nanometer-level positioning of the lower computer through the RTK algorithm built in the Quectel communication module. The environment sensor is used to monitor the environmental data of the environment where the lower computer is located. The image sensor captures optical signals by receiving the shooting instructions of the micro-vision camera and converts them into digital images and transmits them to the image acquisition module; The image acquisition module decodes and processes the signals output by the image sensor and converts them into a format recognizable by the microprocessor. The microprocessor preliminarily collects and processes the sensing data of the environment sensor and the positioning sensor. The microprocessor sends a configuration command to the image acquisition module to set the working parameters of the image sensor and further processes and analyzes the image parameters.

[0008] Preferably, when the microprocessor recognizes that the image parameters do not meet the clarity setting requirements, it sends an instruction to the image sensor to send control signals and data to adjust the working mode and parameters of the image sensor, and re-capture digital image data until the image parameters recognized by the microprocessor meet the clarity setting requirements. Then, the image information is transmitted to the cloud server through the wireless transmission module. The cloud server receives the image data, environmental data, and positioning data transmitted by the microprocessor, stores them in the storage module, and transmits them to the machine learning model of the upper computer again.

[0009] Preferably, the machine learning model includes data collection and preprocessing, model construction, model training and optimization. Among them, the data collection and preprocessing obtains and collects image data containing building cracks through on-site shooting, web crawling, and public data sets, and annotates and preprocesses the collected images to make the image data meet the input requirements of the model. Denoise and enhance the input data, and at the same time extract and represent the features of the input environmental data of different modalities.

[0010] Preferably, the model construction automatically learns and extracts features from the image through the convolutional layer, pooling layer, and fully connected layer structures, judges whether there are cracks in the image, and classifies the cracks to build a neural network model. The model training and optimization divides the labeled crack image data set into a training set and a validation set. The training set is used for model training, and the validation set is used for model parameter adjustment and optimization.

[0011] Preferably, the multi-modal feature fusion extracts different modalities of image data and fuses them with the environmental sensing data of the same period, captures the complementary information between modalities, and constructs time-series data from the regularly collected crack image data and environmental sensing data to predict the crack propagation speed and transmits the result to the service management module.

[0012] Preferably, the service management module receives image data, environmental sensing data, and predicted crack velocity data in different modalities. The service management module includes a monitoring center, device management, building drawing management, and alarm management. The monitoring center sets a regional index based on the positioning sensor data to facilitate viewing the distribution area of the crack monitoring devices. Under the device management, there are a device list, device installation, and data forwarding, which are used to query the basic information, installation location, technical parameters, maintenance records, and data forwarding status of the crack monitoring devices.

[0013] Preferably, the building drawing management includes building management and drawing management. The building management counts the monitoring points and monitoring data of all crack monitoring devices in a certain building. The drawing management counts the building drawings monitored in the area. The alarm management includes monitoring alarm and device alarm. The device alarm monitors the device's own status in real time, pays attention to the hardware failures, communication terminals, and power status during device operation, and pushes device alarm information in case of abnormal situations.

[0014] Preferably, the monitoring alarm further includes an intelligent decision-making unit. The monitoring alarm receives the predicted crack propagation speed data, triggers an alarm and records it when the crack propagation threshold is exceeded. At the same time, it calls the drawing structure of the corresponding building and transmits it to the intelligent decision-making unit. Then, through the comprehensive analysis of the crack data, structural status, and environmental information, a maintenance strategy or risk response plan is generated.

[0015] Secondly, a building crack monitoring method based on machine learning is provided, including the following steps: S1: The lower computer is distributed and installed on the building object, regularly collects crack image data through a camera, regularly collects environmental data through a sensor, and performs preliminary analysis and processing through a microprocessor. S2: Transmit the image data and environmental data to the machine learning model, perform data denoising and enhancement on the input data, use the model to automatically learn and extract features from the image, and determine whether there are cracks and classify them. S3: Fuse the image data in different modalities with the environmental sensing data of the same period, capture the complementary information between modalities, and form time-series data to predict the crack propagation speed. S4: Trigger an alarm and record it when the crack propagation threshold is exceeded. At the same time, call the drawing structure of the corresponding building, and perform a comprehensive analysis of the crack data, structural status, and environmental information to generate a maintenance strategy or risk response plan.

[0016] Compared with the prior art, the beneficial effects of the present invention are: The machine learning model of the present invention is combined with multi-modal feature fusion. The input data is denoised and enhanced through the machine learning model to ensure the clarity of the crack image. At the same time, feature extraction and representation are performed on the input environmental data of different modalities. The convolutional layer, pooling layer, and fully connected layer structures are used to automatically learn and extract features from the image, determine whether there are cracks in the image, and classify and identify the cracks. A neural network is built to establish a learning model for building crack behavior recognition. Combining with multi-modal feature fusion, the image data of different modalities is fused with the environmental sensing data of the same period to capture the complementary information between modalities, improve the accuracy of building crack recognition, and form time-series data from the regularly collected crack image data and environmental sensing data to predict the crack propagation speed.

[0017] The microprocessor of the present invention can send instructions to change parameters such as the resolution, frame rate, exposure time, and gain of the image sensor to improve the clarity of the crack image. When the microprocessor recognizes that the image parameters do not meet the clarity setting requirements, it sends instructions to the image sensor, sending control signals and data to adjust the working mode and parameters of the image sensor, and re-grabbing digital image data until the image parameters recognized by the microprocessor meet the clarity setting requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a schematic diagram of the device transmission of the present invention; Figure 2 It is a schematic diagram of the system architecture of the present invention; Figure 3 It is a schematic diagram of the microprocessor working process of the present invention; Figure 4 It is a framework diagram of the machine learning model of the present invention; Figure 5 It is a framework diagram of the multi-modal feature fusion of the present invention; Figure 6 It is a framework diagram of the service management module of the present invention; Figure 7 It is an interface diagram of the system monitoring center of the present invention; Figure 8 It is an interface diagram of the system device management of the present invention; Figure 9 It is an interface diagram of the system building drawing management of the present invention; Figure 10 It is an interface diagram of the system help center of the present invention; Figure 11 It is a crack monitoring diagram of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0020] Please refer to Figure 1 - Figure 11 , the present invention first provides an embodiment: a building crack monitoring system based on machine learning, including a lower computer, a cloud server, and an upper computer. The lower computer is a crack monitoring device, which is internally provided with a micro-distance camera, a positioning sensor, an environmental sensor, a microprocessor, and a wireless transmission module. An image sensor is arranged inside the micro-distance camera. The cloud server is provided with a storage module. The upper computer is an intelligent device with the function of sending operation instructions, and is internally provided with a machine learning model, multi-modal feature fusion, and service management module. Among them, the wireless transmission module is connected to the cloud server through a base station to realize data exchange and communication between the lower computer and the upper computer; Further, the positioning sensor is used for the nanometer-level positioning of the lower computer through the RTK algorithm built in the Quectel communication module. The environmental sensor is used to monitor the environmental data of the environment where the lower computer is located, including environmental data such as temperature, humidity, wind speed, and wind direction. The image sensor captures optical signals by receiving the shooting instructions of the micro-distance camera and converts them into digital images and transmits them to the image acquisition module. The image acquisition module decodes and processes the signals output by the image sensor and converts them into a format that can be recognized by the microprocessor; Further, the microprocessor preliminarily collects and processes the sensing data of the environmental sensor and the positioning sensor. The microprocessor sends a configuration command to the image acquisition module to set the working parameters of the image sensor and further processes and analyzes the image parameters. For example, the microprocessor can send instructions to change parameters such as the resolution, frame rate, exposure time, and gain of the image sensor to improve the clarity of the crack image; Further, when the microprocessor recognizes that the image parameters do not meet the clarity setting requirements, it sends instructions to the image sensor, sends control signals and data to adjust the working mode and parameters of the image sensor, and re-captures digital image data until the image parameters recognized by the microprocessor meet the clarity setting requirements. Then, the image information is transmitted to the cloud server through the wireless transmission module. The cloud server receives the image data, environmental data, and positioning data transmitted by the microprocessor, stores them in the storage module, and transmits them to the machine learning model of the upper computer again; Furthermore, the machine learning model includes data collection and preprocessing, model construction, model training and optimization. Among them, data collection and preprocessing obtain and collect image data containing building cracks through on-site shooting, web crawling, and public data sets, and label and preprocess the collected images to make the image data meet the input requirements of the model. Data denoising and enhancement are performed on the input data. The data characteristics for different scenarios such as bridges and historical buildings may vary. For example, the images of historical buildings may have more complex backgrounds and require stronger denoising and enhancement methods to ensure the clarity of the crack images, laying a good foundation for the subsequent crack recognition accuracy. At the same time, feature extraction and representation are performed on the input environmental data of different modalities; Furthermore, model construction automatically learns and extracts features from images through convolutional layers, pooling layers, and fully connected layer structures, determines whether there are cracks in the images, classifies and identifies the cracks, builds a neural network, and establishes a learning model for building crack behavior recognition. Model training and optimization divide the labeled crack image data set into a training set and a validation set. The training set is used for model training, and the validation set is used for model parameter adjustment and optimization; Furthermore, multi-modal feature fusion extracts different modalities of image data and fuses them with contemporaneous environmental sensing data to capture complementary information between modalities, improve the accuracy of building crack recognition, and form time-series data from the regularly collected crack image data and environmental sensing data to predict the crack propagation speed and transmit the results to the service management module; Furthermore, the service management module receives different modalities of image data, environmental sensing data, and predicted crack speed data. The service management module includes a monitoring center, device management, building drawing management, and alarm management. The monitoring center sets area indexes according to the positioning sensor data to facilitate viewing the distribution areas of crack monitoring devices. Under device management, there are device lists, device installation, and data forwarding, which are used to query the basic information, installation locations, technical parameters, maintenance records, and data forwarding situations of crack monitoring devices; Furthermore, building drawing management includes building management and drawing management. Building management statistics the monitoring points and monitoring data of all crack monitoring devices of a certain building. Drawing management statistics the building drawings monitored in the area. Alarm management includes monitoring alarms and device alarms. Device alarms monitor the status of the device itself, and pay attention to hardware failures, communication terminals, and power status during device operation in real time. In case of abnormal situations, device alarm information is pushed; Furthermore, the monitoring alarm also includes an intelligent decision-making unit. The monitoring alarm receives the predicted crack propagation speed data, triggers an alarm and records it when it exceeds the crack propagation threshold. At the same time, it calls the drawing structure of the corresponding building and transmits it to the intelligent decision-making unit. Then, through the comprehensive analysis of crack data, structural status, and environmental information, a maintenance strategy or risk response plan is generated.

[0021] The present invention further provides an embodiment: a building crack monitoring method based on machine learning, comprising the following steps: S1: The lower computer is distributed and installed on the building object, regularly collects crack image data through a camera, regularly collects environmental data through a sensor, and conducts preliminary analysis and processing through a microprocessor; S2: Transmit the image data and environmental data to the machine learning model, perform data denoising and enhancement on the input data, automatically learn and extract features from the image using the model construction, and determine whether there are cracks and classify them; S3: Fuse the image data of different modalities with the environmental sensing data of the same period, capture the complementary information between modalities, and form time-series data to predict the crack propagation speed; S4: Trigger an alarm and record it when the crack propagation threshold is exceeded. At the same time, call the drawing structure of the corresponding building, and conduct a comprehensive analysis of the crack data, structural status, and environmental information to generate a maintenance strategy or risk response plan.

[0022] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and without departing from the spirit or basic characteristics of the present invention, the present invention can be implemented in other specific forms. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, it is intended to embrace all changes falling within the meaning and scope of the equivalent elements of the claims in the present invention, and any reference signs in the claims should not be regarded as limiting the claims involved.

Claims

1. A building crack monitoring system based on machine learning, comprising a lower computer, a cloud server and a host computer, characterized in that: The lower computer is a crack monitoring device, which is internally provided with a micro-viewing distance camera, a positioning sensor, an environmental sensor, a microprocessor and a wireless transmission module; An image sensor is arranged inside the micro-viewing distance camera; The cloud server is provided with a storage module; The host computer is an intelligent device with the function of sending operation instructions, and is internally provided with a machine learning model, a multimodal feature fusion and a service management module; The wireless transmission module is connected to the cloud server through a base station to realize data exchange and communication between the lower computer and the upper computer.

2. A building crack monitoring system based on machine learning according to claim 1, characterized in that: The positioning sensor is used for nanometer-level precision positioning of the lower computer through the built-in RTK algorithm of the Quectel communication module. The environmental sensor is used to monitor the environmental data of the environment in which the lower computer is located. The image sensor captures the optical signal by receiving the shooting instruction of the micro-viewing distance camera, and converts it into a digital image and transmits it to the image acquisition module. The image acquisition module decodes and processes the signal output by the image sensor and converts it into a format that can be recognized by the microprocessor. The microprocessor preliminarily collects and processes the sensor data of the environmental sensor and the positioning sensor. The microprocessor sends a configuration command to the image acquisition module to set the working parameters of the image sensor and further processes and analyzes the image parameters.

3. The building crack monitoring system based on machine learning according to claim 1 is characterized in that: When the microprocessor recognizes that the image parameters do not meet the clarity setting requirements, it issues instructions to the image sensor, sends control signals and data to adjust the working mode and parameters of the image sensor, and re-captures the digital image data until the image parameters recognized by the microprocessor meet the clarity setting requirements. The image information is then transmitted to the cloud server through the wireless transmission module. The cloud server receives the image data, environmental data and positioning data transmitted by the microprocessor, stores them in the storage module, and transmits them again to the machine learning model of the host computer.

4. The building crack monitoring system based on machine learning according to claim 1 is characterized in that: The machine learning model includes data collection and preprocessing, model construction, model training and optimization, wherein the data collection and preprocessing obtains and collects image data containing building cracks through on-site photography, web crawling, and public data sets, and annotates and preprocesses the collected images so that the image data meets the input requirements of the model, denoises and enhances the input data, and extracts and represents features of input environmental data of different modes.

5. The machine learning-based building crack monitoring system according to claim 4, characterized in that: The model construction automatically learns and extracts features from the image through the convolution layer, pooling layer and fully connected layer structure, determines whether there are cracks in the image, and classifies the cracks to build a neural network model. The model training and optimization divides the labeled crack image data set into a training set and a verification set. The training set is used for model training, and the verification set is used for model parameter adjustment and optimization.

6. The building crack monitoring system based on machine learning according to claim 1 is characterized in that: The multimodal feature fusion extracts image data of different modalities and fuses them with environmental sensor data of the same period, captures complementary information between the modalities, and forms time series data with regularly collected crack image data and environmental sensor data, predicts the crack expansion speed, and transmits the results to the service management module.

7. The building crack monitoring system based on machine learning according to claim 1 is characterized in that: The service management module receives image data of different modalities, environmental sensor data and predicted crack velocity data. The service management module includes a monitoring center, equipment management, building drawing management, and alarm management. The monitoring center sets a regional index based on the positioning sensor data to facilitate viewing the distribution area of ​​the crack monitoring equipment. The equipment management is provided with an equipment list, equipment installation and data forwarding, which are used to query the basic information, installation location, technical parameters, maintenance records and data forwarding status of the crack monitoring equipment.

8. The machine learning-based building crack monitoring system according to claim 7, characterized in that: The building drawing management includes building management and drawing management. The building management counts the monitoring points and monitoring data of all crack monitoring equipment in a building. The drawing management counts the building drawings monitored in the area. The alarm management includes monitoring alarms and equipment alarms. The equipment alarm monitors the equipment's own status and pays real-time attention to hardware failures, communication terminals, and power supply status during equipment operation. If an abnormal situation occurs, the equipment alarm information will be pushed.

9. The machine learning-based building crack monitoring system according to claim 8, characterized in that: The monitoring alarm also includes an intelligent decision-making unit, which receives predicted crack expansion rate data, triggers an alarm and records it when the crack expansion threshold is exceeded, and simultaneously calls the drawing structure of the corresponding building to transmit it to the intelligent decision-making unit, and then generates a maintenance strategy or risk response plan through a comprehensive analysis of the crack data, structural status and environmental information.

10. A method for monitoring building cracks based on machine learning according to any one of claims 1 to 9, comprising the following steps: S1: The lower computer is distributed and installed on the building object, and regularly collects crack image data through cameras, regularly collects environmental data through sensors, and performs preliminary analysis and processing through microprocessors; S2: Transmit image data and environmental data to the machine learning model, perform data denoising and enhancement on the input data, use the model to automatically learn and extract features from the image, determine whether there are cracks and classify them; S3: Fusion of image data of different modalities with environmental sensor data of the same period to capture complementary information between modalities and form time series data to predict crack propagation speed; S4: When the crack extension threshold is exceeded, an alarm is triggered and recorded. At the same time, the drawing structure of the corresponding building is called up to conduct a comprehensive analysis of the crack data, structural status and environmental information to generate a maintenance strategy or risk response plan.

Images