Building structure health monitoring and early warning system based on deep learning
Through a building structure health monitoring and early warning system based on deep learning, using drones to collect data and deep neural network analysis models, the problem of lack of comprehensive analysis capabilities and false alarms in traditional methods is solved, and a comprehensive assessment and timely warning of the health status of building structures is achieved.
Patent Information
- Application Number
- CN202411938597.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-26
- Publication Date
- 2025-05-06
AI Technical Summary
Traditional building structure health monitoring methods rely on manual regular inspections and simple threshold judgments based on sensors, making it difficult to obtain structural status information in real time, and lack the ability to analyze massive data in a comprehensive way, making false alarms or missed reports.
The building structure health monitoring and early warning system based on deep learning is adopted, including the drone acquisition module, data preprocessing module, deep learning analysis module, early warning module and interactive module. The building structure health status evaluation model is constructed through deep neural networks, and complex features are automatically learned and extracted to conduct comprehensive evaluation and early warning.
It has realized a comprehensive assessment of the health status of the building structure from multiple dimensions, judged whether there are safety hazards in the current building, and issued a timely warning, solving the problem of false alarms or missed reports caused by simple judgment and lack of comprehensive analysis capabilities.
Smart Images
Figure CN119941452A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of building engineering technology, and in particular to a building structure health monitoring and early warning system based on deep learning. Background Art
[0002] In the field of construction, as buildings become larger and more complex, they will be affected by factors such as natural environmental erosion, load, and material aging during long-term use, resulting in structural damage and performance degradation, which in turn affects the safety and service life of the building. Traditional building structure health monitoring methods mostly rely on manual regular inspections, simple threshold judgments based on sensors, and conventional data analysis methods.
[0003] Manual periodic inspections are labor-intensive, subjective, difficult to detect hidden damage, and cannot obtain structural status information in real time. Sensor monitoring methods based on simple thresholds can often only make simple judgments on a single parameter, lack the ability to comprehensively analyze massive data, and are prone to false alarms or omissions. Deep learning technology has demonstrated powerful data processing and feature extraction capabilities in multiple fields such as image recognition and speech processing. However, its application in the field of building structure health monitoring has not yet fully realized its potential. In this regard, we propose a building structure health monitoring and early warning system based on deep learning. Summary of the invention
[0004] In order to solve the above technical problems, a building structure health monitoring and early warning system based on deep learning is provided. This technical solution solves the above-mentioned simple judgment, lacks the ability to comprehensively analyze massive data, and is prone to false alarms or missed alarms.
[0005] To achieve the above objectives, the technical solution adopted by the present invention is: a building structure health monitoring and early warning system based on deep learning, including: a drone acquisition module, a data preprocessing module, a deep learning analysis module, an early warning module and an interaction module;
[0006] The drone acquisition module takes real-time photos of the exterior of the building based on the drone, sets points on the building, and regularly takes photos of the points at the same location to obtain image data of the exterior of the building and image data of the building points, and uploads the data;
[0007] The data preprocessing module preprocesses the obtained data, including data cleaning and denoising. Based on the image processing algorithm, it extracts the features in the image and extracts the cracks on the building surface and the set point features;
[0008] The deep learning analysis module constructs a building structure health status assessment model through a deep neural network. It obtains image data of building structure health in advance for training and learning, optimizes and improves the building structure health status assessment model, inputs the collected data into the model, compares the points in images taken at different times, determines whether the positions are consistent, calculates the building settlement, and combines the number of crack characteristics on the building's outer surface with the settlement to comprehensively calculate the building health value.
[0009] The early warning module constructs the safety and health threshold of the building based on the expert analysis method, and compares the assessed health value with the safety and health threshold. If it is greater than the threshold, it is determined that the current building has safety hazards, and an early warning is issued in time to alert the staff;
[0010] The interactive module performs real-time interaction based on the display screen, allowing staff to intuitively view the analysis results and process the results.
[0011] Preferably, the drone acquisition module performs flight shooting operations based on a pre-set flight route and mission planning, and sets points at various key parts of the building. The point locations include the corners of the building, the junctions of beams and columns, and areas prone to structural changes and aging. The drone regularly shoots the points and the exterior surface of the building; the drone acquisition module obtains the overall image data of the building's exterior and the image data of the building's points, and uploads the data to the storage server based on the 5G network.
[0012] Preferably, data cleaning is to identify and eliminate problematic data, and denoising is performed by using a median filtering algorithm to analyze and calculate the grayscale value of each pixel in the image and its surrounding pixels, and replace the original grayscale value of the pixel with the median value to remove salt and pepper noise in the image, making the image smooth and clear, thereby improving the quality of image data.
[0013] Preferably, feature extraction uses an edge detection algorithm to determine the edge positions where the grayscale changes dramatically in the image by calculating the gradient amplitude and direction of the image, outline the cracks on the building's exterior, and extract points in the image based on an image matching algorithm.
[0014] Preferably, the gradient amplitude calculation formula of the image is:
[0015]
[0016] Where G(x, y) is the gradient amplitude of the similarity, (x, y) is the grayscale value of the pixel at (x, y) in the image, and G x is the horizontal direction, G y is the vertical direction;
[0017] The calculation formula for image direction is:
[0018]
[0019] Where θ(x, y) is the calculation result of the image direction;
[0020] The edge position determination method of the image sets a threshold T. If the gradient amplitude G(x, y) of a pixel point ≥ T, the pixel point is considered to be an edge point, that is, the edge position where the grayscale changes sharply on the outer surface of the building is outlined and represented by a binary image. The calculation formula is:
[0021]
[0022] Where E(x, y) is a binary image.
[0023] Preferably, the deep learning analysis module pre-acquires image data for training and learning. First, the image data is annotated and converted into data samples with classification labels so that the deep neural network can understand the actual structural health status corresponding to different image data. The data samples are divided into a training set, a validation set, and a test set. The training set is substituted into the deep neural network for training. The validation set is used to verify the results. After the training process is completed, the trained building structure health status assessment model is evaluated using the test set, the model is optimized, and the collected image data is input into the model for evaluation.
[0024] Preferably, the calculation formula for the number of crack features in the deep learning analysis module is:
[0025]
[0026] Where N is the number of calculated comprehensive crack features, L1 is the length of cracks observable on the building exterior surface in the image, L2 is the minimum statistical crack length threshold of cracks in the image, D is the density coefficient of cracks, which is determined based on historical empirical data, and A represents the total area of the building exterior surface monitored in the image;
[0027] The formula for calculating building settlement is:
[0028]
[0029] Among them, h0 is the current settlement of the building, h1 is the settlement of the point in the image taken by the second drone, and h2 is the settlement of the point in the image taken by the first drone. The current settlement of the building is obtained by calculation.
[0030] Preferably, the building health value calculation formula is:
[0031] B=N*W1+h0*W2
[0032] Where B is the comprehensively calculated building health value, and W1 and W2 are weight coefficients.
[0033] Preferably, the early warning module uses engineering theoretical knowledge and past practical case experience to gradually construct a safety and health threshold suitable for the building through discussion, demonstration and scientific calculation and evaluation, and compares the calculated building health value B with the safety and health threshold to determine whether the building has safety hazards. When it is determined that the current building has safety hazards, an early warning message is issued to alert relevant staff.
[0034] Preferably, when the interactive module analyzes the analysis results, the results are presented in real time on the display screen. On the main interface of the display screen, the trend chart of the building health value changing over time is displayed in the form of an intuitive chart. The horizontal axis is the time axis, marking different monitoring time nodes, and the vertical axis is the quantitative value of the building health value for intuitive display.
[0035] Compared with the prior art, the present invention has the following beneficial effects:
[0036] The present invention constructs a building structure health status assessment model through a deep neural network, which can automatically learn and extract complex and deep features from a large amount of image data of building structure health, and continuously optimize and improve the assessment model, and conduct a comprehensive and efficient assessment of the building structure health status from multiple dimensions and multiple key indicators, and determine whether there are safety hazards in the current building. If it is determined that the building has safety hazards, the early warning module can issue an early warning in time and alert the staff in various ways, thus solving the problem of false alarms or missed alarms caused by simple judgments and lack of comprehensive analysis capabilities for massive data. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Figure 1 Schematic diagram of the system framework of the present invention. DETAILED DESCRIPTION
[0038] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art may think of other obvious variations.
[0039] Reference Figure 1 As shown, the building structure health monitoring and early warning system based on deep learning includes: drone collection module, data preprocessing module, deep learning analysis module, early warning module and interaction module;
[0040] The drone acquisition module takes real-time photos of the exterior of the building based on the drone, sets points on the building, and regularly takes photos of the points at the same location to obtain image data of the exterior of the building and image data of the building points, and uploads the data;
[0041] The data preprocessing module preprocesses the obtained data, including data cleaning and denoising. Based on the image processing algorithm, it extracts the features in the image and extracts the cracks on the building surface and the set point features;
[0042] The deep learning analysis module constructs a building structure health status assessment model through a deep neural network. It obtains image data of building structure health in advance for training and learning, optimizes and improves the building structure health status assessment model, inputs the collected data into the model, compares the points in images taken at different times, determines whether the positions are consistent, calculates the building settlement, and combines the number of crack characteristics on the building's outer surface with the settlement to comprehensively calculate the building health value.
[0043] The early warning module constructs the safety and health threshold of the building based on the expert analysis method, and compares the assessed health value with the safety and health threshold. If it is greater than the threshold, it is determined that the current building has safety hazards, and an early warning is issued in time to alert the staff;
[0044] The interactive module performs real-time interaction based on the display screen, allowing staff to intuitively view the analysis results and process the results.
[0045] This application is based on the use of drones to photograph the exterior of buildings, which can break through the limitations of traditional manual collection methods in terms of height, angle and accessibility. Drones can easily fly to high places, hard-to-reach corners or complex structural areas of buildings for photography, ensuring full coverage of all facades of the building, and efficiently obtaining image data of the building's exterior and building point positions, greatly improving the integrity and efficiency of data collection. There is no need to build complex scaffolding or use large-scale climbing equipment, saving manpower, material resources and time costs.
[0046] Set points on the building and take photos of them regularly at the same location, so that the collected data has good consistency and comparability;
[0047] The building structure health status assessment model is constructed with the help of deep neural networks, which fully utilizes the powerful data mining and feature learning capabilities of deep learning. It can automatically extract complex and deep features and patterns from massive image data, and mine the health-related features of building structures that are difficult to detect manually, thereby more comprehensively and accurately describing the true status of the building structure.
[0048] At the same time, the points in the images taken at different times are compared, the building settlement is calculated, and the building health value is calculated comprehensively based on multiple key factors. This enables a comprehensive assessment of the health status of the building structure from multiple dimensions. The factors are interrelated and mutually confirmed, avoiding the one-sidedness of single-dimensional judgment.
[0049] The drone acquisition module conducts flight photography based on pre-set flight routes and mission planning, and sets points at key locations of the building, including the corners of the building, the junctions of beams and columns, and areas prone to structural changes and aging. The drone regularly photographs the points and the exterior surface of the building; the drone acquisition module obtains the overall image data of the building's exterior and the image data of the building's points, and uploads the data to the storage server based on the 5G network.
[0050] This application conducts flight photography operations through pre-set flight routes and mission planning, which can ensure that the drone covers all facades of the building in an orderly and comprehensive manner; the points are set at the corners of the building, the junctions of beams and columns, and areas prone to structural changes and aging. These parts are the key nodes and weak links in the building structure, and often best reflect the overall structural health of the building; regular photography is carried out specifically for these key points, so that the collected data is more targeted, and can highlight the actual situation of the most critical and most prone to problems in the building structure.
[0051] Data cleaning is to identify and remove problematic data. Denoising uses a median filtering algorithm to analyze and calculate the grayscale value of each pixel in the image and its surrounding pixels, and replaces the original grayscale value of the pixel with the median value to remove salt and pepper noise in the image, making the image smooth and clear, thereby improving the quality of image data.
[0052] The data cleaning of this application can accurately identify and eliminate these "problem data", such as eliminating the missing parts of the image caused by the sudden interruption of the signal during drone collection, the severely distorted images caused by abnormal light, and the data with garbled or wrong format due to equipment hardware failure. The median filtering algorithm analyzes the grayscale value of each pixel and its surrounding pixels, and replaces the original pixel grayscale value with the median value. It can specifically remove this salt and pepper noise, making the image smooth and clear, and the important features of the boundaries of cracks, pits, and key points on the exterior surface of the building that were originally covered by noise can be clearly presented.
[0053] Feature extraction uses an edge detection algorithm to determine the edge positions where the grayscale changes dramatically in the image by calculating the gradient amplitude and direction of the image, outlining the cracks on the building's exterior, and extracting points in the image based on an image matching algorithm.
[0054] The edge detection algorithm of the present application can keenly capture the areas with drastic grayscale changes in the image by calculating the gradient amplitude and direction of the image, and these areas are often where the cracks are located. In this way, the specific coordinate position of the crack on the outer surface of the building can be accurately located. Whether the crack is distributed horizontally, vertically or diagonally, it can be clearly outlined on the image.
[0055] The gradient amplitude calculation formula of the image is:
[0056]
[0057] Where G(x, y) is the gradient amplitude of the similarity, (x, y) is the grayscale value of the pixel at (x, y) in the image, and G x is the horizontal direction, G y is the vertical direction;
[0058] The calculation formula for image direction is:
[0059]
[0060] Where θ(x, y) is the calculation result of the image direction;
[0061] The edge position determination method of the image sets a threshold T. If the gradient amplitude G(x, y) of a pixel point ≥ T, the pixel point is considered to be an edge point, that is, the edge position where the grayscale changes sharply on the outer surface of the building is outlined and represented by a binary image. The calculation formula is:
[0062]
[0063] Where E(x, y) is a binary image.
[0064] The deep learning analysis module acquires image data in advance for training and learning. First, the image data is annotated and converted into data samples with classification labels, so that the deep neural network can understand the actual structural health status corresponding to different image data. The data samples are divided into training set, validation set and test set, and the training set is substituted into the deep neural network for training. The validation set is used to verify the results. After the training process is completed, the trained building structure health status assessment model is evaluated using the test set, the model is optimized, and the collected image data is input into the model for evaluation.
[0065] This application annotates the pre-acquired image data and assigns a corresponding structural health status classification label to each image based on professional knowledge in the construction field and actual structural health standards. This can establish a clear connection between the data and the actual structural health of the building. After completing the training, verification and optimization of the model, the actual collected image data is input into the model. The model can then use the previously learned knowledge and rules to quickly and automatically evaluate the structural health status of the building.
[0066] The calculation formula for the number of crack features in the deep learning analysis module is:
[0067]
[0068] Where N is the number of calculated comprehensive crack features, L1 is the length of cracks observable on the building exterior surface in the image, L2 is the minimum statistical crack length threshold of cracks in the image, D is the density coefficient of cracks, which is determined based on historical empirical data, and A represents the total area of the building exterior surface monitored in the image;
[0069] The formula for calculating building settlement is:
[0070]
[0071] Among them, h0 is the current settlement of the building, h1 is the settlement of the point in the image taken by the second drone, and h2 is the settlement of the point in the image taken by the first drone. The current settlement of the building is obtained by calculation.
[0072] The comprehensive consideration approach of this application avoids the limitation of relying on a single factor to judge the crack condition of the building's exterior surface, and can quantitatively evaluate the crack condition of the building's exterior surface from multiple dimensions. Only focusing on the crack length may ignore the impact of some shorter but densely distributed cracks on the health of the building structure. The formula introduces density coefficient and area factors to more comprehensively reflect the actual crack distribution characteristics, so that the final number of crack characteristics can more accurately reflect the structural health of the building's exterior surface in this regard. The images taken by drones are used to obtain point settlement, giving full play to the convenience of drone data collection and the ease of processing of image data.
[0073] The building health value calculation formula is:
[0074] B=N*W1+h0*W2
[0075] Where B is the comprehensively calculated building health value, and W1 and W2 are weight coefficients.
[0076] The early warning module uses engineering theory knowledge and past practical case experience to gradually build a safety and health threshold suitable for the building through discussion, demonstration and scientific calculation and evaluation. The calculated building health value B is compared with the safety and health threshold to determine whether the building has safety hazards. When it is determined that the current building has safety hazards, an early warning message is issued to alert relevant staff.
[0077] This application compares the calculated building health value with the safety and health threshold, which is a clear and quantitative judgment method. When the health value exceeds the pre-set threshold range, it can accurately determine that the current building has safety hazards, avoiding the ambiguity and uncertainty that may arise from relying on human subjective observation and experience judgment. The early warning module can not only issue an alarm in time when a safety hazard occurs, but also help staff discover the changing trend of the building's health status in advance and achieve preventive maintenance.
[0078] When the interactive module analyzes the results, it presents the results in real time on the display screen. On the main interface of the display screen, the trend of the building health value changing over time is displayed in the form of an intuitive chart. The horizontal axis is the time axis, marking different monitoring time nodes, and the vertical axis is the quantitative value of the building health value for intuitive display.
[0079] This application presents the analysis results on the display screen in real time, which means that the staff can obtain the latest information on the health status of the building structure at the first time. During the use of the building, the structural health status may change at any time. Whether it is potential safety hazards caused by natural factors or human factors, they can be quickly known to the staff through real-time display. This timeliness allows the staff to take corresponding measures when the problem first appears, avoid further expansion of safety hazards, maximize the safety of the building and the people and property therein, and realize real-time control of the building's safety status.
[0080] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments, and the above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention, and these changes and improvements fall within the scope of the present invention claimed.
Claims
1. Building structure health monitoring and early warning system based on deep learning, characterized by: include: UAV acquisition module, data preprocessing module, deep learning analysis module, early warning module and interaction module; The drone acquisition module takes real-time photos of the exterior of the building based on the drone, sets points on the building, and regularly takes photos of the points at the same location to obtain image data of the exterior of the building and image data of the building points, and uploads the data; The data preprocessing module preprocesses the obtained data, including data cleaning and denoising. Based on the image processing algorithm, it extracts the features in the image and extracts the cracks on the building surface and the set point features; The deep learning analysis module constructs a building structure health status assessment model through a deep neural network. It obtains image data of building structure health in advance for training and learning, optimizes and improves the building structure health status assessment model, inputs the collected data into the model, compares the points in images taken at different times, determines whether the positions are consistent, calculates the building settlement, and combines the number of crack characteristics on the building's outer surface with the settlement to comprehensively calculate the building health value. The early warning module constructs the safety and health threshold of the building based on the expert analysis method, and compares the assessed health value with the safety and health threshold. If it is greater than the threshold, it is determined that the current building has safety hazards, and an early warning is issued in time to alert the staff; The interactive module performs real-time interaction based on the display screen, allowing staff to intuitively view the analysis results and process the results.
2. The building structure health monitoring and early warning system based on deep learning according to claim 1 is characterized in that: The drone acquisition module conducts flight photography based on pre-set flight routes and mission planning, and sets points at key locations of the building, including the corners of the building, the junctions of beams and columns, and areas prone to structural changes and aging. The drone regularly photographs the points and the exterior surface of the building; the drone acquisition module obtains the overall image data of the building's exterior and the image data of the building's points, and uploads the data to the storage server based on the 5G network.
3. The building structure health monitoring and early warning system based on deep learning according to claim 1 is characterized in that: Data cleaning is to identify and remove problematic data. Denoising uses a median filtering algorithm to analyze and calculate the grayscale value of each pixel in the image and its surrounding pixels, and replaces the original grayscale value of the pixel with the median value to remove salt and pepper noise in the image, making the image smooth and clear, thereby improving the quality of image data.
4. The building structure health monitoring and early warning system based on deep learning according to claim 1 is characterized in that: Feature extraction uses an edge detection algorithm to determine the edge positions where the grayscale changes dramatically in the image by calculating the gradient amplitude and direction of the image, outlining the cracks on the building's exterior, and extracting points in the image based on an image matching algorithm.
5. The building structure health monitoring and early warning system based on deep learning according to claim 4 is characterized in that: The gradient amplitude calculation formula of the image is: Where G(x, y) is the gradient amplitude of the similarity, (x, y) is the grayscale value of the pixel at (x, y) in the image, and G x is the horizontal direction, G y is the vertical direction; The calculation formula for image direction is: Where θ(x, y) is the calculation result of the image direction; The edge position determination method of the image sets a threshold T. If the gradient amplitude G(x, y) of a pixel point ≥ T, the pixel point is considered to be an edge point, that is, the edge position where the grayscale changes sharply on the outer surface of the building is outlined and represented by a binary image. The calculation formula is: Where E(x, y) is a binary image.
6. The building structure health monitoring and early warning system based on deep learning according to claim 1 is characterized in that: The deep learning analysis module acquires image data in advance for training and learning. First, the image data is annotated and converted into data samples with classification labels, so that the deep neural network can understand the actual structural health status corresponding to different image data. The data samples are divided into training set, validation set and test set, and the training set is substituted into the deep neural network for training. The validation set is used to verify the results. After the training process is completed, the trained building structure health status assessment model is evaluated using the test set, the model is optimized, and the collected image data is input into the model for evaluation.
7. The building structure health monitoring and early warning system based on deep learning according to claim 1 is characterized in that: The calculation formula for the number of crack features in the deep learning analysis module is: Where N is the number of calculated comprehensive crack features, L1 is the length of cracks observable on the building exterior surface in the image, L2 is the minimum statistical crack length threshold of cracks in the image, D is the density coefficient of cracks, which is determined based on historical empirical data, and A represents the total area of the building exterior surface monitored in the image; The formula for calculating building settlement is: Among them, h0 is the current settlement of the building, h1 is the settlement of the point in the image taken by the second drone, and h2 is the settlement of the point in the image taken by the first drone. The current settlement of the building is obtained by calculation.
8. The building structure health monitoring and early warning system based on deep learning according to claim 7 is characterized in that: The building health value calculation formula is: B=N*W1+h0*W2 Where B is the comprehensively calculated building health value, and W1 and W2 are weight coefficients.
9. The building structure health monitoring and early warning system based on deep learning according to claim 1 is characterized in that: The early warning module uses engineering theory knowledge and past practical case experience to gradually build a safety and health threshold suitable for the building through discussion, demonstration and scientific calculation and evaluation. The calculated building health value B is compared with the safety and health threshold to determine whether the building has safety hazards. When it is determined that the current building has safety hazards, an early warning message is issued to alert relevant staff.
10. The building structure health monitoring and early warning system based on deep learning according to claim 1, characterized in that: When the interactive module analyzes the results, it presents the results in real time on the display screen. On the main interface of the display screen, the trend of the building health value changing over time is displayed in the form of an intuitive chart. The horizontal axis is the time axis, marking different monitoring time nodes, and the vertical axis is the quantitative value of the building health value for intuitive display.