A BIM-based steel structure deformation monitoring and processing method and system

Abstract

The present invention discloses a method and system for monitoring and processing steel structure deformation based on BIM, which is used in the field of steel structure safety monitoring. The method includes the following steps: obtaining a digital image obtained by scanning a steel structure with a laser scanner, and establishing a BIM model based on the digital image; using the GrabCut algorithm to process the digital image to detect the surface deformation of the steel structure and obtain the deformed area; using three-dimensional reconstruction technology to obtain the three-dimensional information data of the deformed area; comparing the three-dimensional information data of the deformed area with the three-dimensional information data in the BIM model to detect the precise position and deformation degree of the steel structure deformation; using an improved CNN model to evaluate the steel structure deformation based on the three-dimensional information; and formulating a steel structure deformation treatment plan. Through machine learning and deep learning technologies, the present invention can automatically detect deformations in digital images and three-dimensional information, reducing the workload of manual detection.

Description

[0001] This application is a divisional application of the application filed on October 7, 2023, with the application number 202311278790.5 and the invention title "A Method and System for Monitoring and Processing Steel Structure Deformation Based on BIM". Technical Field

[0002] The present invention relates to the field of steel structure safety monitoring. Specifically, it particularly relates to a method and system for monitoring and processing steel structure deformation based on BIM. Background Art

[0003] Steel structure projects have been widely adopted in engineering construction scenarios worldwide due to a series of advantages such as light weight, strong seismic resistance, fast construction speed, and environmental friendliness. Especially in recent years, the development of light steel structures has been even more rapid, becoming an important trend. However, the research on the detection and monitoring of various performance indicators affecting the stability of steel structure projects in our country is still in its infancy, and many related studies have not yet started.

[0004] BIM is short for Building Information Modeling. It is a dynamic building design and management process that can create and manage the physical and functional information of buildings and infrastructure. A BIM model is a highly accurate digital representation that can be used for design decision-making, construction planning, performance prediction, and facility operation throughout the building life cycle.

[0005] Steel structure deformation monitoring refers to the real-time or regular detection of whether the steel structure is deformed, the degree of deformation, the location of deformation, etc. The purpose of this monitoring is to ensure the integrity and safety of the structure and timely discover and solve potential problems.

[0006] Most of the existing methods rely on manual visual inspection, which requires a large amount of labor and time, with low efficiency. Moreover, the results of manual inspection may be affected by subjective factors, making it difficult to guarantee accuracy. Additionally, manual inspection is difficult to achieve real-time monitoring of steel structures, which may lead to missed detection of some potential hidden dangers and affect the safety of the structure. For some complex deformation situations, existing methods may be difficult to provide satisfactory judgment and treatment solutions, and a large amount of labor and time need to be invested in research.

[0007] Regarding the problems in the related art, no effective solutions have been proposed yet. Summary of the Invention

[0008] In order to overcome the above problems, the present invention aims to propose a method and system for monitoring and processing steel structure deformation based on BIM, aiming to solve the problem that the results of manual inspection may be affected by subjective factors and it is difficult to guarantee accuracy.

[0009] To this end, the specific technical solution adopted by the present invention is as follows:

[0010] According to one aspect of the present invention, a method for monitoring and processing the deformation of a steel structure based on BIM is provided. The method for monitoring and processing the deformation of the steel structure includes the following steps:

[0011] S1. Obtain the digital image obtained by scanning the steel structure with a laser scanner, and establish a BIM model based on the digital image;

[0012] S2. Process the digital image using the GrabCut algorithm, and analyze the processed digital image to detect the surface deformation of the steel structure and obtain the deformed area;

[0013] S3. Use three-dimensional reconstruction technology to obtain the three-dimensional information data of the deformed area;

[0014] S4. Compare the three-dimensional information data of the deformed area with the three-dimensional information data in the BIM model to detect the exact position and deformation degree of the steel structure deformation;

[0015] S5. According to the steel structure detection results, use the improved CNN model and evaluate the steel structure deformation based on the three-dimensional information to judge the actual deformation situation;

[0016] S6. Obtain the evaluation result of the deformation situation and formulate a steel structure deformation treatment plan.

[0017] Optionally, the step of processing the digital image using the GrabCut algorithm, analyzing the processed digital image to detect the surface deformation of the steel structure, and obtaining the deformed area includes the following steps:

[0018] S21. Label the input digital image, label the normal area as background pixels, and label the suspected deformed area as foreground pixels;

[0019] S22. Construct a color statistical model according to the labeled foreground pixels and background pixels;

[0020] S23. Classify the unlabeled area pixels using the color statistical model and judge the unlabeled area pixels;

[0021] S24. Use the image boundary information to identify the deformed area boundary;

[0022] S25. Repeat the steps of S23-S24, and optimize the color statistical model and the deformed area boundary;

[0023] S26. Evaluate the segmentation result of the boundary of the deformation region, re-label the inaccurate regions, and repeat the steps of S21 - S25.

[0024] Optionally, the classification of the unlabeled region pixels using the color statistical model and the judgment of the unlabeled region pixels include the following steps:

[0025] S231. Obtain the unlabeled region according to the already labeled foreground pixels and background pixels;

[0026] S232. Extract the pixel colors of the unlabeled region, input the pixel colors of the unlabeled region into the color statistical model, and calculate the probabilities of the pixel colors of the unlabeled region being foreground and background;

[0027] S233. According to the foreground and background probabilities of each pixel, if the probability of the foreground is higher than that of the background, the pixel is classified as foreground, otherwise it is classified as background, and pixel classification is performed;

[0028] S234. Obtain the classification result and generate the initial image segmentation result.

[0029] Optionally, the identification of the deformation region boundary using the image boundary information includes the following steps:

[0030] S241. Use a Gaussian filter to eliminate the noise in the image;

[0031] S242. Calculate the gradient intensity and direction of the image, and find the candidate points of the boundary;

[0032] S243. Determine the true edge information through the double - threshold algorithm and non - maximum suppression;

[0033] S244. Use the Hough transform to perform shape analysis on the edge information and extract geometric features;

[0034] S245. Use statistical methods to compare the differences in boundary features between the deformation region and the normal region;

[0035] S246. According to the comparison result, set a threshold, and mark the regions with boundary feature differences greater than the threshold as deformation regions;

[0036] Among them, the determination of the true edge information through the double - threshold algorithm and non - maximum suppression includes the following steps:

[0037] S2431. Use the Sobel operator to calculate the gradient values of each pixel point in the horizontal and vertical directions respectively;

[0038] S2432. Calculate the gradient intensity of each pixel point according to the gradient values in the horizontal and vertical directions;

[0039] S2433. Retain the pixel points with the maximum gradient intensity in the same direction, and regard other pixel points as non-edges and suppress them;

[0040] S2434. Set a high threshold and a low threshold. Among them, the pixel points with gradient intensity greater than the high threshold are confirmed as strong edges, the pixel points with gradient intensity less than the low threshold are confirmed as non-edges, and the pixel points between the high threshold and the low threshold are regarded as weak edges;

[0041] S2435. If there are strong edge pixel points in the neighborhood of the weak edge pixel points, then the weak edge is also confirmed as an edge, otherwise it is confirmed as a non-edge.

[0042] Optionally, the obtaining of the three-dimensional information data of the deformed area by using the three-dimensional reconstruction technology includes the following steps:

[0043] S31. Obtain the digital image of the deformed area;

[0044] S32. If there are digital images from multiple perspectives, then register the digital images and establish the corresponding relationship between the feature points in the digital images;

[0045] S33. Judge whether there are images from multiple perspectives. If so, select the multi-view stereo algorithm. If not, select the monocular stereo algorithm;

[0046] S34. Select a three-dimensional reconstruction algorithm according to the judgment result to obtain the three-dimensional information of the deformed area;

[0047] S35. Process the three-dimensional information, and the processing at least includes three-dimensional information completion, smoothing and noise reduction.

[0048] Optionally, the comparison of the three-dimensional information data of the deformed area with the three-dimensional information data in the BIM model to detect the exact position and deformation degree of the steel structure deformation includes the following steps:

[0049] S41. Obtain the three-dimensional information of the deformed area;

[0050] S42. Use the ICP algorithm to register the three-dimensional information data of the deformed area with the three-dimensional information data of the BIM model and map them to a unified coordinate system;

[0051] S43. Use the morphological difference method in computer vision to calculate the shape difference between the three-dimensional information data of the deformed area and the three-dimensional information data of the BIM model, and judge whether there is a large morphological difference;

[0052] S44. According to the judgment result of the morphological difference, determine the position of the deformation and evaluate the degree of the deformation;

[0053] S45. Record the position and degree of deformation, and provide a visual display interface for information input and output.

[0054] Optionally, the step of registering the three-dimensional information data of the deformation area with the three-dimensional information data of the BIM model using the ICP algorithm and mapping them into a unified coordinate system includes the following steps:

[0055] S421. Set initial transformation parameters;

[0056] S422. Find the nearest neighbor points in the three-dimensional information data of the BIM model for each point in the three-dimensional information data of the deformation area to form point pairs;

[0057] S423. Calculate the distance between the point pairs of the nearest neighbor points, and find a transformation that minimizes the sum of the squared errors of this distance;

[0058] S424. Update the current transformation parameters using the obtained transformation;

[0059] S425. Repeat the steps of S422 - S424 until the change in the transformation parameters is less than a certain threshold or the number of iterations exceeds the set maximum number of times;

[0060] S426. Map the three-dimensional information data of the deformation area into the coordinate system of the BIM model using the obtained transformation parameters.

[0061] Optionally, the step of evaluating the steel structure deformation based on the steel structure detection results using an improved CNN model and judging the actual deformation situation based on three-dimensional information includes the following steps:

[0062] S51: Obtain the three-dimensional information data containing the steel structure deformation, and process the three-dimensional information data, including denoising, normalization, and scaling;

[0063] S52: Construct a pre-trained improved CNN model;

[0064] S53: Use a dataset containing known deformation situations to train the pre-trained improved CNN model;

[0065] S54: Input the pre-processed three-dimensional information data into the trained improved CNN model, and the model will output the predicted deformation information;

[0066] S55: Analyze the results predicted by the trained improved CNN model to determine the actual deformation situation of the steel structure. The deformation situation includes at least the size, position, and shape of the deformation, and feedback the analysis results of the deformation situation.

[0067] Optionally, obtaining the evaluation result of the deformation condition and formulating a steel structure deformation treatment plan includes the following steps:

[0068] S61. Obtain the evaluation result of the deformation condition. If the steel structure has slight deformation, carry out reinforcement treatment;

[0069] S62. If the steel structure has severe deformation, carry out demolition and reconstruction or replacement;

[0070] S63. If the steel structure has complex deformation during severe deformation, judge the complex deformation according to the knowledge graph and optimize the construction treatment plan;

[0071] The judgment of the complex deformation according to the knowledge graph includes the following steps:

[0072] Construct a knowledge graph containing steel structure deformation knowledge;

[0073] Input the deformation evaluation result of the complex deformation into the knowledge graph, and search for known deformation cases similar to the input data in the knowledge graph;

[0074] According to the found similar cases, propose a plan and suggestions for dealing with the input deformation condition;

[0075] Based on the input deformation data and professional knowledge, optimize and adjust the construction treatment plan.

[0076] According to another aspect of the present invention, a BIM-based steel structure deformation monitoring and processing system is provided. The system includes: a data acquisition and model construction module, an image processing module, a deformation detection module, a deformation positioning module, a deformation evaluation module, and a treatment plan formulation module;

[0077] The data acquisition and model construction module is connected to the deformation detection module through the image processing module. The deformation detection module is connected to the deformation evaluation module through the deformation positioning module. The deformation evaluation module is connected to the treatment plan formulation module;

[0078] The data acquisition and model construction module is used to obtain the digital image obtained by scanning the steel structure with a laser scanner and establish a BIM model based on the digital image;

[0079] The image processing module is used to process the digital image by using the GrabCut algorithm, analyze the processed digital image, detect the surface deformation condition of the steel structure, and obtain the deformation area;

[0080] The deformation detection module is used to obtain the three-dimensional information data of the deformation area by using three-dimensional reconstruction technology;

[0081] The deformation positioning module is used to compare the three-dimensional information data of the deformed area with the three-dimensional information data in the BIM model to detect the exact position and deformation degree of the steel structure deformation;

[0082] The deformation evaluation module is used to evaluate the steel structure deformation based on the three-dimensional information by using an improved CNN model according to the steel structure detection results and judge the actual deformation situation;

[0083] The processing scheme formulation module is used to obtain the evaluation result of the deformation situation and formulate a processing scheme for the steel structure deformation.

[0084] Compared with the prior art, the present application has the following beneficial effects:

[0085] 1. By regularly scanning the steel structure with a laser scanner, the present invention can monitor the deformation of the steel structure in real time, providing a basis for the timely discovery and treatment of deformation; by comparing with the BIM model, the position of the deformation can be accurately located, which provides accurate information for subsequent maintenance and reinforcement; through machine learning and deep learning technologies, the deformation in digital images and three-dimensional information can be automatically detected, reducing the workload of manual detection; through the machine learning model, the severity of the deformation can be automatically evaluated, providing a reference for the selection of the maintenance plan.

[0086] 2. By using the knowledge graph technology, the present invention can automatically judge complex deformation situations according to historical cases and provide recommendations for processing schemes, improving work efficiency; the knowledge graph technology can provide recommendations for deformation processing schemes according to historical cases, but the final scheme formulation still needs to be optimized in combination with professional knowledge, so that artificial intelligence and artificial expert knowledge can be comprehensively utilized to achieve better results; through the BIM model and the knowledge graph, the informatization management of the whole life cycle of the steel structure can be realized, providing information support for future operation and maintenance.

[0087] 3. By real-time monitoring and accurately positioning the deformation, the present invention can timely discover and handle potential safety hazards, reducing the risks brought by the damage of the steel structure. Because accurate information for the maintenance work can be provided according to the severity and position of the deformation, the quality and efficiency of the maintenance work are improved. Through automated detection and evaluation, the workload of manual detection is reduced, saving a large amount of manpower and material resources, thus reducing the overall operation cost.

[0088] 4. Timely detection and maintenance can extend the service life of the steel structure and improve its economic benefits. Informatization management enables the data of the whole life cycle of the steel structure to be saved and utilized, which is conducive to energy conservation, emission reduction and the construction of green buildings. Brief Description of the Drawings

[0089] With the following description of the embodiments, the above-mentioned characteristics, features, and advantages of the present invention, as well as the implementation manners and methods thereof, become more understandable. The embodiments are described in detail in conjunction with the accompanying drawings. Shown herein in schematic diagrams:

[0090] Figure 1 is a flowchart of a BIM-based steel structure deformation monitoring and processing method according to an embodiment of the present invention;

[0091] Figure 2 is a schematic block diagram of a BIM-based steel structure deformation monitoring and processing system according to an embodiment of the present invention.

[0092] In the figure:

[0093] 1. Data acquisition and model construction module; 2. Image processing module; 3. Deformation detection module; 4. Deformation positioning module; 5. Deformation evaluation module; 6. Processing plan formulation module. Specific implementation manners

[0094] In order to enable those skilled in the art of this technology to better understand the solutions of this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this application.

[0095] According to an embodiment of the present invention, a BIM-based steel structure deformation monitoring and processing method and system are provided.

[0096] Now, the present invention will be further described in conjunction with the accompanying drawings and specific implementation manners. As Figure 1 shown, according to an embodiment of the present invention, a BIM-based steel structure deformation monitoring and processing method is provided. The steel structure deformation monitoring and processing method includes the following steps:

[0097] S1. Obtain the digital image obtained by scanning the steel structure with a laser scanner, and establish a BIM model based on the digital image.

[0098] It should be explained that laser scanning is a non-contact and high-precision measurement technology that can quickly capture the shape and appearance of the target object and generate high-precision 3D data. In the inspection of steel structures, a laser scanner can scan the steel structure comprehensively and without dead angles to obtain accurate digital images.

[0099] BIM (Building Information Modeling) is a model-based design and management method in which all relevant building design information is embedded in a 3D model. Based on the acquired digital images, the BIM model can accurately reflect the actual state of the steel structure.

[0100] First, use a laser scanner to scan the steel structure, and the obtained digital images reflect the actual state of the steel structure. Then, based on these digital images, establish a BIM model of the steel structure. This model contains detailed information about the steel structure, including shape, size, material properties, etc.

[0101] S2. Use the GrabCut algorithm to process the digital images, analyze the processed digital images, detect the surface deformation of the steel structure, and obtain the deformed area.

[0102] Preferably, the step of using the GrabCut algorithm to process the digital images, analyze the processed digital images, detect the surface deformation of the steel structure, and obtain the deformed area includes the following steps:

[0103] S21. Label the input digital images, label the normal area as background pixels, and label the suspected deformed area as foreground pixels;

[0104] S22. Construct a color statistical model based on the labeled foreground pixels and background pixels;

[0105] S23. Use the color statistical model to classify the unlabeled area pixels and judge the unlabeled area pixels;

[0106] S24. Use the image boundary information to identify the deformed area boundary;

[0107] S25. Repeat the steps of S23 - S24, and optimize the color statistical model and the deformed area boundary;

[0108] S26. Evaluate the segmentation result of the deformed area boundary, re-label the inaccurate area, and repeat the steps of S21 - S25.

[0109] Preferably, the step of using the color statistical model to classify the unlabeled area pixels and judge the unlabeled area pixels includes the following steps:

[0110] S231. Obtain the unlabeled area according to the already labeled foreground pixels and background pixels;

[0111] S232. Extract the pixel colors of the unlabeled area, input the pixel colors of the unlabeled area into the color statistical model, and calculate the probabilities of the pixel colors of the unlabeled area being foreground and background.

[0112] S233. According to the foreground and background probabilities of each pixel, if the probability of the foreground is higher than that of the background, the pixel is classified as the foreground; otherwise, it is classified as the background, and pixel classification is performed.

[0113] S234. Obtain the classification result and generate an initial image segmentation result.

[0114] It should be noted that the extension of calculating the probabilities of the pixel colors in the unlabeled area as foreground and background is as follows:

[0115] For the entire image, assign a probability of belonging to the foreground or background to each pixel. Generally, a high probability can be assigned to the foreground area, a low probability to the background area, and a probability of 0.5 to the unlabeled area. Based on the foreground and background sample points labeled by the user, use the Gaussian mixture model (GMM) to establish the color models of the foreground and background respectively. For each unlabeled pixel, calculate its probabilities using the foreground and background GMM models, that is, the probabilities of belonging to the foreground or background. Construct a graph model to link the probabilities of adjacent pixels, and iteratively optimize the foreground and background probabilities of all pixels to obtain the overall optimal segmentation result, and update the foreground and background GMM models with the optimized result.

[0116] Preferably, the identifying the boundary of the deformed area by using the image boundary information includes the following steps:

[0117] S241. Use a Gaussian filter to eliminate the noise in the image.

[0118] S242. Calculate the gradient intensity and direction of the image to find the candidate points of the boundary.

[0119] S243. Determine the true edge information through a double-threshold algorithm and non-maximum suppression.

[0120] S244. Use the Hough transform to perform shape analysis on the edge information and extract geometric features.

[0121] S245. Use statistical methods to compare the differences in the boundary features between the deformed area and the normal area.

[0122] S246. According to the comparison result, set a threshold, and mark the area where the boundary feature difference is greater than the threshold as the deformed area.

[0123] Among them, the determining the true edge information through a double-threshold algorithm and non-maximum suppression includes the following steps:

[0124] S2431. Use the Sobel operator to calculate the gradient values of each pixel point in the horizontal and vertical directions respectively.

[0125] S2432. Calculate the gradient intensity of each pixel based on the gradient values in the horizontal and vertical directions;

[0126] S2433. In the same direction, retain the pixel with the maximum gradient intensity, and consider other pixels as non-edges and suppress them;

[0127] S2434. Set a high threshold and a low threshold. Among them, pixels with gradient intensity greater than the high threshold are identified as strong edges, pixels with gradient intensity less than the low threshold are identified as non-edges, and pixels with gradient intensity between the high threshold and the low threshold are considered weak edges;

[0128] S2435. If there are strong edge pixels in the neighborhood of a weak edge pixel, then the weak edge is also identified as an edge; otherwise, it is identified as a non-edge.

[0129] It should be noted that the boundary of the deformed area can be identified using edge detection algorithms, such as the Canny operator combined with the Hough transform. Canny includes steps such as Gaussian filtering, calculating the image gradient, double-threshold detection, and non-maximum suppression. It can effectively extract the image edges. The Hough transform can detect the boundaries of various shapes. In addition, the GrabCut algorithm combines edge detection and iterative optimization, which can effectively segment digital images and detect the deformed area of the steel structure, providing a basis for subsequent deformation evaluation and analysis.

[0130] S3. Use three-dimensional reconstruction technology to obtain the three-dimensional information data of the deformed area.

[0131] Preferably, the obtaining of the three-dimensional information data of the deformed area using three-dimensional reconstruction technology includes the following steps:

[0132] S31. Obtain the digital image of the deformed area;

[0133] S32. If there are digital images from multiple perspectives, register the digital images and establish the corresponding relationship between the feature points in the digital images;

[0134] S33. Determine whether there are images from multiple perspectives. If so, select the multi-view stereo algorithm; if not, select the monocular stereo algorithm;

[0135] S34. Select a three-dimensional reconstruction algorithm according to the judgment result to obtain the three-dimensional information of the deformed area;

[0136] S35. Process the three-dimensional information, and the processing includes at least three-dimensional information completion, smoothing, and noise reduction.

[0137] It should be noted that a digital image of the deformed area is obtained. This may require multi-view images to obtain richer information. If there are multi-view images, image registration is performed. This can be achieved using algorithms such as SIFT and SURF. In addition, 3D reconstruction technology involves techniques for recovering 3D information from 2D images. Commonly used algorithms include: Multi-view stereo technology: Using images from two or more viewpoints, corresponding points are found by matching feature points in the images, and then the 3D coordinates are calculated using the principle of triangulation. Monocular stereo technology: Using a single-view image, the 3D information of pixel points is judged based on clues in the image such as texture and shadow. Structured light 3D scanning: Scanning the object surface with structured light to obtain 3D information. Optical flow algorithm: Tracking feature points in the image and calculating 3D information based on the movement trajectories of the feature points. 3D reconstruction technology can effectively recover 3D information from 2D images, providing a more comprehensive and accurate basis for deformation detection and evaluation. However, in practical applications, the most suitable algorithm still needs to be selected in combination with specific images and environments.

[0138] S4. Compare the 3D information data of the deformed area with the 3D information data in the BIM model to detect the exact location and degree of deformation of the steel structure.

[0139] Preferably, the step of comparing the 3D information data of the deformed area with the 3D information data in the BIM model to detect the exact location and degree of deformation of the steel structure includes the following steps:

[0140] S41. Obtain the 3D information of the deformed area;

[0141] S42. Use the ICP algorithm to register the 3D information data of the deformed area with the 3D information data in the BIM model and map them to a unified coordinate system;

[0142] S43. Use the morphological difference method in computer vision to calculate the shape difference between the 3D information data of the deformed area and the 3D information data in the BIM model, and judge whether there is a large morphological difference;

[0143] S44. Determine the location of the deformation and evaluate the degree of deformation according to the judgment result of the morphological difference;

[0144] S45. Record the location and degree of the deformation and provide a visual display interface for information input and output.

[0145] Preferably, the step of using the ICP algorithm to register the 3D information data of the deformed area with the 3D information data in the BIM model and map them to a unified coordinate system includes the following steps:

[0146] S421. Set initial transformation parameters;

[0147] S422. Find the nearest neighbor points in the 3D information data of the BIM model for each point in the 3D information data of the deformation region to form point pairs;

[0148] S423. Calculate the distance between the paired matching points of the nearest neighbor points and find a transformation that minimizes the sum of squared errors of this distance;

[0149] S424. Update the current transformation parameters using the obtained transformation;

[0150] S425. Repeat the steps of S422 - S424 until the change in the transformation parameters is less than a certain threshold or the number of iterations exceeds the set maximum number of iterations;

[0151] S426. Map the 3D information data of the deformation region to the coordinate system of the BIM model using the obtained transformation parameters.

[0152] It should be noted that the ICP (Iterative Closest Point) algorithm is a 3D point cloud registration algorithm. It finds the optimal registration transformation between two sets of point clouds through iteration. The main steps include: selecting initial transformation parameters. Finding the nearest neighbor points for each point in the two sets of point clouds to form point pairs. A point pair refers to the nearest neighbor point found in the target dataset for each point in the source dataset. That is, for each point in the source dataset, find the nearest point in the target dataset, and these two points form a point pair. Calculate the distance between the point pairs and find the transformation that can minimize the distance. Update the current transformation using the new transformation. Perform iteration until the change in the transformation parameters is less than the threshold or the maximum number of iterations is reached. The morphological difference method is used to compare two 3D shapes and find the differences between them. The main steps include: converting the two 3D shapes to the same coordinate system. Sampling the surfaces of the two shapes to obtain discrete point sets. For each point, find the nearest point on the other shape. Calculate the distance between each point pair, and if it exceeds the threshold, it is considered that there are significant differences in this area. Record and display the difference areas. The combination of these technologies can achieve an accurate comparison of the 3D information of the deformation region and the 3D information of the BIM model, detect the location and degree of deformation, and provide a basis for subsequent evaluation and processing.

[0153] In addition, during the application of the Iterative Closest Point (ICP) algorithm, in order to control the number of iterations and the accuracy, the change threshold and the maximum number of iterations of the transformation parameters are usually set. The setting of these two thresholds needs to balance the calculation speed and accuracy, and the specific setting is usually determined according to the task requirements and calculation resources.

[0154] Change threshold of transformation parameters: This is a very small positive number, representing the maximum allowable range of change in transformation parameters. If in two consecutive iterations, the changes in all transformation parameters (such as translation parameters and rotation parameters) are less than this threshold, then it can be considered that the algorithm has converged and the iteration stops. The setting of this threshold usually needs to be determined according to the specific scale of the data and the requirements for registration accuracy.

[0155] Maximum number of iterations: This is a positive integer that limits the maximum number of iterations of the algorithm. For example, it can be set to 100 times. Even if the algorithm has not fully converged, if the number of iterations has reached this upper limit, then the iteration also needs to stop. The purpose of setting the maximum number of iterations is to prevent the algorithm from falling into infinite iteration and ensure that the algorithm can be completed within a limited time. The setting of the maximum number of iterations usually needs to be determined according to the requirements for registration accuracy and the constraints of computing resources.

[0156] S5. According to the steel structure detection results, use the improved CNN model and based on three-dimensional information to evaluate the deformation of the steel structure and judge the actual deformation situation.

[0157] Preferably, the step of using the improved CNN model and based on three-dimensional information to evaluate the deformation of the steel structure and judge the actual deformation situation according to the steel structure detection results includes the following steps:

[0158] S51: Obtain three-dimensional information data containing the deformation of the steel structure, and process the three-dimensional information data, including denoising, normalization, and scaling;

[0159] S52: Construct a pre-trained improved CNN model;

[0160] S53: Use a data set containing known deformation situations to train the pre-trained improved CNN model;

[0161] S54: Input the preprocessed three-dimensional information data into the trained improved CNN model, and the model will output the predicted deformation information;

[0162] S55: Analyze the results predicted by the trained improved CNN model to determine the actual deformation situation of the steel structure. The deformation situation includes at least the size, position, and shape of the deformation, and feedback the analysis results of the deformation situation. In the present invention, the improved CNN model is a 3D CNN model.

[0163] It should be explained that the improved CNN model is a deep learning model for image processing. Common improved CNN (Convolutional Neural Network) models include:

[0164] 1. 3D CNN: It can directly process three-dimensional information and extract three-dimensional features. It uses three-dimensional convolutional kernels and pooling layers, and can better process three-dimensional information.

[0165] 2. ResNet: It introduces residual blocks, which can effectively solve the problem of gradient vanishing in deep networks, enabling the network to be constructed deeper. This can improve the prediction accuracy of the model.

[0166] 3. DenseNet: Each layer is directly connected to all other layers, which can effectively utilize features and improve the utilization efficiency of parameters. This can also improve the prediction accuracy of the model.

[0167] 4. Inception: Different-sized convolutional kernels are used in parallel in the same module, which can effectively extract features at different scales. This can also improve the prediction accuracy of the model. The present invention is preferably a 3D CNN.

[0168] In addition, "feedback" may have several interpretations, specifically depending on the actual application environment and requirements:

[0169] Feedback to the system: In this case, the analysis results will be fed back into the system, and the system can make self-adjustments based on this to optimize the prediction effect. For example, a system that borrows the reinforcement learning method will learn based on the feedback of the prediction results and update and optimize the parameters of the model.

[0170] Feedback to the user: In this scenario, for example, professional engineers or managers of building structures can make corresponding decisions based on the feedback results. For example, if the report indicates serious structural deformation, the maintenance team may need to take immediate action to repair or reinforce. If the report shows small-scale and minor deformation, it may be possible to postpone the treatment.

[0171] Feedback to the database: In some cases, the feedback results may be stored in the database for future analysis and reference. For example, the results of each inspection of a building can be recorded for long-term monitoring and trend analysis.

[0172] S6. Obtain the evaluation results of the deformation situation and formulate a treatment plan for the deformation of the steel structure.

[0173] Preferably, the step of obtaining the evaluation results of the deformation situation and formulating a treatment plan for the deformation of the steel structure

[0174] includes the following steps:

[0175] S61. Obtain the evaluation results of the deformation situation. If the steel structure shows minor deformation, carry out reinforcement treatment;

[0176] S62. If the steel structure shows serious deformation, carry out demolition and reconstruction or replacement;

[0177] S63. If the steel structure shows severe deformation or complex deformation, the complex deformation shall be judged according to the knowledge graph, and the construction treatment plan shall be optimized.

[0178] The judgment of the complex deformation according to the knowledge graph includes the following steps:

[0179] Construct a knowledge graph containing the knowledge of steel structure deformation;

[0180] Input the deformation evaluation result of the complex deformation into the knowledge graph, and search for known deformation cases similar to the input data in the knowledge graph;

[0181] According to the found similar cases, propose a plan and suggestions for dealing with the input deformation situation;

[0182] Based on the input deformation data and professional knowledge, optimize and adjust the construction treatment plan.

[0183] It should be noted that the knowledge graph is a structure for organizing knowledge, which uses the entity-relationship model to describe the concepts in a certain field and the relationships between concepts. The knowledge graph technology can be used for the judgment of complex deformation and the recommendation of solutions. Using the knowledge graph technology can realize the judgment of complex deformation situations and the recommendation of solutions.

[0184] According to another embodiment of the present invention, as Figure 2 shown, a BIM-based steel structure deformation monitoring and processing system is also provided. The system includes: a data acquisition and model construction module 1, an image processing module 2, a deformation detection module 3, a deformation positioning module 4, a deformation evaluation module 5, and a processing plan formulation module 6;

[0185] The data acquisition and model construction module 1 is connected to the deformation detection module 3 through the image processing module 2. The deformation detection module 3 is connected to the deformation evaluation module 5 through the deformation positioning module 4. The deformation evaluation module 5 is connected to the processing plan formulation module 6;

[0186] The data acquisition and model construction module 1 is used to obtain the digital image obtained by scanning the steel structure with a laser scanner, and establish a BIM model based on the digital image;

[0187] The image processing module 2 is used to process the digital image by using the GrabCut algorithm, analyze the processed digital image, detect the surface deformation situation of the steel structure, and obtain the deformation area;

[0188] The deformation detection module 3 is used to obtain the three-dimensional information data of the deformation area by using the three-dimensional reconstruction technology;

[0189] The deformation positioning module 4 is used to compare the three-dimensional information data of the deformed area with the three-dimensional information data in the BIM model to detect the exact position and deformation degree of the steel structure deformation;

[0190] The deformation evaluation module 5 is used to evaluate the steel structure deformation based on the three-dimensional information by using an improved CNN model according to the steel structure detection results to judge the actual deformation situation;

[0191] The processing solution formulation module 6 is used to obtain the evaluation result of the deformation situation and formulate a processing solution for the steel structure deformation.

[0192] In summary, by means of the above technical solutions of the present invention, the present invention can regularly scan the steel structure through a laser scanner to monitor the deformation of the steel structure in real time, providing a basis for the timely discovery and treatment of deformation; by comparing with the BIM model, the position of the deformation can be accurately located, which provides accurate information for subsequent repair and reinforcement; through machine learning and deep learning technologies, the deformation in digital images and three-dimensional information can be automatically detected, reducing the workload of manual detection; through the machine learning model, the severity of the deformation can be automatically evaluated, providing a reference for the selection of the repair plan; through the knowledge graph technology, the complex deformation situation can be automatically judged according to historical cases, and recommendations for processing solutions can be provided to improve work efficiency; the knowledge graph technology can provide recommendations for deformation processing solutions according to historical cases, but the final solution still needs to be optimized in combination with professional knowledge, so that artificial intelligence and artificial expert knowledge can be comprehensively utilized to achieve better results; through the BIM model and the knowledge graph, the informatization management of the entire life cycle of the steel structure can be realized, providing information support for future operation and maintenance; the present invention can timely discover and handle potential safety hazards by monitoring and accurately positioning the deformation in real time, reducing the risk brought by the damage of the steel structure, because accurate information can be provided for the repair work according to the severity and position of the deformation, thereby improving the quality and efficiency of the repair work. Through automated detection and evaluation, the workload of manual detection is reduced, saving a large amount of manpower and material resources, thereby reducing the overall operating cost. Timely detection and maintenance can extend the service life of the steel structure and improve its economic benefits. Informatization management enables the data of the entire life cycle of the steel structure to be saved and utilized, which is beneficial to energy conservation and emission reduction and the construction of green buildings.

[0193] Although the present invention has been disclosed above with preferred embodiments, the embodiments are only for the purpose of illustration and exemplification, and are not intended to limit the present invention. Those skilled in the art can make several modifications and refinements without departing from the spirit and scope of the present invention. The scope of protection claimed by the present invention shall be subject to what is described in the claims.

Claims

1. A method for monitoring and processing the deformation of a steel structure based on BIM, characterized in that, the method for monitoring and processing the deformation of the steel structure comprises the following steps: S1. Obtain the digital images scanned by a laser scanner for the steel structure, and establish a BIM model based on the digital images; the BIM model includes the shape, size, and material properties of the steel structure; S2. Process the digital images using the GrabCut algorithm, and analyze the processed digital images to detect the surface deformation of the steel structure and obtain the deformed area; S3. Use 3D reconstruction technology to obtain the 3D information data of the deformed area; S4. Compare the 3D information data of the deformed area with the 3D information data in the BIM model to detect the exact position and deformation degree of the steel structure deformation; S5. According to the steel structure detection results, use an improved CNN model and based on the 3D information to evaluate the steel structure deformation and judge the actual deformation situation; S6. Obtain the evaluation results of the deformation situation and formulate a steel structure deformation treatment plan; The step of processing the digital images using the GrabCut algorithm, analyzing the processed digital images, detecting the surface deformation of the steel structure, and obtaining the deformed area includes the following steps: S21. Label the input digital images, label the normal area as background pixels, and label the suspected deformed area as foreground pixels; S22. Construct a color statistical model according to the labeled foreground pixels and background pixels; S23. Use the color statistical model to classify the unlabeled area pixels and judge the unlabeled area pixels; S24. Use the image boundary information to identify the deformed area boundary; S25. Repeat the steps of S23 - S24, and optimize the color statistical model and the deformed area boundary; S26. Evaluate the segmentation results of the deformed area boundary, re - label the inaccurate areas, and repeat the steps of S21 - S25; The step of using the image boundary information to identify the deformed area boundary includes the following steps: S241. Use a Gaussian filter to eliminate the noise in the image; S242. Calculate the gradient intensity and direction of the image, and find the candidate points of the boundary; S243. Determine the true edge information through a double - threshold algorithm and non - maximum suppression; S244. Use the Hough transform to perform shape analysis on the edge information and extract geometric features; S245. Use statistical methods to compare the differences in the boundary features between the deformed area and the normal area; S246. According to the comparison results, set a threshold, and mark the area with the boundary feature difference greater than the threshold as the deformed area; Among them, the step of determining the true edge information through a double - threshold algorithm and non - maximum suppression includes the following steps: S2431. Use the Sobel operator to calculate the gradient values of each pixel point in the horizontal and vertical directions respectively; S2432. Calculate the gradient intensity of each pixel point according to the gradient values in the horizontal and vertical directions; S2433. In the same direction, retain the pixel point with the maximum gradient intensity, and regard other pixel points as non - edges and suppress them; S2434. Set a high threshold and a low threshold. Among them, the pixel points with gradient intensity greater than the high threshold are identified as strong edges, the pixel points less than the low threshold are identified as non-edges, and the pixel points between the high threshold and the low threshold are regarded as weak edges; S2435. If there are strong edge pixel points in the neighborhood of the weak edge pixel points, then the weak edge is also identified as an edge, otherwise it is identified as a non-edge. The improved CNN model is a 3D CNN model; The edge detection algorithm used to identify the boundary of the deformed area is specifically: the Canny operator combined with the Hough transform; The steps for obtaining the three-dimensional information data of the deformed area by using the three-dimensional reconstruction technology include the following: S31. Obtain the digital image of the deformed area; S32. If there are digital images from multiple perspectives, register the digital images and establish the corresponding relationship between the feature points in the digital images; S33. Determine whether there are images from multiple perspectives. If so, select the multi-view stereo algorithm. If not, select the monocular stereo algorithm; S34. Select a three-dimensional reconstruction algorithm according to the judgment result to obtain the three-dimensional information of the deformed area; S35. Process the three-dimensional information. The processing includes at least three-dimensional information completion, smoothing, and noise reduction.

2. A method for monitoring and processing the deformation of a steel structure based on BIM according to claim 1, characterized in that, The steps for classifying the pixel points in the unlabeled area by using the color statistical model and judging the pixel points in the unlabeled area include the following: S231. Obtain the unlabeled area according to the already labeled foreground pixel points and background pixel points; S232. Extract the pixel colors of the unlabeled area, input the pixel colors of the unlabeled area into the color statistical model, and calculate the probabilities of the pixel colors of the unlabeled area being foreground and background; S233. According to the foreground and background probabilities of each pixel, if the probability of the foreground is higher than the probability of the background, the pixel is classified as the foreground, otherwise it is classified as the background, and pixel classification is performed; S234. Obtain the classification result and generate the initial image segmentation result.

3. A method for monitoring and processing the deformation of a steel structure based on BIM according to claim 1, characterized in that, The steps for comparing the three-dimensional information data of the deformed area with the three-dimensional information data in the BIM model to detect the exact position and deformation degree of the steel structure deformation include the following: S41. Obtain the three-dimensional information of the deformed area; S42. Use the ICP algorithm to register the three-dimensional information data of the deformed area with the three-dimensional information data in the BIM model and map them to a unified coordinate system; S43. Use the morphological difference method in computer vision to calculate the shape difference between the three-dimensional information data of the deformed area and the three-dimensional information data in the BIM model, and judge whether there is a large morphological difference; S44. Determine the position of the deformation and evaluate the degree of deformation according to the judgment result of the morphological difference; S45. Record the position and degree of deformation and provide a visual display interface for information input and information output The step of judging whether there is a large morphological difference is: Transform two three-dimensional shapes into the same coordinate system, sample the surfaces of the two shapes to obtain a discrete point set. For each point, find the nearest point on the other shape, calculate the distance between each pair of points. If the distance exceeds the threshold, it is considered that there are significant differences in this area, and record and display the difference area.

4. A method for monitoring and processing steel structure deformation based on BIM according to claim 3, characterized in that, registering the three-dimensional information data of the deformed area with the three-dimensional information data of the BIM model using the ICP algorithm and mapping them into a unified coordinate system includes the following steps: S421. Set initial transformation parameters; S422. Find the nearest neighbor points in the three-dimensional information data of the BIM model for each point in the three-dimensional information data of the deformed area to form point pairs; S423. Calculate the distance between the point pairs of the nearest neighbor points and find a transformation that minimizes the sum of the squared errors of this distance; S424. Update the current transformation parameters using the obtained transformation; S425. Repeat the steps of S422 - S424 until the change in the transformation parameters is less than a certain threshold or the number of iterations exceeds the set maximum number; S426. Map the three-dimensional information data of the deformed area into the coordinate system of the BIM model using the obtained transformation parameters.

5. A method for monitoring and processing steel structure deformation based on BIM according to claim 1, characterized in that, evaluating the steel structure deformation based on the three-dimensional information using an improved CNN model according to the steel structure detection results and judging the actual deformation situation includes the following steps: S51: Obtain the three-dimensional information data containing the steel structure deformation and process the three-dimensional information data, including denoising, normalization, and scaling; S52: Construct a pre-trained improved CNN model; S53: Use a data set containing known deformation situations to train the pre-trained improved CNN model; S54: Input the pre-processed three-dimensional information data into the trained improved CNN model, and the model will output the predicted deformation information; S55: Analyze the results predicted by the trained improved CNN model to determine the actual deformation situation of the steel structure. The deformation situation at least includes the size, position, and shape of the deformation, and feedback the analysis results of the deformation situation.

6. A method for monitoring and processing steel structure deformation based on BIM according to claim 1, characterized in that, obtaining the evaluation results of the deformation situation and formulating a steel structure deformation treatment plan includes the following steps: S61. Obtain the evaluation results of the deformation situation. If the steel structure shows slight deformation, carry out reinforcement treatment; S62. If the steel structure shows severe deformation, carry out demolition and reconstruction or replacement; S63. If complex deformation occurs in the severe deformation of the steel structure, judge the complex deformation according to the knowledge graph and optimize the construction treatment plan; judging the complex deformation according to the knowledge graph includes the following steps: Construct a knowledge graph containing steel structure deformation knowledge; Input the deformation evaluation results of complex deformations into the knowledge graph and search for known deformation cases similar to the input data in the knowledge graph; Based on the found similar cases, propose solutions and suggestions for dealing with the input deformation situation; Optimize and adjust the construction treatment plan based on the input deformation data and professional knowledge.

7. A BIM-based steel structure deformation monitoring and processing system for implementing the BIM-based steel structure deformation monitoring and processing method described in any one of claims 1-6, characterized in that, the system includes: a data acquisition and model construction module, an image processing module, a deformation detection module, a deformation positioning module, a deformation evaluation module, and a treatment plan formulation module; the data acquisition and model construction module is connected to the deformation detection module through the image processing module, the deformation detection module is connected to the deformation evaluation module through the deformation positioning module, and the deformation evaluation module is connected to the treatment plan formulation module; the data acquisition and model construction module is used to obtain digital images scanned by a laser scanner on the steel structure and establish a BIM model based on the digital images; the image processing module is used to process the digital images using the GrabCut algorithm, analyze the processed digital images, detect the surface deformation of the steel structure, and obtain the deformation area; the deformation detection module is used to obtain three-dimensional information data of the deformation area using three-dimensional reconstruction technology; the deformation positioning module is used to compare the three-dimensional information data of the deformation area with the three-dimensional information data in the BIM model to detect the precise position and deformation degree of the steel structure deformation; the deformation evaluation module is used to evaluate the steel structure deformation based on the three-dimensional information using an improved CNN model according to the steel structure detection results and judge the actual deformation situation; the treatment plan formulation module is used to obtain the evaluation results of the deformation situation and formulate a steel structure deformation treatment plan.

Images