A method and system for identifying building material defects

Abstract

The present invention relates to the technical field of building material identification, and discloses a building material defect identification method and system, comprising: periodically acquiring image data and environmental data of a building to be tested through an acquisition device, recording the position information of the acquisition device, performing hierarchical grid image registration on image data of adjacent periods based on the acquisition device's position information to obtain aligned images; obtaining a defect mask based on the aligned images, identifying and marking defective areas based on the defect mask, comparing the defective areas of the current period with those of the previous period, and obtaining time series data of the defective areas; generating a gating coefficient based on the environmental data, constructing a time series model based on the gating coefficient and the time series data, quantitatively predicting the future evolution trend of the defective areas, and outputting the prediction results. The method of the present invention can achieve millimeter-level defect tracking and cycle-level risk warning, and is suitable for structural safety monitoring of key components such as bridges, tunnels, and curtain walls.

Description

Technical Field

[0001] The present invention relates to the technical field of building material identification, and in particular to a building material defect identification method and system. Background Art

[0002] As infrastructure expands and its service life increases, structural defects such as aging, cracking, and spalling of building materials are becoming increasingly prominent. This is especially true for key components like bridges, tunnels, and curtain walls. Surface defects not only impact structural safety but also significantly impact long-term maintenance costs. To reduce manual inspection costs and improve the efficiency of maintenance decisions, an increasing number of monitoring systems are incorporating image recognition and trend prediction techniques to digitally model and assess the risk of defect evolution.

[0003] Existing defect recognition and prediction methods generally rely on single-image registration or univariate time series modeling, which suffer from two key deficiencies: First, the accuracy of image alignment is limited by viewing angle, illumination, and non-rigid structural deformation, leading to the accumulation of defect tracking errors and affecting the quality of the time series; second, prediction models often ignore the regulatory effects of external environmental factors, such as temperature, humidity, or load changes on crack evolution, resulting in delayed model response or misjudgment of risk levels. Therefore, a comprehensive defect recognition solution that integrates high-precision registration mechanisms with environmentally aware prediction models is urgently needed to improve recognition accuracy and early warning effectiveness. Summary of the Invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] Therefore, the technical problem solved by the present invention is that the existing defect identification method has the problem of defect tracking error accumulation and ignores the impact of the external environment on building defects.

[0006] To solve the above technical problems, in a first aspect, the present invention provides the following technical solutions: a method for identifying building material defects, comprising: periodically acquiring image data and environmental data of a building to be tested by a collection device, recording the position information of the collection device, and performing hierarchical grid image registration on image data of adjacent periods based on the position information of the collection device to obtain an aligned image;

[0007] Obtain a defect mask based on the aligned image, identify and mark the defect area according to the defect mask, compare the defect area in this cycle with that in the previous cycle, and obtain time series data of the defect area;

[0008] Generate gating coefficients based on environmental data, build a time series model based on the gating coefficients and time series data, quantitatively predict the future evolution trend of the defect area and output the prediction results.

[0009] As a preferred solution of the method for identifying building material defects according to the present invention, the environmental data includes environmental temperature, humidity and structural load;

[0010] The position information of the acquisition device includes the position and attitude angle of the acquisition device.

[0011] As a preferred embodiment of the method for identifying building material defects described in the present invention, the hierarchical grid image registration includes converting the posture information of adjacent periods into a homogeneous coordinate transformation matrix, and combining the internal parameters of the acquisition device to calculate the initial geometric transformation relationship of the image data of the current period relative to the image data of the previous period, and obtaining the initial homography matrix ,according to Get coarse alignment image ;

[0012] Will Divided into Equal-sized subgrids , the image data of the previous cycle Divide the same grid to ensure one-to-one correspondence between sub-grids;

[0013] Extract within each subgrid feature points, only matching is allowed between corresponding sub-grid areas that are spatially adjacent, and the affine matrix is ​​estimated by local RANSAC and remove outliers;

[0014] Construct a joint optimization objective function for all sub-grids and solve it to obtain the final affine transformation parameters of each sub-grid;

[0015] Based on the final affine transformation parameters of each sub-grid, the image data of the current cycle is subjected to geometric transformation and pixel resampling on a sub-grid basis, and the image data of the current cycle is transformed into a spatial coordinate system that is strictly aligned with the image data of the previous cycle to obtain an aligned image.

[0016] As a preferred solution of the method for identifying building material defects described in the present invention, the joint optimization objective function is based on the energy minimization model of non-rigid image registration, and the initial homography matrix generated by the pose information is introduced. As a priori guidance, use the subgrid affine matrix Instead of a global single transformation, it is expressed as:

[0017] ;

[0018] in, Indicates the image in the current cycle Affine matrix of j subgrids; Indicates that it belongs to Pixel points in j sub-grid areas; represents the initial homography matrix; Represents the image subgrid area after coarse alignment; represents the regularization term weight coefficient; express The affine matrix of the adjacent subgrid; Represents minimization of the joint optimization objective function; represents the two-norm.

[0019] As a preferred embodiment of the method for identifying building material defects according to the present invention, the step of obtaining time series data of the defective area includes performing a pixel-by-pixel comparison of the current cycle image data with the previous cycle image data based on the aligned image to generate a change mask image;

[0020] Performing connected domain analysis on the change mask graph to extract all changed regions and calculating the structural feature index of each changed region, wherein the structural feature index includes area, maximum length, main axis direction, shape complexity, and center of mass;

[0021] Each changed area is uniquely numbered, and a time series of changes in structural feature indicators is established. For each changed area currently detected, the spatial overlap rate with the defect area of ​​the previous cycle is calculated to determine whether it is a continuation of the same defect;

[0022] If the spatial overlap between the current change region and the defect region of the previous cycle is greater than or equal to the set threshold, it is determined to be a continuation of the defect region of the previous cycle;

[0023] If the spatial overlap between the current change area and the defect area of ​​the previous cycle is less than the set threshold, it is judged as a newly generated defect and assigned a new number;

[0024] The structural characteristic indicators of each numbered defect in the current cycle are stored in the database according to the timestamp to obtain the time series data of the defect area.

[0025] As a preferred embodiment of the method for identifying building material defects according to the present invention, the method of constructing a time series model based on the gating coefficient and the time series data includes obtaining the time series data of each numbered defective area, arranging the data in ascending order by timestamp, and forming a main sequence;

[0026] Synchronously organize the environmental data collected in the same period, establish a sliding window of fixed length for each type of environmental data, calculate the cumulative statistics of the environmental data within the sliding window, and splice the environmental data and cumulative statistics to generate an auxiliary sequence;

[0027] Based on the LSTM model, an environmental gating mechanism is introduced to calculate dynamic weights through auxiliary sequences to control the hidden state update process;

[0028] A fitting term for the area growth rate is introduced in model training, and a growth rate error regularization term is added to the total loss function to construct the total loss function.

[0029] As a preferred solution of the method for identifying building material defects described in the present invention, the quantitative prediction of the future evolution trend of the defect area and the output of the prediction results include obtaining the evolution trend data of the future cycle of each defect number according to the time series model, pre-setting a risk factor set, establishing a multi-factor threshold judgment rule in combination with the risk factor set, and outputting the prediction results.

[0030] In a second aspect, the present invention provides a building material defect identification system using any of the methods described in the present invention, wherein: an image processing module periodically acquires image data and environmental data of a building to be tested through an acquisition device, records the position information of the acquisition device, and performs hierarchical grid image registration on image data of adjacent periods based on the position information of the acquisition device to obtain an aligned image;

[0031] The recognition module obtains a defect mask based on the aligned image, identifies and marks the defect area according to the defect mask, compares the defect area in the current cycle with that in the previous cycle, and obtains time series data of the defect area;

[0032] The prediction module generates gating coefficients based on environmental data, builds a time series model based on the gating coefficients and time series data, quantitatively predicts the future evolution trend of the defect area, and outputs the prediction results.

[0033] Beneficial effects of the present invention: The present invention significantly improves the pixel-level alignment accuracy of cross-period images through hierarchical grid image registration guided by camera posture, ensuring the continuity of defect area numbering and the accuracy of evolution indicators; the time series prediction model that introduces an environmental gating mechanism can dynamically adjust the prediction weight according to environmental disturbances such as temperature and humidity, thereby enhancing the model's response to crack mutations; at the same time, combined with the growth rate regularization term, the model is guided to pay attention to the speed of defect development, thereby improving the ability to identify early risks; finally, the prediction results, gating strength and defect trends are combined to achieve multi-factor risk level determination and graphical output. The overall solution can achieve millimeter-level defect tracking and cycle-level risk warning, and is suitable for structural safety monitoring of key components such as bridges, tunnels, and curtain walls, with high precision and engineering practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0035] Figure 1 This is an overall flow chart of a building material defect identification method provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0036] To make the above-mentioned objects, features, and advantages of the present invention more clearly understood, the following detailed description of the specific embodiments of the present invention is given in conjunction with the accompanying drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in this field without creative work should fall within the scope of protection of the present invention.

[0037] Example 1, reference Figure 1 , as one embodiment of the present invention, provides a method for identifying building material defects, comprising:

[0038] S1: The image data and environmental data of the building to be measured are periodically acquired through the acquisition device, and the posture information of the acquisition device is recorded. According to the posture information of the acquisition device, the image data of adjacent periods are hierarchically grid-aligned to obtain an aligned image.

[0039] Furthermore, acquisition equipment mounted on drones, railcars, or mobile inspection arms periodically captures image data and environmental data from the target building structure area. The image data is high-resolution images of the building surface, and the environmental status data includes, but is not limited to, ambient temperature, humidity, structural load, and vibration information at the time of image acquisition. This data is used for subsequent defect evolution trend analysis.

[0040] In order to improve the comparability of image data in the time dimension, the spatial posture information of the acquisition device is recorded at each acquisition, including position coordinates (obtained by RTK or GPS module) and attitude angles (obtained by IMU).

[0041] It's important to note that spatial pose information is a combination of the acquisition device's position and orientation / rotation in the world coordinate system, often expressed as a six-dimensional vector or homogeneous transformation matrix. On construction sites, image acquisition often occurs at varying angles and with unreproducible positioning. Fusion of IMU (angle) and RTK (position) provides an initial geometric estimate, significantly improving the success rate and accuracy of subsequent image registration.

[0042] Furthermore, in terms of image registration, existing methods often rely on pure feature matching, making it difficult to address issues such as large variations in illumination, occlusion, and viewing angles on construction sites. To achieve high-precision alignment of periodically acquired images, this paper proposes a hierarchical grid image registration method that incorporates camera pose information. This method constructs initial geometric constraints based on the camera pose (IMU or RTK), and then iteratively corrects them within a local rigid subgrid, ensuring global consistency while avoiding feature drift. The specific steps of hierarchical grid image registration include:

[0043] The first step is to obtain the spatial pose data corresponding to the current cycle image and the previous cycle image from the acquisition device. By converting the camera poses of two adjacent cycles into a homogeneous coordinate transformation matrix and combining it with the camera intrinsic parameters, the initial geometric transformation relationship of the current image with the previous cycle image, namely the initial homography matrix, is calculated. This initial transformation matrix can be regarded as a rough projection relationship of the current image onto the image of the previous cycle. It has global geometric consistency but does not take into account local deformation.

[0044] Specifically, the original camera six-dimensional pose vector 、 Parsed as a homogeneous matrix 、 , calculate the relative attitude:

[0045] ;

[0046] in, Represents a period and cycle The relative posture of the acquisition equipment; Represents a period The camera's six-dimensional pose vector; Represents a period The camera's six-dimensional pose vector. Assume that the building surface being measured can be approximated as a plane , according to the camera internal parameters Find the initial homography matrix :

[0047] ;

[0048] in, represents the relative rotation matrix; Represents the relative translation vector; Transpose of the matrix representing the plane normal; Indicates the distance from the plane to the acquisition device. and Depend on Decompose to obtain The cycle The original image is projected onto the cycle The coordinate system of the original image is used to obtain the coarse alignment image .

[0049] The second step is to Divided into Equal-sized subgrids , the image data of the previous cycle The same grid is divided so that each sub-grid maintains roughly the same position as the corresponding area in the previous cycle image, providing a spatial basis for the subsequent construction of the local affine transformation model.

[0050] The third step is to use SuperPoint or ORB to extract the Local image feature points (such as corner points, edge points) are detected and feature matching operations are performed in a limited area. To prevent global mismatching, only matching is allowed between corresponding grid areas that are adjacent in space to obtain an initial set of matching point pairs. The affine matrix is ​​estimated by local RANSAC. And remove outliers.

[0051] The fourth step is to construct a joint optimization objective function for all subgrids with two constraints:

[0052] The first part of the constraint makes the affine transformation of each sub-grid as close as possible to the global projection result obtained in the first step; the second part of the constraint ensures that the affine transformation between adjacent grids has a certain smoothness to prevent sudden distortion.

[0053] In traditional methods of non-rigid image registration, the optimization objective function usually consists of two energies, including a data term and a regularization term. The data term (DataTerm) is used to minimize the local reprojection error between the registered images to ensure image content alignment; the regularization term (SmoothnessTerm) is used to limit the difference in transformation parameters between adjacent regions, avoid local transformation mutations, and maintain the spatial smoothness of the deformation field.

[0054] Based on this basic structure, the present invention combines the camera pose information to construct a global guide term and introduces the initial homography matrix generated by the pose information. As a priori guidance term, and constructing an affine model at the grid level, using the subgrid affine matrix Instead of a global single transformation, a joint optimization objective function is constructed, which is expressed as:

[0055] ;

[0056] in, Indicates the image in the current cycle Affine matrix of j subgrids; Indicates that it belongs to Pixel points in j sub-grid areas; represents the initial homography matrix; Represents the image subgrid area after coarse alignment; represents the regularization term weight coefficient; express The affine matrix of the adjacent subgrid; Represents minimization of the joint optimization objective function; represents the two-norm.

[0057] By solving this joint optimization problem, the final affine transformation parameters of each sub-grid are obtained. This transformation not only reflects the global relative position but also adapts to micro-scale structural changes such as thermal expansion and contraction and local bending.

[0058] In the fifth step, based on the optimized affine transformation parameters of each subgrid, the current cycle's image undergoes a grid-by-grid geometric transformation and pixel resampling, transforming the entire image into a spatial coordinate system that is strictly aligned with the previous cycle's image to obtain an aligned image. The resulting registered image maintains global consistency and is elastically deformable locally, supporting subsequent pixel-level defect evolution analysis.

[0059] It's important to note that because image acquisition distances for building facades, bridges, and other structures can often be 10–30 meters, the pose errors (centimeter-level, 0.1–0.3° angular deviation) generated by IMU / RTK are amplified when the images are projected onto the building component surface, resulting in pixel-level misalignment >5px. Defects are typically only 2–10px wide, and these misalignments can directly mask early crack evolution. Therefore, in the third feature matching step, the entire image is divided into small grids, each of which only allows for local affine micro-transformations. This confines pose errors to the local grid area and prevents amplification. For example, with a 16×16 subgrid, the required computations involve only ≤256 subgrids, with ≤30 feature points per grid. This enables centimeter-level pose alignment to sub-pixel alignment using low-cost hardware, compatibility with material thermal expansion and occlusion, and continuous output of defect evolution curves to meet the needs of long-term monitoring scenarios.

[0060] S2: Obtain a defect mask image based on the aligned image, identify and mark the defect area according to the defect mask image, compare the defect area of ​​the current cycle with that of the previous cycle, and obtain time series data of the defect area.

[0061] Furthermore, based on the aligned image, the current cycle image data is compared pixel by pixel with the previous cycle image data to generate a change mask map. Specifically, the current aligned image and the previous cycle image are taken, and the frame difference method is used to perform grayscale difference calculation on the two aligned images to generate a difference map. Edge detection (such as Canny and Sobel) is performed on the two images respectively, and their difference is superimposed on the difference map to enhance structural changes such as cracks and edge erosion; dynamic threshold segmentation is applied to the difference map to obtain a binary mask map, and isolated noise points are removed through corrosion and expansion operations, local holes are filled, and a clear change area is formed.

[0062] Connected domain analysis is performed on the change mask image to extract all changed areas (suspected defect areas) and calculate the structural feature indicators of each changed area. The structural feature indicators include but are not limited to area, maximum length, main axis direction, shape complexity, and center of mass. The structural feature indicators of each changed area are output for subsequent matching and numbering.

[0063] To track the evolution of the same defect over time, each changed region is uniquely numbered, and a time series of changes in structural feature indicators is established. For each detected changed region, the spatial overlap rate with the defect region of the previous cycle is calculated to determine whether it is a continuation of the same defect.

[0064] If the spatial overlap ratio between the current change area and the defect area of ​​the previous cycle is greater than or equal to the set threshold, it is judged as a continuation of the defect area of ​​the previous cycle; if the spatial overlap ratio between the current change area and the defect area of ​​the previous cycle is less than the set threshold, it is judged as a newly generated defect and assigned a new number.

[0065] The structural characteristic indicators of each numbered defect in the current cycle are stored in the database according to the timestamp to obtain the time series data of the defect area.

[0066] By performing difference analysis between aligned images, extracting defect change regions, and integrating target matching strategies to manage numbering, this method effectively constructs the evolution trajectory of defect targets over time, providing high-precision historical data support for subsequent evolution trend prediction and risk warning modules. This method boasts high computational efficiency, a clear structure, and strong scalability, making it suitable for deployment in systems such as engineering inspections and long-term monitoring.

[0067] S3: Generate gating coefficients based on environmental data, build a time series model based on the gating coefficients and time series data, quantitatively predict the future evolution trend of the defect area, and output the prediction results.

[0068] Furthermore, the time series data of the defective area is used in combination with the on-site environmental variables (temperature, humidity, load, etc.) to quantitatively predict the future expansion trend of building material defects and output the risk level and repair recommendations. The time series model constructed based on the gating coefficient and time series data includes obtaining the time series data of each numbered defective area, arranging them in ascending order by timestamp, and forming a main sequence. .

[0069] Environmental data collected during the same period are collated simultaneously. Since the environmental data are the current instantaneous values ​​at the time of image acquisition and do not take into account the cumulative exposure effects (such as concrete swelling after a week of high humidity), the model is slow to respond to "chronic environmental stress" and predictions may still lag.

[0070] Therefore, in addition to the original environmental data, a fixed-length sliding window is established for each environmental data, the cumulative statistics of the environmental data in the sliding window are calculated, and the environmental data and cumulative statistics are spliced ​​to generate an auxiliary sequence Cumulative statistics include but are not limited to mean temperature / humidity, maximum load peak, and cumulative number of load cycles.

[0071] It's important to note that cumulative statistics require only a sliding window calculation during data ingestion, with virtually no additional overhead on inference speed. The dual-channel gating of instantaneous and cumulative allows the model to generate progressively higher gating coefficients for "multi-day high humidity" or "load fatigue," amplifying crack growth signals early on.

[0072] Furthermore, in the long-term evolution monitoring scenario of building material defects, environmental conditions are key drivers of crack generation and propagation, including:

[0073] High temperature causes thermal expansion cracks;

[0074] High humidity causes the material to absorb water and swell, and the coating to peel off;

[0075] High loads or frequent vibrations induce fatigue cracks or edge erosion.

[0076] However, traditional time series-based prediction methods (such as LSTM and ARIMA) only consider the historical indicator changes of the defect itself and are unable to perceive the triggering effect of these external disturbances on structural changes, resulting in large prediction errors and untimely warnings, which are more obvious when the structural state suddenly changes.

[0077] To this end, the present invention proposes to introduce an environmental gating mechanism into the time series model, which controls the update amplitude of the model state through a set of weight coefficients dynamically calculated from environmental variables, so that the model has immediate sensitivity to external abnormal stimuli. The gated generation of is represented as:

[0078] ;

[0079] The hidden state correction formula is expressed as:

[0080] ;

[0081] in, represents the gating coefficient vector; Represents the Sigmoid function; represents the learnable weight matrix; represents the bias vector; represents the new hidden state after the introduction of environmental gating; It means bitwise multiplication; Represents a period The hidden state calculated by traditional LSTM; Represents a period Hidden state computed by traditional LSTM.

[0082] When environmental variables are stable, the gating coefficient tends to retain historical states to avoid overfitting. When external temperature and humidity loads change suddenly, the gating coefficient increases, prompting the model to quickly update its state and reflect the change promptly. It should be noted that sudden changes here do not refer to short-term changes (such as seconds or hours), but rather large environmental changes caused by seasonal changes or severe weather.

[0083] In order to make the model more sensitive to the rapid expansion of defects in a short period of time, the present invention introduces a fitting term for the area growth rate in the model training:

[0084] ;

[0085] in, represents the relative growth rate; Represents a period Defect area; Represents a period defect area.

[0086] Add the growth rate error regularization term to the total loss function:

[0087] ;

[0088] in, represents the growth rate error regularization term; The weight coefficient that represents the contribution ratio of controlling the two losses; represents the predicted growth rate; Represents the actual growth rate. The mean square error total loss function is constructed based on the growth rate error regularization term.

[0089] Furthermore, the evolution trend data of each defect number in the future cycle is obtained based on the time series model. In order to comprehensively evaluate the defect risk, a risk factor set is pre-set, in which each dimension factor is derived from the data results generated by the aforementioned creation point, including but not limited to the predicted area, predicted growth rate and environmental response index. The predicted area and predicted growth rate are directly obtained based on the time series model, and the environmental response index is obtained by the gate coefficient. The average value indicates the sensitivity of the model to environmental changes.

[0090] Combined with the risk factor set, multi-factor threshold judgment rules are established to output prediction results. Prediction results include structured warning information, visual output (charts and maps), and operation and maintenance priority sorting recommendations. Structured warning information is a structured record of each defect's risk level, current prediction indicators, and historical trends. Visual output includes time trend curves (area, growth rate), risk level heat maps, and defect number distribution maps on the structure.

[0091] The method of the present invention can achieve millimeter-level defect tracking and cycle-level risk warning. It is suitable for structural safety monitoring of key components such as bridges, tunnels, and curtain walls, and has high precision, high interpretability, and engineering practical value.

[0092] Example 2: In an exemplary embodiment, a building material defect identification system is further provided, comprising:

[0093] The image processing module periodically acquires the image data and environmental data of the building to be tested through the acquisition device, and records the posture information of the acquisition device. According to the posture information of the acquisition device, the image data of adjacent periods are layered grid image registration to obtain aligned images.

[0094] The recognition module obtains a defect mask image based on the aligned image, identifies and marks the defect area according to the defect mask image, compares the defect area of ​​the current cycle with that of the previous cycle, and obtains time series data of the defect area.

[0095] The prediction module generates gating coefficients based on environmental data, builds a time series model based on the gating coefficients and time series data, quantitatively predicts the future evolution trend of the defect area, and outputs the prediction results.

[0096] If the above functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or the portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0097] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0098] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, and then editing, interpreting, or processing in another suitable manner as necessary, and then storing it in a computer memory.

[0099] It should be understood that various components of the present invention may be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods may be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof may be used: a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0100] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for identifying building material defects, characterized in that: include: The image data and environmental data of the building to be tested are periodically acquired through the acquisition device, and the position information of the acquisition device is recorded. According to the position information of the acquisition device, the image data of adjacent periods are subjected to hierarchical grid image registration to obtain an aligned image; Obtain a defect mask based on the aligned image, identify and mark the defect area according to the defect mask, compare the defect area in this cycle with that in the previous cycle, and obtain time series data of the defect area; Generate gating coefficients based on environmental data, build a time series model based on the gating coefficients and time series data, quantitatively predict the future evolution trend of the defect area, and output the prediction results; The environmental data include ambient temperature, humidity and structural load; The position information of the acquisition device includes the position and attitude angle of the acquisition device; The hierarchical grid image registration comprises: The pose information of adjacent cycles is converted into a homogeneous coordinate transformation matrix, and combined with the internal parameters of the acquisition device, the initial geometric transformation relationship of the image data of the current cycle relative to the image data of the previous cycle is calculated to obtain the initial homography matrix ,according to Get coarse alignment image ; Will Divided into Equal-sized subgrids , the image data of the previous cycle Divide the same grid to ensure one-to-one correspondence between sub-grids; Extract within each subgrid feature points, only matching is allowed between corresponding sub-grid areas that are spatially adjacent, and the affine matrix is ​​estimated by local RANSAC and remove outliers; Construct a joint optimization objective function for all sub-grids and solve it to obtain the final affine transformation parameters of each sub-grid; Based on the final affine transformation parameters of each sub-grid, the image data of the current cycle is subjected to geometric transformation and pixel resampling on a sub-grid basis, and the image data of the current cycle is transformed into a spatial coordinate system aligned with the image data of the previous cycle to obtain an aligned image.

2. A method for identifying building material defects according to claim 1, characterized in that: The joint optimization objective function is based on the energy minimization model of non-rigid image registration and introduces the initial homography matrix generated by pose information. As a priori guidance term, use the affine matrix Instead of a global single transformation, it is expressed as: ； in, Indicates the image in the current cycle Affine matrix of j subgrids; Indicates that it belongs to Pixel points in j sub-grid areas; represents the initial homography matrix; represents equal-sized subgrids after coarse alignment; represents the regularization term weight coefficient; express The affine matrix of the adjacent subgrid; Represents minimization of the joint optimization objective function; represents the two-norm.

3. A method for identifying building material defects according to claim 2, characterized in that: The obtaining of time series data of the defective area includes: Based on the aligned image, the current cycle image data is compared with the previous cycle image data pixel by pixel to generate a change mask image; Performing connected domain analysis on the change mask graph to extract all changed regions and calculating the structural feature index of each changed region, wherein the structural feature index includes area, maximum length, main axis direction, shape complexity, and center of mass; Each changed area is uniquely numbered, and a time series of changes in structural feature indicators is established. For each changed area currently detected, the spatial overlap rate with the defect area of ​​the previous cycle is calculated to determine whether it is a continuation of the same defect; If the spatial overlap between the current change region and the defect region of the previous cycle is greater than or equal to the set threshold, it is determined to be a continuation of the defect region of the previous cycle; If the spatial overlap between the current change area and the defect area of ​​the previous cycle is less than the set threshold, it is judged as a newly generated defect and assigned a new number; The structural characteristic indicators of each numbered defect in the current cycle are stored in the database according to the timestamp to obtain the time series data of the defect area.

4. A method for identifying building material defects according to claim 3, characterized in that: The constructing of a time series model based on the gating coefficient and the time series data includes: Obtain the time series data of each numbered defect area and arrange them in ascending order by timestamp to form a main sequence; Synchronously organize the environmental data collected in the same period, establish a sliding window of fixed length for each type of environmental data, calculate the cumulative statistics of the environmental data within the sliding window, and splice the environmental data and cumulative statistics to generate an auxiliary sequence; Based on the LSTM model, an environmental gating mechanism is introduced to calculate dynamic weights through auxiliary sequences to control the hidden state update process; A fitting term for the area growth rate is introduced in model training, and a growth rate error regularization term is added to the total loss function to construct the total loss function.

5. A method for identifying building material defects according to claim 4, characterized in that: The quantitative prediction of the future evolution trend of the defect area and output of the prediction result includes: The evolution trend data of each defect number in the future cycle is obtained based on the time series model, a risk factor set is pre-set, and a multi-factor threshold judgment rule is established based on the risk factor set to output the prediction result.

6. A building material defect recognition system, applied to a building material defect recognition method according to any one of claims 1 to 5, characterized in that: include: The image processing module periodically acquires image data and environmental data of the building to be measured through an acquisition device, records the position information of the acquisition device, and performs hierarchical grid image registration on the image data of adjacent periods based on the position information of the acquisition device to obtain an aligned image; The recognition module obtains a defect mask based on the aligned image, identifies and marks the defect area according to the defect mask, compares the defect area in the current cycle with that in the previous cycle, and obtains time series data of the defect area; The prediction module generates gating coefficients based on environmental data, builds a time series model based on the gating coefficients and time series data, quantitatively predicts the future evolution trend of the defect area, and outputs the prediction results.

Images