Building material defect identification method and system

Abstract

The invention relates to the technical field of building material identification, and discloses a building material defect identification method and system, and the method comprises the steps: periodically obtaining image data and environment data of a to-be-detected building through a collection device, recording the pose information of the collection device, and carrying out the recognition of the to-be-detected building according to the pose information of the collection device; performing layered grid image registration on the image data of the adjacent periods to obtain an aligned image; acquiring a defect mask pattern based on the aligned image, identifying and marking a defect region according to the defect mask pattern, comparing the defect region of the current period with the defect region of the previous period, and acquiring time sequence data of the defect region; and generating a gating coefficient according to the environment data, constructing a time sequence model based on the gating coefficient and the time sequence data, performing quantitative prediction on the future evolution trend of the defect area, and outputting a prediction result. The method can realize millimeter-level defect tracking and periodic-level risk early warning, and is suitable for structural safety monitoring of key components such as bridges, tunnels, curtain walls and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of building material identification, and specifically to a method and system for identifying building material defects. Background Art

[0002] With the expansion of the scale of infrastructure and the extension of service life, structural defect problems such as aging, cracks, and spalling of building materials have become increasingly prominent. Especially in key components such as bridges, tunnels, and curtain walls, surface defects not only affect structural safety but also pose a great pressure on long-term operation and maintenance costs. In order to reduce the cost of manual inspection and improve the scientificity of maintenance decisions, more and more monitoring systems have tried to introduce image recognition and trend prediction means to digitally model and risk assess the defect evolution process.

[0003] Existing defect identification and prediction methods generally rely on single image registration or univariate time series modeling, and there are two key deficiencies: firstly, the alignment accuracy between images is limited by the viewing angle, illumination, and non-rigid deformation of the structure, resulting in the accumulation of defect tracking errors and affecting the quality of time series; secondly, prediction models often ignore the regulatory effects of external environmental factors such as temperature, humidity, or load changes on crack evolution, resulting in model response lags or misjudgment of risk levels. Therefore, there is an urgent need for a comprehensive defect identification scheme that integrates a high-precision registration mechanism and an environmental perception prediction model to improve the accuracy of identification and the effectiveness of early warning. Summary of the Invention

[0004] In view of the above problems, the present invention is proposed.

[0005] Therefore, the technical problem solved by the present invention is: the existing defect identification methods have problems of accumulation of defect tracking errors and ignoring the influence of the external environment on building defects.

[0006] To solve the above technical problem, in the first aspect, the present invention provides the following technical solution: a method for identifying building material defects, including: periodically obtaining image data and environmental data of a building to be measured through a collection device, and recording the pose information of the collection device; according to the pose information of the collection device, performing hierarchical grid image registration on the image data of adjacent periods to obtain aligned images; obtaining a defect mask map based on the aligned images, identifying and marking the defect areas according to the defect mask map, comparing the defect areas of this period with those of the previous period, and obtaining time series data of the defect areas; generating a gating coefficient according to the environmental data, constructing a time series model based on the gating coefficient and the time series data, and quantitatively predicting the future evolution trend of the defect areas and outputting a prediction result.

[0007] As a preferred solution of a method for identifying building material defects according to the present invention, wherein: the environmental data includes environmental temperature, humidity, and structural load; The pose information of the acquisition device includes the position and attitude angle of the acquisition device.

[0008] As a preferred solution of a building material defect identification method according to the present invention, wherein: the hierarchical grid image registration includes converting the pose information of adjacent cycles into a homogeneous coordinate transformation matrix, and combining the internal reference parameters of the acquisition device to calculate the initial geometric transformation relationship of the image data in the current cycle relative to the image data in the previous cycle, and obtaining the initial homography matrix , according to to obtain a roughly aligned image ; Divide into equal-sized sub-grids , divide the image data of the previous cycle into the same grids to ensure one-to-one correspondence between sub-grids; Extract feature points in each sub-grid, only allow matching between spatially adjacent corresponding sub-grid regions, estimate the affine matrix through local RANSAC and remove outliers; Construct a joint optimization objective function for all sub-grids, and solve to obtain the final affine transformation parameters of each sub-grid; Based on the final affine transformation parameters of each sub-grid, perform geometric transformation and pixel resampling on a sub-grid-by-sub-grid basis for the image data in the current cycle, and transform the image data in the current cycle to a spatial coordinate system that is strictly aligned with the image data in the previous cycle to obtain an aligned image.

[0009] As a preferred solution of a building material defect identification method according to the present invention, wherein: the joint optimization objective function is based on the energy minimization model of image non-rigid registration, and introduces the initial homography matrix generated by the pose information as a prior guidance term, and uses the sub-grid affine matrix ; wherein, represents the affine matrix of the j-th sub-grid in the current cycle image; represents the pixel point belonging to the j-th sub-grid region; represents the initial homography matrix; represents the image sub-grid region after rough alignment; represents the regularization term weight coefficient; represents the affine matrix of the adjacent sub-grid of Denotes minimizing the joint optimization objective function; Denotes the second norm.

[0010] As a preferred solution of a method for identifying defects in building materials according to the present invention, wherein: the obtaining of the time series data of the defect area includes, based on the aligned images, performing pixel-by-pixel comparison between the image data of the current period and the image data of the previous period to generate a change mask image; Performing connected component analysis on the change mask image, extracting all change areas, and calculating the structural feature indexes of each change area, where the structural feature indexes include area, maximum length, major axis direction, shape complexity, and centroid; Assigning a unique number to each change area and establishing a time series of changes in the structural feature indexes. For each currently detected change area, calculating the spatial overlap rate with the defect area of the previous period to determine whether it is a continuation of the same defect; If the spatial overlap rate value between the current change area and the defect area of the previous period is greater than or equal to the set threshold, it is determined to be a continuation of the defect area of the previous period; If the spatial overlap rate value between the current change area and the defect area of the previous period is less than the set threshold, it is determined to be a newly generated defect and a new number is assigned; Storing the structural feature indexes of each numbered defect in the current period into the database according to the time stamp to obtain the time series data of the defect area.

[0011] As a preferred solution of a method for identifying defects in building materials according to the present invention, wherein: the constructing of the time series model based on the gating coefficient and the time series data includes obtaining the time series data of each numbered defect area, arranging them in ascending order of time stamp to form a main sequence; Synchronously sorting out the environmental data collected in the same period, establishing a sliding window with a fixed length for each type of environmental data, calculating the cumulative statistic of the environmental data within the sliding window, and splicing the environmental data and the cumulative statistic to generate an auxiliary sequence; Based on the LSTM model, introducing an environmental gating mechanism, calculating the dynamic weight through the auxiliary sequence, and controlling the hidden state update process; Introducing a fitting term for the area growth rate in the model training and adding a growth rate error regularization term to the total loss function to construct the total loss function.

[0012] As a preferred solution of a method for identifying defects in building materials according to the present invention, wherein: the quantitatively predicting the future evolution trend of the defect area and outputting the prediction result includes obtaining the evolution trend data of each defect number in the future period according to the time series model, presetting a risk factor set, establishing a multi-factor threshold decision rule in combination with the risk factor set, and outputting the prediction result.

[0013] In a second aspect, the present invention proposes a building material defect recognition system adopting any of the methods of the present invention, wherein: an image processing module periodically acquires image data and environmental data of a building to be measured through an acquisition device, records the pose information of the acquisition device, and performs hierarchical grid image registration on the image data of adjacent periods according to the pose information of the acquisition device to obtain aligned images; a recognition module obtains a defect mask map based on the aligned images, identifies and marks defect regions according to the defect mask map, compares the defect regions of the current period with those of the previous period, and obtains time series data of the defect regions; a prediction module generates a gating coefficient according to the environmental data, constructs a time series model based on the gating coefficient and the time series data, quantitatively predicts the future evolution trend of the defect regions, and outputs prediction results.

[0014] Advantages of the present invention: Through hierarchical grid image registration guided by the camera pose, the present invention significantly improves the pixel-level alignment accuracy of cross-period images, ensuring the continuity of defect region numbers and the accuracy of evolution indicators; The time series prediction model introducing the environmental gating mechanism can dynamically adjust the prediction weights according to environmental disturbances such as temperature and humidity, enhancing the model's response ability to crack mutations; At the same time, combined with the growth rate regularization term, the model is guided to focus on the defect development speed, improving the early risk identification ability; Finally, combining the prediction results, gating intensity and defect trends, multi-factor risk level determination and chart output are realized. The overall solution can achieve millimeter-level defect tracking and cycle-level risk warning, and is applicable to the structural safety monitoring of key components such as bridges, tunnels, and curtain walls, with high precision and engineering practical value. Description of the Drawings

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to these drawings without creative efforts.

[0016] Figure 1 It is the overall flowchart of a building material defect recognition method provided by an embodiment of the present invention. Detailed Embodiments

[0017] To make the above objects, features, and advantages of the present invention more apparent and understandable, the following provides a detailed description of the specific embodiments of the present invention in conjunction with the accompanying drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0018] Example 1, referring to Figure 1 , which is an embodiment of the present invention, provides a method for identifying defects in building materials, including: S1: Periodically obtain image data and environmental data of the building to be measured through a collection device, and record the pose information of the collection device. According to the pose information of the collection device, perform hierarchical grid image registration on the image data of adjacent periods to obtain aligned images.

[0019] Furthermore, through a collection device installed on a drone, a rail vehicle, or a mobile detection arm, periodically obtain image data and environmental data of the target building structure area. The image data is a high-resolution image of the building surface, and the environmental state data includes, but is not limited to, environmental temperature, humidity, structural load, and vibration information during image acquisition, etc., for subsequent defect evolution trend analysis.

[0020] To improve the comparability of image data in the time dimension, record the spatial pose information of the collection device during each acquisition, including position coordinates (obtained through an RTK or GPS module) and attitude angles (obtained through an IMU).

[0021] It should be noted that the spatial pose information is a combination of the position and orientation / rotation angle of the collection device in the world coordinate system, often represented as a six-dimensional vector or a homogeneous transformation matrix. At the building site, image acquisition often has non-fixed angles and non-reproducible positions. Integrating IMU (angle) and RTK (position) can provide an initial geometric estimate, significantly improving the success rate and accuracy of subsequent image registration.

[0022] Furthermore, in terms of image registration, existing registrations mostly rely on pure feature matching and are difficult to solve problems such as large changes in site lighting / occlusion / viewpoint. To achieve high-precision alignment of periodically acquired images, the present invention proposes a hierarchical grid image registration method that integrates camera pose information. By combining camera pose (IMU or RTK) to construct initial geometric constraints and iteratively correct within local rigid sub-grids, it ensures global consistency and avoids feature drift. The specific steps of the hierarchical grid image registration include: In the first step, obtain the spatial pose data corresponding to the current cycle image and the previous cycle image from the acquisition device. By converting the camera poses of two adjacent cycles into a homogeneous coordinate transformation matrix and combining the camera internal parameter, calculate the initial geometric transformation relationship of the current image relative to the previous cycle image, that is, the initial homography matrix. This initial transformation matrix can be regarded as the rough projection relationship of the current image onto the previous cycle image, which has global geometric consistency but does not consider local deformation yet.

[0023] Specifically, parse the original six-dimensional camera pose vector 、 into homogeneous matrices 、 , and calculate the relative pose: ; Among them, represents the relative pose of the acquisition device in cycle and cycle ; represents the six-dimensional camera pose vector in cycle ; represents the six-dimensional camera pose vector in cycle . Assume that the measured building surface can be approximated as a plane , and calculate the initial homography matrix according to the camera internal parameter : ; Among them, represents the relative rotation matrix; represents the relative translation vector; represents the transpose of the matrix of the plane normal; represents the distance from the plane to the acquisition device. and are obtained by decomposing . Use to project the original image in cycle onto the coordinate system of the original image in cycle to obtain the coarsely aligned image .

[0024] In the second step, divide into equal-sized sub-grids of , and divide the image data of the previous cycle into the same grids, so that each sub-grid is roughly in the same position as the corresponding area in the previous cycle image, providing a spatial basis for constructing a local affine transformation model in the following.

[0025] In the third step, within each sub-grid, use SuperPoint or ORB to extract local image feature points (such as corner points and edge points), and perform region-limited feature matching operations. To prevent global misregistration, only allow matching between spatially adjacent corresponding grid regions to obtain an initial set of matching point pairs. Estimate the affine matrix through local RANSAC and remove outliers.

[0026] Fourth, for all sub-grids, construct a joint optimization objective function that includes two parts of constraints: The first part of the constraint makes the affine transformation of each sub-grid as close as possible to the global projection result obtained in the first step; the second part of the constraint ensures a certain smoothness of the affine transformation between adjacent grids to prevent sudden distortions.

[0027] In traditional methods of image non-rigid registration, the optimization objective function usually consists of two parts of energy, including a data term and a regularization term. The data term (DataTerm) is used to minimize the local reprojection error between the registered images to ensure the alignment of the image content; the regularization term (SmoothnessTerm) is used to limit the difference in transformation parameters between adjacent regions to avoid sudden local transformations and maintain the spatial smoothness of the deformation field.

[0028] Based on this basic structure, the present invention constructs a global guidance term in combination with camera pose information, introduces the initial homography matrix generated by the pose information as a prior guidance term, and constructs an affine model at the grid level, using the sub-grid affine matrix to replace the global single transformation and construct a joint optimization objective function, expressed as: ; where represents the affine matrix of the j-th sub-grid in the current cycle image; represents the pixel points belonging to the j-th sub-grid region; represents belonging to the j-th sub-grid region; represents the initial homography matrix; represents the image sub-grid region after rough alignment; represents the regularization term weight coefficient; represents the affine matrix of the adjacent sub-grid of; represents minimizing the joint optimization objective function; represents the second norm.

[0029] By solving this joint optimization problem, the final affine transformation parameters of each sub-grid are obtained. This transformation not only reflects the global relative pose but also can adapt to micro-scale structural changes such as thermal expansion and contraction and local bending.

[0030] In the fifth step, based on the optimized affine transformation parameters of each sub-grid, perform grid-by-grid geometric transformation and pixel resampling on the image of the current period, so as to transform the entire image into the spatial coordinate system that is strictly aligned with the image of the previous period, and obtain the aligned image. The obtained registered image maintains consistency globally and has the ability of elastic deformation in the local area, which can support the subsequent defect evolution analysis at the pixel level.

[0031] It should be noted that since the image acquisition distance of building facades, bridges, etc. is often 10–30m, the pose error given by IMU / RTK (centimeter level, 0.1–0.3° angle deviation) is amplified when projecting the image onto the surface of building components, resulting in pixel-level misalignment > 5px; defects are usually only 2-10px wide, and the misalignment will directly mask the early crack evolution. Therefore, in the third step of feature matching, the entire image is divided into small grids, and only local affine micro-transformations are allowed in each small grid, confining the pose error within the local grid range to prevent amplification. Taking a 16×16 sub-grid as an example, the required operations only involve ≤256 sub-grids, with ≤30 feature points in each grid, achieving centimeter-level pose → sub-pixel-level alignment under low-cost hardware conditions, being compatible with material thermal expansion deformation and occlusion, and continuously outputting the defect evolution curve to meet the requirements of long-term monitoring scenarios.

[0032] S2: Obtain the defect mask map based on the aligned image, identify and mark the defect areas according to the defect mask map, and compare the defect areas of this period with those of the previous period to obtain the time series data of the defect areas.

[0033] Furthermore, based on the aligned image, perform pixel-by-pixel comparison on the image data of the current period and the image data of the previous period to generate a change mask map. Specifically, take the currently registered image and the image of the previous period, use the frame difference method to perform gray difference calculation on the two aligned images to generate a difference map, perform edge detection (such as Canny, Sobel) on the two images respectively, and superimpose their differences onto the difference map to enhance structural changes such as cracks and edge erosion; apply dynamic threshold segmentation to the difference map to obtain a binary mask map, and remove isolated noise points and fill local holes through erosion and dilation operations to form a clear change area.

[0034] Perform connected component analysis on the change mask map, extract all change areas (suspected defect areas), and calculate the structural feature indicators of each change area. The structural feature indicators include but are not limited to area, maximum length, major axis direction, shape complexity, and centroid, and output the structural feature indicators of each change area for subsequent matching and numbering.

[0035] In order to track the evolution state of the same defect in the time dimension, each changed area is assigned a unique number, and a time series of the changes in the structured feature indicators is established. For each currently detected changed area, the spatial overlap rate with the defect area in the previous cycle is calculated to determine whether it is a continuation of the same defect. If the spatial overlap rate value between the current changed area and the defect area in the previous cycle is greater than or equal to the set threshold, it is determined to be a continuation of the defect area in the previous cycle; if the spatial overlap rate value between the current changed area and the defect area in the previous cycle is less than the set threshold, it is determined to be a newly generated defect and is assigned a new number.

[0036] The structured feature indicators of each numbered defect in the current cycle are stored in the database according to the time stamp, and the time series data of the defect area is obtained.

[0037] By performing difference analysis between the aligned images, extracting the defect change areas, and combining the target matching strategy to complete the number management, the evolution trajectory of the defect target in the time dimension is effectively constructed, providing high-precision historical data support for the subsequent evolution trend prediction and risk warning modules. This method has the advantages of high computational efficiency, clear structure, and strong scalability, and is suitable for deployment in systems such as engineering inspection and long-term monitoring.

[0038] S3: Generate a gating coefficient according to the environmental data, construct a time series model based on the gating coefficient and the time series data, quantitatively predict the future evolution trend of the defect area, and output the prediction result.

[0039] Furthermore, using the time series data of the defect area, combined with on-site environmental variables (temperature, humidity, load, etc.), quantitatively predict the future expansion trend of building material defects, and output the risk level and maintenance suggestions. The construction of the time series model based on the gating coefficient and the time series data includes obtaining the time series data of each numbered defect area, arranging them in ascending order of time stamp to form a main sequence 。

[0040] Synchronously organize the environmental data collected in the same cycle. Since the environmental data is the current instantaneous value during image acquisition and does not consider the cumulative exposure effect (such as concrete absorbing water and swelling after a week of high humidity), the model is insensitive to "chronic environmental stress" and the prediction may still lag.

[0041] Therefore, in addition to the original environmental data, a sliding window with a fixed length is established for each type of environmental data, and the cumulative statistic of the environmental data within the sliding window is calculated. The environmental data and the cumulative statistic are spliced to generate an auxiliary sequence 。The cumulative statistics include but are not limited to the temperature / humidity mean, the maximum load peak, and the cumulative load cycle times.

[0042] It should be noted that the cumulative statistic only needs to be calculated by sliding window when ingesting data, with almost no additional overhead on the inference speed. The gated instantaneous + cumulative dual-channel can enable the model to gradually generate a higher gating coefficient for "high humidity for multiple days" or "load fatigue", and amplify the crack propagation signal earlier.

[0043] Furthermore, in the scenario of long-term evolution monitoring of building material defects, environmental conditions are the key driving factors for crack generation and propagation, including: High temperature causes thermal expansion cracks; High humidity causes the material to absorb water and expand, and the coating to peel off; High load or frequent vibration induces fatigue cracks or edge erosion.

[0044] However, traditional time series-based prediction methods (such as LSTM, ARIMA) only consider the historical index changes of the defects themselves and cannot perceive the triggering effect of these external disturbances on the structural changes, resulting in large prediction errors and untimely warnings, which is more obvious when the structural state changes suddenly.

[0045] Therefore, the present invention proposes to introduce an environmental gating mechanism into the time series model. By a set of weight coefficients dynamically calculated from environmental variables, the update amplitude of the model state is controlled, enabling the model to have immediate sensitivity to external abnormal stimuli. Based on the auxiliary sequence The gating generation based on it is expressed as: ; The hidden state correction formula is expressed as: ; Among them, represents the gating coefficient vector; represents the Sigmoid function; represents the learnable weight matrix; represents the bias vector; represents the new hidden state after introducing the environmental gating; represents element-wise multiplication; represents the period The hidden state calculated by traditional LSTM; represents the period The hidden state calculated by traditional LSTM.

[0046] When the environmental variables are stable, the gating coefficient tends to retain the historical state to avoid overfitting; when the external temperature, humidity, and load change suddenly, the gating coefficient increases, prompting the model to update the state quickly and reflect the changes in a timely manner. It should be noted that the sudden change here is not a change in a short period of time (such as seconds, hours, etc.), but a large-scale change in the environment caused by seasonal changes or the influence of bad weather.

[0047] To make the model more sensitive to the rapid expansion of defects in the short term, the present invention introduces a fitting term for the area growth rate in model training: ; wherein, represents the relative growth rate; represents the period of the defect area; represents the period of the defect area.

[0048] Add a growth rate error regularization term to the total loss function: ; wherein, represents the growth rate error regularization term; represents the weight coefficient that controls the contribution ratio of the two losses; represents the predicted growth rate; represents the actual growth rate. Based on the growth rate error regularization term, a mean square error total loss function is constructed.

[0049] Furthermore, according to the time series model, the evolution trend data of each defect number in the future period is obtained. To comprehensively evaluate the defect risk, a risk factor set is preset in advance, where each dimension of the factor is derived from the data results generated by the aforementioned creative points, including but not limited to the predicted area, predicted growth rate, and environmental response index. The predicted area and predicted growth rate are directly obtained according to the time series model, and the environmental response index takes the average value of the gating coefficient to represent the sensitivity of the model to environmental changes.

[0050] Combined with the risk factor set, a multi-factor threshold determination rule is established to output the prediction result. The prediction result includes structured warning information, visual output (charts and maps), and operation and maintenance priority ranking suggestions. The structured warning information is a structured record obtained by organizing the risk level, current prediction indicators, and historical trends of each defect: The visual output includes time trend curves (area, growth rate), risk level heat maps, and distribution maps of defect numbers on the structure.

[0051] The method of the present invention can achieve millimeter-level defect tracking and periodic risk warning, and is applicable to the structural safety monitoring of key components such as bridges, tunnels, and curtain walls, with high precision, high interpretability, and engineering practical value.

[0052] Example 2, In an exemplary embodiment, a building material defect identification system is further provided, including, The image processing module periodically acquires image data and environmental data of the building to be measured through a collection device, records the pose information of the collection device, and performs hierarchical grid image registration on the image data of adjacent periods according to the pose information of the collection device to obtain aligned images.

[0053] The recognition module obtains a defect mask image based on the aligned images, identifies and marks the defect areas according to the defect mask image, and compares the defect areas of this period with those of the previous period to obtain the time series data of the defect areas.

[0054] The prediction module generates a gating coefficient according to the environmental data, constructs a time series model based on the gating coefficient and the time series data, quantitatively predicts the future evolution trend of the defect areas, and outputs the prediction results.

[0055] If the above functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that makes a contribution to the prior art, or a part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0056] The logic and / or steps represented in the flowchart or described in other ways herein, for example, can be considered as a definite sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or used in combination with these instruction execution systems, apparatus, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device.

[0057] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection (electronic device) having one or more wirings, a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable media can even be paper or other suitable media on which the program can be printed, as the program can be obtained electronically, for example, by optically scanning the paper or other media, then editing, interpreting, or otherwise processing it as appropriate, and then storing it in a computer memory.

[0058] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one or a combination of the following techniques known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0059] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A method for identifying building material defects, characterized in that, Including: Periodically obtain the image data and environmental data of the building to be measured through a collection device, record the pose information of the collection device, and perform hierarchical grid image registration on the image data of adjacent periods according to the pose information of the collection device to obtain aligned images; Obtain a defect mask map based on the aligned images, identify and mark the defect areas according to the defect mask map, compare the defect areas of the current period with those of the previous period, and obtain the time series data of the defect areas; Generate a gating coefficient according to the environmental data, construct a time series model based on the gating coefficient and the time series data, quantitatively predict the future evolution trend of the defect areas and output the prediction results.

2. The method for identifying defects of building materials according to claim 1, characterized in that: The environmental data includes environmental temperature, humidity and structural load; The pose information of the collection device includes the position and attitude angle of the collection device.

3. The method for identifying defects in building materials according to claim 2, characterized in that: The hierarchical grid image registration includes: Convert the pose information of adjacent cycles into a homogeneous coordinate transformation matrix, and combine it with the internal parameter of the acquisition device to calculate the initial geometric transformation relationship of the image data in the current cycle relative to the image data in the previous cycle, and obtain the initial homography matrix , according to obtain the coarsely aligned image ; Divide into equal-sized sub-grids , divide the image data of the previous cycle into the same grids, ensuring one-to-one correspondence between sub-grids; Extract within each sub-grid feature points, and only allow matching between corresponding sub-grid regions that are spatially adjacent. Estimate the affine matrix through local RANSAC and remove outliers; Construct a joint optimization objective function for all sub-grids and solve to obtain the final affine transformation parameters of each sub-grid; Based on the final affine transformation parameters of each sub-grid, perform geometric transformation and pixel resampling on a sub-grid-by-sub-grid basis for the image data of the current period, and transform the image data of the current period to the spatial coordinate system that is strictly aligned with the image data of the previous period to obtain aligned images.

4. The method for identifying defects in building materials according to claim 3, characterized in that: The combined optimization objective function is based on the energy minimization model of image non-rigid registration, and introduces the initial homography matrix generated by pose information as a prior guidance term, and uses the sub-grid affine matrix to replace the global single transformation, which is expressed as: ； Among them, represents the affine matrix of the j-th sub-grid in the current cycle image; represents the pixel points belonging to the j-th sub-grid area; represents the initial homography matrix; represents the image sub-grid area after rough alignment; represents the regularization term weight coefficient; represents the affine matrix of the adjacent sub-grid of; represents minimizing the joint optimization objective function; represents the two-norm.

5. The method for identifying defects of building materials according to claim 4, wherein: The obtaining of the time series data of the defect areas includes: Based on the aligned images, perform pixel-by-pixel comparison between the image data of the current period and the image data of the previous period to generate a change mask map; Perform connected component analysis on the change mask map, extract all change areas, and calculate the structural feature indexes of each change area. The structural feature indexes include area, maximum length, main axis direction, shape complexity, and centroid; Assign a unique number to each change area, establish a time series of changes in the structural feature indexes, and for each currently detected change area, calculate the spatial overlap rate with the defect areas of the previous period to determine whether it is a continuation of the same defect; If the spatial overlap rate value between the current change area and the defect area of the previous period is greater than or equal to the set threshold, it is determined to be a continuation of the defect area of the previous period; If the spatial overlap rate value between the current change area and the defect area of the previous period is less than the set threshold, it is determined to be a newly generated defect and a new number is assigned; Store the structural feature indexes of each numbered defect in the current period into the database according to the time stamp to obtain the time series data of the defect areas.

6. The method for identifying defects of building materials according to claim 5, wherein: The constructing of the time series model based on the gating coefficient and the time series data includes: Obtain the time series data of each numbered defect area, arrange them in ascending order of time stamp to form a main sequence; Synchronously sort out the environmental data collected in the same period, establish a sliding window with a fixed length for each type of environmental data, calculate the cumulative statistics of the environmental data within the sliding window, and splice the environmental data and the cumulative statistics to generate an auxiliary sequence; Based on the LSTM model, introduce an environmental gating mechanism, calculate the dynamic weights through the auxiliary sequence, and control the hidden state update process; Introduce a fitting term for the area growth rate in the model training, add a growth rate error regularization term to the total loss function, and construct the total loss function.

7. The method for identifying defects in building materials according to claim 6, characterized in that: The quantitatively predicting the future evolution trend of the defect areas and outputting the prediction results includes: Obtain the evolution trend data of each defect number in the future period according to the time series model, preset the risk factor set, establish a multi-factor threshold judgment rule in combination with the risk factor set, and output the prediction result.

8. A building material defect recognition system is applied to the building material defect recognition method described in any one of claims 1 to 7, and is characterized in that, Including: The image processing module periodically obtains the image data and environmental data of the building to be measured through the acquisition device, records the pose information of the acquisition device, and performs hierarchical grid image registration on the image data of adjacent periods according to the pose information of the acquisition device to obtain the aligned image; The recognition module obtains the defect mask map based on the aligned image, identifies and marks the defect area according to the defect mask map, compares the defect area of this period with that of the previous period, and obtains the time series data of the defect area; The prediction module generates a gating coefficient according to the environmental data, constructs a time series model based on the gating coefficient and the time series data, quantitatively predicts the future evolution trend of the defect area, and outputs the prediction result.

Images