A construction engineering quality defect visual identification and diagnosis method

Abstract

The application relates to the field of building engineering detection and computer vision technology and discloses a building engineering quality defect visual identification and diagnosis method, which comprises the following steps: collecting a multi-illumination direction image sequence, separating an initial surface normal graph and an albedo graph by using photometric stereo calculation, inferring a normalized stiffness field graph through a deep neural network, constructing a discrete physical grid to apply a virtual load, solving a mechanical equilibrium equation to calculate a displacement field and performing a differentiable reverse rendering, reversely optimizing network parameters by using the difference between a rendered image and an observed image, and judging defects and outputting diagnosis data based on the optimized stiffness field graph. By introducing a self-supervised closed loop of physical mechanics constraints, the application effectively eliminates the interference of surface stains on detection, realizes quantitative inference from visual features to material constitutive properties, and improves the robustness and physical interpretability of defect identification.

Description

Technical Field

[0001] This invention relates to the fields of building engineering inspection and computer vision technology, specifically a method for visual recognition and diagnosis of quality defects in building engineering. Background Technology

[0002] Concrete and masonry structures dominate building construction. However, with increasing service life, factors such as load, environmental erosion, and material aging inevitably lead to quality defects on building surfaces, including cracks, peeling, and hollow areas. If these defects are not detected and repaired promptly, they can escalate into serious structural safety hazards. Therefore, regular, efficient, and accurate inspection of building surfaces is a crucial step in ensuring project quality and public safety.

[0003] Traditional building surface defect detection mainly relies on manual visual inspection or tapping, which is not only inefficient and labor-intensive, but also heavily influenced by the subjective experience of the inspectors, and poses safety risks in high-altitude operations. In recent years, with the development of computer vision and deep learning technologies, automated detection technology based on two-dimensional image processing has gradually become a research hotspot. This type of technology typically uses convolutional neural networks to extract features from acquired building surface images, and achieves defect classification and localization through supervised learning.

[0004] However, existing visual inspection technologies primarily rely on the grayscale or color information of single two-dimensional images for analysis. This single-modal data processing approach has significant limitations in complex engineering environments. Since the pixel brightness of an optical image is the result of the coupling effect of surface geometry, material reflectivity, and ambient lighting conditions, it is difficult to effectively distinguish between true structural defects (such as cracks and pits) with three-dimensional depth information and two-dimensional surface attachments (such as water stains, oil stains, graffiti, or shadows) based solely on two-dimensional texture features. For example, strip-shaped water stains or shadows are easily confused with cracks in terms of grayscale features, leading to numerous false alarms and severely reducing the reliability of the detection results.

[0005] Furthermore, existing deep learning detection models typically construct a mapping relationship from image pixels to semantic labels. Their outputs only remain at the superficial level of "whether a defect exists" or "the geometric outline of the defect," lacking a deep characterization of the material's internal mechanical properties. In practical engineering assessments, simply knowing the location of a defect is insufficient to comprehensively determine the structural safety. Engineers need to obtain physical and mechanical parameters such as material stiffness reduction and changes in elastic modulus. Current technologies cannot establish a quantitative correlation between visual features and material constitutive properties, making it difficult to directly apply detection data to subsequent structural mechanics calculations and safety assessments, thus limiting the application value of visual inspection technology in deep structural health monitoring. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a visual identification and diagnosis method for quality defects in building engineering. It solves the technical problem that existing detection technologies based on single two-dimensional visual features are easily affected by surface stains, lighting shadows, and material textures, making it difficult to accurately distinguish between non-structural surface attachments and real structural defects. At the same time, it solves the problem that existing technologies lack physical and mechanical interpretability and cannot provide quantitative assessment indicators of defect severity based on material constitutive properties.

[0007] This invention provides a visual recognition and diagnosis method for quality defects in building engineering, which aims to guide visual feature learning through physical and mechanical constraints to achieve accurate identification of structural defects on building surfaces.

[0008] This method mainly includes four processing stages: data acquisition and surface feature decoupling, material constitutive parameter inference, physical consistency verification closed loop, and defect diagnosis and output.

[0009] In the data acquisition and surface feature decoupling stage, the system first acquires a sequence of multiple observation images of the same building surface. This is achieved by controlling the light source array to project incident light at different angles according to a preset time sequence and controlling the image sensor to acquire images synchronously. Subsequently, the system performs photometric stereoscopic calculation on the observation image sequence. This calculation process is based on the Lambertian reflection model, which assumes that the brightness value of an image pixel is equal to the product of the dot product of the surface albedo scalar, the surface unit normal vector, and the illumination direction unit vector. The system constructs an observation brightness vector containing the brightness values ​​of all observation frames and an illumination matrix containing all known illumination direction vectors, establishing a system of linear equations. The least squares method is used to solve this system of linear equations, thereby separating the surface pseudo-normal vector. By calculating the magnitude and normalizing the surface pseudo-normal vector, an albedo map representing the reflectivity of the material surface and an initial normal map representing the surface micro-geometry are obtained, respectively.

[0010] In the material constitutive parameter inference stage, the system utilizes a deep neural network to establish a mapping from visual features to physical properties. The initial normal map and albedo map obtained earlier are concatenated along the channel dimension and fed into the deep neural network as input data. This deep neural network can employ a fully convolutional encoder-decoder structure or a visual Transformer structure, with its output layer connected to a sigmoid activation function. The network regresses to predict the mechanical properties at pixels on the building surface, outputting a normalized stiffness field map with values ​​constrained between 0 and 1. Each pixel value in this map represents the relative elastic modulus of the material at the corresponding spatial location, characterizing the material's ability to resist deformation.

[0011] In the closed-loop stage of physical consistency verification, this invention introduces a differentiable physical simulation mechanism to verify and correct the above stiffness prediction results. First, a discretized physical mesh model is constructed, discretizing the image domain into a physical node mesh, where each pixel position in the observed image sequence corresponds to a node in the mesh. Using the harmonic averaging method, the stiffness of the unit is calculated by combining the numerical values ​​of two adjacent nodes in the normalized stiffness field diagram, and then a sparse global stiffness matrix is ​​assembled.

[0012] Subsequently, a virtual uniformly distributed load vector is applied to the physical mesh model, with the load direction set to be perpendicular to the inward direction of the image plane. A linear equilibrium equation is established that correlates the global stiffness matrix, the displacement field to be solved, and the virtual uniformly distributed load vector. The linear equilibrium equation is then solved for differentiability using a gradient-preserving Jacobi iteration algorithm or a multigrid solution strategy to calculate the displacement field containing the vertical displacement components of all nodes.

[0013] Next, geometric surface updates and differentiable rendering are performed. The calculated displacement field is superimposed on the reference height field to obtain the updated height field. The gradient vectors of the updated height field in the horizontal and vertical directions are calculated using a differentiable difference operator and then normalized to generate a deformation normal map. For each pixel location in the image, the product of the dot product of the corresponding albedo map value, the corresponding vector of the deformation normal map, and the original observed illumination vector is calculated to generate a virtual rendered image.

[0014] Finally, the difference between the virtual rendered image and the observed image is calculated. The pixel-level difference between the virtual rendered image and the observed image is calculated as the photometric reconstruction loss, and the spatial gradient of the normalized stiffness field map is calculated as the stiffness smoothing regularization term. The photometric reconstruction loss and the stiffness smoothing regularization term are weighted and summed to obtain the total loss function. The gradient of the total loss function with respect to the deep neural network parameters is calculated using automatic differentiation, and backpropagation is performed to update the network parameters. This process forces the stiffness distribution of the network output to be able to explain the observed image lighting changes under the constraints of physical laws.

[0015] In the defect diagnosis and output phase, the system receives the normalized stiffness field map optimized in the physical consistency verification closed-loop phase and performs numerical analysis on it to determine defects. A stiffness critical threshold is set, and the normalized stiffness field map is traversed. Pixels with stiffness values ​​less than the threshold are marked as potential defect points, generating a binary mask image. Morphological opening operations are performed on the binary mask image to remove noise and restore the main body boundaries. Subsequently, a connected component labeling algorithm is executed to identify independent defect objects.

[0016] Furthermore, the system classifies and quantifies the defective objects. It calculates the major axis length, minor axis length, and aspect ratio of the defective object. When the aspect ratio is greater than a preset shape threshold, the defect type is determined to be a crack; otherwise, it is determined to be spalling or a hole. Simultaneously, the minimum stiffness value of the defective object's region in the normalized stiffness field map is extracted, and the difference between 1 and this minimum stiffness value is calculated as the mechanical damage index. Finally, a structured data file containing the defect location coordinates, defect type, and mechanical damage index is generated.

[0017] This invention provides a method for visual identification and diagnosis of quality defects in building engineering. It has the following beneficial effects: 1. This invention separates surface normals and albedo through photometric stereoscopic calculation and combines it with a physical consistency verification closed loop, effectively eliminating the interference of surface stains on defect identification. Surface color changes are classified as albedo differences, and the physical closed loop ensures that only areas with geometric depth changes are inferred as low-stiffness areas, thereby reducing the false detection rate caused by water stains, oil stains, or graffiti, and improving the robustness of identification under complex lighting and surface conditions.

[0018] 2. This invention constructs a self-supervised learning loop that includes differentiable physical simulation and reverse rendering, establishes a direct mapping between visual representation and physical mechanical properties, and applies virtual loads to the physical grid and solves the mechanical equilibrium equations. The system uses the physical conservation law to constrain the parameter updates of the neural network, so that the output stiffness field map not only conforms to the visual light and shadow laws, but also has clear physical and mechanical interpretation, overcoming the dependence of traditional pure data-driven methods on a large number of manually labeled samples.

[0019] 3. This invention can output pixel-level normalized stiffness field maps and quantified mechanical damage indices, achieving a leap from qualitative identification to quantitative diagnosis. By analyzing the minimum stiffness value and geometric characteristics of the defective region, this method can not only accurately distinguish different defect types such as cracks and holes, but also provide numerical indicators reflecting the degree of degradation of local mechanical properties of materials, providing accurate data support for the safety assessment and maintenance decisions of building structures. Attached Figure Description

[0020] Figure 1 This is a structural block diagram of the visual recognition and diagnosis system for building engineering quality defects of the present invention; Figure 2 This is a flowchart illustrating the overall process of the visual recognition and diagnosis method for building engineering quality defects according to the present invention.

[0021] Among them, 100 is a data acquisition device; 110 is an image sensor; 120 is a programmable light source array; 130 is a synchronous controller; 200 is a data processing terminal; 210 is a central processing unit; 220 is a memory; 300 is a surface feature decoupling module; 400 is a material parameter inference module; 500 is a physical deformation simulation module; 600 is a rendering consistency verification module; and 700 is a defect judgment module. Detailed Implementation

[0022] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Example 1: Visual Recognition and Diagnosis System and Method for Quality Defects in Building Engineering See attached document Figure 1 This invention provides a visual recognition and diagnosis system for quality defects in building engineering, comprising a data acquisition device 100 and a data processing terminal 200. The data acquisition device 100 and the data processing terminal 200 establish a communication connection through a wired data interface or a wireless communication protocol for transmitting image data and control signals.

[0024] The data acquisition device 100 is configured to acquire a sequence of images of the surface of a building under inspection from multiple illumination directions. The data acquisition device 100 includes an image sensor 110, a programmable light source array 120, and a synchronization controller 130. The image sensor 110 is fixedly mounted at the center of the data acquisition device 100, with its lens optical axis perpendicular to the surface of the building under inspection. The programmable light source array 120 includes a number of... Independent point light source units (e.g.) ),Should The individual point light source units are arranged in a ring or hemispherical shape around the image sensor 110. The synchronization controller 130 is electrically connected to the image sensor 110 and the programmable light source array 120, and is used for control. Each independent point light source unit is turned on sequentially according to a preset timing sequence, and the image sensor 110 is controlled to synchronously acquire one frame of image at the moment each point light source unit is turned on, thereby obtaining an image containing... A sequence of observed images. In this sequence of observed images, each frame corresponds to a known illumination direction vector.

[0025] The data processing terminal 200 includes a central processing unit 210 and a memory 220. The memory 220 stores computer-executable instructions and sequences of observed images transmitted by the data acquisition device 100. The central processing unit 210 is configured to execute the instructions stored in the memory 220 to achieve visual identification and diagnosis of defects in building construction quality. The central processing unit 210 includes a graphics processing unit configured to support tensor parallel computation and automatic differentiation. Functionally, the logic modules running inside the central processing unit 210 include a surface feature decoupling module 300, a material parameter inference module 400, a physical deformation simulation module 500, a rendering consistency verification module 600, and a defect determination module 700.

[0026] The surface feature decoupling module 300 is configured to receive a sequence of observed images and separate the geometric normal information and material albedo information of the building surface based on the principle of photometric stereo vision. For the first image in the sequence of observed images... Frame image, in which From 1 to Integers between [a certain value], any pixel position on the building surface. Brightness value at Following the Lambert reflection model, its mathematical expression is: ; in: Indicates pixel position The surface albedo scalar value at a given location is physically equivalent to the proportion of light energy reflected from the material's surface. Indicates pixel position The unit normal vector at a point describes the microscopic orientation of the surface at that point; Indicates the first The known illumination direction unit vector corresponds to each point light source unit. The surface feature decoupling module 300 constructs the observation vector. and lighting matrix Under the approximate condition of neglecting nonlinear factors such as shadows and highlights, according to the formula To establish an overdetermined system of equations, and considering the numerical stability of matrix inversion, this embodiment employs singular value decomposition (SVD) or Moore-Penrose generalized inverse matrix solving to obtain the intermediate vector. : ; in, express The pseudo-inverse matrix is ​​used to handle ill-conditioned or singular cases that may exist in the lighting matrix.

[0027] Initial normal map Each element in Depend on The albedo map was calculated. Each element in Depend on Calculations show that This represents the L2 norm. Based on photometric principles, this step decouples the original image data into normal features characterizing geometry and albedo features characterizing surface color, laying the foundation for subsequent material analysis that removes the effects of illumination.

[0028] The material parameter inference module 400 is configured to use a deep neural network to infer the initial normal map. Albedo map Feature encoding is performed to infer the spatially varying mechanical properties of the building surface. The material parameter inference module 400 outputs a normalized stiffness field map. The normalized stiffness field diagram It has the same spatial resolution as the input image. Each element in Indicates the position of the corresponding pixel The relative stiffness coefficient at the location has a value range of (0,1). The lower the value, the weaker the material's resistance to deformation at that location.

[0029] In this embodiment, the deep neural network employs a fully convolutional encoder and decoder architecture (e.g., a variant of U-Net). The input layer receives an initial normal map. The tensor formed by stitching together the 3-channel map and the albedo map A (3-channel, if in color) along the channel dimension. The encoder section contains multiple downsampling modules, each consisting of a convolutional layer, a batch normalization layer, and a LeakyReLU activation function, used to extract multi-scale texture and geometric features. The decoder section contains corresponding upsampling modules used to restore spatial resolution and generate pixel-level predictions. The network's output layer is connected to a sigmoid activation function to ensure the output normalized stiffness field map. The values ​​in the expression are strictly constrained to the interval (0,1). Output definition: Representing coordinates The relative elastic modulus of the material at that location.

[0030] ; in: The parameter is The neural network mapping function. This stiffness field diagram. This refers to the distribution of physical parameters to be solved, which is also the direct basis for subsequent defect judgment. The design aims to establish a nonlinear mapping relationship from visual appearance features to potential physical and mechanical properties.

[0031] The physical deformation simulation module 500 is configured to construct a differentiable physical simulation layer based on a discrete mesh. The physical deformation simulation module 500 discretizes the image domain into a physical node mesh and, based on the normalized stiffness field diagram... Assemble the global stiffness matrix For any two adjacent nodes in the grid and The stiffness of the element connecting these two nodes Defined as: ; in: and Representing nodes respectively and nodes The relative stiffness coefficient at the location; The preset reference material constant value depends on the average physical modulus of the object being measured; To prevent small constants with a denominator of zero (e.g., 10) 6 This is used to ensure the stability of numerical calculations. The physical deformation simulation module 500 applies a virtual uniformly distributed load vector in the normal direction of the mesh plane. And solve the linear equilibrium equations to obtain the displacement field. The equilibrium equation is: ; This equation describes the elastic deformation response of material surfaces with different stiffness distributions under virtual external forces. The physical deformation simulation module 500 uses the Jacobi iteration algorithm with a fixed number of iterations to solve the above equilibrium equation, outputting a displacement field containing the vertical displacement components of all nodes. Choosing a fixed number of iterations aims to support the backpropagation of gradients within the deep learning framework, making the entire solution process differentiable.

[0032] The rendering consistency verification module 600 first checks the displacement field. Update surface height field To force the neural network to interpret the observed geometric texture through physical deformation, a reference height field is set. For a smooth reference surface that does not contain details of surface microtexture (e.g., the zero plane), the update formula is: ; in: As a reference height field; For position The vertical displacement at that point; This is the deformation amplification factor, used to enhance the deformation characteristics caused by minute defects. Subsequently, the rendering consistency verification module 600 calculates the updated height field... Calculate the deformation normal diagram And combined with albedo map Illumination direction vector corresponding to the original observation Generate virtual rendered images The formula for generating the formula is: ; The rendering consistency verification module 600 calculates the virtual rendered image. Compared with real observation images The pixel-level differences between the images are used as photometric loss, and the network parameters in the material parameter inference module 400 are updated through the backpropagation mechanism of the central processing unit 210. Based on the principle of "analysis-by-synthesis," when the virtual rendered image approximates the real image, the stiffness field S output by the network is considered to reflect the true physical property distribution. The defect determination module 700 is configured to receive the optimized normalized stiffness field map. and will The median value is less than the preset defect threshold. Connected regions are marked as quality defect regions. The defect determination module 700 generates diagnostic data containing the defect location coordinates and the corresponding stiffness reduction degree.

[0033] See attached document Figure 2 The method provided by this invention establishes a differentiable mapping relationship between visual representation and physical and mechanical properties, and uses the law of physical conservation to constrain the learning process of the neural network. The method mainly includes a data acquisition and surface decoupling step S100, a material constitutive parameter inference step S200, a physical consistency verification closed-loop step S300, and a defect diagnosis and output step S400.

[0034] In the data acquisition and surface decoupling step S100, the system performs a conversion from raw photometric data to intrinsic surface properties. Based on synchronous control logic, the programmable light source array projects incident light at different angles according to a preset time sequence, and synchronously triggers the image sensor each time the illumination changes, acquiring a multi-frame observation image sequence for the same building surface. Subsequently, photometric stereoscopic calculation is performed on this observation image sequence. This step eliminates the influence of external illumination direction and intensity by solving a pixel-level linear equation system, decomposing the image information into two independent components: an initial normal map representing the surface micro-geometry. Albedo diagram representing the reflectivity of a material surface Initial normal map It records the details of the building's surface texture, albedo map. The surface color and stain information were recorded. These two components serve as input data for subsequent physical parameter inference.

[0035] In the material constitutive parameter inference step S200, the system performs a mapping from the visual feature space to the physical parameter space. The initial normal map obtained in step S100 is then used... Albedo map The concatenation is used as the input tensor and fed into a pre-built encoder neural network. This neural network does not directly output defect category labels; instead, it is configured to regress and predict the normalized stiffness coefficients at each pixel on the building surface. The output of this step is a normalized stiffness field map. In the normalized stiffness field diagram In this model, the pixel value represents the degree of mechanical continuity of the material at that location. For defect-free concrete or masonry surfaces, the network's initial output tends to have high stiffness values ​​(close to 1.0); for potential cracked or hollow areas, the network, through subsequent training steps, will shift its output towards low stiffness values ​​(close to 0.0). This step establishes a hypothesis about the physical properties to be verified.

[0036] In the physical consistency verification closed-loop step S300, the system constructs a self-supervised learning loop that includes feedback from forward physical simulation and reverse rendering to correct the stiffness prediction results in step S200. This step specifically includes the following sub-processes: First, a mesh-based simulation of differentiable physical deformation is performed. The system is based on the normalized stiffness field diagram. A discretized physical mesh model is constructed, and a virtual vertical load is applied to the mesh surface. The system solves the mechanical equilibrium equations based on the stiffness matrix to calculate the displacement field of the mesh nodes under the virtual load. In this process, if step S200 predicts that a certain region has a sudden drop in stiffness (i.e., a potential defect), then that region will produce significant displacement deformation under virtual load; if it predicts that a certain region has intact stiffness, then the displacement deformation will be minimal.

[0037] Secondly, inverse differentiable rendering is performed. The system renders the displacement field obtained from the physical simulation. Superimposed on the reference surface height, the new normal map after deformation is calculated using a differential operator. Using the new normal diagram Albedo map obtained in step S100 And the known illumination vectors corresponding to the original observed image are used to synthesize a virtual rendered image. The virtual rendered image The simulation simulated the visual effect that should appear under the current lighting conditions, assuming that the currently predicted stiffness distribution is real.

[0038] Finally, consistency loss calculation and parameter optimization are performed. The virtual rendered image is calculated. Compared with real observation images The photometric error is caused by the presence of cracks and shadows in the real image. If the network predicts high stiffness at that location, the physical simulation will not produce concave deformation, and the rendered image will also lack shadows, leading to an increase in photometric error. The system uses automatic differentiation to calculate the gradient of the photometric error relative to the neural network parameters. The gradient flow propagates backward along the path of "photometric error → rendered image → deformation normal → displacement field → global stiffness matrix → normalized stiffness field → neural network parameters," iteratively updating the weights of the neural network and forcing the network to output a stiffness distribution that conforms to the real physical laws of light and shadow.

[0039] In the defect diagnosis and output step S400, the system is based on the optimized normalized stiffness field diagram. The system makes the final determination based on the normalized stiffness field diagram. A threshold segmentation algorithm is applied. When the pixel stiffness value within a connected region remains below a preset defect threshold... At that time, the area is determined to be a structural defect. This step can effectively distinguish between "surface stains" and "real cracks". For surface stains, step S100 has separated them into an albedo map. In step S300, the photometric consistency can be fitted by albedo without reducing stiffness, thus the final predicted stiffness value remains high and will not be misjudged as a defect. The system finally outputs diagnostic result data including the defect location mask, defect type, and stiffness reduction factor.

[0040] Example 2: Practical Engineering Application Case In a specific engineering application scenario, this invention is used to detect quality defects on the concrete walls, beam and column sides, or basement roof surfaces of in-service buildings. Inspectors mount the data acquisition device 100 on a mobile inspection bracket, handheld inspection terminal, or inspection robot platform, ensuring the optical axis of the image sensor 110's lens faces the building surface to be inspected, and arranging the programmable light source array 120 around the image sensor 110. During inspection, the synchronization controller 130 controls multiple point light sources in the programmable light source array 120 to be turned on sequentially according to a preset time sequence, and controls the image sensor 110 to acquire one image frame each time a light source is turned on, thereby obtaining a sequence of observation images from multiple illumination directions for the same building surface area.

[0041] After receiving the observed image sequence, the data processing terminal 200 processes it according to... Figure 2The method flow shown is as follows: First, the surface feature decoupling module 300 performs photometric stereoscopic calculations on multiple frames of observed images to obtain an initial normal map and an albedo map. Non-structural visual disturbances on the building surface, such as water stains, oil stains, graffiti, color differences, or shadows, are mainly reflected in local variations in the albedo map; areas with geometric depth variations or mechanical continuity anomalies, such as cracks, holes, peeling, and hollow edges, are mainly reflected in abnormal variations in the initial normal map and the subsequent normalized stiffness field map.

[0042] Subsequently, the material parameter inference module 400 inputs the initial normal map and albedo map into the deep neural network and outputs a normalized stiffness field map. The physical deformation simulation module 500 constructs a discretized physical mesh based on the normalized stiffness field map and applies a virtual uniformly distributed load perpendicular to the surface of the building to be detected, solving for the displacement field. The rendering consistency verification module 600 generates a deformation normal map based on the displacement field and combines it with the albedo map and illumination direction to generate a virtual rendering image. By comparing the difference between the virtual rendering image and the actual observed image, the network parameters of the material parameter inference module 400 are updated.

[0043] During the defect diagnosis phase, the defect determination module 700 performs threshold segmentation on the optimized normalized stiffness field map. When the normalized stiffness value of a certain region is lower than the preset stiffness critical threshold, the region is marked as a potential structural defect region; when a region only shows significant changes in the albedo map and the normalized stiffness value is not lower than the stiffness critical threshold, the region is determined to be a non-structural surface attachment and is not output as a structural quality defect. For low-stiffness connected regions with a large aspect ratio, the defect determination module 700 determines them as cracks; for low-stiffness connected regions with a small aspect ratio and a large area, they are determined to be spalling or holes.

[0044] Therefore, this embodiment can distinguish between surface stains and actual structural defects in actual building engineering inspections, and output the defect location coordinates, defect type, minimum stiffness value and mechanical damage index, thereby providing data basis for subsequent maintenance decisions, engineering quality assessment or structural health monitoring.

[0045] Example 3: Specific Implementation Method of Physical Consistency Verification and Defect Diagnosis Based on Spring-Mass Mesh This step aims to separate intrinsic surface properties independent of illumination from illumination-affected observation images. The data processing terminal receives... Frame observation image sequence ,in Each frame of image Corresponding to a pre-defined illumination direction vector Light direction vector The calibration process is as follows: Before the data acquisition device 100 leaves the factory or is used, a standard high-reflectivity sphere (such as a chrome sphere) is placed at the center of the field of view; the light sources are turned on sequentially, and by extracting the pixel coordinates of the highlights on the surface of the sphere, combined with the sphere radius and the coordinates of the sphere center, the precise unit direction vector of each light source relative to the optical center of the camera is calculated. This calibration data is stored in the memory 220 for subsequent calculations.

[0046] The system assigns coordinates to each pixel in the image. Perform photometric stereo calculation. Construct the observed brightness vector. ,in Constructing the illumination matrix , of which Behavior Based on the Lambert reflection model, establish a system of linear equations: ; in: Let be the pseudo-normal vector of the surface to be solved, which contains information about the surface albedo and geometric normals. Since... The system of equations can be solved using the least squares method: ; In obtaining vectors Then, the system separates the albedo using the following formula. and unit normal vector : ; ; The above calculations are based on the geometric properties of vector algebra, ensuring that the decomposed normal vectors are unit vectors. The system iterates through all pixel coordinates, and the final output resolution is... Initial normal map and resolution (or Albedo map (depending on whether it is a color camera) .

[0047] The network is configured to map visual features to a physical stiffness field. The network input layer receives an initial normal map. Albedo map Tensors spliced ​​along the channel dimension ,in This represents the number of channels in the albedo map. The main network employs a fully convolutional encoder-decoder structure. The encoder section contains multiple downsampling modules, each consisting of a convolutional layer, a batch normalization layer, and a LeakyReLU activation function, used to extract multi-scale texture features. The decoder section contains corresponding upsampling modules used to restore spatial resolution. The network's output layer is connected to a sigmoid activation function to ensure the output normalized stiffness field map. The values ​​in the expression are strictly constrained to the interval (0,1). Output definition: Representing coordinates The relative elastic modulus of the material at that location.

[0048] ; in: The parameter is The neural network mapping function. This stiffness field diagram. This refers to the distribution of physical parameters to be solved, which is also the direct basis for subsequent defect judgment.

[0049] This step transforms the stiffness field output by the neural network into a physical displacement field, and the entire calculation process remains differentiable.

[0050] Mesh discretization and stiffness matrix assembly: The system divides the image domain into a regular quadrilateral grid, with each pixel... This corresponds to a node in the mesh. For connected nodes in the mesh... and nodes The spring element, its element stiffness coefficient The stiffness is calculated from the predicted values ​​of the two nodes. This embodiment uses the harmonic averaging method to define the element stiffness to enhance sensitivity to low-stiffness regions (defects). ; in: The preset reference material constants, To prevent small constants with a denominator of zero (e.g.) This ensures numerical stability when the stiffness value approaches 0. Based on the element stiffness coefficients, the system assembles a global stiffness matrix. ,in This represents the total number of nodes. Global stiffness matrix. For a sparse matrix, its diagonal elements off-diagonal elements (when for If it is a neighboring node, then it is 0; otherwise, it is 0.

[0051] Virtual loads and differentiable solutions: The system defines a virtual load vector. All its elements are set to constants. (For example This simulates a uniformly distributed pressure perpendicular to the surface. This virtual load vector... This does not refer to the actual gravity or external environmental load on the building surface, but rather a mathematical probe signal introduced to detect the local stiffness characteristics of materials. The direction of this load is always defined as an inward direction perpendicular to the image plane, regardless of the actual angle of the camera relative to the direction of gravity when taking the picture. Its physical meaning lies in simulating "the response that each point on the surface should produce under a unit pressure," thereby inferring material properties through differences in response.

[0052] Establish the linear equilibrium equation: To implement backpropagation within a deep learning framework, this embodiment does not employ a non-differentiable direct inversion algorithm, but instead uses a fixed-step Jacobian iteration method as the differentiable solver. The matrix... Decompose into a diagonal matrix Off-diagonal residual matrix (Right now Displacement field The iterative update formula is: ; Iteration initial value Set it to zero vector. The system performs a fixed number of iterations T (e.g., T=10) and outputs the final displacement field. The displacement field Preservation of information about the input stiffness field The computational graph supports automatic gradient calculation. Considering that high-resolution images (such as 4K resolution) can lead to an excessively large global stiffness matrix dimension, thus affecting the convergence speed of the Jacobian iteration, this embodiment further employs a multi-grid solution strategy. Specifically, the physical deformation simulation module 500 first processes the input stiffness field... Downsampling is performed to solve the physical equilibrium equations on a low-resolution grid to obtain the rough displacement field. Subsequently, bilinear interpolation is used to upsample the rough displacement field as the initial solution for the high-resolution grid. This strategy ensures accurate propagation of physical gradients while reducing the number of iterations required to reach equilibrium. .

[0053] This step supervises the learning of the stiffness field by verifying the visual consistency after physical deformation.

[0054] Geometry surface update: The system utilizes displacement field Update the surface height information. Establish a reference height field. The updated height field is the zero plane (or the initial height obtained from the photometric solid integral). for: ; in: It is the deformation amplification factor, used to amplify microscopic physical deformations to the visible range.

[0055] Normal recalculation: Based on the updated height field The normal map after deformation is calculated using a differentiable central difference operator. First, calculate the unnormalized gradient vector. : ; ; ; Then normalization is performed: ; Image re-rendering and loss calculation: Albedo map obtained by surface feature decoupling in this embodiment and the original observed illumination vector Combined with the updated normal map Generate virtual images : ; The system calculates the total loss function. The loss function includes photometric reconstruction loss. and stiffness smoothing regularization term : ; ; ; in, This is a weighting coefficient, typically ranging from 0.01 to 0.1, used to balance the data fitting term and the prior constraint term. This ensured consistency between physical assumptions and visual observations. This is based on the physical prior that material distribution usually has spatial continuity.

[0056] By minimizing the total loss The system calculates the gradient. And update the network parameters. This process teaches the network to distinguish: only when the shadow changes in the image are caused by geometric deformation (i.e., displacement due to reduced stiffness) will it update the network parameters. The predicted value is low; if the surface itself turns black (due to...). (Explanation), then Maintain high values. Specifically, network parameters. The update employs an adaptive moment estimation optimizer. During the training phase, the initial learning rate is set to... A cosine annealing strategy is employed to gradually reduce the learning rate during training. To prevent gradient explosion during physical simulation, the displacement field is adjusted during backpropagation. The gradient is truncated, and the gradient norm threshold is set to 1.0. Furthermore, to accelerate network convergence in the initial stage, a warm-up phase can be pre-defined. In this iteration, the physical simulation layer is temporarily frozen, and only the albedo map is used. and initial normal map A general autoencoder based on photometrics is trained, and then connected to the physical layer for end-to-end joint fine-tuning.

[0057] The defect determination module 700 receives the normalized stiffness field diagram, optimized by physical consistency verification, output from the material parameter inference module 400. This image is a single-channel floating-point matrix with the same dimensions as the original observed image. The value of each element in the matrix It represents the mechanical integrity of the corresponding spatial location.

[0058] The defect determination module 700 first performs binarization segmentation of the defect region. A stiffness critical threshold is then set. (For example This value is based on the normalized modulus of elasticity reduction limit of a large number of standard concrete test blocks. This threshold is based on the allowable modulus of elasticity reduction limit of a reference material (such as C30 concrete). Module traversal. Generate a binary mask image from all pixels in the array. : ; In binary mask image In the diagram, the region with a value of 1 is the potential "mechanical defect region".

[0059] Here, based on the aforementioned physical deformation simulation mechanism of this invention, for regions with only surface color changes (such as water stains, oil stains, graffiti) but continuous surface geometry, since they have been separated in the albedo decoupling stage and cannot generate consistent shadows conforming to the lighting model through stiffness reduction in the physical simulation stage, the neural network will predict the stiffness value of the region when it converges. The value is close to 1.0. Therefore, this thresholding operation automatically filters out unstructured visual noise, retaining only defects with geometric depth and mechanical significance.

[0060] To eliminate single-pixel noise interference, the defect detection module 700 then... Perform morphological opening. Define a 3×3 structuring element, first perform erosion to remove isolated noise points, then perform dilation to restore the boundaries of the defective main body, resulting in the denoised mask image. The reason for choosing a 3×3 structuring element is that it can effectively smooth sensor noise at the single-pixel level while preserving the skeletal features of tiny cracks. Defect geometric feature analysis and type classification: 700 pairs of defect detection modules The connected component labeling algorithm is executed to identify each independent connected region in the graph as an independent defect object. ( ,in (Total number of defects detected).

[0061] For each defective object The module calculates its geometric feature parameters, specifically including: Area: The total number of pixels within a connected region.

[0062] Major and minor axis lengths: By calculating the eigenvalues ​​of the inertia matrix of the connected region, an equivalent ellipse is fitted, thus obtaining the major axis length. and minor axis length .

[0063] Aspect ratio: defined as .

[0064] Minimum stiffness value: Extracted from the original stiffness field of this region. The minimum value in is denoted as .

[0065] Based on the above geometric features, the defect determination module 700 executes the classification logic: like (For example ),and If the length exceeds a preset threshold, the object is considered defective. The type is crack.

[0066] like Determine the defective object The types are peeling or holes.

[0067] Defect severity quantification and result output: The data processing terminal 200 encapsulates the analysis results into standardized diagnostic report data.

[0068] First, calculate each defective object. Mechanical damage index. It is not based on visual grayscale, but directly on inferred physical parameters: ; The closer the value is to 1.0, the more severe the loss of material stiffness at that location (such as through cracks); the closer the value is to 0, the less severe the damage (such as surface microcracks).

[0069] The final output consists of the following two parts: Visualization of the mapping layer: The system generates a result image in RGB format. This image is based on the original observation image. A semi-transparent heatmap is overlaid on top of the base map. The color mapping of the heatmap is based on the normalized stiffness field map. ,in The area is displayed in red (representing high risk). The identified areas are displayed as transparent or green (representing safety). Simultaneously, a bounding box is drawn around the identified defective object, with the defect type and label next to the box. Numerical value.

[0070] Structured data file: The system generates a JSON or XML file containing detailed information on all defects for subsequent engineering archiving or maintenance decisions. Data field definitions are as follows: DefectID: A unique identifier for a defect.

[0071] Type: Enumeration values ​​("CRACK", "SPALLING").

[0072] Location: The position of the defect centroid in the image coordinate system. .

[0073] Geometry includes area (number of pixels), length (number of pixels), and average width (number of pixels).

[0074] Physics: Includes minimum stiffness value and mechanical damage index .

[0075] Confidence: The reciprocal of the final photometric loss value based on the physical consistency verification stage, representing the confidence level of the diagnosis.

[0076] In this way, this embodiment not only outputs the visual location of the defect, but also provides a quantitative risk assessment index based on physical constitutive parameters, providing mechanically interpretable data support for the health monitoring of building structures.

[0077] Example 4: Alternative Implementation Based on Finite Element Model In Example 3, a simplified spring-mass mesh model was used for the physical layer. In this example, the physical deformation simulation module 500 was configured to perform more refined mechanical modeling using a finite element method based on continuum mechanics.

[0078] In this embodiment, image pixels are mapped to four-node rectangular isoparametric cells. For each cell in the mesh... Its element stiffness matrix It is no longer determined solely by the scalar spring constant, but by the geometric matrix of the element. (Strain-displacement matrix) and material constitutive matrix The result was obtained through integration: ; Where: material constitutive matrix The normalized stiffness field is directly controlled by the material parameter inference module 400. Assuming the building surface material is an isotropic linear elastic material and is under plane stress, then the corresponding pixel... The constitutive matrix at a given location is defined as: ; in: The baseline elastic modulus; Poisson's ratio of the material; This is the relative stiffness coefficient predicted by the neural network.

[0079] In this alternative, the forward computation process involves all element stiffness matrices Assembled into a global stiffness matrix and solve Since matrix assembly and matrix multiplication are both differentiable operations, this alternative scheme still maintains the end-to-end gradient propagation characteristics, allowing the network to learn more complex mechanical behaviors, such as shear deformation and lateral deformation caused by the Poisson effect.

[0080] Example 5: Alternative rendering implementation method for high-gloss architectural surfaces In Examples 1 to 3, the rendering consistency verification module 600 uses the Lambertian diffuse reflection model. However, for ceramic tiles, polished stone, or varnished building surfaces, their optical properties include specular reflection components.

[0081] In this alternative embodiment, the rendering consistency verification module 600 is configured to perform differentiable rendering using a Brin-Von der Vélez lighting model. In this case, the surface feature decoupling module 300 and the material parameter inference module 400 are configured to additionally output a roughness map. ) and specular reflectance diagram ( Virtual rendering image The calculation formula is extended to a linear superposition of the diffuse reflection component and the specular reflection component: ; ; ; in: It is a half-angle vector; This is the gaze vector (usually perpendicular to the image plane). The exponent term... From roughness diagram Transformed from (e.g.) This extended model ensures that the system can still converge correctly when dealing with materials with specular properties, avoiding misinterpreting specular highlights as photometric anomalies caused by geometric deformation.

[0082] Example 6: An Alternative Network Implementation Based on Vision Transformer In Embodiments 1 to 3, the material parameter inference module 400 employs a fully convolutional network. To better handle long cracks or large-area voids that span the entire field of view, this alternative embodiment uses a VisionTransformer architecture based on a self-attention mechanism as the backbone network.

[0083] In this embodiment, the input tensor First, the image is segmented into fixed-size tiles, then linearly projected into sequence vectors and embedded into a Transformer encoder. The encoder contains multiple layers of multi-head self-attention modules. For the first... element Its update rules involve the aggregation of global context information: ; in: , , These are the query, key, and value matrices, respectively.

[0084] Through an attention mechanism, the network can capture the dependencies between pixels that are far apart in an image. For example, when one end of a crack is occluded, the network can infer the stiffness reduction of the occluded region based on the texture features of the other end of the crack and the global stress concentration pattern. The decoder then reshapes the sequence vectors into a two-dimensional feature map and upsamples it to the original resolution to output the stiffness field. This scheme offers superior feature representation capabilities when dealing with structural defects with long-range correlations.

Claims

1. A method for visual identification and diagnosis of quality defects in building engineering, characterized in that, include: Data acquisition and surface feature decoupling steps: control the light source array to project incident light at different angles according to a preset time sequence, control the image sensor to synchronously acquire multiple frames of observation image sequences for the same building surface, perform photometric stereoscopic calculation on the observation image sequence, and separate the initial normal map representing the surface micro-geometry and the albedo map representing the material surface reflectivity. Material constitutive parameter inference steps: The initial normal map and the albedo map are input into the deep neural network as input data. The deep neural network is used to regress and predict the mechanical properties at the pixel points on the building surface, and the normalized stiffness field map is output. Physical consistency verification closed-loop steps: Construct a discretized physical mesh model based on the normalized stiffness field map, apply a virtual uniformly distributed load vector to the physical mesh model, calculate the displacement field by solving the mechanical equilibrium equation, update the surface height according to the displacement field and calculate the deformation normal map, generate a virtual rendering image by combining the deformation normal map and the albedo map, calculate the difference between the virtual rendering image and the observed image sequence, and update the network parameters of the deep neural network in reverse. Defect diagnosis and output steps: Receive the normalized stiffness field map optimized by the physical consistency verification closed-loop step, perform numerical analysis on the normalized stiffness field map, determine the location of the defect area, and generate diagnostic data.

2. The method for visual identification and diagnosis of quality defects in building engineering according to claim 1, characterized in that, In the data acquisition and surface feature decoupling step, the photometric stereo calculation process includes: Based on the Lambert reflection model, it is determined that the image pixel brightness value is equal to the product of the surface albedo scalar and the dot product of the surface unit normal vector and the illumination direction unit vector; Construct an observation brightness vector containing brightness values ​​of all observation frames and an illumination matrix containing all known illumination direction vectors; Establish a system of linear equations that includes the observed brightness vector, the illumination matrix, and the surface pseudo-normal vector to be solved. Solve the system of linear equations using the least squares method to obtain the surface pseudo-normal vector. The albedo map is obtained by calculating the magnitude of the pseudo-normal vector of the surface, and the initial normal map is obtained by normalizing the pseudo-normal vector of the surface.

3. The method for visual identification and diagnosis of quality defects in building engineering according to claim 1, characterized in that, In the material constitutive parameter inference step, the deep neural network is configured as a fully convolutional encoder-decoder structure or a visual Transformer structure. The input layer of the deep neural network receives a tensor formed by splicing the initial normal map and the albedo map along the channel dimension; The output layer of the deep neural network is connected to a sigmoid activation function, and the output value is constrained to the normalized stiffness field map between 0 and 1. Each pixel value in the normalized stiffness field diagram represents the relative elastic modulus of the material at the corresponding spatial location.

4. The method for visual identification and diagnosis of quality defects in building engineering according to claim 1, characterized in that, In the physical consistency verification closed-loop step, the process of constructing the discretized physical mesh model includes: The image domain is discretized into a physical node grid, where each image pixel in the physical node grid corresponds to a node in the grid. Define the element stiffness connecting adjacent nodes, and calculate the element stiffness using the harmonic average method combined with the numerical values ​​of the two adjacent nodes in the normalized stiffness field diagram. The global stiffness matrix is ​​assembled based on the unit stiffness, and the global stiffness matrix is ​​a sparse matrix.

5. The method for visual identification and diagnosis of quality defects in building engineering according to claim 4, characterized in that, In the closed-loop step of the physical consistency verification, the process of solving the mechanical equilibrium equations includes: All elements in the virtual uniformly distributed load vector are set to constants, and the direction is defined as an inward direction perpendicular to the image plane; Establish a linear equilibrium equation relating the global stiffness matrix, the displacement field to be solved, and the virtual uniformly distributed load vector; The linear equilibrium equations are solved in a differentiable manner using the Jacobi iterative algorithm that preserves gradient information or a multigrid solution strategy, and the output is the displacement field containing the vertical displacement components of all nodes.

6. The method for visual identification and diagnosis of quality defects in building engineering according to claim 1, characterized in that, In the physical consistency verification closed-loop step, the process of calculating the deformation normal map includes: The displacement field is superimposed on the reference height field to obtain the updated height field; The gradient vectors of the updated height field in the horizontal and vertical directions are calculated using a differentiable difference operator; The gradient vector is normalized to generate the deformation normal map.

7. The method for visual identification and diagnosis of quality defects in building engineering according to claim 1, characterized in that, In the physical consistency verification closed-loop step, the process of generating the virtual rendered image and updating the network parameters includes: For each pixel location in the image, the product of the corresponding value of the albedo map, the corresponding vector of the deformation normal map, and the dot product of the original observed illumination vector is calculated to obtain the virtual rendered image. The pixel-level difference between the virtual rendered image and the real observed image is calculated as the photometric reconstruction loss; The spatial gradient of the normalized stiffness field plot is calculated as a stiffness smoothing regularization term. The total loss function is obtained by weighted summation of the photometric reconstruction loss and the stiffness smoothing regularization term. The gradient of the total loss function with respect to the parameters of the deep neural network is calculated using automatic differentiation and backpropagation is performed.

8. The method for visual identification and diagnosis of quality defects in building engineering according to claim 1, characterized in that, In the defect diagnosis and output step, the process of determining the location of the defect area includes: Set a critical stiffness threshold; Traverse all pixels in the normalized stiffness field map, mark pixels with stiffness values ​​less than the stiffness critical threshold as potential defect points, and generate a binary mask map. Perform a morphological opening operation on the binary mask image to remove isolated noise points and restore the boundary of the defect body; A connected component labeling algorithm is applied to the morphologically processed mask image to identify independent connected regions as independent defect objects.

9. The method for visual identification and diagnosis of quality defects in building engineering according to claim 8, characterized in that, The defect diagnosis and output step also includes classifying the defective object by type: Calculate the geometric feature parameters of the defective object, including the length of the major axis, the length of the minor axis, and the aspect ratio; Compare the aspect ratio with a preset shape threshold; When the aspect ratio is greater than the preset shape threshold, the defect object is determined to be a crack. When the aspect ratio is less than or equal to the preset shape threshold, the type of the defective object is determined to be peeling or hole.

10. The method for visual identification and diagnosis of quality defects in building engineering according to claim 8, characterized in that, The defect diagnosis and output step also includes generating quantitative diagnostic indicators: Extract the minimum stiffness value of the region where the defective object is located in the normalized stiffness field diagram; Calculate the difference between 1 and the minimum stiffness value, and define the resulting difference as the mechanical damage index; The output is a structured data file containing the location coordinates of the defective object, the defect type, and the mechanical damage index.

Images