Image recognition-based wall joint defect monitoring method and system, and medium

Abstract

The present application relates to the technical field of image recognition processing, and specifically includes a wall body jointing defect monitoring method and system based on image recognition, and a medium, comprising: bilateral filtering of wall body jointing images, segmentation and extraction of geometric, texture and color features, construction of a multi-scale feature topology process vector, focusing on defects, establishment of a cause graph combining related features, reasoning of cause probability and triggering of monitoring reminders. The technical problems of low contrast of small defects in the wall background, fuzzy defect type distinction and limited wall body jointing defect monitoring accuracy are solved, and the technical effects of extracting jointing geometric, texture and color features through bilateral filtering and adaptive threshold segmentation, combining multi-scale feature topology for dimension alignment, introducing an attention mechanism to effectively focus on small defect areas, accurately reasoning defect cause probability according to the process feature vector, combining the defect type mask and the cause knowledge graph established based on the wall body base features, and improving the wall body jointing defect monitoring accuracy are achieved.

Description

Technical Field

[0001] This invention relates to the field of image recognition and processing technology, specifically to a method, system, and medium for monitoring wall grout defects based on image recognition. Background Technology

[0002] Wall grout sealing is a key process in building decoration projects. Its construction quality directly affects the aesthetics, waterproofing, and durability of the wall. Existing wall grout sealing defects are mostly based on single defect characteristics, with a high rate of missed detection for minor defects. In addition, the detection process only stays at the defect identification level, making it difficult to trace the source of material shortages to the construction process or material characteristics. It also has a weak ability to distinguish grout sealing defects against complex wall backgrounds and cannot meet the needs of fine decoration projects for refined monitoring.

[0003] In summary, existing technologies suffer from technical problems such as low contrast of minor defects against the wall background, unclear distinction of defect types, and limited accuracy in monitoring wall grout defects. Summary of the Invention

[0004] This application provides a method, system, and medium for monitoring wall grout defects based on image recognition, aiming to solve the technical problems in the prior art where small defects have low contrast against the wall background, unclear distinction between defect types, and limited accuracy in monitoring wall grout defects.

[0005] In view of the above problems, the technical solution to achieve the present application is as follows:

[0006] In a first aspect, this application provides a method for monitoring wall grout defects based on image recognition. The method includes: performing bilateral filtering on an image of the wall grout area; extracting grout geometric features, grout texture features, and grout color features through adaptive threshold segmentation; constructing a multi-scale feature topology; aligning the grout geometric features, grout texture features, and grout color features dimensionally; configuring a process feature vector; focusing on minute defect areas using an attention mechanism; outputting defect type masks including missing material, bubbles, cracks, and discoloration; and wall base features under pixel-level outward sampling of defect edges; establishing a causal knowledge graph based on the process feature vector, combined with the defect type mask and wall base features; using the causal knowledge graph to infer the probability of defect causes; and triggering defect monitoring alerts based on the grout quality qualification standards.

[0007] Preferably, a fixed industrial camera is deployed at the end of the construction robotic arm to collect frontal images of the grouting area in real time during the wall grouting operation; when the construction robotic arm moves to the preset acquisition point, the camera focal length is controlled to acquire a local close-up image; the frontal image of the grouting area and the local close-up image are subjected to multi-view fusion processing to obtain the image of the wall grouting area.

[0008] Preferably, the spatial domain standard deviation and gray-level similarity standard deviation of the bilateral filter are set as dynamic parameters. The spatial domain standard deviation is adaptively adjusted according to the width of the seam, and the gray-level similarity standard deviation is positively correlated with the local contrast of the image. The image edge preservation degree before and after bilateral filtering is used. If the image edge preservation degree is lower than the dynamic threshold, the filtering adjustment parameters are iterated again.

[0009] Preferably, by using max pooling and deconvolution operations, the first-scale feature associated with the geometric features of the grout, the second-scale feature associated with the texture features of the grout, and the third-scale feature associated with the color features of the grout are adjusted to the same dimension; the process feature vector is determined by weighted fusion of the first-scale feature, the second-scale feature, and the third-scale feature based on the cosine similarity of the third-scale feature.

[0010] Preferably, a dual-channel attention unit is constructed; wherein, the spatial attention branch extracts edge features through the Sobel operator to generate a spatial weight map, and the channel attention branch calculates feature importance through global average pooling to generate a channel weight map; the spatial weight map and the channel weight map are multiplied to obtain the fused attention weight, and the focus of the tiny defect region is visualized through an attention heatmap.

[0011] Preferably, the defect type mask has a defect area ratio and a defect length distribution; morphological closing operations are used to fill the internal holes of the contour to obtain the defect contour; the ratio of the defect area to the total area of ​​the grout is obtained, and the defect contour is described using Freeman chain code; the encoding characteristics of the defect contour using Freeman chain code are used to convert it into the corresponding defect length distribution.

[0012] Preferably, the causal knowledge graph includes an entity layer and a relation layer; a graph neural network is used to reason about the causal knowledge graph, and the process feature vector is input; at the same time, the defect type mask and wall base features are used for encoding conversion, and then the message passing mechanism of the graph neural network is used to mine the associated features in the entity layer and the relation layer.

[0013] Preferably, the system integrates with the BIM (Building Information Modeling) system, receives grout material parameters including elastic modulus and bonding strength, and maps these parameters to material entity nodes corresponding to the entity layer in the causal knowledge graph. It also evaluates the grout curing rate index based on environmental temperature and humidity data, mapping this index to environmental entity nodes corresponding to the entity layer in the causal knowledge graph. Furthermore, it configures associated edges in the relationship layer of the causal knowledge graph, trains edge weights using historical defect cases, and aggregates adjacent entity nodes corresponding to the associated edges according to the edge weights using a message passing mechanism.

[0014] In a second aspect, this application provides a wall grout defect monitoring system based on image recognition. The system includes: a segmentation and extraction module: performing bilateral filtering on the wall grout area image and extracting grout geometric features, grout texture features, and grout color features through adaptive threshold segmentation; a dimension alignment module: constructing a multi-scale feature topology, dimensionally aligning the grout geometric features, grout texture features, and grout color features, and configuring a process feature vector; a focusing module: focusing on minute defect areas using an attention mechanism, outputting defect type masks including missing material, bubbles, cracks, and discoloration, and wall base features under pixel-level outward sampling of defect edges; and a defect monitoring and alerting module: establishing a causal knowledge graph based on the process feature vector, combined with the defect type mask and wall base features, using the causal knowledge graph to infer the probability of defect causes, and triggering a defect monitoring alert based on the grout quality qualification standard.

[0015] In a third aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps in the above-described image recognition-based wall grout defect monitoring method.

[0016] In summary, one or more technical solutions provided in this application achieve the following technical effects: extracting geometric, texture, and color features of the grout through bilateral filtering and adaptive threshold segmentation; performing dimensional alignment by combining multi-scale feature topology; introducing an attention mechanism to effectively focus on micro-defect areas; and accurately inferring the probability of defect causes based on the process feature vector, combined with the defect type mask and the causal knowledge graph established by the wall base features, thereby improving the technical accuracy of wall grout defect monitoring. Attached Figure Description

[0017] Figure 1 This application provides a flowchart illustrating a method for monitoring wall grout defects based on image recognition.

[0018] Figure 2 This application provides a structural schematic diagram of a wall grout defect monitoring system based on image recognition.

[0019] Figure labeling: Segmentation and extraction module M100, Dimension alignment module M200, Focusing module M300, Defect monitoring and alerting module M400. Detailed Implementation

[0020] Example 1: The present application will be described in detail below with reference to the accompanying drawings, as follows... Figure 1 As shown, this application provides a method for monitoring wall grout defects based on image recognition, wherein the method includes:

[0021] S1: Perform bilateral filtering on the wall grout area image, and extract the grout geometric features, grout texture features, and grout color features through adaptive threshold segmentation; S2: Construct a multi-scale feature topology, align the grout geometric features, grout texture features, and grout color features in dimensions, and configure the process feature vector.

[0022] Specifically, bilateral filtering is an image processing technique that can smooth and denoise images while preserving edge information. By introducing a dual filtering mechanism in both the spatial and grayscale domains, it achieves a good denoising effect. Adaptive threshold segmentation adaptively determines the threshold based on the local grayscale values ​​of the image, dividing the image into different regions to extract target features. The construction of multi-scale feature topology aims to integrate and associate features at different scales to better describe the features of the target object. Dimension alignment adjusts the dimensions of different features to be consistent for subsequent fusion and processing, forming process feature vectors for subsequent causal knowledge graph construction.

[0023] Execution steps: By performing bilateral filtering on the image of the wall grout area, noise interference in the image can be effectively removed while retaining the key edge information of the grout area. This provides a clearer and more accurate image foundation for subsequent feature extraction. For example, when processing complex images containing wall grout, bilateral filtering can distinguish the edge of the grout from the surrounding background, making subsequent feature extraction more accurate. Furthermore, adaptive threshold segmentation technology is used to extract the geometric, texture, and color features of the grout, accurately identifying various feature information of the grout area from the image, providing basic data for subsequent analysis and judgment.

[0024] When processing images of wall grout, adaptive threshold segmentation can automatically adjust the threshold based on changes in the grayscale values ​​of local areas, thereby accurately segmenting the geometric shape, texture details, and color distribution features of the grout. Constructing a multi-scale feature topology, aligning these features of different dimensions, and configuring them into process feature vectors can comprehensively integrate and quantify various feature information of the grout, enabling various features to be analyzed and processed in the same dimension, providing comprehensive and accurate feature input for the subsequent construction of a causal knowledge graph.

[0025] S3: Focus on minute defect areas using an attention mechanism, and output defect type masks including material shortage, bubbles, cracks, and discoloration, as well as wall base features under pixel-level outward sampling of defect edges; S4: Based on the process feature vector, combined with the defect type mask and wall base features, establish a causal knowledge graph, use the causal knowledge graph to infer the probability of defect causes, and trigger defect monitoring reminders according to the quality qualification standards of the grout.

[0026] Specifically, the attention mechanism is a neural network mechanism that simulates human visual attention, automatically focusing on key areas in an image and suppressing interfering information from irrelevant areas, thereby improving the model's ability to perceive specific features; the defect type mask is a binary image used to identify areas with different defect types in an image, such as missing material, bubbles, cracks, and discoloration; pixel-level outward sampling refers to performing pixel-level extended sampling at the defect edges to obtain more complete wall base feature information; the causal knowledge graph is a graph-based data model used to represent and reason about the complex relationships between wall grout defects and various causes, including entity layers and relation layers; the graph neural network is a neural network used to process graph-structured data, using a message passing mechanism to propagate and aggregate features in the graph; the message passing mechanism is the mechanism in the graph neural network used to pass and aggregate information between nodes.

[0027] Execution steps: The attention mechanism precisely focuses on minute defect areas in the image, and then concentrates computing resources to analyze these minute defect areas. Furthermore, when processing a wall grouting image containing bubble defects, the attention mechanism can increase the weight of the bubble area, thus paying more attention to this area in subsequent feature extraction, effectively improving the detection accuracy of bubble defects. At the same time, by obtaining wall base features through pixel-level outward sampling, the relationship between defects and wall base can be more comprehensively reflected, providing richer information for subsequent cause analysis.

[0028] By using process feature vectors as input and combining them with defect type masks and wall base features, a causal knowledge graph containing rich semantic information can be constructed. For example, by associating features such as grout width and texture complexity in the process feature vectors with information such as bubble area ratio and crack length distribution in the defect type mask, the causal knowledge graph can infer that bubble defects generally originate from incomplete grouting during construction, and crack defects are related to insufficient elastic modulus of the material. In the above steps, preferably, using the causal knowledge graph for inference improves the accuracy of judging the causes of wall grout defects. Combined with the grout quality qualification standard to trigger defect monitoring reminders, it can promptly remind construction personnel of the specific defect type and possible causes, facilitating targeted improvement measures and effectively improving the overall quality of wall grouting.

[0029] Furthermore, the method of this application includes bilateral filtering of the image of the wall grout area:

[0030] A fixed industrial camera is deployed at the end of a construction robotic arm to collect frontal images of the grouting area in real time during the wall grouting operation. When the construction robotic arm moves to a preset acquisition point, the camera's focal length is controlled to acquire a local close-up image. The frontal image of the grouting area and the local close-up image are fused together from multiple perspectives to obtain the image of the wall grouting area.

[0031] Specifically, a fixed industrial camera is an image acquisition device with a fixed installation position but adjustable shooting angle and focal length. Deploying it at the end of a construction robotic arm allows the camera to change position and angle as the robotic arm moves to adapt to the shooting needs of different wall grouting areas. Preset acquisition points refer to the robotic arm positions pre-set according to the process requirements of wall grouting operations and the wall structure. When the robotic arm reaches these points, it triggers the camera to acquire close-up images. Multi-view fusion processing refers to fusing images taken from different perspectives to generate a more complete and accurate image of the wall grouting area.

[0032] Execution steps: By deploying a fixed industrial camera at the end of the construction robotic arm, real-time frontal images can be acquired during the wall grouting process, ensuring dynamic monitoring of the grouting process; when the construction robotic arm moves to the preset acquisition point, the camera focal length is controlled to obtain a close-up image of the local area, which can more clearly capture the detailed features of the grouting area. Furthermore, the camera focal length can be adjusted by sending pulse signals through the control card, and the focal length value is linearly related to the number of pulses; when the robotic arm reaches a preset point, a specific number of pulse signals are sent to adjust the camera focal length to a state suitable for shooting close-up images, thereby obtaining high-definition local images.

[0033] Multi-view fusion processing of frontal and close-up images can generate images of wall grouting areas containing rich details and complete scenes. Fusion algorithms typically consider features such as grayscale, texture, and edges of the images, aligning and fusing images from different perspectives through techniques such as feature matching and image registration. For example, using a multi-view fusion algorithm based on feature point matching, feature points in frontal and close-up images can be matched and fused using methods such as weighted averaging to obtain a complete wall grouting image, thereby improving the feature detail retention rate and providing a more reliable image foundation for subsequent bilateral filtering and feature extraction.

[0034] Furthermore, the method of this application also includes bilateral filtering processing on the image of the wall grout sealing area, and further includes:

[0035] The spatial domain standard deviation and gray-level similarity standard deviation of the bilateral filter are set as dynamic parameters. The spatial domain standard deviation is adaptively adjusted according to the width of the seam, and the gray-level similarity standard deviation is based on the positive correlation of local image contrast. The edge preservation of the image before and after bilateral filtering is measured. If the edge preservation of the image is lower than the dynamic threshold, the filtering parameters are iterated and adjusted again.

[0036] Specifically, the spatial domain standard deviation refers to the parameter that controls the influence of pixel spatial distance on filtering during bilateral filtering, while the gray-level similarity standard deviation is the parameter that controls the influence of pixel gray-level value difference on filtering. The image edge preservation degree before and after bilateral filtering refers to the degree to which the image edge information is preserved relative to the original image after bilateral filtering. The dynamic threshold is a threshold set according to the image processing requirements to determine whether the image edge preservation degree meets the requirements.

[0037] Execution steps: The spatial domain standard deviation is adaptively adjusted according to the width of the seam, so that the filtering process is more adaptable to the characteristics of images with different seam widths. For example, when processing images with wider seam widths, appropriately increasing the spatial domain standard deviation can more effectively smooth noise in larger areas; while when processing images with narrower seam widths, decreasing the spatial domain standard deviation can better preserve fine edge information.

[0038] Meanwhile, by setting the gray-level similarity standard deviation based on the positive correlation of local image contrast, the filtering intensity can be dynamically adjusted according to the contrast of local areas of the image. For areas with high contrast, the gray-level similarity standard deviation can be appropriately increased so that the filtering can better preserve edge details; for areas with low contrast, the gray-level similarity standard deviation can be decreased to avoid over-smoothing.

[0039] By evaluating the edge preservation of the image before and after bilateral filtering, it is possible to determine whether the current filtering parameters can meet the requirements of image edge preservation and noise reduction. For example, when the image edge preservation is lower than the dynamic threshold, such as below 0.8, it indicates that the current filtering parameters may cause a significant loss of edge information. In this case, iterating and adjusting the filtering parameters can optimize the filtering effect. In the above steps, by improving the image edge preservation, the edge features of the seam area can be better preserved, providing a clearer and more accurate image basis for subsequent feature extraction and defect detection.

[0040] Furthermore, by dimensionally aligning the geometric features, texture features, and color features of the grout, and configuring a process feature vector, the method of this application includes:

[0041] By performing max pooling and deconvolution operations, the first-scale feature associated with the geometric features of the grout, the second-scale feature associated with the texture features of the grout, and the third-scale feature associated with the color features of the grout are adjusted to the same dimension; the feature vector is determined by weighted fusion based on the cosine similarity of the first-scale feature, the second-scale feature, and the third-scale feature.

[0042] Specifically, max pooling is a downsampling technique in neural networks that reduces the spatial size of a feature map by sliding a window across it and taking the maximum value within the window, while preserving important feature information. Deconvolution is an upsampling technique that restores a low-resolution feature map to a high-resolution one by learning an upsampled representation of the feature map. Cosine similarity is used to measure the similarity of two vector directions by calculating the ratio of the dot product of two vectors to the product of their magnitudes; the closer the value is to 1, the higher the similarity. The process feature vector is a vector representation obtained by fusing the geometric, texture, color, and other multi-dimensional features of the grout, and is used for subsequent cause analysis and defect monitoring.

[0043] Execution steps: The first-scale feature associated with the geometric features of the grout is downsampled by max pooling to extract key geometric information and reduce the amount of data. Then, the second-scale feature associated with the texture features and the third-scale feature associated with the color features of the grout are upsampled by deconvolution to adjust their spatial size to be the same as the first-scale feature after max pooling. For example, if the spatial size of the original geometric feature map is 32×32, it becomes 16×16 after max pooling. Then, the texture feature map (original size 16×16) and the color feature map (original size 8×8) are upsampled to 16×16 by deconvolution.

[0044] The cosine similarity between the first, second, and third scale features is obtained. The features are then weighted and fused according to the similarity. If the cosine similarity between geometric and texture features is 0.7, between geometric and color features is 0.5, and between texture and color features is 0.6, then weights (such as normalized similarity values) can be assigned based on these similarity values. The weights are then multiplied by the corresponding feature maps, and the weighted feature maps are summed to obtain the fused process feature vector. Through the above steps, the fusion accuracy of features at different scales can be improved, making the process feature vector more accurately reflect the real features of the grout area. This provides high-quality feature input for subsequent construction of causal knowledge graphs and defect monitoring.

[0045] Furthermore, by focusing on minute defect areas using an attention mechanism, the method of this application includes:

[0046] A dual-channel attention unit is constructed; wherein, the spatial attention branch extracts edge features through the Sobel operator to generate a spatial weight map, and the channel attention branch calculates feature importance through global average pooling to generate a channel weight map; the spatial weight map and the channel weight map are multiplied to obtain the fused attention weight, and the focus of the tiny defect region is visualized through an attention heatmap.

[0047] Specifically, the dual-channel attention unit refers to a neural network structure that combines spatial attention and channel attention. The spatial attention branch uses the Sobel operator to extract edge features. The Sobel operator is a differential operator used in image processing that highlights edge information by calculating the image gradient, thereby generating a spatial weight map. This spatial weight map emphasizes regions with obvious edges in the image. The channel attention branch calculates feature importance through global average pooling. Global average pooling averages each channel of the feature map to obtain global feature information for each channel, thus generating a channel weight map. This channel weight map reflects the importance of features from different channels. Dot multiplication involves element-wise multiplication of the spatial and channel weight maps to obtain a fused attention weight. This weight integrates information from both spatial and channel dimensions, used to highlight more critical regions in the image. The attention heatmap uses color intensity to represent the magnitude of attention weights; darker colors indicate more concentrated attention, allowing for intuitive focus on small defect areas.

[0048] Execution steps: The spatial attention branch uses the Sobel operator to extract edge features, which can highlight areas in the image where there are edge changes. These areas often contain boundary information of defects. For example, when processing wall grout images containing bubble defects, the edges of the bubbles usually produce obvious grayscale changes. The spatial weight map calculated by the Sobel operator will increase the weight values ​​of these edge areas, making them easier for the model to pay attention to.

[0049] The channel attention branch evaluates feature importance through global average pooling, which can filter out feature channels that contribute more to defect detection from the channel dimension and suppress interference information from irrelevant channels. For example, for wall grout images with prominent texture features, the channel attention branch will calculate that channels related to texture have higher importance weights, while channels that are not sensitive to color changes have relatively lower weights.

[0050] By performing a dot product operation on the spatial weight map and the channel weight map, a fused attention weight is obtained, enabling the model to consider both spatial and channel dimensions comprehensively and more accurately locate micro-defect areas. The attention heatmap is then used for visualization, which areas in the image the model pays more attention to. In the above steps, the use of the dual-channel attention mechanism improves the detection accuracy of micro-defects in wall grout lines.

[0051] Furthermore, the method of this application also includes:

[0052] The defect type mask has the defect area ratio and defect length distribution; morphological closing operation is used to fill the internal holes of the outline to obtain the defect outline; the ratio of the defect area to the total area of ​​the grout is obtained, and the defect outline is described by Freeman chain code; the encoding characteristics of the defect outline by Freeman chain code are used to convert it into the corresponding defect length distribution.

[0053] Specifically, a defect type mask is a binary image used to identify regions with different types of defects in an image. The defect area ratio refers to the proportion of the defect area to the total area of ​​the grout joint. The defect length distribution refers to the length characteristics of the defect contour in different directions. Morphological closing is an image processing operation that first dilates the image and then erodes it to fill small holes inside the contour, making the contour more complete and continuous. Here, the contour refers to the boundary line of the defect area. Filling the holes yields a more accurate defect contour. Freeman chain code is used to describe the encoding method of two-dimensional contour curves. It represents the shape of the contour by tracing the edge of the contour and recording the changes in direction. The defect length distribution is a statistical description of the length characteristics of the defect contour, reflecting the complexity and distribution of the defect shape.

[0054] Execution steps: The defect type mask has two quantitative indicators: defect area ratio and defect length distribution. These indicators can provide specific data support for subsequent defect assessment. For example, when the defect area ratio exceeds 0.05 or the defect length distribution shows that there are long cracks, the defect can be judged to be more serious. Using morphological closing operations to fill the voids inside the contour can make the defect contour more complete and avoid the error in defect area or length caused by contour interruption. For example, when processing wall grout images containing material shortage defects, the defect contour after filling the voids can more realistically reflect the actual size and shape of the material shortage area.

[0055] Obtain the ratio of the defect area to the total grout area. This ratio can serve as an important indicator for assessing the severity of the defect. For example, when the ratio reaches 0.1, it indicates that 10% of the grout area has defects and needs to be addressed promptly. Use Freeman chain codes to describe the defect outline, which can convert the shape information of the outline into a processable digital code, facilitating subsequent analysis and processing. Specifically, by analyzing the frequency of directional changes in the Freeman chain codes, the complexity of the defect outline can be determined. For example, frequent directional changes in the chain codes may indicate that the defect outline is irregular and has many edge fluctuations.

[0056] By utilizing the encoding characteristics of Freeman chain codes to convert them into corresponding defect length distributions, the length features of the defect contour in different directions can be obtained, providing richer information for defect type identification. Specifically, discoloration defects may exhibit a long and smooth contour length distribution, while cracking defects may have a short and complex contour length distribution. Extracting these quantitative features helps improve the accuracy and reliability of wall grout defect monitoring, providing strong support for subsequent defect cause analysis and quality control.

[0057] Furthermore, based on the aforementioned process feature vector, combined with the defect type mask and wall base characteristics, a causal knowledge graph is established. The method of this application includes:

[0058] The causal knowledge graph includes an entity layer and a relation layer; a graph neural network is used to reason about the causal knowledge graph, and the process feature vector is input; at the same time, the defect type mask and wall base features are used for encoding conversion, and then the message passing mechanism of the graph neural network is used to mine the associated features in the entity layer and the relation layer.

[0059] Specifically, the entity layer refers to the set of nodes in the causal knowledge graph that represent different entities (such as grout materials, construction process parameters, defect types, etc.), while the relation layer is the set of edges that represent the relationships between entities (such as causal relationships, association relationships, etc.). A graph neural network is a deep learning model based on graph-structured data, capable of learning and reasoning about entities and relationships in the causal knowledge graph. The process feature vector is a vector representing the characteristics of the wall grouting process after feature fusion processing, serving as the input to the graph neural network. The defect type mask is a binary image that identifies different defect types in the image. The wall base features are feature information of the wall base obtained through pixel-level outward sampling. Encoding conversion refers to transforming this feature information into a form suitable for graph neural network processing. The message passing mechanism is a rule in the graph neural network used to pass and update feature information between entity nodes, thereby mining associated features in the entity and relation layers.

[0060] Execution steps: The entity layer and relation layer of the causal knowledge graph construct a semantic network containing entities such as grout material, construction process, and defect type, as well as their interrelationships. Specifically, the entity layer contains nodes corresponding to grout composition, construction temperature, and bubble defects, while the relation layer contains edges corresponding to the influence of grout composition on bubble defects and the correlation between construction temperature and cracking defects.

[0061] The process feature vector is input into a graph neural network (Graph Neural Network). Based on the structure of the causal knowledge graph, the Graph Neural Network can learn the latent representations of each entity and relationship. Furthermore, through training, the Graph Neural Network can learn the association weights between construction temperature nodes and crack defect nodes, representing the degree of influence of construction temperature on crack defects. Specifically, constructing the Graph Neural Network includes using an existing graph convolutional network framework to quantify the features of entity layer nodes (such as material entity nodes, environmental entity nodes, and defect type nodes) in the causal knowledge graph. For material entity nodes, the elastic modulus (MPa) and bonding strength (N / m²) of the grout material are used as parameters. 2 The features are directly obtained from the BIM model and standardized to values ​​in the range [0, 1]. For environmental entity nodes, the curing rate index (calculated based on temperature and humidity, e.g., 0.8 at 25℃ and 50% humidity) is used as the core feature, combined with construction time (hours) to form a two-dimensional feature vector. For defect type nodes, the defect area ratio and mean defect length distribution are extracted based on the defect type mask, and the defect type (material shortage / bubbles / cracking / discoloration) is identified using One-Hot encoding to form a three-dimensional feature vector. The quantized entity node features are used as input to a graph neural network, with each node corresponding to a feature vector. These features are uniformly mapped to a multi-dimensional feature space through an embedding layer to ensure compatibility of features for different types of entities. Based on the weights of the associated edges in the causal knowledge graph relationship layer, convolutional operations are used to aggregate adjacent node information, such as the aggregation of material and environmental node features for defect type nodes. After mapping the global feature vector through a fully connected layer, the probability distribution of defect causes is output through a softmax function, such as... Among them, z k For the k-th type of cause score output by the fully connected layer, such as material, construction process, environmental impact, and substrate, P(cause) k The probability of this cause is z. m This refers to the raw score (unnormalized value) of the m-th type of defect cause output by the fully connected layer of the graph neural network; the difference between the predicted probability and the actual cause label is calculated using the cross-entropy loss function, and the Adam optimizer is used for iterative training until the loss function converges.

[0062] Simultaneously, after encoding and converting using defect type masks and wall base features, the message passing mechanism of a graph neural network is used to mine associated features at the entity and relation layers. This allows for the combination of the appearance features of defects with the causal relationships in the causal knowledge graph. Specifically, when processing wall grout images containing material shortage defects, based on the association between grouting pressure and material shortage defects in the causal knowledge graph, the feature information of the grouting pressure node is passed to the material shortage defect node through the message passing mechanism. Combined with the features of the material shortage defect in the current image, a comprehensive analysis is performed, thereby more accurately determining whether the material shortage defect is related to insufficient grouting pressure. Through these steps, the accuracy of determining the causes of wall grout defects is improved, providing a more precise analysis of defect causes, which helps to take targeted improvement measures and effectively improve the construction quality of wall grouting.

[0063] The message passing mechanism of a graph neural network is used to mine associated features in the entity and relation layers. Specifically, the process feature vector (the dimensional alignment result of the grout geometry, texture, and color features) is used as the core input of the graph neural network. Simultaneously, defect type masks (including defect area ratio and length distribution) and wall base features (obtained through pixel-level outward sampling of defect edges) are encoded and transformed to generate feature vectors adapted to the knowledge graph. A graph convolutional network (GCN) is used as the basic framework, with entity layer nodes (such as material entities, environmental entities, and defect entities) of the causal knowledge graph as network nodes, and relational edges (such as material elastic modulus → cracking defects, environmental humidity → bubble defects) as connections between nodes. The first layer of inference aggregates the features of adjacent nodes of each entity node through convolution operations. For example, the bubble defect node aggregates the features of its associated grout material bonding strength node and environmental humidity node. The calculation formula is as follows: in, The initial features are defined as follows: ω1 and ω2 are edge weights (obtained through training with historical cases), deg() is the node degree, and σ is the ReLU activation function. The second layer of inference performs high-order aggregation on the features output from the first layer, incorporating global information from the process feature vector. For example, the aggregated features of bubble defects are further fused with the looseness features of the grout texture in the process feature vector to output the probability distribution of defect causes. For example, the probability of insufficient material adhesion causing bubbles is 65%, and the probability of excessive environmental humidity causing bubbles is 30%. The final feature vector is converted into causal probabilities using the softmax function. If the probability of a certain cause exceeds the threshold of the grout quality qualification standard, it is determined to be the main cause, and the corresponding monitoring reminder is triggered.

[0064] The entity layer and relation layer are used for feature mining. Further, the entity layer features are used for feature mining. Specifically, the entity layer includes nodes such as material entities (e.g., elastic modulus, bond strength), environmental entities (e.g., curing rate index), and defect entities (e.g., missing material, bubbles). The graph neural network uses a message passing mechanism to mine potential relationships between different entity nodes. For example, when the cosine similarity between the length distribution characteristics (mean 10mm) of the crack defect node and the characteristics of the elastic modulus node of the grout material (0.3MPa, lower than the standard value of 0.4MPa) is higher than 0.8, the graph neural network will strengthen the association weight between the two, revealing the causal relationship of low elastic modulus → cracking. Combined with the material parameters input from the BIM model, the entity node features are bound to actual engineering data. For example, the frequency of crack defects corresponding to materials with an elastic modulus of 0.3MPa in historical cases is 72%, further verifying the association strength.

[0065] The entity layer and relation layer perform feature mining. Furthermore, the weights of the relational edges are optimized. Specifically, the initial weights of the relational edges are set based on historical defect cases. The graph neural network optimizes the weights through iterative training. For example, the initial weight of "ambient humidity > 80% → bubble defect" is 0.6. If 80% of the newly input "bubble defect" cases have ambient humidity > 80%, the edge weight is increased to 0.75. If a case has normal humidity but a material curing rate index < 0.5 and a bubble defect appears, the edge weight of "material curing rate → bubble defect" is strengthened. The edge weights are dynamically adjusted through an attention mechanism to adapt to the feature associations of different scenarios.

[0066] The entity layer and the relationship layer are used for feature mining. Further, cross-layer feature fusion is performed. Specifically, the node features of the entity layer and the edge weights of the relationship layer are fused across layers through the fully connected layer of the graph neural network. For example, the entity feature of material shortage defect is multiplied with the edge weight of "insufficient grouting pressure → material shortage", and then fused with the porosity feature of the wall base layer to output the comprehensive score corresponding to the grouting process. This realizes the collaborative reasoning of entity attributes and relationship strength. In the above steps, the graph neural network aggregates entity features through hierarchical convolution, dynamically optimizes relationship edge weights, and fuses multi-dimensional information across layers to achieve accurate reasoning of the causal knowledge graph. At the same time, it mines the potential correlation features between the entity layer and the relationship layer to improve the monitoring accuracy of wall grout defects.

[0067] Furthermore, by utilizing the message passing mechanism of graph neural networks to perform association feature mining at the entity layer and relation layer, the method of this application includes:

[0068] Integrating with BIM (Building Information Modeling), the system receives grout material parameters including elastic modulus and bonding strength, and maps these parameters to material entity nodes corresponding to the entity layer in the causal knowledge graph. Based on environmental temperature and humidity data, it evaluates the grout curing rate index and maps it to environmental entity nodes corresponding to the entity layer in the causal knowledge graph. In the relationship layer of the causal knowledge graph, it configures associated edges, trains edge weights using historical defect cases, and aggregates adjacent entity nodes corresponding to the associated edges according to the edge weights using a message passing mechanism.

[0069] Specifically, BIM (Building Information Modeling) is a digital model that integrates information from the entire lifecycle of a building project, including multi-dimensional data on building components, materials, and the environment. Elastic modulus and bond strength are important physical performance indicators for grout materials, measuring the material's resistance to deformation and its adhesion to the substrate, respectively. By mapping these material parameters to material entity nodes in a causal knowledge graph, the actual material properties can be connected to the data structure within the causal knowledge graph. The grout curing rate index measures the curing speed of grout under specific environmental conditions and is closely related to environmental temperature and humidity. Mapping this index to environmental entity nodes reflects the impact of environmental factors on grout quality. Configuring associated edges in the relational layer of the causal knowledge graph and training edge weights using historical defect cases quantifies the association strength between different entity nodes. The message passing mechanism is a method in graph neural networks used to update node features. By weighted aggregation of features of adjacent nodes based on edge weights, potential association features between entities can be mined.

[0070] Execution steps: Integration with BIM (Building Information Modeling) enables real-time acquisition of key parameters such as the elastic modulus and bond strength of the sealant. Specifically, according to GB / T 14683-2017 "Silicone and Modified Silicone Building Sealants", the tensile modulus (a type of elastic modulus) of silicone building sealants should be ≤0.4MPa or ≤0.6MPa at 23℃, and the adhesion at a given elongation should be undamaged. These parameters directly affect the quality and durability of the sealant.

[0071] After mapping these parameters to material entity nodes in the causal knowledge graph, the relationship between material properties and defect types can be established. The curing rate index of the grout is evaluated based on environmental temperature and humidity data and mapped to environmental entity nodes, which can reflect the impact of environmental factors on the quality of the grout. Commonly, when the environmental humidity is below 30% or above 80%, the curing rate of the grout will be significantly slowed down, which may lead to defects such as bubbles and cracks.

[0072] By configuring associated edges in the relational layer of the causal knowledge graph and training edge weights through historical defect cases, the graph neural network can learn the association strength between different entity nodes. Specifically, by analyzing a large number of historical defect cases, it was found that the association weight between grout materials with low elastic modulus and bubble defects is relatively high, while the association weight between materials with insufficient bonding strength and cracking defects is relatively high. Through a message passing mechanism, the adjacent entity nodes corresponding to the associated edges are weighted and aggregated according to the edge weights. The influence of material characteristics, environmental factors, and construction technology on defects is comprehensively considered, thereby more accurately inferring the causes of defects and significantly improving the accuracy and reliability of wall grout defect monitoring.

[0073] In summary, the beneficial effects of the embodiments of this application are:

[0074] This application employs bilateral filtering on the wall grout area image, and extracts grout geometric, texture, and color features through adaptive threshold segmentation. A multi-scale feature topology is constructed, aligning the geometric, texture, and color features dimensionally, and configuring process feature vectors. An attention mechanism focuses on minute defect areas, outputting defect type masks (including missing material, bubbles, cracks, and discoloration) and wall substrate features under pixel-level outward sampling of defect edges. Based on the process feature vectors, combined with the defect type masks and wall substrate features, a causal knowledge graph is established. This graph is used to infer the probability of defect causes, and defect monitoring alerts are triggered based on grout quality standards. This application provides a wall grout defect monitoring method, system, and medium based on image recognition. This technology achieves the following results: geometric, texture, and color features of wall grout are extracted through bilateral filtering and adaptive threshold segmentation; dimension alignment is performed by combining multi-scale feature topology; an attention mechanism is introduced to effectively focus on small defect areas; and a cause knowledge graph is established based on process feature vectors, defect type masks, and wall base features to accurately infer the probability of defect causes, thereby improving the technical effect of wall grout defect monitoring.

[0075] Example 2 is based on the same inventive concept as the image recognition-based wall grout defect monitoring method in the previous examples, such as... Figure 2 As shown in the figure, this application provides a wall grout defect monitoring system based on image recognition, wherein the system includes:

[0076] The segmentation and extraction module M100 performs bilateral filtering on the wall grout area image and extracts the geometric features, texture features, and color features of the grout through adaptive threshold segmentation.

[0077] Dimension Alignment Module M200: Constructs a multi-scale feature topology, aligns the geometric features, texture features, and color features of the grout in dimensions, and configures the process feature vector.

[0078] Focusing Module M300: Focuses on tiny defect areas using an attention mechanism, outputting defect type masks including material shortage, bubbles, cracks, and discoloration, as well as wall base features under pixel-level outward sampling of defect edges.

[0079] Defect monitoring and reminder module M400: Based on the process feature vector, combined with the defect type mask and wall base features, a cause knowledge graph is established. The cause knowledge graph is used to infer the probability of defect causes, and a defect monitoring reminder is triggered according to the quality qualification standard of the grout.

[0080] Furthermore, the segmentation and extraction module M100 is used to perform the following method:

[0081] A fixed industrial camera is deployed at the end of a construction robotic arm to collect frontal images of the grouting area in real time during the wall grouting operation. When the construction robotic arm moves to a preset acquisition point, the camera's focal length is controlled to acquire a local close-up image. The frontal image of the grouting area and the local close-up image are fused together from multiple perspectives to obtain the image of the wall grouting area.

[0082] Furthermore, the segmentation and extraction module M100 is also used to perform the following method:

[0083] The spatial domain standard deviation and gray-level similarity standard deviation of the bilateral filter are set as dynamic parameters. The spatial domain standard deviation is adaptively adjusted according to the width of the seam, and the gray-level similarity standard deviation is based on the positive correlation of local image contrast. The edge preservation of the image before and after bilateral filtering is measured. If the edge preservation of the image is lower than the dynamic threshold, the filtering parameters are iterated and adjusted again.

[0084] Furthermore, the dimension alignment module M200 is used to perform the following method:

[0085] By performing max pooling and deconvolution operations, the first-scale feature associated with the geometric features of the grout, the second-scale feature associated with the texture features of the grout, and the third-scale feature associated with the color features of the grout are adjusted to the same dimension; the feature vector is determined by weighted fusion based on the cosine similarity of the first-scale feature, the second-scale feature, and the third-scale feature.

[0086] Furthermore, the focusing module M300 is used to perform the following method:

[0087] A dual-channel attention unit is constructed; wherein, the spatial attention branch extracts edge features through the Sobel operator to generate a spatial weight map, and the channel attention branch calculates feature importance through global average pooling to generate a channel weight map; the spatial weight map and the channel weight map are multiplied to obtain the fused attention weight, and the focus of the tiny defect region is visualized through an attention heatmap.

[0088] Furthermore, the focusing module M300 is also used to perform the following methods:

[0089] The defect type mask has the defect area ratio and defect length distribution; morphological closing operation is used to fill the internal holes of the outline to obtain the defect outline; the ratio of the defect area to the total area of ​​the grout is obtained, and the defect outline is described by Freeman chain code; the encoding characteristics of the defect outline by Freeman chain code are used to convert it into the corresponding defect length distribution.

[0090] Furthermore, the defect monitoring and alerting module M400 is used to perform the following method:

[0091] The causal knowledge graph includes an entity layer and a relation layer; a graph neural network is used to reason about the causal knowledge graph, and the process feature vector is input; at the same time, the defect type mask and wall base features are used for encoding conversion, and then the message passing mechanism of the graph neural network is used to mine the associated features in the entity layer and the relation layer.

[0092] Furthermore, the defect monitoring and alerting module M400 is also used to perform the following methods:

[0093] Integrating with BIM (Building Information Modeling), the system receives grout material parameters including elastic modulus and bonding strength, and maps these parameters to material entity nodes corresponding to the entity layer in the causal knowledge graph. Based on environmental temperature and humidity data, it evaluates the grout curing rate index and maps it to environmental entity nodes corresponding to the entity layer in the causal knowledge graph. In the relationship layer of the causal knowledge graph, it configures associated edges, trains edge weights using historical defect cases, and aggregates adjacent entity nodes corresponding to the associated edges according to the edge weights using a message passing mechanism.

[0094] In Example 3, based on the same inventive concept as the image recognition-based wall grout defect monitoring method in the foregoing embodiments, this application also provides a computer-readable storage medium storing a computer program, which, when executed, implements the steps of any one of the methods described in Example 1 above.

[0095] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this specification. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are possible or may be advantageous.

[0096] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0097] This specification and accompanying drawings are merely illustrative examples of this application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from its scope. Therefore, if such modifications and modifications fall within the scope of this application and its equivalents, this application intends to include such modifications and modifications.

Claims

1. A method for monitoring defects in wall grout joints based on image recognition, characterized in that, The method includes: Bilateral filtering is applied to the image of the wall grout area, and the geometric features, texture features, and color features of the grout are extracted by adaptive threshold segmentation. Construct a multi-scale feature topology, align the dimensions of the grout geometric features, grout texture features, and grout color features, and configure the process feature vector, including: By using max pooling and deconvolution operations, the first-scale feature associated with the geometric features of the grout, the second-scale feature associated with the texture features of the grout, and the third-scale feature associated with the color features of the grout are adjusted to the same dimension. The process feature vector is determined by weighted fusion of features based on the cosine similarity of the first scale feature, the second scale feature, and the third scale feature; Focusing on minute defect areas using an attention mechanism, the output includes defect type masks such as material shortage, bubbles, cracks, and discoloration, as well as wall base features under pixel-level outward sampling of defect edges; Based on the process feature vector, combined with the defect type mask and wall base features, a causal knowledge graph is established. The causal knowledge graph is used to infer the probability of defect causes, and defect monitoring reminders are triggered according to the quality qualification standards of the grout.

2. The method for monitoring wall grout defects based on image recognition as described in claim 1, characterized in that, The method for performing bilateral filtering on images of wall grout sealing areas includes: A fixed industrial camera is deployed at the end of the construction robotic arm to collect real-time frontal images of the grouting area during the wall grouting operation. When the construction robotic arm moves to the preset acquisition point, the camera focal length is controlled to acquire a close-up image of the local area; The front view image of the grouting area and the close-up image of the local area are fused from multiple perspectives to obtain the image of the grouting area of ​​the wall.

3. The method for monitoring wall grout defects based on image recognition as described in claim 2, characterized in that, The method further includes performing bilateral filtering on the image of the wall grout sealing area. The spatial domain standard deviation and gray-level similarity standard deviation of the bilateral filter are set as dynamic parameters. The spatial domain standard deviation is adaptively adjusted according to the width of the seam, and the gray-level similarity standard deviation is based on the positive correlation of local image contrast. The image edge preservation is evaluated before and after bilateral filtering. If the image edge preservation is lower than the dynamic threshold, the filtering parameters are iterated and adjusted again.

4. The method for monitoring wall grout defects based on image recognition as described in claim 1, characterized in that, The method of focusing on minute defect areas using an attention mechanism includes: Construct a dual-channel attention unit; Among them, the spatial attention branch extracts edge features through the Sobel operator to generate a spatial weight map, and the channel attention branch calculates feature importance through global average pooling to generate a channel weight map; The spatial weight map and the channel weight map are multiplied to obtain the fused attention weight, and the focus of the tiny defect area is visualized through an attention heatmap.

5. The method for monitoring wall grout defects based on image recognition as described in claim 4, characterized in that, The defect type mask has the defect area ratio and defect length distribution; Use morphological closing operations to fill the holes inside the contour to obtain the defect contour; Obtain the ratio of the defect area to the total grout area, and use Freeman chain code to describe the defect outline; By utilizing the encoding characteristics of Freeman chain codes for defect contours, the corresponding defect length distribution is converted.

6. The method for monitoring wall grout defects based on image recognition as described in claim 5, characterized in that, Based on the aforementioned process feature vector, and combined with the defect type mask and wall base characteristics, a causal knowledge graph is established. The method includes: The causal knowledge graph includes an entity layer and a relation layer; A graph neural network is used to reason about the causal knowledge graph, and the process feature vector is input. Simultaneously, the defect type mask and wall base features are used for encoding conversion. Then, the message passing mechanism of the graph neural network is used to mine associated features in the entity layer and relation layer.

7. The method for monitoring wall grout defects based on image recognition as described in claim 6, characterized in that, The method utilizes the message passing mechanism of graph neural networks to perform association feature mining at the entity layer and relation layer, and includes: Integrating with BIM (Building Information Model), it receives grout material parameters including elastic modulus and bonding strength, and maps the grout material parameters to material entity nodes corresponding to the entity layer in the causal knowledge graph; The curing rate index of the grout is evaluated based on environmental temperature and humidity data, and the curing rate index is mapped to the environmental entity node corresponding to the entity layer in the causal knowledge graph. In the relational layer of the causal knowledge graph, associated edges are configured, edge weights are trained through historical defect cases, and adjacent entity nodes corresponding to the associated edges are weighted and aggregated according to the edge weights through a message passing mechanism.

8. A wall grout defect monitoring system based on image recognition, characterized in that, The system is used to implement the image recognition-based wall joint defect monitoring method according to any one of claims 1-7, wherein the system comprises: Segmentation and extraction module: Performs bilateral filtering on the wall grout area image, and extracts the geometric features, texture features and color features of the grout through adaptive threshold segmentation; Dimension Alignment Module: Constructs a multi-scale feature topology, aligns the geometric features, texture features, and color features of the grout dimensionally, and configures the process feature vector, including: By using max pooling and deconvolution operations, the first-scale feature associated with the geometric features of the grout, the second-scale feature associated with the texture features of the grout, and the third-scale feature associated with the color features of the grout are adjusted to the same dimension. The process feature vector is determined by weighted fusion of features based on the cosine similarity of the first scale feature, the second scale feature, and the third scale feature; Focusing module: Focuses on minute defect areas using an attention mechanism, outputting defect type masks including material shortage, bubbles, cracks, and discoloration, as well as wall base features under pixel-level outward sampling of defect edges; Defect monitoring and alert module: Based on the process feature vector, combined with the defect type mask and wall base features, a cause knowledge graph is established. The cause knowledge graph is used to infer the probability of defect causes, and a defect monitoring alert is triggered according to the quality qualification standard of the grout.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the wall grout defect monitoring method based on image recognition as described in any one of claims 1-7.

Images