Wall body seam beautifying defect monitoring method and system based on image recognition and medium

Abstract

The invention relates to the related technical field of image recognition processing, in particular to a wall body seam beautifying defect monitoring method and system based on image recognition and a medium, and the method comprises the following steps: carrying out bilateral filtering on a wall body seam beautifying image, segmenting and extracting geometric, texture and color features, constructing a multi-scale feature topological matching process vector, focusing defects, and constructing a cause map by combining related features; reasoning cause probability and triggering monitoring reminding. The technical problems that the contrast ratio of tiny defects in a wall background is low, defect type distinguishing is fuzzy, and the wall body seam beautifying defect monitoring precision is limited are solved, seam beautifying geometric, texture and color features are extracted through bilateral filtering and self-adaptive threshold segmentation, dimension alignment is carried out in combination with multi-scale feature topology, and the seam beautifying defect monitoring precision is improved. An attention mechanism is introduced to effectively focus a small defect area, and the defect cause probability is accurately deduced according to a process feature vector in combination with a cause knowledge graph established by defect type masks and wall base layer features, so that the wall seam beautifying defect monitoring precision is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition processing, and particularly relates to a wall body jointing defect monitoring method and system based on image recognition and a medium. BACKGROUND

[0002] As a key process of building decoration engineering, the construction quality of wall body jointing directly affects the aesthetics, waterproofness and durability of the wall body. The existing wall body jointing defects are mainly aimed at single defect characteristics, and the detection rate of small defects is high. In addition, the detection process only stays at the defect identification level, and it is difficult to trace the defect source to the construction process or material properties. The ability to distinguish jointing defects in complex wall backgrounds is weak, and it cannot meet the needs of fine decoration engineering for fine monitoring.

[0003] In summary, the prior art has the technical problems of low contrast of small defects in the wall background, blurred defect type distinction, and limited wall body jointing defect monitoring accuracy. SUMMARY

[0004] The present application provides a wall body jointing defect monitoring method, system and medium based on image recognition, which aims to solve the technical problems of low contrast of small defects in the wall background, blurred defect type distinction, and limited wall body jointing defect monitoring accuracy in the prior art.

[0005] In view of the above problems, the technical scheme of the present application is as follows:

[0006] In a first aspect, the present application provides a wall body jointing defect monitoring method based on image recognition, wherein the method comprises: performing bilateral filtering processing on a wall body jointing area image, extracting jointing geometric features, jointing texture features and jointing color features through adaptive threshold segmentation; constructing a multi-scale feature topology, aligning the dimensions of the jointing geometric features, jointing texture features and jointing color features, and configuring a process feature vector; focusing on the small defect area with an attention mechanism, and outputting a wall body base layer feature under defect edge pixel-level outer expansion sampling, including defect type mask, such as material shortage, bubbles, cracking and discoloration; based on the process feature vector, combining the defect type mask and the wall body base layer feature, establishing a cause knowledge graph, using the cause knowledge graph to infer the defect cause probability, and triggering a defect monitoring reminder according to the jointing quality qualification standard.

[0007] Preferably, a fixed industrial camera is deployed at the end of a construction robot arm, and real-time jointing area front view images are collected during the wall body jointing operation process; the camera focal length is controlled when the construction robot arm moves to the preset collection point, and a local close-up image is obtained; the jointing area front view image and the local close-up image are subjected to multi-view fusion processing to obtain the wall body jointing area image.

[0008] Preferably, the spatial domain standard deviation of the bilateral filtering is set as a dynamic parameter, which is self-adaptively adjusted according to the width of the joint, and the gray similarity standard deviation is positively correlated with the local contrast of the image; the edge retention degree of the image before and after the bilateral filtering is compared, and if the edge retention degree is lower than a dynamic threshold, the filtering parameters are re-iterated and adjusted.

[0009] Preferably, the first scale feature associated with the joint geometry, the second scale feature associated with the joint texture, and the third scale feature associated with the joint color are adjusted to the same dimension through max-pooling and deconvolution operations; and the feature weighted fusion is performed according to the cosine similarity of the first scale feature, the second scale feature, and the third scale feature to determine the process feature vector.

[0010] Preferably, a double-channel attention unit is constructed; wherein the spatial attention branch extracts edge features through a Sobel operator to generate a spatial weight map, and the channel attention branch calculates feature importance through global average pooling to generate a channel weight map; the spatial weight map and the channel weight map are point-multiplied to obtain a fused attention weight, and the micro-defect area is visualized and focused through an attention heat map.

[0011] Preferably, the defect type mask has a defect area ratio and a defect length distribution; a morphological closing operation is used to fill the internal cavities of the contour to obtain a defect contour; the ratio of the defect area to the total joint area is obtained, and the defect contour is described using Freeman chain code; the coding characteristics of the defect contour are converted into a corresponding defect length distribution using Freeman chain code.

[0012] Preferably, the cause knowledge graph comprises an entity layer and a relationship layer; the process feature vector is input into the cause knowledge graph for reasoning using a graph neural network; at the same time, the defect type mask and the wall base feature are encoded and converted, and then the associated feature mining is performed on the entity layer and the relationship layer using the message passing mechanism of the graph neural network.

[0013] Preferably, the cause knowledge graph is integrated with a BIM (Building Information Modeling) model, receives joint material parameters including elastic modulus and bonding strength, maps the joint material parameters to material entity nodes corresponding to the entity layer in the cause knowledge graph, evaluates a joint curing rate index based on environmental temperature and humidity data, maps the joint curing rate index to environmental entity nodes corresponding to the entity layer in the cause knowledge graph, configures associated edges in the relationship layer of the cause knowledge graph, trains edge weights through historical defect cases, and aggregates adjacent entity nodes corresponding to the associated edges according to the edge weights through the message passing mechanism.

[0014] In a second aspect of the present application, a wall joint defect monitoring system based on image recognition is provided, wherein the system comprises: a segmentation extraction module: performing bilateral filtering processing on a wall joint area image, and extracting joint geometry features, joint texture features and joint color features through adaptive threshold segmentation; a dimension alignment module: constructing a multi-scale feature topology, performing dimension alignment on the joint geometry features, joint texture features and joint color features, and configuring a process feature vector; a focusing module: focusing on a small defect area using an attention mechanism, and outputting a wall base layer feature under defect edge pixel level outer expansion sampling including a defect type mask, a bubble, a cracking, a discoloration and the like; and a defect monitoring reminding module: establishing a cause knowledge graph based on the process feature vector in combination with the defect type mask and the wall base layer feature, reasoning a defect cause probability using the cause knowledge graph, and triggering a defect monitoring reminder according to a joint quality qualified standard.

[0015] In a third aspect of the present application, a computer readable storage medium having a computer program stored thereon is provided, and the computer program is executed by a processor to implement the steps of the wall joint defect monitoring method based on image recognition.

[0016] In summary, one or more technical solutions provided in the present application achieve the technical effect of improving wall joint defect monitoring precision by extracting joint geometry, texture and color features through bilateral filtering and adaptive threshold segmentation, performing dimension alignment in combination with a multi-scale feature topology, effectively focusing on a small defect area by introducing an attention mechanism, accurately reasoning a defect cause probability based on a process feature vector in combination with a defect type mask and a wall base layer feature to establish a cause knowledge graph, and the like. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 A flowchart of the wall joint defect monitoring method based on image recognition is provided for the present application.

[0018] Figure 2 A structural diagram of the wall joint defect monitoring system based on image recognition is provided for the present application.

[0019] Legend of reference signs: segmentation extraction module M100, dimension alignment module M200, focusing module M300, defect monitoring reminding module M400. DETAILED DESCRIPTION

[0020] In Example 1, the present application will be specifically described below in combination with the drawings, as shown in the drawings, the present application provides a wall joint defect monitoring method based on image recognition, wherein the method comprises: Figure 1

[0021] ​S1: bilateral filtering processing is performed on the wall jointing area image, and adaptive threshold segmentation is used to extract jointing geometric features, jointing texture features and jointing color features; S2: a multi-scale feature topology is constructed, and the jointing geometric features, jointing texture features and jointing color features are dimensionally aligned to configure a process feature vector.

[0022] Specifically, bilateral filtering processing is an image processing technique that can smooth and denoise an image while preserving edge information. By introducing a dual filtering mechanism in the spatial and gray domains, a better denoising effect is achieved. Adaptive threshold segmentation is a method of determining the threshold value according to the local gray value of the image to divide the image into different regions and extract the target features. The construction of the multi-scale feature topology aims to integrate and correlate features at different scales to better describe the characteristics of the target object. Dimensional alignment adjusts the dimensions of different features to be consistent for subsequent fusion and processing, forming a process feature vector for subsequent cause knowledge graph construction.

[0023] Execution steps: By performing bilateral filtering processing on the wall jointing area image, noise interference in the image can be effectively removed while preserving the key edge information of the jointing area, providing a clearer and more accurate image basis for subsequent feature extraction. For example, when processing a complex image containing wall jointing, bilateral filtering can distinguish the edges of the jointing from the surrounding background, making subsequent feature extraction more accurate. Furthermore, adaptive threshold segmentation technology is used to extract the geometric, texture and color features of the jointing, accurately identifying various feature information of the jointing area from the image, providing basic data for subsequent analysis and judgment.

[0024] In processing the wall jointing image, adaptive threshold segmentation can automatically adjust the threshold value according to the gray value changes in the local area, accurately segmenting the geometric shape, texture details and color distribution features of the jointing. Constructing a multi-scale feature topology and dimensionally aligning these different dimensional features into a process feature vector can comprehensively integrate and quantitatively represent various feature information of the jointing, allowing various features to be analyzed and processed in the same dimension, providing comprehensive and accurate feature input for subsequent cause knowledge graph construction.

[0025] S3: focusing on the micro-defect area with attention mechanism, outputting a defect type mask including material shortage, bubbles, cracking, discoloration, and wall base features under pixel-level outer expansion sampling of defect edges; S4: based on the process feature vector, combining the defect type mask and wall base features, establishing a cause knowledge graph, using the cause knowledge graph to infer the defect cause probability, and triggering a defect monitoring reminder according to the jointing quality qualification standard.

[0026] Specifically, the attention mechanism is a neural network mechanism that simulates human visual attention, which can automatically focus on the key area in the image and suppress irrelevant information in the non-relevant area, thereby improving the perception ability of the model to specific features; the defect type mask is a binary image used to identify the area of different defect types in the image, such as material shortage, bubble, cracking, discoloration, etc.; the pixel-level external expansion sampling refers to the pixel-level expansion sampling at the edge of the defect to obtain more complete wall base feature information; the cause knowledge graph is a graph structure-based data model used to represent and reason the complex relationship between wall jointing defects and various causes, including entity layer and relationship layer; the graph neural network is a neural network used to process graph structure data, which propagates and aggregates features in the graph through a message passing mechanism; the message passing mechanism is a mechanism used to pass and aggregate information between nodes in the graph neural network.

[0027] The execution step: the attention mechanism precisely focuses on the micro-defect area in the image, and then concentrates the computing resources to analyze the micro-defect area. Further, when processing a wall jointing image containing a bubble defect, the attention mechanism can increase the weight of the bubble area, so as to pay more attention to this area in the subsequent feature extraction, effectively improving the detection accuracy of the bubble defect. At the same time, the pixel-level external expansion sampling obtains the wall base features, which can more comprehensively reflect the relationship between the defect and the wall base, and provide more information for the subsequent cause analysis.

[0028] The process feature vector is taken as the input, combined with the defect type mask and the wall base features, which can construct a cause knowledge graph containing rich semantic information. For example, by associating the jointing width, texture complexity and other features in the process feature vector with the bubble area ratio, cracking length distribution and other information in the defect type mask, the cause knowledge graph can infer that the bubble defect is generally caused by insufficient injection during construction, and the cracking defect is related to the insufficient elastic modulus of the material. In the above steps, preferably, the cause knowledge graph is used for reasoning to improve the accuracy of judging the causes of wall jointing defects, and the jointing quality standard is combined to trigger the defect monitoring reminder, which can timely remind the construction personnel of the specific defect type and the cause possibility, facilitate to take targeted improvement measures, and effectively improve the overall quality of the wall jointing.

[0029] Further, the wall jointing area image is processed by bilateral filtering, and the method comprises the following steps:

[0030] The fixed industrial camera is deployed at the end of the construction robot arm and collects real-time jointing area front view images during the wall jointing operation; the camera focal length is controlled when the construction robot arm moves to the preset collection point to obtain a local close-up image; and the jointing area front view image and the local close-up image are subjected to multi-view fusion processing to obtain the wall jointing area image.

[0031] Specifically, the fixed industrial camera is an image acquisition device with fixed installation position but adjustable shooting angle and focal length. It is deployed at the end of the construction robot arm, so that the camera can change position and angle with the movement of the robot arm to adapt to the shooting needs of different wall jointing areas. The preset collection points are the positions of the robot arm preset according to the process requirements of wall jointing operation and wall structure. When the robot arm reaches these points, the camera is triggered to collect close-up images. The multi-view fusion processing refers to the fusion of images taken from different angles to generate more complete and accurate images of the wall jointing area.

[0032] Execution steps: By deploying the fixed industrial camera at the end of the construction robot arm, real-time collection of front view images during wall jointing operation can be realized to ensure dynamic monitoring of the jointing process. When the construction robot arm moves to the preset collection points, local close-up images can be obtained by controlling the focal length of the camera to capture the detailed features of the jointing area more clearly. Further, the focal length of the camera can be adjusted by sending pulse signals through the control card, and the focal length value is linearly related to the number of pulses. When the robot arm reaches a preset point, a specific number of pulse signals are sent to adjust the focal length of the camera to a state suitable for shooting local close-ups, thereby obtaining high-definition local images.

[0033] Multi-view fusion processing of front view images and local close-up images can generate wall jointing area images containing rich details and complete scenes. The fusion algorithm usually considers features such as image gray scale, texture, and edge, and aligns and fuses images from different angles through techniques such as feature matching and image registration. For example, using a multi-view fusion algorithm based on feature point matching, the feature points in the front view images and local close-up images can be matched, and the complete wall jointing image can be obtained by weighted averaging and other methods, thereby improving the feature detail retention rate and providing a more reliable image basis for subsequent bilateral filtering processing and feature extraction.

[0034] Further, the wall jointing area image is subjected to bilateral filtering processing, and the method of the present application further comprises:

[0035] The spatial domain standard deviation and the gray scale similarity standard deviation of the bilateral filtering are set as dynamic parameters. The spatial domain standard deviation is self-adaptively adjusted according to the jointing width, and the gray scale similarity standard deviation is positively correlated based on the local contrast of the image. The image edge retention degree before and after bilateral filtering is determined, and if the image edge retention degree is lower than the dynamic threshold, the filtering parameters are re-iterated and adjusted.

[0036] Specifically, the spatial domain standard deviation refers to a parameter for controlling the influence of the spatial distance of pixels on the filtering in the bilateral filtering process, and the gray similarity standard deviation refers to a parameter for controlling the influence of the difference in the gray value of pixels on the filtering; the edge retention degree of the image before and after the bilateral filtering processing refers to the retention degree of the edge information of the image after the bilateral filtering processing relative to the original image, and the dynamic threshold is a threshold value set according to the image processing requirement and used for judging whether the edge retention degree of the image meets the requirement.

[0037] The spatial domain standard deviation is adaptively adjusted according to the joint width, so that the filtering process is more suitable for the characteristics of images with different joint widths. For example, when processing an image with a wide joint width, the spatial domain standard deviation is appropriately increased, so that the noise in a larger area can be more effectively smoothed; and when processing an image with a narrow joint width, the spatial domain standard deviation is reduced, so that the edge information of a small size can be better retained.

[0038] Meanwhile, the gray similarity standard deviation is positively correlated with the local contrast of the image, so that the filtering strength can be dynamically adjusted according to the local contrast of the image. For a region with a high contrast, the gray similarity standard deviation is appropriately increased, so that the filtering can better retain the edge details; and for a region with a low contrast, the gray similarity standard deviation is reduced, so that excessive smoothing can be avoided.

[0039] By evaluating the edge retention degree of the image before and after the bilateral filtering processing, it can be judged whether the current filtering parameter can meet the requirement of edge-preserving denoising of the image. For example, when the edge retention degree of the image is lower than the dynamic threshold, such as lower than 0.8, it is indicated that the current filtering parameter may cause a large amount of edge information to be lost. At this time, the filtering parameter is re-iterated and adjusted, so that the filtering effect can be optimized. In the above step, the edge features of the joint region are better retained by improving the edge retention degree of the image, so that a clearer and more accurate image basis is provided for subsequent feature extraction and defect monitoring.

[0040] Further, the joint geometric features, joint texture features and joint color features are dimensionally aligned to configure a process feature vector, and the method comprises the following steps:

[0041] The first scale feature associated with the joint geometric features, the second scale feature associated with the joint texture features and the third scale feature associated with the joint color features are adjusted to the same dimension through max-pooling and deconvolution operations; and the process feature vector is determined by performing feature weighted fusion according to the cosine similarity of the first scale feature, the second scale feature and the third scale feature.

[0042] Specifically, max-pooling is a down-sampling technique in neural networks that reduces the spatial dimensions of feature maps while preserving important feature information by sliding a window over the feature map and taking the maximum value within the window; de-convolution is an up-sampling technique that restores low-resolution feature maps to high-resolution by learning an up-sampled representation of the feature maps; cosine similarity is an index for measuring the similarity of the directions of two vectors, and the value is closer to 1, indicating a higher similarity; the process feature vector is a vector representation obtained by fusing the geometry, texture, color, and other multi-dimensional features of the sealant, which is used for subsequent cause analysis and defect monitoring.

[0043] The execution steps are as follows: the first scale feature associated with the sealant geometry feature is down-sampled by a max-pooling operation to extract key geometric information while reducing the amount of data, then the second scale feature associated with the sealant texture feature and the third scale feature associated with the sealant color feature are up-sampled by a de-convolution operation to adjust their spatial dimensions to the same as the first scale feature after max-pooling, for example, the spatial dimensions of the original geometry feature map are 32x32, which are changed to 16x16 after max-pooling, then the texture feature map (original size 16x16) and the color feature map (original size 8x8) are up-sampled to 16x16 by de-convolution.

[0044] The cosine similarity between the first, second, and third scale features is obtained, and the features are weighted and fused according to the similarity size, if the cosine similarity between the geometry feature and the texture feature is 0.7, the cosine similarity between the geometry feature and the color feature is 0.5, and the cosine similarity between the texture feature and the color feature is 0.6, then the weights can be assigned according to these similarity values (such as normalized similarity values), the weights are multiplied by the corresponding feature maps, and the weighted feature maps are added to obtain the fused process feature vector. Through the above steps, the fusion accuracy of different scale features can be improved, and the process feature vector can more accurately reflect the true features of the sealant area, providing high-quality feature input for subsequent cause knowledge graph construction and defect monitoring.

[0045] Further, focusing on the micro-defect area with attention mechanism, the method of the present application includes:

[0046] A double-channel attention unit is constructed; wherein the spatial attention branch extracts edge features by a Sobel operator to generate a spatial weight map, and the channel attention branch calculates feature importance by global average pooling to generate a channel weight map; the spatial weight map and the channel weight map are point-multiplied to obtain a fused attention weight, and the micro-defect area is visualized and focused by an attention heat map.

[0047] Specifically, the dual-channel attention unit refers to a neural network structure combining spatial attention and channel attention. The spatial attention branch extracts edge features using a Sobel operator, which is a differential operator used in image processing that highlights edge information by calculating the gradient of the image, thereby generating a spatial weight map that emphasizes areas with obvious edges in the image. The channel attention branch calculates feature importance through global average pooling, which averages each channel of the feature map to obtain global feature information for each channel, thereby generating a channel weight map that reflects the importance of different channel features. Point multiplication refers to the element-wise multiplication of the spatial weight map and the channel weight map to obtain a fused attention weight that integrates information from both the spatial and channel dimensions to highlight more critical areas in the image. The attention heat map represents the size of the attention weight through color depth, with darker colors indicating more focused attention, thereby allowing for intuitive focusing on small defect areas.

[0048] The spatial attention branch uses the Sobel operator to extract edge features, which can highlight areas with edge changes in the image, which often contain boundary information of defects. For example, when processing a wall joint image containing bubble defects, the edges of the bubbles usually produce obvious gray scale changes. The spatial weight map after Sobel operator calculation will increase the weight value of these edge regions, making them more easily focused on by the model.

[0049] The channel attention branch evaluates feature importance through global average pooling, which can filter out feature channels that contribute more to defect detection from the channel dimension and suppress irrelevant channel interference information. For example, for a wall joint image with prominent texture features, the channel attention branch will calculate that the texture-related channels have higher importance weights, while the channels that are not sensitive to color changes have relatively low weights.

[0050] Point multiplication of the spatial weight map and the channel weight map obtains a fused attention weight, allowing the model to consider both spatial and channel dimensions for more accurate positioning of small defect areas. Visualization through the attention heat map intuitively shows which areas the model pays more attention to in the image. In the above steps, the use of the dual-channel attention mechanism improves the detection accuracy of small defects in wall joint images.

[0051] Further, the method of the present application further comprises:

[0052] The defect type mask has the defect area ratio and defect length distribution; morphological closing operation is used to fill the internal holes of the outline to obtain the defect outline; the ratio of the defect area to the total area of ​​the grout is obtained, and the defect outline is described by Freeman chain code; the encoding characteristics of the defect outline by Freeman chain code are used to convert it into the corresponding defect length distribution.

[0053] Specifically, a defect type mask is a binary image used to identify regions with different types of defects in an image. The defect area ratio refers to the proportion of the defect area to the total area of ​​the grout joint. The defect length distribution refers to the length characteristics of the defect contour in different directions. Morphological closing is an image processing operation that first dilates the image and then erodes it to fill small holes inside the contour, making the contour more complete and continuous. Here, the contour refers to the boundary line of the defect area. Filling the holes yields a more accurate defect contour. Freeman chain code is used to describe the encoding method of two-dimensional contour curves. It represents the shape of the contour by tracing the edge of the contour and recording the changes in direction. The defect length distribution is a statistical description of the length characteristics of the defect contour, reflecting the complexity and distribution of the defect shape.

[0054] Execution steps: The defect type mask has two quantitative indicators: defect area ratio and defect length distribution. These indicators can provide specific data support for subsequent defect assessment. For example, when the defect area ratio exceeds 0.05 or the defect length distribution shows that there are long cracks, the defect can be judged to be more serious. Using morphological closing operations to fill the voids inside the contour can make the defect contour more complete and avoid the error in defect area or length caused by contour interruption. For example, when processing wall grout images containing material shortage defects, the defect contour after filling the voids can more realistically reflect the actual size and shape of the material shortage area.

[0055] Obtain the ratio of the defect area to the total grout area. This ratio can serve as an important indicator for assessing the severity of the defect. For example, when the ratio reaches 0.1, it indicates that 10% of the grout area has defects and needs to be addressed promptly. Use Freeman chain codes to describe the defect outline, which can convert the shape information of the outline into a processable digital code, facilitating subsequent analysis and processing. Specifically, by analyzing the frequency of directional changes in the Freeman chain codes, the complexity of the defect outline can be determined. For example, frequent directional changes in the chain codes may indicate that the defect outline is irregular and has many edge fluctuations.

[0056] By utilizing the encoding characteristics of Freeman chain codes to convert them into corresponding defect length distributions, the length features of the defect contours in different directions can be obtained, providing richer information for defect type identification. Specifically, discoloration defects may exhibit a long and smooth contour length distribution, while cracking defects may have a short and complex contour length distribution. Extracting these quantitative features helps improve the accuracy and reliability of wall grout defect monitoring, providing strong support for subsequent defect cause analysis and quality control.

[0057] Furthermore, based on the aforementioned process feature vector, combined with the defect type mask and wall base characteristics, a causal knowledge graph is established. The method of this application includes:

[0058] The causal knowledge graph includes an entity layer and a relation layer; a graph neural network is used to reason about the causal knowledge graph, and the process feature vector is input; at the same time, the defect type mask and wall base features are used for encoding conversion, and then the message passing mechanism of the graph neural network is used to mine the associated features in the entity layer and the relation layer.

[0059] Specifically, the entity layer refers to the set of nodes in the causal knowledge graph that represent different entities (such as grout materials, construction process parameters, defect types, etc.), while the relation layer is the set of edges that represent the relationships between entities (such as causal relationships, association relationships, etc.). A graph neural network is a deep learning model based on graph-structured data, capable of learning and reasoning about entities and relationships in the causal knowledge graph. The process feature vector is a vector representing the characteristics of the wall grouting process after feature fusion processing, serving as the input to the graph neural network. The defect type mask is a binary image that identifies different defect types in the image. The wall base features are feature information of the wall base obtained through pixel-level outward sampling. Encoding conversion refers to transforming this feature information into a form suitable for graph neural network processing. The message passing mechanism is a rule in the graph neural network used to pass and update feature information between entity nodes, thereby mining associated features in the entity and relation layers.

[0060] Execution steps: The entity layer and relation layer of the causal knowledge graph construct a semantic network containing entities such as grout material, construction process, and defect type, as well as their interrelationships. Specifically, the entity layer contains nodes corresponding to grout composition, construction temperature, and bubble defects, while the relation layer contains edges corresponding to the influence of grout composition on bubble defects and the correlation between construction temperature and cracking defects.

[0061] The process feature vector is input into a graph neural network (Graph Neural Network). Based on the structure of the causal knowledge graph, the Graph Neural Network can learn the latent representations of each entity and relationship. Furthermore, through training, the Graph Neural Network can learn the association weights between construction temperature nodes and crack defect nodes, representing the degree of influence of construction temperature on crack defects. Specifically, constructing the Graph Neural Network includes using an existing graph convolutional network framework to quantify the features of entity layer nodes (such as material entity nodes, environmental entity nodes, and defect type nodes) in the causal knowledge graph. For material entity nodes, the elastic modulus (MPa) and bonding strength (N / m²) of the grout material are used as parameters. 2 The features are directly obtained from the BIM model and standardized to values ​​in the range [0, 1]. For environmental entity nodes, the curing rate index (calculated based on temperature and humidity, e.g., 0.8 at 25℃ and 50% humidity) is used as the core feature, combined with construction time (hours) to form a two-dimensional feature vector. For defect type nodes, the defect area ratio and mean defect length distribution are extracted based on the defect type mask, and the defect type (material shortage / bubbles / cracking / discoloration) is identified using One-Hot encoding to form a three-dimensional feature vector. The quantized entity node features are used as input to a graph neural network, with each node corresponding to a feature vector. These features are uniformly mapped to a multi-dimensional feature space through an embedding layer to ensure compatibility of features for different types of entities. Based on the weights of the associated edges in the causal knowledge graph relationship layer, convolutional operations are used to aggregate adjacent node information, such as the aggregation of material and environmental node features for defect type nodes. After mapping the global feature vector through a fully connected layer, the probability distribution of defect causes is output through a softmax function, such as... Among them, z k For the k-th type of cause score output by the fully connected layer, such as material, construction process, environmental impact, and substrate, P(cause) k The probability of this cause is z. m This refers to the raw score (unnormalized value) of the m-th type of defect cause output by the fully connected layer of the graph neural network; the difference between the predicted probability and the actual cause label is calculated using the cross-entropy loss function, and the Adam optimizer is used for iterative training until the loss function converges.

[0062] Simultaneously, after encoding and converting using defect type masks and wall base features, the message passing mechanism of a graph neural network is used to mine associated features at the entity and relation layers. This allows for the combination of the appearance features of defects with the causal relationships in the causal knowledge graph. Specifically, when processing wall grout images containing material shortage defects, based on the association between grouting pressure and material shortage defects in the causal knowledge graph, the feature information of the grouting pressure node is passed to the material shortage defect node through the message passing mechanism. Combined with the features of the material shortage defect in the current image, a comprehensive analysis is performed, thereby more accurately determining whether the material shortage defect is related to insufficient grouting pressure. Through these steps, the accuracy of determining the causes of wall grout defects is improved, providing a more precise analysis of defect causes, which helps to take targeted improvement measures and effectively improve the construction quality of wall grouting.

[0063] The message passing mechanism of a graph neural network is used to mine associated features in the entity and relation layers. Specifically, the process feature vector (the dimensional alignment result of the grout geometry, texture, and color features) is used as the core input of the graph neural network. Simultaneously, defect type masks (including defect area ratio and length distribution) and wall base features (obtained through pixel-level outward sampling of defect edges) are encoded and transformed to generate feature vectors adapted to the knowledge graph. A graph convolutional network (GCN) is used as the basic framework, with entity layer nodes (such as material entities, environmental entities, and defect entities) of the causal knowledge graph as network nodes, and relational edges (such as material elastic modulus → cracking defects, environmental humidity → bubble defects) as connections between nodes. The first layer of inference aggregates the features of adjacent nodes of each entity node through convolution operations. For example, the bubble defect node aggregates the features of its associated grout material bonding strength node and environmental humidity node. The calculation formula is as follows: in, The initial features are defined as follows: ω1 and ω2 are edge weights (obtained through training with historical cases), deg() is the node degree, and σ is the ReLU activation function. The second layer of inference performs high-order aggregation on the features output from the first layer, incorporating global information from the process feature vector. For example, the aggregated features of bubble defects are further fused with the looseness features of the grout texture in the process feature vector to output the probability distribution of defect causes. For example, the probability of insufficient material adhesion causing bubbles is 65%, and the probability of excessive environmental humidity causing bubbles is 30%. The final feature vector is converted into causal probabilities using the softmax function. If the probability of a certain cause exceeds the threshold of the grout quality qualification standard, it is determined to be the main cause, and the corresponding monitoring reminder is triggered.

[0064] The entity layer and relation layer are used for feature mining. Further, the entity layer features are used for feature mining. Specifically, the entity layer includes nodes such as material entities (e.g., elastic modulus, bond strength), environmental entities (e.g., curing rate index), and defect entities (e.g., missing material, bubbles). The graph neural network uses a message passing mechanism to mine potential relationships between different entity nodes. For example, when the cosine similarity between the length distribution characteristics (mean 10mm) of the crack defect node and the characteristics of the elastic modulus node of the grout material (0.3MPa, lower than the standard value of 0.4MPa) is higher than 0.8, the graph neural network will strengthen the association weight between the two, revealing the causal relationship of low elastic modulus → cracking. Combined with the material parameters input from the BIM model, the entity node features are bound to actual engineering data. For example, the frequency of crack defects corresponding to materials with an elastic modulus of 0.3MPa in historical cases is 72%, further verifying the association strength.

[0065] The entity layer and relation layer perform feature mining. Furthermore, the weights of the relational edges are optimized. Specifically, the initial weights of the relational edges are set based on historical defect cases. The graph neural network optimizes the weights through iterative training. For example, the initial weight of "ambient humidity > 80% → bubble defect" is 0.6. If 80% of the newly input "bubble defect" cases have ambient humidity > 80%, the edge weight is increased to 0.75. If a case has normal humidity but a material curing rate index < 0.5 and a bubble defect appears, the edge weight of "material curing rate → bubble defect" is strengthened. The edge weights are dynamically adjusted through an attention mechanism to adapt to the feature associations of different scenarios.

[0066] The entity layer and the relationship layer are used for feature mining. Further, cross-layer feature fusion is performed. Specifically, the node features of the entity layer and the edge weights of the relationship layer are fused across layers through the fully connected layer of the graph neural network. For example, the entity feature of material shortage defect is multiplied with the edge weight of "insufficient grouting pressure → material shortage", and then fused with the porosity feature of the wall base layer to output the comprehensive score corresponding to the grouting process. This realizes the collaborative reasoning of entity attributes and relationship strength. In the above steps, the graph neural network aggregates entity features through hierarchical convolution, dynamically optimizes relationship edge weights, and fuses multi-dimensional information across layers to achieve accurate reasoning of the causal knowledge graph. At the same time, it mines the potential correlation features between the entity layer and the relationship layer to improve the monitoring accuracy of wall grout defects.

[0067] Furthermore, by utilizing the message passing mechanism of graph neural networks to perform association feature mining at the entity layer and relation layer, the method of this application includes:

[0068] Integrating with BIM (Building Information Modeling), the system receives grout material parameters including elastic modulus and bonding strength, and maps these parameters to material entity nodes corresponding to the entity layer in the causal knowledge graph. Based on environmental temperature and humidity data, it evaluates the grout curing rate index and maps it to environmental entity nodes corresponding to the entity layer in the causal knowledge graph. In the relationship layer of the causal knowledge graph, it configures associated edges, trains edge weights using historical defect cases, and aggregates adjacent entity nodes corresponding to the associated edges according to the edge weights using a message passing mechanism.

[0069] Specifically, BIM (Building Information Modeling) is a digital model that integrates information from the entire lifecycle of a building project, including multi-dimensional data on building components, materials, and the environment. Elastic modulus and bond strength are important physical performance indicators for grout materials, measuring the material's resistance to deformation and its adhesion to the substrate, respectively. By mapping these material parameters to material entity nodes in a causal knowledge graph, the actual material properties can be connected to the data structure within the causal knowledge graph. The grout curing rate index measures the curing speed of grout under specific environmental conditions and is closely related to environmental temperature and humidity. Mapping this index to environmental entity nodes reflects the impact of environmental factors on grout quality. Configuring associated edges in the relational layer of the causal knowledge graph and training edge weights using historical defect cases quantifies the association strength between different entity nodes. The message passing mechanism is a method in graph neural networks used to update node features. By weighted aggregation of features of adjacent nodes based on edge weights, potential association features between entities can be mined.

[0070] Execution steps: Integration with BIM (Building Information Modeling) enables real-time acquisition of key parameters such as the elastic modulus and bond strength of the sealant. Specifically, according to GB / T 14683-2017 "Silicone and Modified Silicone Building Sealants", the tensile modulus (a type of elastic modulus) of silicone building sealants should be ≤0.4MPa or ≤0.6MPa at 23℃, and the adhesion at a given elongation should be undamaged. These parameters directly affect the quality and durability of the sealant.

[0071] After mapping these parameters to material entity nodes in the causal knowledge graph, the relationship between material properties and defect types can be established. The curing rate index of the grout is evaluated based on environmental temperature and humidity data and mapped to environmental entity nodes, which can reflect the impact of environmental factors on the quality of the grout. Commonly, when the environmental humidity is below 30% or above 80%, the curing rate of the grout will be significantly slowed down, which may lead to defects such as bubbles and cracks.

[0072] By configuring associated edges in the relational layer of the causal knowledge graph and training edge weights through historical defect cases, the graph neural network can learn the association strength between different entity nodes. Specifically, by analyzing a large number of historical defect cases, it was found that the association weight between grout materials with low elastic modulus and bubble defects is relatively high, while the association weight between materials with insufficient bonding strength and cracking defects is relatively high. Through a message passing mechanism, the adjacent entity nodes corresponding to the associated edges are weighted and aggregated according to the edge weights. The influence of material characteristics, environmental factors, and construction technology on defects is comprehensively considered, thereby more accurately inferring the causes of defects and significantly improving the accuracy and reliability of wall grout defect monitoring.

[0073] In summary, the beneficial effects of the embodiments of this application are:

[0074] This application employs bilateral filtering on the wall grout area image, and extracts grout geometric, texture, and color features through adaptive threshold segmentation. A multi-scale feature topology is constructed, aligning the geometric, texture, and color features dimensionally, and configuring process feature vectors. An attention mechanism focuses on minute defect areas, outputting defect type masks (including missing material, bubbles, cracks, and discoloration) and wall substrate features under pixel-level outward sampling of defect edges. Based on the process feature vectors, combined with the defect type masks and wall substrate features, a causal knowledge graph is established. This graph is used to infer the probability of defect causes, and defect monitoring alerts are triggered based on grout quality standards. This application provides a wall grout defect monitoring method, system, and medium based on image recognition. This technology achieves the following results: geometric, texture, and color features of wall grout are extracted through bilateral filtering and adaptive threshold segmentation; dimension alignment is performed by combining multi-scale feature topology; an attention mechanism is introduced to effectively focus on small defect areas; and a cause knowledge graph is established based on process feature vectors, defect type masks, and wall base features to accurately infer the probability of defect causes, thereby improving the technical effect of wall grout defect monitoring.

[0075] Example 2 is based on the same inventive concept as the image recognition-based wall grout defect monitoring method in the previous examples, such as... Figure 2 As shown in the figure, this application provides a wall grout defect monitoring system based on image recognition, wherein the system includes:

[0076] The segmentation and extraction module M100 performs bilateral filtering on the wall grout area image and extracts the geometric features, texture features, and color features of the grout through adaptive threshold segmentation.

[0077] Dimension Alignment Module M200: Constructs a multi-scale feature topology, aligns the geometric features, texture features, and color features of the grout in dimensions, and configures the process feature vector.

[0078] Focusing Module M300: Focuses on tiny defect areas using an attention mechanism, outputting defect type masks including material shortage, bubbles, cracks, and discoloration, as well as wall base features under pixel-level outward sampling of defect edges.

[0079] Defect monitoring and reminder module M400: Based on the process feature vector, combined with the defect type mask and wall base features, a cause knowledge graph is established. The cause knowledge graph is used to infer the probability of defect causes, and a defect monitoring reminder is triggered according to the quality qualification standard of the grout.

[0080] Furthermore, the segmentation and extraction module M100 is used to perform the following method:

[0081] A fixed industrial camera is deployed at the end of a construction robotic arm to collect frontal images of the grouting area in real time during the wall grouting operation. When the construction robotic arm moves to a preset acquisition point, the camera's focal length is controlled to acquire a local close-up image. The frontal image of the grouting area and the local close-up image are fused together from multiple perspectives to obtain the image of the wall grouting area.

[0082] Furthermore, the segmentation and extraction module M100 is also used to perform the following method:

[0083] The spatial domain standard deviation and gray-level similarity standard deviation of the bilateral filter are set as dynamic parameters. The spatial domain standard deviation is adaptively adjusted according to the width of the seam, and the gray-level similarity standard deviation is based on the positive correlation of local image contrast. The edge preservation of the image before and after bilateral filtering is measured. If the edge preservation of the image is lower than the dynamic threshold, the filtering parameters are iterated and adjusted again.

[0084] Furthermore, the dimension alignment module M200 is used to perform the following method:

[0085] By performing max pooling and deconvolution operations, the first-scale feature associated with the geometric features of the grout, the second-scale feature associated with the texture features of the grout, and the third-scale feature associated with the color features of the grout are adjusted to the same dimension; the feature vector is determined by weighted fusion based on the cosine similarity of the first-scale feature, the second-scale feature, and the third-scale feature.

[0086] Furthermore, the focusing module M300 is used to perform the following method:

[0087] A dual-channel attention unit is constructed; wherein, the spatial attention branch extracts edge features through the Sobel operator to generate a spatial weight map, and the channel attention branch calculates feature importance through global average pooling to generate a channel weight map; the spatial weight map and the channel weight map are multiplied to obtain the fused attention weight, and the focus of the tiny defect region is visualized through an attention heatmap.

[0088] Furthermore, the focusing module M300 is also used to perform the following methods:

[0089] The defect type mask has the defect area ratio and defect length distribution; morphological closing operation is used to fill the internal holes of the outline to obtain the defect outline; the ratio of the defect area to the total area of ​​the grout is obtained, and the defect outline is described by Freeman chain code; the encoding characteristics of the defect outline by Freeman chain code are used to convert it into the corresponding defect length distribution.

[0090] Furthermore, the defect monitoring and alerting module M400 is used to perform the following method:

[0091] The causal knowledge graph includes an entity layer and a relation layer; a graph neural network is used to reason about the causal knowledge graph, and the process feature vector is input; at the same time, the defect type mask and wall base features are used for encoding conversion, and then the message passing mechanism of the graph neural network is used to mine the associated features in the entity layer and the relation layer.

[0092] Furthermore, the defect monitoring and alerting module M400 is also used to perform the following methods:

[0093] Integrating with BIM (Building Information Modeling), the system receives grout material parameters including elastic modulus and bonding strength, and maps these parameters to material entity nodes corresponding to the entity layer in the causal knowledge graph. Based on environmental temperature and humidity data, it evaluates the grout curing rate index and maps it to environmental entity nodes corresponding to the entity layer in the causal knowledge graph. In the relationship layer of the causal knowledge graph, it configures associated edges, trains edge weights using historical defect cases, and aggregates adjacent entity nodes corresponding to the associated edges according to the edge weights using a message passing mechanism.

[0094] In Example 3, based on the same inventive concept as the image recognition-based wall grout defect monitoring method in the foregoing embodiments, this application also provides a computer-readable storage medium storing a computer program, which, when executed, implements the steps of any one of the methods described in Example 1 above.

[0095] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this specification. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are possible or may be advantageous.

[0096] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0097] This specification and accompanying drawings are merely illustrative examples of this application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from its scope. Therefore, if such modifications and modifications fall within the scope of this application and its equivalents, this application intends to include such modifications and modifications.

Claims

1. A method for monitoring defects in wall grout joints based on image recognition, characterized in that, The method includes: Bilateral filtering is applied to the image of the wall grout area, and the geometric features, texture features, and color features of the grout are extracted by adaptive threshold segmentation. Construct a multi-scale feature topology, align the dimensions of the grout geometric features, grout texture features and grout color features, and configure the process feature vector; Focusing on minute defect areas using an attention mechanism, the output includes defect type masks such as material shortage, bubbles, cracks, and discoloration, as well as wall base features under pixel-level outward sampling of defect edges; Based on the process feature vector, combined with the defect type mask and wall base features, a causal knowledge graph is established. The causal knowledge graph is used to infer the probability of defect causes, and defect monitoring reminders are triggered according to the quality qualification standards of the grout.

2. The method for monitoring wall grout defects based on image recognition as described in claim 1, characterized in that, The method for performing bilateral filtering on images of wall grout sealing areas includes: A fixed industrial camera is deployed at the end of the construction robotic arm to collect real-time frontal images of the grouting area during the wall grouting operation. When the construction robotic arm moves to the preset acquisition point, the camera focal length is controlled to acquire a close-up image of the local area; The front view image of the grouting area and the close-up image of the local area are fused from multiple perspectives to obtain the image of the grouting area of ​​the wall.

3. The method for monitoring wall grout defects based on image recognition as described in claim 2, characterized in that, The method further includes performing bilateral filtering on the image of the wall grout sealing area. The spatial domain standard deviation and gray-level similarity standard deviation of the bilateral filter are set as dynamic parameters. The spatial domain standard deviation is adaptively adjusted according to the width of the seam, and the gray-level similarity standard deviation is based on the positive correlation of local image contrast. The image edge preservation is evaluated before and after bilateral filtering. If the image edge preservation is lower than the dynamic threshold, the filtering parameters are iterated and adjusted again.

4. The method for monitoring wall grout defects based on image recognition as described in claim 3, characterized in that, The method involves dimensionally aligning the geometric features, texture features, and color features of the grout joint, and configuring a process feature vector. By using max pooling and deconvolution operations, the first-scale feature associated with the geometric features of the grout, the second-scale feature associated with the texture features of the grout, and the third-scale feature associated with the color features of the grout are adjusted to the same dimension. The process feature vector is determined by weighted fusion of features based on the cosine similarity of the first scale feature, the second scale feature, and the third scale feature.

5. The method for monitoring wall grout defects based on image recognition as described in claim 1, characterized in that, The method of focusing on minute defect areas using an attention mechanism includes: Construct a dual-channel attention unit; Among them, the spatial attention branch extracts edge features through the Sobel operator to generate a spatial weight map, and the channel attention branch calculates feature importance through global average pooling to generate a channel weight map; The spatial weight map and the channel weight map are multiplied to obtain the fused attention weight, and the focus of the tiny defect area is visualized through an attention heatmap.

6. The method for monitoring wall grout defects based on image recognition as described in claim 5, characterized in that, The defect type mask has the defect area ratio and defect length distribution; Use morphological closing operations to fill the holes inside the contour to obtain the defect contour; Obtain the ratio of the defect area to the total grout area, and use Freeman chain code to describe the defect outline; By utilizing the encoding characteristics of Freeman chain codes for defect contours, the corresponding defect length distribution is converted.

7. The method for monitoring wall grout defects based on image recognition as described in claim 6, characterized in that, Based on the aforementioned process feature vector, and combined with the defect type mask and wall base characteristics, a causal knowledge graph is established. The method includes: The causal knowledge graph includes an entity layer and a relation layer; A graph neural network is used to reason about the causal knowledge graph, and the process feature vector is input. Simultaneously, the defect type mask and wall base features are used for encoding conversion. Then, the message passing mechanism of the graph neural network is used to mine associated features in the entity layer and relation layer.

8. The method for monitoring wall grout defects based on image recognition as described in claim 7, characterized in that, The method utilizes the message passing mechanism of graph neural networks to perform association feature mining at the entity layer and relation layer, and includes: Integrating with BIM (Building Information Model), it receives grout material parameters including elastic modulus and bonding strength, and maps the grout material parameters to material entity nodes corresponding to the entity layer in the causal knowledge graph; The curing rate index of the grout is evaluated based on environmental temperature and humidity data, and the curing rate index is mapped to the environmental entity node corresponding to the entity layer in the causal knowledge graph. In the relational layer of the causal knowledge graph, associated edges are configured, edge weights are trained through historical defect cases, and adjacent entity nodes corresponding to the associated edges are weighted and aggregated according to the edge weights through a message passing mechanism.

9. A wall joint defect monitoring system based on image recognition, characterized in that, The system is used to implement the image recognition-based wall joint defect monitoring method according to any one of claims 1-8, wherein the system comprises: Segmentation and extraction module: Performs bilateral filtering on the wall grout area image, and extracts the geometric features, texture features and color features of the grout through adaptive threshold segmentation; Dimension Alignment Module: Constructs a multi-scale feature topology, aligns the geometric features, texture features, and color features of the grout in dimensions, and configures the process feature vector; Focusing module: Focuses on minute defect areas using an attention mechanism, outputting defect type masks including material shortage, bubbles, cracks, and discoloration, as well as wall base features under pixel-level outward sampling of defect edges; Defect monitoring and alert module: Based on the process feature vector, combined with the defect type mask and wall base features, a cause knowledge graph is established. The cause knowledge graph is used to infer the probability of defect causes, and a defect monitoring alert is triggered according to the quality qualification standard of the grout.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the wall grout defect monitoring method based on image recognition as described in any one of claims 1-8.

Images