A multi-view detection method and application system for brickwork construction quality

Abstract

The present invention discloses a multi-perspective detection method and application system for brick masonry construction quality. In the data processing part, the method combines web crawler technology and proprietary data for data enhancement, obtains rich training data through multi-perspective transformation, and improves the generalization ability of the model. It solves the problem of insufficient data in existing methods when processing complex scenes. In the testing stage, the EPnP posture estimation algorithm and brick boundary intersection auxiliary correction are used to effectively solve the image angle problem and improve the accuracy of detection. Existing methods often have poor effects when processing image angle changes, resulting in deviations in detection results. In the evaluation stage, the corner defect detection function is introduced, and the boundary line is completed through the geometric extrapolation method, which improves the accuracy of horizontality and verticality evaluation. At the same time, the defect detection function also provides strong support for construction quality monitoring. In contrast, existing methods often ignore the impact of corner defects on the results during the evaluation process, resulting in inaccurate evaluation results.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and deep learning, and in particular relates to a multi-view detection method and application system for brickwork construction quality. Background Art

[0002] Brick masonry is a common structural form in construction projects, and its construction quality directly affects the stability and service life of the building. The construction quality of brick masonry is mainly reflected in the arrangement accuracy of bricks, the thickness of mortar joints, the verticality and horizontality of masonry, and other aspects. Therefore, how to effectively detect and evaluate the construction quality of brick masonry has become an important topic in the current construction industry. Traditional brick masonry construction quality detection methods mostly rely on manual inspection or simple instrument measurement. These methods have many shortcomings, such as low detection efficiency, high labor costs, and large errors. In order to improve the accuracy and efficiency of detection, in recent years, computer vision, artificial intelligence, and deep learning technologies have gradually been introduced into the assessment of brick masonry construction quality, especially in the application of target detection, image processing, and deep learning models, showing great potential.

[0003] Currently, traditional methods for brickmasonry construction quality inspection primarily include manual inspection, measurement tool inspection, and image-based inspection. While manual inspection is simple and easy to perform, it relies on manual intervention, making it prone to omissions and misjudgments. This is particularly inefficient and subjective at complex and large-scale construction sites. Measurement tool inspection requires the use of measuring instruments (such as levels, verticality gauges, and calipers) to physically measure the brickmasonry. While these tools may be effective for small-scale inspections, they struggle to comprehensively cover complex structures and large-scale brickmasonry construction, and their efficiency and accuracy remain limited. In recent years, image processing technology has been introduced to brickmasonry construction quality inspection. By capturing images of brickmasonry at construction sites and combining them with image analysis and processing techniques, accurate measurement and assessment of brickwork can be achieved. Image inspection methods process brickmasonry images to extract information such as brick boundaries, mortar joint thickness, verticality, and horizontality. This enables non-contact inspection, reduces manual intervention, and improves inspection efficiency and accuracy. However, traditional image processing methods still face some challenges, such as complex background, low image resolution, occlusion or stains on the brickwork surface, etc., which affect the accuracy of detection. Summary of the Invention

[0004] In view of the deficiencies in the above-mentioned background technology, the present invention aims to provide a multi-view detection method and application system for brick masonry construction quality, which solves the problems of insufficient detection efficiency and accuracy of traditional brick masonry construction quality detection methods.

[0005] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is as follows: On the one hand, the present invention provides a multi-view detection method for brickwork construction quality, which is characterized by comprising:

[0006] Step 1: Obtain images containing brickwork construction scenes to form an original dataset. The original dataset covers brickwork construction scene images with different resolutions, lighting conditions, and shooting angles. Use data augmentation technology to process the original dataset to form an enhanced dataset, and annotate the enhanced dataset to form a complete dataset containing category and location information. Input the complete dataset into the detection model for pre-training and fine-tuning to establish a brickwork construction quality detection model.

[0007] Step 2: Use digital equipment to shoot the actual brick wall construction scene to form a high-resolution, multi-viewpoint actual brick wall image dataset;

[0008] Step 3: Input the actual brick wall image dataset into the brickwork construction quality inspection model for target detection, obtain the initial detection frame positions of the bricks and the initial detection frame positions of the mortar joints, and extract the boundary intersection set between the bricks based on the initial detection frame positions of the bricks and the initial detection frame positions of the mortar joints;

[0009] Step 4: Correct the boundary intersection point set between bricks using the 3D point set of the standard brick layout to obtain a corrected data set;

[0010] Step 5: After performing boundary line fitting, defect detection, horizontality and verticality evaluation, and mortar joint thickness measurement on the images in the corrected dataset, it is evaluated whether the brickwork construction quality meets the standards.

[0011] The multi-view detection method for brick masonry construction quality in the present invention can construct a brick masonry construction quality detection model under limited image data conditions, significantly improving the accuracy and adaptability of brick and mortar joint detection, replacing manual detection of brick masonry construction quality, improving the efficiency of brick masonry construction quality detection, reducing human errors, and providing a more scientific and objective evaluation basis for construction quality.

[0012] Furthermore, in step 1, the original dataset is obtained from the Internet using crawler technology or photographed with a digital device. Specifically, crawler technology (such as Scrapy) can be used to crawl brickwork images from the Internet at multiple resolutions, multiple scenes, and multiple perspectives. These data are derived from photos of brick walls taken at different angles and lighting conditions, enriching the diversity of the dataset. At the same time, brickwork image data can also be collected by digital devices, and the brickwork image data collected from the Internet can be combined with the self-collected brickwork image data to form an original dataset, ensuring that the original dataset covers more actual construction scenarios. This original dataset not only provides a foundation for the subsequent establishment and fine-tuning of the brickwork construction quality detection model, but also provides a diverse source of raw materials for multi-perspective enhancement operations. Ultimately, a high-precision brickwork construction quality detection model is established, improving the precision and accuracy of target detection.

[0013] Furthermore, in step 1, the data enhancement technology includes rotation transformation, scaling and cropping, and lighting and color adjustment of the image; the enhanced data set is recorded as :

[0014]

[0015] in, is the original dataset, Represents rotation, scaling, and shearing operations respectively.

[0016] The purpose of setting up the enhanced dataset is to generate diverse samples in the spatial and color domains through data augmentation technology, expand the scale of the dataset, improve the adaptability of the subsequently established brick masonry construction quality inspection model to diverse scenarios, and improve the robustness of the brick masonry construction quality inspection model.

[0017] Specifically, the rotation transformation process is as follows: To simulate the impact of shooting angle deviation on the image, the image is randomly transformed as follows:

[0018]

[0019] in, The angle is randomly rotated (for example, randomly generated within the range of [-30°, 30°]. This operation can simulate the position changes of bricks when photographed at different angles and train the detection model to adapt to multi-angle detection tasks.

[0020] The process of scaling and cropping is to randomly scale and crop the image to enhance the detection model's adaptability to different resolutions and field of view. The scaling ratio is a random value. , cropping is done by defining the center point ( x c , y c) and the crop frame size ( w , h ) is generated dynamically to ensure that the target brick remains in the cropped image.

[0021] The process of light and color adjustment is: simulate the change of light conditions, and randomly change the brightness b , contrast c and saturation s : ,in I is the original image, I ’ This method enhances the robustness of the model to different lighting environments.

[0022] Furthermore, in step 1, the detection model is a YOLOv8 model. The principle behind establishing the brickwork construction quality detection model is as follows: the system uses the YOLOv8 model for brick and mortar joint detection, combining pre-training and fine-tuning techniques. Pre-training is first performed on a large public dataset (i.e., the original dataset) to obtain common feature representations. Fine-tuning is then performed on an augmented dataset to optimize the model's adaptability to the specific task, ultimately establishing the brickwork construction quality detection model. To further improve the detection accuracy of the brickwork construction quality detection model, specific post-processing strategies are introduced, including non-maximum suppression (NMS) to remove duplicate detection results, ultimately outputting a set of bounding boxes for bricks and mortar joints.

[0023] Specifically, the method for forming a complete dataset containing category and location information is as follows: use the LabelImg tool to annotate the original dataset and the enhanced dataset, generate the bounding boxes of the brick and mortar joint areas, form a complete dataset, parse the complete dataset, and save it in the YOLO annotation format. The complete dataset is recorded as ( x , y , w , h ),in, x , y Respectively represent the horizontal and vertical coordinates of the center point of the pixel calibration point in the image; w , h Represents the width and length of the crop box size respectively.

[0024] The same annotation strategy is used to generate labels for each image in the augmented dataset, ensuring consistent quality. This multi-view data processing and augmentation technology not only expands and increases data diversity, but also effectively mitigates the model's overfitting problem with limited data during training, providing a solid data foundation for subsequent brick detection, posture correction, and construction quality assessment.

[0025] Furthermore, in step 1, the brickwork construction quality detection model includes a CNN layer for feature extraction, a linear layer for class mapping, and a Softmax layer for outputting the probability distribution of bricks and mortar joints. After the brickwork construction quality detection model is established, the non-maximum suppression (NMS) algorithm is used to remove redundant detection frames, retaining only the target area with the highest confidence: , its goal is to optimize the confidence of boundary classification and make the detection results more reliable.

[0026] In step 1 of the invention, through multi-view data processing and data enhancement technology, not only the expansion and diversity of data are achieved, but also the overfitting problem of the detection model on limited data is effectively alleviated during the training process, providing a solid data foundation for subsequent brick detection, posture correction and construction quality assessment.

[0027] Furthermore, in step 3, the initial detection frame position of the brick and the initial detection frame position of the mortar joint are recorded as {b i};

[0028] Extract the boundary intersection set between bricks through the vertex coordinates of the initial detection box of bricks and mortar joints:

[0029]

[0030] in, P k The two-bit pixel coordinates of each point, ( x k , y k ) are the horizontal and vertical coordinates of the two-dimensional pixel respectively; N is the total number of detected intersections.

[0031] Furthermore, in step 4, the method for obtaining the corrected data set is:

[0032] Using the boundary intersection set between bricks P and the 3D point set of the standard brick layout Q To establish a projection relationship:

[0033]

[0034] in, R is the rotation matrix, T is the translation vector, is the projection point set of the image space; according to the EPnP algorithm, R and T , perform inverse transformation on the input image in the complete dataset, correct the shooting angle deviation, and obtain the corrected dataset Specifically, the boundary intersection set between bricks is corrected by combining a pose estimation algorithm with image angle correction. The pose estimation algorithm and image angle correction can effectively reduce the interference of shooting angle deviation on the horizontality and verticality assessment of bricks, thereby improving detection accuracy and robustness.

[0035] Specifically, the process of the pose estimation algorithm is to calculate the rotation and translation parameters of the brickwork plane in three-dimensional space by inputting the pixel coordinates of the calibration points in the image. In this solution, the EPnP (EfficientPerspective-n-Point) algorithm is selected for pose estimation. Its main purpose is to derive three-dimensional pose information from two-dimensional image points. Assume that there is a n calibration points, and the two-dimensional pixel coordinates of each point are , the corresponding three-dimensional world coordinates are The camera intrinsic parameter matrix is K , the external parameter matrix is ,in R is the rotation matrix, T is the translation vector. The EPnP algorithm solves the extrinsic parameters by minimizing the projection error:

[0036] ,

[0037]

[0038] in is the actual projection point coordinate. In data preprocessing, the brick boundary intersection points or feature points are accurately extracted to ensure that the calibration points input to the EPnP algorithm have high-quality correspondence. The selection strategy of these points optimizes the uniformity of the calibration point distribution and reduces the error accumulation caused by boundary defects. The output of the EPnP algorithm includes the rotation matrix R and translation vectors T Furthermore, the solution can be optimized by combining the geometric constraints in the shooting scene. For example, if the brickwork is parallel to the ground, the ground plane normal vector can be constrained to (0, 0, 1), thereby optimizing R precision.

[0039] The process of image angle correction is as follows: after completing the pose estimation, the goal of image angle correction is to align the horizontal edge of the brickwork with the horizontal axis of the image and the vertical edge with the vertical axis of the image, thereby providing standardized input for the subsequent horizontality and verticality evaluation. The implementation of affine transformation is based on the known external parameters , design an affine transformation to adjust the image to the standard posture. Let the pixel coordinates of any point in the original image be ( x ,, y ), the coordinates of the corrected point are ( x ′,y ′), the transformation formula is:

[0040]

[0041] in H is the affine transformation matrix, which is obtained by solving the pose estimation result. Specifically, the generation process of the affine matrix includes the following steps:

[0042] 1) Extract the key boundary feature points of the brickwork and determine the horizontal and vertical baselines;

[0043] 2) According to R Decomposition result, calculate the rotation angle of the baseline in the image θ ;

[0044] 3) Generate rotation correction matrix: .

[0045] In addition, to further improve the correction accuracy, this solution designs an auxiliary correction strategy based on the intersection of brick boundaries. Detect all brick boundaries in the image and extract the intersection set of boundary lines. . Assume that the intersection point set after standard correction is , use the least squares method to fit the transformation matrix: .

[0046] In summary, in step 4, the EPnP algorithm is combined with geometric constraints to improve the rotation matrix through multi-stage optimization. R The solution accuracy is high. In particular, in actual construction scenarios, the geometric relationship between brickwork and the ground provides strong constraints, effectively eliminating noise interference. The overall process design, combining pose estimation and image correction, not only improves the accuracy of brick horizontality and verticality assessment, but also further enhances the robustness of the algorithm through innovative defect identification and geometric optimization strategies, providing theoretical support and technical assurance for multi-view construction quality inspection.

[0047] Furthermore, the deviation angle of the brick boundary line in the correction data set is calculated :

[0048]

[0049] Ensure the deviation angle Within the allowable error range, x 1. x 2. y 1. y 2 are the pixel coordinates of the brick boundary line, x 1 ’ 、 x 2 ’ 、 y1 ’ 、 y 2 ’ are the pixel coordinates of the brick boundary lines after correction.

[0050] Furthermore, in step 5, the boundary line fitting includes: using the boundary points of the bricks in the correction data set, fitting the straight line equation by the least squares method:

[0051]

[0052] in, m , c To fit the parameters, minimize the error function:

[0053]

[0054] in,( x i , y i ) is the point set of the brick boundary; m , c is the fitting parameter;

[0055] Defect detection includes: analyzing the Euclidean distance of adjacent boundary points and angle changes :

[0056]

[0057] in,{ p i} is a discrete point set, p i+1 - p i is the distance between each point and its neighboring points; if >threshod1 or >threshod2, determine it as a defect point and complete the boundary through geometric extrapolation;

[0058] The horizontality and verticality assessment involves comparing the fitted boundary line with the standard line and calculating the horizontality and verticality errors:

[0059]

[0060] in, y i is the fitted horizontal boundary line, y std is the standard horizontal boundary line, is the brick level error, x i is the fitted vertical boundary line, xstd is the standard vertical boundary line, is the verticality error of the brick, N is the number of discrete points in the brick area;

[0061] The mortar joint thickness measurement includes: calculating the minimum distance between adjacent boundary lines based on the brick boundary line fitting results:

[0062]

[0063]

[0064] in, is the thickness of the mortar joint, 、 is the fitting result of the adjacent boundary line of the brick; the mortar joint thickness result is compared with the construction standard to evaluate whether the brick masonry construction quality meets the standard.

[0065] Specifically, when evaluating the horizontality and verticality of bricks, the boundary line equation is fitted using the least squares method to reduce the impact of noise interference on the boundary line calculation. Assume that the point set of the brick boundary is , the horizontal boundary can be fitted with a straight line equation: , calculated by minimizing the sum of squared errors m and b :

[0066]

[0067] Similarly, the vertical boundaries are fitted as: , combined with the included angle between the fitted boundary line and the standard reference line, calculate the horizontal and vertical offset angles:

[0068] .

[0069] By comparing the offset angles with construction quality standards, a quantitative assessment of the horizontality and verticality of the bricks is completed.

[0070] In order to solve the problem of damaged or missing corners on the brick boundary, the system designed a defect detection method based on the distance and angle changes between adjacent points. , calculate the distance between each point and its neighboring points :

[0071]

[0072] And define the angle change as:

[0073]

[0074] in, .like and If the value exceeds the set threshold, it is determined to be a defect point. For the detected defect area, the geometric extrapolation method is used to complete the boundary line. Assume that the endpoint of the defect area is P s and P e , the extrapolation line equation can be obtained by fitting the slope of the straight line in the neighborhood of the endpoint:

[0075] .

[0076] The extrapolated line is seamlessly connected with the complete boundary line to ensure the continuity and reliability of the assessment results.

[0077] In addition, the brickwork construction quality inspection model is used to mark bricks and mortar joints as two types of targets, and the bounding box set of mortar joints is obtained:

[0078]

[0079] Among them, each bounding box B i Indicates the area of ​​the mortar joint. The mortar joint thickness T is defined as the width of the mortar joint area between bricks. Usually, the mortar joint thickness can be calculated by the distance between the mortar joint boundary and the nearest point of the adjacent brick boundary. For a mortar joint area B i The thickness calculation can be simplified to the shortest distance between the mortar joint boundary line and the adjacent brick boundary line. The specific steps are as follows:

[0080] 1) Assume the upper boundary line of the mortar joint area is , the lower boundary line is .They can be expressed by linear equations respectively:

[0081]

[0082] 2) The lower boundary line of the upper brick is , the upper boundary line of the lower brick is , which are also expressed as:

[0083]

[0084] 3) The calculation of the mortar joint thickness Ti is:

[0085]

[0086] in, 、 They are the y-coordinate values ​​of the mortar joint and brick boundary corresponding to position x respectively.

[0087] In actual scenarios, mortar joints are divided into horizontal mortar joints and vertical mortar joints. The calculation methods of the two are slightly different:

[0088] Horizontal mortar joints: Measured along the y direction, the average distance between the upper and lower boundaries:

[0089]

[0090] Vertical gray seam: measured along the x-direction, the average distance between the left and right borders:

[0091]

[0092] in, M is the number of discrete points in the gray seam area.

[0093] First, the present invention designs a defect detection algorithm based on the distance and angle changes of adjacent points, and combines it with a geometric extrapolation strategy to complete the boundary completion of the defect area. This method improves the accuracy of horizontal and vertical calculations while reducing the interference of defects on the evaluation results. Second, a post-processing module specifically for bricks and mortar joints is added to the traditional detection framework to improve the confidence of boundary recognition, and the robustness of the detection results is further enhanced by combining non-maximum suppression. Through least squares fitting and angle offset calculation, the standardization of the evaluation process is achieved, while adapting to the diverse quality assessment needs under different construction scenarios. Third, the introduction of a comprehensive measurement strategy based on boundary fitting geometric methods and discrete point calculations not only improves the accuracy of mortar joint thickness calculations, but also ensures the robustness of the method under a variety of resolutions and shooting conditions.

[0094] On the other hand, the present invention also provides an application system of a multi-view detection method for brickwork construction quality, comprising:

[0095] Data input unit, used to input original data sets;

[0096] A data processing unit, used to perform data enhancement processing and labeling processing on the original data set and the enhanced data set respectively;

[0097] The computational model training unit is used to train the detection model and input the complete data set into the trained brickwork construction quality detection model to obtain the initial detection frame position of the bricks, the initial detection frame position of the mortar joints, and the boundary intersection set between the bricks;

[0098] The data analysis unit performs posture estimation and image angle correction on the complete data set to obtain a corrected data set, and then calculates and outputs the brickwork construction quality assessment results on the corrected data set;

[0099] A display unit displays the brickwork construction quality assessment results.

[0100] The beneficial effects of the present invention are:

[0101] 1. Unlike the existing technology that only relies on a small amount of single data to train the model, the present invention can build a high-performance model under limited data conditions, significantly improving the accuracy and adaptability of brick and mortar joint detection.

[0102] 2. Unlike existing methods that only use original images for detection and evaluation, the present invention can effectively eliminate errors caused by shooting angles and brick defects, greatly improving the assessment accuracy of horizontality and verticality, and meeting the needs of high-precision construction quality inspection.

[0103] 3. Compared with traditional manual measurement or simple pixel distance measurement methods, this invention significantly improves the accuracy of mortar joint thickness calculation and reduces human errors through automated evaluation, providing a more scientific and objective evaluation basis for construction quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0104] Figure 1 The figure is a flow chart of the multi-view detection method for brickwork construction quality in the present invention. DETAILED DESCRIPTION

[0105] The specific embodiments of the present invention are described below to facilitate understanding of the present invention by those skilled in the art. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations utilizing the concepts of the present invention are protected.

[0106] like Figure 1 As shown, the present invention provides step 1, obtaining pictures containing brick masonry construction scenes to form an original data set, which covers pictures of brick masonry construction scenes with different resolutions, lighting conditions and shooting angles; using data enhancement technology to process the original data set to form an enhanced data set, and annotating the enhanced data set to form a complete data set containing category and location information; inputting the complete data set into a detection model for pre-training and fine-tuning, and establishing a brick masonry construction quality detection model.

[0107] In step 1, the original data set is obtained from the Internet through crawler technology or photographed by digital devices. Specifically, the Scrapy crawler is used to crawl multi-resolution, multi-scene, and multi-perspective brickwork images from the Internet. These data come from brick wall photos under different angles and lighting conditions, enriching the diversity of the data set. At the same time, brickwork image data can also be self-collected through digital devices, and the brickwork image data collected on the Internet can be combined with the self-collected brickwork image data to form an original data set, ensuring that the original data set covers more actual construction scenes. This original data set not only provides a basis for the establishment and fine-tuning of the subsequent brickwork construction quality detection model, but also provides diversified original materials for multi-perspective enhancement operations. Finally, a high-precision brickwork construction quality detection model is established to improve the precision and accuracy of target detection.

[0108] Data enhancement techniques include image rotation, scaling, cropping, and lighting and color adjustment; the enhanced dataset is denoted as :

[0109]

[0110] in, is the original dataset, Represents rotation, scaling, and shearing operations respectively.

[0111] The purpose of setting up the enhanced dataset is to generate diverse samples in the spatial and color domains through data augmentation technology, expand the scale of the dataset, improve the adaptability of the subsequently established brick masonry construction quality inspection model to diverse scenarios, and improve the robustness of the brick masonry construction quality inspection model.

[0112] Specifically, the rotation transformation process is as follows: To simulate the impact of shooting angle deviation on the image, the image is randomly transformed as follows:

[0113] ,

[0114] in, The angle is randomly rotated (for example, randomly generated within the range of [-30°, 30°]. This operation can simulate the position changes of bricks when photographed at different angles and train the detection model to adapt to multi-angle detection tasks.

[0115] The scaling and cropping process is to randomly scale and crop the image to enhance the detection model's adaptability to different resolutions and field of view. The scaling ratio is a random value. , cropping is done by defining the center point ( x c , y c ) and the crop frame size ( w ,h ) is generated dynamically to ensure that the target brick remains in the cropped image.

[0116] The process of light and color adjustment is: simulate the change of light conditions, and randomly change the brightness b , contrast c and saturation s: ,in I is the original image, This method enhances the robustness of the model to different lighting environments.

[0117] The detection model is the YOLOv8 model. The brickwork quality inspection model is built using the following principles: The system uses the YOLOv8 model for brick and mortar joint detection, combining pre-training and fine-tuning techniques. Pre-training is first performed on a large public dataset (i.e., the original dataset) to obtain common feature representations. Fine-tuning is then performed on an augmented dataset to optimize the model's adaptability to the specific task, ultimately establishing a brickwork quality inspection model. To further improve the accuracy of the brickwork quality inspection model, specific post-processing strategies are introduced, including non-maximum suppression (NMS) to remove duplicate detection results. The model ultimately outputs a set of bounding boxes for bricks and mortar joints.

[0118] Specifically, the method for forming a complete dataset containing category and location information is as follows: use the LabelImg tool to annotate the original dataset and the enhanced dataset, generate the bounding boxes of the brick and mortar joint areas, form a complete dataset, parse the complete dataset, and save it in the YOLO annotation format. The complete dataset is recorded as ( x , y , w , h ),in, x , y Respectively represent the horizontal and vertical coordinates of the center point of the pixel calibration point in the image; w , h Represents the width and length of the crop box size respectively.

[0119] The same annotation strategy is used to generate labels for each image in the augmented dataset, ensuring consistent quality. This multi-view data processing and augmentation technology not only expands and increases data diversity, but also effectively mitigates the model's overfitting problem with limited data during training, providing a solid data foundation for subsequent brick detection, posture correction, and construction quality assessment.

[0120] Furthermore, in step 1, the brickwork construction quality detection model includes a CNN layer for feature extraction, a linear layer for class mapping, and a Softmax layer for outputting the probability distribution of bricks and mortar joints. After the brickwork construction quality detection model is established, the non-maximum suppression (NMS) algorithm is used to remove redundant detection frames, retaining only the target area with the highest confidence: , its goal is to optimize the confidence of boundary classification and make the detection results more reliable.

[0121] Through multi-view data processing and data enhancement technology, not only can data expansion and diversity be improved, but the overfitting problem of the detection model on limited data can also be effectively alleviated during the training process, providing a solid data foundation for subsequent brick detection, posture correction and construction quality assessment.

[0122] Step 2: Use digital equipment to shoot the actual brick wall construction scene to form a high-resolution, multi-view actual brick wall image dataset.

[0123] Step 3: Input the actual brick wall image dataset into the brickwork construction quality inspection model for target detection, obtain the initial detection frame positions of the bricks and the initial detection frame positions of the mortar joints, and extract the boundary intersection set between the bricks based on the initial detection frame positions of the bricks and the initial detection frame positions of the mortar joints.

[0124] In step 3, the initial detection frame positions of bricks and mortar joints are recorded as { b i};

[0125] Extract the boundary intersection set between bricks through the vertex coordinates of the initial detection box of bricks and mortar joints:

[0126]

[0127] in, P k The two-bit pixel coordinates of each point, ( x k , y k ) are the horizontal and vertical coordinates of the two-dimensional pixel respectively; N is the total number of intersections detected.

[0128] Step 4: Correct the boundary intersection point set between bricks using the 3D point set of the standard brick layout to obtain a corrected data set.

[0129] In step 4, the method for obtaining the corrected dataset is:

[0130] Using the boundary intersection set between bricks P and the 3D point set of the standard brick layout Q To establish a projection relationship:

[0131]

[0132] in, R is the rotation matrix, T is the translation vector, is the projection point set of the image space; according to the EPnP algorithm, R and T , perform inverse transformation on the input image in the complete dataset, correct the shooting angle deviation, and obtain the corrected dataset Specifically, the boundary intersection set between bricks is corrected by combining a pose estimation algorithm with image angle correction. The pose estimation algorithm and image angle correction can effectively reduce the interference of shooting angle deviation on the horizontality and verticality assessment of bricks, thereby improving detection accuracy and robustness.

[0133] Specifically, the process of the pose estimation algorithm is to calculate the rotation and translation parameters of the brickwork plane in three-dimensional space by inputting the pixel coordinates of the calibration points in the image. In this solution, the EPnP (EfficientPerspective-n-Point) algorithm is selected for pose estimation. Its main purpose is to derive three-dimensional pose information from two-dimensional image points. Assume that there is a n calibration points, and the two-dimensional pixel coordinates of each point are , the corresponding three-dimensional world coordinates are The camera intrinsic parameter matrix is K , the external parameter matrix is ,in R is the rotation matrix, T is the translation vector. The EPnP algorithm solves the extrinsic parameters by minimizing the projection error:

[0134] ,

[0135] ,

[0136] in is the actual projection point coordinate. In data preprocessing, the brick boundary intersection points or feature points are accurately extracted to ensure that the calibration points input to the EPnP algorithm have high-quality correspondence. The selection strategy of these points optimizes the uniformity of the calibration point distribution and reduces the error accumulation caused by boundary defects. The output of the EPnP algorithm includes the rotation matrix R and translation vectors T Furthermore, the solution can be optimized by combining the geometric constraints in the shooting scene. For example, if the brickwork is parallel to the ground, the ground plane normal vector can be constrained to (0, 0, 1), thereby optimizing R precision.

[0137] The process of image angle correction is as follows: after completing the pose estimation, the goal of image angle correction is to align the horizontal edge of the brickwork with the horizontal axis of the image and the vertical edge with the vertical axis of the image, thereby providing standardized input for the subsequent horizontality and verticality evaluation. The implementation of affine transformation is based on the known external parameters , design an affine transformation to adjust the image to the standard posture. Let the pixel coordinates of any point in the original image be ( x , y ), the coordinates of the corrected point are ( x′ , y′ ), the transformation formula is:

[0138]

[0139] in H is the affine transformation matrix, which is obtained by solving the pose estimation result. Specifically, the generation process of the affine matrix includes the following steps:

[0140] 1) Extract the key boundary feature points of the brickwork and determine the horizontal and vertical baselines;

[0141] 2) According to R Decomposition result, calculate the rotation angle of the baseline in the image θ ;

[0142] 3) Generate rotation correction matrix: .

[0143] In addition, to further improve the correction accuracy, this solution designs an auxiliary correction strategy based on the intersection of brick boundaries. Detect all brick boundaries in the image and extract the intersection set of boundary lines. . Assume that the intersection point set after standard correction is , use the least squares method to fit the transformation matrix: .

[0144] In summary, in step 4, the EPnP algorithm is combined with geometric constraints to improve the rotation matrix through multi-stage optimization. R The solution accuracy is high. In particular, in actual construction scenarios, the geometric relationship between brickwork and the ground provides strong constraints, effectively eliminating noise interference. The overall process design, combining pose estimation and image correction, not only improves the accuracy of brick horizontality and verticality assessment, but also further enhances the robustness of the algorithm through innovative defect identification and geometric optimization strategies, providing theoretical support and technical assurance for multi-view construction quality inspection.

[0145] Furthermore, the deviation angle of the brick boundary line in the correction data set is calculated :

[0146]

[0147] Ensure the deviation angle Within the allowable error range, x 1. x 2. y 1. y 2 are the pixel coordinates of the brick boundary line, x 1 ’ 、 x 2 ’ 、 y 1 ’ 、 y 2 ’ are the pixel coordinates of the brick boundary lines after correction.

[0148] Step 5: After performing boundary line fitting, defect detection, horizontality and verticality evaluation, and mortar joint thickness measurement on the images in the corrected dataset, it is evaluated whether the brickwork construction quality meets the standards.

[0149] In step 5, boundary line fitting involves fitting the straight line equation using the least squares method using the boundary points of the bricks in the rectified dataset:

[0150]

[0151] in, m , c To fit the parameters, minimize the error function:

[0152]

[0153] in,( x i , y i ) is the point set of the brick boundary; m , c is the fitting parameter;

[0154] Defect detection includes: analyzing the Euclidean distance of adjacent boundary points and angle changes :

[0155]

[0156] in,{ p i} is a discrete point set, p i+1 - p i is the distance between each point and its neighboring points; if >threshod1 or >threshod2, determine it as a defect point and complete the boundary through geometric extrapolation;

[0157] The horizontality and verticality assessment involves comparing the fitted boundary line with the standard line and calculating the horizontality and verticality errors:

[0158]

[0159] in, y i is the fitted horizontal boundary line, y std is the standard horizontal boundary line, is the brick level error, x i is the fitted vertical boundary line, x std is the standard vertical boundary line, is the verticality error of the brick, N is the number of discrete points in the brick area;

[0160] The mortar joint thickness measurement includes: calculating the minimum distance between adjacent boundary lines based on the brick boundary line fitting results:

[0161]

[0162]

[0163] in, is the thickness of the mortar joint, 、 is the fitting result of the adjacent boundary line of the brick; the mortar joint thickness result is compared with the construction standard to evaluate whether the brick masonry construction quality meets the standard.

[0164] Specifically, when evaluating the horizontality and verticality of bricks, the boundary line equation is fitted using the least squares method to reduce the impact of noise interference on the boundary line calculation. Assume that the point set of the brick boundary is , the horizontal boundary can be fitted with a straight line equation: , calculated by minimizing the sum of squared errors m and b :

[0165]

[0166] Similarly, the vertical boundaries are fitted as: , combined with the included angle between the fitted boundary line and the standard reference line, calculate the horizontal and vertical offset angles:

[0167] .

[0168] By comparing the offset angles with construction quality standards, a quantitative assessment of the horizontality and verticality of the bricks is completed.

[0169] In order to solve the problem of damaged or missing corners on the brick boundary, the system designed a defect detection method based on the distance and angle changes between adjacent points. , calculate the distance between each point and its neighboring points :

[0170]

[0171] And define the angle change as:

[0172]

[0173] in, .like and If the value exceeds the set threshold, it is determined to be a defect point. For the detected defect area, the geometric extrapolation method is used to complete the boundary line. Assume that the endpoints of the defect area are P s and P e , the extrapolation line equation can be obtained by fitting the slope of the straight line in the neighborhood of the endpoint:

[0174] .

[0175] The extrapolated line is seamlessly connected with the complete boundary line to ensure the continuity and reliability of the assessment results.

[0176] In addition, the brickwork construction quality inspection model is used to mark bricks and mortar joints as two types of targets, and the bounding box set of mortar joints is obtained:

[0177]

[0178] Each bounding box B i Indicates the area of ​​the mortar joint. The mortar joint thickness T is defined as the width of the mortar joint area between bricks. Usually, the mortar joint thickness can be calculated by the distance between the mortar joint boundary and the nearest point of the adjacent brick boundary. For a mortar joint area B i The thickness calculation can be simplified to the shortest distance between the mortar joint boundary line and the adjacent brick boundary line. The specific steps are as follows:

[0179] 1) Assume the upper boundary line of the mortar joint area is , the lower boundary line is .They can be expressed by linear equations respectively:

[0180]

[0181] 2) The lower boundary line of the upper brick is , the upper boundary line of the lower brick is , which are also expressed as:

[0182]

[0183] 3) Thickness of mortar joints T i The calculation is:

[0184]

[0185] in, 、 The corresponding positions x The mortar joints and brick boundaries y Coordinate value.

[0186] In actual scenarios, mortar joints are divided into horizontal mortar joints and vertical mortar joints. The calculation methods of the two are slightly different:

[0187] Horizontal mortar joints: along y Directional measure, average distance between upper and lower bounds:

[0188]

[0189] Vertical mortar joints: along x Directional measure, average distance between left and right boundaries:

[0190]

[0191] in, M is the number of discrete points in the gray seam area.

[0192] First, the present invention designs a defect detection algorithm based on the distance and angle changes of adjacent points, and combines it with a geometric extrapolation strategy to complete the boundary completion of the defect area. This method improves the accuracy of horizontal and vertical calculations while reducing the interference of defects on the evaluation results. Second, a post-processing module specifically for bricks and mortar joints is added to the traditional detection framework to improve the confidence of boundary recognition, and the robustness of the detection results is further enhanced by combining non-maximum suppression. Through least squares fitting and angle offset calculation, the standardization of the evaluation process is achieved, while adapting to the diverse quality assessment needs under different construction scenarios. Third, the introduction of a comprehensive measurement strategy based on boundary fitting geometric methods and discrete point calculations not only improves the accuracy of mortar joint thickness calculations, but also ensures the robustness of the method under a variety of resolutions and shooting conditions.

[0193] The present invention also provides an application system of a multi-view detection method for brickwork construction quality, comprising:

[0194] Data input unit, used to input original data sets;

[0195] A data processing unit, used to perform data enhancement processing and labeling processing on the original data set and the enhanced data set respectively;

[0196] The computational model training unit is used to train the detection model and input the complete data set into the trained brickwork construction quality detection model to obtain the initial detection frame position of the bricks, the initial detection frame position of the mortar joints, and the boundary intersection set between the bricks;

[0197] The data analysis unit performs posture estimation and image angle correction on the complete data set to obtain a corrected data set, and then calculates and outputs the brickwork construction quality assessment results on the corrected data set;

[0198] A display unit displays the brickwork construction quality assessment results.

[0199] In summary, the present invention provides a multi-perspective detection method and application system for brick masonry construction quality, which obtains brick masonry images of different resolutions by combining crawler technologies such as Scrapy, and uses transformation methods such as rotation and scaling to achieve multi-perspective data enhancement, providing diverse training data for model training. Ultimately, the fine-tuning of the detection model is achieved, a brick masonry construction quality detection model is established, and the precision and accuracy of target detection are improved. The combined application of posture estimation algorithm and image angle correction: by introducing the EPnP posture estimation algorithm to correct the image angle, and combining the brick boundary intersection information for auxiliary correction, subsequent target detection is made more accurate, providing more accurate input data for horizontal verticality assessment and mortar joint thickness calculation. By introducing the defect detection function, the presence of brick defects is judged based on the example and angle changes between adjacent points, and the boundary line is completed in combination with the geometric extrapolation method to enhance the accuracy of horizontal verticality assessment. At the same time, the mortar joint thickness is calculated based on the parallel boundary lines to ensure that the detection results meet the construction standards, which can effectively improve the automation level and accuracy of construction quality management and monitoring.

Claims

1. A multi-view detection method for brickwork construction quality, characterized in that: include: Step 1: Acquire pictures containing brickwork construction scenes to form an original dataset, which covers pictures of brickwork construction scenes with different resolutions, lighting conditions, and shooting angles; The original dataset is processed using data enhancement technology to form an enhanced dataset, and the enhanced dataset is annotated to form a complete dataset containing category and location information; The complete dataset was input into the detection model for pre-training and fine-tuning to establish a brickwork construction quality detection model; Step 2: Use digital equipment to shoot the actual brick wall construction scene to form a high-resolution, multi-viewpoint actual brick wall image dataset; Step 3: Input the actual brick wall image dataset into the brickwork construction quality inspection model for target detection, obtain the initial detection frame positions of the bricks and the initial detection frame positions of the mortar joints, and extract the boundary intersection set between the bricks based on the initial detection frame positions of the bricks and the initial detection frame positions of the mortar joints; Step 4: Correct the boundary intersection point set between bricks by using the 3D point set of the standard brick layout to obtain the corrected data set; P and the 3D point set of the standard brick layout Q To establish a projection relationship: in, R is the rotation matrix, T is the translation vector, is the projection point set of the image space; according to the EPnP algorithm, R and T , perform inverse transformation on the input image in the complete dataset, correct the shooting angle deviation, and obtain the corrected dataset ; Step 5: After performing boundary line fitting, defect detection, horizontality and verticality evaluation, and mortar joint thickness measurement on the images in the corrected dataset, it is evaluated whether the brickwork construction quality meets the standards.

2. The multi-view detection method for brickwork construction quality according to claim 1, characterized in that: In step 1, the original dataset is obtained from the Internet through crawler technology or photographed by digital devices.

3. The multi-view detection method for brickwork construction quality according to claim 2, characterized in that: In step 1, the data enhancement technology includes image rotation, scaling and cropping, and lighting and color adjustment; the enhanced data set is recorded as : in, is the original dataset, Represents rotation, scaling, and shearing operations respectively.

4. The multi-view detection method for brickwork construction quality according to claim 3, characterized in that: In step 1, the detection model is the YOLOv8 model; the method for forming a complete data set containing category and location information is as follows: use the LabelImg tool to annotate the original data set and the enhanced data set, generate the bounding boxes of the brick and mortar joint areas, form a complete data set, parse the complete data set, and save it in the YOLO annotation format. The complete data set is recorded as ( x , y , w , h ),in, x , y Respectively represent the horizontal and vertical coordinates of the center point of the pixel calibration point in the image; w , h Represents the width and length of the crop box size respectively.

5. The multi-view detection method for brickwork construction quality according to claim 4, characterized in that: In step 1, the brickwork construction quality detection model includes a CNN layer for feature extraction, a linear layer for class mapping, and a Softmax layer for outputting the probability distribution of bricks and mortar joints. After the brickwork construction quality detection model is established, the non-maximum suppression (NMS) algorithm is used to remove redundant detection boxes, retaining only the target area with the highest confidence: .

6. The multi-view detection method for brickwork construction quality according to claim 4, characterized in that: In step 3, the initial detection frame position of the brick and the initial detection frame position of the mortar joint are recorded as {b i }; Extract the boundary intersection set between bricks through the vertex coordinates of the initial detection box of bricks and mortar joints: in, P k The two-bit pixel coordinates of each point, ( x k , y k ) are the horizontal and vertical coordinates of the two-dimensional pixel respectively; N is the total number of detected intersections.

7. The multi-view detection method for brickwork construction quality according to claim 1, characterized in that: Calculate the deviation angle of the brick boundary line in the correction dataset : ； Ensure the deviation angle Within the allowable error range, x 1. x 2. y 1. y 2 are the pixel coordinates of the brick boundary line, x 1 ’ 、 x 2 ’ 、 y 1 ’ 、 y 2 ’ are the pixel coordinates of the brick boundary lines after correction.

8. The multi-view detection method for brickwork construction quality according to claim 1, characterized in that: In step 5, boundary line fitting includes: using the boundary points {pi} of the bricks in the rectified dataset, fitting the straight line equation by the least squares method: in, m , c To fit the parameters, minimize the error function: in,( x i , y i ) is the point set of the brick boundary; m , c is the fitting parameter; Defect detection includes: analyzing the Euclidean distance of adjacent boundary points and angle changes : in,{ p i } is a discrete point set, p i+1 - p i is the distance between each point and its neighboring points; if >threshod1 or >threshod2, determine it as a defect point and complete the boundary through geometric extrapolation; The horizontality and verticality assessment involves comparing the fitted boundary line with the standard line and calculating the horizontality and verticality errors: in, y i is the fitted horizontal boundary line, y std is the standard horizontal boundary line, is the brick level error, x i is the fitted vertical boundary line, x std is the standard vertical boundary line, is the verticality error of the brick, N is the number of discrete points in the brick area; The mortar joint thickness measurement includes: calculating the minimum distance between adjacent boundary lines based on the brick boundary line fitting results: in, is the thickness of the mortar joint, 、 is the fitting result of the adjacent boundary line of the brick; the mortar joint thickness result is compared with the construction standard to evaluate whether the brick masonry construction quality meets the standard.

9. An application system based on the multi-view detection method for brickwork construction quality according to any one of claims 1 to 8, characterized in that: include: Data input unit, used to input original data sets; A data processing unit, used to perform data enhancement processing and labeling processing on the original data set and the enhanced data set respectively; The computational model training unit is used to train the detection model and input the complete data set into the trained brickwork construction quality detection model to obtain the initial detection frame position of the bricks, the initial detection frame position of the mortar joints, and the boundary intersection set between the bricks; The data analysis unit performs posture estimation and image angle correction on the complete data set to obtain a corrected data set, and then calculates and outputs the brickwork construction quality assessment results on the corrected data set; A display unit displays the brickwork construction quality assessment results.

Images