Multi-view detection method and application system for brick masonry construction quality

Abstract

The invention discloses a multi-view detection method for brick masonry construction quality and an application system, and the method combines a web crawler technology and own data for data enhancement in a data processing part, obtains rich training data through multi-view transformation, and improves the generalization ability of a model. The problem that data is insufficient when an existing method is used for processing a complex scene is solved. In the test stage, through an EPnP attitude estimation algorithm and brick boundary intersection point auxiliary correction, the image angle problem is effectively solved, and the detection accuracy is improved. An existing method is often poor in effect when image angle changes are processed, so that a detection result is deviated. In the evaluation stage, a corner defect detection function is introduced, boundary line completion is carried out through a geometric extrapolation method, and the accuracy of levelness and perpendicularity evaluation is improved. And meanwhile, the defect detection function also provides powerful support for monitoring the construction quality. Compared with the prior art, the existing method often neglects the influence of corner defects on the result in the evaluation process, so that the evaluation result is not accurate enough.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and deep learning, and in particular relates to a multi-viewing angle detection method and application system for brickwork construction quality. Background Art

[0002] Brick masonry is a common structural form in construction projects, and its construction quality directly affects the stability and service life of the building. The construction quality of brick masonry is mainly reflected in the arrangement accuracy of bricks, the thickness of mortar joints, the verticality and horizontality of masonry, and other aspects. Therefore, how to effectively detect and evaluate the construction quality of brick masonry has become an important topic in the current construction industry. Traditional brick masonry construction quality detection methods mostly rely on manual inspection or simple instrument measurement. These methods have many shortcomings, such as low detection efficiency, high labor costs, and large errors. In order to improve the accuracy and efficiency of detection, in recent years, computer vision, artificial intelligence, and deep learning technologies have gradually been introduced into the evaluation of brick masonry construction quality, especially in the application of target detection, image processing, and deep learning models, showing great potential.

[0003] At present, the traditional methods for brick masonry construction quality inspection mainly include manual inspection, measuring tool inspection and image-based inspection methods. Although manual inspection is simple and easy, it is prone to omissions and misjudgments due to its reliance on manual operation. Especially in complex and large-scale construction sites, manual inspection is inefficient and highly subjective. Measuring tool inspection requires the use of measuring instruments (such as levels, verticality meters, calipers, etc.) to perform physical measurements of brick masonry. These tools may be effective for small-scale inspections, but they are difficult to fully cover complex structures and large-area brick masonry construction, and their efficiency and accuracy are still limited. In recent years, image processing technology has been introduced into brick masonry construction quality inspection. By collecting brick masonry images at the construction site and combining image analysis and processing technology, accurate measurement and evaluation of brick masonry can be achieved. Image detection methods process images of brick masonry to extract information such as brick boundaries, mortar joint thickness, verticality and horizontality, and can achieve non-contact detection, reduce manual intervention, and improve the efficiency and accuracy of detection. However, traditional image processing methods still have some challenges, such as complex background, low image resolution, occlusion or stains on the brickwork surface, which affect the accuracy of detection. Summary of the invention

[0004] In view of the deficiencies in the above-mentioned background technology, the present invention aims to provide a multi-view detection method and application system for brickwork construction quality, which solves the problem that traditional brickwork construction quality detection methods have deficiencies in detection efficiency and detection accuracy.

[0005] In order to achieve the above-mentioned purpose, the technical solution adopted by the present invention is as follows: On the one hand, the present invention provides a multi-viewing method for detecting the construction quality of brickwork, which is characterized by comprising: Step 1: Obtain pictures containing brickwork construction scenes to form an original data set, which covers pictures of brickwork construction scenes with different resolutions, lighting conditions and shooting angles; use data enhancement technology to process the original data set to form an enhanced data set, and annotate the enhanced data set to form a complete data set containing category and location information; input the complete data set into the detection model for pre-training and fine-tuning, and establish a brickwork construction quality detection model; Step 2: Use digital equipment to shoot the actual brick wall construction scene to form a high-resolution, multi-view actual brick wall image dataset; Step 3: Input the actual brick wall image dataset into the brickwork construction quality detection model for target detection, and obtain the initial detection frame position of the bricks, the initial detection frame position of the mortar joints, and the boundary intersection set between the bricks; Step 4, correcting the boundary intersection point set between bricks through the three-dimensional point set of the standard brick layout to obtain a corrected data set; Step 5: After performing boundary line fitting, defect detection, horizontality and verticality evaluation, and mortar joint thickness measurement on the images in the corrected data set, it is evaluated whether the brickwork construction quality meets the standards.

[0006] A multi-view detection method for brick masonry construction quality in the present invention can construct a brick masonry construction quality detection model under limited image data conditions, significantly improving the accuracy and adaptability of brick and mortar joint detection, replacing manual detection of brick masonry construction quality, improving the efficiency of detecting brick masonry construction quality, reducing human errors, and providing a more scientific and objective evaluation basis for construction quality.

[0007] Furthermore, in step 1, the original data set is obtained from the Internet through crawler technology or photographed by a digital device. Specifically, crawler technology (such as Scrapy) can be used to crawl brick masonry images with multiple resolutions, multiple scenes, and multiple perspectives from the Internet. These data are derived from brick wall photos under different angles and lighting conditions, enriching the diversity of the data set. At the same time, brick masonry image data can also be collected by digital devices, and the brick masonry image data collected on the Internet can be combined with the self-collected brick masonry image data to form an original data set, ensuring that the original data set covers more actual construction scenes. This original data set not only provides a basis for the establishment and fine-tuning of the subsequent brick masonry construction quality detection model, but also provides diversified original materials for multi-perspective enhancement operations. Finally, a high-precision brick masonry construction quality detection model is established to improve the precision and accuracy of target detection.

[0008] Furthermore, in step 1, the data enhancement technology includes rotation transformation, scaling and cropping, and lighting and color adjustment of the image; the enhanced data set is recorded as :

[0009] in, is the original dataset, Represents rotation, scaling, and shearing operations respectively.

[0010] The purpose of setting the enhanced dataset is to generate diverse samples in space and color domain through data enhancement technology, expand the scale of the dataset, improve the adaptability of the subsequently established brickmasonry construction quality inspection model to diverse scenarios, and improve the robustness of the brickmasonry construction quality inspection model.

[0011] Specifically, the process of rotation transformation is as follows: to simulate the influence of shooting angle deviation on the image, the angle transformation operation is randomly performed on the image, as described below:

[0012] in, is a random rotation angle (for example, randomly generated in the range of [-30°, 30°]). This operation can simulate the position changes of bricks when photographed at different angles and train the detection model to adapt to multi-angle detection tasks.

[0013] The process of scaling and cropping is to randomly scale and crop the image to enhance the detection model's adaptability to different resolutions and field of view. The scaling ratio is a random value. , cropping is done by defining the center point ( x c , y c ) and the crop box size ( w , h ) is generated dynamically to ensure that the target brick remains in the cropped image.

[0014] The process of light and color adjustment is: simulate the change of light conditions, and randomly change the brightness b , contrast c and Saturation s : ,in I is the original image, I ’ This method enhances the robustness of the model to different lighting environments.

[0015] Furthermore, in step 1, the detection model is a YOLOv8 model; the principle of establishing a brickwork construction quality detection model is as follows: the system uses a YOLOv8 model to detect bricks and mortar joints, combining pre-training and fine-tuning techniques. First, pre-training is performed on a large public dataset (i.e., the original dataset) to obtain a general feature representation; then fine-tuning is performed in combination with an enhanced dataset to optimize the model's adaptability to specific tasks, and finally a brickwork construction quality detection model is established. In order to further improve the detection accuracy of the brickwork construction quality detection model, a specific post-processing strategy is introduced, including non-maximum suppression (NMS), which is used to remove duplicate detection results, and finally a set of bounding boxes of bricks and mortar joints is output.

[0016] Specifically, the method for forming a complete data set containing category and location information is as follows: use the LabelImg tool to annotate the original data set and the enhanced data set, generate the bounding boxes of the brick and mortar joint areas, form a complete data set, parse the complete data set, and save it in the YOLO annotation format. The complete data set is recorded as ( x , y , w , h ),in, x , y Respectively represent the horizontal and vertical coordinates of the center point of the pixel calibration point in the image; w , h Respectively represent the width and length of the crop box size.

[0017] For each image in the enhanced data set, the same annotation strategy is used to generate corresponding labels to ensure the consistency of the enhanced data quality. Through the above multi-view data processing and data enhancement technology, not only the data expansion and diversity are achieved, but also the overfitting problem of the model to limited data is effectively alleviated during the training process, providing a solid data foundation for subsequent brick detection, posture correction and construction quality assessment.

[0018] Furthermore, in step 1, the brickwork construction quality detection model includes a CNN layer for extracting features, a linear layer for completing category mapping, and a Softmax layer for outputting probability distribution of bricks and mortar joints; after the brickwork construction quality detection model is established, the non-maximum suppression (NMS) algorithm is used to remove redundant detection frames, and only the target area with the highest confidence is retained: , whose goal is to optimize the confidence of boundary classification and make the detection results more reliable.

[0019] In step 1 of the invention, through multi-view data processing and data enhancement technology, not only the expansion and diversity of data are achieved, but also the overfitting problem of the detection model on limited data is effectively alleviated during the training process, providing a solid data foundation for subsequent brick detection, posture correction and construction quality assessment.

[0020] Furthermore, in step 3, the initial detection frame position of the brick and the initial detection frame position of the mortar joint are recorded as {b i}; Through the vertex coordinates of the initial detection box of bricks and mortar joints, the boundary intersection set between bricks is extracted:

[0021] in, P k is the two-bit pixel coordinate of each point, ( x k , y k ) are the horizontal and vertical coordinates of the two-dimensional pixel respectively; N is the total number of detected intersections.

[0022] Furthermore, in step 4, the method for obtaining the corrected data set is: Using the boundary intersection set between bricks P and the 3D point set of the standard brick layout Q To establish a projection relationship:

[0023] in, R is the rotation matrix, T is the translation vector, is the projection point set in the image space; solved by the EPnP algorithm R and T , perform inverse transformation on the input image in the complete dataset, correct the shooting angle deviation, and obtain the corrected dataset Specifically, the boundary intersection set between bricks is corrected by combining the posture estimation algorithm and image angle correction. The posture estimation algorithm and image angle correction can effectively reduce the interference of shooting angle deviation on the horizontality and verticality evaluation of bricks, and improve the detection accuracy and robustness.

[0024] Specifically, the process of the pose estimation algorithm is: by inputting the pixel coordinates of the calibrated points in the image, the rotation and translation parameters of the brickwork plane in the three-dimensional space are solved. In this scheme, the EPnP (EfficientPerspective-n-Point) algorithm is selected for pose estimation, and its main purpose is to derive three-dimensional pose information from two-dimensional image points. Assume that there is a n calibration points, the two-dimensional pixel coordinates of each point are , the corresponding three-dimensional world coordinates are The camera intrinsic parameter matrix is K , the external parameter matrix is ,in R is the rotation matrix, T is the translation vector. The EPnP algorithm solves the external parameters by minimizing the projection error: ,

[0025] in is the actual projection point coordinate. In data preprocessing, the brick boundary intersection points or feature points are accurately extracted to ensure that the calibration points input to the EPnP algorithm have high-quality correspondence. The selection strategy of these points optimizes the uniformity of the calibration point distribution and reduces the error accumulation caused by boundary defects. The output of the EPnP algorithm includes the rotation matrix R and translation vector T Furthermore, the solution can be optimized by combining the geometric constraints in the shooting scene. For example, if the brickwork is parallel to the ground, the ground plane normal vector can be constrained to (0, 0, 1), thereby optimizing R accuracy.

[0026] The process of image angle correction is as follows: after completing the pose estimation, the goal of image angle correction is to align the horizontal edge of the brickwork with the horizontal axis of the image and the vertical edge with the vertical axis of the image, thereby providing standardized input for subsequent horizontal and vertical evaluation. The implementation of affine transformation is based on the known external parameters , design an affine transformation to adjust the image to the standard posture. Let the pixel coordinates of any point in the original image be ( x ,, y ), the coordinates of the corrected point are ( x ′, y ′), the transformation formula is:

[0027] in H is the affine transformation matrix, which is obtained by solving the posture estimation result. Specifically, the generation process of the affine matrix includes the following steps: 1) Extract the characteristic points of the key boundaries of the brickwork and determine the horizontal and vertical reference lines; 2) According to R Decomposition result, calculate the rotation angle of the baseline in the image θ ; 3) Generate rotation correction matrix: .

[0028] In addition, to further improve the correction accuracy, this scheme designs an auxiliary correction strategy based on the intersection of brick boundaries. Detect all brick boundaries in the image and extract the intersection set of boundary lines. . Assume that the intersection point set after standard correction is , use the least squares method to fit the transformation matrix: .

[0029] In summary, in step 4, the EPnP algorithm is combined with geometric constraints to improve the rotation matrix through multi-stage optimization. R The solution accuracy is improved. Especially in actual construction scenarios, the geometric relationship between brickwork and the ground provides strong constraints, which can effectively eliminate the interference of noise points. The overall process design combining posture estimation and image correction not only improves the accuracy of brick horizontality and verticality assessment, but also further enhances the robustness of the algorithm through innovative defect recognition and geometric optimization strategies, providing theoretical support and technical guarantee for multi-view construction quality inspection.

[0030] Furthermore, the deviation angle of the brick boundary line in the correction data set is calculated :

[0031] Ensure the deviation angle Within the allowable error range, x 1. x 2. y 1. y 2 are the pixel coordinates of the brick boundary lines, x 1 ’ , x 2 ’ , y 1 ’ , y 2 ’ They are the pixel coordinates of the brick boundary lines after correction.

[0032] Furthermore, in step 5, the boundary line fitting includes: using the boundary points of the bricks in the correction data set, fitting the straight line equation by the least square method:

[0033] in, m , c To fit the parameters, minimize the error function:

[0034] in,( x i , y i ) is the point set of the brick boundary; m ,c is the fitting parameter; Defect detection includes: Analysis of the Euclidean distance of neighboring boundary points and angle changes :

[0035] in,{ p i} is a discrete point set, p i+1 - p i is the distance between each point and its neighboring points; if >threshod1 or >threshod2, determine it as a defect point and complete the boundary through geometric extrapolation; The horizontality and verticality assessment includes: comparing the fitted boundary line with the standard line and calculating the horizontality and verticality errors:

[0036] in, y i is the fitted horizontal boundary line, y std is the standard horizontal boundary line, is the brick level error, x i is the fitted vertical boundary line, x std is the standard vertical boundary line, is the verticality error of the brick, N is the number of discrete points in the brick area; The measurement of mortar joint thickness includes: calculating the minimum distance between adjacent boundary lines based on the brick boundary line fitting results:

[0037]

[0038] in, is the thickness of the mortar joint, , is the fitting result of the brick adjacent boundary line; the mortar joint thickness result is compared with the construction standard to evaluate whether the brick masonry construction quality meets the standard.

[0039] Specifically, when evaluating the horizontality and verticality of bricks, the least squares method is used to fit the boundary line equation, thereby reducing the impact of noise interference on the boundary line calculation. Assume that the point set of the brick boundary is , the horizontal boundary can be fitted to the equation of a straight line: , calculated by minimizing the sum of squared errors m and b :

[0040] Similarly, the vertical boundaries are fitted as: , combined with the included angle between the fitted boundary line and the standard reference line, calculate the horizontal and vertical offset angles: .

[0041] By comparing the offset angles with construction quality standards, a quantitative assessment of the horizontality and verticality of the bricks is completed.

[0042] In view of the possible damage or missing corners of the brick boundary, the system has designed a defect detection method based on the distance and angle changes between adjacent points. For the fitted boundary line L, its discrete point set is extracted , calculate the distance between each point and its neighboring points :

[0043] And define the angle change as:

[0044] in, .like and If the value exceeds the set threshold, it is determined to be a defect point. For the detected defect area, the geometric extrapolation method is used to complete the boundary line. Assume that the endpoint of the defect area is P s and P e , the extrapolation line equation can be obtained by fitting the slope of the straight line in the neighborhood of the endpoint: .

[0045] The extrapolated line is seamlessly connected with the complete boundary line to ensure the continuity and reliability of the assessment results.

[0046] In addition, the bricks and mortar joints are marked as two types of targets through the brickwork construction quality inspection model, and the bounding box set of the mortar joints is obtained:

[0047] Among them, each bounding box B i Indicates the area of ​​the mortar joint. The mortar joint thickness T is defined as the width of the mortar joint area between bricks. Usually, the mortar joint thickness can be calculated by the distance between the mortar joint boundary and the nearest point of the adjacent brick boundary. For a mortar joint area B i The thickness calculation can be simplified to the shortest distance between the mortar joint boundary line and the adjacent brick boundary line. The specific steps are as follows: 1) Assume the upper boundary line of the mortar joint area is , the lower boundary line is .They can be expressed by linear equations respectively:

[0048] 2) The lower boundary line of the upper brick is , the upper boundary line of the lower brick is , which can also be expressed as:

[0049] 3) The calculation of the mortar joint thickness Ti is:

[0050] in, , They are the y-coordinate values ​​of the mortar joint and brick boundary corresponding to position x respectively.

[0051] In actual scenarios, there are two types of mortar joints: horizontal mortar joints and vertical mortar joints. The calculation methods for the two are slightly different: Horizontal gray seam: measured along the y direction, the average distance between the upper and lower boundaries:

[0052] Vertical gray seam: measured along the x direction, the average distance between the left and right borders:

[0053] in, M is the number of discrete points in the gray seam area.

[0054] First, the present invention designs a defect detection algorithm based on the distance and angle changes of neighboring points, and completes the boundary completion of the defect area in combination with the geometric extrapolation strategy. This method improves the accuracy of horizontal and vertical calculations while reducing the interference of defects on the evaluation results. Second, a post-processing module specifically for bricks and mortar joints is added to the traditional detection framework to improve the confidence of boundary recognition, and the robustness of the detection results is further enhanced by combining non-maximum suppression. Through least squares fitting and angle offset calculation, the standardization of the evaluation process is achieved, while adapting to the diverse quality assessment needs in different construction scenarios. Third, the introduction of a comprehensive measurement strategy based on boundary fitting geometric methods and discrete point calculations not only improves the accuracy of mortar joint thickness calculations, but also ensures the robustness of the method under a variety of resolutions and shooting conditions.

[0055] On the other hand, the present invention also provides an application system of a multi-viewing angle detection method for brickwork construction quality, which comprises: A data input unit, used to input the original data set; A data processing unit, used to perform data enhancement processing and labeling processing on the original data set and the enhanced data set respectively; The computing model training unit is used to train the detection model and input the complete data set into the trained brickwork construction quality detection model to obtain the initial detection frame position of the bricks, the initial detection frame position of the mortar joints and the boundary intersection set between the bricks; The data analysis unit performs posture estimation and image angle correction on the complete data set to obtain a corrected data set, and calculates the corrected data set to output a brickwork construction quality assessment result; A display unit displays the brickwork construction quality assessment results.

[0056] The beneficial effects of the present invention are: 1. Unlike the prior art that only relies on a small amount of single data to train the model, the present invention can build a high-performance model under limited data conditions, significantly improving the accuracy and adaptability of brick and mortar joint detection.

[0057] 2. Unlike the existing methods that only perform detection and evaluation through original images, the present invention can effectively eliminate the errors caused by shooting angles and brick defects, greatly improving the evaluation accuracy of horizontality and verticality, and meeting the needs of high-precision construction quality inspection.

[0058] 3. Compared with traditional manual measurement or simple pixel distance measurement methods, the present invention significantly improves the accuracy of mortar joint thickness calculation and reduces human errors through automated evaluation, providing a more scientific and objective evaluation basis for construction quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 The present invention is a flowchart of the multi-viewing angle detection method for brickwork construction quality. DETAILED DESCRIPTION

[0060] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0061] like Figure 1As shown, the present invention provides step 1, obtaining pictures containing brickwork construction scenes to form an original data set, which covers pictures of brickwork construction scenes with different resolutions, lighting conditions and shooting angles; using data enhancement technology to process the original data set to form an enhanced data set, and annotating the enhanced data set to form a complete data set containing category and location information; inputting the complete data set into a detection model for pre-training and fine-tuning, and establishing a brickwork construction quality detection model.

[0062] In step 1, the original data set is obtained from the Internet through crawler technology or photographed by digital devices. Specifically, the Scrapy crawler is used to crawl brickwork pictures with multiple resolutions, multiple scenes, and multiple perspectives from the Internet. These data come from brick wall photos under different angles and lighting conditions, enriching the diversity of the data set. At the same time, brickwork picture data can also be collected by digital devices, and the brickwork picture data collected on the Internet is combined with the self-collected brickwork picture data to form an original data set, ensuring that the original data set covers more actual construction scenes. This original data set not only provides a basis for the establishment and fine-tuning of the subsequent brickwork construction quality detection model, but also provides diversified original materials for multi-perspective enhancement operations. Finally, a high-precision brickwork construction quality detection model is established to improve the precision and accuracy of target detection.

[0063] Data enhancement techniques include image rotation, scaling, cropping, and lighting and color adjustment; the enhanced data set is recorded as :

[0064] in, is the original dataset, Represents rotation, scaling, and shearing operations respectively.

[0065] The purpose of setting the enhanced dataset is to generate diverse samples in space and color domain through data enhancement technology, expand the scale of the dataset, improve the adaptability of the subsequently established brickmasonry construction quality inspection model to diverse scenarios, and improve the robustness of the brickmasonry construction quality inspection model.

[0066] Specifically, the process of rotation transformation is as follows: to simulate the influence of shooting angle deviation on the image, the angle transformation operation is randomly performed on the image, as described below: , in, is a random rotation angle (for example, randomly generated in the range of [-30°, 30°]). This operation can simulate the position changes of bricks when photographed at different angles and train the detection model to adapt to multi-angle detection tasks.

[0067] The process of scaling and cropping is to randomly scale and crop the image to enhance the detection model's adaptability to different resolutions and field of view. The scaling ratio is a random value. , cropping is done by defining the center point ( x c , y c ) and the crop box size ( w , h ) is generated dynamically to ensure that the target brick remains in the cropped image.

[0068] The process of light and color adjustment is: simulate the change of light conditions, and randomly change the brightness b , contrast c and saturation s: ,in I is the original image, This method enhances the robustness of the model to different lighting environments.

[0069] The detection model is the YOLOv8 model; the principle of establishing the brickwork construction quality detection model is as follows: the system uses the YOLOv8 model to detect bricks and mortar joints, combining pre-training and fine-tuning techniques. First, pre-training is performed on a large public dataset (i.e., the original dataset) to obtain general feature representations; then fine-tuning is performed in combination with the enhanced dataset to optimize the model's adaptability to specific tasks, and finally a brickwork construction quality detection model is established. In order to further improve the detection accuracy of the brickwork construction quality detection model, a specific post-processing strategy is introduced, including non-maximum suppression (NMS), which is used to remove duplicate detection results, and finally output a set of bounding boxes for bricks and mortar joints.

[0070] Specifically, the method for forming a complete data set containing category and location information is as follows: use the LabelImg tool to annotate the original data set and the enhanced data set, generate the bounding boxes of the brick and mortar joint areas, form a complete data set, parse the complete data set, and save it in the YOLO annotation format. The complete data set is recorded as ( x , y , w , h ),in, x , y Respectively represent the horizontal and vertical coordinates of the center point of the pixel calibration point in the image; w , h Respectively represent the width and length of the crop box size.

[0071] For each image in the enhanced data set, the same annotation strategy is used to generate corresponding labels to ensure the consistency of the enhanced data quality. Through the above multi-view data processing and data enhancement technology, not only the data expansion and diversity are achieved, but also the overfitting problem of the model to limited data is effectively alleviated during the training process, providing a solid data foundation for subsequent brick detection, posture correction and construction quality assessment.

[0072] Furthermore, in step 1, the brickwork construction quality detection model includes a CNN layer for extracting features, a linear layer for completing category mapping, and a Softmax layer for outputting probability distribution of bricks and mortar joints; after the brickwork construction quality detection model is established, the non-maximum suppression (NMS) algorithm is used to remove redundant detection frames, and only the target area with the highest confidence is retained: , whose goal is to optimize the confidence of boundary classification and make the detection results more reliable.

[0073] Through multi-view data processing and data enhancement technology, not only the expansion and diversity of data are achieved, but also the overfitting problem of the detection model on limited data is effectively alleviated during the training process, providing a solid data foundation for subsequent brick detection, posture correction and construction quality assessment.

[0074] Step 2: Use digital equipment to shoot the actual brick wall construction scene to form a high-resolution, multi-perspective actual brick wall image dataset.

[0075] Step 3: Input the actual brick wall image dataset into the brickwork construction quality inspection model for target detection to obtain the initial detection frame position of the bricks, the initial detection frame position of the mortar joints, and the boundary intersection set between the bricks.

[0076] In step 3, the initial detection frame positions of bricks and mortar joints are recorded as { b i}; Through the vertex coordinates of the initial detection box of bricks and mortar joints, the boundary intersection set between bricks is extracted:

[0077] in, P k is the two-bit pixel coordinate of each point, ( x k , y k ) are the horizontal and vertical coordinates of the two-dimensional pixel respectively; N is the total number of intersections detected.

[0078] Step 4: Correct the boundary intersection point set between bricks using the three-dimensional point set of the standard brick layout to obtain a corrected data set.

[0079] In step 4, the method for obtaining the corrected data set is: Using the boundary intersection set between bricks P and the 3D point set of the standard brick layout Q To establish a projection relationship:

[0080] in, R is the rotation matrix, T is the translation vector, is the projection point set in the image space; solved by the EPnP algorithm R and T , perform inverse transformation on the input image in the complete dataset, correct the shooting angle deviation, and obtain the corrected dataset Specifically, the boundary intersection set between bricks is corrected by combining the posture estimation algorithm and image angle correction. The posture estimation algorithm and image angle correction can effectively reduce the interference of shooting angle deviation on the horizontality and verticality evaluation of bricks, and improve the detection accuracy and robustness.

[0081] Specifically, the process of the pose estimation algorithm is: by inputting the pixel coordinates of the calibrated points in the image, the rotation and translation parameters of the brickwork plane in the three-dimensional space are solved. In this scheme, the EPnP (EfficientPerspective-n-Point) algorithm is selected for pose estimation, and its main purpose is to derive three-dimensional pose information from two-dimensional image points. Assume that there is a n calibration points, the two-dimensional pixel coordinates of each point are , the corresponding three-dimensional world coordinates are The camera intrinsic parameter matrix is K , the external parameter matrix is ,in R is the rotation matrix, T is the translation vector. The EPnP algorithm solves the external parameters by minimizing the projection error: , , in is the actual projection point coordinate. In data preprocessing, the brick boundary intersection points or feature points are accurately extracted to ensure that the calibration points input to the EPnP algorithm have high-quality correspondence. The selection strategy of these points optimizes the uniformity of the calibration point distribution and reduces the error accumulation caused by boundary defects. The output of the EPnP algorithm includes the rotation matrix R and translation vector TFurthermore, the solution can be optimized by combining the geometric constraints in the shooting scene. For example, if the brickwork is parallel to the ground, the ground plane normal vector can be constrained to (0, 0, 1), thereby optimizing R accuracy.

[0082] The process of image angle correction is as follows: after completing the pose estimation, the goal of image angle correction is to align the horizontal edge of the brickwork with the horizontal axis of the image and the vertical edge with the vertical axis of the image, thereby providing standardized input for subsequent horizontal and vertical evaluation. The implementation of affine transformation is based on the known external parameters , design an affine transformation to adjust the image to the standard posture. Let the pixel coordinates of any point in the original image be ( x , y ), the coordinates of the corrected point are ( x′ , y′ ), the transformation formula is:

[0083] in H is the affine transformation matrix, which is obtained by solving the posture estimation result. Specifically, the generation process of the affine matrix includes the following steps: 1) Extract the characteristic points of the key boundaries of the brickwork and determine the horizontal and vertical reference lines; 2) According to R Decomposition result, calculate the rotation angle of the baseline in the image θ ; 3) Generate rotation correction matrix: .

[0084] In addition, to further improve the correction accuracy, this scheme designs an auxiliary correction strategy based on the intersection of brick boundaries. Detect all brick boundaries in the image and extract the intersection set of boundary lines. . Assume that the intersection point set after standard correction is , use the least squares method to fit the transformation matrix: .

[0085] In summary, in step 4, the EPnP algorithm is combined with geometric constraints to improve the rotation matrix through multi-stage optimization. R The solution accuracy is improved. Especially in actual construction scenarios, the geometric relationship between brickwork and the ground provides strong constraints, which can effectively eliminate the interference of noise points. The overall process design combining posture estimation and image correction not only improves the accuracy of brick horizontality and verticality assessment, but also further enhances the robustness of the algorithm through innovative defect recognition and geometric optimization strategies, providing theoretical support and technical guarantee for multi-view construction quality inspection.

[0086] Furthermore, the deviation angle of the brick boundary line in the correction data set is calculated :

[0087] Ensure the deviation angle Within the allowable error range, x 1. x 2. y 1. y 2 are the pixel coordinates of the brick boundary lines, x 1 ’ , x 2 ’ , y 1 ’ , y 2 ’ They are the pixel coordinates of the brick boundary lines after correction.

[0088] Step 5: After performing boundary line fitting, defect detection, horizontality and verticality evaluation, and mortar joint thickness measurement on the images in the corrected data set, it is evaluated whether the brickwork construction quality meets the standards.

[0089] In step 5, the boundary line fitting includes: using the boundary points of the bricks in the correction data set, fitting the straight line equation by the least squares method:

[0090] in, m , c To fit the parameters, minimize the error function:

[0091] in,( x i , y i ) is the point set of the brick boundary; m , c is the fitting parameter; Defect detection includes: Analysis of the Euclidean distance of neighboring boundary points and angle changes :

[0092] in,{ p i} is a discrete point set, p i+1 - p i is the distance between each point and its neighboring points; if >threshod1 or >threshod2, determine it as a defect point and complete the boundary through geometric extrapolation; The horizontality and verticality assessment includes: comparing the fitted boundary line with the standard line and calculating the horizontality and verticality errors:

[0093] in, y i is the fitted horizontal boundary line, y std is the standard horizontal boundary line, is the brick level error, x i is the fitted vertical boundary line, x std is the standard vertical boundary line, is the verticality error of the brick, N is the number of discrete points in the brick area; The measurement of mortar joint thickness includes: calculating the minimum distance between adjacent boundary lines based on the brick boundary line fitting results:

[0094]

[0095] in, is the thickness of the mortar joint, , is the fitting result of the brick adjacent boundary line; the mortar joint thickness result is compared with the construction standard to evaluate whether the brick masonry construction quality meets the standard.

[0096] Specifically, when evaluating the horizontality and verticality of bricks, the least squares method is used to fit the boundary line equation, thereby reducing the impact of noise interference on the boundary line calculation. Assume that the point set of the brick boundary is , the horizontal boundary can be fitted to the equation of a straight line: , calculated by minimizing the sum of squared errors m and b :

[0097] Similarly, the vertical boundaries are fitted as: , combined with the included angle between the fitted boundary line and the standard reference line, calculate the horizontal and vertical offset angles: .

[0098] By comparing the offset angles with construction quality standards, a quantitative assessment of the horizontality and verticality of the bricks is completed.

[0099] In view of the possible damage or missing corners of the brick boundary, the system has designed a defect detection method based on the distance and angle changes between adjacent points. For the fitted boundary line L, its discrete point set is extracted , calculate the distance between each point and its neighboring points :

[0100] And define the angle change as:

[0101] in, .like and If the value exceeds the set threshold, it is determined to be a defect point. For the detected defect area, the geometric extrapolation method is used to complete the boundary line. Assume that the endpoints of the defect area are P s and P e , the extrapolation line equation can be obtained by fitting the slope of the straight line in the neighborhood of the endpoint: .

[0102] The extrapolated line is seamlessly connected with the complete boundary line to ensure the continuity and reliability of the assessment results.

[0103] In addition, the bricks and mortar joints are marked as two types of targets through the brickwork construction quality inspection model, and the bounding box set of the mortar joints is obtained:

[0104] Each bounding box B i Indicates the area of ​​the mortar joint. The mortar joint thickness T is defined as the width of the mortar joint area between bricks. Usually, the mortar joint thickness can be calculated by the distance between the mortar joint boundary and the nearest point of the adjacent brick boundary. For a mortar joint area B i The thickness calculation can be simplified to the shortest distance between the mortar joint boundary line and the adjacent brick boundary line. The specific steps are as follows: 1) Assume the upper boundary line of the mortar joint area is , the lower boundary line is .They can be expressed by linear equations respectively:

[0105] 2) The lower boundary line of the upper brick is , the upper boundary line of the lower brick is , which can also be expressed as:

[0106] 3) Thickness of mortar joints T i The calculation is:

[0107] in, , The corresponding positions are x The mortar joints and brick borders y Coordinate value.

[0108] In actual scenarios, there are two types of mortar joints: horizontal mortar joints and vertical mortar joints. The calculation methods for the two are slightly different: Horizontal mortar joints: along y Directional measure, average distance between upper and lower boundaries:

[0109] Vertical mortar joints: along x Directional measure, average distance between left and right borders:

[0110] in, M is the number of discrete points in the gray seam area.

[0111] First, the present invention designs a defect detection algorithm based on the distance and angle changes of neighboring points, and completes the boundary completion of the defect area in combination with the geometric extrapolation strategy. This method improves the accuracy of horizontal and vertical calculations while reducing the interference of defects on the evaluation results. Second, a post-processing module specifically for bricks and mortar joints is added to the traditional detection framework to improve the confidence of boundary recognition, and the robustness of the detection results is further enhanced by combining non-maximum suppression. Through least squares fitting and angle offset calculation, the standardization of the evaluation process is achieved, while adapting to the diverse quality assessment needs in different construction scenarios. Third, the introduction of a comprehensive measurement strategy based on boundary fitting geometric methods and discrete point calculations not only improves the accuracy of mortar joint thickness calculations, but also ensures the robustness of the method under a variety of resolutions and shooting conditions.

[0112] The present invention also provides an application system of a multi-viewing angle detection method for brickwork construction quality, comprising: A data input unit, used to input the original data set; A data processing unit, used to perform data enhancement processing and labeling processing on the original data set and the enhanced data set respectively; The computing model training unit is used to train the detection model and input the complete data set into the trained brickwork construction quality detection model to obtain the initial detection frame position of the bricks, the initial detection frame position of the mortar joints and the boundary intersection set between the bricks; The data analysis unit performs posture estimation and image angle correction on the complete data set to obtain a corrected data set, and calculates the corrected data set to output a brickwork construction quality assessment result; A display unit displays the brickwork construction quality assessment results.

[0113] In summary, a multi-view detection method and application system for the quality of brick masonry construction in the present invention obtains brick masonry images of different resolutions by combining crawler technologies such as Scrapy, and uses transformation methods such as rotation and scaling to achieve multi-view data enhancement, providing diversified training data for model training. Finally, the fine-tuning of the detection model is achieved, a brick masonry construction quality detection model is established, and the precision and accuracy of target detection are improved. The combined application of posture estimation algorithm and image angle correction: by introducing the EPnP posture estimation algorithm to correct the image angle, and combining the brick boundary intersection information for auxiliary correction, the subsequent target detection is more accurate, and more accurate input data is provided for horizontal verticality evaluation and mortar joint thickness calculation. By introducing the defect detection function, it is judged whether there is a brick defect based on the example and angle change between adjacent points, and the boundary line is completed in combination with the geometric extrapolation method to enhance the accuracy of horizontal verticality evaluation. At the same time, the mortar joint thickness is calculated according to the parallel boundary lines to ensure that the detection results meet the construction standards, which can effectively improve the automation level and accuracy of construction quality management and monitoring.

Claims

1. A multi-view detection method for brickwork construction quality, characterized in that: include: Step 1: Acquire pictures containing brickwork construction scenes to form an original data set, where the original data set covers pictures of brickwork construction scenes with different resolutions, lighting conditions, and shooting angles; The original data set is processed using data enhancement technology to form an enhanced data set, and the enhanced data set is annotated to form a complete data set containing category and location information; The complete data set is input into the detection model for pre-training and fine-tuning to establish a brickwork construction quality detection model; Step 2: Use digital equipment to shoot the actual brick wall construction scene to form a high-resolution, multi-view actual brick wall image dataset; Step 3: Input the actual brick wall image dataset into the brickwork construction quality detection model for target detection, and obtain the initial detection frame position of the bricks, the initial detection frame position of the mortar joints, and the boundary intersection set between the bricks; Step 4, correcting the boundary intersection point set between bricks through the three-dimensional point set of the standard brick layout to obtain a corrected data set; Step 5: After performing boundary line fitting, defect detection, horizontality and verticality evaluation, and mortar joint thickness measurement on the images in the corrected data set, it is evaluated whether the brickwork construction quality meets the standards.

2. The multi-view detection method for brickwork construction quality according to claim 1 is characterized in that: In step 1, the original data set is obtained from the Internet through crawler technology or photographed by digital devices.

3. The multi-viewing method for detecting the construction quality of brickwork according to claim 2, characterized in that: In step 1, the data enhancement technology includes rotation transformation, scaling and cropping, and lighting and color adjustment of the image; the enhanced data set is recorded as : in, is the original dataset, Represents rotation, scaling, and shearing operations respectively.

4. The multi-viewing method for detecting the construction quality of brickwork according to claim 3 is characterized in that: In step 1, the detection model is a YOLOv8 model; the method for forming a complete data set containing category and location information is: using the LabelImg tool to annotate the original data set and the enhanced data set, generating the bounding boxes of the brick and mortar joint areas, forming a complete data set, parsing the complete data set, and saving it in the YOLO annotation format. The complete data set is recorded as ( x , y , w , h ),in, x , y Respectively represent the horizontal and vertical coordinates of the center point of the pixel calibration point in the image; w , h Represents the width and length of the crop box size respectively.

5. The multi-viewing method for detecting the construction quality of brickwork according to claim 4, characterized in that: In step 1, the brickwork construction quality detection model includes a CNN layer for extracting features, a linear layer for completing category mapping, and a Softmax layer for outputting probability distribution of bricks and mortar joints. After the brickwork construction quality detection model is established, the non-maximum suppression (NMS) algorithm is used to remove redundant detection frames and only retain the target area with the highest confidence: .

6. The multi-viewing method for detecting the construction quality of brickwork according to claim 4, characterized in that: In step 3, the initial detection frame position of the brick and the initial detection frame position of the mortar joint are recorded as {b i }; Through the vertex coordinates of the initial detection box of bricks and mortar joints, the boundary intersection set between bricks is extracted: in, P k is the two-bit pixel coordinate of each point, ( x k , y k ) are the horizontal and vertical coordinates of the two-dimensional pixel respectively; N is the total number of detected intersections.

7. The multi-viewing method for detecting the construction quality of brickwork according to claim 6, characterized in that: In step 4, the method for obtaining the corrected data set is: Using the boundary intersection set between bricks P and the 3D point set of the standard brick layout Q To establish a projection relationship: in, R is the rotation matrix, T is the translation vector, is the projection point set in the image space; solved by the EPnP algorithm R and T , perform inverse transformation on the input image in the complete dataset, correct the shooting angle deviation, and obtain the corrected dataset .

8. The multi-viewing method for detecting the construction quality of brickwork according to claim 6, characterized in that: Calculate the deviation angle of the brick boundary line in the corrected dataset : ； Ensure the deviation angle Within the allowable error range, x 1. x 2. y 1. y 2 are the pixel coordinates of the brick boundary lines, x 1 ’ , x 2 ’ , y 1 ’ , y 2 ’ They are the pixel coordinates of the brick boundary lines after correction.

9. The multi-viewing method for detecting the construction quality of brickwork according to claim 7, characterized in that: In step 5, the boundary line fitting includes: using the boundary points {pi} of the bricks in the correction data set, fitting the straight line equation by the least squares method: in, m , c To fit the parameters, minimize the error function: in,( x i , y i ) is the point set of the brick boundary; m , c is the fitting parameter; Defect detection includes: Analysis of the Euclidean distance of neighboring boundary points and angle changes : in,{ p i } is a discrete point set, p i+1 - p i is the distance between each point and its neighboring points; if >threshod1 or >threshod2, determine it as a defect point and complete the boundary through geometric extrapolation; The horizontality and verticality assessment includes: comparing the fitted boundary line with the standard line and calculating the horizontality and verticality errors: in, y i is the fitted horizontal boundary line, y std is the standard horizontal boundary line, is the brick level error, x i is the fitted vertical boundary line, x std is the standard vertical boundary line, is the verticality error of the brick, N is the number of discrete points in the brick area; The measurement of mortar joint thickness includes: based on the brick boundary line fitting results, calculating the minimum distance between adjacent boundary lines: in, is the thickness of the mortar joint, , is the fitting result of the brick adjacent boundary line; the mortar joint thickness result is compared with the construction standard to evaluate whether the brick masonry construction quality meets the standard.

10. An application system based on the multi-viewing angle detection method for brickwork construction quality according to any one of claims 1 to 9, characterized in that: include: A data input unit, used to input the original data set; A data processing unit, used to perform data enhancement processing and labeling processing on the original data set and the enhanced data set respectively; The computing model training unit is used to train the detection model and input the complete data set into the trained brickwork construction quality detection model to obtain the initial detection frame position of the bricks, the initial detection frame position of the mortar joints and the boundary intersection set between the bricks; The data analysis unit performs posture estimation and image angle correction on the complete data set to obtain a corrected data set, and calculates the corrected data set to output a brickwork construction quality assessment result; A display unit displays the brickwork construction quality assessment results.

Images