Construction method of high-precision visual module of intelligent building polishing robot and visual module
By designing specialized target points and progressive training strategies, combined with the YOLO architecture and post-processing logic, the robustness problem of the vision module of the intelligent grinding robot in complex environments was solved, achieving high-precision construction quality and safety, and reducing reliance on manual labor.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANGHAI RESEARCH INSTITUTE OF BUILDING SCIENCES CO LTD
- Filing Date
- 2025-12-29
- Publication Date
- 2026-04-21
AI Technical Summary
In the construction industry, the vision module of intelligent grinding robots is not adaptable enough to complex environments, and the robustness of the algorithm is limited, resulting in a high misjudgment rate and an inability to form an effective construction closed loop.
Specialized target points were designed, a progressive training strategy and a five-stage dataset optimization were adopted, and the target detection model and post-processing logic of the YOLO architecture were combined to enhance the model's anti-interference ability and recognition accuracy. High-precision polishing area recognition was achieved through vision and LiDAR collaborative localization.
It significantly reduced the false negative rate, improved the robustness of the model and the quality of construction, and achieved automation and safety in the construction process, while reducing reliance on manual labor.
Smart Images
Figure CN121904501A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent building robot technology, and in particular to a method for constructing a high-precision vision module for an intelligent building grinding robot and the vision module itself. Background Technology
[0002] Due to low levels of informatization and intelligentization, extensive management, and over-reliance on manual labor, the construction industry has long suffered from problems such as low quality and long construction periods, which has further led to frequent safety accidents.
[0003] In recent years, as the construction industry has gradually entered a stock-based era, urban renewal projects have been accounting for an increasing proportion of the industry. Reinforcement, renovation, and decoration are crucial aspects of urban renewal. Before reinforcing existing concrete buildings with carbon fiber cloth, steel plate bonding, or surface decoration, it is necessary to remove impurities such as laitance and oil stains from the cement concrete surface, and to grind and polish the cement concrete base surface of the components. Traditional building grinding operations mainly rely on manual labor, exposing workers to high dust and noise environments for extended periods, which can easily lead to occupational lung diseases and the risk of falls from heights. Although the industry has introduced intelligent grinding robots, their vision modules still have significant deficiencies: 1. Insufficient environmental adaptability: Existing target points are mostly single-color black and white target points, and the false detection rate increases sharply when dust obscures the target or the lighting changes. Existing robots rely on 3D vision or robotic arm control, but the problem of robust target recognition in complex backgrounds has not been solved.
[0004] 2. Limitations in Algorithm Robustness: Traditional target detection models lack negative samples and high-intensity interference simulations in their training data, leading to increased misclassification rates in actual construction. When target points are partially occluded or insufficient in number, the model cannot accurately classify the effective area, restricting the robot's autonomous decision-making capabilities.
[0005] Therefore, the real-time performance, anti-interference ability, and accuracy of vision modules remain bottlenecks in the industry. In dynamic construction scenarios, sudden changes in lighting and dust interference reduce the signal-to-noise ratio of images. Traditional target detection models have not optimized their training processes for this purpose and cannot form a good closed loop of "recognition-execution-feedback". This problem seriously restricts the improvement of construction quality. Summary of the Invention
[0006] To address the aforementioned deficiencies and shortcomings of existing technologies, this invention aims to provide a method for constructing a high-precision vision module for an intelligent building grinding robot and a vision module thereof. This method integrates improved target detection and post-processing logic, and can construct a vision module suitable for the application scenario of reinforcement, renovation and decoration construction of existing buildings.
[0007] The objective of this invention can be achieved through the following technical solutions: A method for constructing a high-precision vision module for an intelligent building polishing robot includes the following steps: Step S1: Prepare specialized target points: Design and fabricate specific recognition targets as target points, which are used to set at the four corners of the polishing area; Step S2, Progressive Training Strategy: Construct an object detection model and train the model using a five-stage dataset optimization strategy. The five stages include basic positive sample training, noise perturbation introduction, negative sample and complex pattern enhancement, high-intensity interference addition, and full data integration and generalization. Step S3, Post-processing Dual Verification: Based on the trained target detection model, inference is performed on the test image to identify specialized target points, a confidence threshold is set to filter the detection results, and the number of similar target points is counted. Only when the number of valid target points meets the preset number and the confidence is greater than the set threshold, the current image is determined to be a valid image, and the grinding area is determined according to the position of the target points. Step S4, Grinding quality evaluation: A closed-loop evaluation of the grinding area is performed based on image analysis technology.
[0008] A further improvement of the present invention is that, in step S1, the identification target is a circular target point with a white "+" shaped pattern.
[0009] A further improvement of the present invention is that, in step S1, the thickness difference between the white "+" shaped pattern of the identified target and the substrate of the circular target point is greater than 1 mm, so that the difference in flatness between the two is easily recognized by the lidar mounted on the robot.
[0010] A further improvement of the present invention is that, in step S2, the five-stage dataset optimization strategy specifically includes: Phase 1: Construct a basic positive sample dataset and perform data augmentation using rotation, flipping, and brightness adjustment techniques; Phase Two: Introduce noise disturbances and early stopping mechanisms during training, and set the learning rate to 0.005; Phase 3: Add negative samples and complex labeling style data. The negative samples include images with fewer than 4 target points to enhance the model's adaptability to occlusion or target loss. Phase 4: Add high-intensity interference data, including cropping the image, processing exposure offset, and adding dust noise simulation to improve the model's anti-interference ability; Phase 5: Integrate all the datasets from the first four phases and perform final training on the model to enhance its generalization ability.
[0011] A further improvement of the present invention is that, in step S3, the target detection model adopts the YOLO architecture, and the specific steps of the post-processing dual verification include: Load the optimal model weights obtained from training the object detection model; The test set images are read sequentially, input into the model for inference, and the detection results are extracted. The detection results include the target point location, category, and confidence level. Set a confidence threshold and filter out reliable targets with a confidence level greater than the threshold; Determine if the number of valid target points identified in the image is 4: if yes, it is considered a valid image and saved to the valid image folder; otherwise, it is considered an invalid image and saved to the invalid image folder.
[0012] A further improvement of the present invention is that, in step S4, the specific criteria for evaluating the polishing quality include: SSIM consistency comparison: Calculate the structural similarity of images before and after polishing, and mark the areas with differences as substandard; Noise detection: After median filtering to remove noise from the image, the grayscale value is analyzed. If the grayscale value fluctuates within ±15%, the area is determined to be unevenly polished. Edge integrity: Canny edge detection is used to identify contours. If the detected contour break length is greater than 10mm, it is located as over-polishing.
[0013] A further improvement of the present invention is that the method further includes a coordinated localization step using vision and radar: During the polishing process, if the confidence level of the target point recognized by visual image recognition is greater than 0.75, the target point is located directly using visual image recognition. If the confidence level of the target points identified within the visible range is less than 0.75, then the lidar is activated for assisted identification and positioning, and cross-validation is performed with the image recognition results.
[0014] The present invention also provides a vision module, which is constructed using the method described above.
[0015] A further improvement of the present invention is that the vision module specifically includes: High-resolution cameras and lidar are used to capture images of the polishing area and to assist in the identification and positioning of target points, respectively. Edge devices are equipped with target detection models. The image post-processing module communicates with the edge device, performs dual verification, and associates with the robot control unit to generate a polishing path.
[0016] Compared with existing technologies, (1) Breakthrough in environmental adaptability: The design of circular target points with white "+" tape enhances texture features and three-dimensional recognition (flatness difference > 1mm), and the missed detection rate can be significantly reduced under dust cover.
[0017] (2) Improved robustness of the algorithm: Five-stage training (noise perturbation / negative samples / high-intensity interference) effectively improves the model's mAP50-95, recall rate and other indicators. In addition, it can effectively solve the problem of misjudgment caused by occlusion and missing markers.
[0018] (3) Construction closed-loop optimization: The post-processing mechanism based on confidence threshold and the number of marker points can greatly reduce the possibility of invalid grinding; combined with SSIM consistency comparison and Canny edge detection quality evaluation system, the "identification-execution-feedback" full process automation is realized. Attached Figure Description
[0019] Figure 1 This is a flowchart illustrating the construction process of the present invention (visual module); Figure 2 This refers to the image recognition workflow; Figure 3 These are the original positive sample images; Figure 4 It is a data-enhanced user interface; Figure 5 These are data-augmented samples; Figure 6 This is the post-processing flowchart; Figure 7 This is a graph showing the analysis of the dataset training results; Detailed Implementation
[0020] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.
[0021] like Figure 1 The embodiments of the present invention provide a method for constructing a vision module for an intelligent building grinding robot according to the attached diagram. Figure 1 The process is designed to achieve the triple benefits of improved construction quality, reduced reliance on manual labor, and elimination of safety hazards, providing core technological support for the intelligent transformation of the construction industry.
[0022] 1. In the intelligent sanding robot system, the vision module is the core for autonomously identifying sanding areas and assisting in decision-making and control. It primarily provides the basis for sanding path planning and task execution by automatically detecting and identifying target points in the construction wall image. This module is designed based on deep learning, integrating target detection algorithms and image post-processing strategies to achieve accurate target point extraction and region validity judgment. It consists of three parts: image acquisition equipment, target detection model, and image post-processing module. (Appendix) Figure 1 Appendix Figure 2 ) (1) Image acquisition equipment The robot's onboard camera captures real-time images of the wall. Considering the interference from high dust levels, obstructions, and changing lighting at the polishing site, a high-resolution camera with strong anti-interference capabilities is required. In this embodiment, the robot is also equipped with a lidar for assisting in the identification and positioning of target points.
[0023] (2) Target detection model The system utilizes the YOLOv8 target detection algorithm to detect target points in images. This algorithm boasts a streamlined architecture, fast execution speed, and stable deployment even on edge devices with limited computing resources. It accurately outputs the location, category, and confidence score data of each target point, laying a solid data foundation for subsequent image analysis and judgment. The target detection model typically identifies target points located at the four corners of a rectangular polished area.
[0024] (3) Image post-processing module The post-processing module determines the validity of an image by analyzing the target detection results. If the number of similar target points in an image is ≥4 and the confidence level of each point is higher than a set threshold, the image is considered valid; otherwise, it is considered invalid.
[0025] 2. To ensure the training effect of the visual module's object detection algorithm, high-quality and representative image data are required. High-resolution cameras can be used for image acquisition. During acquisition, it is important to maintain image clarity and stability to prevent blurring and jitter from affecting data quality. LiDAR can also be used as an auxiliary tool if necessary. Positive samples are attached. Figure 3 As shown.
[0026] 3. In scenarios where LiDAR is used, increase the thickness of the tape (white "+" pattern) so that the flatness difference between it and the circular substrate is greater than 1mm, thereby enhancing the recognition stability of the 3D sensing (LiDAR).
[0027] 4. Data annotation is then performed. The quality of the annotation directly affects the model training effect. Annotation strictly follows a unified standard and is manually reviewed to ensure quality. To improve model robustness, various augmentation processes are applied to the original data (see appendix). Figure 4 The following are the methods for adding them: (1) Image rotation: random angle rotation; (2) Brightness adjustment: Simulates different lighting changes; (3) Image distortion: slight deformation; (4) Add noise: simulate a high dust concentration environment.
[0028] Furthermore, similar enhancement methods can be used to increase interference and invalid samples. The enhanced samples are shown in the attached image. Figure 5 As shown. Once the above work is completed, model training can be carried out based on this.
[0029] 5. After completing the training, the changes in various indicators during the verification phase should be compiled (as shown in the attached document). Figure 7 As shown, the metrics include localization error (val / box_loss), classification error (val / cls_loss), bounding box regression accuracy (val / dfl_loss), precision (metrics / precision(B)), recall (metrics / recall(B)), average precision (metrics / mAP50(B) and metrics / mAP50-95(B)), which are used to evaluate the model's performance in object detection tasks from multiple dimensions and can comprehensively evaluate the model's performance under different requirements.
[0030] 6. Since the interface that needs to be reinforced after the building construction is generally rectangular, the valid samples can be assumed to be rectangular target points. Based on this, an image post-processing algorithm can be designed and implemented to classify the validity of the test set images. By counting the number of target points with confidence higher than the threshold in each image, the valid images and invalid images are automatically divided according to the judgment criteria of "four valid target points".
[0031] A post-processing script can be written in Python to call the model inference interface and the trained weight file to detect test images one by one. The processing flow is as follows (see appendix). Figure 6 ): (1) Load the optimal model weights obtained from YOLOv8 training; (2) Read the test set images sequentially, perform inference, and extract the detection results; (3) Set a confidence threshold (which can be set to 0.5) to filter out reliable targets; (4) If the number of valid target points identified in the image is 4, it is considered a valid image; otherwise, it is invalid. (5) Based on the judgment results, save the images into the valid / invalid image folders respectively; (6) Total time spent on output processing.
[0032] Based on this result, dual verification can be performed quickly to examine and improve training effectiveness.
[0033] The code for this process is as follows: import os import shutil import YOLO from ultralytics import Image from PIL import time model = YOLO(r'C:\Users\Administrator\runs\detect\yolov8s-marker-train012\weights\best.pt') image_folder = "D: / data2 / roboflow012 / test / images" valid_folder = "D: / data2 / roboflow012 / valid_images" invalid_folder = "D: / data2 / roboflow012 / invalid_images" os.makedirs(valid_folder, exist_ok=True) os.makedirs(invalid_folder, exist_ok=True) CONF_THRESH = 0. start_time = time.time() for img_file in os.listdir(image_folder): if not img_file.lower().endswith(('.jpg', '.jpeg', '.png')): continue img_path = os.path.join(image_folder, img_file) results = model(img_path)[0] count = 0 for box in results.boxes: if box.conf > CONF_THRESH: count += 1 if count == 4: shutil.copy(img_path, os.path.join(valid_folder, img_file)) else: shutil.copy(img_path, os.path.join(invalid_folder, img_file)) To continuously improve and iterate the model and enhance visual accuracy, it is necessary to build an automatic evaluation system for polishing quality based on image analysis (see attached). Figure 1 Specifically: (1) Consistency comparison: Calculate the SSIM structural similarity of the images before and after polishing, and mark the difference areas as not meeting the standard; (2) Noise detection: After median filtering and noise reduction, the smoothness of grayscale is analyzed, and abnormal fluctuation areas are identified as uneven polishing. (3) Edge integrity: Canny edge detection identifies contour breaks and locates over-polished areas.
[0034] In actual grinding operations, when visibility is high (the confidence level of identifying the target point is greater than 0.75), the target point is located directly using visual image recognition; when visibility is low (the confidence level of identifying the target point within the visible range is less than 0.75), LiDAR is used for assisted identification and positioning, and the results are cross-validated with the image recognition results.
[0035] After confirming through post-processing algorithms that the current image is a valid image containing four valid marker points, the system automatically calculates and plans the rectangular area to be polished and the robot's execution path based on the image coordinates of these four marker points, and outputs this information to the robot's motion control system. The control system drives the robot to move along the planned path and controls the polishing head to perform automated work within the rectangular area.
[0036] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.
Claims
1. A method for constructing a high-precision vision module for an intelligent building grinding robot, characterized in that, Includes the following steps: Step S1: Prepare specialized target points: Design and fabricate specific recognition targets as target points, which are used to set at the four corners of the polishing area; Step S2, Progressive Training Strategy: Construct an object detection model and train the model using a five-stage dataset optimization strategy. The five stages include basic positive sample training, noise perturbation introduction, negative sample and complex pattern enhancement, high-intensity interference addition, and full data integration and generalization. Step S3, Post-processing Dual Verification: Based on the trained target detection model, inference is performed on the test image to identify specialized target points, a confidence threshold is set to filter the detection results, and the number of similar target points is counted. Only when the number of valid target points meets the preset number and the confidence is greater than the set threshold, the current image is determined to be a valid image, and the grinding area is determined according to the position of the target points. Step S4, Grinding quality evaluation: A closed-loop evaluation of the grinding area is performed based on image analysis technology.
2. The method for constructing a high-precision vision module for an intelligent building grinding robot according to claim 1, characterized in that, In step S1, the identification target is a circular target point with a white "+" shaped pattern.
3. The method for constructing a high-precision vision module for an intelligent building grinding robot according to claim 2, characterized in that, In step S1, the thickness difference between the white "+" shaped pattern of the target and the substrate of the circular target is greater than 1 mm, so that the difference in flatness between the two can be easily identified by the lidar mounted on the robot.
4. The method for constructing a high-precision vision module for an intelligent building grinding robot according to claim 1, characterized in that, In step S2, the five-stage dataset optimization strategy specifically includes: Phase 1: Construct a basic positive sample dataset and perform data augmentation using rotation, flipping, and brightness adjustment techniques; Phase Two: Introduce noise disturbances and early stopping mechanisms during training, and set the learning rate to 0.005; Phase 3: Add negative samples and complex labeling style data. The negative samples include images with fewer than 4 target points to enhance the model's adaptability to occlusion or target loss. Phase 4: Add high-intensity interference data, including cropping the image, processing exposure offset, and adding dust noise simulation to improve the model's anti-interference ability; Phase 5: Integrate all the datasets from the first four phases and perform final training on the model to enhance its generalization ability.
5. The method for constructing a high-precision vision module for an intelligent building grinding robot according to claim 1, characterized in that, In step S3, the target detection model adopts the YOLO architecture, and the specific steps of the post-processing dual verification include: Load the optimal model weights obtained from training the object detection model; The test set images are read sequentially, input into the model for inference, and the detection results are extracted. The detection results include the target point location, category, and confidence level. Set a confidence threshold and filter out reliable targets with a confidence level greater than the threshold; Determine if the number of valid target points identified in the image is 4: if yes, it is considered a valid image and saved to the valid image folder; if no, it is considered an invalid image and saved to the invalid image folder.
6. The method for constructing a high-precision vision module for an intelligent building grinding robot according to claim 1, characterized in that, In step S4, the specific criteria for evaluating the polishing quality include: SSIM consistency comparison: Calculate the structural similarity of images before and after polishing, and mark the areas with differences as substandard; Noise detection: After median filtering to remove noise from the image, the grayscale value is analyzed. If the grayscale value fluctuates within ±15%, the area is determined to be unevenly polished. Edge integrity: Canny edge detection is used to identify contours. If the detected contour break length is greater than 10mm, it is located as over-polishing.
7. A method for constructing a high-precision vision module for an intelligent building grinding robot according to claim 5, characterized in that, This method also includes a visual and radar co-localization step: During the polishing process, if the confidence level of the target point recognized by visual image recognition is greater than 0.75, the target point is located directly using visual image recognition. If the confidence level of the target points identified within the visible range is less than 0.75, then the lidar is activated for assisted identification and positioning, and cross-validation is performed with the image recognition results.
8. A visual module, characterized in that, It is constructed using the method described in any one of claims 1 to 6.
9. The vision module according to claim 8, characterized in that, Specifically, it includes: High-resolution cameras and lidar are used to capture images of the polishing area and to assist in the identification and positioning of target points, respectively. Edge devices, equipped with target detection models; The image post-processing module communicates with the edge device, performs dual verification, and associates with the robot control unit to generate a polishing path.