A BIM-visual fusion-based automatic scratching control method for laminated slab

Abstract

This application relates to the field of prefabricated building manufacturing and automation control technology, and discloses an automatic surface roughening control method for composite slabs based on BIM-visual fusion. The method includes: when the formwork is in place, acquiring multi-view images and inputting them into a deep learning model to extract the edge contours of the slab surface, the center lines of the ribs, and the position features of the embedded parts; simultaneously, acquiring BIM model data from the MES system. Cross-modal registration is performed between the visual features and the BIM data, and rigid body transformation parameters are solved to compensate for pose deviations, obtaining the registered obstacle positions; anisotropic expansion is performed on each obstacle to generate a three-dimensional passable area model; based on this, a layered hybrid path planning method is used to generate a collision-free three-dimensional path covering the preset roughening area, and execution code containing force-position hybrid control instructions is generated and sent to the robot. This application achieves teach-free, zero-collision, high-coverage, and constant-depth adaptive control for composite slab roughening, significantly improving production efficiency and roughening quality.

Description

Technical Field

[0001] This application relates to the field of prefabricated building manufacturing and automation control technology, and in particular to an automatic roughening control method for composite slabs based on BIM-visual fusion. Background Technology

[0002] Composite slabs are one of the main prefabricated components in prefabricated buildings. Their surfaces need to be roughened before the initial setting of concrete to form a rough surface, so as to enhance the bonding performance with the subsequent concrete layer. The national standard "Technical Specification for Prefabricated Concrete Structures" (JGJ 1-2014) stipulates that the depth of the concave and convex surfaces of composite slabs should not be less than 4 mm, and the roughened area should not be less than 80% of the surface. At present, the roughening of composite slab surfaces mainly relies on manual hand-held roughening tools. Manual roughening has the following problems: (1) The roughening depth is inconsistent and is affected by the worker's physical strength, experience and concrete condition, which can easily lead to uneven depths or missed areas; (2) The surface of composite slabs is covered with various obstacles such as ribs, junction boxes, lifting lugs, and embedded screws. Manual operation requires frequent avoidance, which is inefficient and easily damages molds or components; (3) The initial setting time window of concrete is short (usually 3-6 hours), and it is difficult for manual workers to complete the stable roughening of a large number of slabs within a limited time; (4) The labor intensity is high, and the occupational health risks such as dust and vibration are high.

[0003] Some precast component manufacturers have attempted to use gantry robots or teach-type robotic arms for surface roughening. However, existing automation solutions have significant shortcomings: ① They rely on manual teaching and programming, which is time-consuming and error-prone when there are various types of composite slabs and frequent production changes; ② They can only move along fixed paths and cannot adaptively adjust according to the actual posture deviations of the slab surface (mold positioning errors, composite slab placement tilt, etc.), resulting in a high risk of collision between the roughening tool and the reinforcing bars; ③ They lack the ability to compensate for changes in concrete stiffness with initial setting time, making it difficult to maintain a constant roughening depth; ④ They do not fully utilize the integration of BIM (Building Information Modeling) preset data and machine vision, making it impossible to achieve intelligent roughening with no teaching required, zero collisions, and constant depth.

[0004] In the field of industrial robot control and data processing, there are already some technical solutions involving BIM and vision fusion, path planning, or semantic map construction, but none of them have proposed effective solutions to the aforementioned problems in the composite slab roughening scenario. Therefore, there is an urgent need for an intelligent roughening method that can integrate BIM prior data with machine vision, achieve sub-millimeter-level registration, adopt anisotropic expansion obstacle avoidance, and combine force-position hybrid control. Summary of the Invention

[0005] (a) Technical problems to be solved

[0006] In view of the above-mentioned shortcomings and deficiencies of the prior art, this application provides an automatic roughening control method for composite slabs based on BIM-visual fusion, which overcomes the following problems existing in the prior art:

[0007] Low cross-modal data fusion: There is a discrepancy between the theoretical obstacle positions provided by the BIM model and the actual model pose. Existing solutions either rely solely on real-time vision (losing prior information) or simply overlay BIM and vision (insufficient registration accuracy), failing to achieve sub-millimeter level alignment, resulting in inaccurate path planning benchmarks.

[0008] The obstacle avoidance strategy is static and isotropic: Existing obstacle avoidance methods mostly adopt isotropic expansion (i.e., the safe distance is the same in all directions around the obstacle), without considering the characteristic that the roughening tool has been guaranteed to be collision-free along the direction of travel by path planning, resulting in excessive shrinkage of the roughening area, especially in the high-density rib area where the coverage rate is seriously reduced.

[0009] Path planning does not take into account both coverage and motion smoothness: Existing path planning mostly aims at the shortest path or simple bow-shaped scanning, lacking comprehensive optimization of the smoothness of the robot arm joint motion and the penalty for avoiding protruding ribs, resulting in large motion impact and high risk of local collisions during actual execution;

[0010] Force control and path planning are independent of each other: In the existing solution, force position control and path planning are decoupled, and force control requirements (such as constant depth and rib edge transition) cannot be included in unified optimization during the path generation stage, which leads to sudden changes in the roughening depth or collision impact during execution.

[0011] (II) Technical Solution

[0012] To achieve the above objectives, the main technical solutions adopted in this application include:

[0013] This application provides an automatic roughening control method for composite slabs based on BIM-visual fusion, the specific steps of which include:

[0014] S1. When the mold table loading the composite plate to be roughened arrives at the preset station, multiple real-time visual images of the surface of the composite plate to be roughened are acquired from different angles; the multiple real-time visual images are input into a pre-trained deep learning model to obtain the edge contour features and obstacle features of the composite plate to be roughened, the obstacle features including the center line of the rib and the position features of the embedded parts.

[0015] S2. Obtain the BIM model data corresponding to the composite slab to be roughened from the MES system through the communication interface. The BIM model data includes the slab outline dimensions, obstacle coordinates, preset roughening area, preset roughening depth, and preset roughening direction. The obstacle coordinates include the three-dimensional coordinates and height of the center line of the protruding rib and the position of the embedded parts.

[0016] S3. Perform cross-modal registration of the edge contour features and obstacle features with the BIM model data, and solve the rigid body transformation parameters from the BIM model coordinate system to the actual model table coordinate system; use the rigid body transformation parameters to transform the obstacle coordinates in the BIM model data to the actual model table coordinate system to obtain the registered obstacle positions.

[0017] S4. Based on the registered obstacle positions, perform anisotropic expansion on each obstacle to generate a three-dimensional passable area model.

[0018] S5. Based on the preset roughened area, within the three-dimensional passable area model, a layered hybrid path planning algorithm is used to generate a collision-free three-dimensional path covering the preset roughened area.

[0019] Based on the collision-free 3D path and the preset roughening depth, execution code including force-position hybrid control instructions is generated, and the execution code is sent to the robot actuator via an industrial bus.

[0020] Optionally, in some embodiments of this application, the training process of the deep learning model is included before S1:

[0021] S01. Obtain the training dataset; the training dataset includes multiple historical images of the surface of the composite slab, each historical image is pre-annotated with edge contour features and obstacle features; the obstacle features include the center line of the rib and the position features of the embedded parts;

[0022] S02. An improved Mask R-CNN instance segmentation network is used as the initial network; the improved Mask R-CNN instance segmentation network includes, in addition to the mask branch and bounding box branch of the standard Mask R-CNN, an additional rib centerline regression branch is added; the rib centerline regression branch outputs the coordinate sequence of the rib centerline;

[0023] S03. The initial network is trained using the training dataset. The loss function during training includes classification loss, bounding box regression loss, mask segmentation loss, and rib centerline continuity constraint loss.

[0024] The continuity constraint loss of the rib centerline is calculated by summing the squared Euclidean distances between adjacent key points on the rib centerline predicted by the Mask R-CNN instance segmentation network, and is used to penalize the breakage of the rib centerline.

[0025] When the loss function converges, training ends, and the pre-trained deep learning model is obtained.

[0026] Optionally, in some embodiments of this application, the cross-modal registration of the edge contour features and obstacle features with the BIM model data in step S3 includes:

[0027] Establish the projection relationship between the three-dimensional geometric elements in the BIM model data and the camera imaging plane;

[0028] The iterative nearest point algorithm is used to project the three-dimensional coordinates of the rib centerline in the BIM model data onto the camera image plane to obtain the projection line; the first distance error between the projection line and the rib centerline output by the pre-trained deep learning model is calculated.

[0029] The positional features of the rib centerline output by the pre-trained deep learning model are back-projected onto the BIM model plane to obtain the back-projection line; the second distance error between the back-projection line and the rib centerline in the BIM model data is calculated.

[0030] With the goal of minimizing the sum of the first distance error and the second distance error, the rigid body transformation parameters from the BIM model coordinate system to the actual model platform coordinate system are solved.

[0031] Optionally, in some embodiments of this application, the cross-modal registration further includes:

[0032] Extract the preset roughening direction from the BIM model data, and extract the surface texture direction from the multiple real-time visual images;

[0033] The angular deviation between the preset roughening direction and the surface texture direction is taken as the third distance error and added to the optimization objective of the sum of the first distance error and the second distance error.

[0034] With the goal of minimizing the weighted sum of the first distance error, the second distance error, and the third distance error, the rigid body transformation parameters from the BIM model coordinate system to the actual model table coordinate system are solved to achieve dual-constraint registration of pose and texture direction.

[0035] Optionally, in some embodiments of this application, the anisotropic expansion of each obstacle in step S4 includes:

[0036] The expansion distance perpendicular to the direction of travel of the texturing tool is greater than or equal to the sum of the radius of the texturing tool and the preset safety margin, and the expansion distance along the direction of travel is set to the preset positioning error compensation value.

[0037] Optionally, in some embodiments of this application, the step S5, which involves generating a collision-free 3D path covering the preset roughening area using a hierarchical hybrid path planning algorithm, includes:

[0038] Based on the preset roughened area, within the three-dimensional passable area model, an initial global path is generated by the global planning layer, and then the initial global path is corrected by the local planning layer to obtain a collision-free three-dimensional path covering the preset roughened area.

[0039] The global planning layer uses RRT. The Connect algorithm, whose cost function includes a path length term, a robotic arm joint motion smoothness term, and an avoidance penalty term inversely proportional to the distance from the centerline of the rib, is used to search for an initial global path from the starting point to the ending point.

[0040] The local planning layer uses a laser displacement sensor installed at the end of the robotic arm to detect changes in the height of the front plate in real time, and combines the dynamic window method to perform local trajectory correction on the initial global path, outputting a collision-free 3D path.

[0041] Optionally, in some embodiments of this application, generating the execution code including the force-position hybrid control instruction in step S5 includes:

[0042] The target contact force is determined based on the preset roughening depth;

[0043] During the process of the roughening tool traveling along the collision-free three-dimensional path, the contact force between the roughening tool and the surface of the composite plate is collected in real time, and the vertical position of the roughening tool is dynamically adjusted according to the deviation between the contact force and the target contact force to keep the roughening depth constant at the preset roughening depth.

[0044] Optionally, in some embodiments of this application, the force-position hybrid control command employs an impedance control strategy and includes feedforward correction of the target contact force:

[0045] The concrete pouring time of the composite slab to be roughened and the ambient temperature and humidity data of the current production workshop are obtained from the MES system through the communication interface; based on the pouring time and ambient temperature and humidity data, the pre-established concrete stiffness prediction mapping table is queried to obtain the estimated stiffness value of the current concrete; before the roughening operation begins, the initial target contact force is weighted and corrected according to the estimated stiffness value to obtain the corrected target contact force.

[0046] Based on the preset roughening depth and the actual height of the plate surface detected by the laser displacement sensor installed at the end of the robotic arm, the theoretical downward pressure of the end tool is calculated in a feedforward manner; when the contact force collected in real time deviates from the corrected target contact force, the vertical position of the end of the robotic arm is dynamically adjusted to form a position-force closed-loop control.

[0047] Optionally, in some embodiments of this application, a model reuse and lightweight registration step is included before S2:

[0048] Determine whether the model of the current composite board to be roughened is the same as the model of the composite board processed last time;

[0049] If they are the same, skip the operation of obtaining BIM model data from the MES system in S2, directly use the BIM model data obtained last time, and reuse the rigid body transformation parameters and the registered obstacle positions from the previous solution; at the same time, perform a lightweight iterative nearest point fine registration, only optimizing the translation amount and not the rotation amount, in order to improve positioning accuracy while maintaining computational efficiency.

[0050] If they are different, then execute S2 and S3.

[0051] Optionally, in some embodiments of this application, a safe distance adaptive adjustment during the path tracking phase is further included after S5:

[0052] During the robot's journey along the collision-free 3D path, the local curvature of the collision-free 3D path is calculated in real time.

[0053] When the local curvature exceeds a preset threshold, an additional safety distance proportional to the curvature is added on top of the safety distance perpendicular to the direction of travel of the texturing tool; the safety distance is the sum of the radius of the texturing tool and the preset safety margin.

[0054] When the local curvature is less than a preset threshold, it is restored to the sum of the radius and the preset safety margin.

[0055] (III) Beneficial Effects

[0056] The automatic roughening control method for composite slabs based on BIM-visual fusion proposed in this application can achieve the following beneficial effects:

[0057] By aligning the theoretical coordinates of obstacles in the BIM model data with the actual features detected by vision at the sub-millimeter level, the influence of mold platform positioning deviation and stack plate placement tilt is eliminated, thereby solving the problem of insufficient registration accuracy caused by the simple superposition of "BIM + vision" in the existing technology, and providing an accurate obstacle position benchmark for subsequent path planning;

[0058] By setting a large expansion distance (≥ tool radius + safety margin) perpendicular to the direction of travel of the roughening tool, and only setting a positioning error compensation value in the direction of travel, absolute zero collision is guaranteed, and excessive shrinkage of the roughening area is avoided. In particular, the roughening coverage is significantly improved in the gaps between high-density ribs.

[0059] Through layered hybrid path planning, a collision-free 3D path covering the entire preset roughened area is automatically generated within the 3D passable area model. No manual teaching or programming is required, enabling rapid adaptive production changeover for different models of composite plates. At the same time, the path planning process comprehensively considers obstacle avoidance constraints and coverage integrity, avoiding omissions or repetitions in traditional bow-shaped scanning.

[0060] By including force-position hybrid control instructions in the execution code, an instruction basis is provided for achieving constant roughening depth control during subsequent robot execution. This is different from traditional pure position control or open-loop force control schemes, and an interface is reserved for adaptive compensation of dynamic stiffness changes during the initial setting of concrete.

[0061] In summary, this application achieves zero-collision, high-coverage automated operation of composite slab roughening through the synergistic effect of BIM-visual cross-modal registration, anisotropic expansion, and hierarchical hybrid path planning, effectively overcoming the technical defects of existing technologies such as low data fusion, static obstacle avoidance strategies, and low production changeover efficiency. Attached Figure Description

[0062] Figure 1 This is a flowchart of an automatic roughening control method for composite slabs based on BIM-visual fusion, as described in this application.

[0063] Figure 2 This is a schematic diagram of the overall structure of an automatic roughening control system for composite slabs based on BIM-visual fusion according to an embodiment of this application.

[0064] Among them, 100-truss-type motion base, 101-X-axis guide rail, 102-Y-axis moving crossbeam, 103-first support column, 104-second support column, 105-Y-axis sliding mounting platform, 200-six-axis robotic arm, 201-roughening tool, 300-laminated plate to be roughened, 400-industrial camera, 500-control cabinet, 600-production mold table for supporting laminated plate. Detailed Implementation

[0065] To better explain and facilitate understanding of this application, the following detailed description of the application is provided in conjunction with the accompanying drawings and specific embodiments.

[0066] In existing technologies, methods for controlling roughening of composite slabs can be mainly categorized into the following three types:

[0067] The first type is the manual roughening method: workers manually roughen the surface of the slab with a roughening tool before the concrete initially sets. This method relies on operational experience, has poor consistency in roughening depth, and is prone to omissions; the surface of the composite slab is full of obstacles such as ribs and junction boxes, making manual avoidance inefficient and prone to collision damage; at the same time, it is labor-intensive and poses high occupational health risks, making it difficult to meet the quality and cycle time requirements of large-scale production.

[0068] The second type is the teach-in robot texturing method: the robot's motion trajectory is recorded in advance through manual teaching, and it repeats the taught path during operation. This method is only suitable for batch production of a single plate type. Once the composite plate model or the layout of the ribs changes, re-teaching and programming are required, resulting in long production changeover times. Furthermore, it cannot automatically compensate for mold table positioning errors and actual pose deviations caused by the tilt of the composite plate placement. The path may not align with the actual obstacle position, posing a collision risk. At the same time, it lacks the ability to adapt to the time-varying characteristics of concrete stiffness, making it difficult to maintain a constant texturing depth.

[0069] The third type is the purely vision-guided roughening method: This method uses industrial cameras to capture images of the panel surface, identifies obstacles using visual algorithms, and plans obstacle avoidance paths online. This type of method relies solely on real-time image information and lacks prior design data provided by the BIM model (such as rib coordinates, embedded part locations, and pre-defined roughening areas). The reliability of identification decreases when obstacles are obscured by mud or when lighting changes. Furthermore, the planning algorithm typically does not comprehensively consider the smoothness of the robotic arm's movement and the roughening coverage, resulting in unstable path quality.

[0070] To address this, this application provides an automatic roughening control method for composite slabs based on BIM-visual fusion. By aligning BIM data with visual features at the sub-millimeter level through cross-modal registration, and by balancing safety and coverage using anisotropic expansion, combined with hierarchical hybrid path planning and force-position hybrid control, it achieves adaptive roughening with no teaching required, zero collision, and constant depth, overcoming the shortcomings of the aforementioned prior art.

[0071] To better explain and facilitate understanding of this application, a detailed description of its embodiments is provided below in conjunction with the accompanying drawings. While exemplary embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a clearer and more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0072] Example 1:

[0073] Figure 1 According to this application, an automatic roughening control method for composite slabs based on BIM-visual fusion is proposed, such as... Figure 1 As shown, the automatic roughening control method for composite slabs based on BIM-visual fusion includes:

[0074] S1. When the mold table loading the composite plate to be roughened arrives at the preset station, multiple real-time visual images of the surface of the composite plate to be roughened are acquired from different angles; the multiple real-time visual images are input into a pre-trained deep learning model to obtain the edge contour features and obstacle features of the composite plate to be roughened, the obstacle features including the center line of the rib and the position features of the embedded parts.

[0075] S2. Obtain the BIM model data corresponding to the composite slab to be roughened from the MES system through the communication interface. The BIM model data includes the slab outline dimensions, obstacle coordinates, preset roughening area, preset roughening depth, and preset roughening direction. The obstacle coordinates include the three-dimensional coordinates and height of the center line of the protruding rib and the position of the embedded parts.

[0076] S3. Perform cross-modal registration of the edge contour features and obstacle features with the BIM model data, and solve the rigid body transformation parameters from the BIM model coordinate system to the actual model table coordinate system; use the rigid body transformation parameters to transform the obstacle coordinates in the BIM model data to the actual model table coordinate system to obtain the registered obstacle positions.

[0077] S4. Based on the registered obstacle positions, perform anisotropic expansion on each obstacle to generate a three-dimensional passable area model.

[0078] S5. Based on the preset roughened area, within the three-dimensional passable area model, a layered hybrid path planning algorithm is used to generate a collision-free three-dimensional path covering the preset roughened area.

[0079] Based on the collision-free 3D path and the preset roughening depth, execution code including force-position hybrid control instructions is generated, and the execution code is sent to the robot actuator via an industrial bus.

[0080] This embodiment utilizes a deep learning model to automatically extract the edge contours, rib centerlines, and embedded part location features of the slab surface from multi-view images, achieving accurate identification of obstacles on the composite slab surface. By acquiring BIM model data from the MES system, prior design information such as slab surface contours, obstacle coordinates, preset roughening areas, depth, and direction is introduced, compensating for the shortcomings of pure visual methods in terms of information completeness. Cross-modal registration aligns the visually extracted features with the BIM data, solves the rigid body transformation parameters, and transforms the obstacle coordinates in the BIM coordinate system to the actual mold table coordinate system, effectively eliminating the limitations of the mold table positioning. The system addresses actual positional and orientation deviations such as positional errors and tilting of the composite plate. Through anisotropic expansion, a three-dimensional passable area model is generated, maximizing the area of ​​the roughened region while ensuring absolute avoidance of obstacles. This solves the problems of excessive shrinkage of the roughened region and severe reduction in the coverage of high-density rib areas caused by traditional isotropic expansion. Through layered hybrid path planning, a collision-free three-dimensional path covering the preset roughened region is automatically generated within the passable area model, realizing teachless adaptive path planning for different plate types. This significantly improves the efficiency of multi-product changeover while taking into account obstacle avoidance safety and the integrity of roughened coverage.

[0081] Example 2:

[0082] This embodiment provides an automatic surface roughening control method for composite slabs based on BIM-visual fusion. This method is automatically executed by a robot control system and includes an offline training phase and an online operation phase. The specific steps are described in detail below.

[0083] I. Offline Training Phase: Building a Pre-trained Deep Learning Model

[0084] Step S01: Obtain the training dataset;

[0085] A large number of historical images of composite slab surfaces of different types, under different lighting conditions, and in different concrete states were collected. Each image was manually annotated, and the annotations included:

[0086] Edge contours of the composite board (pixel-level mask);

[0087] Centerline of rib: A sequence of key points marked along the extension direction of the rib;

[0088] Location characteristics of embedded parts (junction boxes, lifting lugs, embedded bolts): marked with bounding boxes.

[0089] All labeled data and their corresponding images together constitute the training dataset.

[0090] Step S02: Construct an improved Mask R-CNN instance segmentation network;

[0091] Based on the standard Mask R-CNN network, its original classification branch, bounding box regression branch and mask branch are retained, and an additional rib centerline regression branch is added on top of this.

[0092] The original standard branches have the following functions:

[0093] (1) Classification branch: used to determine the category of each detection target, including ribs, junction boxes, lifting lugs, screws, etc.;

[0094] (2) Boundary box regression branch: used to output the position of the outer rectangle of the embedded parts and obstacles;

[0095] (3) Mask branch: used to output the instance segmentation mask of the target to achieve pixel-level segmentation.

[0096] The added rib centerline regression branch consists of several convolutional layers and fully connected layers. The input is the shared feature map output by the feature pyramid network, and the output is the coordinate sequence of ordered key points on the rib centerline, so that each rib corresponds to a set of continuous key point coordinates.

[0097] Step S03: Design the loss function and train the network;

[0098] The total loss function consists of four parts: classification loss, bounding box regression loss, mask segmentation loss, and rib centerline continuity constraint loss.

[0099] (1) Classification loss: Cross-entropy loss is used to supervise the target category;

[0100] (2) Bounding box regression loss: Smooth L1 loss is used to supervise the position of the target bounding box;

[0101] (3) Mask segmentation loss: Binary cross-entropy loss is used to supervise the pixel-level mask of the target;

[0102] (4) Continuity constraint loss of rib centerline: Calculate the sum of squared Euclidean distances between adjacent key points on the predicted rib centerline, i.e. L cont For the continuity constraint loss of the rib centerline, p i To predict the i-th keypoint on the centerline, N is the total number of keypoints on a single rib centerline. This loss term is used to penalize centerline breaks, forcing the network to output a continuous, smooth centerline.

[0103] The initial network was iteratively trained using the training dataset until the total loss function converged. After training, a pre-trained deep learning model was obtained. During inference, this model can simultaneously output: the edge contour mask of the composite slab, the keypoint sequence of the rib centerline, and the bounding box and category of the embedded part.

[0104] It's also important to note that protruding ribs are the most critical, complex, and impactful obstacle on the surface of the composite slab, significantly affecting the roughening path planning. They are slender structures with extremely high aspect ratios, making them prone to breakage in conventional segmentation networks. Specialized centerline regression and continuity loss mechanisms are needed to ensure accuracy. In contrast, embedded parts (e.g., junction boxes, lifting lugs) are typically discrete, regularly shaped small objects, which can be handled well by the bounding box regression branch of the standard instance segmentation network, requiring no additional customization of the network structure. Therefore, the improvement in this embodiment addresses the challenge of protruding ribs; for embedded parts, the standard branch is sufficient.

[0105] II. Online Operation Phase: Composite Slab Roughening Control Process

[0106] Step S1: Triggering the mold stage to position and acquiring multi-view images;

[0107] As the mold table carrying the roughened composite slab moves along the production line to the robot station, position sensors (such as photoelectric switches) mounted on the truss columns detect the mold table's positioning signal. The robot control system then synchronously triggers at least two industrial cameras to acquire multiple real-time visual images of the composite slab surface from different angles (e.g., directly above and to the side front). These images cover the entire slab surface and have partial overlap to eliminate occlusion and distortion from a single viewpoint.

[0108] Step S2: Visual feature extraction;

[0109] Multiple real-time visual images are simultaneously input into a pre-trained deep learning model. The model first extracts multi-scale features through a feature pyramid network, and then processes these features through various branches:

[0110] The mask branch outputs the instance segmentation mask of the panel edge contour and obstacles (ribs, embedded parts); the bounding box branch outputs the rectangular box position of the embedded parts; and the rib centerline regression branch outputs the key point sequence of the rib centerline.

[0111] Furthermore, to meet the sub-millimeter accuracy requirements of subsequent cross-modal registration, the keypoint sequence output by the rib centerline regression branch is refined using sub-pixel parabolic fitting. Specifically, the rib centerline regression branch, while outputting the keypoint coordinates, also generates a centerline response feature map of the same size as the input image. Each pixel value in this feature map represents the response intensity at that location belonging to the rib centerline.

[0112] Along the direction perpendicular to the detected rib centerline, take the feature response value (such as the feature map response intensity at that location) and fit a parabola; use the vertex coordinates of the parabola as the sub-pixel precision rib centerline position to replace the original pixel-level coordinates.

[0113] Specifically, sub-pixel level parabolic fitting refinement is performed on each key point on the center line of the rib:

[0114] Let the image coordinates of the current keypoint be (u0, v0), and its normal direction angle be θ (obtained by the local perpendicularity of the centerline at this point). Along the normal direction, taking k pixels to the left and right of (u0, v0) as the center, for a total of 2k+1 sampling points. The coordinates of the sampling points are as follows:

[0115] ;

[0116] The characteristic response value at each sampling point is f(i) (obtained from the centerline response characteristic map by bilinear interpolation).

[0117] With i as the independent variable and f(i) as the dependent variable, fit a quadratic function: f(i) = ai 2 +bi+c;

[0118] The coefficients a, b, and c are solved using the least squares method. The sub-pixel offset is then: ;

[0119] The corrected sub-pixel level keypoint coordinates are: ;

[0120] Here, parameter k is the number of sampling points on one side, usually taken as k=2 or 3, meaning the total number of sampling points is 5 or 7. The above fitting is performed sequentially on all key points on the detected rib centerline to obtain the rib centerline with sub-pixel accuracy.

[0121] After subpixel refinement, the positioning accuracy of the rib centerline is improved from pixel level (approximately 0.3 mm) to subpixel level (approximately 0.05 mm), providing more accurate matching features for the cross-modal registration in the subsequent step S4.

[0122] Thus, the edge contour features and obstacle features of the composite slab to be roughened are obtained. Among them, the obstacle features include: the center line of the rib (a curve composed of continuous key points) and the position features of the embedded parts (boundary box coordinates).

[0123] Step S3: Obtain BIM model data;

[0124] The robot control system sends a request to the MES system via an Ethernet communication interface to obtain the BIM model data of the composite slab corresponding to the current work order. The BIM model data contains at least the following information:

[0125] Plate surface outline dimensions (theoretical boundary);

[0126] Obstacle coordinates: three-dimensional coordinates and height of the center line of the protruding rib, and the position of the embedded parts (junction box, lifting lug, screw rod);

[0127] Preset roughening area (the geometric range on the board surface that needs to be roughened);

[0128] Preset roughening depth (not less than 4mm);

[0129] Preset the roughening direction (usually along the length of the board or according to process requirements).

[0130] Step S4: Cross-modal registration;

[0131] Due to positioning errors (±2~5mm) in the formwork docking and the potential slight tilt of the composite slabs, the theoretical coordinates in the BIM model do not coincide with the actual slab positions. Therefore, it is necessary to perform cross-modal registration between the visually extracted features and the BIM data.

[0132] The specific registration process is as follows:

[0133] ① Establish the projection relationship between the three-dimensional geometric elements in the BIM model data and the camera imaging plane, that is, determine the transformation matrix from the BIM coordinate system to the image pixel coordinate system based on the camera intrinsic and extrinsic parameters.

[0134] ② Use the Iterative Closest Point (ICP) algorithm for bidirectional projection matching:

[0135] Forward projection: Project the three-dimensional coordinates of the rib centerline in the BIM model onto the camera image plane to obtain a series of projection points, which are then connected to form a projection line; calculate the first distance error (point-line distance) between the projection line and the rib centerline detected in step S2.

[0136] Back projection: Back project the key points on the center line of the rib detected in step S2 onto the BIM model plane (using depth information or plane assumption) to obtain back projection points, and connect them to form a back projection line; calculate the second distance error between the back projection line and the center line of the rib in the BIM model.

[0137] ③ With the goal of minimizing the sum of the first distance error and the second distance error, iteratively optimize and solve the rigid body transformation parameters (including rotation matrix and translation vector) from the BIM model coordinate system to the actual model platform coordinate system.

[0138] ④ Using the rigid body transformation parameters obtained from the solution, transform all obstacle coordinates (center lines of protruding ribs, positions of embedded parts) in the BIM model data to the actual model table coordinate system to obtain the registered obstacle positions. These positions are the actual obstacle coordinates that the robot should base on in subsequent planning.

[0139] After registration is completed, the system obtains the precise position of the obstacle in the actual model platform coordinate system. The next step is to construct a model of the area where the robot can safely pass through.

[0140] Step S5: Anisotropic expansion and traversable region modeling;

[0141] In order to maximize the area that can be roughened without colliding with obstacles, each registered obstacle is anisotropically expanded.

[0142] Assume the roughening tool is a circular disc with radius r, and the preset safety margin is d. safe The positioning error compensation value is d loc (Typically 2~5mm). Definition:

[0143] Expansion distance perpendicular to the direction of the burring tool's travel: D lat ≥r+d safe (In practice, the minimum value of the equal sign is usually taken);

[0144] Expansion distance along the direction of the burring tool's travel: D lon =d loc .

[0145] That is, for each obstacle (especially protruding ribs), expand outward laterally (perpendicular to the direction of travel) by D, using its centerline as a reference. lat Expanding outwards longitudinally (parallel to the direction of travel) D lon This creates an elliptical restricted area. After superimposing the restricted areas of all obstacles, subtracting them from the overall area of ​​the board, the remaining free area becomes the 3D passable area model. This model subsequently only allows the robot's end effector to move within it.

[0146] Step S6: Layered hybrid path planning;

[0147] Based on the pre-defined roughened area, a collision-free and fully covered 3D path is generated within the 3D walkable area model. A layered blending strategy is employed.

[0148] Global planning layer: using RRT The Connect algorithm (Bidirectional Fast Expanding Random Tree) searches for an initial global path from the starting point (a corner point of a pre-defined roughening region) to the ending point (the opposite corner point). The cost function of this algorithm combines three objectives:

[0149] Path length item: Prefers shorter paths;

[0150] Robotic arm joint motion smoothness term: penalizes drastic changes in joint angles between adjacent path points;

[0151] Penalty for avoiding protruding ribs: It is inversely proportional to the distance from the path point to the center line of the nearest protruding rib, guiding the path to stay as far away from the protruding rib as possible.

[0152] This yields an initial global path that is collision-free and has smooth motion.

[0153] Local Planning Layer: As the robot travels along the initial global path, a laser displacement sensor mounted at the end of the robotic arm detects real-time changes in the height of the platform in front (with micrometer-level accuracy). Using the Dynamic Window Method (DWA), multiple candidate velocity commands are generated within the current velocity window, and the optimal command is selected based on an evaluation function to correct the local trajectory of the initial global path. The evaluation function includes: obstacle avoidance score (positively correlated with the distance to the nearest obstacle on the predicted trajectory), depth consistency score (negatively correlated with the absolute value of vertical acceleration on the predicted trajectory), and velocity smoothness score. The final output is a collision-free 3D path that satisfies both global coverage and adapts to local terrain changes.

[0154] Step S7: Generation and execution of force-position hybrid control commands;

[0155] Before the roughening operation begins, the system performs online concrete stiffness identification: a flat area on the slab surface is selected (such as the gap between reinforcing bars or the edge of the slab), and the robotic arm is controlled to press down at multiple different depths in a trial manner. The contact force corresponding to each depth is recorded by a six-dimensional force sensor, and the actual stiffness coefficient of the concrete is estimated based on this. The target contact force F that matches the preset roughening depth is then dynamically calculated. target .

[0156] Based on the preset roughening depth (e.g., 4mm) and the target contact force identified above, execution code containing force-position hybrid control instructions is generated and sent to the robot actuator (including gantry and six-axis robotic arm) via EtherCAT industrial bus.

[0157] During the roughening tool's movement along the aforementioned collision-free 3D path, the following controls are executed:

[0158] 1. A six-dimensional force / torque sensor installed between the end flange of the robotic arm and the roughening tool is used to collect the contact force F between the roughening tool and the surface of the laminated plate in real time. actual .

[0159] 2. Calculate the contact force deviation F=F target -F actual .

[0160] 3. An impedance control strategy is adopted to dynamically adjust the vertical position (Z-axis) of the robotic arm's end effector based on the deviation: when the contact force is less than the target force, the end effector presses down; otherwise, it lifts up. The adjustment amount is determined by a proportional-integral-derivative (PID) or impedance control law.

[0161] This creates a position-force closed-loop control, ensuring that the actual roughening depth remains constant at the preset roughening depth, with the error controlled within ±0.3mm.

[0162] Step S8: Security monitoring and anomaly handling;

[0163] During the roughening process, the control system monitors in real time at a frequency of no less than 100Hz:

[0164] Current values ​​of the motors at each joint of a six-axis robotic arm;

[0165] The vibration amplitude of a vibration sensor (accelerometer) installed at the end of a robotic arm.

[0166] When any joint current exceeds a preset threshold (e.g., 150% of the rated current) or vibration amplitude exceeds a safe range (e.g., 0.5g), an abnormal collision or torque over-limit is detected, immediately triggering an emergency stop signal, cutting off the robot's power supply, and issuing an audible and visual alarm. Operation can only resume after the operator has reset the device.

[0167] Step S9: Coverage closed-loop detection;

[0168] After the robot completes the preset roughening path, images of the roughened area are captured again using the same set of industrial cameras. A deep learning model is then used to detect the area coverage of the roughened area. If any un-roughened areas are found (e.g., due to obstruction by local ridges), a short local supplementary path is automatically generated based on the location and contour of the missing area. This supplementary path is then sent to the robot actuator for supplementary roughening until the coverage reaches the preset requirement (e.g., ≥95%).

[0169] III. Optimization Measures for Different Panel Types

[0170] Same-model reuse mechanism: Before step S2, the control system determines whether the model of the current composite plate to be roughened is the same as the model processed in the previous step. If they are the same, the operation of obtaining BIM data from the MES system is skipped, and the previously obtained BIM model data is used directly. The rigid body transformation parameters and the registered obstacle positions are reused from the previous solution. At the same time, in order to compensate for the slight drift after the model table has been running for a long time, a lightweight ICP fine registration is performed: only the translation (X, Y direction) is optimized, and the rotation is not optimized. This ensures accuracy and greatly improves calculation efficiency (the registration time per operation is reduced from about 1 second to less than 50 milliseconds).

[0171] Curvature-adaptive safety distance: During path tracking, the local curvature of the current path point is calculated in real time (calculated using the rate of change of direction of adjacent path points). When the local curvature exceeds a preset threshold (e.g., curvature radius < 500mm), the lateral expansion distance D is increased. lat≥r+d safe In addition to the above, an extra safety distance proportional to the magnitude of curvature is added. Where κ is the curvature of the current path point, and k is a preset scaling factor. When the curvature is less than a threshold, the original D is restored. lat This mechanism significantly reduces the risk of the tool colliding with the ribs on the side when turning.

[0172] This embodiment automatically extracts the features of the panel edge and obstacles through an offline-trained deep learning model. Combined with online acquisition and cross-modal registration of BIM data, it can adapt to any type of composite panel without manual teaching and programming. The iterative nearest point algorithm of bidirectional projection matching is used to register the theoretical BIM coordinates with the actual visual features at the sub-millimeter level, effectively compensating for the positioning error of the mold platform (±2~5mm) and the tilt of the composite panel placement, providing an accurate obstacle position benchmark for path planning.

[0173] Anisotropic expansion strategy sets safety distances differently based on the tool's direction of movement: the lateral (perpendicular to the direction of travel) expansion distance equals the sum of the tool radius and the safety margin, ensuring absolute zero collision; the longitudinal (direction of travel) expansion distance only sets a positioning error compensation value to avoid excessive shrinkage of the roughening area; layered hybrid path planning (global RRT) -Connect+ Local Dynamic Window Method) comprehensively considers path length, joint motion smoothness and rib avoidance, outputs a collision-free 3D path covering the entire preset roughening area, the path is smooth and without omission; force-position hybrid control command collects end contact force in real time, dynamically adjusts vertical position according to deviation from target force, forming closed-loop control;

[0174] By acquiring pouring time and ambient temperature and humidity data through MES, and querying the stiffness mapping table for feedforward correction, combined with online identification of actual stiffness through trial pressing, the target force is dynamically updated to ensure that slabs of different batches at different times within the initial setting window of concrete can maintain a constant roughening depth. For slabs of the same model, BIM data, registration parameters, and planned paths are reused, and only lightweight translational fine registration is performed, reducing the single-slab preparation time from seconds to milliseconds, significantly reducing the communication load and computing overhead of MES. During the path tracking stage, local curvature is calculated in real time, and the lateral safety distance is adaptively increased at curves to effectively suppress the risk of tool side collision with protruding ribs, further ensuring zero collision. Joint current and vibration signals are monitored in real time, and an emergency stop alarm is triggered in case of abnormality. After the operation, the coverage is visually detected, and local paths are automatically reissued to achieve closed-loop control of full roughening coverage.

[0175] In summary, this embodiment systematically solves the core problems in the prior art, such as low data fusion, static obstacle avoidance strategies, low production changeover efficiency, and unstable depth control, through the synergistic effect of BIM-visual fusion, anisotropic expansion, hierarchical path planning, and force-position hybrid control. It significantly improves the automation level, processing quality, and production efficiency of composite slab roughening.

[0176] Example 3:

[0177] This embodiment provides a BIM-visual fusion-based automatic surface roughening control system for composite slabs, used to implement the surface roughening control methods described in Embodiments 1 and 2. A schematic diagram of the overall structure of the BIM-visual fusion-based automatic surface roughening control system for composite slabs is shown below. Figure 2 As shown in the attached diagram. The system structure, workflow, and correspondence between the system and its steps will be explained in detail below, with reference to the accompanying drawings and hardware components.

[0178] I. System Overall Structure

[0179] like Figure 2 As shown, the automatic roughening control system for composite slabs in this embodiment includes: a gantry truss mechanism, a mold table conveying mechanism, a vision acquisition unit, an end effector unit, a control cabinet, and a host computer system.

[0180] 1. Gantry Truss Mechanism

[0181] Truss-type motion base 100: This serves as the load-bearing frame for the entire system, used to install and support all moving parts.

[0182] Support columns 103 and 104: These are the basic support components of the truss-type motion base 100, namely the first support column 103 and the second support column 104. They are arranged symmetrically and together support the upper guide rail and crossbeam to ensure the stability of the mechanism operation.

[0183] X-axis guide rail 101: Fixed to the top of the first support column 103 and the second support column 104, extending vertically along the assembly line, providing X-axis linear motion guidance for the Y-axis moving crossbeam 102.

[0184] Y-axis movable crossbeam 102: can be slidably mounted on X-axis guide rail 101, and can reciprocate along X-axis guide rail.

[0185] Y-axis sliding mounting platform 105: It can be slidably mounted on the Y-axis sliding track of the Y-axis moving crossbeam 102, forming the mounting carrier of the execution unit. It is used to support components such as the six-axis robotic arm 200, the roughening tool 201, and sensors. It can move with the crossbeam in the X and Y directions and cooperate with the Z-axis lifting assembly to achieve three-dimensional spatial movement.

[0186] 2. Mold table conveying mechanism

[0187] Production mold 600 for supporting composite slabs: It is a support platform for composite slabs to be roughened, arranged along the production line direction, and can be transported to the preset roughening station.

[0188] The laminated board 300 to be roughened: placed on the production mold table 600, is the object of the roughening operation.

[0189] A photoelectric sensor is installed at the mold table's position to send a station trigger signal to the control system.

[0190] 3. Visual Acquisition Unit

[0191] Industrial Camera 400: Figure 2 Only one camera is shown for illustration. In the actual system, multiple industrial cameras are installed at different positions and angles on the gantry truss to simultaneously acquire image data of the surface of the composite slab 300 to be roughened from multiple perspectives. This eliminates occlusion, shadows, and blind spots, ensuring the completeness and accuracy of obstacle identification. The cameras are positioned above the truss, with an imaging range covering the entire surface of the slab to be roughened, providing reliable image input for subsequent visual feature extraction and BIM-visual cross-modal registration.

[0192] 4. Control cabinet 500

[0193] Installed below the truss-type motion base 100, it serves as the control core of the system, with built-in motion controller, I / O module, EtherCAT bus communication unit, etc., used to receive commands from the host computer and drive the truss and robotic arm to move, while receiving sensor feedback signals to achieve closed-loop control.

[0194] 5. End-effector

[0195] The Y-axis sliding mounting stage 105 is equipped with a six-axis robotic arm 200, a roughening tool 201, a six-dimensional force / torque sensor, and a laser displacement sensor.

[0196] The roughening tool 201 has a disc-shaped structure and is used to roughen the surface of the laminated board.

[0197] A six-dimensional force / torque sensor is installed between the end flange of the six-axis robotic arm 200 and the roughening tool 201 to collect contact force signals in real time.

[0198] Laser displacement sensors are used to detect height undulations on the board surface, providing data support for local path correction and depth control.

[0199] II. System Workflow

[0200] System workflow overview:

[0201] When the production mold 600 carrying the composite slab transports the composite slab 300 to the roughening station, the position detection sensor on the truss column sends a positioning signal. The control system then triggers the industrial camera 400 mounted on the X-axis guide rail 101 to simultaneously capture images of the composite slab surface and upload the images to the host computer.

[0202] The deep learning inference module in the host computer extracts the edge contour of the slab, the center line of the ribs, and the position features of the embedded parts. At the same time, the BIM data parsing module obtains the BIM model data of the current composite slab from the MES system through the communication interface. The cross-modal registration module aligns the visual features with the BIM data at the sub-millimeter level, solves the rigid body transformation parameters, and transforms the obstacle coordinates. The path planning module generates a collision-free 3D path covering the preset roughened area based on the passable area model constructed by anisotropic expansion. The force-position hybrid control module converts the path and the preset roughening depth into executable code containing force control instructions, which is sent to the motion controller through the EtherCAT bus to drive the gantry truss mechanism (X-axis guide rail 101, Y-axis moving crossbeam 102) to drive the Y-axis sliding mounting platform 105 and the six-axis robotic arm 200 to move in coordination.

[0203] During the roughening process, a six-dimensional force / torque sensor provides real-time feedback on the contact force. The control system dynamically adjusts the vertical position of the end to maintain a constant roughening depth. The force control signal is fed back to the control cabinet 500, achieving closed-loop control. After the operation is completed, the gantry and robotic arm automatically return to the zero position, the mold table moves out of the workstation, and enters the next cycle.

[0204] After power-on initialization, this system enters automatic operation mode and executes the following procedures:

[0205] 1. Mold positioning and image acquisition

[0206] When the mold table is transported to the preset roughening station, the photoelectric sensor detects the position signal, and the control system synchronously triggers multiple industrial cameras to collect real-time images of the laminated board surface from multiple perspectives.

[0207] 2. Visual Feature Recognition

[0208] The host computer inputs the image into the pre-trained deep learning model and outputs the edge contour of the composite plate, the key point sequence of the center line of the rib, and the location box of the embedded part, thus completing the feature extraction of the obstacle.

[0209] 3. BIM Data Acquisition and Cross-Modal Registration

[0210] The control system retrieves the BIM model data of the corresponding composite slab from the MES system via Ethernet, including slab dimensions, 3D coordinates of obstacles, designed roughened areas, and roughening depth. Using the centerline of the protruding rib as the matching feature, bidirectional projection ICP registration is performed to solve the rigid body transformation parameters from the BIM coordinate system to the actual mold table coordinate system, thus completing the obstacle coordinate transformation.

[0211] 4. Anisotropic expansion and construction of walkable regions

[0212] Based on the registered obstacle positions, anisotropic expansion is performed on the protruding ribs and embedded parts: the expansion distance perpendicular to the roughening travel direction is not less than the sum of the roughening tool radius and the safety margin; the expansion distance in the travel direction is set as the positioning error compensation value. After expansion, a three-dimensional passable area model is formed.

[0213] 5. Hierarchical Hybrid Path Planning: The global planning layer adopts RRT. The Connect algorithm generates an initial path covering the roughened area within the passable region. The cost function integrates path length, joint smoothness, and rib avoidance penalty. The local planning layer uses a dynamic window method to correct the trajectory in real time based on the plate height detected by the laser displacement sensor, resulting in a final collision-free 3D path.

[0214] 6. Force-position hybrid control executes roughening.

[0215] The control system determines the target contact force based on the preset roughening depth and sends motion commands via the EtherCAT bus. During the roughening process, a six-dimensional force sensor collects the contact force in real time and uses an impedance control strategy to dynamically adjust the Z-axis position at the end, forming a position-force closed loop to ensure a constant roughening depth.

[0216] 7. The safety monitoring and abnormal protection system monitors the motor current and end-effector vibration signals of each joint in real time. When the current exceeds the limit or the vibration exceeds the standard, it will immediately stop and alarm.

[0217] 8. Coverage detection and rescanning

[0218] After the roughening process is completed, the vision unit acquires images of the board surface again, identifies the missed areas, and automatically generates a re-scanning path to achieve closed-loop control of the roughening coverage.

[0219] Furthermore, this system supports automatic identification of composite slab models. When continuously processing composite slabs of the same model, the system directly reuses historical BIM data and rigid body transformation parameters, and performs lightweight ICP fine registration, optimizing only the X and Y translation amounts, without optimizing the rotation amounts, thereby improving registration efficiency and ensuring continuous production cycle time.

[0220] During path tracking, the system calculates the path curvature in real time. When the curvature is large, the expansion distance perpendicular to the direction of travel is automatically increased. The additional safety distance is positively correlated with the curvature, preventing the roughening tool from laterally colliding with the ribs at the turning points, thus improving operational safety and stability.

[0221] This embodiment, through the integrated structural design of gantry truss, multi-view industrial cameras, and end-effector force control mechanism, provides complete hardware support for cross-modal registration, anisotropic expansion obstacle avoidance, hierarchical path planning, and force-position hybrid roughening in Embodiment 2. This enables the method steps to be stably executed on a real production line, solving the problem of the difficulty in engineering pure algorithm solutions. By using multiple industrial cameras to acquire images from different angles, combined with the truss fixed installation method, the entire area of ​​the composite plate can be completely covered, avoiding centerline breakage and missed detection problems caused by ribs and embedded parts. After being combined with a deep learning model, the detection accuracy and robustness of ribs and embedded parts are significantly improved.

[0222] The combined motion of the gantry truss and end effector balances large-scale operation with high-precision control. The truss mechanism enables large stroke and high-speed movement to meet the requirement of full coverage of the composite slab. The end effector of the robotic arm enables small displacement and force control fine adjustment, balancing production efficiency and consistency of roughening depth. The overall structure is suitable for the production line cycle of precast component plants and can also ensure the accuracy of the roughening process.

[0223] The dual sensing of force sensor and laser displacement sensor enables a true three-dimensional force-position collaborative control device. The device integrates a six-dimensional force sensor and a laser displacement sensor to achieve closed-loop contact force and real-time correction of plate height, respectively. This allows the roughening tool to adapt to the undulations of the concrete surface and stably maintain the designed roughening depth, solving the problems of overcutting and undercutting in traditional open-loop control.

[0224] Anisotropic expansion obstacle avoidance and curvature adaptive safety distance can be directly executed in the device, significantly improving safety. The device calculates the expansion distance and safety margin in real time through the controller, and automatically expands the avoidance area when turning or approaching ribs. From the hardware execution level, it avoids the roughening tool from colliding with ribs and embedded parts, reduces the risk of equipment damage, and improves the stability of continuous operation.

[0225] Supports the reuse of parameters for the same type of board, greatly improving the production efficiency of the production line. The device has a built-in model recognition and historical position parameter caching mechanism. When continuously producing the same type of composite board, there is no need to repeatedly obtain BIM data and perform full registration. Only lightweight fine registration is performed, which greatly shortens the processing time of a single board and increases the production line capacity.

[0226] The multi-layered safety monitoring hardware configuration ensures rapid and reliable abnormal response. The device integrates motor current monitoring, end-point vibration monitoring, emergency stop circuit, and audible and visual alarms. It can trigger protection in milliseconds in the event of collision, jamming, or overload, ensuring the safety of equipment, molds, and personnel, and meeting the safety standards for industrialized production of precast components.

[0227] The device has a high degree of modularity, is compatible with different models of composite plates, and has strong versatility. The vision unit, execution unit, and control unit all adopt modular design. By changing the roughening tool head, adjusting the camera angle, and updating the BIM model, it can adapt to composite plates with various widths, thicknesses, and reinforcement forms, thereby reducing the cost of production line transformation.

[0228] Example 4:

[0229] Finally, this application also proposes a computing device, which includes a processor and a memory. The memory stores a computer program, and the processor executes the instructions stored in the memory so that the computer device performs the automatic roughening control method for composite slabs based on BIM-visual fusion described in the above embodiments.

[0230] It should be noted that any reference numerals placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In claims that enumerate several means, several of these means may be embodied by the same hardware. The use of the terms first, second, third, etc., is merely for convenience of expression and does not indicate any order. These terms can be understood as part of the component names.

[0231] Furthermore, it should be noted that in the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0232] In this application, unless otherwise expressly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0233] In this application, unless otherwise expressly specified and limited, "above" or "below" the second feature can mean that the first and second features are in direct contact, or that they are in indirect contact through an intermediate medium. Furthermore, "above," "on top of," and "over" the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0234] In the description of this specification, the terms "one embodiment," "some embodiments," "embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0235] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make modifications, alterations, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A method for automatic roughening control of composite slabs based on BIM-visual fusion, characterized in that, include: S1. When the mold table loading the composite plate to be roughened arrives at the preset station, multiple real-time visual images of the surface of the composite plate to be roughened are collected from different angles and input into the pre-trained deep learning model to obtain the edge contour features and obstacle features of the composite plate to be roughened. The obstacle features include the center line of the rib and the position features of the embedded parts. S2. Obtain the BIM model data corresponding to the composite slab to be roughened from the MES system through the communication interface, including the slab outline dimensions, obstacle coordinates, preset roughening area, preset roughening depth, and preset roughening direction; the obstacle coordinates include the three-dimensional coordinates and height of the rib centerline and the position of the embedded parts. S3. Perform cross-modal registration of the edge contour features and obstacle features with the BIM model data, and solve the rigid body transformation parameters from the BIM model coordinate system to the actual model table coordinate system; use the rigid body transformation parameters to transform the obstacle coordinates in the BIM model data to the actual model table coordinate system to obtain the registered obstacle positions. S4. Based on the registered obstacle positions, perform anisotropic expansion on each obstacle to generate a three-dimensional passable area model. S5. Based on the preset roughened area, within the three-dimensional passable area model, a layered hybrid path planning algorithm is used to generate a collision-free three-dimensional path covering the preset roughened area. Based on the collision-free 3D path and the preset roughening depth, execute code including force-position hybrid control instructions is generated.

2. The method according to claim 1, characterized in that, The training process of the deep learning model is also included before S1: S01. Obtain the training dataset; the training dataset includes multiple historical images of the surface of the composite plate, each historical image is pre-annotated with edge contour features and obstacle features; The obstacle features include the center line of the rib and the positional features of the embedded parts; S02. An improved Mask R-CNN instance segmentation network is used as the initial network; the improved Mask R-CNN instance segmentation network includes, in addition to the mask branch and bounding box branch of the standard Mask R-CNN, an additional rib centerline regression branch is added; the rib centerline regression branch outputs the coordinate sequence of the rib centerline; S03. The initial network is trained using the training dataset. The loss function during training includes classification loss, bounding box regression loss, mask segmentation loss, and rib centerline continuity constraint loss. The continuity constraint loss of the rib centerline is calculated by summing the squared Euclidean distances between adjacent key points on the rib centerline predicted by the Mask R-CNN instance segmentation network, and is used to penalize the breakage of the rib centerline. When the loss function converges, training ends, and the pre-trained deep learning model is obtained.

3. The method according to claim 1, characterized in that, The cross-modal registration of the edge contour features and obstacle features with the BIM model data in step S3 includes: Establish the projection relationship between the three-dimensional geometric elements in the BIM model data and the camera imaging plane; The iterative nearest point algorithm is used to project the three-dimensional coordinates of the rib centerline in the BIM model data onto the camera image plane to obtain the projection line; the first distance error between the projection line and the rib centerline output by the pre-trained deep learning model is calculated. The positional features of the rib centerline output by the pre-trained deep learning model are back-projected onto the BIM model plane to obtain the back-projection line; the second distance error between the back-projection line and the rib centerline in the BIM model data is calculated. With the goal of minimizing the sum of the first distance error and the second distance error, the rigid body transformation parameters from the BIM model coordinate system to the actual model platform coordinate system are solved.

4. The method according to claim 3, characterized in that, The cross-modal registration also includes: Extract the preset roughening direction from the BIM model data, and extract the surface texture direction from the multiple real-time visual images; The angular deviation between the preset roughening direction and the surface texture direction is taken as the third distance error and added to the optimization objective of the sum of the first distance error and the second distance error. With the goal of minimizing the weighted sum of the first distance error, the second distance error, and the third distance error, the rigid body transformation parameters from the BIM model coordinate system to the actual model table coordinate system are solved to achieve dual-constraint registration of pose and texture direction.

5. The method according to claim 1, characterized in that, The anisotropic expansion of each obstacle in S4 includes: The expansion distance perpendicular to the direction of travel of the texturing tool is greater than or equal to the sum of the radius of the texturing tool and the preset safety margin, and the expansion distance along the direction of travel is set to the preset positioning error compensation value.

6. The method according to claim 1, characterized in that, The step S5, which involves using a hierarchical hybrid path planning algorithm to generate a collision-free 3D path covering the preset roughened area, includes: Based on the preset roughened area, within the three-dimensional passable area model, an initial global path is generated by the global planning layer, and then the initial global path is corrected by the local planning layer to obtain a collision-free three-dimensional path covering the preset roughened area. The global planning layer uses RRT. The Connect algorithm, whose cost function includes a path length term, a robotic arm joint motion smoothness term, and an avoidance penalty term inversely proportional to the distance from the centerline of the rib, is used to search for an initial global path from the starting point to the ending point. The local planning layer uses a laser displacement sensor installed at the end of the robotic arm to detect changes in the height of the front plate in real time, and combines the dynamic window method to perform local trajectory correction on the initial global path, outputting a collision-free 3D path.

7. The method according to claim 1, characterized in that, The execution code generated in S5, which includes force-position hybrid control instructions, includes: The target contact force is determined based on the preset roughening depth; During the process of the roughening tool traveling along the collision-free three-dimensional path, the contact force between the roughening tool and the surface of the composite plate is collected in real time, and the vertical position of the roughening tool is dynamically adjusted according to the deviation between the contact force and the target contact force to keep the roughening depth constant at the preset roughening depth.

8. The method according to claim 7, characterized in that, The force-position hybrid control command employs an impedance control strategy and includes feedforward correction of the target contact force: The concrete pouring time of the composite slab to be roughened and the ambient temperature and humidity data of the current production workshop are obtained from the MES system through the communication interface; based on the pouring time and ambient temperature and humidity data, the pre-established concrete stiffness prediction mapping table is queried to obtain the estimated stiffness value of the current concrete; before the roughening operation begins, the initial target contact force is weighted and corrected according to the estimated stiffness value to obtain the corrected target contact force. Based on the preset roughening depth and the actual height of the plate surface detected by the laser displacement sensor installed at the end of the robotic arm, the theoretical downward pressure of the end tool is calculated in a feedforward manner; when the contact force collected in real time deviates from the corrected target contact force, the vertical position of the end of the robotic arm is dynamically adjusted to form a position-force closed-loop control.

9. The method according to claim 1, characterized in that, Prior to S2, there are also model reuse and lightweight registration steps: Determine whether the model of the current composite board to be roughened is the same as the model of the composite board processed last time; If they are the same, skip the operation of obtaining BIM model data from the MES system in S2, directly use the BIM model data obtained last time, and reuse the rigid body transformation parameters and the registered obstacle positions from the previous solution; at the same time, perform a lightweight iterative nearest point fine registration, only optimizing the translation amount and not the rotation amount, in order to improve positioning accuracy while maintaining computational efficiency. If they are different, then execute S2 and S3.

10. The method according to claim 1, characterized in that, Following S5, adaptive adjustment of the safe distance during the path tracking phase is also included. During the robot's journey along the collision-free 3D path, the local curvature of the collision-free 3D path is calculated in real time. When the local curvature exceeds a preset threshold, an additional safety distance proportional to the curvature is added on top of the safety distance perpendicular to the direction of travel of the texturing tool; the safety distance is the sum of the radius of the texturing tool and the preset safety margin. When the local curvature is less than a preset threshold, it is restored to the sum of the radius and the preset safety margin.

Images