Automatic Hole-Finding Installation Method for Precast Slabs Integrating Visual and Force Feedback
By using a Bayesian joint optimization framework that integrates vision and force feedback, combined with a distributed tactile sensor array and a contact state diagram model, the problems of insufficient visual accuracy and lack of global guidance in force feedback during prefabricated panel installation are solved, thus achieving an efficient and reliable automated installation process.
Patent Information
- Application Number
- CN202611544529.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-30
- Publication Date
- 2026-11-17
AI Technical Summary
In the existing precast slab installation process, the vision system has difficulty ensuring the accuracy of the reference holes under light and dust interference. The pure force perception method lacks global guidance, resulting in low installation efficiency and easy tilting and jamming.
A Bayesian joint optimization framework integrating vision and force feedback is constructed. By combining a distributed tactile sensor array and a contact state graph model, multi-hole contact probability inference and adaptive strategy switching are performed to achieve high-precision installation throughout the entire process from coarse alignment to press-fitting.
High-precision automatic installation of precast panels was achieved under complex working conditions, which significantly improved the accuracy of hole matching and the robustness of the hole finding process, avoided tilting and jamming, and ensured the smoothness and reliability of the installation process.
Smart Images

Figure CN123115969A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of building construction and automated assembly technology, specifically relating to an automatic hole-finding and installation method for precast slabs that integrates vision and force feedback. Background Technology
[0002] Precast slab installation is a crucial process in prefabricated construction, its fundamental task being to precisely align the multiple reference holes pre-drilled on the precast slab with the positioning pins fixed on the installation surface. Currently, construction sites commonly employ a combination of manual visual inspection and remote crane operation. This process is highly dependent on operator experience and susceptible to errors in perspective and distance judgment, leading to repeated lifting and adjustments and low efficiency. With the development of automation technology, researchers have attempted to introduce machine vision to guide hoisting equipment for automatic hole finding. However, in actual construction environments, lighting conditions vary significantly, and the concrete surface is subject to severe interference from dust and laitance layers. Furthermore, due to factors such as casting shrinkage and demolding deformation of the precast slab itself, the actual positions of the reference holes differ from those on the design drawings, making it difficult for the vision system to guarantee sufficient absolute accuracy in extracting the hole center coordinates. When the geometric arrangement of multiple reference holes is approximately symmetrical or evenly spaced, single visual information is insufficient to correctly distinguish the correspondence between the holes, easily leading to matching ambiguities.
[0003] Pure force-guided methods rely on contact force feedback to infer the relative positions of holes and pins, unaffected by external environmental imaging conditions. However, before multiple holes simultaneously contact each other, only a single hole makes contact with the pin. The contact force vector at that point only reflects local geometric constraints and cannot directly provide position information for the remaining uncontacted holes. The search strategy lacks global guidance and often requires probing in multiple directions, which is time-consuming. Furthermore, the wedge-shaped contact surface formed by the chamfer at the hole opening and the chamfer at the pin end creates a strong nonlinear relationship between the contact force direction and the hole center deviation. When there are manufacturing deviations in the friction coefficient and chamfer angle, the correspondence between the force direction and the deviation direction will shift significantly, increasing the difficulty of adjusting the direction decision. When multiple holes are simultaneously inserted into the pin, the uneven force on each hole can cause the precast slab to tilt, resulting in an overturning moment. If the adjustment strategy is inappropriate, it can easily lead to jamming. In this case, it is difficult to determine which side of the hole position deviation caused the jamming based solely on local force information, and the traditional probing retreat strategy lacks specificity. Related studies, such as Asada and Tsuji's contact state analysis in active force sensing assembly and Whitney's discussion on jamming conditions in quasi-static assembly of rigid components, all point out that in the pin hole assembly process, the geometric interpretation of force information depends on an accurate contact state model. However, this model is difficult to accurately establish in actual scenarios with porous, large-size, and low-stiffness components. Existing methods mostly process visual and force information separately or only switch them in high-level decision-making, and have not yet effectively solved the problem of fusing the two information to overcome their inherent limitations. Summary of the Invention
[0004] This invention provides an automatic hole-finding and installation method for prefabricated panels that integrates vision and force feedback. It can construct a Bayesian joint optimization framework that deeply integrates vision and distributed tactile force feedback, and combine it with a contact state diagram model to realize multi-hole contact probability inference and single-hole / multi-hole adaptive strategy switching. Thus, under complex working conditions such as light and dust interference, visual matching ambiguity, multi-hole tilting and jamming, and visual blind spots at the end of the installation, it can automatically complete the entire process of high-precision installation of prefabricated panels from rough alignment, hole finding, import and press-fitting.
[0005] To achieve these objectives and other advantages of the present invention, an automatic hole-finding and installation method for prefabricated panels integrating vision and force feedback is provided, comprising: S1. The vision system acquires images of the reference holes and target holes on the mounting surface of the precast slab, extracts the sub-pixel hole centers, constructs a local coordinate system for the precast slab with at least three non-collinear reference holes, calculates the initial pose deviation, and drives the hoisting equipment to the coarse alignment area. S2. Contact force is detected by a distributed tactile sensor array integrated around the root of each locating pin on the mounting surface. When contact is detected for the first time, the pressure value of each sensor unit is recorded. Combined with the prior geometry of the hole group, the posterior probability of each candidate hole being a contact point is calculated using a contact state diagram model. When the posterior probability entropy exceeds a preset threshold, a controlled micro-amplitude planar excitation is applied, the force distribution change is observed, and the posterior probability is updated. The target hole with the highest probability and its predicted deviation direction are determined, and the posterior probability distribution of the hole center deviation direction is obtained, hereinafter referred to as the force direction probability. The number of contact holes is determined based on the number of holes with posterior probabilities exceeding the preset threshold. S3. Using the predicted deviation direction as a priori, the vision system is triggered to switch to local high resolution mode. Secondary extraction is performed in the neighborhood of the predicted hole center. The force direction probability and the confidence of the visual hole center matching are jointly optimized by Bayesian to obtain the hole position matching with the highest confidence. S4. Update the target pose accordingly and find the hole in a plane perpendicular to the installation direction according to the geometric constraints of the hole group: when the number of contact holes is less than 2, the adjustment direction is determined based on the contact force direction of a single hole and the geometric constraints of the chamfer at the hole opening. When the number of contact holes is greater than or equal to 2, the pose is adjusted with the minimum variance of the normal contact force of each hole as the target until at least two non-collinear holes enter the guide section. S5. Press-fit along the installation direction, monitor the axial pressing force and overturning moment in real time. When the rate of change of the pressing force exceeds the first slope threshold, determine the orifice interference and apply low-frequency micro-amplitude vibration to assist the feed. If the safety threshold is exceeded, reverse micro-retreat and return to S3. The installation in place determination is based on the force feedback seating characteristics, including the axial pressing force reaching the preset threshold and the rate of change of the axial force being lower than the second slope threshold. The mapping relationship between force direction and hole center deviation is initially provided by the CAD model and updated through online incremental calibration during installation; the distributed tactile sensor array provides direct observation of the contact force of each hole, and the contact state diagram model automatically switches between single-hole and multi-hole adjustment strategies according to the contact probability of each hole.
[0006] Preferably, the distributed tactile sensing array includes multiple pressure sensing units uniformly arranged circumferentially along each positioning pin. The output of each pressure sensing unit is conditioned to form a contact force distribution vector. The contact state diagram model uses the contact state of each hole-pin as the hidden state, the contact force distribution vector as the observation, and the prior geometry of the hole group as the prior probability. In step S2, only the force distribution vector and the prior are used for preliminary posterior calculation. After local high-resolution visual extraction in step S3, the visual hole center matching is used as the new observation to update the posterior. When the posterior probability is lower than a preset threshold, the pose is updated based on the visual matching result.
[0007] Preferably, in step S2, the initial contact detection uses wavelet packet decomposition of the force signal to extract high-frequency impact features, and combines the rising edge slope for judgment; when both the high-frequency impact feature energy and the rising edge slope exceed the threshold, it is determined to be a valid contact; at the same time, the quadrant of the contact point is determined according to the force difference of each unit in the distributed tactile sensing array; the controlled micro-amplitude planar excitation is to apply an orthogonal circular or cross-shaped trajectory with an amplitude smaller than the guide section gap in a plane perpendicular to the installation direction, and the excitation frequency avoids the natural frequency of the hoisting system, and the ambiguity of the contact direction is eliminated by observing the amplitude and phase changes of the force distribution vector.
[0008] Preferably, the Bayesian joint optimization in step S3 is as follows: using the force direction probability as the prior and the visual hole center matching confidence as the likelihood, a joint objective function is constructed and the hole center position is iteratively solved; when the posterior probability entropy exceeds the threshold or the force and visual deviation exceeds the threshold, the force direction is determined to be invalid, the target position is updated based on the visual matching result, and the local high-resolution search window is expanded to cover the entire candidate hole area or switched back to the large field of view mode.
[0009] Preferably, in step S4, when a single hole is in contact, the adjustment direction is determined based on the direction of the single hole contact force and the geometric constraints of the hole chamfer. Specifically, the position of the contact point on the chamfer is estimated based on the direction of the contact force, the half angle of the hole chamfer, and the friction angle, thereby determining the direction of the hole center deviation. The friction angle is obtained by friction testing of the precast plate hole and the positioning pin material before installation, or estimated online based on historical installation data. Planar micro-motion is applied along this direction and the change in contact force is monitored. When the contact state diagram model detects that the probability of contact of the second hole exceeds a preset threshold, it automatically switches to the multi-hole force balance adjustment mode.
[0010] Preferably, during the multi-hole force balance adjustment in step S4, the distributed tactile sensing array outputs the contact pressure of each hole, calculates the normal contact force of each hole based on the geometric relationship of the hole group, performs low-pass filtering or moving average on the normal contact force sequence, constructs the objective function of the variance of the normal contact force of each hole, and adopts a gradient descent or trial adjustment strategy; when the difference in contact force of each hole or the overturning moment exceeds the threshold, it is determined that the tilt is stuck and the reverse micro-retreat is performed to adjust the posture.
[0011] Preferably, the online incremental calibration specifically involves: initial parameters provided by the CAD model and a simple calibration before installation; during installation, when the vision system successfully extracts the hole center and the confidence level of the force direction probability matching exceeds a preset threshold, the matching pair is used as an incremental sample, and the force-visual coordinate transformation parameters are updated using recursive least squares or Kalman filtering; when the confidence level is lower than the threshold, the sample is discarded and the parameters of the previous time step are retained.
[0012] Preferably, after coarse alignment, the vision system tracks multiple deformation monitoring marks set on the edge of the precast slab in real time, calculates the mark displacement using digital image correlation and interpolates to obtain the elastic deformation field, and corrects the coordinates of each target hole as the predicted hole center in step S3; when the mark moves out of the field of view, compensation is performed based on the actual position of the hole that has entered the guide section and the geometric constraints of the hole group.
[0013] Preferably, in step S5, the frequency of the low-frequency micro-amplitude vibration avoids the natural frequency of the precast plate-positioning pin system, the amplitude is less than the gap of the guide section, and the vibration is only started when the rate of change of the pressing force exceeds the first slope threshold. If the pressing force continues to rise after vibration, the vibration is stopped and the hole is re-searched after a reverse micro-retraction. The debris at the hole opening is dislodged through the micro-retraction process.
[0014] Preferably, when the end-stage vision system cannot observe the reference hole, it switches to a blind hole-finding mode based on a distributed tactile sensor array. Specifically, the contact state diagram model is used to determine the contact state of each hole. When a hole enters the guide section, its contact force distribution changes from single-point local contact to circumferential multi-point contact, and the force distribution characteristics change. Combined with the geometric constraints of the hole group, a spiral search is performed in the plane perpendicular to the installation direction until the force distribution of all preset holes shows the contact characteristics of the guide section and the axial pressing force reaches the sitting characteristics.
[0015] The present invention has at least the following beneficial effects: First, this invention achieves collaborative decision-making at the probabilistic level between two heterogeneous sensory information types: visual and force feedback, by constructing a Bayesian joint optimization framework that deeply integrates visual and force feedback. Specifically, upon initial contact, the distributed tactile sensor array uses a contact state graph model to infer the posterior probability of each candidate hole. When the probability entropy exceeds a threshold, a controlled micro-amplitude planar excitation is actively applied to eliminate ambiguity in the contact direction and obtain the force direction probability. The visual system uses this direction as a prior to trigger local high-resolution secondary extraction, and performs Bayesian joint optimization using the force direction probability and the visual hole center matching confidence as priors and likelihoods, enabling the two types of information to mutually calibrate and complement each other. When the posterior probability entropy exceeds a threshold or the force and visual deviation exceeds a threshold, the force direction is determined to be invalid and automatically switches to a vision-dominated recovery mode. When the posterior probability is lower than a preset threshold, the pose is updated primarily based on the visual matching result. This fusion mechanism effectively suppresses the hole center extraction bias and matching ambiguity caused by single vision under light and dust interference. At the same time, it overcomes the inherent limitation that pure force perception cannot infer the global hole position relationship under single-point contact conditions, and significantly improves the accuracy of hole position matching and the robustness of the hole finding process.
[0016] Secondly, this invention automatically switches between single-hole adjustment and multi-hole force balance adjustment strategies based on the contact probability of each hole using a contact state diagram model, and achieves adaptive control of the entire process from hole finding and introduction to pressing by combining hole group geometric constraints. In single-hole contact, the position of the contact point on the chamfer is estimated based on the contact force direction, the half angle of the hole opening chamfer, and the friction angle, thereby determining the direction of the hole center deviation. Planar micro-motion is applied along this direction and the change in contact force is monitored to form a closed-loop verification. In multi-hole contact, the distributed tactile sensor array outputs the contact pressure of each hole, and the normal contact force of each hole is calculated in combination with the geometric relationship of the hole group, and a normal contact force variance objective function is constructed. Gradient descent or trial adjustment strategy is adopted to optimize the pose with the goal of minimizing the variance. When the difference in contact force of each hole or the overturning moment exceeds the threshold, tilting and jamming are judged and reverse micro-retraction is performed to adjust the posture. In the pressing stage, the axial pressing force and overturning moment are monitored in real time. When the rate of change of pressing force exceeds the threshold, low-frequency micro-vibration is applied to assist the feed. If the safety threshold is exceeded, reverse micro-retraction is performed and the hole is re-searched. The installation is determined based on the force feedback seating feature that the axial pressing force reaches the preset threshold and the rate of change of axial force is lower than the second slope threshold. This adaptive strategy covers all installation stages, including single-hole finding, dual-hole introduction, multi-hole force balancing, press-fit feeding, and positioning determination. It effectively solves the tilting and jamming problem in multi-hole synchronous installation, ensuring the smoothness and reliability of the installation process.
[0017] Third, this invention, through online incremental calibration and deformation field compensation mechanisms, enables the system to maintain high precision continuously under long-term operation and large-size flexible component conditions. Regarding online incremental calibration, initial parameters are provided by the CAD model and a simple calibration before installation. During installation, when the vision system successfully extracts the hole center and the confidence level of the force direction probability matching exceeds a preset threshold, the matching pair is used as an incremental sample to update the force-visual coordinate transformation parameters using recursive least squares or Kalman filtering. When the confidence level is below the threshold, the sample is discarded while maintaining the parameters from the previous moment, achieving continuous tracking and correction of system parameters drifting with operating conditions without interrupting the installation process. Regarding deformation field compensation, after coarse alignment, the vision system tracks multiple deformation monitoring marks set on the edge of the precast slab in real time, calculates the mark displacement using digital image correlation, and interpolates to obtain the elastic deformation field, correcting the coordinates of each target hole as the predicted hole center. When the mark moves out of the field of view, compensation is performed based on the actual position of the hole that has entered the guide section and the geometric constraints of the hole group. When the final-stage vision system cannot observe the reference holes, it switches to a blind hole-finding mode based on a distributed tactile sensor array. This mode uses a contact state diagram model to determine the contact state of each hole. When a hole enters the guide section, its contact force distribution changes from single-point local contact to circumferential multi-point contact. Combined with the geometric constraints of the hole group, a spiral search is performed in a plane perpendicular to the installation direction until the force distribution of all preset holes exhibits the contact characteristics of the guide section and the axial pressing force reaches the seating characteristic. These mechanisms collectively ensure the continuous effectiveness of the system's position reference throughout the entire process and under all operating conditions, achieving high-precision installation with full coverage from coarse alignment to final seating.
[0018] Other advantages, objectives and features of the present invention will become apparent in part from the following description, and in part from those skilled in the art through study and practice of the invention. Attached Figure Description
[0019] Fig. 1 This is a schematic diagram of the main process of the precast slab automatic hole-finding and installation method that integrates vision and force feedback according to the present invention. Fig. 2 The diagram illustrates the four fault tolerance and recovery logic flows. Detailed Implementation
[0020] The present invention will now be described in further detail so that those skilled in the art can implement it based on the description.
[0021] It should be understood that terms such as “having,” “comprising,” and “including” as used herein do not exclude the presence or addition of one or more other elements or combinations thereof.
[0022] It should be noted that, unless otherwise specified, the experimental methods described in the following embodiments are conventional methods, and the reagents and materials mentioned are commercially available. In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "setting" should be interpreted broadly. For example, they can refer to fixed connection or setting, detachable connection or setting, or integral connection or setting. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances. The terms "lateral," "longitudinal," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this invention and simplifying the description. They do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention.
[0023] like Figs. 1-2 As shown, the automatic hole-finding installation method for precast slabs, which integrates vision and force feedback, provided by this invention, begins its complete implementation process in the rough alignment stage. In this invention, the positioning pin adopts a stepped structure design, comprising, from top to bottom, a chamfered guide section, a straight cylindrical guide section, and a root fixing section. The chamfered guide section is located at the top of the positioning pin and has a conical or arc-shaped chamfer, used to guide the hole opening into place during the descent of the precast slab. The chamfer half-angle is typically 15 to 45 degrees, with an optimal value of 30 degrees. The straight cylindrical guide section is immediately below the chamfered guide section, and its outer diameter has a millimeter-level design gap with the inner diameter of the precast slab's reference hole. This gap is referred to as the guide section gap in this invention, and can be exemplarily 0.5 mm to 2 mm, with an optimal value of 1 mm. The length of the guide section is typically 1 to 2 times the diameter of the positioning pin, used to provide a straight guiding fit after the hole opening passes the chamfered area, ensuring that the precast slab descends vertically along the installation direction. The root fixing section is located at the bottom of the locating pin, and its diameter can be larger than that of the guide section to provide sufficient structural strength for fixing the locating pin to the mounting surface.
[0024] In step S1, the vision system first acquires images of the reference holes on the precast slab and the preset target holes on the mounting surface. It extracts the subpixel-level coordinates of the center of each hole through subpixel edge detection and ellipse fitting algorithms. Then, it selects at least three non-collinear reference holes as reference points. Using the correspondence between these reference points in the local coordinate system of the precast slab and the global coordinate system of the mounting surface, it constructs the local coordinate system of the precast slab and calculates its initial pose deviation relative to the mounting surface. The initial pose deviation includes the translation component in the plane and the rotation component around the normal of the mounting direction. It drives the hoisting equipment to move the precast slab to the coarse alignment area, so that the precast slab and the mounting surface are close to a predetermined distance in the height direction but have not yet made contact. The predetermined distance can be set according to the actual hoisting accuracy, for example, a reasonable value range between 10 mm and 50 mm. This value is only for illustrative reference and not the only limitation.
[0025] After entering the hole-finding stage, step S2 initiates contact detection and contact state inference based on force feedback. A distributed tactile sensor array is integrated circumferentially at the root of each locating pin on the mounting surface. This array consists of multiple pressure sensing units evenly arranged around the circumference of the locating pin. When the precast slab slowly descends under the lifting equipment and first contacts the chamfered tip of one or more locating pins, the tactile sensor array immediately captures the contact force distribution. To improve the reliability of contact detection, the initial contact determination uses wavelet packet decomposition of the force signal to extract the high-frequency impact characteristic energy generated at the moment of contact. Simultaneously, the rising slope of the force signal is monitored. Only when both the high-frequency impact characteristic energy and the rising slope exceed their respective preset thresholds is it considered a valid contact, rather than a false trigger caused by vibration or noise. Furthermore, based on the force differences of the sensing units in each direction within the same circumferential array, the quadrant region of the locating pin where the contact point is located can be preliminarily determined. Once the first contact is confirmed, the system records the pressure value of each sensing unit and calls the pre-built contact state diagram model. This contact state diagram model takes whether each hole pin pair is in contact as the hidden state, takes the force distribution vector of all currently observed sensing arrays as the observation value, and takes the geometric prior of the hole group, that is, the design relative position relationship of each hole on the prefabricated plate, as the prior probability. It calculates the posterior probability of each candidate hole becoming the actual contact point under a given force observation. When the information entropy of these posterior probability distributions exceeds a preset threshold, it indicates significant ambiguity in the contact point judgment. For example, if the posterior probabilities of multiple candidate holes are similar, the system actively applies a controlled micro-amplitude planar excitation. This involves making the precast slab move slightly along a preset orthogonal circular or cross-shaped trajectory in a plane perpendicular to the installation direction. The amplitude of this slight movement must be less than the gap of the guide section of the positioning pin. This gap is typically in the millimeter range and can be between 0.5 mm and 2 mm depending on the actual design. The excitation frequency must avoid the natural frequency of the hoisting equipment to prevent resonance. By observing the amplitude change and phase lag of the force distribution vector of each sensing unit under this excitation, geometric ambiguity in the contact direction can be effectively eliminated, thereby updating the posterior probabilities of each candidate hole. After updating, the candidate hole with the highest posterior probability is determined as the target contact hole. Based on the mapping relationship between the force distribution vector at this contact hole and the geometric constraint of the hole chamfer, the deviation direction of the hole center relative to the pin center is inferred, and the posterior probability distribution of this deviation direction is obtained. This distribution is called the force direction probability. In addition, based on the number of holes whose posterior probability exceeds a preset threshold, the system automatically identifies the number of holes that have already been contacted, providing a basis for subsequent strategy switching.
[0026] In this invention, the mapping relationship between the force direction and the hole center deviation specifically includes the following parameters: the radial and tangential components of the contact force vector in the locating pin, the half-angle of the hole chamfer, the friction coefficient or friction angle between the precast plate material and the locating pin material, and the position coordinates of the contact point on the chamfered surface. The mathematical model for this mapping relationship adopts a nonlinear contact force model based on static equilibrium. That is, by measuring the contact force direction and combining it with known chamfer geometric parameters and friction parameters, a functional relationship between the contact force direction and the hole center deviation direction is established. Specifically, when the hole chamfer contacts the pin end chamfer, the contact force acts along the normal direction of the chamfered surface at the contact point. Through force decomposition and equilibrium equations, the position of the contact point on the chamfered surface can be deduced, and the offset of this position relative to the chamfer symmetry center directly corresponds to the deviation direction and approximate deviation of the hole center relative to the pin center. The initial mapping parameters are theoretically provided by the CAD models of the precast slab and the mounting surface. A simple on-site calibration is performed before installation to establish the initial transformation relationship between the visual coordinate system and the force coordinate system. This simple calibration can be completed using standard blocks of known dimensions or single-hole trial installation. During actual installation, this mapping relationship is continuously updated through online incremental calibration to adapt to changes in friction coefficient, chamfer wear, and other working conditions.
[0027] Step S3 involves deep fusion of force and visual information. Using the force direction probability obtained in step S2 as prior information, the system triggers a switch from the current large field-of-view mode to a local high-resolution mode. This involves re-acquiring images within the predicted aperture center neighborhood and performing a more precise secondary extraction of the aperture center. This neighborhood can be, for example, a square window centered on the predicted aperture center with a side length approximately three to five times the guide segment gap, thus obtaining the visual aperture center coordinates and their matching confidence. Subsequently, Bayesian joint optimization is performed. Specifically, the force direction probability is treated as a prior distribution, and the visual aperture center matching confidence is treated as a likelihood function. A joint objective function is constructed, and the most probable aperture center position is iteratively solved to obtain the aperture position matching result with the highest confidence. During this fusion process, if the posterior probability entropy exceeds the threshold again, or the deviation between the force direction and the visual detection result exceeds the preset deviation threshold, it is determined that the current force direction information has become invalid. At this time, the system automatically degrades to updating the target pose based on the visual matching result, while expanding the local high-resolution search window to cover the entire candidate aperture area. If necessary, it may even switch back to the large field of view mode to reacquire the global reference.
[0028] After obtaining a high-confidence hole position match, step S4 updates the target installation pose of the precast plate according to the matching result and performs collaborative hole finding in a plane perpendicular to the installation direction according to the geometric constraints of the hole group. When the system determines that there are fewer than two contact holes, that is, only one hole is in contact, a single-hole adjustment strategy is adopted: based on the contact force direction of the single hole, the half-angle parameter of the hole chamfer, and the friction angle between the precast plate and the positioning pin material, the specific position of the contact point on the chamfered surface is estimated through the static balance relationship between force and geometry, and then the deviation direction of the hole center relative to the pin center is deduced. The friction angle can be obtained through standard friction test before installation or estimated online based on historical installation data. Then, plane micro-motion is applied along the deviation direction, and the change of contact force is continuously monitored during the micro-motion to verify whether the adjustment direction is correct. When the contact state diagram model detects that the posterior probability of the second hole exceeds the preset threshold, the system automatically switches to multi-hole adjustment mode. When the number of contact holes reaches two or more, the system switches to a multi-hole force equalization adjustment strategy: Utilizing the contact pressure of each hole output by the distributed tactile sensor array, combined with the geometric relationship of the hole group (i.e., the design distance and angle between each hole), the normal contact force at each contact hole is calculated. These normal contact force sequences are then low-pass filtered or averaged to remove high-frequency noise. The variance of the normal contact force of each hole is then constructed as the objective function. Gradient descent or a trial-and-error fine-tuning strategy is used to adjust the pose of the precast plate in the plane, minimizing the variance of the normal contact force of each hole, thereby achieving uniform force distribution across all holes. During the above adjustment process, if the difference in contact force between holes or the overturning moment exceeds a corresponding threshold, it is determined that the precast plate has tilted and jammed. At this point, a reverse micro-retreat action is performed, and the attitude angle of the precast plate is adjusted, then the hole search continues. The above single-hole or multi-hole adjustment is repeated until at least two non-collinear holes successfully enter the guide section of the locating pin, i.e., the hole opening has crossed the chamfered area and entered the straight cylindrical guide mating area.
[0029] The system then proceeds to the pressing stage in step S5. A pressing force is applied along the installation direction, i.e., the vertical direction, to ensure the precast slab moves smoothly downwards. Simultaneously, the axial pressing force and overturning moment are monitored in real time. During the pressing process, if the rate of change of the axial pressing force exceeds the first slope threshold, it indicates interference or jamming between the orifice and the pin. In this case, the system applies low-frequency micro-vibration to assist feeding. The frequency of this vibration must avoid the inherent frequency of the system formed by the precast slab and the positioning pin, and the amplitude must still be less than the guide section gap. This vibration is only initiated when the rate of change of the pressing force exceeds the threshold and is not used as a normal excitation. It should be noted that the first slope threshold is used to determine orifice interference and trigger vibration assistance; its exemplary selection range is 1 Newton per second to 5 Newtons per second, with an optimal value set at 3 Newtons per second. The second slope threshold is used to determine that the installation is in place; its value should be much smaller than the first slope threshold, with an exemplary selection range of 0.1 Newtons per second to 0.5 Newtons per second, and an optimal value set at 0.2 Newtons per second, to characterize a stable seating state where the pressing force change tends to be stable. If the pressing force continues to rise after vibration, the vibration is stopped and the device is slightly retracted in the reverse direction, allowing debris at the orifice to fall off naturally during the retraction process. Then, the process returns to step S3 to re-perform high-precision hole matching and hole finding. When the axial pressing force exceeds the safety threshold, the device is similarly slightly retracted in the reverse direction and returns to step S3. The final determination of proper installation is based on the seating characteristics of the force feedback, i.e., the axial pressing force reaches the preset seating force threshold, and the rate of change of the axial force is lower than the second slope threshold, indicating that the precast plate has been completely placed on the mounting surface, and all holes and positioning pins have reached the designed mating depth, indicating that the installation is complete.
[0030] Throughout the above process, the initial parameters of the mapping relationship between force direction and hole center deviation are provided by the CAD model and a simple calibration before installation. During actual installation, whenever the vision system successfully extracts the hole center and the matching confidence level of the force direction probability exceeds a preset threshold, the system uses this matching pair as an incremental sample and updates the coordinate transformation parameters between force perception and vision online using recursive least squares or Kalman filtering algorithms. If the confidence level is below the threshold, the sample is discarded while maintaining the parameters from the previous moment, thus achieving continuous calibration of the mapping relationship. The distributed tactile sensor array always provides direct observation of the contact force of each hole, and the contact state diagram model automatically and smoothly switches between single-hole adjustment and multi-hole adjustment strategies based on changes in the contact probability of each hole, ensuring that the hole-finding process has the optimal control strategy at different contact stages.
[0031] By deeply integrating the global positioning capability of vision with the local contact perception capability of distributed tactile sensing at the probabilistic level, the system not only significantly suppresses the impact of changes in lighting, dust obstruction, and concrete surface texture interference on the accuracy of visual hole center extraction in the construction environment, but also effectively overcomes the limitation of pure force-guided methods lacking global hole position information in single-point contact states. This allows for accurate differentiation of hole correspondences even when multiple holes are arranged in a near-symmetrical or equidistant manner, avoiding matching ambiguities. Meanwhile, the contact state diagram model-driven single-hole and multi-hole adaptive switching strategy makes the transition from initial contact to multi-hole introduction smoother. Multi-hole force balance adjustment effectively prevents tilting and jamming caused by uneven force on each hole. Vibration assistance and force change rate monitoring during the pressing stage further enhance anti-interference capability, while online incremental calibration and blind hole-finding mode ensure the continuous accuracy and reliability of the system throughout the entire process and under all working conditions. Ultimately, this achieves fully automated and highly robust installation of precast slabs from rough alignment to final placement.
[0032] In one technical solution, the distributed tactile sensing array specifically includes multiple pressure sensing units uniformly arranged circumferentially around each positioning pin. The number of these pressure sensing units is determined according to the diameter of the positioning pin and the required angular resolution. For example, 8 or 12 pressure sensing units can be uniformly arranged around each positioning pin. The pressure signal output by each pressure sensing unit is amplified, filtered, and converted from analog to digital by a signal conditioning circuit to form a vector characterizing the circumferential contact force distribution of the positioning pin. This vector is the basic observation data for the subsequent contact state diagram model.
[0033] The contact state diagram model uses the contact state between each hole and each positioning pin as a hidden state variable. This hidden state is a binary variable, taking the value of contact or non-contact. The force distribution vector output by all sensor arrays at the current moment is used as the observation variable, and the hole group geometry prior is used as the prior probability. This prior reflects the inherent probability that each hole will make contact when there is no observation information. This probability is jointly determined by the current pose of the prefabricated plate and the relative positional relationship of the hole group. In the specific implementation of step S2, the system only relies on the above-mentioned force distribution observation vector and hole group geometry prior to perform preliminary posterior probability calculation. That is, without introducing high-resolution visual observation, the system independently determines the probability that each candidate hole is an actual contact point based on force information.
[0034] After the local high-resolution vision completes the secondary extraction and obtains the visual aperture center matching result in subsequent step S3, the system uses this visual matching result as a new observation dimension to perform a Bayesian update on the original posterior probability, so that the fused posterior probability reflects both force and visual evidence. When the fused and updated posterior probability is still lower than the preset threshold, it indicates that the current force observation has low reliability, possibly due to sensor array contamination, excessive signal noise, or overly complex contact conditions. In this case, the system updates the target pose primarily based on the visual matching result, with the vision system undertaking the main localization decision-making function, thereby ensuring the normal operation of the system even when force information is unreliable.
[0035] By establishing a bidirectional interactive update mechanism between force and visual observations within a Bayesian framework, the two types of sensory information can be adaptively weighted and fused under dynamic changes in confidence levels. This significantly improves the robustness of the fused localization results under various types of interference. Furthermore, it can smoothly switch to a vision-dominated mode when force information degrades, ensuring the system's continued operation even in the event of single-modal failure.
[0036] In one technical solution, the initial contact detection employs wavelet packet decomposition of the force signal. The system performs multi-scale wavelet packet decomposition on the original force signal, extracting the signal energy within the high-frequency subband as the high-frequency impact characteristic energy. Simultaneously, it calculates the rising edge slope of the force signal's leading edge, which reflects the speed at which the contact force is established. When both the high-frequency impact characteristic energy and the rising edge slope exceed their respective preset thresholds, the system determines that the contact is valid. The preset threshold for the high-frequency impact characteristic energy can be adaptively adjusted according to the background vibration and noise level at the site, for example, set to 3 to 5 times the background noise energy. The preset threshold for the rising edge slope depends on the feed speed of the hoisting equipment and the stiffness characteristics of the installed component; for example, it can be set to 0.5 Newtons per millisecond to 2 Newtons per millisecond, with an optimal value of 1 Newton per millisecond. These values are for reference only and not the only limitation.
[0037] Based on the relative magnitudes of the pressure values measured by each sensing unit located at different azimuth angles in the same distributed tactile sensing array, the system can initially determine which quadrant region the contact point is located in circumferentially around the positioning pin. For example, when the pressure of the sensing unit located at 0 degrees is the greatest and the pressure at 180 degrees is the least, the contact point is roughly located near the 0-degree azimuth. This quadrant information provides an initial reference for subsequent excitation response analysis.
[0038] The controlled micro-amplitude planar excitation specifically involves applying a tiny motion trajectory with an amplitude smaller than the gap in the guide section of the positioning pin within a plane perpendicular to the installation direction. This trajectory is an orthogonal circle or cross shape, and the reasonable range for its amplitude is 30% to 70% of the guide section gap, with the optimal value being 50%. For example, when the guide section gap is 1 mm, the amplitude is 0.5 mm. The excitation frequency must avoid the system natural frequency of the hoisting equipment and its load-bearing components. This natural frequency is obtained through modal testing before installation, and the excitation frequency is typically selected as a frequency point at least 20% lower or higher than the natural frequency. Under this excitation, the system observes the amplitude and phase responses of the force distribution vector of each sensing unit. By analyzing the variation of these responses under excitation in different directions, the directional ambiguity caused by the contact point being located at a chamfered symmetrical position is effectively eliminated, thereby accurately determining the deviation direction of the hole center relative to the pin center.
[0039] By introducing high-frequency impact characteristics and rising edge slope as dual criteria for contact detection, the interference of background vibration and random noise is effectively filtered out, significantly improving the accuracy of the first contact determination. At the same time, by actively applying controlled micro-amplitude planar excitation and observing the force response change, the ambiguity in the contact direction judgment caused by the chamfer geometric symmetry is fundamentally eliminated, making the deviation direction inference under single-point contact state reliable and verifiable.
[0040] In one technical solution, the core of Bayesian joint optimization lies in using the force perception direction probability as a prior distribution. This prior distribution represents the system's existing belief in the hole center deviation direction before visual observation. Simultaneously, the hole center matching confidence extracted by the visual system in a local high-resolution mode is used as a likelihood function. This likelihood function reflects the relative probability of each possible hole center position under the current visual observation results. The system multiplies the prior and likelihood and normalizes the result to construct a joint objective function. This objective function is defined as the distribution of the posterior probability density function in the hole center position parameter space. Then, an iterative optimization algorithm is used to solve for the hole center position that maximizes this posterior probability density. This iterative optimization can employ common numerical optimization techniques such as gradient ascent or expectation-maximization algorithms.
[0041] During the optimization process, if the posterior probability entropy after fusion exceeds the preset threshold, it indicates that the consistency between visual and force information is poor and the fusion result still has a large degree of uncertainty. Alternatively, when the deviation between the force direction and the visual detection result exceeds the preset deviation threshold, which is set according to the positioning accuracy required by the system, for example, it is set to twice the allowable deviation of the hole center. For example, it can be 0.2 mm to 0.5 mm, with the optimal value being 0.3 mm. At this time, the system determines that the current force direction information has failed. The reasons for failure may include local failure of the sensor array, mismatch of contact state model parameters, or abnormal wear of the hole chamfer.
[0042] Once the force sensing is determined to be ineffective, the system immediately switches to a decision-making mode based on visual matching results. This means that the target pose is updated by directly using the visually extracted hole center coordinates. At the same time, the size of the local high-resolution search window is expanded to cover the area of all candidate holes, ensuring that the vision system can simultaneously observe all possible contact points. If a matching result with sufficient confidence cannot be obtained after expanding the window, the system switches back to the large field-of-view working mode to reacquire the global field of view and re-establish the preliminary correspondence of the hole positions.
[0043] By constructing a joint objective function that integrates force perception priors and visual likelihoods for iterative optimization, the hole location matching decision benefits from the complementary advantages of the two types of heterogeneous information. Furthermore, by setting entropy thresholds and deviation thresholds as force perception failure criteria, a clear hierarchical recovery strategy is established, enabling the system to quickly degrade without interruption when force perception is abnormal, effectively extending the effective working time of the system under complex working conditions.
[0044] In one technical solution, when the system determines that only one hole is in contact with the locating pin, the direction of the contact force of that single hole and the geometric constraint of the chamfer at the hole opening jointly determine the solution method for the adjustment direction. Both the top of the locating pin and the hole opening of the precast plate are chamfered. When the two chamfers contact, they form a wedge-shaped mating surface. The contact force acts along the normal direction of this wedge-shaped surface. By measuring the actual direction of the contact force and combining it with the known half-angle parameters of the hole opening chamfer and the material friction angle, the specific position of the contact point on the chamfered surface can be deduced using the force decomposition and balance relationship, thereby determining the deviation direction of the hole center relative to the pin center.
[0045] The friction angle is a key parameter affecting the accuracy of deviation direction calculation. This friction angle is obtained through standard friction and wear tests on the same batch of precast slab orifice materials and locating pin materials before installation. Alternatively, it can be estimated online during actual installation using historical contact data, such as by statistically analyzing contact force data from multiple successful installations to update the estimated friction angle. Once the deviation direction is determined, the system applies small-amplitude planar micro-motions along that direction and continuously monitors the change in contact force during the micro-motion process. If the contact force increases with the direction of micro-motion, it indicates that the adjustment direction is correct; otherwise, it needs to be adjusted in the opposite direction.
[0046] When the contact state diagram model detects that the posterior probability of the second hole exceeds a preset threshold, which is the same as the first contact determination threshold, for example, it is set between 70% and 90%, and the optimal value is set to 80%, indicating that the second hole has made substantial contact with the corresponding positioning pin. At this time, the system automatically switches from single-hole adjustment mode to multi-hole force balance adjustment mode.
[0047] By jointly solving the contact force direction and the orifice chamfer geometric constraint, the deviation direction under the single-hole contact state can be directly inferred, avoiding the trial search in multiple possible directions and shortening the time consumption of the single-hole finding stage. At the same time, through the contact force closed-loop verification and probability threshold control mode switching mechanism, the timing accuracy and transition smoothness of the conversion from single-hole strategy to multi-hole strategy are ensured.
[0048] In one technical solution, under the multi-hole simultaneous contact working mode, the distributed tactile sensing array outputs the contact pressure value at each contacted hole. The system combines the geometric relationship of the hole group, including the designed spacing between holes and the designed angle of each hole relative to the center of the precast slab, and calculates the normal contact force at each contact hole using a force and moment balance equation. The normal contact force refers to the component force perpendicular to the axis of the positioning pin, which is the main factor causing the precast slab to shift and tilt in the plane. To eliminate the interference of high-frequency random noise on the force control decision, the system performs low-pass filtering or moving average processing on the time series of each normal contact force. The cutoff frequency of the filter is exemplarily set to 10 Hz to 20 Hz, with an optimal value of 15 Hz. The window length of the moving average is exemplarily set to 5 to 10 sampling periods.
[0049] After filtering, the system constructs an objective function with the variance of the normal contact force of each hole as the evaluation index. The smaller the variance, the more uniform the force on each hole, and the closer the precast plate's posture is to the ideal alignment state. Subsequently, the gradient descent method or a trial-and-error adjustment strategy is used to optimize the pose. The gradient descent method requires calculating the gradient direction of the variance with respect to the pose parameters and stepping along the gradient descent direction. The trial-and-error adjustment strategy applies small displacements in the plane along orthogonal directions and observes the trend of variance changes to select a favorable direction.
[0050] During the adjustment process, if the difference in contact force between holes exceeds the preset difference threshold or the overturning moment exceeds the preset moment threshold, it indicates that the precast slab has tilted in space. The tilting and jamming may be caused by excessive hole position deviation on one side or the deflection deformation of the precast slab itself. At this time, the system performs a reverse micro-retraction action, that is, it retracts a small distance upward along the installation direction. The retraction distance is 1 mm to 3 mm for example, and the optimal value is 2 mm. Then, the attitude angle of the precast slab, that is, the rotation angle around the horizontal axis, is adjusted in the retraction state, and then it is lowered again and the hole search continues.
[0051] By constructing a multi-hole equilibrium control strategy with the minimization of the variance of the normal contact force of each hole as the core, the force distribution during the simultaneous introduction of multiple holes tends to be uniform, effectively suppressing the overturning tendency caused by unilateral load. At the same time, through dual threshold monitoring of force difference and overturning moment, as well as recovery methods such as reverse micro-retreat combined with attitude adjustment, the probability of tilting and jamming during multi-hole synchronous installation and the difficulty of handling jamming are reduced.
[0052] In one technical solution, the core of online incremental calibration lies in continuously updating system parameters using successful matching samples accumulated during actual installation, enabling the system to adapt to environmental changes and drift caused by long-term operation. Initial parameters are provided by theoretical values from the CAD models of the prefabricated slab and mounting surface, and a simple on-site calibration is performed before installation to establish the initial transformation relationship between the visual coordinate system and the force coordinate system. This simple calibration can be completed using standard blocks of known dimensions or single-hole trial installation.
[0053] In actual installation, when the vision system successfully extracts the center coordinates of the hole and the confidence level of the force direction probability at the same moment exceeds the preset threshold, the system will include the vision-force matching pair as an incremental sample into the calibration dataset. The exemplary selection range of the confidence threshold is 75% to 95%, and the optimal value is set to 90%.
[0054] For the acquired incremental samples, the system uses either recursive least squares or Kalman filtering to update the force-visual coordinate transformation parameters. Recursive least squares is suitable for situations where system parameters change relatively smoothly, as it has lower computational complexity and stable convergence. Kalman filtering, on the other hand, is suitable for situations with strong process and measurement noise, and can provide optimal estimation of parameter changes. If the confidence level of the current matching pair is lower than a preset threshold, it indicates that the sample's reliability is insufficient, and introducing such a sample may actually degrade calibration accuracy. In this case, the system discards the sample and maintains the parameter values from the previous time step, waiting for subsequent high-confidence samples to arrive before performing the next update.
[0055] By selectively using high-confidence matching samples for online calibration, the drawbacks of offline calibration requiring interruption of the installation process are avoided, and the contamination of parameter estimation by low-quality samples is effectively suppressed, enabling the system to maintain high positioning accuracy during long-term continuous operation.
[0056] In one technical solution, the precast slab, as a large-sized concrete component, undergoes elastic deformation during hoisting due to factors such as its own weight, uneven distribution of hoisting points, and temperature changes. This deformation causes the actual position of the reference hole to deviate from its designed position, directly affecting the hole-finding accuracy if not compensated. Therefore, after rough alignment, the vision system tracks multiple deformation monitoring marks pre-set on the edge of the precast slab in real time. These marks can be high-contrast artificial targets or natural texture features, with at least three marks that are not collinear to cover the main deformation area of the precast slab. The vision system uses digital image correlation to process continuously acquired image sequences, calculating the displacement components of each monitoring mark in two orthogonal directions within the horizontal plane.
[0057] Based on the displacement data of these discrete marker points, an elastic deformation field of the entire precast slab plane is constructed using interpolation algorithms such as bilinear interpolation or thin-plate spline interpolation. This deformation field reflects the local offset at any position on the precast slab. The system superimposes this deformation field onto the target hole coordinates obtained from the design drawings to obtain the corrected hole coordinates. These corrected coordinates are then used as the reference for predicting the hole center in step S3, enabling the vision system to perform accurate searches near the actual hole positions after deformation, avoiding deviations in the search window caused by component deformation.
[0058] As the precast slab continues to descend to the final installation stage, the edge markers may become obscured as they move out of the vision system's field of view. At this point, the system switches to a compensation strategy, calculating the overall deformation state of the precast slab based on the actual positions of the holes that have entered the positioning pin guide section. Considering the geometric constraints of the hole group, i.e., the relative positions between the holes are fixed in the design, the system uses the rigid body transformation assumption or the least squares fitting method to compensate for the remaining holes that have not yet entered the guide section, ensuring that a certain degree of deformation compensation capability can still be maintained after the markers fail.
[0059] By using digital image correlation to track deformation monitoring markers in real time and interpolate displacement fields, the elastic deformation of large precast slabs caused by factors such as self-weight and temperature during hoisting is effectively compensated. At the same time, after the markers fail, the remaining hole positions are calculated from the actual positions of the already introduced holes, achieving uninterrupted deformation compensation coverage throughout the entire process and improving the accuracy of hole position prediction under flexible component working conditions.
[0060] In one technical solution, the frequency of the low-frequency micro-amplitude vibration must be selected to avoid the natural frequency of the mechanical system formed by the precast slab and the positioning pin, so as to avoid resonance caused by vibration excitation, which could lead to loss of installation accuracy or damage to components. This natural frequency can be measured before installation by hammering or frequency sweeping. The vibration frequency is typically selected to be less than 0.7 times or more than 1.3 times the natural frequency. Specifically, if the natural frequency is 50 Hz, the vibration frequency can be selected as 35 Hz or 65 Hz, and the optimal value is selected as 40 Hz based on the actual frequency avoidance requirements. The vibration amplitude must be less than the single-sided gap of the positioning pin guide section. This gap is usually in the millimeter range. The amplitude is typically 20% to 60% of the guide section gap, and the optimal value is 40% of the guide section gap. For example, when the guide section gap is 1 mm, the amplitude is 0.4 mm.
[0061] The vibration is not applied continuously, but only when the rate of change of the axial pressing force exceeds a first slope threshold. An exemplary selection range for the first slope threshold is 1 Newton per second to 5 Newtons per second, with the optimal value set at 3 Newtons per second, indicating that significant mechanical interference has occurred between the orifice and the pin.
[0062] If the pressing force continues to rise without any relief after vibration is applied, it indicates that the interference is severe or that hard debris is stuck at the orifice. Vibration alone cannot effectively solve the problem. At this time, the system immediately stops vibrating and performs a reverse micro-retraction operation, that is, retracts a preset distance in the opposite direction of the installation direction. The preset distance is 2 mm to 5 mm for example, and the optimal value is 3 mm. Through the relative movement between the orifice and the pin during the micro-retraction process, the debris stuck between the two is dislodged under the action of gravity or friction. Then the system returns to step S3 to perform high-precision hole matching and hole finding again.
[0063] By setting the start conditions and exit mechanism for vibration-assisted feeding, it ensures that low-energy vibration can assist in passing through without damaging components when there is slight jamming, and automatically switches to the reverse micro-retraction and debris shedding process when vibration is ineffective. This avoids the risk of equipment and component damage caused by long-term ineffective vibration, and improves the adaptability to local interference at the orifice during the pressing stage.
[0064] In one technical solution, at the final stage of prefabricated panel installation, when the panel has descended to near the mounting surface, the panel itself obstructs the field of view of the vision system. This prevents the reference holes and deformation monitoring marks originally used for positioning from being captured by the camera. At this point, the system automatically switches to a blind hole-finding mode based on a distributed tactile sensor array, relying entirely on force information to complete the final import and pressing. In blind hole-finding mode, the contact state diagram model continues to run and continuously judges the contact state between each hole and the positioning pin. The judgment is based on the fact that when a hole just enters the guide section of the positioning pin, the contact force distribution characteristics at that hole will change significantly. That is, from the original single-point or small-area force mode formed by only the chamfer of the hole opening and the chamfer of the pin end in partial contact, it changes to the force mode in which the circumferential wall of the hole and the cylindrical surface of the pin form a multi-point or continuous area contact. This change is manifested on the sensor array as multiple azimuth angle sensing units simultaneously detecting pressure, and the pressure distribution tends to be uniform along the circumferential direction.
[0065] The system determines whether a hole has successfully entered the guide section by identifying abrupt changes in this force distribution characteristic. For the remaining holes that have not yet entered the guide section, the system combines the known geometric constraints of the hole group, i.e., the relative positions between each hole are fixed on the precast slab, and the actual position of the hole that has entered the guide section can be accurately located by its contact force distribution. Using this as a reference point, the approximate orientation of the remaining holes that have not yet entered the guide section is calculated using rigid body geometry.
[0066] Subsequently, the system executes a spiral search trajectory in a plane perpendicular to the installation direction. This spiral search starts from the currently estimated theoretical positions of each hole and gradually expands the search radius from the inside out. The step increment of the search radius is set to 0.1 mm to 0.5 mm for example, with an optimal value of 0.2 mm, until all the tactile sensing arrays corresponding to the preset holes exhibit guide section contact characteristics. That is, the force distribution of all holes changes from local contact to circumferential multi-point contact, and the axial pressing force reaches the aforementioned seating characteristics, that is, the axial pressing force reaches the preset seating threshold and the axial force change rate is lower than the second slope threshold, indicating that all holes have synchronously entered the guide section and completed the final seating.
[0067] By constructing a blind hole-finding strategy based on force distribution characteristic criteria and hole group geometric constraints, the system can still autonomously complete the introduction and pressing of all holes in the final stage of installation when vision is completely ineffective. This achieves the automatic installation capability without blind spots from rough alignment to final placement, effectively solving the inherent problem of traditional vision-guided methods being unable to continue working in the final stage of installation due to obstructed vision.
[0068] The number of devices and processing scale described herein are for the purpose of simplifying the description of the invention. Applications, modifications, and variations of the invention will be readily apparent to those skilled in the art.
[0069] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. For those skilled in the art, other modifications can be easily made. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details.
Claims
1. A precast slab automatic hole-finding installation method integrating vision and force feedback, characterized in that, include: S1. The vision system acquires images of the reference holes and target holes on the mounting surface of the precast slab, extracts the sub-pixel hole centers, constructs a local coordinate system for the precast slab with at least three non-collinear reference holes, calculates the initial pose deviation, and drives the hoisting equipment to the coarse alignment area. S2. Contact force is detected by a distributed tactile sensor array integrated around the root of each locating pin on the mounting surface. When contact is detected for the first time, the pressure value of each sensor unit is recorded. Combined with the prior geometry of the hole group, the posterior probability of each candidate hole being a contact point is calculated using a contact state diagram model. When the posterior probability entropy exceeds a preset threshold, a controlled micro-amplitude planar excitation is applied, the force distribution change is observed, and the posterior probability is updated. The target hole with the highest probability and its predicted deviation direction are determined, and the posterior probability distribution of the hole center deviation direction is obtained, hereinafter referred to as the force direction probability. The number of contact holes is determined based on the number of holes with posterior probabilities exceeding the preset threshold. S3. Using the predicted deviation direction as a priori, the vision system is triggered to switch to local high resolution mode. Secondary extraction is performed in the neighborhood of the predicted hole center. The force direction probability and the confidence of the visual hole center matching are jointly optimized by Bayesian to obtain the hole position matching with the highest confidence. S4. Update the target pose accordingly and find the hole in a plane perpendicular to the installation direction according to the geometric constraints of the hole group: when the number of contact holes is less than 2, the adjustment direction is determined based on the contact force direction of a single hole and the geometric constraints of the chamfer at the hole opening. When the number of contact holes is greater than or equal to 2, the pose is adjusted with the minimum variance of the normal contact force of each hole as the target until at least two non-collinear holes enter the guide section. S5. Press-fit along the installation direction, monitor the axial pressing force and overturning moment in real time. When the rate of change of the pressing force exceeds the first slope threshold, determine the orifice interference and apply low-frequency micro-amplitude vibration to assist the feed. If the safety threshold is exceeded, reverse micro-retreat and return to S3. The installation in place determination is based on the force feedback seating characteristics, including the axial pressing force reaching the preset threshold and the rate of change of the axial force being lower than the second slope threshold. The mapping relationship between force direction and hole center deviation is initially provided by the CAD model and updated through online incremental calibration during installation; the distributed tactile sensor array provides direct observation of the contact force of each hole, and the contact state diagram model automatically switches between single-hole and multi-hole adjustment strategies according to the contact probability of each hole.
2. The precast panel automatic hole-finding and installation method integrating vision and force feedback according to claim 1, characterized in that, The distributed tactile sensing array includes multiple pressure sensing units uniformly arranged circumferentially along each positioning pin. The output of each pressure sensing unit is conditioned to form a contact force distribution vector. The contact state diagram model uses the contact state of each hole-pin as the hidden state, the contact force distribution vector as the observation, and the prior geometry of the hole group as the prior probability. In step S2, only the force distribution vector and the prior are used for preliminary posterior calculation. After local high-resolution visual extraction in step S3, the visual hole center matching is used as the new observation to update the posterior. When the posterior probability is lower than a preset threshold, the pose is updated based on the visual matching result.
3. The precast panel automatic hole-finding installation method integrating vision and force feedback according to claim 1, characterized in that, In step S2, the initial contact detection uses wavelet packet decomposition of the force signal to extract high-frequency impact features, and combines the rising edge slope for judgment; when both the high-frequency impact feature energy and the rising edge slope exceed the threshold, it is determined to be a valid contact; at the same time, the quadrant of the contact point is determined based on the force difference of each unit in the distributed tactile sensing array. The controlled micro-amplitude planar excitation is to apply an orthogonal circular or cross-shaped trajectory with an amplitude smaller than the gap of the guide section in a plane perpendicular to the installation direction. The excitation frequency avoids the natural frequency of the hoisting system, and the ambiguity of the contact direction is eliminated by observing the amplitude and phase changes of the force distribution vector.
4. The precast slab automatic hole-finding installation method integrating vision and force feedback according to claim 1, characterized in that, In step S3, the Bayesian joint optimization is as follows: using the force direction probability as the prior and the visual hole center matching confidence as the likelihood, a joint objective function is constructed and the hole center position is iteratively solved; when the posterior probability entropy exceeds the threshold or the force and visual deviation exceeds the threshold, the force direction is determined to be invalid, the target position is updated based on the visual matching result, and the local high-resolution search window is expanded to cover the entire candidate hole area or switched back to the large field of view mode.
5. The precast slab automatic hole-finding installation method integrating vision and force feedback according to claim 1, characterized in that, In step S4, when a single hole is in contact, the adjustment direction is determined based on the direction of the single hole contact force and the geometric constraints of the hole chamfer. Specifically, the position of the contact point on the chamfer is estimated based on the direction of the contact force, the half angle of the hole chamfer, and the friction angle, thereby determining the direction of the hole center deviation. The friction angle is obtained by friction testing of the precast plate hole and the positioning pin material before installation, or estimated online based on historical installation data. Planar micro-motion is applied along this direction and the change in contact force is monitored. When the contact state diagram model detects that the probability of contact of the second hole exceeds the preset threshold, it automatically switches to the multi-hole force balance adjustment mode.
6. The precast slab automatic hole-finding installation method integrating vision and force feedback according to claim 1, characterized in that, In step S4, during the multi-hole force balance adjustment, the distributed tactile sensing array outputs the contact pressure of each hole, calculates the normal contact force of each hole based on the geometric relationship of the hole group, performs low-pass filtering or moving average on the normal contact force sequence, constructs the objective function of the variance of the normal contact force of each hole, and adopts gradient descent or trial adjustment strategy; when the difference in contact force of each hole or the overturning moment exceeds the threshold, it is determined that the tilt is stuck and the reverse micro-retreat is performed to adjust the posture.
7. The precast slab automatic hole-finding installation method integrating vision and force feedback according to claim 1, characterized in that, The online incremental calibration is as follows: the initial parameters are provided by the CAD model and a simple calibration before installation; during installation, when the vision system successfully extracts the hole center and the confidence level of the force direction probability matching exceeds the preset threshold, the matching pair is used as the incremental sample, and the force-visual coordinate transformation parameters are updated by recursive least squares or Kalman filtering. When the confidence level is below the threshold, the sample is discarded and the parameters from the previous time step are retained.
8. The precast slab automatic hole-finding installation method integrating vision and force feedback according to claim 1, characterized in that, After coarse alignment, the vision system tracks multiple deformation monitoring marks set on the edge of the precast slab in real time, calculates the mark displacement using digital image correlation and interpolates to obtain the elastic deformation field, and corrects the coordinates of each target hole as the predicted hole center in step S3; when the mark moves out of the field of view, compensation is performed based on the actual position of the hole that has entered the guide section and the geometric constraints of the hole group.
9. The precast slab automatic hole-finding installation method integrating vision and force feedback according to claim 1, characterized in that, In step S5, the frequency of the low-frequency micro-vibration avoids the natural frequency of the precast plate-positioning pin system, the amplitude is less than the gap of the guide section, and the vibration is only started when the rate of change of the pressing force exceeds the first slope threshold. If the pressing force continues to rise after vibration, the vibration is stopped and the hole is re-searched after a reverse micro-retraction. The debris at the hole opening is dislodged through the micro-retraction process.
10. The precast slab automatic hole-finding installation method integrating vision and force feedback according to claim 1, characterized in that, When the end-stage vision system cannot observe the reference hole, it switches to a blind hole-finding mode based on a distributed tactile sensor array. Specifically, the contact state diagram model is used to determine the contact state of each hole. When a hole enters the guide section, its contact force distribution changes from single-point local contact to circumferential multi-point contact, and the force distribution characteristics change. Combined with the geometric constraints of the hole group, a spiral search is performed in the plane perpendicular to the installation direction until the force distribution of all preset holes shows the contact characteristics of the guide section and the axial pressing force reaches the sitting characteristics.
