A robot curtain wall installation method and system based on assembly state perception

Abstract

This invention belongs to the field of building decoration and installation technology, and particularly relates to a robotic curtain wall installation method and system based on assembly state perception. The method includes: inputting data into an assembly state perception model to obtain the current curtain wall contact state; determining whether the current curtain wall contact state is a parallel state; if not, mapping the current curtain wall contact state to obtain a corresponding robot pose adjustment action according to the contact state-robot action mapping relationship; executing the mapped robot pose adjustment action to adjust the curtain wall contact state until the current curtain wall contact state is a parallel state; if the current curtain wall contact state is a parallel state, performing an assembly attempt until the curtain wall installation is completed. This invention improves curtain wall assembly efficiency by sensing the current curtain wall contact state and adjusting the robot pose action, performing an assembly attempt after the current curtain wall contact state is parallel.

Description

Technical Field

[0001] This invention belongs to the field of building decoration and installation technology, and particularly relates to a robotic curtain wall installation method and system based on assembly status perception. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] In the construction and decoration industry, the installation volume of various types of panels, such as curtain walls, lightweight partition walls, and integrated doors and windows, is very large. Among them, the construction and installation of various curtain walls is a particularly difficult and labor-intensive task. Building curtain walls are made of a variety of materials, including marble, glass, and metal. The panels are heavy and large in size. During the installation process, scaffolding needs to be erected, and other auxiliary facilities are required. The installation work is completed manually by a large number of construction workers, resulting in slow construction progress and poor consistency in installation quality.

[0004] The application of construction robots has brought opportunities to curtain wall assembly tasks. Compared with traditional wet operations, curtain wall assembly mainly adopts the dry hanging method for construction. Among them, SE type hangers are more common. They have the advantages of reliable connection, high strength and good ductility of metal hangers, and reduced building weight. However, the SE hook has a special structure, complex assembly process, and complex assembly trajectory, which leads to the nonlinearity, coupling and system instability of the entire assembly process. Summary of the Invention

[0005] To overcome the shortcomings of the prior art, the present invention provides a robotic curtain wall installation method and system based on assembly state perception.

[0006] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0007] The first aspect of this invention provides a robotic curtain wall installation method based on assembly state perception, comprising:

[0008] Real-time acquisition and preprocessing of contact state time series data of the robot during the assembly process;

[0009] The preprocessed data is input into the assembly status perception model to obtain the current curtain wall contact status;

[0010] Determine whether the current curtain wall contact state is parallel. If not, map the current curtain wall contact state to obtain the corresponding robot pose adjustment action according to the contact state-robot action mapping relationship. Execute the mapped robot pose adjustment action to adjust the curtain wall contact state until the current curtain wall contact state is parallel.

[0011] If the current curtain wall is in a parallel contact state, proceed with the assembly attempt until the curtain wall installation is completed.

[0012] A second aspect of the present invention provides a robotic curtain wall installation system based on assembly state perception, comprising:

[0013] The information acquisition and preprocessing module is configured to: acquire time-series data of the robot's contact state during the assembly process in real time, and perform preprocessing.

[0014] The current curtain wall contact status sensing module is configured to input preprocessed data into the assembly status sensing model to obtain the current curtain wall contact status.

[0015] The robot pose adjustment module is configured to: determine whether the current curtain wall contact state is parallel; if not, map the current curtain wall contact state to obtain the corresponding robot pose adjustment action according to the contact state-robot action mapping relationship; execute the mapped robot pose adjustment action to adjust the curtain wall contact state until the current curtain wall contact state is parallel.

[0016] The assembly attempt module is configured to attempt assembly until the curtain wall installation is completed if the current curtain wall contact state is parallel.

[0017] The above one or more technical solutions have the following beneficial effects:

[0018] (1) This invention senses the current contact state of the curtain wall, adjusts the robot's posture and actions based on the contact state-robot action mapping relationship, and performs assembly attempts after the current contact state of the curtain wall is in a parallel state, thereby completing the installation of the curtain wall and improving the curtain wall assembly efficiency.

[0019] (2) The assembly state perception network based on RNN of this invention has better recognition accuracy and efficiency than other state perception networks.

[0020] (3) The present invention combines position control and force control. Position control is used as the inner control loop to ensure that the robotic arm moves along a trajectory. At the same time, combined admittance control is used as the outer control loop to adjust the movement trajectory, so as to realize different assembly actions in different task stages and realize curtain wall installation operation.

[0021] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0022] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0023] Figure 1 (a) and (b) in the figure are schematic diagrams of SE-type hangers and curtain wall installation, respectively.

[0024] Figure 2 This is a flowchart of a robotic curtain wall installation method based on assembly state perception, as shown in the first embodiment.

[0025] Figure 3 This is a structural diagram of the assembly state perception model of the first embodiment.

[0026] Figure 4 Assembly attempt flowchart for the first embodiment. Detailed Implementation

[0027] Example 1

[0028] like Figure 1 As shown in the figure, this embodiment discloses a robotic curtain wall installation method based on assembly state perception, including a non-contact stage and a contact stage.

[0029] Based on the characteristics of the curtain wall assembly process, this invention divides the assembly process into two stages: a non-contact stage and a contact stage. By collecting information from the six-dimensional force sensor at the end of the robotic arm and the motion state information of the robotic arm itself, the current contact state during the curtain wall installation process is analyzed. Under the mapping relationship between the assembly state and the robotic arm's action, the robot is guided to achieve force and position control.

[0030] Phase 1: Non-contact phase;

[0031] Based on environmental information collected by the camera, the installation location is roughly determined. The robot then moves the curtain wall to the target assembly position and enters the contact phase. The specific steps are as follows:

[0032] Step 101: Use a camera to capture images of the target area.

[0033] Step 102: Use the Gaussian filtering algorithm to remove image noise, use the HSV color space to extract the four curtain wall hangings in the image, and use the Shi-Tomasi corner detection algorithm to extract the feature point set of the four hangings.

[0034] Step 103: Use a convex hull extraction algorithm to obtain the convex hull of the feature point set of the pendant. The centroid of the convex hull is the assembly target position. Based on the assembly target position, estimate the target pose and use robot inverse kinematics to calculate θ = (θ1, θ2, θ3, θ4, θ5, θ6), where θ i The values ​​for the six joints guide the robot to achieve the target pose.

[0035] Phase Two: Contact Phase;

[0036] During the contact phase, force sequences and robot motion state information are collected in real time. A state-assembly state perception model is constructed based on an RNN, and the current state is identified according to the assembly state perception model. A curtain wall assembly state-robot action knowledge base is built. Based on the state perception model, the current contact state is detected in real time and mapped to the corresponding robot pose adjustment action. Based on the result, the robotic arm is controlled to make corresponding pose adjustments. Then, the current new assembly state is detected again, and pose adjustments are continued until the curtain wall installation operation is completed. The specific steps are as follows:

[0037] Step 201: Determine whether the robot has entered the assembly contact process based on the Z-axis force of the robot arm;

[0038] Step 202: Collect real-time time-series data of the robot's contact state during the assembly process and perform preprocessing;

[0039] Step 203: Input the preprocessed data into the assembly status perception model to obtain the current curtain wall contact status;

[0040] Step 204: Determine whether the current curtain wall contact state is parallel. If not, map the current curtain wall contact state to obtain the corresponding robot pose adjustment action according to the contact state-robot action mapping relationship; execute the mapped robot pose adjustment action to adjust the curtain wall contact state until the current curtain wall contact state is parallel.

[0041] Step 205: If the current curtain wall contact state is parallel, then attempt to assemble until the curtain wall installation is completed.

[0042] In step 201, it is determined whether the robotic arm has entered the assembly contact stage. During the contact process, the robot is controlled using a force-position hybrid model. Force control on the assembly direction Z-axis adopts an admittance model, with constant force. Make contact, without applying additional force to other axes, and use position control to keep the robotic arm in a vertical forward trend. When the force on the tool coordinate Z axis is greater than the set value f. z >F z The contact was deemed successful.

[0043] In step 202, during the contact process, the robotic arm will collect continuous contact state time series data in real time, including six-dimensional force information of the curtain wall assembly contact process and the position and pose information of the robotic arm end effector. f =(f x f y f z , τ x , τ y , τ z ,x,y,z,rx,ry,rz).

[0044] After determining contact, the system will perform window sampling based on the contact time point, extracting n consecutive sampling time sequences. This serves as a sample of contact state data; and sample labels λ = {0, 1, 2, 3, 4, 5, 6, 7, 8} are manually added, corresponding to nine different contact states: parallel, forward tilt, backward tilt, left tilt, right tilt, left forward tilt, right forward tilt, left backward tilt, and right backward tilt. Preprocessing includes normalization, filtering, and other processing.

[0045] In step 203, the assembly state perception model adopts a recurrent neural network structure, including an input layer, an output layer, and a hidden layer connected in sequence, such as... Figure 3 As shown;

[0046] The number of nodes in the input layer and hidden layer is set to T. The input K is the 12-dimensional contact state of the current curtain wall, including six-dimensional force information of the curtain wall assembly contact process and the pose information of the robotic arm end effector. f =(f x f y f z , τ x , τ y , τ z ,x,y,z,rx,ry,rz).

[0047] After normalization, filtering, and other processing, the input to the input layer is x. (i) The input u of the hidden layer i =W i-1 ·h (i-1) +x (i) h0 = 0, select the softmax function as the activation function, and set the output o to utilize the last hidden node. (T) =h (T) X serves as the input to the output layer and is processed by the softmax function to obtain the output value y. The output value y is the label value corresponding to the current curtain wall assembly state. By comparing it with the pre-set curtain wall state labels, the curtain wall collision state can be classified.

[0048] A sufficient number of collision process samples are collected in advance to establish a sample library, and random network weights W are used. i Training is performed by periodically training after determining the learning rate, calculating the loss value using the cross-entropy loss function, and using gradient descent to update the network weights and the trained network model.

[0049] In step 204, based on the curtain wall assembly process, assembly actions are designed to form an "Assembly State - Robot Actions" knowledge base, as shown in Table 1:

[0050] Table 1 shows the constructed "Assembly State - Robot Action" knowledge base.

[0051] type Assembly status robot actions 0 parallel stop 1 Lean forward <![CDATA[Move backward and lean backward by θ1 degrees]]> 2 Lean back <![CDATA[Move backward and lean forward by θ2 degrees]]> 3 Left-leaning <![CDATA[Move backward and turn right by θ3 degrees]]> 4 Right-leaning <![CDATA[Move backward and turn left by θ4 degrees]]> 5 Left forward tilt <![CDATA[Move backward and tilt backward to the right by θ5 degrees]]> 6 Right forward tilt <![CDATA[Move backward and tilt left backward by θ6 degrees]]> 7 Left posterior tilt <![CDATA[Move backward and lean forward to the right by θ7 degrees]]> 8 Right back tilt <![CDATA[Move backward and lean forward left by θ8 degrees]]>

[0052] Real-time acquisition of state sequence s during the contact phase f The pre-trained assembly state perception model is input to perform real-time contact state perception and output the assembly state label λ. Then, it is compared with the "assembly state-robot action" knowledge base. Based on the mapping between state and action, the robot is controlled to execute the designed action and adjust the curtain wall state until the current contact state is a parallel state.

[0053] In step 205, after achieving a parallel state, the current parallel contact state is recorded, and an assembly attempt is performed, specifically including:

[0054] Step A: After reaching the parallel state, record the current parallel contact state and the current contact pose as the initial contact pose. Use the coordinate system of the end tool of the robotic arm as the assembly coordinate system, record the initial reading on the Z-axis of the current assembly direction as d, and mark it as the position of the curtain wall contacting the outer surface of the hanger. When the curtain wall contacts the inner surface of the hanger, the absolute value of the difference between the reading on the Z-axis of the assembly direction and the initial reading will exceed the preset threshold (5mm). At this time, perform the engagement operation and record the initial number of attempts p = 1.

[0055] In this assembly coordinate system, the positive X-axis direction is to the left of the curtain wall, the positive Y-axis direction is downward of the curtain wall, and the positive Z-axis direction is forward of the curtain wall. Furthermore, the XOY plane of the assembly coordinate system is parallel to the curtain wall, and the Z-axis of the assembly coordinate system is perpendicular to the curtain wall.

[0056] Step B: Control the robotic arm to move backward a first preset distance (1cm) to ensure that it is not currently in contact, and then move upward a second preset distance (3mm);

[0057] Step C: Control the robotic arm to move forward along the Z-axis in the assembly direction with constant force. Make contact, record the current contact state, and record the current assembly direction Z-axis reading as d'. Determine whether the engagement is successful based on whether the absolute value of the difference between the current assembly direction Z-axis reading d' and the initial reading d exceeds the threshold. If the absolute value exceeds the threshold, the curtain wall is successfully engaged. Control the robotic arm to move 5mm along the positive Y-axis of the assembly direction to complete the installation.

[0058] Step D: If the curtain wall does not engage and is currently in a parallel contact state, proceed to step B.

[0059] Step E: If the engagement fails and the current curtain wall contact state changes from parallel to forward tilt, the curtain wall and the hanger are parallel in relative posture, but the overall position of the curtain wall is slightly above the hanger. Due to the position, the upper part of the curtain wall can contact the hanger, but the lower part cannot. Based on the force analysis, the current curtain wall contact state is forward tilted. The robotic arm is then controlled to return to a new position p (mm) above the initial contact pose, and the process jumps to step B, updating the number of attempts: p = p + 1.

[0060] Throughout the process, the robot's motion is controlled using a force-position hybrid model, employing admittance control on the Z-axis and position control on other axes.

[0061] Example 2

[0062] This embodiment discloses a robotic curtain wall installation system based on assembly state perception, including:

[0063] The information acquisition and preprocessing module is configured to: acquire time-series data of the robot's contact state during the assembly process in real time, and perform preprocessing.

[0064] The current curtain wall contact status sensing module is configured to input preprocessed data into the assembly status sensing model to obtain the current curtain wall contact status.

[0065] The robot pose adjustment module is configured to: determine whether the current curtain wall contact state is parallel; if not, map the current curtain wall contact state to obtain the corresponding robot pose adjustment action according to the contact state-robot action mapping relationship; execute the mapped robot pose adjustment action to adjust the curtain wall contact state until the current curtain wall contact state is parallel.

[0066] The assembly attempt module is configured to attempt assembly until the curtain wall installation is completed if the current curtain wall contact state is parallel.

[0067] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0068] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A robotic curtain wall installation method based on assembly state perception, characterized in that, include: Real-time acquisition and preprocessing of contact state time series data of the robot during the assembly process; The preprocessed data is input into the assembly status perception model to obtain the current curtain wall contact status; Determine whether the current curtain wall contact state is parallel. If not, map the current curtain wall contact state to obtain the corresponding robot pose adjustment action according to the contact state-robot action mapping relationship. Execute the mapped robot pose adjustment action to adjust the curtain wall contact state until the current curtain wall contact state is parallel. If the current curtain wall is in a parallel contact state, proceed with the assembly attempt until the curtain wall installation is completed; include: Step A: After reaching the parallel state, record the current parallel contact state and the current contact pose as the initial contact pose. Use the coordinate system of the end tool of the robotic arm as the assembly coordinate system, record the initial reading on the Z-axis of the current assembly direction as d, and mark it as the position of the curtain wall contacting the outer surface of the hanger. When the curtain wall contacts the inner surface of the hanger, the absolute value of the difference between the reading on the Z-axis of the assembly direction and the initial reading will exceed the preset threshold. At this time, perform the engagement operation and record the initial number of attempts p=1. In this assembly coordinate system, the positive direction of the X-axis is to the left of the curtain wall, the positive direction of the Y-axis is downward of the curtain wall, and the positive direction of the Z-axis is forward of the curtain wall. Furthermore, the XOY plane of the assembly coordinate system is parallel to the curtain wall, and the Z-axis of the assembly coordinate system is perpendicular to the curtain wall. Step B: Control the robotic arm to move backward a first preset distance to ensure that there is no contact at present, and then move upward a second preset distance; Step C: Control the robotic arm to move forward along the Z-axis in the assembly direction with constant force. Make contact, record the current contact state, and record the current assembly direction Z-axis reading as d'. Determine whether the engagement is successful based on whether the absolute value of the difference between the current assembly direction Z-axis reading d' and the initial reading d exceeds the threshold. If the absolute value exceeds the threshold, the curtain wall is successfully engaged. Control the robotic arm to move 5mm along the positive Y-axis of the assembly direction to complete the installation. Step D: If the curtain wall does not engage and is currently in a parallel contact state, proceed to step B. Step E: If the engagement fails and the current curtain wall contact state changes from parallel to forward tilt, control the robotic arm to return to a new position p distance above the initial contact pose, jump to step B, and update the number of attempts: p = p + 1.

2. The robotic curtain wall installation method based on assembly state perception as described in claim 1, characterized in that, The contact state time series data includes: six-dimensional force information at the end of the robotic arm and joint angle information of the robotic arm during the curtain wall assembly contact process.

3. The robotic curtain wall installation method based on assembly state perception as described in claim 1, characterized in that, The assembly state perception model adopts a recurrent neural network structure, including an input layer, an output layer, and a hidden layer connected in sequence; The number of nodes in the input layer and hidden layer is set to The input of the input layer is Input of hidden layer , ,choose The function is activated, and the output is set to utilize the last hidden node. As input to the output layer, and after The function processes and calculates to obtain the output value. The output value y is the label value of the current curtain wall contact state.

4. The robotic curtain wall installation method based on assembly state perception as described in claim 1, characterized in that, Determining whether a robot has entered the assembly contact process based on the Z-axis force of its robotic arm includes: The robot's motion execution is controlled using a force-position hybrid model. Force control on the Z-axis of the assembly direction adopts an admittance model, applying a constant force to achieve contact. No additional force is applied to other axes, and position control is used to keep the robotic arm in a vertical forward trend. When the Z-axis force is greater than the set value, it is determined that the contact is successful, and the robot enters the assembly contact process.

5. The robotic curtain wall installation method based on assembly state perception as described in claim 1, characterized in that, The curtain wall contact states also include: forward tilt, backward tilt, left tilt, right tilt, left forward tilt, right forward tilt, left backward tilt, and right backward tilt; The corresponding robot actions are mapped based on the current contact state of the curtain wall, including the robot's movement direction, angle adjustment direction, and adjustment angle.

6. The robotic curtain wall installation method based on assembly state perception as described in claim 1, characterized in that, Also includes: In the non-contact stage, the robot locates the curtain wall installation position by analyzing and processing the collected images of the assembly target.

7. A robotic curtain wall installation system based on assembly status perception, characterized in that, include: The information acquisition and preprocessing module is configured to: acquire time-series data of the robot's contact state during the assembly process in real time, and perform preprocessing. The current curtain wall contact status sensing module is configured to input preprocessed data into the assembly status sensing model to obtain the current curtain wall contact status. The robot pose adjustment module is configured to: determine whether the current curtain wall contact state is parallel; if not, map the current curtain wall contact state to obtain the corresponding robot pose adjustment action according to the contact state-robot action mapping relationship; execute the mapped robot pose adjustment action to adjust the curtain wall contact state until the current curtain wall contact state is parallel. The assembly attempt module is configured to: if the current curtain wall contact state is parallel, then perform an assembly attempt until the curtain wall installation is completed; include: Step A: After reaching the parallel state, record the current parallel contact state and the current contact pose as the initial contact pose. Use the coordinate system of the end tool of the robotic arm as the assembly coordinate system, record the initial reading on the Z-axis of the current assembly direction as d, and mark it as the position of the curtain wall contacting the outer surface of the hanger. When the curtain wall contacts the inner surface of the hanger, the absolute value of the difference between the reading on the Z-axis of the assembly direction and the initial reading will exceed the preset threshold. At this time, perform the engagement operation and record the initial number of attempts p=1. In this assembly coordinate system, the positive direction of the X-axis is to the left of the curtain wall, the positive direction of the Y-axis is downward of the curtain wall, and the positive direction of the Z-axis is forward of the curtain wall. Furthermore, the XOY plane of the assembly coordinate system is parallel to the curtain wall, and the Z-axis of the assembly coordinate system is perpendicular to the curtain wall. Step B: Control the robotic arm to move backward a first preset distance to ensure that there is no contact at present, and then move upward a second preset distance; Step C: Control the robotic arm to move forward along the Z-axis in the assembly direction with constant force. Make contact, record the current contact state, and record the current assembly direction Z-axis reading as d'. Determine whether the engagement is successful based on whether the absolute value of the difference between the current assembly direction Z-axis reading d' and the initial reading d exceeds the threshold. If the absolute value exceeds the threshold, the curtain wall is successfully engaged. Control the robotic arm to move 5mm along the positive Y-axis of the assembly direction to complete the installation. Step D: If the curtain wall does not engage and is currently in a parallel contact state, proceed to step B. Step E: If the engagement fails and the current curtain wall contact state changes from parallel to forward tilt, control the robotic arm to return to a new position p distance above the initial contact pose, jump to step B, and update the number of attempts: p = p + 1.

8. The robotic curtain wall installation system based on assembly state perception as described in claim 7, characterized in that, The contact state time series data includes: six-dimensional force information at the end of the robotic arm and joint angle information of the robotic arm during the curtain wall assembly contact process.

9. A robotic curtain wall installation system based on assembly state perception as described in claim 7, characterized in that, The assembly state perception model adopts a recurrent neural network structure, including an input layer, an output layer, and a hidden layer connected in sequence; The number of nodes in the input layer and hidden layer is set to The input of the input layer is Input of hidden layer , ,choose The function is activated, and the output is set to utilize the last hidden node. As input to the output layer, and after The function processes and calculates to obtain the output value. The output value y is the label value of the current curtain wall contact state.

Images