Double-arm cooperative masonry method and system

Abstract

According to a masonry task, a feasible path of a main arm and a feasible path of a slave arm are obtained, tasks of the main arm and the slave arm are subjected to labor division, the operation mode of one-hand grabbing and one-hand gluing of a mechanical arm is achieved, the two arms can efficiently cooperate in the respective task fields, and the work efficiency is improved. The control complexity is reduced, the system efficiency is improved, the cost of each group of executable pose actions in the multiple groups of executable pose actions is solved, the pose actions to be executed are selected according to the cost, inverse kinematics solving is conducted on each masonry point, and the pose actions to be executed are obtained. And operation cost calculation is performed on the solved pose action, and the greedy algorithm is combined to take the shortest execution time as an optimization target, so that the execution efficiency of the action path of the mechanical arm can be effectively improved, the total task duration is remarkably reduced, the beat control of engineering construction is facilitated, the problem of too tight coupling of a cooperation mode of two arms is solved, and the operation efficiency is high.

Description

Technical Field

[0001] The invention belongs to the technical field of mechanical arm operation, and relates to a double-arm collaborative masonry method and system. Background Art

[0002] In traditional manual masonry processes, workers typically use both hands for different operations: one hand is responsible for grasping and placing masonry pieces, while the other is responsible for applying adhesive. This manual method has certain flexibility and efficiency, but it still has many limitations in complex environments and large-scale operations, such as low work efficiency, high labor intensity, and difficulty in ensuring accuracy.

[0003] With the continuous advancement of robotics and automated equipment, automated masonry systems have been gradually proposed, aiming to improve construction efficiency, reduce manual intervention, and optimize construction quality. In some existing automated masonry systems, robotic arms are used to mimic human masonry movements. Typically, one robotic arm is responsible for grasping and placing masonry pieces, while another or the same robotic arm is responsible for applying adhesive. However, existing dual-arm systems still face many technical challenges in terms of collaborative modes, path planning, and collision detection, which limit their application and effectiveness in actual construction.

[0004] Defects and shortcomings of existing technology:

[0005] First, the dual-arm collaboration model is inefficient

[0006] Current dual-arm collaborative systems often use a tightly coupled or synchronized mode, requiring the two robotic arms to operate in strict synchronization. This results in low system efficiency when performing multiple tasks simultaneously and makes it difficult to cope with complex operating environments. A loosely coupled mode, on the other hand, allows the two robotic arms to work independently while simultaneously aiming at the same target, enabling more flexible response to diverse tasks within the masonry process. However, this aspect of the existing technology is relatively understudied.

[0007] Second, collision detection and safety issues

[0008] In dual-arm systems, the relative motion of the manipulators and the overlap of their operating spaces create a collision risk. Existing collision detection technologies mostly rely on simple static detection methods, which cannot reflect real-time changes in dynamic environments. However, the SAT algorithm proposed in this embodiment of the present invention can monitor collision risks in real time, significantly improving system safety and reducing potential failures and damage.

[0009] Third, task allocation and operational efficiency

[0010] In existing automated masonry systems, the intelligence and automation levels of task allocation are relatively low, and usually require human intervention or relatively simple task allocation strategies.

[0011] While the current dual-arm collaborative masonry system has achieved a certain degree of automation, it still suffers from issues such as overly tight coupling of the collaborative model, delayed collision detection, and low operational efficiency. These shortcomings limit the application of existing technologies in actual construction. Summary of the Invention

[0012] The purpose of the present invention is to solve the problems in the prior art that the collaborative mode of the two arms is too tightly coupled, the collision detection during the execution of the two arms is not real-time, and the operating efficiency is low. A dual-arm collaborative masonry method and system are provided, which reasonably distributes the masonry tasks to different robotic arms through the master-slave control mode, and optimizes the posture and motion of each masonry point in combination with the greedy algorithm, thereby achieving efficient automated task allocation and execution. The loosely coupled dual-arm collaborative mode proposed in the present invention, the posture and motion optimization method based on operation cost, and the real-time collision detection technology combined with the SAT algorithm all help to improve the system's collaborative efficiency, safety and operating accuracy, and provide a new technical solution for the realization of automated masonry tasks.

[0013] In order to achieve the above object, the present invention adopts the following technical solutions:

[0014] A double-arm collaborative masonry method comprises the following steps:

[0015] S1: According to the masonry task, obtain the main arm feasible path and the slave arm feasible path, sample the main arm feasible path and the slave arm feasible path, and obtain the main arm sampling point set and the slave arm sampling point set;

[0016] S2: Based on the master arm sampling point set and the slave arm sampling point set, solve multiple sets of executable pose actions for each sampling point;

[0017] S3: Obtain an operation cost model for double-arm masonry, solve the cost of each set of executable posture actions in multiple sets of executable posture actions, compare each set of costs, and determine the executable posture action corresponding to the minimum cost as the posture action to be executed;

[0018] S4: Based on the posture action to be executed, the collaborative masonry of the master arm and the slave arm begins.

[0019] A further improvement of the present invention is:

[0020] The S3 includes:

[0021] Assume that there are n sampling points in total, and the set of multiple executable pose actions of n sampling points is the initialization action sequence set;

[0022] Calculate the cost of each set of executable pose actions in the multiple sets of executable pose actions corresponding to the i-th sampling point in the initialization action sequence set, and obtain the cost set (C1, C2, C3, ... Cx ), where x represents the number of groups of executable pose actions corresponding to the sampling point;

[0023] Compare the costs of C1 and C2. If the cost of C2 is less than C3, compare the cost of C2 with the cost of C3, and repeat the comparison until the minimum cost is obtained. The posture action corresponding to the minimum cost is determined as the posture action to be executed.

[0024] In S3, the operation cost model of the two-arm masonry is expressed by the following formula:

[0025] C action =αC move +βC interaction +γC pose

[0026] Among them, C move is the moving distance cost between sampling points, C interaction is the interaction cost of the two arms in the shared workspace, C pose is the posture difference cost caused by different postures, and α, β, and γ are the weight coefficients between each cost item.

[0027] In S2 and S3, the moving distance cost C between the sampling points move It is expressed by the following formula:

[0028] C move =C move1 +C move2

[0029]

[0030] The interaction cost C of the two arms in the shared workspace interaction It is expressed by the following formula:

[0031] C interaction =Time(P conflict )

[0032] Where, P conflict represents the overlapping part of the paths, Time(·) represents the overlapping time;

[0033] The pose difference cost C caused by the different poses pose It is expressed by the following formula:

[0034]

[0035] Where k is a constant coefficient related to the velocity and path of the joint.

[0036] The S3 further includes introducing collision detection, which is used to detect the feasibility of the posture action to be executed.

[0037] In S3, collision detection includes:

[0038] S3.1: Obtain the spatial position information of the master arm and the slave arm;

[0039] S3.2: Set a safety distance threshold. Based on the spatial position information of the master arm and the slave arm, determine whether the distance between the master arm and the slave arm is within the safety distance threshold. If the distance between the master arm and the slave arm is greater than or equal to the safety distance threshold, detect the next sampling point. If the distance between the master arm and the slave arm is less than the safety distance threshold, execute S3.3.

[0040] S3.3: Use the SAT algorithm to detect the motion of the master arm and the slave arm. If a collision occurs, select the executable posture action with the second smallest cost based on the cost of each group of executable posture actions at the sampling point, and repeat this step to perform motion detection until the master arm and the slave arm do not collide, then detect the next sampling point.

[0041] Said S3.3 includes:

[0042] The cost of each group of executable posture actions at the sampling point is sorted from small to large. If the executable posture action corresponding to the minimum cost collides, the executable posture actions corresponding to the cost are selected in turn according to the sorting of costs from small to large for detection until the master arm and the slave arm do not collide. The posture action corresponding to the current cost is determined as the posture action to be executed.

[0043] A double-arm collaborative masonry system, comprising:

[0044] A path sampling module is used to obtain a master arm feasible path and a slave arm feasible path according to the masonry task, sample the master arm feasible path and the slave arm feasible path, and obtain a master arm sampling point set and a slave arm sampling point set;

[0045] A pose solving module is used to solve multiple sets of executable pose actions of the sampling points based on the master arm sampling point set and the slave arm sampling point set;

[0046] The posture optimization module is used to obtain the operation cost model of the double-arm masonry, solve the cost of each group of executable posture actions in multiple groups of executable posture actions, compare each group of costs, and determine the executable posture action corresponding to the minimum cost as the posture action to be executed;

[0047] The masonry module is used to start the collaborative masonry of the master arm and the slave arm based on the posture action to be executed.

[0048] A terminal device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the steps of any one of the methods of the present invention when executing the computer program.

[0049] A computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of any method described in the present invention.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] The present invention discloses a dual-arm collaborative masonry method. According to the masonry task, the feasible path of the main arm and the feasible path of the slave arm are obtained, the tasks of the main arm and the slave arm are divided, and the "one-hand grasping, one-hand gluing" operation mode of the robot arm is realized. The dual arms can efficiently collaborate in their respective task areas, reducing the control complexity and improving the system efficiency. The cost of each group of executable posture actions in multiple groups of executable posture actions is solved, and the posture action to be executed is selected according to the cost. By performing inverse kinematics solution on each masonry point, the operation cost of the solved posture action is calculated and combined with the greedy algorithm to take the shortest execution time as the optimization goal, the execution efficiency of the robot arm motion path can be effectively improved, the total task time is significantly reduced, which is beneficial to the rhythm control of engineering construction, overcomes the problem of overly tight coupling of the dual-arm collaboration mode, and has high operation efficiency.

[0052] Furthermore, the present invention takes into account the potential collision risk between the two arms when operating in a shared space, and introduces collision detection. Collision detection is used to detect the feasibility of the posture action to be executed, and can realize real-time dynamic collision judgment and avoidance, effectively improving the operational safety and reliability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0054] Figure 1 is a flow chart of a method according to an embodiment of the present invention;

[0055] Figure 2 Schematic diagram of masonry tasks and robot arm position conversion according to an embodiment of the present invention;

[0056] Figure 3 This is a sampling diagram of masonry action points according to an embodiment of the present invention;

[0057] Figure 4 This is a flowchart of the posture motion optimization of a robotic arm based on operation cost according to an embodiment of the present invention;

[0058] Figure 5 Schematic diagram of the separation axis and separation plane according to an embodiment of the present invention;

[0059] Figure 6 Collision detection under the perspective of a depth camera according to an embodiment of the present invention;

[0060] Figure 7 This is an action sequence of the masonry method before the improvement of the embodiment of the present invention (a is the main arm sucking masonry adhesive from the blue disc, and the main arm is in the waiting working state 1; b is the main arm sucking masonry adhesive from the blue disc, and the main arm is in the waiting working state 2; c is the main arm sucking masonry adhesive from the blue disc, and the main arm is in the waiting working state 3; d is the slave arm completing the masonry adhesive application from the initial point to the masonry point 1, the main arm grabs the masonry from the conveyor belt and continues to wait for the slave arm to apply the masonry adhesive in state 1; e is the slave arm completing the masonry adhesive application from the initial point to the masonry point 1, The main arm grabs the masonry from the conveyor belt and continues to wait for the slave arm to apply the adhesive (state 2); f is the slave arm completing the application of masonry adhesive from the initial point to masonry point 1, the main arm grabs the masonry from the conveyor belt and continues to wait for the slave arm to apply the adhesive (state 3); g is the main arm placing the masonry at masonry point 1, and after the slave arm completes the application, it returns to the initial point and continues to wait (state 1); h is the main arm placing the masonry at masonry point 1, and after the slave arm completes the application, it returns to the initial point and continues to wait (state 2); i is the main arm placing the masonry at masonry point 1, and after the slave arm completes the application, it returns to the initial point and continues to wait (state 3);

[0061] Figure 8The improved method of the embodiment of the present invention is a masonry process after the masonry is finished. (a is when the slave arm performs the adhesive application operation on the masonry point 1, the main arm completes the grabbing of the masonry and moves to the public workspace to wait for the slave arm to complete the application state one; b is when the slave arm performs the adhesive application operation on the masonry point 1, the main arm completes the grabbing of the masonry and moves to the public workspace to wait for the slave arm to complete the application state two; c is when the slave arm performs the adhesive application operation on the masonry point 1, the main arm completes the grabbing of the masonry and moves to the public workspace to wait for the slave arm to complete the application state three; d is when the slave arm performs the adhesive application operation on the masonry point 1, the main arm completes the grabbing of the masonry and moves to the public workspace to wait for the slave arm to complete the application state four; e is when the slave arm completes the application and moves away from the masonry point 1, the main arm moves toward the masonry point 1 in state one; f is when the slave arm completes the application and moves away from the masonry point 1, the main arm moves toward the masonry point 1 in state one. g is the movement state 3 when the slave arm completes the smearing and moves away from the masonry point 1, and the main arm moves toward the masonry point 1; h is the movement state 4 when the slave arm completes the smearing and moves away from the masonry point 1, and the main arm moves toward the masonry point 1; i is when the main arm is placing the masonry, the slave arm has begun to prepare for the smearing operation of the next piece of masonry, and the main arm immediately performs the work state 1 of grabbing and placing the next piece of masonry; j is when the main arm is placing the masonry, the slave arm has begun to prepare for the smearing operation of the next piece of masonry, and the main arm immediately performs the work state 2 of grabbing and placing the next piece of masonry; k is when the main arm is placing the masonry, the slave arm has begun to prepare for the smearing operation of the next piece of masonry, and the main arm immediately performs the work state 3 of grabbing and placing the next piece of masonry; l is when the main arm is placing the masonry, the slave arm has begun to prepare for the smearing operation of the next piece of masonry, and the main arm immediately performs the work state 4 of grabbing and placing the next piece of masonry);

[0062] Figure 9 The figure shows the comparison of the construction time of a single masonry block before and after the improvement of the embodiment of the present invention. DETAILED DESCRIPTION

[0063] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0064] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort shall fall within the scope of protection of the present invention.

[0065] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0066] In the description of the embodiments of the present invention, it should be noted that if the terms "upper," "lower," "horizontal," "inner," etc. appear, the orientation or positional relationship indicated is based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship in which the inventive product is typically placed when in use. These terms are merely for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or component referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limitations on the present invention. In addition, the terms "first," "second," etc. are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0067] In addition, if the term "horizontal" appears, it does not mean that the component must be absolutely horizontal, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", and does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0068] In the description of the embodiments of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "disposed," "installed," "connected," and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to internal connections between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0069] The present invention is described in further detail below with reference to the accompanying drawings:

[0070] See also Figure 1The embodiment of the present invention discloses a dual-arm collaborative masonry method. When manually laying bricks, one arm is used to complete the grasping and placement of bricks, while the other arm completes the application of adhesive to the bricks. The dual-arm system of the embodiment of the present invention imitates this masonry process and designs the robotic arms in a master-slave control mode. The master arm completes the grasping and placement of bricks, while the slave arm completes the application of adhesive to the bricks. The master arm's masonry grasping action is a fixed sequence. In combination with the characteristics of masonry work, the dual-arm collaboration of the embodiment of the present invention adopts a loosely coupled mode, that is, sequential operations are performed on the same target point in a shared workspace to complete a specific task. In this mode, the two arms perform different tasks, have different starting points and the same target point, and there is relative motion at the end of the manipulator. First, based on the masonry task and basic information of the masonry, the set of masonry points for the masonry task is determined; secondly, for each masonry point, the path planning algorithm is used to plan the manipulator path and perform periodic sampling to generate a set of sampling points for the masonry points; then, multiple sets of inverse solutions for the manipulator at each sampling point are calculated through inverse kinematics, the modeling operation cost is modeled, and a greedy algorithm is used to optimize the solution with the shortest execution time as the goal, obtaining the optimal posture motion of each sampling point in the masonry point, and executing the masonry task corresponding to the masonry point accordingly. During the posture motion optimization process, considering that there is still a probability of collision between the two arms, the SAT algorithm is introduced for real-time monitoring of collision detection to improve system safety. The specific steps include:

[0071] Step 1: For the masonry task, first calculate the masonry point set of K masonry points through geometric relationship calculation. Use the spatial position relationship transformation to obtain the masonry point coordinate set for each task of the master and slave arms, and at the same time, clarify the execution order of each masonry point. Let i = 0, where i represents the masonry point.

[0072] Step 2: i = i + 1. Initialize the i-th masonry point, and return the master and slave arms to their respective starting positions.

[0073] Step 3: For the masonry task of the i-th masonry point, combined with the constraints such as the dual-arm workspace and kinematic limitations, a path planning algorithm is used to plan the paths of the two arms separately to obtain the feasible paths of the master and slave arms for the i-th masonry point.

[0074] Step 4: Periodically sample the paths of the master and slave arms to obtain the sampling point sets of the master and slave arms at the current masonry point.

[0075] Step 5: Sequentially take out the sampling points, and use the operation cost-based robot arm posture motion optimization method to determine the optimal posture motion of the master and slave arms, and the two arms execute accordingly.

[0076] Step 6: Determine whether the current masonry point is the last point in the masonry point set. If not (i≠K), jump to step 2 and enter the action optimization solution and masonry task execution for the next masonry point; if (i=K), go to step 7 and all masonry tasks are completed.

[0077] Step 7: Output action execution data and end.

[0078] Furthermore, in step 1, obtaining the set of masonry points includes the following steps:

[0079] In the embodiment of the present invention, the length of one side L weight =990mm, height L height = 135mm target wall masonry is the task, such as Figure 2 As shown, combined with the "Quanshun" masonry process, the main arm is set on the left side of the masonry task area, the slave arm is set on the right side of the masonry task area, and the wall masonry position is located in the public working space of the two arms.

[0080] Combined with the geometric position relationship in space, such as Figure 2 As shown, calculate the coordinates of the geometric center point of each masonry piece, that is, obtain the masonry point set of the masonry task. The specific steps are as follows:

[0081] Step 1.1: Taking the world coordinate system as the reference coordinate system, the basic information L of the wall masonry task is known weight , L height 、P start 、P end , use formula (1) to calculate the required number of masonry:

[0082]

[0083] Where N FULL The number of masonry units required for the masonry task, N HALF The number of masonry (half bricks) that need to be cut for the masonry task, d is the mortar joint width, l, w, h are the length, width and height specifications of the masonry respectively.

[0084] Step 1.2: The process of generating the geometric center point of the masonry is divided into multiple cases:

[0085] First, we need to distinguish between odd-numbered and even-numbered layers;

[0086] Secondly, considering the staggered laying method of the masonry, it is assumed that the first and last masonry blocks of the even-numbered layers are half bricks.

[0087] The geometric center point of the first masonry is known to be P start In the case of (x1, y1, z1), the coordinates of the geometric center point of each masonry (world coordinate system) are calculated by formula (2).

[0088]

[0089] In the formula, i represents the number of masonry blocks in each layer, j represents the number of layers, the first formula is the calculation method for odd-numbered layers, the second formula is the calculation method for the first half brick of even-numbered layers, and the third formula is the calculation method for the remaining masonry blocks of even-numbered layers.

[0090] Step 1.3: Convert the world coordinate system to the robot coordinate system, and use formula (3) to calculate the rotation angle C of the masonry task point (the geometric center point of the masonry) in the robot coordinate system.

[0091] Where, P start (x1, y1, z1) is the coordinate of the geometric center point of the first block, and P end (x end ,y end ,z end ) The coordinates of the geometric center point corresponding to the last piece of masonry.

[0092]

[0093] Step 1.4: Based on the coordinates of the geometric center points of the masonry calculated in step 1.2, the rotation angle calculated in step 1.3, and the geometric relationship between the coordinate systems, calculate the coordinates of the geometric center points of each masonry in the robot arm coordinate system (the master arm calls it the placement point, and the slave arm calls it the smear point).

[0094] by Figure 2 Taking the wall construction task shown above as an example, according to the calculation steps above, it can be calculated that 11 whole bricks and 2 half bricks are required to complete this wall construction task. The set of masonry points expressed in the robot arm coordinate system is shown in Table 1.

[0095] Table 1 Set of masonry points for masonry tasks

[0096]

[0097] Note: To simplify the masonry process, the main arm grabbing point (the position where the main arm takes bricks) is set to a fixed position. The coordinate units in the table are all centimeters.

[0098] A dual-arm collaborative structure that mimics manual operation, with clear division of labor and efficient task collaboration: Based on the typical operation mode of "one hand grasping and the other hand applying glue" in manual masonry, the present invention designs a master-slave controlled dual-arm system, which divides the grasping and placement tasks and the adhesive application tasks, allowing the dual arms to efficiently collaborate within their respective task areas, reducing control complexity and improving system efficiency.

[0099] Furthermore, the path planning in step 3 and the periodic sampling in step 4 include the following steps:

[0100] After obtaining the set of masonry points (main arm placement points and slave arm smearing points), the paths of the two arms are planned separately in combination with the path planning algorithm to obtain their respective feasible paths.

[0101] The robot arm's movement speed was set to 30%, and the feasible path of the two arms was sampled and decomposed with a sampling period of 3 seconds. Each masonry point obtained two sampling point sets, one for the master arm and one for the slave arm.

[0102] Taking the masonry target point 1 in Table 1 as an example, the sampling decomposition results are as follows: Figure 3 As shown, the blue and red point sets are the sampling points of the master arm path (blue represents the masonry grabbing process, and red represents the masonry placement process), and the orange and yellow point sets are the sampling points of the slave arm path.

[0103] Furthermore, in step 5, the method for optimizing the posture motion of the manipulator based on the operation cost includes the following steps:

[0104] For each masonry point, after obtaining the set of robotic arm sampling points, it is necessary to determine the final execution posture of the robotic arm at each sampling point, and then drive the robotic arm to move to complete the masonry task corresponding to the masonry point. Different sampling point posture and action combinations will result in different robotic arm dual-arm collaboration effects. This embodiment proposes a method for optimizing the posture and action of the robotic arm based on operation cost. The specific implementation steps are as follows:

[0105] Step 5.1: Sequentially select one sampling point from each of the master arm and slave arm sampling point sets.

[0106] Step 5.2: Use inverse kinematics analysis to determine multiple sets of inverse solutions corresponding to the sampling points, i.e., multiple sets of executable poses for the robotic arm. Generally speaking, each sampling point corresponds to eight sets of inverse solutions, i.e., eight sets of poses for each robotic arm.

[0107] Step 5.3: Using execution time as the core modeling operation cost, a greedy algorithm is used to optimize the inverse solution combination obtained in Step 5.2. This results in the optimal posture motion for each of the master and slave arms at the current sampling point, namely the optimal execution pose for the master arm and the optimal execution pose for the slave arm. Both arms execute accordingly, and collision detection is performed using the SAT algorithm during execution to ensure safety.

[0108] Step 5.4: Determine whether all sampling points in the sampling set have been optimized. If yes, proceed to step 5.5; if not, skip to step 5.1.

[0109] Step 5.5: The optimization of the sampling point set is completed, that is, the optimization of the current masonry point is completed.

[0110] Assume that the master arm sampling point set has a total of M sampling points and the slave arm sampling point set has a total of N sampling points. Take a pair of sampling points taken from the master arm and slave arm sampling point sets as an example to describe the pose motion optimization process. Figure 4 shown.

[0111] Operation cost refers to the average cost of completing each operation under certain circumstances. Operation cost optimization refers to reducing the cost of task operations by adjusting the task execution strategy when performing a task or process.

[60] When applied specifically to the field of robotics, operation costs usually fall into three categories.

[0112] (1) Time cost: The time required to perform a task is usually the most important type of cost, especially in scenarios where tasks need to be completed in real time.

[0113] (2) Energy consumption cost: The energy consumption of the robotic arm during task execution, especially in long-term tasks, may significantly affect the system efficiency.

[0114] (3) Distance cost: refers to the distance the robot moves from its current position to the target task point. A longer path usually means higher time and energy consumption costs.

[0115] Based on the actual execution of a task, the operational cost is typically characterized by a quantitative description of the robot's movement path, joint motion, and other aspects. For example, the time cost can be estimated by calculating the path length from the robot's current position to the target position and converting this into time based on the robot's speed. The energy cost can be calculated based on the robot's motion characteristics and joint loads.

[0116] Furthermore, operation cost optimization includes:

[0117] In dual-arm collaboration, the primary goal of optimizing operation costs is to improve task execution efficiency. Furthermore, dual-arm collaboration involves not only the individual movements of the robotic arms but also the coordination between them, particularly their interaction within the shared workspace. Therefore, cost optimization in dual-arm collaborative masonry construction in this embodiment of the present invention not only aims to reduce the execution time of each robotic arm but also considers the interaction and collision avoidance between the two arms within the workspace.

[0118] Operation cost optimization mainly includes two steps: modeling and solving. The following will explain in detail how to model and optimize the operation cost in the dual-arm collaborative masonry application.

[0119] (1) Operation cost modeling

[0120] To model the operation cost, we first need to define a cost function to describe the cost of each action. This chapter decomposes the operation cost into three parts based on the execution characteristics of the masonry task:

[0121] a. Robot arm movement distance cost: The cost of the robot arm moving distance from one sampling point to the next sampling point, usually calculated using the actual length of the path.

[0122] b. Dual-arm interaction cost: The two arms may interact in a shared workspace, resulting in a collaboration cost.

[0123] c. Posture difference cost: The cost caused by the time difference required to calculate the change from the current posture of the robot arm to the target posture.

[0124] Based on the above three parts, the operation cost function of two-arm masonry is defined as:

[0125] C action =αC move +βC interaction +γC pose (4)

[0126] Where C move is the moving distance cost between sampling points, C interaction is the interaction cost of the two arms in the shared workspace, C pose is the posture difference cost caused by different postures, and α, β, and γ are the weight coefficients between each cost item.

[0127] For the masonry task proposed in the embodiment of the present invention, the operation cost should consider the execution time of the master arm and the slave arm from the last sampling point to the current sampling point respectively, which includes the change of posture, the choice of path, and the interaction cost when the two arms collaborate.

[0128] The embodiment of the present invention calculates the moving distance cost C between sampling points by Euclidean distance move . Assume that the current position of the main arm end effector is (x current1 ,y current1 , z current1 ), the target position is (x target1 ,y target1 , z target1 ), the current position of the slave arm end effector is (x current2 ,y current2 , z current2 ), target position (x target2 ,y target2 , z target2 ) The current master-slave arm distance cost can be calculated as:

[0129] C move =C move1 +C move2

[0130]

[0131] The two robotic arms may interact or collide in the shared workspace, so the interaction cost of the two arms in the shared workspace is C interaction The interaction time of the two manipulators in the space needs to be calculated, which is usually determined by the overlap between the manipulator paths and the target task. The greater the intersection of the paths of the two manipulators, the higher the possibility of collision, and the longer the interaction time to avoid collision. interaction The interaction time of the manipulator in the shared space is calculated by combining the manipulator envelope and the sampling point distance threshold to perform intersection calculation, as shown in formula (6).

[0132] C interaction =Time(P conflict ) (6)

[0133] Where, P conflict represents the path overlap. If the distance between the sampling points of the two-arm path is less than a given threshold, it is considered that there is path overlap between the two sampling points. Time(·) represents the overlap time. Assuming that the robot moves at a constant speed v, the overlap time can be calculated by the length of the path overlap.

[0134] The pose of a robotic arm refers to the spatial position and posture of its end effector. The pose difference refers to the difference between the current pose and the target pose, and the pose change is usually represented by the difference in joint angles. Since the pose difference between the two sampling points is mainly θ angle (joint angle difference), let the corresponding poses of the two sampling points be (x current1 ,y current1 , z current1， θ1, θ2, θ3) and (x current2 ,y current2 , z current2， θ'1, θ'2, θ'3), therefore, the pose difference cost C caused by different poses pose Expressed as:

[0135]

[0136] Where k is a constant coefficient related to the speed and path of the joint, which is determined by the maximum joint speed of the robot arm, the complexity of the path, etc.

[0137] (2) Operation cost optimization solution

[0138] The embodiment of the present invention adopts a greedy algorithm to perform cost optimization, and the specific steps are as follows.

[0139] Step 1: Assume that there are n sampling points in total. Randomly initialize the robot arm posture action selection of the i-th sample in the action sequence set, where i = 1. Calculate the current operation cost, including the movement distance cost between sampling points, the interaction cost of the two arms in the shared workspace, and the posture difference cost caused by different postures.

[0140] Step 2: Assume that the costs corresponding to the eight solutions for action i are (C1, C2, C3, ... C8). Compare the costs corresponding to C1 and C2. If the cost of C2 is smaller, choose C2 and continue comparing it with C3. Repeat this process until the solution with the smallest cost is found.

[0141] Step 3: After all n sampling points are calculated, the robot arm pose selection for all sampling points is obtained and the optimization is completed.

[0142] Inverse kinematics is used to calculate the operation cost of the action corresponding to each posture and combined with a greedy algorithm to optimize the execution efficiency: By solving the inverse kinematics of each masonry point, calculating the operation cost of the solved posture action and combining it with a greedy algorithm to take the shortest execution time as the optimization goal, it can effectively improve the execution efficiency of the robot arm's motion path, significantly reduce the total task duration, and facilitate the rhythm control of engineering construction.

[0143] Furthermore, the collision detection in this embodiment includes:

[0144] During the motion sequence optimization process, considering the optimization cost of the interaction between the two arms (the working distance of the two arms in the common workspace at the same time), there is still a probability of collision. The embodiment of the present invention combines the SAT collision detection algorithm to monitor the masonry process in real time. The SAT algorithm determines whether a collision occurs based on the projections of the objects in various directions. Only when the projections of the two objects in all directions do not overlap, it is determined that there is no collision. In other words, if the projections of the two objects in any direction overlap, it is considered that a collision has occurred.

[0145] like Figure 5 As shown in the figure, for two separated convex polygons A and B, if a projection axis is found such that the projected line segments of the two convex polygons on this axis do not overlap, then the two polygons will not collide. This projection axis is called the separating axis. The separating plane is the plane where the faces of the two objects intersect. The plane generated by the intersection of the face normal vectors is the potential separating plane. The separating plane determines whether the two objects can be separated. If their projections on this plane overlap, a collision occurs.

[0146] When the SAT algorithm detects a collision, it reselects the inverse solution and combines it with the operation cost optimization. The specific steps are:

[0147] Step 1: Use the images collected by the masonry system camera (the eye is installed outside the hand) to detect the working area of ​​the two arms in real time and obtain the spatial position information of the two robotic arms. The eye is installed outside the hand means that the camera is installed on the outside of the robotic arm.

[0148] Step 2: Set the safety distance threshold to 150 mm (based on the actual requirements of the task and the size of the robotic arm). If the distance between the master arm and the slave arm is greater than or equal to the safety threshold, determine the next sampling point; if it is less than the safety threshold, jump to step 3.

[0149] Step 3: Use the SAT algorithm to perform collision detection on the two robotic arms. If a collision occurs, a new set of inverse solutions is selected for that sampling point for optimization. Specifically, the costs of each set of executable poses for the sampling point are sorted from small to large. If a collision occurs between the executable poses corresponding to the minimum cost, the executable poses corresponding to the costs are selected in order from small to large. This is done until the master and slave arms are free of collision. The pose corresponding to the current cost is then determined as the pose to be executed.

[0150] If no collision occurs, determine the next sampling point.

[0151] Step 4: After all sampling points have been judged and the system determines that there is no collision risk, the task is executed and the optimization ends.

[0152] like Figure 6 As shown in the red detection box in the above figure, we can see that there may be collisions between the two arms during the masonry process. However, through the real-time monitoring and feedback of the collision situation by the camera, and the real-time control feedback of the robotic arm using the collision detection algorithm, the occurrence of collisions during the masonry process is effectively avoided.

[0153] The fusion collision detection algorithm (SAT) performs real-time safety monitoring to ensure operational safety: Considering the potential collision risk between the two arms when operating in a shared space, the present invention introduces a collision detection algorithm based on the separating axis theorem (SAT), which can realize real-time dynamic collision judgment and avoidance, effectively improving the operational safety and reliability of the system.

[0154] Simulation experiment of double-arm collaborative masonry method:

[0155] In order to verify the effectiveness of the masonry method after introducing operation cost optimization, a comparative experiment of the masonry methods before and after improvement was carried out, and the experimental results were analyzed.

[0156] Masonry has time and space avoidance, that is, at the same time, only one robotic arm is moving in the public workspace, and the continuous masonry actions of this process are sampled at intervals of 3s.

[0157] like Figure 7As shown in the figure, since the two arms do not join any collaborative method at this time, the masonry process at this time is that the two robotic arms perform tasks in chronological order. After the main arm completes the grasping and placement process of the corresponding masonry according to the masonry target point, the slave arm performs the adhesive application work.

[0158] exist Figure 7 In the example, the two arms perform tasks in chronological order, resulting in low collaborative efficiency. Figures (a)-(c) show the main arm sucking masonry adhesive from the blue disc, while the main arm is waiting for work; Figures (d)-(f) show the slave arm completing the application of masonry adhesive from the initial point to masonry point 1, while the main arm grabs the masonry from the conveyor belt and continues to wait for the slave arm to apply it. At this time, although the two arms move simultaneously, the collaborative nature of the two-arm masonry is still not reflected; Figures (g)-(i) show the main arm placing the masonry at masonry point 1, while the slave arm completes the application and returns to the initial point to continue waiting. From the masonry method before the improvement, it can be seen that the dual-arm system at this time lacks dual-arm collaboration, resulting in low masonry efficiency and waste of time and energy.

[0159] Masonry method after operation cost optimization:

[0160] The masonry simulation results based on the improved masonry method are as follows Figure 8 As shown in the figure, the two arms complete the tasks of grasping and placing masonry and applying adhesive, respectively. This figure demonstrates the effectiveness of dual-arm collaboration after introducing operation cost optimization. The following section will analyze the optimization performance of each stage in detail.

[0161] exist Figure 8 In the figure, (a)-(d) show that while the slave arm is applying adhesive to masonry point 1, the master arm has finished grabbing the masonry and moved to the common workspace to wait for the slave arm to finish applying adhesive. Figures (e)-(h) show that while the slave arm has finished applying adhesive and is moving away from masonry point 1, the master arm is moving toward masonry point 1, demonstrating the system's dual-arm collaboration capabilities. Figures (i)-(l) show that while the master arm is placing a masonry piece, the slave arm has already begun preparing to apply adhesive to the next piece, and the master arm immediately grabs and places the next piece. This reflects the coherence of the masonry system.

[0162] The above process demonstrates the three main advantages of the masonry method after operation cost optimization: first, it significantly shortens the interval between task executions. By introducing operation cost optimization, the coordination of the two arms is significantly improved during the optimization and execution of each set of sampling points, avoiding unnecessary delays and improving overall masonry efficiency. Second, the coordination between the master and slave arms is significantly improved, and the two coordinate more efficiently within the same workspace. Finally, masonry continuity is increased, with standby redundancy time between masonry points almost zero. This is because the action optimization process executes each action one by one, and the masonry points show continuity, and the corresponding action optimization of the sampling points within the masonry point is also consistent. Considering the significant reduction in operation costs within the common work area, especially the reduction in time cost, the improved masonry efficiency of a single block is greatly improved.

[0163] Experimental data analysis:

[0164] For the simulation of a single block of masonry, this embodiment of the present invention sets the robotic arm's operating speed to 30%, and defines the time from the main arm completing the masonry grabbing process to the start of grabbing the second block as the masonry laying time for a single block of masonry. Table 2 compares the masonry laying time for a single block of masonry before and after the improved algorithm. The experimental results in the table show that compared with the pre-improvement masonry method, the improved masonry method significantly improves the efficiency of single-block masonry, with the highest relative improvement reaching 28.6%. For the masonry tasks proposed in this chapter, the average masonry efficiency increased by 25%.

[0165] Table 2 Comparison of simulation laying time of single masonry

[0166]

[0167]

[0168] from Figure 9 As can be seen from the bar chart shown, the improved method shows time optimization in the construction of each block of masonry, further proving that after introducing operation cost optimization, the dual-arm collaborative masonry method can significantly improve masonry efficiency and reduce execution time.

[0169] The embodiment of the present invention discloses a dual-arm collaborative masonry system, comprising:

[0170] A path sampling module is used to obtain a master arm feasible path and a slave arm feasible path according to the masonry task, sample the master arm feasible path and the slave arm feasible path, and obtain a master arm sampling point set and a slave arm sampling point set;

[0171] A pose solving module is used to solve multiple sets of executable pose actions of the sampling points based on the master arm sampling point set and the slave arm sampling point set;

[0172] The posture optimization module is used to obtain the operation cost model of the double-arm masonry, solve the cost of each group of executable posture actions in multiple groups of executable posture actions, compare each group of costs, and determine the executable posture action corresponding to the minimum cost as the posture action to be executed;

[0173] The masonry module is used to start the collaborative masonry of the master arm and the slave arm based on the posture action to be executed.

[0174] A schematic diagram of a terminal device provided in one embodiment of the present invention. The terminal device in this embodiment includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of each of the aforementioned method embodiments are implemented. Alternatively, when the processor executes the computer program, the functions of each module / unit in each of the aforementioned device embodiments are implemented.

[0175] The computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to accomplish the present invention.

[0176] The terminal device may be a computing device such as a desktop computer, a notebook computer, a PDA, a cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0177] The processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0178] The memory may be used to store the computer programs and / or modules, and the processor implements various functions of the terminal device by running or executing the computer programs and / or modules stored in the memory and calling the data stored in the memory.

[0179] If the module / unit integrated in the terminal device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, and can also be completed by a computer program to instruct the relevant hardware. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, it can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0180] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A double-arm collaborative masonry method, characterized in that: The following steps are involved: S1: According to the masonry task, obtain the main arm feasible path and the slave arm feasible path, sample the main arm feasible path and the slave arm feasible path, and obtain the main arm sampling point set and the slave arm sampling point set; S2: Based on the master arm sampling point set and the slave arm sampling point set, solve multiple sets of executable pose actions for each sampling point; S3: Obtain an operation cost model for double-arm masonry, solve the cost of each set of executable posture actions in multiple sets of executable posture actions, compare each set of costs, and determine the executable posture action corresponding to the minimum cost as the posture action to be executed; S4: Based on the posture action to be executed, the collaborative masonry of the master arm and the slave arm begins.

2. A double-arm collaborative masonry method according to claim 1, characterized in that: The S3 includes: Assume that there are n sampling points in total, and the set of multiple executable pose actions of n sampling points is the initialization action sequence set; Calculate the cost of each set of executable pose actions in the multiple sets of executable pose actions corresponding to the i-th sampling point in the initialization action sequence set, and obtain the cost set (C1, C2, C3, ... C x ), where x represents the number of groups of executable pose actions corresponding to the sampling point; Compare the costs of C1 and C2. If the cost of C2 is less than C3, compare the cost of C2 with the cost of C3, and repeat the comparison until the minimum cost is obtained. The posture action corresponding to the minimum cost is determined as the posture action to be executed.

3. A double-arm collaborative masonry method according to claim 1, characterized in that: In S3, the operation cost model of the two-arm masonry is expressed by the following formula: C action =αC move +βC interaction +γC pose Among them, C move is the moving distance cost between sampling points, C interaction is the interaction cost of the two arms in the shared workspace, C pose is the posture difference cost caused by different postures, and α, β, and γ are the weight coefficients between each cost item.

4. A double-arm collaborative masonry method according to claim 3, characterized in that: In S2 and S3, the moving distance cost C between the sampling points move It is expressed by the following formula: C move =C move1 +C move2 The interaction cost C of the two arms in the shared workspace interaction It is expressed by the following formula: C interaction =Time(P conflict ) Where, P conflict represents the overlapping part of the paths, Time(·) represents the overlapping time; The pose difference cost C caused by the different poses pose It is expressed by the following formula: Where k is a constant coefficient related to the velocity and path of the joint.

5. A double-arm collaborative masonry method according to claim 1, characterized in that: The S3 further includes introducing collision detection, which is used to detect the feasibility of the posture action to be executed.

6. A double-arm collaborative masonry method according to claim 5, characterized in that: In S3, collision detection includes: S3.1: Obtain the spatial position information of the master arm and the slave arm; S3.2: Set a safety distance threshold. Based on the spatial position information of the master arm and the slave arm, determine whether the distance between the master arm and the slave arm is within the safety distance threshold. If the distance between the master arm and the slave arm is greater than or equal to the safety distance threshold, detect the next sampling point. If the distance between the master arm and the slave arm is less than the safety distance threshold, execute S3.

3. S3.3: Use the SAT algorithm to detect the motion of the master arm and the slave arm. If a collision occurs, select the executable posture action with the second smallest cost based on the cost of each group of executable posture actions at the sampling point, and repeat this step to perform motion detection until the master arm and the slave arm do not collide, then detect the next sampling point.

7. A double-arm collaborative masonry method according to claim 1, characterized in that: Said S3.3 includes: The cost of each group of executable posture actions at the sampling point is sorted from small to large. If the executable posture action corresponding to the minimum cost collides, the executable posture actions corresponding to the cost are selected in turn according to the sorting of costs from small to large for detection until the master arm and the slave arm do not collide. The posture action corresponding to the current cost is determined as the posture action to be executed.

8. A double-arm collaborative masonry system, characterized in that: include: A path sampling module is used to obtain a master arm feasible path and a slave arm feasible path according to the masonry task, sample the master arm feasible path and the slave arm feasible path, and obtain a master arm sampling point set and a slave arm sampling point set; A pose solving module is used to solve multiple sets of executable pose actions of the sampling points based on the master arm sampling point set and the slave arm sampling point set; The posture optimization module is used to obtain the operation cost model of the double-arm masonry, solve the cost of each group of executable posture actions in multiple groups of executable posture actions, compare each group of costs, and determine the executable posture action corresponding to the minimum cost as the posture action to be executed; The masonry module is used to start the collaborative masonry of the master arm and the slave arm based on the posture action to be executed.

9. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images