A curtain wall installation control system and method for a robotic arm based on path planning

Abstract

This application is applicable to the field of manipulator control technology, and provides a curtain wall installation control system and method for a robotic arm based on path planning. The system includes: a robotic cart, a robotic arm, an attitude sensor, a vision sensor, and a controller; the attitude sensor acquires the initial position information and initial attitude information of the robotic arm at the starting point of the operation task and sends them to the controller; the vision sensor acquires the environmental image of the robotic arm at the starting point of the operation task and sends it to the controller; the controller determines the initial planned path of the robotic arm according to the initial position information, initial attitude information, and a preset teaching trajectory library, calculates the similarity between the environmental image and the taught environmental image, and determines the target planned path of the robotic arm according to the similarity and the initial planned path; after grasping the glass curtain wall, the robotic arm moves the glass curtain wall to the end point of the operation task near the curtain wall installation frame according to the target planned path. This application can improve the versatility and adaptability of the curtain wall installation control system.

Description

Technical Field

[0001] This application relates to the field of manipulator control, and particularly to a curtain wall installation control system and method for a robotic arm based on path planning. Background Art

[0002] With the continuous development of modern architectural design, glass curtain walls have been widely used due to their advantages such as being lightweight, beautiful, not easily polluted, safe and reliable, and easy to maintain.

[0003] Currently, during the construction and installation of glass curtain walls, manual labor is mainly relied on. And since curtain walls generally have large sizes and weights, usually multiple workers need to cooperate. This not only results in low installation efficiency and high labor intensity, but also poses a high safety risk during high-altitude operations.

[0004] To solve the problems during curtain wall installation, it is usually considered to use curtain wall installation robots instead of manual labor. However, the inventors found that: due to the relatively complex process of installing curtain walls and the large differences in the installation trajectories of curtain walls at different positions in the physical space, when using curtain wall installation robots, for different working scenarios, it is usually necessary to perform targeted teaching on the robotic arm of the curtain wall installation robot, so that the path planned for the robotic arm after teaching meets the task requirements in the new working scenario. This leads to poor versatility and adaptability of curtain wall installation robots, thus affecting the popularization and application of curtain wall installation robots. Summary of the Invention

[0005] This application provides a curtain wall installation control system and method for a robotic arm based on path planning to solve the problem of poor versatility and adaptability of curtain wall installation robots.

[0006] In a first aspect, this application provides a curtain wall installation control system for a robotic arm based on path planning, including: a robot trolley, a robotic arm, an attitude sensor, a vision sensor, and a controller. The robotic arm is arranged on the robot trolley and is used to grab the glass curtain wall. The attitude sensor and the vision sensor are both arranged on the robotic arm. The robot trolley, the robotic arm, the attitude sensor, and the vision sensor are all communicatively connected to the controller;

[0007] The attitude sensor is used to obtain the initial position information and initial attitude information of the robotic arm at the starting point of the operation task and send them to the controller;

[0008] The vision sensor is used to obtain the environmental image of the robotic arm at the starting point of the operation task and send it to the controller;

[0009] The controller is configured to determine an initial planned path of the robotic arm based on the initial position information, the initial attitude information, and a preset teaching trajectory library, calculate the similarity between the environmental image and the teaching environmental image, and determine a target planned path of the robotic arm according to the similarity and the initial planned path;

[0010] The robotic arm is configured to move the glass curtain wall to the end point of the operation task close to the curtain wall installation frame according to the target planned path after grasping the glass curtain wall.

[0011] In a second aspect, the present application provides a curtain wall installation control method for a robotic arm based on path planning, which is applied to a controller in the curtain wall installation control system of the robotic arm based on path planning as described in the first aspect above. The control method includes:

[0012] Obtain the initial position information and initial attitude information of the robotic arm on the robot trolley at the starting point of the operation task, and obtain the environmental image where the robotic arm is located;

[0013] Determine an initial planned path of the robotic arm according to the initial position information, the initial attitude information, and a preset teaching trajectory library;

[0014] Calculate the similarity between the environmental image and the teaching environmental image;

[0015] Determine a target planned path of the robotic arm according to the similarity and the initial planned path, so as to move the glass curtain wall to the end point of the operation task close to the curtain wall installation frame according to the target planned path after the robotic arm grasps the glass curtain wall.

[0016] The present application provides a curtain wall installation control system and method for a robotic arm based on path planning. The system includes a robotic cart, a robotic arm, an attitude sensor, a vision sensor, and a controller. The robotic arm is arranged on the robotic cart and is used to grasp a glass curtain wall. The attitude sensor and the vision sensor are both arranged on the robotic arm. The robotic cart, the robotic arm, the attitude sensor, and the vision sensor are all communicatively connected to the controller. The attitude sensor is used to obtain the initial position information and initial attitude information of the robotic arm at the starting point of the operation task and send them to the controller. The vision sensor is used to obtain the environmental image of the robotic arm at the starting point of the operation task and send it to the controller. The controller is used to determine the initial planned path of the robotic arm according to the initial position information, the initial attitude information, and a preset teaching trajectory library, calculate the similarity between the environmental image and the taught environmental image, and determine the target planned path of the robotic arm according to the similarity and the initial planned path. The robotic arm is used to move the glass curtain wall to the end point of the operation task close to the curtain wall installation frame according to the target planned path after grasping the glass curtain wall. By introducing the attitude sensor and the vision sensor, the present application can re-obtain the initial position information and initial attitude information of the robotic arm at the starting point of the operation task and the environmental image of the robotic arm at the starting point of the operation task when the operation scenario changes. On the one hand, the initial planned path of the robotic arm is determined specifically based on the initial position information and the initial attitude information. On the other hand, the initial planned path is adjusted specifically based on the similarity between the environmental image and the taught environmental image to obtain the target planned path of the robotic arm. Therefore, when the operation scenario changes, the target planned path of the robotic arm in the changed operation scenario can be accurately obtained without re-teaching the robotic arm, thereby improving the versatility of the curtain wall installation control system, enhancing the adaptability of the system to complex and changeable construction environments, and further facilitating the popularization and application of curtain wall installation robots. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained without creative efforts.

[0018] Figure 1 is a schematic structural diagram of a curtain wall installation control system for a robotic arm based on path planning provided by an embodiment of the present application;

[0019] Figure 2 is a schematic diagram of the setting of the vision sensor provided by an embodiment of the present application;

[0020] Figure 3 is a schematic diagram of the setting of the light intensity sensor provided by an embodiment of the present application;

[0021] Figure 4 Schematic flow chart of the curtain wall installation control method for a robotic arm based on path planning provided by an embodiment of the present application. Detailed implementation manners

[0022] In the following description, specific details such as specific system structures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0023] To make the objectives, technical solutions, and advantages of the present application clearer, the following will be described through specific embodiments with reference to the accompanying drawings.

[0024] Figure 1 Schematic structural diagram of the curtain wall installation control system for a robotic arm based on path planning provided by an embodiment of the present application. As Figure 1 shown, the curtain wall installation control system for a robotic arm based on path planning includes: a robotic cart 1, a robotic arm 2, an attitude sensor 3, a vision sensor 4, and a controller 5. The robotic arm 2 is disposed on the robotic cart 1 and is used to grasp the glass curtain wall. Both the attitude sensor 3 and the vision sensor 4 are disposed on the robotic arm 2. The robotic cart 1, the robotic arm 2, the attitude sensor 3, and the vision sensor 4 are all communicatively connected to the controller 5.

[0025] The attitude sensor 3 is configured to obtain the initial position information and initial attitude information of the robotic arm 2 at the starting point of the operation task and send them to the controller 5.

[0026] The vision sensor 4 is configured to obtain the environmental image of the robotic arm 2 at the starting point of the operation task and send it to the controller 5.

[0027] The controller 5 is configured to determine the initial planned path of the robotic arm according to the initial position information, initial attitude information, and a preset teaching trajectory library, calculate the similarity between the environmental image and the taught environmental image, and determine the target planned path of the robotic arm 2 according to the similarity and the initial planned path.

[0028] The robotic arm 2 is configured to move the glass curtain wall to the end point of the operation task close to the curtain wall installation frame according to the target planned path after grasping the glass curtain wall.

[0029] Exemplarily, as Figure 1 shown, the curtain wall installation control system for a robotic arm based on path planning may further include at least one electric suction cup 7 and a suction cup holder 6. The suction cup holder 6 is fixedly installed on the robotic arm 2, and at least one electric suction cup 7 is fixedly installed on the suction cup holder 6.

[0030] The controller 5 is further configured to send a grasping instruction to at least one electric suction cup 7 to control at least one electric suction cup 7 to grasp the glass curtain wall.

[0031] During the construction and installation of the glass curtain wall by using the curtain wall installation robot, it is first necessary to control the curtain wall installation robot to grasp the glass curtain wall. For example, the glass curtain wall is grasped by the electric suction cup at the end of the robotic arm of the curtain wall installation robot. After grasping the glass curtain wall, the robot after grasping the glass curtain wall quickly approaches the position to be installed through the rough positioning stage, that is, the end point of the operation task close to the curtain wall installation frame, and then the fine installation of the glass curtain wall is completed through the fine positioning stage.

[0032] During the rough positioning stage, due to the different building structure forms at the curtain wall installation site, the different positions and types of safety protection settings, etc., there are often different space limitations at the curtain wall installation site. As a result, after learning the planned path of the robotic arm through teaching in one operation scenario, it is often inapplicable in other operation scenarios. Therefore, in this embodiment, by setting the attitude sensor and the vision sensor on the robotic arm, when the operation scenario changes, the initial position information, the initial attitude information of the robotic arm at the starting point of the operation task, and the differences in the environmental images of the robotic arm at the starting point of the operation task can be re-analyzed. Thus, the initial planned path of the robotic arm is dynamically adjusted according to the differences in the initial position information and the initial attitude information, and the initial planned path is adjusted again according to the environmental image. Therefore, without teaching the robotic arm in different operation scenarios, the target planned path of the robotic arm in different operation scenarios can be obtained more accurately, thereby improving the versatility of the curtain wall installation control system, enhancing the adaptability of the system to complex and changeable construction environments, and thus contributing to the popularization and application of the curtain wall installation robot.

[0033] Moreover, based on the target planned path determined in this embodiment, the robotic arm can reach the end point of the operation task smoothly and avoid passing through redundant paths, thereby significantly improving the operation efficiency of the curtain wall installation, ensuring the safety and stability of the entire system, effectively reducing the safety risks during the curtain wall construction and installation process, and ensuring the stability of the construction personnel and construction equipment.

[0034] In one embodiment, the taught environmental images may include the left front taught environmental image, the right front taught environmental image, and the obliquely above taught environmental image. As Figure 2 shown, the vision sensor may include a first camera 41, a second camera 42, and a third camera 43; the first camera 41 is disposed on the left side of the robotic arm 2, the second camera 42 is disposed on the right side of the robotic arm 2, and the third camera 43 is disposed above the robotic arm 2.

[0035] The first camera 41 is used to acquire the left-front environment image at a first angle relative to the robotic arm 2 and send it to the controller 5.

[0036] The second camera 42 is used to acquire the right-front environment image at a second angle relative to the robotic arm 2 and send it to the controller 5.

[0037] The third camera 43 is used to acquire the obliquely-up environment image at a third angle relative to the robotic arm 2 and send it to the controller 5.

[0038] The controller 5 is used for:

[0039] Calculating a first similarity between the left-front environment image and the left-front taught environment image, a second similarity between the right-front environment image and the right-front taught environment image, and a third similarity between the obliquely-up environment image and the obliquely-up taught environment image.

[0040] Determining the current working space of the robotic arm based on the left-front environment image, the right-front environment image, and the obliquely-up environment image.

[0041] Calculating a fourth similarity between the current working space and the taught working space.

[0042] Obtaining the similarity between the environment image and the taught environment image based on the first similarity, the second similarity, the third similarity, and the fourth similarity.

[0043] In this embodiment, considering that the glass curtain wall is generally large in size, when the vision sensor is installed on the robotic arm and the robotic arm has grasped the glass curtain wall, it is difficult to directly acquire the environment image around the curtain wall installation frame through the vision sensor. Therefore, it is considered to respectively set a vision sensor (i.e., the first camera 41, the second camera 42, and the third camera 43) on the left side, the right side, and the upper side of the robotic arm, so as to indirectly evaluate the difference between the current working environment and the working environment during teaching through the left-front environment image, the right-front environment image, and the obliquely-up environment image of the robotic arm, that is, the similarity between the environment image and the taught environment image, so as to accurately determine the target planning path of the robotic arm in the current working environment based on the similarity between the environment image and the taught environment image and the initial planned path.

[0044] In one embodiment, as Figure 3 shown, the curtain wall installation control system of the robotic arm based on path planning may further include three light intensity sensors 8; the three light intensity sensors 8 are respectively arranged on one side of the first camera 41, the second camera 42, and the third camera 43, and are used for collecting the light intensity in the image acquisition directions of the first camera 41, the second camera 42, and the third camera 43 and sending it to the controller 5.

[0045] The controller 5 is used for:

[0046] Compare each light intensity with the minimum light intensity and the maximum light intensity respectively.

[0047] If a certain light intensity is less than the minimum light intensity or greater than the maximum light intensity, the target similarity is corrected according to this light intensity, and the target similarity is the first similarity, the second similarity or the third similarity corresponding to the position where the light intensity sensor that collects this light intensity is located.

[0048] In this embodiment, in order to more accurately determine the target planning path of the robotic arm in the current working environment, considering the influence of the light intensity on the acquired environmental image, light intensity sensors are respectively arranged on one side of the first camera, one side of the second camera, and one side of the third camera, so as to collect the light intensity in the image acquisition directions of the first camera, the second camera, and the third camera. When the light intensity is too small or too large, it will cause the acquired environmental image to be unavailable or have more noise, and further cause a large deviation when calculating the similarity with the taught environmental image. Therefore, if a certain light intensity is less than the minimum light intensity or greater than the maximum light intensity, the calculated similarity is corrected according to this light intensity.

[0049] Exemplarily, assume that the light intensity collected by the light intensity sensor on one side of the first camera is less than the minimum light intensity or greater than the maximum light intensity, then the target similarity is the first similarity, and the first similarity is corrected according to this light intensity.

[0050] Similarly, assume that the light intensity collected by the light intensity sensor on one side of the second camera is less than the minimum light intensity or greater than the maximum light intensity, then the target similarity is the second similarity, and the second similarity is corrected according to this light intensity.

[0051] Assume that the light intensity collected by the light intensity sensor on one side of the third camera is less than the minimum light intensity or greater than the maximum light intensity, then the target similarity is the third similarity, and the third similarity is corrected according to this light intensity.

[0052] Optionally, correcting the target similarity according to this light intensity may include:

[0053] If this light intensity is less than the minimum light intensity, calculate the difference between the minimum light intensity and this light intensity, and denote it as the target difference.

[0054] If this light intensity is greater than the maximum light intensity, calculate the difference between this light intensity and the maximum light intensity, and denote it as the target difference.

[0055] Determine the average light intensity according to the minimum light intensity and the maximum light intensity, and calculate the ratio of the target difference to the average light intensity.

[0056] Correct the target similarity according to the ratio.

[0057] In this embodiment, when the light intensity is less than the minimum light intensity or greater than the maximum light intensity, correct the target similarity according to the light intensity. The average value of the minimum light intensity and the maximum light intensity (i.e., the average light intensity) can be used as a reference to evaluate the degree of deviation of the difference between the light intensity and the minimum light intensity or the maximum light intensity from the reference. The greater the degree of deviation, the greater the calculated ratio and the greater the degree of correction of the target similarity.

[0058] Exemplarily, through a calibration test, the degree of correction of the target similarity at different ratios can be determined, so as to conveniently correct the target similarity according to the ratio after calculating the ratio of the target difference to the average light intensity.

[0059] In one embodiment, as Figure 3 shown, the curtain wall installation control system of the robotic arm based on path planning may further include three light intensity sensors; the three light intensity sensors are respectively arranged on one side of the first camera 41, the second camera 42, and the third camera 43, and are used to collect the light intensity in the image acquisition directions of the first camera 41, the second camera 42, and the third camera 43 and send it to the controller 5.

[0060] The controller 5 is used for:

[0061] Before calculating the first similarity, the second similarity, and the third similarity, compare each light intensity with the minimum light intensity and the maximum light intensity respectively.

[0062] If a certain light intensity is less than the minimum light intensity or greater than the maximum light intensity, record the first similarity, the second similarity, or the third similarity corresponding to the position of the light intensity sensor that collects this light intensity as the target similarity.

[0063] Judge whether the number of target similarities is one.

[0064] If the number of target similarities is one, correct the confidence level of the target similarity according to this light intensity.

[0065] Obtaining the similarity between the environmental image and the taught environmental image according to the first similarity, the second similarity, the third similarity, and the fourth similarity may include:

[0066] If the confidence level of the corrected target similarity is lower than the confidence level threshold, obtain the similarity between the environmental image and the taught environmental image according to the similarities other than the target similarity among the first similarity, the second similarity, and the third similarity and the fourth similarity.

[0067] Optionally, after determining whether the number of target similarities is one, it may further include:

[0068] If the number of target similarities is greater than one, adjust the image acquisition directions of the cameras corresponding to each target similarity.

[0069] Re-acquire the light intensity in the adjusted image acquisition direction until the number of target similarities is less than or equal to one.

[0070] In this embodiment, considering that when the influence of the light intensity on the acquired environmental image is relatively large, that is, when the light intensity is less than the minimum light intensity or greater than the maximum light intensity, the credibility of the acquired environmental image is relatively low. When the light intensity collected by the light intensity sensor on one side of one of the first camera, the second camera, or the third camera is less than the minimum light intensity or greater than the maximum light intensity, discard the similarity calculated based on the environmental image collected by this camera, and use the similarities calculated based on the environmental images collected by other cameras to evaluate and determine the similarity between the environmental image and the taught environmental image.

[0071] When the light intensity collected by the light intensity sensors on at least two of the first camera, the second camera, or the third camera is less than the minimum light intensity or greater than the maximum light intensity, discard the environmental images collected by at least two cameras, adjust the image acquisition directions of the cameras, and then re-acquire the light intensity in the adjusted image acquisition direction.

[0072] Exemplarily, if only the light intensity collected by the light intensity sensor on one side of the first camera is less than the minimum light intensity or greater than the maximum light intensity, then determine the similarity between the environmental image and the taught environmental image according to the second similarity, the third similarity, and the fourth similarity. If the light intensities collected by the light intensity sensors on one side of the first camera and one side of the second camera are both less than the minimum light intensity or greater than the maximum light intensity, then adjust the image acquisition directions of the first camera and the second camera, and re-acquire the light intensity in the adjusted image acquisition direction until at most one of the re-acquired light intensities is less than the minimum light intensity or greater than the maximum light intensity, and then calculate the similarity between the corresponding environmental image and the taught environmental image and determine the final similarity.

[0073] In one embodiment, determining the current working space of the robotic arm according to the left front environmental image, the right front environmental image, and the obliquely upper environmental image may include:

[0074] According to the left front environmental image, the right front environmental image, and the obliquely upper environmental image, identify the building structure form where the curtain wall installation frame corresponding to the glass curtain wall is located and the positions and types of the safety protection facilities around the robotic arm.

[0075] Determine the current working space of the robotic arm according to the building structure form where the curtain wall installation frame is located and the position and type of the safety protection facilities.

[0076] Exemplarily, a building structure form recognition model and a safety protection facility recognition model can be trained to identify the building structure form where the curtain wall installation frame corresponding to the glass curtain wall is located according to the left front environment image, the right front environment image, the obliquely upper environment image, and the building structure form recognition model. Identify the position and type of the safety protection facilities around the robotic arm according to the left front environment image, the right front environment image, the obliquely upper environment image, and the safety protection facility recognition model. Then, after identifying the building structure form where the curtain wall installation frame is located and the position and type of the safety protection facilities, inversely deduce the current working space of the robotic arm. So as to evaluate the fourth similarity between the environment image and the taught environment image as a whole according to the current working space of the robotic arm and the taught working space.

[0077] In an embodiment, the curtain wall installation control system of the robotic arm based on path planning may further include a wind speed sensor, and the wind speed sensor is used to collect the environmental wind speed at the position where the robotic arm 2 is located and send it to the controller 5.

[0078] The controller 5 is used for:

[0079] Judge whether the environmental wind speed is greater than the wind speed threshold.

[0080] If the environmental wind speed is greater than the wind speed threshold, control the robotic arm 2 to move the glass curtain wall to a preset safe position and stop working.

[0081] In this embodiment, considering the influence of wind load on curtain wall installation, a wind speed sensor is also set. When it is determined based on the wind speed sensor that the environmental wind speed at the position where the robotic arm is located is greater than the wind speed threshold, control the robotic arm to move the glass curtain wall to a preset safe position, such as on a working platform, and stop working, so as to improve the safety and stability of curtain wall installation.

[0082] The present application provides a curtain wall installation control system for a robotic arm based on path planning. The system includes a robotic cart, a robotic arm, an attitude sensor, a vision sensor, and a controller. The robotic arm is arranged on the robotic cart and is used to grasp a glass curtain wall. The attitude sensor and the vision sensor are both arranged on the robotic arm. The robotic cart, the robotic arm, the attitude sensor, and the vision sensor are all communicatively connected to the controller. The attitude sensor is used to obtain the initial position information and the initial attitude information of the robotic arm at the starting point of the operation task and send them to the controller. The vision sensor is used to obtain the environmental image of the robotic arm at the starting point of the operation task and send it to the controller. The controller is used to determine the initial planned path of the robotic arm according to the initial position information, the initial attitude information, and a preset teaching trajectory library, calculate the similarity between the environmental image and the taught environmental image, and determine the target planned path of the robotic arm according to the similarity and the initial planned path. The robotic arm is used to move the glass curtain wall to the end point of the operation task close to the curtain wall installation frame according to the target planned path after grasping the glass curtain wall. By introducing the attitude sensor and the vision sensor, the present application can re-obtain the initial position information and the initial attitude information of the robotic arm at the starting point of the operation task and the environmental image of the robotic arm at the starting point of the operation task when the operation scenario changes. On the one hand, the initial planned path of the robotic arm is determined specifically based on the initial position information and the initial attitude information. On the other hand, the initial planned path is adjusted specifically based on the similarity between the environmental image and the taught environmental image to obtain the target planned path of the robotic arm. Therefore, when the operation scenario changes, the target planned path of the robotic arm in the changed operation scenario can be accurately obtained without re-teaching the robotic arm, thereby improving the versatility of the curtain wall installation control system, enhancing the adaptability of the system to complex and changeable construction environments, and further facilitating the popularization and application of curtain wall installation robots.

[0083] The following are method embodiments of the present application. For details not described in detail, reference may be made to the corresponding vision control system embodiments of the curtain wall installation control system for a robotic arm based on path planning above.

[0084] Figure 4 The implementation flowchart of the curtain wall installation control method for a robotic arm based on path planning provided by an embodiment of the present application is shown and described in detail as follows:

[0085] In step 401, the initial position information and the initial attitude information of the robotic arm on the robotic cart at the starting point of the operation task are obtained, and the environmental image where the robotic arm is located is obtained.

[0086] In an embodiment of the present application, the initial position information and the initial attitude information of the robotic arm on the robotic cart at the starting point of the operation task can be obtained by using an attitude sensor, and the environmental image where the robotic arm is located can be obtained by using a vision sensor.

[0087] In step 402, determine the initial planned path of the robotic arm according to the initial position information, initial attitude information, and the preset teaching trajectory library.

[0088] In the embodiment of the present application, the preset teaching trajectory library can be a teaching trajectory library obtained in the teaching operation scenario. After obtaining the preset teaching trajectory library, according to the initial position information and initial attitude information, the teaching trajectory in the case that is more similar to the initial position information and initial attitude information can be selected from the preset teaching trajectory library as the initial planned path of the robotic arm.

[0089] In step 403, calculate the similarity between the environmental image and the teaching environmental image.

[0090] In the embodiment of the present application, in order to improve versatility and adaptability, in the case of only obtaining the preset teaching trajectory library in the teaching operation scenario, the similarity between the environmental image of the current operation scenario where the robotic arm is located and the teaching environmental image can be evaluated, so that the initial planned path obtained based on the preset teaching trajectory library is more applicable to the current operation scenario.

[0091] In step 404, determine the target planned path of the robotic arm according to the similarity and the initial planned path, so that after the robotic arm grabs the glass curtain wall, the glass curtain wall can be moved to the end point of the operation task close to the curtain wall installation frame according to the target planned path.

[0092] The present application provides a curtain wall installation control method for a robotic arm based on path planning, which is applied to a controller in a curtain wall installation control system of a robotic arm based on path planning. The control method includes: obtaining the initial position information and initial attitude information of the robotic arm on the robot trolley at the starting point of the operation task, and obtaining the environmental image where the robotic arm is located; determining the initial planned path of the robotic arm according to the initial position information, initial attitude information, and the preset teaching trajectory library; calculating the similarity between the environmental image and the teaching environmental image; determining the target planned path of the robotic arm according to the similarity and the initial planned path, so that after the robotic arm grabs the glass curtain wall, the glass curtain wall can be moved to the end point of the operation task close to the curtain wall installation frame according to the target planned path. The present application can re-obtain the initial position information and initial attitude information of the robotic arm at the starting point of the operation task and the environmental image of the robotic arm at the starting point of the operation task when the operation scenario changes. On the one hand, determine the initial planned path of the robotic arm specifically based on the initial position information and initial attitude information. On the other hand, specifically adjust the initial planned path based on the similarity between the environmental image and the teaching environmental image to obtain the target planned path of the robotic arm. Therefore, when the operation scenario changes, the target planned path of the robotic arm in the changed operation scenario can be accurately obtained without re-teaching the robotic arm, thereby improving the versatility of the curtain wall installation control system and enhancing the adaptability of the system to complex and changeable construction environments, and further contributing to the popularization and application of curtain wall installation robots.

[0093] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included within the protection scope of the present application.

Claims

1. A curtain wall installation control system based on a robot arm with path planning, characterized in that: include: A robot car, a mechanical arm, a posture sensor, a visual sensor and a controller, wherein the mechanical arm is arranged on the robot car and is used to grab the glass curtain wall, the posture sensor and the visual sensor are both arranged on the mechanical arm, and the robot car, the mechanical arm, the posture sensor and the visual sensor are all communicatively connected with the controller; The posture sensor is used to obtain the initial position information and initial posture information of the robot arm at the starting point of the operation task and send them to the controller; The visual sensor is used to obtain an environmental image of the robot arm at the starting point of the operation task and send it to the controller; The controller is used to determine the initial planning path of the robot arm according to the initial position information, the initial posture information and a preset teaching trajectory library, and calculate the similarity between the environment image and the teaching environment image, and determine the target planning path of the robot arm according to the similarity and the initial planning path; The robotic arm is used to move the glass curtain wall to an end point of the operation task close to the curtain wall installation frame according to the target planning path after grabbing the glass curtain wall; The teaching environment image includes a left front teaching environment image, a right front teaching environment image, and an obliquely upper teaching environment image, and the visual sensor includes a first camera for acquiring a left front environment image at a first angle relative to the robotic arm, a second camera for acquiring a right front environment image at a second angle relative to the robotic arm, and a third camera for acquiring an obliquely upper environment image at a third angle relative to the robotic arm; The controller is used to: Calculating a first similarity between the left front environment image and the left front teaching environment image, a second similarity between the right front environment image and the right front teaching environment image, and a third similarity between the obliquely upper environment image and the obliquely upper teaching environment image; Determine the current working space of the robot arm according to the three environment images; Calculating a fourth similarity between the current operation space and the teaching operation space; Obtaining a similarity between the environment image and the teaching environment image according to the first similarity, the second similarity, the third similarity, and the fourth similarity; It also includes: three light intensity sensors; the three light intensity sensors are respectively arranged on one side of the three cameras, and are used to collect the light intensity in the image acquisition direction of the three cameras and send it to the controller; The controller is used to: Before calculating the first similarity, the second similarity and the third similarity, each of the illumination intensities is compared with a minimum illumination intensity and a maximum illumination intensity respectively; If a certain light intensity is less than the minimum light intensity value or greater than the maximum light intensity value, the first similarity, the second similarity or the third similarity corresponding to the position where the light intensity sensor collecting the light intensity is located is recorded as the target similarity; Determining whether the number of target similarities is one; If the number of the target similarity is one, the target similarity is corrected according to the light intensity; If the confidence of the corrected target similarity is lower than the confidence threshold, obtaining the similarity between the environment image and the teaching environment image according to the similarities other than the target similarity among the first similarity, the second similarity and the third similarity and the fourth similarity; If the number of the target similarities is greater than one, adjusting the image acquisition direction of the camera corresponding to each target similarity; The illumination intensity in the adjusted image acquisition direction is re-acquired until the number of target similarities is less than or equal to one.

2. The curtain wall installation control system of the robot arm based on path planning according to claim 1, characterized in that: The first camera is arranged on the left side of the robotic arm, the second camera is arranged on the right side of the robotic arm, and the third camera is arranged above the robotic arm.

3. The curtain wall installation control system of the robot arm based on path planning according to claim 2, characterized in that: The target similarity is corrected according to the light intensity, including: If the light intensity is less than the minimum light intensity, then the difference between the minimum light intensity and the light intensity is calculated and recorded as the target difference; If the light intensity is greater than the maximum light intensity, then the difference between the light intensity and the maximum light intensity is calculated and recorded as the target difference; Determine a light intensity mean value according to the light intensity minimum value and the light intensity maximum value, and calculate a ratio of the target difference value to the light intensity mean value; The target similarity is modified according to the ratio.

4. The curtain wall installation control system of the robot arm based on path planning according to claim 1, characterized in that: Determining the current working space of the robot arm according to the left front environment image, the right front environment image, and the upper oblique environment image includes: According to the left front environment image, the right front environment image and the oblique upper environment image, identifying the building structure type where the curtain wall installation frame corresponding to the glass curtain wall is located and the location and type of the safety protection facilities around the robotic arm; The current working space of the robotic arm is determined according to the building structure type where the curtain wall installation frame is located and the location and type of the safety protection facilities.

5. The curtain wall installation control system of the robot arm based on path planning according to claim 1, characterized in that: It also includes a wind speed sensor, which is used to collect the ambient wind speed at the location of the mechanical arm and send it to the controller; The controller is used to: Determining whether the ambient wind speed is greater than a wind speed threshold; If the ambient wind speed is greater than the wind speed threshold, the robotic arm is controlled to move the glass curtain wall to a preset safe position and stop working.

6. The curtain wall installation control system of the robot arm based on path planning according to claim 1, characterized in that: It also includes at least one electric suction cup and a suction cup frame, wherein the suction cup frame is fixedly mounted on the mechanical arm, and the at least one electric suction cup is fixedly mounted on the suction cup frame; The controller is also used to send a grabbing instruction to the at least one electric suction cup to control the at least one electric suction cup to grab the glass curtain wall.

7. A curtain wall installation control method for a robot arm based on path planning, applied to a visual control system in a curtain wall installation control system for a robot arm based on path planning as claimed in any one of claims 1 to 6, characterized in that: The control method comprises: Obtaining the initial position information and initial posture information of the mechanical arm on the robot car at the starting point of the operation task, and obtaining the environment image where the mechanical arm is located; Determining an initial planning path of the robotic arm according to the initial position information, the initial posture information and a preset teaching trajectory library; Calculating the similarity between the environment image and the teaching environment image; The target planning path of the robot arm is determined according to the similarity and the initial planning path, so that after the robot arm grabs the glass curtain wall, the glass curtain wall is moved to an end point of the operation task close to the curtain wall installation frame according to the target planning path.

Images