Curtain wall installation control system and method of mechanical arm based on path planning

Abstract

The invention is suitable for the technical field of manipulator control, and provides a curtain wall installation control system and method of a mechanical arm based on path planning. The system comprises a robot trolley, a mechanical arm, an attitude sensor, a visual sensor and a controller, the attitude sensor obtains initial position information and initial attitude information of the mechanical arm at an operation task starting point and sends the initial position information and the initial attitude information to the controller; the visual sensor obtains an environment image of the mechanical arm at the operation task starting point and sends the environment image to the controller; the controller determines an initial planning path of the mechanical arm according to the initial position information, the initial attitude information and a preset teaching track library, calculates the similarity between the environment image and the teaching environment image, and determines a target planning path of the mechanical arm according to the similarity and the initial planning path; and after the mechanical arm grabs the glass curtain wall, the glass curtain wall is moved to the end point of the operation task close to the curtain wall mounting frame according to the target planning path. The universality and adaptability of the curtain wall installation control system can be improved.

Description

Technical Field

[0001] The present application relates to the field of robot control, and in particular to a curtain wall installation control system and method of a robot arm based on path planning. Background Art

[0002] With the continuous development of modern architectural design, glass curtain walls have been widely used due to their advantages such as lightness, beauty, low pollution, safety, reliability and easy maintenance.

[0003] At present, the construction and installation of glass curtain walls mainly rely on manpower, and because curtain walls are generally large in size and weight, they usually require multiple workers to work together. Not only is the installation efficiency low and the labor intensity high, but there is also a high safety risk when working at height.

[0004] In order to solve the problems in the curtain wall installation process, it is usually considered to use curtain wall installation robots instead of manual labor. However, the inventors found that: since the process of curtain wall installation is relatively complicated, and the installation trajectories of curtain walls at different locations in the physical space are also quite different, when using curtain wall installation robots, for different operation scenarios, it is usually necessary to conduct targeted teaching of the mechanical arm of the curtain wall installation robot, so that the path planned for the mechanical arm after teaching meets the task requirements in the new operation scenario, resulting in poor versatility and adaptability of curtain wall installation robots, thus affecting the promotion and application of curtain wall installation robots. Summary of the invention

[0005] The present application provides a curtain wall installation control system and method for a mechanical arm based on path planning to solve the problem of poor versatility and adaptability of the curtain wall installation robot.

[0006] In a first aspect, the present application provides a curtain wall installation control system of a mechanical arm based on path planning, comprising: a robot car, a mechanical arm, a posture sensor, a visual sensor and a controller, wherein the mechanical arm is arranged on the robot car and used to grab a glass curtain wall, the posture sensor and the visual sensor are both arranged on the mechanical arm, and the robot car, the mechanical arm, the posture sensor and the visual sensor are all communicatively connected with the controller; The posture sensor is used to obtain the initial position information and initial posture information of the robot arm at the starting point of the operation task and send them to the controller; The visual sensor is used to obtain an environmental image of the robot arm at the starting point of the operation task and send it to the controller; The controller is used to determine the initial planning path of the robot arm according to the initial position information, the initial posture information and a preset teaching trajectory library, and calculate the similarity between the environment image and the teaching environment image, and determine the target planning path of the robot arm according to the similarity and the initial planning path; The robotic arm is used to move the glass curtain wall to an end point of the operation task close to the curtain wall installation frame according to the target planning path after grabbing the glass curtain wall.

[0007] In a second aspect, the present application provides a curtain wall installation control method of a robot arm based on path planning, which is applied to a controller in a curtain wall installation control system of a robot arm based on path planning as described in the first aspect above, and the control method comprises: Obtaining the initial position information and initial posture information of the mechanical arm on the robot car at the starting point of the operation task, and obtaining the environment image where the mechanical arm is located; Determining an initial planning path of the robotic arm according to the initial position information, the initial posture information and a preset teaching trajectory library; Calculating the similarity between the environment image and the teaching environment image; The target planning path of the robot arm is determined according to the similarity and the initial planning path, so that after the robot arm grabs the glass curtain wall, the glass curtain wall is moved to an end point of the operation task close to the curtain wall installation frame according to the target planning path.

[0008] The present application provides a curtain wall installation control system and method of a mechanical arm based on path planning, the system includes a robot car, a mechanical arm, a posture sensor, a visual sensor and a controller, the mechanical arm is arranged on the robot car and is used to grab the glass curtain wall, the posture sensor and the visual sensor are both arranged on the mechanical arm, the robot car, the mechanical arm, the posture sensor and the visual sensor are all connected to the controller for communication; the posture sensor is used to obtain the initial position information and initial posture information of the mechanical arm at the starting point of the operation task and send them to the controller; the visual sensor is used to obtain the environmental image of the mechanical arm at the starting point of the operation task and send it to the controller; the controller is used to determine the initial planning path of the mechanical arm according to the initial position information, the initial posture information and the preset teaching trajectory library, and calculate the similarity between the environmental image and the teaching environmental image, and determine the target planning path of the mechanical arm according to the similarity and the initial planning path; the mechanical arm is used to move the glass curtain wall to the end point of the operation task close to the curtain wall installation frame according to the target planning path after grabbing the glass curtain wall. By introducing posture sensors and visual sensors, the present application can re-acquire the initial position information and initial posture information of the robot arm at the starting point of the work task and the environmental image of the robot arm at the starting point of the work task when the work scene changes. On the one hand, the initial planned path of the robot arm is determined based on the initial position information and the initial posture information. On the other hand, the initial planned path is adjusted based on the similarity between the environmental image and the teaching environment image to obtain the target planned path of the robot arm. Therefore, when the work scene changes, the target planned path of the robot arm in the changed work scene can be accurately obtained without re-teaching the robot arm, thereby improving the versatility of the curtain wall installation control system, enhancing the system's adaptability to complex and changeable construction environments, and further contributing to the promotion and application of curtain wall installation robots. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0010] Figure 1 It is a structural schematic diagram of a curtain wall installation control system of a mechanical arm based on path planning provided in an embodiment of the present application; Figure 2 is a schematic diagram of the setting of the visual sensor provided in the embodiment of the present application; Figure 3 is a schematic diagram of the configuration of a light intensity sensor provided in an embodiment of the present application; Figure 4A schematic flow chart of a curtain wall installation control method for a robotic arm based on path planning provided in an embodiment of the present application. DETAILED DESCRIPTION

[0011] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0012] In order to make the purpose, technical solutions and advantages of the present application clearer, specific embodiments will be described below in conjunction with the accompanying drawings.

[0013] Figure 1 This is a schematic diagram of the structure of the curtain wall installation control system based on the path planning of the robot arm provided in the embodiment of the present application. Figure 1 As shown, the curtain wall installation control system of the robotic arm based on path planning includes: a robot car 1, a robotic arm 2, a posture sensor 3, a visual sensor 4 and a controller 5. The robotic arm 2 is arranged on the robot car 1 and is used to grab the glass curtain wall. The posture sensor 3 and the visual sensor 4 are both arranged on the robotic arm 2. The robot car 1, the robotic arm 2, the posture sensor 3 and the visual sensor 4 are all communicatively connected with the controller 5.

[0014] The posture sensor 3 is used to obtain the initial position information and initial posture information of the robot arm 2 at the starting point of the operation task and send them to the controller 5.

[0015] The visual sensor 4 is used to obtain the environmental image of the robot arm 2 at the starting point of the operation task and send it to the controller 5.

[0016] The controller 5 is used to determine the initial planning path of the robot arm according to the initial position information, the initial posture information and the preset teaching trajectory library, and calculate the similarity between the environment image and the teaching environment image, and determine the target planning path of the robot arm 2 according to the similarity and the initial planning path.

[0017] The robot arm 2 is used to move the glass curtain wall to an end point of the operation task close to the curtain wall installation frame according to the target planning path after grabbing the glass curtain wall.

[0018] For example, Figure 1 As shown, the curtain wall installation control system of the robot arm based on path planning can also include at least one electric suction cup 7 and a suction cup frame 6, the suction cup frame 6 is fixedly mounted on the robot arm 2, and at least one electric suction cup 7 is fixedly mounted on the suction cup frame 6.

[0019] The controller 5 is further used to send a grabbing instruction to at least one electric suction cup 7 to control at least one electric suction cup 7 to grab the glass curtain wall.

[0020] Among them, in the process of using the curtain wall installation robot to realize the construction and installation of the glass curtain wall, it is first necessary to control the curtain wall installation robot to grab the glass curtain wall, for example, by using the electric suction cup at the end of the mechanical arm of the curtain wall installation robot to grab the glass curtain wall. After grabbing the glass curtain wall, the robot that grabs the glass curtain wall is quickly moved close to the installation position, that is, close to the end point of the operation task of the curtain wall installation frame, through the rough positioning stage, and then the fine installation of the glass curtain wall is completed through the fine positioning stage.

[0021] Among them, in the rough positioning stage, due to the different architectural structures at the curtain wall installation site, the different locations and types of the safety protection settings, etc., the curtain wall installation site often has different space restrictions, which leads to the fact that after learning the planned path of the robot arm through teaching in one operation scene, it is often not applicable in other operation scenes. Therefore, this embodiment uses the posture sensor and visual sensor set on the robot arm to re-analyze the initial position information, initial posture information and environmental image of the robot arm at the starting point of the operation task when the operation scene changes, so as to dynamically adjust the initial planning path of the robot arm according to the difference in initial position information and initial posture information, and adjust the initial planning path again according to the environmental image, so that there is no need to teach the robot arm in different operation scenes, and the target planning path of the robot arm in different operation scenes can be obtained more accurately, thereby improving the versatility of the curtain wall installation control system, enhancing the system's adaptability to complex and changeable construction environments, and thus contributing to the promotion and application of curtain wall installation robots.

[0022] Moreover, based on the target planning path determined in this embodiment, the robotic arm can reach the end of the work task smoothly and avoid passing through unnecessary paths, thereby significantly improving the operating efficiency of the curtain wall installation, ensuring the safety and stability of the entire system, effectively reducing the safety risks in the curtain wall construction and installation process, and ensuring the stability of construction personnel and construction equipment.

[0023] In one embodiment, the teaching environment image may include a left front teaching environment image, a right front teaching environment image, and an upper oblique teaching environment image. Figure 2 As shown, the visual sensor may include a first camera 41, a second camera 42 and a third camera 43; the first camera 41 is arranged on the left side of the robotic arm 2, the second camera 42 is arranged on the right side of the robotic arm 2, and the third camera 43 is arranged above the robotic arm 2.

[0024] The first camera 41 is used to obtain a left front environment image at a first angle relative to the robot arm 2 and send the image to the controller 5 .

[0025] The second camera 42 is used to obtain an image of the right front environment at a second angle relative to the robot arm 2 and send the image to the controller 5 .

[0026] The third camera 43 is used to obtain an upper oblique environmental image at a third angle relative to the robot arm 2 and send the image to the controller 5 .

[0027] The controller 5 is used for: A first similarity between the left front environment image and the left front teaching environment image, a second similarity between the right front environment image and the right front teaching environment image, and a third similarity between the obliquely upper environment image and the obliquely upper teaching environment image are calculated.

[0028] The current working space of the robot arm is determined according to the left front environment image, the right front environment image and the upper oblique environment image.

[0029] A fourth similarity between the current work space and the teaching work space is calculated.

[0030] The similarity between the environment image and the teaching environment image is obtained according to the first similarity, the second similarity, the third similarity and the fourth similarity.

[0031] In this embodiment, considering that the glass curtain wall is generally large in size, when the visual sensor is installed on the robotic arm and the robotic arm has grasped the glass curtain wall, it is difficult to directly obtain the environmental image around the curtain wall installation frame through the visual sensor. Therefore, it is considered to set a visual sensor (that is, the first camera 41, the second camera 42 and the third camera 43) on the left side, right side and top of the robotic arm respectively, so as to indirectly evaluate the difference between the current working environment and the working environment during teaching through the left front environmental image, the right front environmental image and the upper oblique environmental image of the robotic arm, that is, the similarity between the environmental image and the teaching environmental image, so as to accurately determine the target planning path of the robotic arm in the current working environment based on the similarity between the environmental image and the teaching environmental image and the initial planning path.

[0032] In one embodiment, if Figure 3 As shown, the curtain wall installation control system of the path planning-based robotic arm can also include three light intensity sensors 8; the three light intensity sensors 8 are respectively arranged on one side of the first camera 41, the second camera 42 and the third camera 43, and are used to collect the light intensity in the image acquisition direction of the first camera 41, the second camera 42 and the third camera 43 and send it to the controller 5.

[0033] The controller 5 is used for: Each light intensity is compared to the minimum light intensity and the maximum light intensity.

[0034] If a certain light intensity is less than the minimum light intensity or greater than the maximum light intensity, the target similarity is corrected according to the light intensity, and the target similarity is the first similarity, the second similarity or the third similarity corresponding to the position of the light intensity sensor that collects the light intensity.

[0035] In this embodiment, in order to more accurately determine the target planning path of the robot arm in the current working environment, considering the influence of light intensity on the acquired environmental image, light intensity sensors are respectively set on one side of the first camera, one side of the second camera and one side of the third camera, so as to collect the light intensity in the image acquisition direction of the first camera, the second camera and the third camera. When the light intensity is too small or too large, the acquired environmental image will be unusable or have more noise, which will lead to a large deviation in the calculation of the similarity with the teaching environment image. Therefore, if a certain light intensity is less than the minimum light intensity or greater than the maximum light intensity, the calculated similarity is corrected according to the light intensity.

[0036] Exemplarily, assuming that the light intensity collected by the light intensity sensor on the first camera side is less than the minimum light intensity or greater than the maximum light intensity, the target similarity is the first similarity, and the first similarity is corrected according to the light intensity.

[0037] Similarly, assuming that the light intensity collected by the light intensity sensor on the second camera side is less than the minimum light intensity or greater than the maximum light intensity, the target similarity is the second similarity, and the second similarity is corrected according to the light intensity.

[0038] Assuming that the light intensity collected by the light intensity sensor on the third camera side is less than the minimum light intensity or greater than the maximum light intensity, the target similarity is the third similarity, and the third similarity is corrected according to the light intensity.

[0039] Optionally, correcting the target similarity according to the light intensity may include: If the light intensity is less than the minimum light intensity, the difference between the minimum light intensity and the light intensity is calculated and recorded as the target difference.

[0040] If the light intensity is greater than the maximum light intensity, the difference between the light intensity and the maximum light intensity is calculated and recorded as the target difference.

[0041] The average light intensity is determined according to the minimum light intensity and the maximum light intensity, and the ratio of the target difference to the average light intensity is calculated.

[0042] The target similarity is corrected according to the ratio.

[0043] In this embodiment, when the light intensity is less than the minimum light intensity or greater than the maximum light intensity, the target similarity is corrected according to the light intensity. The average of the minimum light intensity and the maximum light intensity (that is, the light intensity mean) can be used as a benchmark to evaluate the degree to which the difference between the light intensity and the minimum light intensity or the maximum light intensity deviates from the benchmark. When the degree of deviation is greater, the calculated ratio is greater, and the degree of correction of the target similarity is also greater.

[0044] Exemplarily, the degree of correction of target similarity under different ratios can be determined through calibration tests, so that after the ratio of the target difference to the average light intensity is calculated, the target similarity can be conveniently corrected according to the ratio.

[0045] In one embodiment, if Figure 3 As shown, the curtain wall installation control system of the path planning-based robotic arm can also include three light intensity sensors; the three light intensity sensors are respectively arranged on one side of the first camera 41, the second camera 42 and the third camera 43, and are used to collect the light intensity in the image acquisition direction of the first camera 41, the second camera 42 and the third camera 43 and send it to the controller 5.

[0046] The controller 5 is used for: Before calculating the first similarity, the second similarity and the third similarity, each illumination intensity is compared with the minimum illumination intensity and the maximum illumination intensity respectively.

[0047] If a certain light intensity is less than the minimum light intensity or greater than the maximum light intensity, the first similarity, the second similarity or the third similarity corresponding to the position of the light intensity sensor collecting the light intensity is recorded as the target similarity.

[0048] Determine whether the number of target similarities is one.

[0049] If the number of target similarities is one, the confidence of the target similarity is corrected according to the light intensity.

[0050] Obtaining the similarity between the environment image and the teaching environment image according to the first similarity, the second similarity, the third similarity, and the fourth similarity may include: If the confidence of the corrected target similarity is lower than the confidence threshold, the similarity between the environment image and the teaching environment image is obtained according to the similarities other than the target similarity among the first similarity, the second similarity and the third similarity and the fourth similarity.

[0051] Optionally, after determining whether the number of target similarities is one, the following may also be included: If the number of target similarities is greater than one, the image acquisition direction of the camera corresponding to each target similarity is adjusted.

[0052] Re-acquire the illumination intensity in the adjusted image acquisition direction until the number of target similarities is less than or equal to one.

[0053] In this embodiment, considering that the influence of light intensity on the acquired environmental image is relatively large, that is, when the light intensity is less than the minimum light intensity or greater than the maximum light intensity, the credibility of the acquired environmental image is low, when the light intensity collected by the light intensity sensor on one side of the first camera, the second camera or the third camera is less than the minimum light intensity or greater than the maximum light intensity, the similarity calculated based on the environmental image collected by the camera is discarded, and the similarity calculated based on the environmental images collected by other cameras is used to evaluate and determine the similarity between the environmental image and the teaching environmental image.

[0054] When the light intensity collected by the light intensity sensor on one side of at least two cameras among the first camera, the second camera or the third camera is less than the minimum light intensity or greater than the maximum light intensity, the environmental images collected by at least two cameras are discarded, the image acquisition direction of the camera is adjusted, and the light intensity in the adjusted image acquisition direction is re-collected.

[0055] Exemplarily, if only the light intensity collected by the light intensity sensor on the first camera side is less than the minimum light intensity or greater than the maximum light intensity, the similarity between the environment image and the teaching environment image is determined based on the second similarity, the third similarity, and the fourth similarity. If the light intensity collected by the light intensity sensors on the first camera side and the second camera side are both less than the minimum light intensity or greater than the maximum light intensity, the image acquisition direction of the first camera and the second camera is adjusted, and the light intensity in the adjusted image acquisition direction is re-collected until at most only one of the re-collected light intensities is less than the minimum light intensity or greater than the maximum light intensity, then the similarity between the corresponding environment image and the teaching environment image is calculated and the final similarity is determined.

[0056] In one embodiment, determining the current working space of the robot arm according to the left front environment image, the right front environment image, and the upper oblique environment image may include: According to the left front environment image, the right front environment image and the upper oblique environment image, the building structure type where the curtain wall installation frame corresponding to the glass curtain wall is located and the position and type of the safety protection facilities around the robot arm are identified.

[0057] The current working space of the robot arm is determined according to the building structure type where the curtain wall installation frame is located and the location and type of safety protection facilities.

[0058] Exemplarily, a building structure form recognition model and a safety protection facility recognition model can be trained to identify the building structure form where the curtain wall installation frame corresponding to the glass curtain wall is located based on the left front environment image, the right front environment image, the oblique upper environment image and the building structure form recognition model. According to the left front environment image, the right front environment image, the oblique upper environment image and the safety protection facility recognition model, the location and type of the safety protection facilities around the robot arm are identified. Then, after identifying the building structure form where the curtain wall installation frame is located and the location and type of the safety protection facilities, the current working space of the robot arm is inferred. In order to evaluate the fourth similarity between the environmental image and the teaching environment image as a whole based on the current working space and the teaching working space of the robot arm.

[0059] In one embodiment, the curtain wall installation control system of the robot arm based on path planning may also include a wind speed sensor, which is used to collect the ambient wind speed at the location of the robot arm 2 and send it to the controller 5.

[0060] The controller 5 is used for: Determine whether the ambient wind speed is greater than the wind speed threshold.

[0061] If the ambient wind speed is greater than the wind speed threshold, the robot arm 2 is controlled to move the glass curtain wall to a preset safe position and stop working.

[0062] In this embodiment, considering the influence of wind load on the curtain wall installation, a wind speed sensor is also provided. When the wind speed sensor determines that the ambient wind speed at the location of the robotic arm is greater than the wind speed threshold, the robotic arm is controlled to move the glass curtain wall to a preset safe position such as a working platform and stop working, so as to improve the safety and stability of the curtain wall installation.

[0063] The present application provides a curtain wall installation control system of a mechanical arm based on path planning, the system includes a robot car, a mechanical arm, a posture sensor, a visual sensor and a controller, the mechanical arm is arranged on the robot car and is used to grab the glass curtain wall, the posture sensor and the visual sensor are both arranged on the mechanical arm, the robot car, the mechanical arm, the posture sensor and the visual sensor are all communicatively connected with the controller; the posture sensor is used to obtain the initial position information and initial posture information of the mechanical arm at the starting point of the operation task and send them to the controller; the visual sensor is used to obtain the environmental image of the mechanical arm at the starting point of the operation task and send it to the controller; the controller is used to determine the initial planning path of the mechanical arm according to the initial position information, the initial posture information and the preset teaching trajectory library, and calculate the similarity between the environmental image and the teaching environmental image, and determine the target planning path of the mechanical arm according to the similarity and the initial planning path; the mechanical arm is used to move the glass curtain wall to the end point of the operation task close to the curtain wall installation frame according to the target planning path after grabbing the glass curtain wall. By introducing posture sensors and visual sensors, the present application can re-acquire the initial position information and initial posture information of the robot arm at the starting point of the work task and the environmental image of the robot arm at the starting point of the work task when the work scene changes. On the one hand, the initial planned path of the robot arm is determined based on the initial position information and the initial posture information. On the other hand, the initial planned path is adjusted based on the similarity between the environmental image and the teaching environment image to obtain the target planned path of the robot arm. Therefore, when the work scene changes, the target planned path of the robot arm in the changed work scene can be accurately obtained without re-teaching the robot arm, thereby improving the versatility of the curtain wall installation control system, enhancing the system's adaptability to complex and changeable construction environments, and further contributing to the promotion and application of curtain wall installation robots.

[0064] The following is an embodiment of the method of the present application. For details not described in detail, please refer to the above-mentioned corresponding visual control system embodiment of the curtain wall installation control system of the robotic arm based on path planning.

[0065] Figure 4 The following is a flowchart of a curtain wall installation control method based on path planning provided by a robotic arm according to an embodiment of the present application, which is described in detail as follows: In step 401, the initial position information and initial posture information of the robot arm on the robot vehicle at the starting point of the operation task are obtained, and the environment image where the robot arm is located is obtained.

[0066] In an embodiment of the present application, a posture sensor can be used to obtain the initial position information and initial posture information of the mechanical arm on the robot vehicle at the starting point of the work task, and a visual sensor can be used to obtain an image of the environment in which the mechanical arm is located.

[0067] In step 402, the initial planning path of the robot arm is determined according to the initial position information, the initial posture information and the preset teaching trajectory library.

[0068] In an embodiment of the present application, the preset teaching trajectory library may be a teaching trajectory library obtained in a teaching operation scenario. After obtaining the preset teaching trajectory library, according to the initial position information and the initial posture information, a teaching trajectory that is more similar to the initial position information and the initial posture information may be selected from the preset teaching trajectory library as the initial planning path of the robot arm.

[0069] In step 403, the similarity between the environment image and the teaching environment image is calculated.

[0070] In an embodiment of the present application, in order to improve versatility and adaptability, when only a preset teaching trajectory library is obtained for a teaching operation scenario, the similarity between the environmental image of the current operation scenario of the robot arm and the teaching environment image can be evaluated to make the initial planned path obtained based on the preset teaching trajectory library more suitable for the current operation scenario.

[0071] In step 404, the target planning path of the robot arm is determined according to the similarity and the initial planning path, so that after the robot arm grabs the glass curtain wall, the glass curtain wall is moved to the end point of the operation task close to the curtain wall installation frame according to the target planning path.

[0072] The present application provides a curtain wall installation control method of a mechanical arm based on path planning, which is applied to a controller in a curtain wall installation control system of a mechanical arm based on path planning. The control method comprises: obtaining initial position information and initial posture information of a mechanical arm on a robot vehicle at a starting point of an operation task, and obtaining an environment image in which the mechanical arm is located; determining an initial planning path of the mechanical arm according to the initial position information, initial posture information and a preset teaching trajectory library; calculating the similarity between the environment image and the teaching environment image; determining a target planning path of the mechanical arm according to the similarity and the initial planning path, so that after the mechanical arm grabs the glass curtain wall, the glass curtain wall is moved to an end point of the operation task close to a curtain wall installation frame according to the target planning path. The present application can re-acquire the initial position information and initial posture information of the robot arm at the starting point of the work task and the environmental image of the robot arm at the starting point of the work task when the work scene changes. On the one hand, the initial planned path of the robot arm is determined based on the initial position information and the initial posture information. On the other hand, the initial planned path is adjusted based on the similarity between the environmental image and the teaching environment image to obtain the target planned path of the robot arm. Therefore, when the work scene changes, the target planned path of the robot arm in the changed work scene can be accurately obtained without re-teaching the robot arm, thereby improving the versatility of the curtain wall installation control system, enhancing the system's adaptability to complex and changeable construction environments, and further contributing to the promotion and application of curtain wall installation robots.

[0073] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A curtain wall installation control system based on a robot arm with path planning, characterized in that: include: A robot car, a mechanical arm, a posture sensor, a visual sensor and a controller, wherein the mechanical arm is arranged on the robot car and is used to grab the glass curtain wall, the posture sensor and the visual sensor are both arranged on the mechanical arm, and the robot car, the mechanical arm, the posture sensor and the visual sensor are all communicatively connected with the controller; The posture sensor is used to obtain the initial position information and initial posture information of the robot arm at the starting point of the operation task and send them to the controller; The visual sensor is used to obtain an environmental image of the robot arm at the starting point of the operation task and send it to the controller; The controller is used to determine the initial planning path of the robot arm according to the initial position information, the initial posture information and a preset teaching trajectory library, and calculate the similarity between the environment image and the teaching environment image, and determine the target planning path of the robot arm according to the similarity and the initial planning path; The robotic arm is used to move the glass curtain wall to an end point of the operation task close to the curtain wall installation frame according to the target planning path after grabbing the glass curtain wall.

2. The curtain wall installation control system of the robot arm based on path planning according to claim 1, characterized in that: The teaching environment image includes a left front teaching environment image, a right front teaching environment image, and an obliquely upper teaching environment image, and the visual sensor includes a first camera, a second camera, and a third camera; the first camera is arranged on the left side of the mechanical arm, the second camera is arranged on the right side of the mechanical arm, and the third camera is arranged above the mechanical arm; The first camera is used to obtain a left front environment image at a first angle relative to the robotic arm and send the image to the controller; The second camera is used to obtain an image of the right front environment at a second angle relative to the robotic arm and send the image to the controller; The third camera is used to obtain an upper oblique environmental image at a third angle relative to the robotic arm and send the image to the controller; The controller is used to: Calculating a first similarity between the left front environment image and the left front teaching environment image, a second similarity between the right front environment image and the right front teaching environment image, and a third similarity between the obliquely upper environment image and the obliquely upper teaching environment image; Determine the current working space of the robot arm according to the left front environment image, the right front environment image and the upper oblique environment image; Calculating a fourth similarity between the current operation space and the teaching operation space; The similarity between the environment image and the teaching environment image is obtained according to the first similarity, the second similarity, the third similarity and the fourth similarity.

3. The curtain wall installation control system of the robot arm based on path planning according to claim 2, characterized in that: Also includes: three light intensity sensors; The three light intensity sensors are respectively arranged on one side of the first camera, the second camera and the third camera, and are used to collect the light intensity in the image acquisition direction of the first camera, the second camera and the third camera and send it to the controller; The controller is used to: Compare each of the light intensities with the minimum light intensity and the maximum light intensity respectively; If a certain light intensity is less than the minimum light intensity or greater than the maximum light intensity, the target similarity is corrected according to the light intensity, and the target similarity is the first similarity, the second similarity or the third similarity corresponding to the position where the light intensity sensor that collects the light intensity is located.

4. The curtain wall installation control system of the robot arm based on path planning according to claim 3 is characterized in that: The target similarity is corrected according to the light intensity, including: If the light intensity is less than the minimum light intensity, then the difference between the minimum light intensity and the light intensity is calculated and recorded as the target difference; If the light intensity is greater than the maximum light intensity, then the difference between the light intensity and the maximum light intensity is calculated and recorded as the target difference; Determine a light intensity mean value according to the light intensity minimum value and the light intensity maximum value, and calculate a ratio of the target difference value to the light intensity mean value; The target similarity is modified according to the ratio.

5. The curtain wall installation control system of the robot arm based on path planning according to claim 2, characterized in that: Also includes: three light intensity sensors; The three light intensity sensors are respectively arranged on one side of the first camera, the second camera and the third camera, and are used to collect the light intensity in the image acquisition direction of the first camera, the second camera and the third camera and send it to the controller; The controller is used to: Before calculating the first similarity, the second similarity and the third similarity, each of the illumination intensities is compared with a minimum illumination intensity and a maximum illumination intensity respectively; If a certain light intensity is less than the minimum light intensity value or greater than the maximum light intensity value, the first similarity, the second similarity or the third similarity corresponding to the position where the light intensity sensor collecting the light intensity is located is recorded as the target similarity; Determining whether the number of target similarities is one; If the number of the target similarities is one, the confidence of the target similarities is corrected according to the light intensity; Obtaining the similarity between the environment image and the teaching environment image according to the first similarity, the second similarity, the third similarity and the fourth similarity includes: If the confidence of the corrected target similarity is lower than the confidence threshold, the similarity between the environment image and the teaching environment image is obtained according to the similarities other than the target similarity among the first similarity, the second similarity and the third similarity and the fourth similarity.

6. The curtain wall installation control system of the robot arm based on path planning according to claim 5, characterized in that: After determining whether the number of target similarities is one, the method further includes: If the number of the target similarities is greater than one, adjusting the image acquisition direction of the camera corresponding to each target similarity; The illumination intensity in the adjusted image acquisition direction is re-acquired until the number of target similarities is less than or equal to one.

7. The curtain wall installation control system of the robot arm based on path planning according to claim 2, characterized in that: Determining the current working space of the robot arm according to the left front environment image, the right front environment image, and the upper oblique environment image includes: According to the left front environment image, the right front environment image and the oblique upper environment image, identifying the building structure type where the curtain wall installation frame corresponding to the glass curtain wall is located and the location and type of the safety protection facilities around the robotic arm; The current working space of the robotic arm is determined according to the building structure type where the curtain wall installation frame is located and the location and type of the safety protection facilities.

8. The curtain wall installation control system of the robot arm based on path planning according to claim 1, characterized in that: It also includes a wind speed sensor, which is used to collect the ambient wind speed at the location of the mechanical arm and send it to the controller; The controller is used to: Determining whether the ambient wind speed is greater than a wind speed threshold; If the ambient wind speed is greater than the wind speed threshold, the robotic arm is controlled to move the glass curtain wall to a preset safe position and stop working.

9. The curtain wall installation control system of the robot arm based on path planning according to claim 1, characterized in that: It also includes at least one electric suction cup and a suction cup frame, wherein the suction cup frame is fixedly mounted on the mechanical arm, and the at least one electric suction cup is fixedly mounted on the suction cup frame; The controller is also used to send a grabbing instruction to the at least one electric suction cup to control the at least one electric suction cup to grab the glass curtain wall.

10. A curtain wall installation control method for a robot arm based on path planning, applied to a visual control system in a curtain wall installation control system for a robot arm based on path planning as claimed in any one of claims 1 to 9, characterized in that: The control method comprises: Obtaining the initial position information and initial posture information of the mechanical arm on the robot car at the starting point of the operation task, and obtaining the environment image where the mechanical arm is located; Determining an initial planning path of the robotic arm according to the initial position information, the initial posture information and a preset teaching trajectory library; Calculating the similarity between the environment image and the teaching environment image; The target planning path of the robot arm is determined according to the similarity and the initial planning path, so that after the robot arm grabs the glass curtain wall, the glass curtain wall is moved to an operation task end point close to the curtain wall installation frame according to the target planning path.

Images