Screw locking control method and device and screw locking robot

Abstract

The invention relates to a screw locking control method and device and a screw locking robot, the screw locking robot comprises a movable chassis, a mechanical arm, a force sensor, a screw locking device and a depth camera, the force sensor, the screw locking device and the depth camera are installed on the mechanical arm, and the force sensor is installed on the screw locking device; the method comprises the steps of navigating a screw locking robot to a position corresponding to a first coordinate based on the first coordinate of a target screw hole in a construction drawing; based on the depth camera, the tail end of the mechanical arm is moved to the locking starting position of the target screw hole, and based on a pre-obtained locking plane normal vector, the posture of the tail end of the mechanical arm is adjusted to be consistent with the locking plane normal vector; the screw locking device is controlled to start the locking process, in the locking process, the screw position of each control period and the contact force of the screw locking device are obtained to serve as feedback items of a second-order closed-loop system, closed-loop control is executed, and the problem that the screw locking force is too large or too small in the photovoltaic module installation process is solved.

Description

Technical Field

[0001] The present application relates to the technical field of photovoltaic equipment, and particularly relates to a screw tightening control method, device, and screw-locking robot. Background Art

[0002] During the construction of a photovoltaic power station, a large number of photovoltaic modules need to be fixed to pre-designed positions by screws. Currently, a photovoltaic installation robotic arm or a movable photovoltaic power station construction robot is generally used to assist in the installation of photovoltaic modules through visual positioning technology. However, during the installation process, due to certain errors in visual positioning, when the robot controls the tightening of screws according to the identified position, problems such as insufficient tightening force of the screws that cannot be tightened, excessive tightening force that damages the photovoltaic modules, or broken screws may occur, thereby affecting the installation quality and efficiency of the photovoltaic modules.

[0003] Regarding the problem of excessive or insufficient screw tightening force during the installation of photovoltaic modules in related technologies, no effective solution has been proposed yet. Summary of the Invention

[0004] In this embodiment, a screw tightening control method, device, and screw-locking robot are provided to solve the problem of excessive or insufficient screw tightening force during the installation of photovoltaic modules in related technologies.

[0005] In the first aspect, in this embodiment, a screw tightening control method is provided. The method is applied to a screw-locking robot, which includes a mobile chassis, a robotic arm disposed on the mobile chassis, and a force sensor, a screw tightening device, and a depth camera installed on the robotic arm. The force sensor is installed on the screw tightening device. The method includes:

[0006] Based on the first coordinate of the target screw hole in the construction drawing, navigate the screw-locking robot to the position corresponding to the first coordinate;

[0007] Based on the depth camera, move the end of the robotic arm to the starting position of locking the target screw hole, and adjust the end attitude of the robotic arm based on the pre-acquired locking plane normal vector to make the end attitude of the robotic arm consistent with the locking plane normal vector;

[0008] Control the screw tightening device to start the tightening process, and during the tightening process, obtain the screw position and the contact force of the screw tightening device in each control cycle as feedback terms of a second-order closed-loop system, and perform closed-loop control.

[0009] In some of these embodiments, the obtaining the screw expected position and the contact force as feedback terms of a second-order closed-loop system and performing closed-loop control includes:

[0010] Obtain the error force of the current control cycle based on the screw position, the contact force, and the position of the target screw hole in the current control cycle;

[0011] Obtain the expected screw position of the next control cycle based on the error force of the current control cycle and the error force of the previous control cycle;

[0012] In the next control cycle, control the moving distance of the screw locking device based on the expected screw position;

[0013] Repeat the above cycle during the locking process until the screw moves to the desired position of the target screw hole.

[0014] In some embodiments, the obtaining the error force of the current control cycle based on the screw position, the contact force, and the position of the target screw hole in the current control cycle includes the following methods:

[0015]

[0016] Wherein, e t is the error force of the current control cycle t, F ext is the contact force, x t is the screw position of the current control cycle t, x 0 is the desired position of the target screw hole, M d , D d , H d are predetermined desired parameters, and M d , D d , H d are all greater than 0.

[0017] In some embodiments, the obtaining the expected screw position of the next control cycle based on the error force of the current control cycle and the error force of the previous control cycle includes:

[0018] x t+1 = K p e t + K d (e t - e t-1 )

[0019] Wherein, e t is the error force of the current control cycle t, e t-1 is the error force of the previous control cycle t - 1, x t+1 is the expected screw position of the next control cycle t + 1, K p , K d are predetermined PD controller parameters.

[0020] In some of these embodiments, obtaining the expected position of the screw, the position of the screw, and the contact force of the screw locking device in each control cycle during the locking process includes:

[0021] Based on the force sensor, obtaining the contact force of the screw locking device in each control cycle;

[0022] Based on the starting position of screwing and the moving distance of the screw locking device in each control cycle, obtaining the position of the screw in the current control cycle.

[0023] In some of these embodiments, moving the end of the robotic arm to the starting position of screwing the target screw hole based on the depth camera includes:

[0024] Based on the depth camera, obtaining the second coordinate of the target screw hole in the body coordinate system of the screw locking robot;

[0025] Based on the second coordinate, obtaining the starting position of screwing the target screw hole;

[0026] Moving the end of the robotic arm to the starting position of screwing.

[0027] In some of these embodiments, obtaining the second coordinate of the target screw hole in the body coordinate system of the screw locking robot based on the depth camera includes:

[0028] Based on the depth camera, collecting an image of the screw hole and obtaining the third coordinate of the target screw hole in the screw hole image;

[0029] Based on the internal parameter matrix and external parameter matrix of the depth camera, performing conversion between the body coordinate system and the pixel coordinate system to obtain the second coordinate corresponding to the third coordinate.

[0030] In some of these embodiments, navigating the screw locking robot to the position corresponding to the first coordinate based on the first coordinate of the target screw hole in the construction drawing includes:

[0031] Based on the pre-obtained conversion relationship between the navigation map coordinate system and the construction drawing coordinate system, and the first coordinate, obtaining the fourth coordinate of the target screw hole in the navigation map coordinate system;

[0032] Navigating the screw locking robot to the position corresponding to the fourth coordinate.

[0033] Second aspect, in this embodiment, a screw tightening control device is provided. The device is applied to a screw-locking robot, which includes a mobile chassis, a robotic arm disposed on the mobile chassis, and a force sensor, a screw tightening device, and a depth camera installed on the robotic arm. The force sensor is installed on the screw tightening device. The device includes:

[0034] A navigation module, configured to navigate the screw-locking robot to the position corresponding to the first coordinate based on the first coordinate of the target screw hole in the construction drawing;

[0035] A movement module, configured to move the end of the robotic arm to the starting position of screwing the target screw hole based on the depth camera, and adjust the posture of the end of the robotic arm based on the pre-acquired normal vector of the screwing plane to make the posture of the end of the robotic arm consistent with the normal vector of the screwing plane;

[0036] A control module, configured to control the screw tightening device to start the tightening process, and obtain the screw position and the contact force of the screw tightening device in each control cycle as feedback items of a second-order closed-loop system during the tightening process, and perform closed-loop control.

[0037] Third aspect, in this embodiment, a screw-locking robot is provided. The screw-locking robot includes a mobile chassis, a robotic arm disposed on the mobile chassis, and a force sensor, a screw tightening device, and a depth camera installed on the robotic arm. The force sensor is installed on the screw tightening device, and the screw tightening control device described in the second aspect.

[0038] Compared with the related technologies, in the screw tightening control method provided in this embodiment, based on the first coordinate of the target screw hole in the construction drawing, the screw-locking robot is navigated to the position corresponding to the first coordinate, and the robot is moved to the position where the target screw hole is located; based on the depth camera, the end of the robotic arm is moved to the starting position of screwing the target screw hole, and the attitude of the end of the robotic arm is adjusted based on the pre-acquired normal vector of the screwing plane to make the attitude of the end of the robotic arm consistent with the normal vector of the screwing plane. The screw tightening device is moved to the preparatory position for tightening the screw hole through visual positioning, and the angle of the screw tightening device is perpendicular to the screw hole; by controlling the screw tightening device to start the tightening process, the position of the screw and the contact force of the screw tightening device in each control cycle are obtained as the feedback terms of the second-order closed-loop system, and closed-loop control is performed. During the entire tightening process, the contact force and the screw position are measured in real time, and the measurement data is input into the closed-loop feedback system to correct the moving distance of the screw tightening device during the tightening process, so that the moving distance of the screw tightening device in each control cycle changes with the change of the contact force, solving the problem of excessive or too small screw tightening force during the installation of photovoltaic modules, avoiding damage to the photovoltaic modules, and improving the installation efficiency and installation success rate of the photovoltaic modules.

[0039] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more concise and understandable. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings described herein are used to provide a further understanding of the present application and form a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0041] Figure 1 is a hardware structure block diagram of the screw-locking robot according to some embodiments of the present application;

[0042] Figure 2 is a flowchart of the screw tightening control method according to some embodiments of the present application;

[0043] Figure 3 is a schematic diagram of the second-order control model according to some embodiments of the present application;

[0044] Figure 4 is a flowchart of the screw tightening device performing closed-loop control according to some embodiments of the present application;

[0045] Figure 5 is a flowchart of obtaining the contact force and the screw position according to some embodiments of the present application;

[0046] Figure 6 is a flowchart of moving the robotic arm to the starting position of screwing according to some embodiments of the present application;

[0047] Figure 7 It is a flowchart of obtaining the second coordinates of the target screw hole in some embodiments of the present application;

[0048] Figure 8 It is a flowchart of the screw-locking robot of the present application navigating to the target screw hole;

[0049] Figure 9 It is a flowchart of the screw-locking control method in some preferred embodiments of the present application;

[0050] Figure 10 It is a structural block diagram of the screw-locking control device in some embodiments of the present application. Detailed implementation manners

[0051] For a clearer understanding of the purpose, technical solutions and advantages of the present application, the present application will be described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0052] Unless otherwise defined, the technical terms or scientific terms involved in the present application shall have the general meaning understood by those with ordinary skills in the technical field to which the present application belongs. In the present application, words such as "a", "one", "a kind of", "the", "these" and the like do not represent a limitation in quantity, and they can be singular or plural. The terms "including", "comprising", "having" and any variants thereof involved in the present application are intended to cover non-exclusive inclusion; for example, a process, method, system, product or device including a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent in these processes, methods, products or devices. The terms "connected", "coupled" and the like involved in the present application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The term "plurality" involved in the present application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" may represent: A exists alone, A and B exist simultaneously, and B exists alone. Usually, the character " / " represents an "or" relationship between the associated objects before and after. The terms "first", "second", "third", etc. involved in the present application are only used to distinguish similar objects and do not represent a specific order for the objects.

[0053] The screw-locking control method provided by the embodiments of the present application can be executed on a screw-locking robot or other movable robotic arms. Figure 1It is a hardware structure block diagram of a screw-locking robot according to some embodiments of the present application. As Figure 1 shown, the screw-locking robot includes a mobile chassis 10, a robotic arm 11 disposed on the mobile chassis 10, and a force sensor 12, a screw-locking device 13, and a depth camera 14 mounted on the robotic arm 11. Among them, the force sensor 12 is mounted on the screw-locking device 13 for measuring the contact force of the screw-locking device 13 during the locking process; the depth camera 14 is used to collect screw hole images for visual positioning of the target screw hole.

[0054] Specifically, the robotic arm 11 can be a six-degree-of-freedom robotic arm, which can adjust the position and angle of the screw-locking device 13. The screw-locking device 13 is used to control the alignment of the screw hole and the installation of the screw. The screw-locking device 13 can include an electric screwdriver with an automatically adjustable rotation speed.

[0055] Those of ordinary skill in the art can understand that Figure 1 the structure shown is only schematic and does not limit the structure of the above screw-locking robot. For example, the robotic arm may further include more components than those shown in Figure 1 , or the robotic arm may have a different moving mode from the mobile chassis shown in Figure 1 , which is not limited in this embodiment.

[0056] In this embodiment, a screw-locking control method is provided. Figure 2 It is a flowchart of the screw-locking control method according to some embodiments of the present application. As Figure 2 shown, the process includes the following steps:

[0057] Step S201, based on the first coordinate of the target screw hole in the construction drawing, navigate the screw-locking robot to the position corresponding to the first coordinate.

[0058] The construction drawing can be the installation design drawing of the photovoltaic module, drawn according to the construction drawing coordinate system, and the construction drawing coordinate system can be a two-dimensional plane coordinate system. The construction drawing includes the target screw hole, and the first coordinate of the target screw hole in the construction drawing coordinate system can be obtained from the construction drawing.

[0059] The screw-locking robot can pre-obtain the conversion relationship between the construction drawing coordinate system and the navigation map coordinate system, convert the first coordinate into the corresponding coordinate in the navigation map coordinate system, and then move the screw-locking robot to the position where the coordinate is located through the mobile chassis.

[0060] Among them, the navigation map coordinate system refers to the coordinate system established by the screw-locking robot for map construction and positioning through SLAM or other technologies. Specifically, the screw-locking robot can establish a navigation map coordinate system based on its own depth camera and / or lidar. The positioning and map construction technologies involved are relatively mature and will not be elaborated in this embodiment. The navigation map coordinate system can be a two-dimensional plane coordinate system. According to the positioning points jointly included in the navigation map coordinate system and the construction drawing coordinate system, a coordinate transformation equation can be established.

[0061] Step S202: Based on the depth camera, move the end of the robotic arm to the starting position for locking the target screw hole, and adjust the attitude of the end of the robotic arm based on the pre-acquired normal vector of the locking plane to make the attitude of the end of the robotic arm consistent with the normal vector of the locking plane.

[0062] After the screw-locking robot moves to the position where the target screw hole is located, an image of this position can be collected through the depth camera, and the target screw hole is included in the image. By performing image recognition on the collected image, the position coordinates of the target screw hole in the image can be obtained, and through the coordinate transformation between the pixel coordinate system where the image is located and the body coordinate system of the screw-locking robot, the position coordinates of the target screw hole in the body coordinate system can be obtained. According to this position coordinate, move the end of the robotic arm to the starting position corresponding to this position coordinate.

[0063] Before the tightening process starts, it is necessary to adjust the attitude of the end of the robotic arm to be perpendicular to the target screw hole. According to the image of the photovoltaic module collected by the depth camera, the plane equation of the locking plane where the photovoltaic module is located can be obtained, and then the normal vector perpendicular to the locking plane can be calculated, and the attitude of the end of the robotic arm is adjusted to be consistent with the normal vector of the locking plane. There are various different ways to obtain the normal vector of the locking plane, including through laser point cloud recognition, image feature extraction recognition, etc., which will not be elaborated in this embodiment.

[0064] Based on the normal vector of the locking plane and the starting position for locking, the pose of the end of the robotic arm (which is also the end of the screw tightening device) after tightening can be obtained. Control the end of the robotic arm to be adjusted according to this pose.

[0065] Step S203: Control the screw tightening device to start the tightening process, and obtain the screw position and the contact force of the screw tightening device in each control cycle as the feedback terms of the second-order closed-loop system, and perform closed-loop control.

[0066] The screw locking device includes an electric screwdriver that can automatically adjust the speed, and the contact force of the electric screwdriver can be measured by a force sensor. Before the locking process begins, the end of the electric screwdriver is located at the starting position of the locking, and the contact force is 0. When the locking process starts, the electric screwdriver moves forward driven by the motor and rotates at a predetermined speed. The entire locking process can be divided into multiple control cycles, and the speed of the electric screwdriver can be adjusted in each control cycle. According to the speed of the electric screwdriver and the time of the control cycle, the movement distance of the electric screwdriver within the control cycle can be calculated, thereby obtaining the screw position of each control cycle.

[0067] This embodiment uses a second-order closed-loop system to perform closed-loop control on the entire locking process. Figure 3 is a schematic diagram of a second-order control model of some embodiments of the present application, such as Figure 3 As shown, the second-order control model includes a force feedback control module 31, a position control module 32 and a dynamic control module 33 connected in sequence. The dynamic control module 33 outputs the screw position x and the contact force F of the screw locking device in each control cycle. ext , and use it as the feedback term of the second-order closed-loop system. ext The input force feedback control module 31 inputs the screw position x into the position control module 32. The force feedback control module 31 inputs the screw position x into the position control module 32. 0 and contact force F ext , output position adjustment x d The position control module 32 adjusts the position x d Modify the pre-set screw position trajectory to achieve screw position control.

[0068] Through steps S201 to S203, the screw-locking robot is navigated to the position corresponding to the first coordinate based on the first coordinate of the target screw hole in the construction drawing, and the robot is moved to the position where the target screw hole is located; based on the depth camera, the end of the robotic arm is moved to the starting position of locking the target screw hole, and the attitude of the end of the robotic arm is adjusted based on the pre-acquired locking plane normal vector to make the attitude of the end of the robotic arm consistent with the locking plane normal vector. The screw-locking device is moved to the preparatory position for locking the target screw hole through visual positioning, and the angle of the screw-locking device is perpendicular to the target screw hole; by controlling the screw-locking device to start the locking process, the screw position and the contact force of the screw-locking device in each control cycle are obtained as feedback terms of the second-order closed-loop system, and closed-loop control is performed. During the entire locking process, the contact force and the screw position are measured in real time, and the measurement data is input into the closed-loop feedback system to correct the moving distance of the screw-locking device during the locking process, so that the moving distance of the screw-locking device in each control cycle changes with the change of the contact force, solving the problem of excessive or too small screw locking force during the installation of photovoltaic modules, avoiding damage to the photovoltaic modules, and improving the installation efficiency and installation success rate of the photovoltaic modules.

[0069] In some embodiments, Figure 4 is a flowchart of the screw-locking device of some embodiments of the present application performing closed-loop control, as Figure 4 shown, this process includes the following steps:

[0070] Step S401, based on the screw position, contact force, and position of the target screw hole in the current control cycle, obtain the error force in the current control cycle.

[0071] Specifically, let the current control cycle be t, and the error force in the current control cycle can be calculated by the following formula:

[0072]

[0073] where, e t is the error force in the current control cycle t, F ext is the contact force, x t is the screw position in the current control cycle t, x 0 is the expected position of the target screw hole, M d , D d , H d are pre-determined expected parameters, and M d , D d , H d are all greater than 0.

[0074] where represents the first derivative of x t , Represents the second derivative of xt. Based on the distance between the screw and the target screw hole in the current control cycle t, and the contact force of the screw locking device, the error force in the current control cycle is calculated. M d 、D d 、H d The values are obtained in advance through debugging.

[0075] Step S402, based on the error force in the current control cycle and the error force in the previous control cycle, obtain the expected position of the screw in the next control cycle.

[0076] Specifically, the expected position of the screw in the next control cycle t + 1 can be calculated by the following formula:

[0077] x t-1 =K p e t +K d (e t -e t-1 )

[0078] Where, e t is the error force in the current control cycle t, e t-1 is the error force in the previous control cycle t - 1, x t+1 is the expected position of the screw in the next control cycle t + 1, K p 、K d are the PD controller parameters determined in advance.

[0079] The expected position of the screw in the next control cycle is calculated through PID closed-loop control, where K p is the proportional parameter, and K d is the differential parameter.

[0080] Step S403, in the next control cycle, control the moving distance of the screw locking device based on the expected position of the screw.

[0081] The direction of the moving distance is the direction of the normal vector of the locking plane. According to the expected position of the screw x t+1 , control the moving distance of the screw locking device in the next control cycle, that is, adjust the rotation speed of the electric screwdriver.

[0082] Step S404, repeat the above cycle during the locking process until the screw moves to the desired position of the target screw hole.

[0083] Repeat the above steps S401~S403 in each control cycle. During the locking process, as the screw locking device moves, the error force e t approaches 0 until the end of the screw locking device moves to the desired position x 0 of the target screw hole.

[0084] Through steps S401 to S404, based on the screw position, contact force, and the position of the target screw hole in the current control cycle, the error force in the current control cycle is obtained. Based on the error force in the current control cycle and the error force in the previous control cycle, the expected position of the screw in the next control cycle is obtained, and based on the expected position of the screw, the moving distance of the screw locking device is controlled in the next control cycle, and the above steps are repeated, so that the moving distance of the screw locking device in each control cycle changes with the change of the contact force, and finally the error force is zero, avoiding impact or damage to the photovoltaic module during the screw locking process, and improving the installation efficiency and installation success rate of the photovoltaic module.

[0085] In some embodiments, Figure 5 is a flowchart for obtaining the contact force and screw position in some embodiments of the present application, as Figure 5 shown, the process includes the following steps:

[0086] Step S501, based on the force sensor, obtain the contact force of the screw locking device in each control cycle.

[0087] Through the force sensor installed on the robotic arm, the contact force of the screw locking device can be obtained in real time. This contact force is the resistance generated when the end of the screw locking device contacts the screw or other objects during the locking process. When the contact force is 0, it can be considered that the end of the screw locking device does not contact the screw; when the contact force is too large, it indicates that the screw has been locked or an abnormality has occurred during the locking process. By measuring the contact force, the contact state between the electric screwdriver and the screw can be detected.

[0088] Step S502, based on the starting position of locking and the moving distance of the screw locking device in each control cycle, obtain the screw position in the current control cycle.

[0089] The starting position of locking is the position where the end of the screw locking device is located before the locking process starts, and the distance between the starting position of locking and the position of the target screw hole is pre-determined. And the rotation speed of the electric screwdriver in each control cycle is known, and the moving distance in each control cycle can be calculated, and based on this, the screw position coordinates in each control cycle can be calculated.

[0090] Through steps S501 to S502, based on the force sensor, obtain the contact force of the screw locking device in each control cycle, based on the starting position of locking and the moving distance of the screw locking device in each control cycle, obtain the screw position in the current control cycle, and use the contact force and screw position as the feedback terms of the second-order closed-loop system to correct the moving distance of the screw locking device during the locking process, improving the control accuracy of the locking force and moving distance during the locking process, avoiding damage to the photovoltaic module, and improving the installation efficiency and installation success rate of the photovoltaic module.

[0091] In some embodiments,Figure 6 is a flowchart of a mobile robotic arm moving to the starting position for screwing in some embodiments of the present application. As Figure 6 shown, the process includes the following steps:

[0092] Step S601: Based on the depth camera, obtain the second coordinate of the target screw hole in the body coordinate system of the screw-locking robot.

[0093] Collect an image containing the target screw hole through the depth camera, and identify the position coordinates of the target screw hole in the image through an image recognition algorithm. The position coordinates can be two-dimensional plane coordinates. Based on the internal and external parameters of the depth camera, establish a coordinate transformation equation between the pixel coordinate system where the image is located and the body coordinate system of the screw-locking robot. According to this coordinate transformation equation and the position coordinates of the target screw hole in the pixel coordinate system, calculate the second coordinate of the target screw hole in the body coordinate system. The second coordinate can be three-dimensional coordinates.

[0094] Step S602: Based on the second coordinate, obtain the starting position for screwing the target screw hole.

[0095] According to the pre-determined distance and orientation between the starting position for screwing and the target screw hole, obtain the starting position for screwing the target screw hole. In a specific embodiment, the starting position for screwing is 20 mm below the second coordinate. That is, assuming that the coordinate axes in the three-dimensional coordinate system are the horizontal, vertical, and depth directions respectively, and the second coordinate of the target screw hole is (x, y, z), then the coordinates of the starting position for screwing are (x, y - 20 mm, z).

[0096] Step S603: Move the end of the robotic arm to this starting position for screwing.

[0097] Through the robotic arm control system, move the end of the robotic arm to this starting position for screwing, and adjust the end attitude of the robotic arm to be perpendicular to the target screw hole according to the normal vector of the screwing plane.

[0098] Through steps S601 to S603, by using the depth camera, obtain the second coordinate of the target screw hole in the body coordinate system of the screw-locking robot, which facilitates the positioning of the target screw hole by the screw-locking robot; by using the second coordinate, obtain the starting position for screwing the target screw hole and move the end of the robotic arm to this starting position for screwing, which improves the positioning accuracy of the target screw hole.

[0099] In some embodiments, Figure 7 is a flowchart of obtaining the second coordinate of the target screw hole in some embodiments of the present application. As Figure 7 shown, the process includes the following steps:

[0100] Step S701: Collect the screw hole image based on the depth camera and obtain the third coordinate of the target screw hole in the screw hole image.

[0101] The pixel coordinate system is the coordinate system of the screw hole image. Through image recognition, obtain the third coordinate (u, v) of the target screw hole in the pixel coordinate system. This embodiment does not limit the image recognition algorithm.

[0102] Step S702: Based on the internal parameter matrix and external parameter matrix of the depth camera, perform the conversion between the body coordinate system and the pixel coordinate system to obtain the second coordinate corresponding to the third coordinate.

[0103] Assume that the internal parameter matrix of the depth camera is K. Then, according to the internal parameter matrix K, the third coordinate (u, v) can be converted into the normalized coordinate (x_c, y_c, z_c) in the camera coordinate system. The calculation formula is as follows:

[0104]

[0105] Assume that the external parameter matrix of the depth camera is [R|T], where R is the rotation matrix and T is the translation vector. According to the external parameter matrix, the normalized coordinate (x_c, y_c, z_c) in the camera coordinate system can be converted into the second coordinate (x_r, y_r, z_r) in the body coordinate system of the screw-locking robot. The calculation formula is as follows:

[0106]

[0107] Through steps S701 - S702, by collecting the screw hole image based on the depth camera, obtaining the third coordinate of the target screw hole in the screw hole image, and performing the conversion between the body coordinate system and the pixel coordinate system based on the internal parameter matrix and external parameter matrix of the depth camera to obtain the second coordinate corresponding to the third coordinate, the coordinate conversion of the target screw hole from the planar image to the three-dimensional space is realized, providing accurate positioning for subsequent screw locking.

[0108] In some embodiments, Figure 8 is the flowchart of the screw-locking robot of some embodiments of the present application navigating to the target screw hole. As Figure 8 shown, this process includes the following steps:

[0109] Step S801: Based on the conversion relationship between the navigation map coordinate system and the construction drawing coordinate system obtained in advance, and the first coordinate, obtain the fourth coordinate of the target screw hole in the navigation map coordinate system.

[0110] Through the construction drawing of the photovoltaic power station, the first coordinate (x p , y p). Through the conversion relationship between the navigation map coordinate system and the construction drawing coordinate system, the first coordinate (x p , y p ) can be converted into the fourth coordinate (X p , Y p ) in the navigation map coordinate system, and its conversion equation is as follows:

[0111] x p = (X p - X o ) · cos(θ) + (Y p - Y o ) · sin(θ) + x o

[0112] y p = (X p - X o ) · sin(θ) + (Y p - Y o ) · cos(θ) + y o

[0113] Among them, X p , Y p are the fourth coordinates of the target screw hole, x o , y o are the coordinates of the origin of the navigation map coordinate system (usually (0, 0)); X o , Y o are the coordinates corresponding to the origin of the navigation map coordinate system in the construction drawing coordinate system, and θ is the coordinate azimuth angle of the x-axis of the navigation map coordinate system in the construction drawing coordinate system.

[0114] Step S802, navigate the screw-locking robot to the position corresponding to the fourth coordinate.

[0115] Through the robot navigation system, navigate the screw-locking robot to the position corresponding to the fourth coordinate.

[0116] Through steps S801 - S802, based on the pre-obtained conversion equation between the navigation map coordinate system and the construction drawing coordinate system, and the first coordinate, obtain the fourth coordinate of the target screw hole in the navigation map coordinate system, convert the position of the target screw hole in the construction drawing to the position in the navigation map coordinate system of the screw-locking robot, which facilitates the screw-locking robot to automatically reach this position through navigation, execute the screw tightening work, improves the environmental adaptability of the screw-locking robot, and expands the application scenarios of the screw-locking robot.

[0117] The following describes and explains this embodiment through preferred embodiments. The preferred embodiments of this application are applied to a screw-locking robot, which includes a mobile chassis, a robotic arm disposed on the mobile chassis, and a force sensor, a screw-locking device, and a depth camera mounted on the robotic arm. The force sensor is mounted on the screw-locking device. Figure 9 is a flowchart of a screw-locking control method for some preferred embodiments of this application. As Figure 9 shown, this process includes the following steps:

[0118] Step S901, obtain the coordinate point p1(x p , y p ) of the target screw hole in the construction drawing;

[0119] Step S902, based on the conversion equation between the navigation map coordinate system and the construction drawing coordinate system obtained in advance, convert the coordinate point p1 into the coordinate point p2(X p , Y p ) in the navigation map coordinate system;

[0120] Step S903, through the robot navigation system, navigate the screw-locking robot to point p2;

[0121] Step S904, start the depth camera, and identify the coordinate point a(u, v) of the target screw hole in the image through an image recognition algorithm;

[0122] Step S905, based on the internal parameter matrix K of the depth camera, convert the coordinate point a(u, v) into the normalized coordinate (x_c, y_c, z_c) in the camera coordinate system;

[0123] Step S906, based on the external parameter matrix [R|T] of the depth camera, convert the normalized coordinate (x_c, y_c, z_c) into the coordinate point b(x_r, y_r, z_r) in the robot body coordinate system;

[0124] Step S907, through the robotic arm control system, make the end of the robotic arm stay at a position 20 mm below point b, and its posture remains consistent with the normal vector of the locking plane;

[0125] Step S908, start the locking process, and based on the force sensor, obtain the contact force of the screw-locking device in the current control cycle;

[0126] Step S909, based on the locking start position and the moving distance of the screw-locking device in each control cycle, obtain the screw position in the current control cycle;

[0127] Step S910, based on the screw position, contact force, and the position of the target screw hole in the current control cycle, obtain the error force in the current control cycle;

[0128] Step S911: Based on the error force in the current control cycle and the error force in the previous control cycle, obtain the expected position of the screw in the next control cycle.

[0129] Step S912: In the next control cycle, control the moving distance of the screw locking device based on the expected position of the screw.

[0130] Step S913: Repeat steps S908 - S912 until the screw moves to the desired position in the target screw hole.

[0131] Through steps S901 - S913, the screw - locking robot is automatically navigated to the position where the target screw hole is located, improving the environmental adaptability of the screw - locking robot and expanding its application scenarios; through visual positioning, the screw locking device is moved to the preparatory position for screwing the screw hole, realizing the coordinate conversion of the target screw hole from a planar image to a three - dimensional space, providing accurate positioning for subsequent screw fastening, and the angle of the screw locking device is perpendicular to the screw hole, improving the positioning accuracy of the target screw hole; during the entire tightening process, the contact force and the screw position are measured in real - time, and the measurement data is input into the closed - loop feedback system to correct the moving distance of the screw locking device during the tightening process, so that the moving distance of the screw locking device in each control cycle changes with the change of the contact force, improving the control accuracy of the tightening force and the moving distance during the tightening process, solving the problem of excessive or insufficient screw tightening force during the installation of photovoltaic modules, avoiding impact on photovoltaic modules, and improving the installation efficiency and installation success rate of photovoltaic modules.

[0132] It should be noted that the steps shown in the above process or the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer - executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from here.

[0133] In some embodiments, the present application further provides a screw - locking control device, which is applied to a screw - locking robot. The screw - locking robot includes a mobile chassis, a robotic arm arranged on the mobile chassis, and a force sensor, a screw - locking device, and a depth camera installed on the robotic arm. The force sensor is installed on the screw - locking device. The screw - locking control device is used to implement the above - mentioned embodiments and preferred implementation manners, and those that have been described will not be repeated. The following terms such as "module", "unit", "sub - unit", etc. can be a combination of software and / or hardware that can achieve a predetermined function.

[0134] In some embodiments, Figure 10 is the structural block diagram of the screw - locking control device of some embodiments of the present application. As Figure 10 shown, the device includes:

[0135] The navigation module 1001 is configured to navigate the screw-locking robot to the position corresponding to the first coordinate based on the first coordinate of the target screw hole in the construction drawing.

[0136] The movement module 1002 is configured to move the end of the robotic arm to the starting position of screwing the target screw hole based on the depth camera, and adjust the attitude of the end of the robotic arm based on the pre-acquired normal vector of the screwing plane to make the attitude of the end of the robotic arm consistent with the normal vector of the screwing plane.

[0137] The control module 1003 is configured to control the screw tightening device to start the tightening process, and obtain the screw position and the contact force of the screw tightening device in each control cycle as the feedback items of the second-order closed-loop system during the tightening process, and perform closed-loop control.

[0138] The screw tightening control device of this embodiment navigates the screw-locking robot to the position corresponding to the first coordinate based on the first coordinate of the target screw hole in the construction drawing through the navigation module 1001, and moves the robot to the position where the target screw hole is located; through the movement module 1002, based on the depth camera, moves the end of the robotic arm to the starting position of screwing the target screw hole, and adjusts the attitude of the end of the robotic arm based on the pre-acquired normal vector of the screwing plane to make the attitude of the end of the robotic arm consistent with the normal vector of the screwing plane, and moves the screw tightening device to the preparatory position for tightening the target screw hole through visual positioning, and the angle of the screw tightening device is perpendicular to the target screw hole; through the control module 1003, controls the screw tightening device to start the tightening process, obtains the screw position and the contact force of the screw tightening device in each control cycle as the feedback items of the second-order closed-loop system during the tightening process, and performs closed-loop control. During the entire tightening process, the contact force and the screw position are measured in real time, and the measurement data is input into the closed-loop feedback system to correct the moving distance of the screw tightening device during the tightening process, so that the moving distance of the screw tightening device in each control cycle changes with the change of the contact force, solving the problem of excessive or too small screw tightening force during the installation of photovoltaic modules, avoiding damage to photovoltaic modules, and improving the installation efficiency and installation success rate of photovoltaic modules.

[0139] In some embodiments, the present application further provides a screw-locking robot, which includes a mobile chassis, a robotic arm disposed on the mobile chassis, and a force sensor, a screw tightening device and a depth camera mounted on the robotic arm. The force sensor is mounted on the screw tightening device, and the screw tightening control device in the above embodiment.

[0140] The screw-locking robot of this embodiment moves the screw-locking robot to the position where the target screw hole is located through a moving chassis, collects an image of the target screw hole through a depth camera, obtains the position of the target screw hole based on the image, moves the end of the robotic arm to the starting position of locking the target screw hole, and measures the contact force of the screw-locking device during the locking process through a force sensor. The real-time measured contact force and screw position are used as feedback terms of a second-order closed-loop system to perform closed-loop control, so that the moving distance of the screw-locking device in each control cycle changes with the change of the contact force, solving the problem of excessive or insufficient screw-locking force during the installation of photovoltaic modules, avoiding damage to the photovoltaic modules, and improving the installation efficiency and success rate of the photovoltaic modules.

[0141] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and alternative embodiments, and will not be elaborated in this embodiment.

[0142] It should be understood that the specific embodiments described here are only used to explain this application, rather than to limit it. According to the embodiments provided in this application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of this application.

[0143] Obviously, the drawings are only some examples or embodiments of this application. For those of ordinary skill in the art, this application can also be applied to other similar situations based on these drawings without creative work. Additionally, it can be understood that although the work done during this development process may be complex and time-consuming, for those of ordinary skill in the art, certain design, manufacturing, or production changes based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient disclosure of this application.

[0144] The term "embodiment" in this application means that the specific features, structures, or characteristics described in connection with the embodiment may be included in at least one embodiment of this application. The phrase appears in various positions in the specification does not necessarily mean the same embodiment, nor does it mean being independent or alternative to other embodiments and mutually exclusive. Those of ordinary skill in the art can clearly or implicitly understand that the embodiments described in this application can be combined with other embodiments without conflict.

[0145] The above-described embodiments only represent several implementation manners of this application, and their descriptions are relatively specific and detailed, but should not be construed as limiting the scope of patent protection. It should be noted that for those of ordinary skill in the art, without departing from the concept of this application, several modifications and improvements can still be made, and these all belong to the protection scope of this application. Therefore, the protection scope of this application should be subject to the appended claims.

Claims

1. A screw tightening control method, characterized in that: The method is applied to a screw locking robot, which includes a mobile chassis, a mechanical arm arranged on the mobile chassis, and a force sensor, a screw locking device and a depth camera installed on the mechanical arm, wherein the force sensor is installed on the screw locking device. The method includes: Based on the first coordinate of the target screw hole in the construction drawing, navigating the screw locking robot to a position corresponding to the first coordinate; Based on the depth camera, the end of the robotic arm is moved to the locking starting position of the target screw hole, and the end posture of the robotic arm is adjusted based on the pre-acquired locking plane normal vector, so that the end posture of the robotic arm is consistent with the locking plane normal vector; The screw locking device is controlled to start a locking process, and during the locking process, the screw position and the contact force of the screw locking device in each control cycle are obtained as feedback items of a second-order closed-loop system to perform closed-loop control.

2. The method according to claim 1, characterized in that: The screw position and the contact force of the screw locking device in each control cycle are obtained during the locking process as feedback items of a second-order closed-loop system, and the closed-loop control is performed, including: Based on the screw position, the contact force and the position of the target screw hole in the current control cycle, obtaining the error force of the current control cycle; Based on the error force of the current control cycle and the error force of the previous control cycle, obtaining the expected position of the screw in the next control cycle; In the next control cycle, controlling the moving distance of the screw locking device based on the expected position of the screw; The above cycle is repeated during the tightening process until the screw moves to the desired position of the target screw hole.

3. The method according to claim 2, characterized in that The method of obtaining the error force of the current control cycle based on the screw position, the contact force and the position of the target screw hole of the current control cycle includes the following: Among them, e t is the error force of the current control cycle t, F ext is the contact force, x t is the screw position of the current control cycle t, x0 is the expected position of the target screw hole, M d , D d , H d is a predetermined expected parameter, and M d , D d , H d Both are greater than 0.

4. The method according to claim 2, characterized in that: The obtaining of the expected screw position in the next control cycle based on the error force in the current control cycle and the error force in the previous control cycle includes: x t+1 =K p And t +K d (And t -And t-1 ) Among them, e t is the error force of the current control cycle t, e t-1 is the error force of the previous control cycle t-1, x t+1 is the expected screw position in the next control cycle t+1, K p , K d are pre-determined PD controller parameters.

5. The method according to claim 1, characterized in that The screw position and the contact force of the screw locking device in each control cycle are obtained during the locking process as feedback items of a second-order closed-loop system, and the closed-loop control is performed, including: Based on the force sensor, obtaining the contact force of the screw locking device in each control cycle; Based on the locking starting position and the moving distance of the screw locking device in each control cycle, the screw position of the current control cycle is obtained.

6. The method according to claim 1, characterized in that The step of moving the end of the robot arm to a locking start position of the target screw hole based on the depth camera comprises: Based on the depth camera, obtaining a second coordinate of the target screw hole in the body coordinate system of the screw locking robot; Based on the second coordinate, obtaining a locking starting position of the target screw hole; Move the end of the mechanical arm to the locking starting position.

7. The method according to claim 6, characterized in that The acquiring, based on the depth camera, the second coordinate of the target screw hole in the body coordinate system of the screw locking robot comprises: Capturing a screw hole image based on the depth camera, and acquiring a third coordinate of the target screw hole in the screw hole image; A conversion is performed between the body coordinate system and the pixel coordinate system based on an intrinsic parameter matrix and an extrinsic parameter matrix of the depth camera to obtain a second coordinate corresponding to the third coordinate.

8. The method according to claim 1, characterized in that: The step of navigating the screw locking robot to a position corresponding to the first coordinate of the target screw hole in the construction drawing includes: Based on the pre-acquired conversion relationship between the navigation map coordinate system and the construction drawing coordinate system, and the first coordinate, acquiring a fourth coordinate of the target screw hole in the navigation map coordinate system; Navigate the screw locking robot to the position corresponding to the fourth coordinate.

9. A screw locking control device, characterized in that: The device is applied to a screw locking robot, which includes a mobile chassis, a mechanical arm arranged on the mobile chassis, a force sensor, a screw locking device and a depth camera installed on the mechanical arm, the force sensor is installed on the screw locking device, and the device includes: A navigation module, configured to navigate the screw locking robot to a position corresponding to a first coordinate of a target screw hole in a construction drawing based on the first coordinate of the target screw hole; A moving module, used to move the end of the robotic arm to the locking starting position of the target screw hole based on the depth camera, and adjust the end posture of the robotic arm based on the pre-acquired locking plane normal vector so that the end posture of the robotic arm is consistent with the locking plane normal vector; The control module is used to control the screw locking device to start the locking process, obtain the screw position and the contact force of the screw locking device in each control cycle as feedback items of the second-order closed-loop system during the locking process, and perform closed-loop control.

10. A screw locking robot, characterized in that: The screw locking robot includes a mobile chassis, a mechanical arm arranged on the mobile chassis, and a force sensor, a screw locking device and a depth camera installed on the mechanical arm, the force sensor is installed on the screw locking device, and the screw locking control device according to claim 9.

Images