Robot power transmission line foreign matter cleaning method and system based on AI vision

Abstract

The invention discloses a robot power transmission line foreign matter cleaning method based on AI vision. The method comprises the following steps: acquiring power transmission line foreign matter feature image data; marking the foreign matter feature image data of the power transmission line to obtain a training data set; constructing an initial vision AI model based on a deep learning algorithm; training the initial visual AI model by using the training data set to obtain a visual AI model; a visual AI model is used, and according to a robot motion control strategy, actual power transmission line foreign matter cleaning is carried out. The invention also discloses a system for realizing the robot power transmission line foreign matter cleaning method based on AI vision. According to the method, the target foreign matter on the power transmission line can be quickly identified, the robot is automatically controlled to remove the foreign matter, the automation and intelligence capabilities of the robot are improved, and the working efficiency is improved.

Description

Technical Field

[0001] The present invention belongs to the field of robot control, and in particular relates to a method and system for cleaning foreign matter from a robot power transmission line based on AI vision. Background Art

[0002] With the development of science and technology, the application of robots in power systems is becoming more and more common; power companies are using robots equipped with mechanical arms to perform inspection and maintenance tasks of high-voltage power lines, reducing the need for personnel to be exposed to high-risk environments. Although the application prospects of robots in power systems are broad, there are still problems such as complex technology, high cost, poor environmental adaptability, limited data processing capabilities, and low naturalness of human-machine interaction.

[0003] Regarding a modular live-working robot with a current publication number of CN115582846A, the robot system is currently used to perform maintenance operations on power transmission lines. During actual operations, the robot has a low level of intelligence and a high level of human participation. The robot is still very dependent on humans, and there is still much room for improvement in its efficiency when performing tasks. Summary of the invention

[0004] One of the purposes of the present invention is to provide a robot power transmission line foreign body cleaning method based on AI vision, which can greatly improve the working efficiency of the dual-arm robot.

[0005] A second object of the present invention is to provide a system for implementing the AI ​​vision-based robot power transmission line foreign body cleaning method.

[0006] The present invention provides a method for cleaning foreign matter in a power transmission line using a robot based on AI vision, comprising the following steps:

[0007] S1. Acquire characteristic image data of foreign matter on the power transmission line;

[0008] S2. Marking the foreign body feature image data of the power transmission line obtained in step S1 to obtain a training data set;

[0009] S3. Build an initial visual AI model based on deep learning algorithms;

[0010] S4. Using the training data set obtained in step S2, the initial visual AI model obtained in step S3 is trained to obtain a visual AI model;

[0011] S5. Use the visual AI model obtained in step S4 to perform actual cleaning of foreign objects on the power transmission line according to the robot motion control strategy.

[0012] Step S1 specifically includes: photographing foreign objects in several different mounting postures on a preset transmission line at different angles to obtain characteristic image data of foreign objects on the transmission line.

[0013] Step S2 specifically includes: classifying and labeling the foreign object feature image data of the power transmission line obtained in step S1 according to the foreign object processing method to obtain foreign object training data; combining the foreign object training data with the posture data and depth data corresponding to the foreign object to obtain a training data set.

[0014] Step S3 is specifically as follows: based on the Yolov8 model, an initial visual AI model is constructed; the input of the initial visual AI model is the image data of the moving foreign object in front taken by the robot camera, and the output is the foreign object processing type classification result, the posture data and depth data of the foreign object.

[0015] The classification and labeling are specifically as follows: for foreign objects that can be picked up and removed by the robot gripper, the corresponding transmission line foreign object characteristic data is labeled as "foreign object removal by gripper"; for foreign objects that cannot be removed by the robot gripper, the flammability of the foreign object is judged. If the foreign object is flammable, the corresponding transmission line foreign object characteristic data is labeled as "laser foreign object removal", otherwise it is labeled as "manual foreign object removal".

[0016] Step S5 specifically includes the following steps:

[0017] a. Use the camera on the robot to obtain the image data, posture data and depth data of the robot in front of it in real time;

[0018] b. Input the image data obtained in step a into the AI ​​visual model obtained in step S4 to obtain the foreign body processing type classification result, foreign body posture data and depth data;

[0019] c. According to the foreign body handling type classification result obtained in step b and the posture data obtained in step a, the foreign body handling method is determined based on the robot arm foreign body handling strategy;

[0020] d. Based on the foreign matter treatment method obtained in step c, combined with the depth data of the foreign matter obtained in step b, the foreign matter is cleaned up.

[0021] Step c is specifically as follows:

[0022] When the classification result of the foreign body treatment type is laser foreign body removal, if the posture of the foreign body is that the foreign body is wrapped around the transmission line and the end is raised, the robot gripper approaches the end of the foreign body from the upper side of the foreign body, grabs the end of the foreign body with the gripper and straightens the foreign body, controls the other side of the robot arm to the place where the foreign body is wrapped around the transmission line, and focuses the end laser on the entanglement to perform laser burning, and finally removes the foreign body;

[0023] When the classification result of the foreign body treatment type is laser foreign body removal, if the posture of the foreign body is that the foreign body is entangled on the transmission line and the end is falling, the robot gripper approaches the end of the foreign body from the bottom of the foreign body, grabs the end of the foreign body with the gripper and straightens the foreign body, controls the other side of the robot arm to the place where the foreign body is entangled with the transmission line, and focuses the end laser on the entanglement to perform laser burning, and finally removes the foreign body;

[0024] When the foreign body treatment type classification result is laser foreign body removal, if the posture of the foreign body is that the foreign body is wrapped around the transmission line and the end is flush with the transmission line, the robot gripper approaches the end of the foreign body from a height such as the transmission line, grabs the end of the foreign body with the gripper and straightens the foreign body, controls the other side of the robot arm to the place where the foreign body is wrapped around the transmission line, and focuses the end laser on the entanglement to perform laser burning, and finally removes the foreign body;

[0025] When the foreign object handling type classification result is gripper removal of foreign objects, if the foreign object posture is a non-entangled non-flexible foreign object with a length less than the arm span of the robotic arm, the robot gripper grasps one end of the non-flexible foreign object, controls the movement of the robotic arm, and makes the foreign object separate from the power transmission line, and finally removes the foreign object;

[0026] When the foreign body handling type classification result is gripper removal of foreign bodies, if the foreign body is a non-entangled flexible foreign body whose length is less than the arm span of the robotic arm, the robot gripper grabs the flexible foreign body on either side of the hanging line position, controls the movement of the robotic arm, separates the foreign body from the transmission line, and finally removes the foreign body;

[0027] When the foreign body handling type classification result is the removal of foreign bodies by grippers, if the posture of the foreign body is a non-entangled flexible foreign body whose length is greater than the arm span of the robotic arm, the robot gripper grasps any side of the flexible foreign body close to the hanging line position, controls the movement of the robotic arm, and makes the grasped side of the foreign body longer, and repeats this step until the length of one side of the flexible foreign body is less than the arm span of the robotic arm or it automatically falls off due to gravity; when the length of one side of the flexible foreign body is less than the arm span of the robotic arm, the robot gripper grasps the corresponding side of the foreign body, controls the movement of the robotic arm, and makes the foreign body separate from the transmission line, and finally removes the foreign body;

[0028] When the classification result of the foreign object handling type is manual foreign object removal, and the posture of the foreign object is one that the robot cannot handle, an alarm is directly issued to prompt manual intervention.

[0029] Step d specifically comprises the following steps:

[0030] Obtain historical motion control data of a manually controlled robot arm performing a foreign body removal operation as a control training data set;

[0031] Based on the Deep Deterministic Policy Gradient neural network, an initial motion control decision AI model is constructed;

[0032] The initial motion control decision AI model is trained using the control training data set to obtain a motion control decision AI model;

[0033] The foreign body treatment method judgment obtained in step c and the depth data of the foreign body obtained in step b are input into the motion control decision AI model to obtain the robot foreign body cleaning motion trajectory control data;

[0034] The robot is controlled to move according to the obtained robot foreign object cleaning motion trajectory control data to complete the transmission line foreign object cleaning.

[0035] The present invention also provides a system for implementing the AI ​​vision-based robot power transmission line foreign body cleaning method, comprising a data acquisition module, a data marking module, an AI vision model construction module, an AI vision model training module, and a power transmission line foreign body cleaning module, which are sequentially connected in series;

[0036] The data acquisition module acquires characteristic image data of foreign matter on the power transmission line and uploads the data to the data marking module;

[0037] The data labeling module labels the transmission line foreign body feature image data based on the received data, obtains the training data set, and uploads the data to the AI ​​vision model building module;

[0038] The AI ​​vision model building module builds the initial visual AI model based on the deep learning algorithm and uploads the data to the AI ​​vision model training module;

[0039] The AI ​​visual model training module trains the initial visual AI model according to the received data to obtain the visual AI model, and uploads the data to the transmission line foreign matter cleaning module;

[0040] The transmission line foreign matter cleaning module performs actual transmission line foreign matter cleaning according to the received data and the robot motion control strategy.

[0041] The present invention discloses a method and system for cleaning foreign objects on power transmission lines by a robot based on AI vision, which can quickly identify target foreign objects on the power transmission lines and automatically control the robot to clean them, thereby improving the automation and intelligence capabilities of the robot and improving operating efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a schematic diagram of the process of the present invention;

[0043] Figure 2 It is a structural schematic diagram of the system of the present invention;

[0044] Figure 3 This is a diagram showing the actual application effect when the classification result of the foreign matter treatment type in the method of the present invention is laser foreign matter removal;

[0045] Figure 4 This is a diagram showing the actual application effect of the foreign body handling type classification result of the method of the present invention when the hand claw removes foreign bodies. DETAILED DESCRIPTION

[0046] The present invention provides a method for cleaning foreign matter in power transmission lines using a robot based on AI vision, the flow chart of which is as follows: Figure 1 As shown, the following steps are included:

[0047] S1. Obtaining characteristic image data of foreign matter on the power transmission line, specifically:

[0048] Foreign objects in several different mounting postures on the preset transmission line are photographed at different angles to obtain characteristic image data of foreign objects on the transmission line.

[0049] S2. Label the transmission line foreign body feature image data obtained in step S1 to obtain a training data set, specifically:

[0050] The transmission line foreign object feature image data obtained in step S1 is classified and labeled according to the foreign object processing method to obtain foreign object training data; the foreign object training data is combined with the posture data and depth data corresponding to the foreign object to obtain a training data set.

[0051] The classification and labeling are specifically as follows: for foreign objects that can be picked up and removed by the robot gripper, the corresponding transmission line foreign object characteristic data is labeled as "foreign object removal by gripper"; for foreign objects that cannot be removed by the robot gripper, the flammability of the foreign object is judged. If the foreign object is flammable, the corresponding transmission line foreign object characteristic data is labeled as "laser foreign object removal", otherwise it is labeled as "manual foreign object removal".

[0052] S3. Based on the deep learning algorithm, build the initial visual AI model, specifically:

[0053] Based on the Yolov8 model, an initial visual AI model is constructed; the input of the initial visual AI model is the image data of foreign objects in front of the robot taken by the robot camera, and the output is the classification result of the foreign object processing type, the posture data and depth data of the foreign object.

[0054] S4. Using the training data set obtained in step S2, the initial visual AI model obtained in step S3 is trained to obtain a visual AI model;

[0055] S5. Using the visual AI model obtained in step S4, according to the robot motion control strategy, actual transmission line foreign body cleaning is performed, specifically including the following steps:

[0056] a. Use the camera on the robot to obtain the image data, posture data and depth data of the robot in front of it in real time;

[0057] b. Input the image data obtained in step a into the AI ​​visual model obtained in step S4 to obtain the foreign body processing type classification result, foreign body posture data and depth data;

[0058] c. According to the foreign body handling type classification result obtained in step b and the posture data obtained in step a, the foreign body handling method is determined based on the robot arm foreign body handling strategy;

[0059] d. Based on the foreign matter treatment method obtained in step c, combined with the depth data of the foreign matter obtained in step b, the foreign matter is cleaned up.

[0060] Step c is specifically as follows:

[0061] When the classification result of the foreign body treatment type is laser foreign body removal, if the posture of the foreign body is that the foreign body is wrapped around the transmission line and the end is raised, the robot gripper approaches the end of the foreign body from the upper side of the foreign body, grabs the end of the foreign body with the gripper and straightens the foreign body, controls the other side of the robot arm to the place where the foreign body is wrapped around the transmission line, and focuses the end laser on the entanglement to perform laser burning, and finally removes the foreign body;

[0062] When the classification result of the foreign body treatment type is laser foreign body removal, if the posture of the foreign body is that the foreign body is entangled on the transmission line and the end is falling, the robot gripper approaches the end of the foreign body from the bottom of the foreign body, grabs the end of the foreign body with the gripper and straightens the foreign body, controls the other side of the robot arm to the place where the foreign body is entangled with the transmission line, and focuses the end laser on the entanglement to perform laser burning, and finally removes the foreign body;

[0063] When the foreign body treatment type classification result is laser foreign body removal, if the posture of the foreign body is that the foreign body is wrapped around the transmission line and the end is flush with the transmission line, the robot gripper approaches the end of the foreign body from a height such as the transmission line, grabs the end of the foreign body with the gripper and straightens the foreign body, controls the other side of the robot arm to the place where the foreign body is wrapped around the transmission line, and focuses the end laser on the entanglement to perform laser burning, and finally removes the foreign body;

[0064] When the foreign object handling type classification result is gripper removal of foreign objects, if the foreign object posture is a non-entangled non-flexible foreign object with a length less than the arm span of the robotic arm, the robot gripper grasps one end of the non-flexible foreign object, controls the movement of the robotic arm, and makes the foreign object separate from the power transmission line, and finally removes the foreign object;

[0065] When the foreign body handling type classification result is gripper removal of foreign bodies, if the foreign body is a non-entangled flexible foreign body whose length is less than the arm span of the robotic arm, the robot gripper grabs the flexible foreign body on either side of the hanging line position, controls the movement of the robotic arm, separates the foreign body from the transmission line, and finally removes the foreign body;

[0066] When the foreign body handling type classification result is the removal of foreign bodies by grippers, if the posture of the foreign body is a non-entangled flexible foreign body whose length is greater than the arm span of the robotic arm, the robot gripper grasps any side of the flexible foreign body close to the hanging line position, controls the movement of the robotic arm, and makes the grasped side of the foreign body longer, and repeats this step until the length of one side of the flexible foreign body is less than the arm span of the robotic arm or it automatically falls off due to gravity; when the length of one side of the flexible foreign body is less than the arm span of the robotic arm, the robot gripper grasps the corresponding side of the foreign body, controls the movement of the robotic arm, and makes the foreign body separate from the transmission line, and finally removes the foreign body;

[0067] When the classification result of the foreign object handling type is manual foreign object removal, and the posture of the foreign object is one that the robot cannot handle, an alarm is directly issued to prompt manual intervention.

[0068] Step d specifically comprises the following steps:

[0069] Obtain historical motion control data of a manually controlled robot arm performing a foreign body removal operation as a control training data set;

[0070] Based on the Deep Deterministic Policy Gradient neural network, an initial motion control decision AI model is constructed; the DDPG neural network is an existing model;

[0071] The initial motion control decision AI model is trained using the control training data set to obtain a motion control decision AI model;

[0072] The foreign body treatment method judgment obtained in step c and the depth data of the foreign body obtained in step b are input into the motion control decision AI model to obtain the robot foreign body cleaning motion trajectory control data;

[0073] The robot is controlled to move according to the obtained robot foreign object cleaning motion trajectory control data to complete the transmission line foreign object cleaning.

[0074] Figure 3 This is a diagram showing the effect of the actual application of the method of the present invention.

[0075] The present invention also provides a system for implementing the AI ​​vision-based robot power transmission line foreign body cleaning method, comprising a data acquisition module, a data marking module, an AI vision model construction module, an AI vision model training module, and a power transmission line foreign body cleaning module, which are sequentially connected in series;

[0076] The data acquisition module acquires characteristic image data of foreign matter on the power transmission line and uploads the data to the data marking module;

[0077] The data labeling module labels the transmission line foreign body feature image data based on the received data, obtains the training data set, and uploads the data to the AI ​​vision model building module;

[0078] The AI ​​vision model building module builds the initial visual AI model based on the deep learning algorithm and uploads the data to the AI ​​vision model training module;

[0079] The AI ​​visual model training module trains the initial visual AI model according to the received data to obtain the visual AI model, and uploads the data to the transmission line foreign matter cleaning module;

[0080] The transmission line foreign matter cleaning module performs actual transmission line foreign matter cleaning according to the received data and the robot motion control strategy.

Claims

1. A robot power transmission line foreign body cleaning method based on AI vision, characterized in that: The following steps are involved: S1. Acquire characteristic image data of foreign matter on the power transmission line; S2. Marking the foreign body feature image data of the power transmission line obtained in step S1 to obtain a training data set; S3. Build an initial visual AI model based on deep learning algorithms; S4. Using the training data set obtained in step S2, the initial visual AI model obtained in step S3 is trained to obtain a visual AI model; S5. Use the visual AI model obtained in step S4 to perform actual cleaning of foreign objects on the power transmission line according to the robot motion control strategy.

2. The AI ​​vision-based robot power transmission line foreign body cleaning method according to claim 1 is characterized in that: Step S1 specifically includes: photographing foreign objects in several different mounting postures on a preset transmission line at different angles to obtain characteristic image data of foreign objects on the transmission line.

3. The AI ​​vision-based robot power transmission line foreign body cleaning method according to claim 1 is characterized in that: Step S2 specifically includes: classifying and labeling the foreign object feature image data of the power transmission line obtained in step S1 according to the foreign object processing method to obtain foreign object training data; combining the foreign object training data with the foreign object posture data and depth data to obtain a training data set.

4. The AI ​​vision-based robot power transmission line foreign body cleaning method according to claim 3 is characterized in that: The classification and labeling are specifically as follows: for foreign objects that can be picked up and removed by the robot gripper, the corresponding transmission line foreign object characteristic data is labeled as "hand gripper foreign object removal"; for foreign objects that cannot be removed by the robot gripper, the flammability of the foreign object is judged. If the foreign object is flammable, the corresponding transmission line foreign object characteristic data is labeled as "laser foreign object removal", otherwise it is labeled as "manual foreign object removal".

5. The method for cleaning foreign matter in power transmission lines by a robot based on AI vision according to claim 1 is characterized in that: Step S3 is specifically as follows: based on the Yolov8 model, an initial visual AI model is constructed; the input of the initial visual AI model is the image data of the moving foreign object in front taken by the robot camera, and the output is the foreign object processing type classification result, the posture data and depth data of the foreign object.

6. The method for cleaning foreign matter from power transmission lines using a robot based on AI vision according to claim 1, characterized in that: Step S5 specifically includes the following steps: a. Use the camera on the robot to obtain the image data, posture data and depth data of the robot in front of it in real time; b. Input the image data obtained in step a into the AI ​​visual model obtained in step S4 to obtain the foreign body processing type classification result, foreign body posture data and depth data; c. According to the foreign body handling type classification result obtained in step b and the posture data obtained in step a, the foreign body handling method is determined based on the robot arm foreign body handling strategy; d. Based on the foreign matter treatment method obtained in step c, combined with the depth data of the foreign matter obtained in step b, the foreign matter is cleaned up.

7. The AI ​​vision-based robot power transmission line foreign body cleaning method according to claim 5 is characterized in that: Step c is specifically as follows: When the classification result of the foreign body treatment type is laser foreign body removal, if the posture of the foreign body is that the foreign body is wrapped around the transmission line and the end is raised, the robot gripper approaches the end of the foreign body from the upper side of the foreign body, grabs the end of the foreign body with the gripper and straightens the foreign body, controls the other side of the robot arm to the place where the foreign body is wrapped around the transmission line, and focuses the end laser on the entanglement to perform laser burning, and finally removes the foreign body; When the classification result of the foreign body treatment type is laser foreign body removal, if the posture of the foreign body is that the foreign body is entangled on the transmission line and the end is falling, the robot gripper approaches the end of the foreign body from the bottom of the foreign body, grabs the end of the foreign body with the gripper and straightens the foreign body, controls the other side of the robot arm to the place where the foreign body is entangled with the transmission line, and focuses the end laser on the entanglement to perform laser burning, and finally removes the foreign body; When the foreign body treatment type classification result is laser foreign body removal, if the posture of the foreign body is that the foreign body is wrapped around the transmission line and the end is flush with the transmission line, the robot gripper approaches the end of the foreign body from a height such as the transmission line, grabs the end of the foreign body with the gripper and straightens the foreign body, controls the other side of the robot arm to the place where the foreign body is wrapped around the transmission line, and focuses the end laser on the entanglement to perform laser burning, and finally removes the foreign body; When the foreign object handling type classification result is gripper removal of foreign objects, if the foreign object posture is a non-entangled non-flexible foreign object with a length less than the arm span of the robotic arm, the robot gripper grasps one end of the non-flexible foreign object, controls the movement of the robotic arm, and makes the foreign object separate from the power transmission line, and finally removes the foreign object; When the foreign body handling type classification result is gripper removal of foreign bodies, if the foreign body is a non-entangled flexible foreign body whose length is less than the arm span of the robotic arm, the robot gripper grabs the flexible foreign body on either side of the hanging line position, controls the movement of the robotic arm, separates the foreign body from the transmission line, and finally removes the foreign body; When the foreign body handling type classification result is the removal of foreign bodies by grippers, if the posture of the foreign body is a non-entangled flexible foreign body whose length is greater than the arm span of the robotic arm, the robot gripper grasps any side of the flexible foreign body close to the hanging line position, controls the movement of the robotic arm, and makes the grasped side of the foreign body longer, and repeats this step until the length of one side of the flexible foreign body is less than the arm span of the robotic arm or it automatically falls off due to gravity; when the length of one side of the flexible foreign body is less than the arm span of the robotic arm, the robot gripper grasps the corresponding side of the foreign body, controls the movement of the robotic arm, and makes the foreign body separate from the transmission line, and finally removes the foreign body; When the classification result of the foreign object handling type is manual foreign object removal, and the posture of the foreign object is one that the robot cannot handle, an alarm is directly issued to prompt manual intervention.

8. The AI ​​vision-based robot power transmission line foreign body cleaning method according to claim 6 is characterized in that: Step d specifically comprises the following steps: Obtain historical motion control data of a manually controlled robot arm performing a foreign body removal operation as a control training data set; Based on the Deep Deterministic Policy Gradient neural network, an initial motion control decision AI model is constructed; The initial motion control decision AI model is trained using the control training data set to obtain a motion control decision AI model; The foreign body treatment method judgment obtained in step c and the depth data of the foreign body obtained in step b are input into the motion control decision AI model to obtain the robot foreign body cleaning motion trajectory control data; The robot is controlled to move according to the obtained robot foreign object cleaning motion trajectory control data to complete the transmission line foreign object cleaning.

9. A system for implementing the AI ​​vision-based robot power transmission line foreign body cleaning method according to any one of claims 1 to 8, characterized in that: It includes a data acquisition module, a data labeling module, an AI visual model building module, an AI visual model training module, and a transmission line foreign body cleaning module which are connected in series in sequence; The data acquisition module acquires characteristic image data of foreign matter on the power transmission line and uploads the data to the data marking module; The data labeling module labels the transmission line foreign body feature image data based on the received data, obtains the training data set, and uploads the data to the AI ​​vision model building module; The AI ​​vision model building module builds the initial visual AI model based on the deep learning algorithm and uploads the data to the AI ​​vision model training module; The AI ​​visual model training module trains the initial visual AI model according to the received data to obtain the visual AI model, and uploads the data to the transmission line foreign matter cleaning module; The transmission line foreign matter cleaning module performs actual transmission line foreign matter cleaning according to the received data and the robot motion control strategy.

Images