Industrial robot data acquisition method based on programming and body intelligent cooperation

Abstract

The invention discloses an industrial robot data acquisition method based on programming and intelligent cooperation, relates to an industrial robot control method, and provides an industrial robot data acquisition system and method based on programming and intelligent cooperation. The system comprises a programming mechanical arm (main arm), a robot demonstrator, five RGBD cameras and a humanoid robot (slave arm). The master mechanical arm and the slave mechanical arm conduct cooperative collection, the master arm executes pre-programming actions, and the slave arm receives the joint angle and the tail end pose of the master arm through the ROS and calculates the track of the slave arm. And efficient data acquisition and processing are realized by applying algorithms such as master-slave mechanical arm cooperation, image acquisition and processing, data conversion standardization, model training optimization and cloud edge cooperation. Meanwhile, an alarm mechanism is arranged, and data are recorded and analyzed. The method is high in expansibility, can be applied to the fields of industrial automation, intelligent manufacturing and the like, and helps to improve the industrial production efficiency and quality.

Description

Technical Field

[0001] The present invention relates to an industrial robot control method, and particularly to an industrial robot data acquisition method based on the collaboration of programming and embodied intelligence. Background Art

[0002] Traditional industrial robots mainly rely on teach pendants for programming, resulting in data being in a closed state and unable to be directly applied to the AI training process. Specifically, the following pain points exist: 1. Data acquisition link: Data is collected by means of teleoperation or motion capture technology, and this process highly depends on manual intervention by professionals with high skills. Not only is the labor cost extremely high, but the data acquisition efficiency is also extremely low.

[0003] 2. Insufficient support for large model training: During the large model training process, there is an extreme lack of large-scale and standardized industrial scenario motion trajectory data, severely restricting the model training effect and application expansion.

[0004] 3. Waste of existing data assets: The existing programming programs in the factory contain valuable data resources, but they have not been fully explored and effectively transformed, resulting in these data not forming reusable data assets and causing a great deal of resource idling. Summary of the Invention

[0005] The purpose of the present invention is to provide an industrial robot data acquisition method based on the collaboration of programming and embodied intelligence. This method proposes an industrial robot data acquisition system and method based on the collaboration of programming and embodied intelligence, and uses algorithms such as master-slave manipulator collaboration, image acquisition and processing, data conversion and standardization, model training optimization, and cloud-edge collaboration to achieve efficient data acquisition and processing, and help improve industrial production efficiency and quality.

[0006] The purpose of the present invention is achieved through the following technical solutions: An industrial robot data acquisition method based on the collaboration of programming and embodied intelligence, the method includes the following components and methods: Components include: A programming manipulator structure, serving as the main arm defined by embodied intelligence, providing motion teaching for the humanoid robot; a robot teach pendant, used to control the programming manipulator; RGBD cameras, five binocular depth and color cameras, corresponding to the head, left arm, right arm, left leg, and right leg of the humanoid robot respectively, for collecting humanoid robot image information; a humanoid robot, serving as the embodied intelligence ontology, making specific actions driven by a large model; a master-slave manipulator collaborative acquisition system, the main arm is equipped with a visual servo system to execute pre-programmed actions, and the slave arm (humanoid robot) receives the joint angles and end poses of the main arm in real time through ROS topics and calculates the trajectory that conforms to its own work. Technical methods: The master-slave manipulator collaborative algorithm realizes the motion coordination and data transmission between the master manipulator and the slave manipulator, including master manipulator motion planning, joint angle and end pose transmission, slave manipulator trajectory calculation, as well as ROS communication and inverse kinematics algorithm; The RGBD camera image acquisition and processing technology includes image acquisition, preprocessing and 3D reconstruction, and uses image preprocessing algorithms such as Gaussian filtering and median filtering; The data conversion and standardization algorithm converts the acquired data into a format suitable for AI training, and performs normalization and mean removal standardization operations, using data format conversion tools such as OpenCV and Pandas; The AI model training and optimization algorithm trains the AI model according to the acquired data, and optimizes it through data preprocessing and enhancement, using deep learning frameworks to train the model, adjusting model parameters, and adopting regularization methods and ensemble learning techniques; The cloud-edge collaborative data processing algorithm realizes the collaborative processing of data between the cloud and the edge, including data upload, cloud processing and edge inference, using the HTTP transmission protocol and distributed computing frameworks.

[0007] In the described industrial robot data acquisition method based on the collaboration of programming and embodied intelligence, the programmed manipulator structure executes pre-programmed actions to provide an accurate action teaching reference for the slave manipulator; The robot teach pendant can conveniently operate and control the programmed manipulator to adjust the motion parameters of the master manipulator.

[0008] In the described industrial robot data acquisition method based on the collaboration of programming and embodied intelligence, the RGBD camera accurately acquires multi-part image information of the humanoid robot, providing a rich data basis for subsequent data analysis.

[0009] In the described industrial robot data acquisition method based on the collaboration of programming and embodied intelligence, the master-slave manipulator collaborative acquisition system efficiently realizes the motion synchronization and data interaction between the master and slave manipulators; The master-slave manipulator collaborative algorithm ensures that the actions of the master and slave manipulators are coordinated and consistent, and the data transmission is timely and accurate. In the described industrial robot data acquisition method based on the collaboration of programming and embodied intelligence, the RGBD camera image acquisition and processing technology effectively acquires and processes the images of the humanoid robot, providing high-quality image data for model training; The data conversion and standardization algorithm ensures that the acquired data meets the requirements of AI training, improving the efficiency and effect of model training.

[0010] In the described industrial robot data acquisition method based on the collaboration of programming and embodied intelligence, the AI model training and optimization algorithm effectively improves the performance and generalization ability of the AI model; The cloud-edge collaborative data processing algorithm significantly improves the real-time performance and efficiency of data processing, realizing the reasonable utilization of data.

[0011] A data acquisition method for industrial robots based on the collaboration of programming and embodied intelligence. This method utilizes the ROS communication mechanism to send alarm signals to the monitoring center or related devices in real time when abnormalities or faults occur in the main arm or the slave arm. The HTTP data transmission protocol is adopted to quickly and accurately transmit alarm information from the edge side to the cloud or the designated receiving end.

[0012] The advantages and effects of the present invention are as follows: The present invention applies algorithms such as master-slave robotic arm collaboration, image acquisition and processing, data conversion standardization, model training optimization, and cloud-edge collaboration to achieve efficient data acquisition and processing. At the same time, an alarm mechanism is provided, and data recording and analysis are carried out. The present invention has strong scalability and can be applied to fields such as industrial automation and intelligent manufacturing, helping to improve the efficiency and quality of industrial production. Brief Description of the Drawings

[0013] Figure 1 It is a schematic diagram of the overall system composition of the present invention. Detailed Embodiment

[0014] The present invention will be described in detail below in conjunction with the drawings and the detailed embodiment: System Structure Design The core of this system is an efficient master-slave robotic arm collaborative acquisition system. The main arm: an industrial robot equipped with a vision servo system (such as ABB / KUKA), which executes pre-programmed actions. The slave arm: an embodied intelligence robot (humanoid robot), which receives the joint angles and end poses of the main arm in real time through ROS topics and calculates the working trajectory suitable for the humanoid robot to work. This set of acquisition system aims to not waste high-precision factory data and convert it into excellent data assets for subsequent model training.

[0015] Key Technologies and Algorithms In a data acquisition method for industrial robots based on the collaboration of programming and embodied intelligence, the key technologies and algorithms are the core to achieve efficient data acquisition, data conversion, and model training. The following are the key technologies and algorithms involved in this system: 1. Master-Slave Robotic Arm Collaboration Algorithm The master-slave manipulator collaborative algorithm is one of the core algorithms of the system, which is used to achieve motion coordination and data transmission between the master arm (programming manipulator) and the slave arm (humanoid robot). Master arm motion planning: Predefined motions are planned for the master arm through programming or a vision servo system. Joint angle and end-effector pose transmission: The joint angle and end-effector pose data of the master arm are transmitted to the slave arm in real time through ROS (Robot Operating System) topics. Slave arm trajectory calculation: After receiving the data from the master arm, the slave arm calculates the working trajectory that conforms to the operation of the humanoid robot through the inverse kinematics algorithm. ROS communication: Ensures real-time data transmission between the master arm and the slave arm. Inverse kinematics algorithm: Calculates the working trajectory of the humanoid robot based on the end-effector pose and joint angles.

[0016] 2. RGBD Camera Image Acquisition and Processing Technology The RGBD camera is used to collect image information of the humanoid robot, including depth information and color information, for subsequent data analysis and processing. Image acquisition: The image information of the head, both arms, and both legs of the humanoid robot is collected through five RGBD cameras respectively. Image preprocessing: Preprocessing operations such as denoising, filtering, and enhancement are performed on the collected images. 3D reconstruction: Combining depth information and color information, 3D reconstruction of the humanoid robot is carried out. Image preprocessing algorithms: Such as Gaussian filtering, median filtering, etc., are used to remove image noise.

[0017] 3. Data Conversion and Standardization Algorithm The data conversion and standardization algorithm is used to convert the collected data into a standard format suitable for AI training and ensure the consistency and reliability of the data. Data format conversion: Convert the collected image data, joint angle data, end-effector pose data, etc. into the format required for AI training. Data standardization: Normalization, mean removal, and other standardization operations are performed on the converted data to improve the efficiency and effect of model training. Data format conversion tools: Such as OpenCV, Pandas, etc., are used for data format conversion and processing. Data standardization methods: Such as normalization, standardization, etc., are used to improve the consistency and reliability of the data.

[0018] 4. AI Model Training and Optimization Algorithm The AI model training and optimization algorithm is used to train the AI model based on the collected data and optimize the model to improve its performance and generalization ability. Implementation steps: Data preprocessing and enhancement: Perform preprocessing and enhancement operations on the collected data to improve the generalization ability of the model.

[0019] Model training: Use deep learning frameworks (such as TensorFlow, PyTorch) to train the AI model. Optimize the model through techniques such as adjusting model parameters, using regularization methods, and ensemble learning.

[0020] 5. Cloud-edge collaborative data processing algorithm The cloud-edge collaborative data processing algorithm is used to achieve collaborative data processing between the cloud and the edge to improve the real-time performance and efficiency of data processing. Data upload: Upload the collected data from the edge to the cloud for storage and processing. Cloud processing: Perform large-scale data processing and model training in the cloud. Edge inference: Deploy the trained model to the edge for real-time inference and decision-making. Use the HTTP transmission protocol to achieve data transmission between the cloud and the edge. Adopt a distributed computing framework to achieve large-scale data processing and model training in the cloud. In summary, an industrial robot data acquisition method based on the collaboration of programming and embodied intelligence involves multiple key technologies and algorithms, which together constitute the core of the system, ensuring the efficiency of data acquisition, the accuracy of data conversion, and the reliability of model training.

[0021] Alarm mechanism and information transmission The alarm mechanism and information transmission are crucial parts to ensure the stable operation of the system and timely response to abnormal situations. Through an effective alarm and information transmission mechanism, problems in the system can be detected and handled in a timely manner. This invention utilizes the communication mechanism of ROS (Robot Operating System). When an abnormality or failure occurs in the main arm or the slave arm (such as entering a singularity), an alarm signal can be sent to the monitoring center or related devices in real time. Adopt the HTTP data transmission protocol to ensure that the alarm information can be quickly and accurately transmitted from the edge to the cloud or the designated receiving end, so that relevant personnel can take measures in a timely manner. Establish a remote monitoring system to monitor and respond to the alarm information in real time to ensure that problems are solved in a timely manner.

[0022] Data recording and analysis Data recording and analysis are important means to evaluate the system performance, optimize model training, and improve industrial production efficiency. By recording and analyzing various data in the system, the operating status and performance bottlenecks of the system can be deeply understood. Use data format conversion tools such as OpenCV and Pandas to convert the collected image data, joint angle data, end-effector pose data, etc. into a standard format and store them in a database or cloud storage.

[0023] Adopt data analysis tools and visualization software to deeply analyze and visually display the stored data to discover the patterns and trends in the data. Combine machine learning and deep learning algorithms to perform fault diagnosis and predictive analysis on the data in the system, discover potential problems in advance, and take corresponding measures.

[0024] System scalability and application prospects Adopt the modular design concept, design each part in the system as an independent module, so as to be flexibly combined and expanded according to requirements.

[0025] Cloud-edge collaboration architecture: Utilize cloud-edge collaboration data processing algorithms and distributed computing frameworks to achieve data collaborative processing and model training between the cloud and the edge, improving the processing capacity and response speed of the system. Follow international standards and open protocols to ensure that the system can be seamlessly docked and integrated with other devices and systems. This system can be widely applied to industrial automation production lines and intelligent manufacturing fields to improve production efficiency and quality. Using the collaborative function of humanoid robots and master-slave robotic arms in this system, robot education and training activities can be carried out to cultivate relevant talents. This system can also be used as a scientific research and experimental platform for researching new technologies and new methods in the fields of robot control, artificial intelligence, machine vision, etc.

Claims

1. An industrial robot data acquisition method based on the collaboration of programming and embodied intelligence, characterized in that, The method includes the following components and methods: The components are: A programmable robotic arm structure, serving as the main arm defined by embodied intelligence, providing motion teaching for the humanoid robot; a robot teach pendant, used to control the programmable robotic arm; an RGBD camera, five binocular depth and color cameras, corresponding to the head, left arm, right arm, left leg, and right leg of the humanoid robot respectively, for collecting image information of the humanoid robot; a humanoid robot, serving as the embodied intelligence ontology, making specific actions driven by a large model; a master-slave robotic arm collaborative acquisition system, where the master arm is equipped with a visual servo system to execute pre-programmed actions, and the slave arm (humanoid robot) receives the joint angles and end poses of the master arm in real time through ROS topics and calculates the trajectory suitable for its own work; Technical methods: The master-slave robotic arm collaborative algorithm realizes the action coordination and data transmission between the master arm and the slave arm, including master arm motion planning, joint angle and end pose transmission, slave arm trajectory calculation, as well as ROS communication and inverse kinematics algorithm; the RGBD camera image acquisition and processing technology includes image acquisition, preprocessing, and three-dimensional reconstruction, and uses image preprocessing algorithms such as Gaussian filtering and median filtering; the data conversion and standardization algorithm converts the collected data into a format suitable for AI training and performs normalization and mean removal standardization operations, using data format conversion tools such as OpenCV and Pandas; the AI model training and optimization algorithm trains the AI model based on the collected data, optimizes it through data preprocessing and augmentation, using a deep learning framework to train the model, adjusting model parameters, and adopting regularization methods and ensemble learning techniques; the cloud-edge collaborative data processing algorithm realizes the collaborative processing of data between the cloud and the edge, including data upload, cloud processing, and edge inference, using the HTTP transmission protocol and a distributed computing framework.

2. The industrial robot data acquisition method based on the collaboration of programming and embodied intelligence according to claim 1, wherein The programmable robotic arm structure executes pre-programmed actions, providing an accurate action teaching reference for the slave arm; the robot teach pendant conveniently operates and controls the programmable robotic arm to adjust the action parameters of the master arm.

3. A method for collecting industrial robot data based on the collaboration of programming and embodied intelligence according to claim 1, characterized in that, The RGBD camera accurately collects image information of multiple parts of the humanoid robot, providing a rich data basis for subsequent data analysis.

4. A method for collecting industrial robot data based on the collaboration of programming and embodied intelligence according to claim 1, characterized in that, The master-slave robotic arm collaborative acquisition system efficiently realizes the action synchronization and data interaction between the master and slave arms; the master-slave robotic arm collaborative algorithm ensures that the actions of the master arm and the slave arm are coordinated and the data transmission is timely and accurate.

5. A method for collecting industrial robot data based on the collaboration of programming and embodied intelligence according to claim 1, characterized in that, The RGBD camera image acquisition and processing technology effectively acquires and processes the images of the humanoid robot, providing high-quality image data for model training; the data conversion and standardization algorithm ensures that the collected data meets the requirements of AI training, improving the efficiency and effect of model training.

6. A method for collecting industrial robot data based on the collaboration of programming and embodied intelligence according to claim 1, characterized in that, The AI model training and optimization algorithm effectively improves the performance and generalization ability of the AI model; the cloud-edge collaborative data processing algorithm significantly improves the real-time performance and efficiency of data processing, realizing the rational utilization of data.

7. A method for collecting industrial robot data based on the collaboration of programming and embodied intelligence according to claim 1, characterized in that, This method utilizes the ROS communication mechanism to send alarm signals to the monitoring center or related devices in real time when abnormalities or faults occur in the master arm or the slave arm; it adopts the HTTP data transmission protocol to quickly and accurately transmit the alarm information from the edge side to the cloud or the designated receiving end.

Images