Multi-degree-of-freedom mechanical arm machine vision artificial intelligence guiding method

Abstract

The invention relates to a multi-degree-of-freedom mechanical arm machine vision artificial intelligence guiding method, and aims to improve the self-adaptive capability and accurate control of a mechanical arm in a dynamic environment. According to the method, machine vision data collection, artificial intelligence decision, path planning and optimization and dynamic feedback control are combined, and autonomous guidance of the mechanical arm is achieved. The method comprises the following steps: firstly, collecting environmental data in real time by utilizing machine vision, and processing the data through a deep learning algorithm; secondly, the artificial intelligence decision module generates a motion control instruction based on the environment information; then, a path planning and optimizing module generates an optimal motion track; and finally, the motion trail and posture of the mechanical arm are adjusted through a dynamic adaptation and real-time feedback control module so as to cope with environment changes and task requirements. The method is widely applied to the fields of industrial automation, intelligent manufacturing, mechanical arms, intelligent warehousing and the like, and has remarkable technical advantages and application prospects.

Description

Technical Field

[0001] The present invention relates to a multi-degree-of-freedom robot arm machine vision artificial intelligence guidance method, which belongs to the field of artificial intelligence guidance and automatic control, and is specifically applied to the fields of industrial automation, intelligent manufacturing, precision assembly, medical robot arms, intelligent warehousing, etc. The present invention provides a multi-degree-of-freedom robot arm guidance method based on machine vision data acquisition and combined with artificial intelligence technology. Through the collaborative work of perception, decision-making, motion planning and optimization, and closed-loop feedback mechanism, the multi-degree-of-freedom robot arm can complete various tasks in a dynamic and complex environment and realize highly autonomous intelligent operation. The method has a wide range of applications. Background Art

[0002] With the improvement of industrial automation and intelligence, multi-DOF robotic arms are widely used in various complex tasks such as assembly, welding, spraying, packaging, etc. Traditional multi-DOF robotic arm control methods mainly rely on preset paths and control algorithms, which have poor adaptability and are difficult to handle dynamic changes in the environment and complex working conditions. With the advancement of artificial intelligence technology, the application of deep learning and reinforcement learning in automated control has made significant progress, which can help robotic arms achieve more efficient and intelligent operations.

[0003] At present, traditional multi-DOF manipulator control methods usually adopt model-based control technology. The main feature of these methods is that they rely on known mathematical models and prior knowledge, and generate motion trajectories through real-time computer solutions. Traditional inverse kinematics methods calculate the expected position and posture of the end effector of the manipulator, and then infer the angles of each joint based on geometric relationships and kinematic models. Traditional trajectory planning algorithms are used to generate a smooth path from the initial position to the target position of the end effector of the manipulator; these algorithms can calculate the posture at each time point in the path, thereby generating a continuous motion trajectory, which requires a lot of manpower in design and implementation. Traditional feedback control technology is used to adjust the position of each joint of the manipulator. Although this method is simple and effective, it is suitable for static or simple dynamic environments and cannot cope with extremely complex working conditions.

[0004] However, current technical solutions still have deficiencies in high-precision control and dynamic adaptability of multi-DOF manipulators. In particular, when facing unknown environments, complex tasks and adaptive control, existing technologies often have difficulty in efficiently integrating perception, learning and decision-making processes. The present invention aims to achieve more efficient and accurate task execution through machine vision and artificial intelligence algorithms combined with motion control of multi-DOF manipulators. Summary of the invention

[0005] In order to solve the above problems, the present invention provides a multi-degree-of-freedom robot arm machine vision artificial intelligence guidance method, and the technical solution adopted is as follows:

[0006] Environmental perception and data collection: collect environmental data around the robot in real time through machine vision, integrate the robot's posture, acceleration, motion trajectory and other information to generate a fusion dynamic environment model. By integrating machine vision hardware algorithms, objects, obstacles and target positions can be accurately identified. The fusion dynamic environment function is as follows:

[0007] E t ={F v ,θ t ,v t}

[0008] Among them, E t The dynamic environment model at the current time t, F v The visual features generated by the machine vision system, θ t is the posture data of the robot arm, v t is the acceleration data of the robot arm;

[0009] By using the gradient descent method, the optimization function improves the accuracy of the fusion dynamic environment model. The function is as follows:

[0010] ΔE=||E t -E ref ||

[0011]

[0012] Among them, E t is the state of the environmental model at time t, E ref is the reference environment model state, η is the learning rate, is the error gradient.

[0013] Data processing and feature extraction: pre-process the environmental data collected by the machine vision system, including denoising, data normalization, filtering and other operations. Then, extract features through deep learning algorithms to identify the shape, position, relative motion state and other information of objects in the environment. Perform image recognition through the Convolutional Neural Network (CNN) model and conduct comprehensive analysis in combination with historical image data.

[0014] The data processing steps usually include denoising, data normalization, filtering and other operations. Each operation aims to eliminate noise, reduce data errors and make the data suitable for subsequent deep learning model processing. The key functions are as follows:

[0015] Convolutional neural networks perform convolution operations on images to extract local features:

[0016]

[0017] Among them, I(x,y) is the input image and K(i,j) is the convolution kernel.

[0018] Feature extraction: extract the outline of an object by detecting the area of ​​pixel value changes in the image, analyze the arrangement pattern of pixels in the image, extract texture features, use color information to distinguish different objects, and describe the shape of an object by calculating its size features. Automatically learn complex patterns and features in images through convolutional layers, pooling layers, and fully connected layers, and use reinforcement learning (Graph Neural Networks, GNN) to automatically complete feature extraction.

[0019] Motion planning and path optimization are the core parts of multi-degree-of-freedom manipulators to achieve efficient task execution. The goal is to generate the optimal motion path so that the manipulator can complete operations efficiently and safely under specific environments and task requirements. A comprehensive path optimization method is proposed by combining heuristic search, sampling algorithm, trajectory smoothing and optimization technology. First, an initial path is generated using a heuristic search algorithm, and a feasible path is generated using a sampling algorithm; then, the initial path is smoothed and optimized using a trajectory optimization method to ensure that the path meets kinematic constraints, dynamic constraints and environmental constraints; further, the objective function is designed by comprehensively considering factors such as path length, motion energy consumption, time efficiency, and obstacle distance; finally, combined with image data, the path is adjusted in real time based on feedback control to adapt to the dynamic environment. The key mathematical functions are:

[0020] J=λ1·L+λ2·E+λ3·C

[0021]

[0022]

[0023] Among them, J is the optimization target value, L is the path length, E is the motion energy consumption, C is the collision cost, λ1, λ2, λ3 are the weights of path length, energy consumption and collision cost respectively, τ i is the joint torque at time i.

[0024] Dynamic adaptation and real-time feedback control are crucial components in multi-DOF robotic arm systems, which directly affect the quality of robotic arm movements. The robot's behavior is adjusted and optimized in real time based on environmental changes, target changes, sensor feedback, and execution status. Through a closed-loop control mechanism, it is ensured that the robot can adapt to the dynamically changing environment and perform real-time corrections on motion trajectories and operations. The environmental adaptation and real-time trajectory optimization functions are the core of this part. Obstacles are sensed through sensors, and the path is adjusted through dynamic obstacle avoidance strategies; the gradient descent algorithm is combined to adjust the path smoothness and obstacle avoidance. Related functions:

[0025] d obs =min||p arm -p obstacle ||

[0026] If d obs <d thresh , triggering evasive action and changing the trajectory of the robot arm;

[0027]

[0028] Among them, d obs is the measured value, d thresh is the preset threshold, P is the trajectory point, L(p) is the loss function, and a is the step size.

[0029] Artificial intelligence decision-making, based on input data and environmental perception information, uses algorithms to generate efficient operation strategies, which is a key part of realizing intelligent control and task completion. Its core is to analyze, learn and make decisions on inputs in complex environments through deep learning, reinforcement learning and optimization algorithms. Multi-degree-of-freedom manipulators have the characteristics of discrete and continuous actions. In order to solve the high-dimensional spatial state of multi-degree-of-freedom manipulators and perform complex tasks, a new reinforcement learning strategy is proposed, which includes the combination of state and action value, time difference error, policy gradient term, entropy regularization term and learning rate. The new reinforcement learning function is as follows:

[0030]

[0031] Time difference error: δ t =R(s t ,a t )+γV Φ (s t +1)-V Φ (s t )

[0032] Entropy regularization term: H(π θ )=-∑ a π θ (a|s t )logπθ (a|s t )

[0033] Among them, θ is the reinforcement learning result, R(s t ,a t ) is the immediate reward, V φ (s) is the estimated state value of the neural network, Update the gradient direction of the policy network for the policy gradient method, π θ (a|s t ) is the parameterized strategy distribution, H(π θ ) is the entropy of the side rate distribution, and β is the weight.

[0034] Inter-module collaboration and system architecture. In the AI ​​guidance method of multi-DOF manipulators, the key to ensuring the efficient operation of each module is to adopt communication protocols with different delays and data accuracy. Inter-module collaboration can not only improve the overall performance of the system, but also achieve higher flexibility and autonomy in complex environments and tasks. The system architecture of each module determines the collaboration mechanism, data flow, control logic, and how to enable each module to collaborate efficiently through a reasonable system architecture.

[0035] The interfaces and data protocols for the coordinated work between unified modules need to have the characteristics of efficient communication and low latency. The data of multiple sensors are synchronized in time to ensure that the robot can make accurate decisions at each time step. In the links of environmental perception, path planning and feedback control, through data fusion technology, the data of various sensors can be effectively complementary, maintain a high degree of data consistency, and ensure the system's high-precision decision-making in a dynamic environment.

[0036] Mechanical action execution is a key component of the multi-DOF robotic arm artificial intelligence guidance system, responsible for converting instructions optimized by the decision-making and path planning modules into actual mechanical actions. Combining control signals with the hardware interface of the robotic arm ensures that the robotic arm can accurately and efficiently complete various operation tasks, such as grasping, placing, rotating, etc.

[0037] This part involves the design of control algorithms, precise control of the execution system, and the implementation of feedback adjustment mechanisms to ensure that the robot arm can flexibly respond to various situations in complex dynamic environments. It includes action execution functions, kinematic control and drive, dynamic control, real-time feedback and error correction. The core functions are:

[0038]

[0039] Among them, T is the posture of the end effector, A i (θ i ) is the transformation matrix of the i-th joint, θ iis the angle of the i-th joint;

[0040] q=f(T)

[0041] Where q is the joint angle vector, and T is the target transformation matrix of the end effector;

[0042] e(t)=P target -P actual

[0043] Among them, e(t) is the error, P target is the target position, P actual The current actual position. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A flow chart of a multi-degree-of-freedom robotic arm machine vision artificial intelligence guidance method provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, and are not used to limit the scope of the present invention.

[0046] Method Example:

[0047] A specific embodiment of the present invention discloses a multi-degree-of-freedom robot arm machine vision artificial intelligence guidance method, which realizes the multi-degree-of-freedom robot arm to collect environmental data based on machine vision and realize the robot arm autonomous guidance work through artificial intelligence algorithm. Figure 1 As shown, the method includes:

[0048] Step S1, set up a machine vision module to perceive and collect data on the environment. By collecting the environmental data around the robot in real time, integrating the robot's posture, acceleration, motion trajectory and other information, and using the fusion dynamic environment model to generate environmental data. This module can accurately identify objects, obstacles and target positions by integrating machine vision hardware and perception algorithms. The fusion dynamic environment function is as follows:

[0049] E t ={F v ,θ t ,v t}

[0050] Among them, E t The dynamic environment model at the current time t, F v The visual features generated by the machine vision system, θ t is the posture data of the robot arm, v tis the acceleration data of the robot arm;

[0051] By using the gradient descent method, the optimization function improves the accuracy of the fusion dynamic environment model. The main functions are as follows:

[0052] ΔE=||E t -E ref ||

[0053]

[0054] Among them, E t is the state of the environmental model at time t, E ref is the reference environment model state, η is the learning rate, is the error gradient.

[0055] Step S2, data processing and feature extraction module, processes the environmental data collected by the machine vision system, including denoising, data normalization, filtering and other operations. Then, features are extracted through deep learning algorithms to identify the shape, position, relative motion state and other information of objects in the environment. Image recognition is performed through the CNN network, and comprehensive analysis is performed in combination with historical image data. These processed data and features are important bases for the path planning of the robot arm.

[0056] The data processing steps usually include denoising, data normalization, filtering and other operations. Each operation aims to eliminate noise, reduce data errors and make the data suitable for subsequent deep learning model processing. The key functions are as follows:

[0057] Convolutional neural networks perform convolution operations on images to extract local features:

[0058]

[0059] Among them, I(x,y) is the input image and K(i,j) is the convolution kernel.

[0060] Feature extraction: extract the outline of the object by detecting the area where the pixel intensity changes in the image, analyze the arrangement pattern of pixels in the image, and extract texture features; use color information to distinguish different objects; use the YOLO network (You Only Look Once, YOLO) to identify the size of the object and determine its position and size in the image. The complex patterns and features in the image are automatically learned through the CNN network convolutional layer, pooling layer, and fully connected layer, and the GNN network reinforcement learning is used to automatically complete feature extraction.

[0061] Step S3, motion planning and path optimization module, path planning is the core part of the multi-degree-of-freedom manipulator to achieve efficient task execution. Its goal is to generate the optimal motion path so that the manipulator can complete operations efficiently and safely under specific environments and task requirements. This module combines heuristic search, sampling algorithm, trajectory smoothing and optimization technology to propose a comprehensive path optimization method. First, use the heuristic search algorithm to generate an initial path, and combine the sampling algorithm to generate a feasible path; then, use the trajectory optimization method to smooth and optimize the initial path to ensure that the path meets the kinematic constraints, dynamic constraints and environmental constraints; further, comprehensively consider factors such as path length, motion energy consumption, time efficiency, obstacle distance, etc. to design the objective function; finally, combine sensor data, and adjust the path in real time based on feedback control to adapt to the dynamic environment. The key mathematical function is:

[0062] J=λ1·L+λ2·E+λ3·C

[0063] Among them, J is the optimization target value, L is the path length, E is the motion energy consumption, C is the collision cost, λ1, λ2, λ3 are the weights of path length, energy consumption and collision cost respectively, τ i is the joint torque at time step i.

[0064] Step S4, dynamic adaptation and real-time feedback control module, this module is a crucial component in the multi-degree-of-freedom robot system, and directly affects the movement quality of the robot. First, according to environmental changes, target changes, and sensor feedback data; secondly, the data is sent to the robot for execution; then, according to the position and posture data of the robot, the data error before and after execution is compared; finally, the robot behavior is adjusted and optimized in real time according to the error and the actual collected environmental data. Through PID closed-loop control (Proportional-Integral-Derivative Control, PID), ensure that the robot can adapt to the dynamically changing environment and perform real-time correction of the motion trajectory and operation. The environmental adaptation and real-time trajectory optimization function is the core of this part. Obstacles are perceived through machine vision, and the path is adjusted through dynamic obstacle avoidance strategies; the gradient descent algorithm is combined to adjust the path smoothness and obstacle avoidance. Related functions:

[0065] d obs =min||p arm -p obstacle ||

[0066] If d obs <d thresh , triggering evasive action and changing the trajectory of the robot arm;

[0067]

[0068] Among them, dobs is the measured value, d thresh is the preset threshold, P is the trajectory point, L(p) is the loss function, and a is the step size.

[0069] Step S5, artificial intelligence decision-making module, which is the core module of the robot's automated decision-making. Based on the input data and environmental perception information, the algorithm is used to generate efficient operation strategies, thereby realizing the key parts of intelligent control and task completion. Its core is to analyze, learn and make decisions on inputs in complex environments through deep learning, GNN reinforcement learning and optimization algorithms. Multi-degree-of-freedom robot arms have the characteristics of discrete and continuous actions. In order to solve the high-dimensional spatial state of multi-degree-of-freedom robot arms and perform complex tasks, a new reinforcement learning strategy is proposed, which includes the combination of state and action value, time difference error, policy gradient term, entropy regularization term and learning rate. The new reinforcement learning function is as follows:

[0070]

[0071] Time difference error: δ t =R(s t ,a t )+γV Φ (s t +1)-V Φ (s t )

[0072] Entropy regularization term: H(π θ )=-∑ a π θ (a|s t )logπ θ (a|s t )

[0073] Among them, θ is the reinforcement learning result, R(s t ,a t ) is the immediate reward, V φ (s) is the estimated state value of the neural network, Update the gradient direction of the policy network for the policy gradient method, π θ (a|s t ) is the parameterized strategy distribution, H(π θ ) is the entropy of the side rate distribution, and β is the weight.

[0074] Step S6, inter-module collaboration and system architecture. In the artificial intelligence guidance method of the multi-degree-of-freedom robotic arm, the collaborative work of each module is the key to the efficient operation of the system. Inter-module collaboration can not only improve the overall performance of the system, but also achieve higher flexibility and autonomy in complex environments and tasks. The system architecture of each module determines the collaboration mechanism, data flow, control logic, and how to enable each module to collaborate efficiently through a reasonable system architecture. The time-sensitive network protocol (TSN) is used for data interaction between the image data and the path planning module to ensure the accuracy of data transmission and the deferred vehicle model; the CAN bus protocol (Controller Area Network, CAN) is used between the path planning module and the mechanical action execution module to ensure high reliability, strong anti-interference ability and good real-time performance of instruction transmission; the TCP / IP communication protocol is used between the machine vision device and the computer.

[0075] The interfaces and data protocols for the coordinated work between unified modules need to have the characteristics of efficient communication and low latency. The data of multiple sensors are synchronized in time to ensure that the robot can make accurate movements at each time step. In the links of environmental perception, path planning and feedback control, through data fusion technology, the data of various sensors can be effectively complementary, maintain high data consistency, and ensure high-precision decision-making of the system in a dynamic environment.

[0076] Step S7, mechanical action execution module, which is the core part of the action implementation in the multi-DOF manipulator artificial intelligence guidance system, is responsible for converting the instructions optimized by the decision-making and path planning modules into actual mechanical actions. This module combines the control signal with the hardware interface of the manipulator to ensure that the manipulator can accurately and efficiently complete various operation tasks, such as grasping, placing, rotating, and walking in space.

[0077] This part involves the design of control algorithms, precise control of the execution system, and the implementation of feedback adjustment mechanisms to ensure that the robot arm can flexibly respond to various situations in complex dynamic environments. It includes action execution functions, kinematic control and drive, dynamic control, real-time feedback and error correction. The core functions are:

[0078]

[0079] Among them, T is the posture of the end effector, A i (θ i ) is the transformation matrix of the i-th joint, θ i is the angle of the i-th joint;

[0080] q=f(T)

[0081] Among them, q is the joint angle vector, T is the target transformation matrix of the end effector;

[0082] e(t)=P target -P actual

[0083] Among them, e(t) is the error, P target is the target position, P actual The current actual position.

[0084] Compared with the prior art, the present invention proposes a multi-degree-of-freedom robotic arm machine vision artificial intelligence guidance method. First, in the process of environmental data collection, the present invention designs a fusion dynamic environment model to accurately obtain environmental dynamic data, significantly improve the accuracy and robustness of target object recognition and positioning, and adapt to complex environments. Secondly, deep learning (CNN) and reinforcement learning algorithms are introduced to achieve efficient feature extraction and dynamic path planning, so that the robotic arm can autonomously learn and optimize operation strategies, improve the level of intelligence and flexibility. Finally, an efficient artificial intelligence decision-making algorithm is designed to achieve efficient collaboration between modules and improve the real-time and overall performance of the system.

[0085] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-degree-of-freedom robot arm machine vision artificial intelligence guidance method, characterized in that: The method comprises the following: Use machine vision as an environmental perception module to collect environmental data of the multi-degree-of-freedom robotic arm's operating space in real time; The collected data is transmitted to the data processing and feature extraction module, and the data is processed through the deep learning algorithm to extract environmental features; Through motion line planning and path optimization algorithms, the optimal action path of the robot arm is generated based on the collected environmental data; during the movement of the robot arm, the dynamic adaptation and real-time feedback control algorithms continuously correct the motion errors; Through the artificial intelligence decision-making module, decision data, different paths, error correction, etc. are generated according to the extracted environmental features for analysis to complete the motion control decision of the robot arm; The robotic arm follows guidance instructions to ensure that it can accurately and efficiently complete various grasping, rotation, placement, spatial movement, etc.

2. The method according to claim 1, wherein: The artificial intelligence decision-making module adopts a deep reinforcement learning algorithm to perform adaptive learning based on real-time feedback of environmental conditions and output optimal motion control instructions.

3. A multi-degree-of-freedom robotic arm decision-making strategy based on deep reinforcement learning, characterized in that: The algorithm solves the problem of a robot arm performing complex tasks in a high-dimensional space state by fusing a dynamic environment function. The algorithm includes: visual features, robot arm posture data, robot arm acceleration data, and error gradient terms.

4. The method according to claims 1 and 3, wherein: The motion planning and path optimization module uses an inverse kinematics algorithm to calculate the angles of each joint of the robotic arm to ensure that the end effector can reach the target position, and optimizes the path through an optimization algorithm to avoid collisions and redundant motions.

5. A new decision-making strategy is proposed, which is characterized by: The algorithm analyzes, learns and makes decisions on inputs in complex environments through deep learning, reinforcement learning and optimization; it includes the combination of state and action value, time difference error, policy gradient term, entropy regularization term and learning rate term.

6. The method according to claims 1 and 5, wherein: The artificial intelligence decision-making module also includes a deep neural network, which predicts the motion state of each joint of the robotic arm based on environmental characteristics and generates corresponding control instructions.

7. The method according to claim 1, wherein: The motion planning and path optimization module also includes an algorithm-based path search module for optimizing the movement path of the robotic arm, taking into account multiple factors such as the shortest path, minimum energy consumption, obstacle avoidance, and operational safety.

8. The method according to claim 1, wherein: The environmental perception module includes multiple industrial cameras, and the sensors work together through data fusion technology to provide more accurate environmental perception information.

9. The method according to claim 1, wherein: The multi-degree-of-freedom robotic arm achieves real-time error correction and motion adjustment through dynamic adaptation and closed-loop control systems when performing tasks, ensuring continuous and efficient completion of tasks in complex environments.

Images