Intelligent mechanical arm control system

Abstract

The invention discloses an intelligent mechanical arm control system, and relates to the technical field of intelligent mechanical arms, the system comprises a sensor data acquisition module, a multi-modal data fusion module, an environment and target prediction module, an intelligent decision and control module and a control execution and feedback module; through a cooperative working mechanism of the environment and target prediction module and the intelligent decision-making and control module, future state change of a target object can be accurately predicted, possibility is provided for pre-judgment operation of the mechanical arm, then a decision-making optimization algorithm and a multi-target optimization function are applied, an optimal motion strategy is planned for the mechanical arm, and the optimal motion strategy is provided for the mechanical arm. The system further has the real-time feedback and adjustment capacity, it is ensured that the mechanical arm can automatically adapt to changes in the complex dynamic environment, the operation precision and efficiency of the mechanical arm are improved, the capacity of the mechanical arm for adapting to the complex environment and dealing with emergencies is enhanced, and the production efficiency and safety are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent robotic arms, and particularly to an intelligent robotic arm control system. Background Technique

[0002] With the advent of the Industry 4.0 era, intelligent manufacturing and industrial automation have become the key forces driving the transformation and upgrading of the global manufacturing industry. As the core execution unit, the performance and intelligence level of the robotic arm are directly related to production efficiency and product quality. With the increasing complexity and diversification of production tasks, higher requirements are put forward for the perception ability, decision-making ability and environmental adaptability of the robotic arm.

[0003] The patent with the application number: CN114505856A discloses an intelligent control system for a robotic arm based on big data deep learning. It is difficult to handle complex and changeable actual working scenarios. Especially when facing unknown obstacles or dynamic targets, its decision-making efficiency and accuracy are greatly reduced, ignoring the real-time changes of environmental information and target states, resulting in a large deviation between the decision-making result and the actual demand, and being unable to adjust the control strategy in time to adapt to new situations, and lacking an effective real-time adjustment mechanism, resulting in a great reduction in the accuracy and effectiveness of control instructions. Therefore, an intelligent robotic arm control system has been developed. Summary of the Invention

[0004] The purpose of the present invention is to make up for the deficiencies of the prior art, and provide an intelligent robotic arm control system. The system integrates multiple modules such as sensor data acquisition, multi-modal data fusion, environment and target prediction, intelligent decision-making and control, and control execution and feedback. Through the intelligent decision-making and control module, it can receive and preprocess the fused environmental information and prediction results, optimize the control input sequence using decision optimization algorithms, plan the motion strategies of the robotic arm mobile chassis and each joint, and generate control instructions. At the same time, it also has the ability to adjust instructions in real time.

[0005] To solve the above technical problems, the present invention provides the following technical solution: An intelligent robotic arm control system, the system includes: a sensor data acquisition module, a multi-modal data fusion module, an environment and target prediction module, an intelligent decision-making and control module, and a control execution and feedback module; Sensor data acquisition module: Deploy a variety of sensors on the robotic arm to build a multi-dimensional perception network, and collect the original data of the distance, image, joint angle, sound and temperature field of the environment around the robotic arm at different frequencies in real time, and transmit it to the data processing center for storage; Multi-modal data fusion module: Extract the collected data from the data processing center, perform spatio-temporal calibration and feature extraction, and use fusion algorithms to fuse the feature data of different types of sensors to generate fused environmental information; Environment and Target Prediction Module: Based on the fused environmental information and combined with historical data, it analyzes from both the time dimension and the space dimension, and makes a short-term prediction of the future state of the target object through the target state prediction integration model according to the analysis results; Intelligent Decision-making and Control Module: Receives and preprocesses the fused environmental information and prediction results, optimizes the control input sequence using the decision optimization algorithm, plans the motion strategies of the robotic arm mobile chassis and each joint, generates control instructions and sends them to the Control Execution and Feedback Module. At the same time, it receives the actual motion state information of the robotic arm feedback by the Control Execution and Feedback Module, and uses the robotic arm motion optimization algorithm to adjust the instructions in real time; Control Execution and Feedback Module: Receives the control instructions generated in the Intelligent Decision-making and Control Module, drives the robotic arm to act. During operation, it collects the actual state data of the robotic arm in real time through sensors, evaluates the target state deviation and environmental interference, and feeds them back to the Intelligent Decision-making and Control Module.

[0006] Furthermore, the sensors used in the Sensor Data Acquisition Module are: Distance Sensor: Adopts lidar, installed at the center of the top of the robotic arm; Vision Sensor: Adopts a binocular stereo camera, installed on the outside of the forearm; Position Sensor: Installs high-precision absolute encoders at the rotation axes of each joint; Sound Sensor: Adopts an annular array composed of MEMS microphones, installed around the upper half of the robotic arm housing; Infrared Sensor: Adopts the FLIR Lepton 3.5 thermal imaging module, installed on the side of the end effector.

[0007] Even further, the fusion algorithm is used in the Multimodal Data Fusion Module to fuse the feature data of different types of sensors. The fusion algorithm formula is: , where: is the fused environmental information vector, is the quantum state fusion function, is the coupling coefficient simulating quantum entanglement, is the dynamic weight of the th type of modal data, is the original data of the th type of modality, is the index variable.

[0008] Even further, the specific steps for analyzing the fused environmental information from the time dimension and the space dimension in the Environment and Target Prediction Module are: Analysis in the time dimension: Arrange the fused environmental information in chronological order to form time series data of the target object's state, perform smoothing processing, determine the change trends of position, attitude, and temperature, judge the speed change state, detect whether there is a periodic change pattern in the time series, and simultaneously detect outliers in the time series data; Analysis in the space dimension: Establish a three-dimensional coordinate system for the robotic arm's working area, map the target object and the obstacles in the surrounding environment to the coordinate system, calculate the relative distance and azimuth angle between the target and the surrounding objects according to their spatial coordinate positions in the three-dimensional coordinate system, and extract the characteristics of the spatial structures of enclosed areas and movement channels by observing the distribution patterns of environmental objects, and evaluate the influencing ways and degrees of surrounding environmental factors on the target object's attitude and temperature.

[0009] Furthermore, the Environment and Target Prediction Module performs short-term prediction on the future state of the target object through the Target State Prediction Integration Model according to the analysis results. The calculation formula of the Target State Prediction Integration Model is: , where is the state vector of the predicted target object at the next time point, is the fusion weight of the prediction results in the time dimension and the space dimension, is the state predicted by the time series model, obtained by training with historical time series data, is the th correction value of the target state by the spatial associated object, obtained by statistically analyzing the change pattern of the target state under the spatial relationship in historical data, is the k th weight of the spatial associated object, k is the index variable, is the number of surrounding objects included in the analysis.

[0010] Furthermore, in the Intelligent Decision-making and Control Module, the control input sequence of the robotic arm is optimized according to the decision optimization algorithm. The formula of the decision optimization algorithm is: , where is the robotic arm control input sequence, is the motion energy consumption function, is the distance penalty term between the robotic arm and the obstacle, is the change amount of adjacent control inputs, and are weight coefficients dynamically adjusted according to the obstacle density and the environmental temperature change rate.

[0011] Furthermore, the specific steps for the intelligent decision-making and control module to plan the movement strategy of the robotic arm's mobile chassis and each joint are as follows: Analyze the received fused environmental information and prediction results to determine the type, priority, and accuracy requirements of the current task executed by the robotic arm, as well as the information on the position, attitude, motion trend of the target object, and the distribution of surrounding environmental obstacles. Based on the real-time distance between the target object and the robotic arm, and the parameters of the target predicted motion speed and acceleration, construct a multi-objective optimization function with the completion time, energy consumption, motion smoothness, and safety as the core objectives. According to the actual requirements of the task, set corresponding weight coefficients for each objective, combine the kinematic and dynamic constraints of the robotic arm to form a multi-objective optimization model, and solve the multi-objective optimization model. Through continuous iterative calculations, obtain the control parameters of the traveling speed, steering angle of the robotic arm's mobile chassis, and the rotation angle, angular velocity, and angular acceleration of each joint.

[0012] Furthermore, the intelligent decision-making and control module constructs a multi-objective optimization function with the completion time, energy consumption, motion smoothness, and safety as the core objectives. Specifically, let be the optimization objective function, be the task completion time, be the reference completion time, be the energy consumption, be the reference energy consumption, be the motion smoothness index, be the safety assessment index. Then the formula for the multi-objective optimization function is: ; : ; ; ; ; , where , , , are the weight coefficients corresponding to the respective objectives, and , dynamically adjusted according to the task type, represents the robotic arm kinematic constraint function, represents the robotic arm dynamic constraint function, is the robotic arm joint angle vector, is the joint angular velocity vector, is the joint angular acceleration vector, is the real-time distance between the target object and the robotic arm, measured through sensor data, is the maximum allowable distance value, is the target predicted speed, obtained by predicting the robotic arm motion speed through a prediction model, is the maximum allowable speed value, is the target predicted acceleration, which is calculated by the prediction model based on the data of the manipulator motion state. is the maximum allowable acceleration value.

[0013] Furthermore, the manipulator motion optimization algorithm is used in the intelligent decision-making and control module to adjust the instructions in real time. The formula of the manipulator motion optimization algorithm is: where: is at time the manipulator control instruction adjusted in real time by the optimization algorithm, is the control instruction received by the intelligent decision-making and control module, is the deviation vector between the actual state and the target state of the manipulator, is the estimated value of the environmental interference intensity, is the feedback signal, is the time.

[0014] Compared with the prior art, the intelligent manipulator control system has the following beneficial effects: First, through the collaborative working mechanism of the environment and target prediction module and the intelligent decision-making and control module, the present invention can accurately predict the future state changes of the target object, providing the possibility for the predictive operation of the manipulator. Then, by using the decision optimization algorithm and the multi-objective optimization function, the optimal motion strategy is planned for the manipulator. Moreover, the system also has the ability of real-time feedback and adjustment to ensure that the manipulator can autonomously adapt to changes in a complex dynamic environment, not only improving the operation accuracy and efficiency of the manipulator, but also enhancing its ability to adapt to complex environments and handle emergencies, thus improving production efficiency and safety.

[0015] Second, the control execution and feedback module of the present invention provides key support for the intelligent upgrade of the system. By deploying sensors at key parts of the manipulator, the motion state data of the manipulator are collected in real time to feedback the execution effect to the intelligent decision-making and control module, enabling the manipulator to be upgraded from a tool that passive executes instructions to an intelligent agent with the ability of autonomous state monitoring, effectively enhancing the autonomous decision-making ability of the manipulator and laying a foundation for the stable execution of complex tasks.

[0016] Other advantages, objectives and features of the present invention will be described to some extent in the subsequent specification, and to some extent, will be obvious to those skilled in the art based on the study of the following text, or can be learned from the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0018] Figure 1 It is a schematic structural diagram of an intelligent robotic arm control system; Figure 2 It is a flow chart of an intelligent robotic arm control system. Detailed implementation manners

[0019] To further elaborate on the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following, in combination with the drawings and preferred embodiments, will detail the specific implementation manners, structures, features, and their effects of the present invention as follows.

[0020] Embodiment 1 In an automated warehousing center, the robotic arm needs to accurately grasp, transport, and sort packages of different sizes and weights, while avoiding dynamic obstacles and adapting to the complex lighting and temperature environment in the warehouse.

[0021] Sensor data acquisition module: Install a 20Hz lidar (distance sensor) at the center of the top of the robotic arm to scan the spacing between sorting shelves and the position of packages in real time; deploy a 20Hz binocular stereo camera (vision sensor) on the outer side of the forearm to identify package barcodes and three-dimensional sizes; configure a 100Hz high-precision absolute encoder (position sensor) for each joint rotation axis to monitor the grasping posture of the robotic arm; arrange a 50Hz ring MEMS microphone array (sound sensor) around the shell of the robotic arm to detect abnormal noises during the operation of the conveyor belt; install a 50Hz FLIR Lepton 3.5 thermal imaging module (infrared sensor) on the side of the end effector to sense abnormal surface temperatures of packages, and synchronously transmit the collected raw data (distance, image, joint angle, sound, temperature) to the data processing center for storage.

[0022] Multi-modal data fusion module: Extract the data of each sensor from the data processing center. As Figure 1 shown, the characteristic data of different types of sensors are fused through a fusion algorithm. The formula of the fusion algorithm is: , where: is the fused environmental information vector (such as comprehensive data of package position, size, and conveyor belt speed), is the quantum state fusion function, is the coupling coefficient simulating quantum entanglement, is the The dynamic weights of various modal data (such as the weight of lidar distance data is higher than that of sound data), is the original data of the th modality, is the index variable, is the number of modalities participating in the fusion; the quantum state fusion function simulates the characteristics of quantum entanglement, enhances the spatial and dynamic weights of different modal data is dynamically adjusted according to the scenario, and after data fusion, environmental information including package position, size, barcode information, conveyor belt speed, and environmental safety is generated.

[0023] Environment and target prediction module: Based on the fused environmental information, combined with historical data, through time dimension analysis, arrange the data of package position and conveyor belt speed, determine the changing trends of position and speed through smoothing processing, detect periodic patterns (such as conveyor belt start-stop cycle) and outliers (such as emergency stop events); through spatial dimension analysis, establish a three-dimensional coordinate system with the sorting area, map the positions of packages, conveyor belts, and sorting ports, calculate the relative distance and azimuth angle, analyze the package stacking density in the channel, and according to the analysis results of time dimension and spatial dimension, predict the state of the package at the next time point through the target state prediction integration model, and the calculation formula of its target state prediction integration model is: , where, is the state vector (position, attitude) of the predicted target object at the next time point, is the fusion weight of the time dimension and spatial dimension prediction results (such as the time dimension accounts for 60%), is the prediction value of the time series model, obtained by training historical time series data, is the th correction value of the target state by the spatial correlation object, obtained by statistically analyzing the change law of the target state under the spatial relationship in historical data, is the k th weight of the spatial correlation object, k is the index variable, is the number of surrounding objects included in the analysis (such as = 3).

[0024] Intelligent decision-making and control module: After receiving and preprocessing the fused environmental information and prediction results, optimize the control input sequence through the decision optimization algorithm, and the formula of the decision optimization algorithm is: , where: is the control input sequence of the robotic arm (such as joint angle, moving speed), is the motion energy consumption function, is the distance penalty term between the robotic arm and the obstacle (such as increasing the penalty when the distance to the conveyor belt edge is too close), is the change in the adjacent control input, and is the weight coefficient dynamically adjusted according to the obstacle density and the environmental temperature change rate (automatically adjusted according to the obstacle density, e.g., when packages are stacked increases), and plans the motion strategies of the robotic arm's mobile chassis and each joint through the output control input sequence; At the same time, a multi-objective optimization function with the task completion time, energy consumption, motion smoothness, and safety as the core objectives is constructed. According to the actual requirements of the task, corresponding weight coefficients are set for each objective. Combining the constraints of the robotic arm's kinematics and dynamics, a multi-objective optimization model is formed, and the multi-objective optimization model is solved. Specifically, let be the optimization objective function, be the task completion time, be the reference completion time, be the energy consumption, be the reference energy consumption, be the motion smoothness index, be the safety evaluation index. Then the formula for the multi-objective optimization function is: ; : ; ; ; ; , where , , , are the weight coefficients corresponding to the respective objectives, and , dynamically adjusted according to the task type, represents the robotic arm kinematic constraint function, represents the robotic arm dynamic constraint function, is the robotic arm joint angle vector, is the joint angular velocity vector, is the joint angular acceleration vector, is the real-time distance between the target object and the robotic arm, measured through sensor data, is the allowable maximum distance value, is the target predicted speed, obtained by predicting the robotic arm's motion speed through a prediction model, is the allowable maximum speed value, is the target predicted acceleration, calculated by the prediction model based on the data of the robotic arm's motion state, is the allowable maximum acceleration value. Through continuous iteration calculation, the control parameters of the traveling speed, steering angle of the robotic arm's mobile chassis, and the rotation angle, angular velocity, and angular acceleration of each joint are obtained.

[0025] Control Execution and Feedback Module: Driven by the generated control instructions, the robotic arm performs grasping actions, and real-time data such as joint angles and gripper pressures are collected through sensors. The deviation from the target state is calculated, and disturbances such as conveyor belt vibrations are evaluated. The deviation information is fed back to the Intelligent Decision and Control Module to trigger the robotic arm motion optimization algorithm to adjust the next action. The formula for its robotic arm motion optimization algorithm is: , where: is the robotic arm control instruction adjusted in real-time by the optimization algorithm at time , is the control instruction received from the Intelligent Decision and Control Module, is the deviation vector between the actual state and the target state of the robotic arm, is the estimated value of the environmental disturbance intensity, is the feedback signal, is the time.

[0026] To sum up, in the logistics sorting scenario, through multi-modal sensor fusion and dynamic prediction models, this solution can accurately perceive the position, size, and conveyor belt status of packages, significantly improving sorting efficiency and accuracy. The Intelligent Decision Module reduces the risk of the robotic arm idling and colliding by optimizing energy consumption and obstacle avoidance strategies. Moreover, the closed-loop feedback mechanism can correct grasping deviations in real-time, adapting to complex environments such as conveyor belt vibrations and package accumulation, improving sorting efficiency and supporting 24-hour continuous operation, which is suitable for high-load scenarios such as e-commerce promotions and warehousing logistics.

[0027] Example Two For the delicate tissue separation and hemostasis operations in laparoscopic surgery, the robotic arm is required to have sub-millimeter positioning accuracy, real-time obstacle avoidance (avoiding important blood vessels), and thermal damage monitoring capabilities. At the same time, it is necessary to balance the operation speed and the need for tissue protection.

[0028] Sensor Data Acquisition Module: As Figure 2 shown, a 20Hz micro lidar is installed at the center of the top of the robotic arm to scan the contour of the organs and the distance between the instruments in the surgical field; a 20Hz medical binocular endoscope (visual sensor) is integrated on the outer side of the forearm to obtain high-definition blood vessel and tissue images of the surgical site in real-time; 100Hz high-precision encoders are configured for each joint to monitor the micron-level displacement of the surgical instruments of the robotic arm (with an accuracy of ±0.01mm); a 50Hz medical microphone array is arranged on the upper part of the robotic arm shell to collect the operation sound characteristics of instruments such as ultrasonic scalpels and electrosurgical knives; an infrared thermal imaging module with a frequency of 50Hz is added to the side of the end surgical instrument to monitor the tissue temperature change (resolution 0.1℃), and the collected original data (organ position, tissue image, joint displacement, instrument sound, tissue temperature) are synchronously stored in the data processing center.

[0029] Multi-modal Data Fusion Module: Extract the organ contour data of the lidar, the vascular image features of the endoscope, the instrument position of the encoder, the ultrasonic scalpel frequency signal of the microphone, and the thermal imaging temperature field from the data processing center, and generate fusion information through the fusion algorithm formula. The fusion algorithm formula is: 。

[0030] Environment and Target Prediction Module: Based on the fused environmental information, through time dimension analysis, arrange the instrument movement trajectories (such as the movement speed of the electrosurgical knife) and tissue temperature change data in a time series, predict the position of the instrument at the next time point (error ≤ 0.2 mm) through smoothing processing, analyze the temperature rise rate (the threshold is set to 0.5 °C / s, and an alarm is triggered if exceeded), and detect abnormal jitter of the robotic arm (such as the acceleration mutation when colliding with the operating table); through space dimension analysis, establish a three-dimensional coordinate system with the patient's surgical site, map structures such as the liver, blood vessels, and tumors, calculate the relative distance between the instrument and the portal vein of the liver (the safety distance of ≥ 2 mm needs to be maintained) through the spatial analysis steps of claim 4, evaluate the amplitude of organ displacement caused by respiratory movement (such as the liver fluctuating up and down by 1.5 mm), and combine the analysis results to predict the impact of the instrument through the target state prediction integration model. The calculation formula of the target state prediction integration model is: 。

[0031] Intelligent Decision-making and Control Module: Receive the information such as the fused vascular position and tissue temperature and the prediction results (such as the electrosurgical knife approaching the blood vessel after 0.5 seconds), and optimize the control instructions of the robotic arm using the decision optimization algorithm. The decision optimization algorithm formula is: , plan the movement strategies of the robotic arm's mobile chassis and each joint, and at the same time construct a multi-objective optimization function with the task completion time, energy consumption, movement smoothness, and safety as the core objectives. According to the actual requirements of the task, set the corresponding weight coefficients for each objective, and combine the kinematic and dynamic constraints of the robotic arm to form a multi-objective optimization model. The multi-objective optimization function formula is: ; : ; ; ; ; , through continuous iterative calculation, obtain the control parameters of the traveling speed, steering angle of the robotic arm's mobile chassis, and the rotation angle, angular velocity, and angular acceleration of each joint.

[0032] Control Execution and Feedback Module: Through the generated control instructions, it drives the robotic arm to perform separation and hemostasis operations, and collects data such as the position of the instrument (error ≤ 0.05 mm) and tissue contact force (resolution 0.01 N) in real time through sensors, calculates the deviation from the planned path (such as the temperature in the cauterized area of the electrosurgical knife exceeding the safety threshold by 1 °C), evaluates the intensity of respiratory movement interference, and feeds back the deviation information to the intelligent decision-making module to trigger the robotic arm motion optimization algorithm to adjust the next action. The formula of the robotic arm motion optimization algorithm is: .

[0033] In summary, in the medical surgery scenario, this solution realizes the precise modeling and real-time monitoring of blood vessels and tissues in the surgical area through sub-millimeter sensor accuracy and quantum state data fusion. The spatio-temporal prediction model warns of the contact risk between the instrument and key structures 0.5 seconds in advance, combines with the multi-objective optimization algorithm to control the operation error within 0.1 mm, and at the same time reduces the risk of thermal damage of the electrosurgical knife by 40% through dynamic weight adjustment. The physiological motion compensation mechanism effectively offsets the organ displacement caused by the patient's breathing (compensation accuracy 0.05 mm), improves the stability of laparoscopic surgery, is especially suitable for high-precision operations such as hepatobiliary vascular separation and microsuturing, reduces intraoperative blood loss and the incidence of postoperative complications.

[0034] The above is only a preferred embodiment of the present invention, and it does not impose any form of limitation on the present invention. Although the present invention has been disclosed as above with a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art can make some changes or modifications to the above-disclosed technical content to obtain an equivalent embodiment with equivalent changes. However, as long as the technical content of the present invention is not departed from, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. An intelligent robotic arm control system, characterized in that, The system includes: Sensor data acquisition module: Deploy a variety of sensors on the robotic arm to build a multi-dimensional perception network, and collect the original data of the distance, image, joint angle, sound and temperature field of the environment around the robotic arm at different frequencies in real time, and transmit it to the data processing center for storage; Multi-modal data fusion module: Extract the collected data from the data processing center, perform spatio-temporal calibration and feature extraction, and use the fusion algorithm to fuse the feature data of different types of sensors to generate the fused environmental information; Environment and target prediction module: Based on the fused environmental information and combined with historical data, analyze from the time dimension and the space dimension respectively, and perform short-term prediction on the future state of the target object through the target state prediction integration model according to the analysis results; Intelligent decision-making and control module: Receive and preprocess the fused environmental information and prediction results, use the decision optimization algorithm to optimize the control input sequence, plan the motion strategies of the robotic arm mobile chassis and each joint, and generate control instructions to send to the control execution and feedback module. At the same time, receive the actual motion state information of the robotic arm feedback by the control execution and feedback module, and use the robotic arm motion optimization algorithm to adjust the instructions in real time; Control execution and feedback module: Receive the control instructions generated in the intelligent decision-making and control module, drive the robotic arm to move, and during the operation, collect the actual state data of the robotic arm through sensors in real time, evaluate the target state deviation and environmental interference, and feedback to the intelligent decision-making and control module.

2. The intelligent robotic arm control system according to claim 1, wherein The sensors used in the sensor data acquisition module are: Distance sensor: Use lidar and install it at the center of the top of the robotic arm; Vision sensor: Use a binocular stereo camera and install it on the outside of the forearm; Position sensor: Install high-precision absolute encoders at the rotation axes of each joint; Sound sensor: Use a ring array composed of MEMS microphones and install it around the upper half of the robotic arm shell; Infrared sensor: Use the FLIR Lepton 3.5 thermal imaging module and install it on the side of the end effector.

3. An intelligent robotic arm control system according to claim 1, characterized in that, In the multi-modal data fusion module, a fusion algorithm is used to fuse the feature data of different types of sensors. The formula of the fusion algorithm is as follows: , where: is the fused environmental information vector, is the quantum state fusion function, is the coupling coefficient simulating quantum entanglement, is the dynamic weight of the th modality data, is the original data of the th modality, is the index variable.

4. An intelligent robotic arm control system according to claim 1, characterized in that, The specific steps for analyzing the fused environmental information from the time dimension and the space dimension in the environment and target prediction module are: Analysis in the time dimension: Arrange the fused environmental information in chronological order to form the time series data of the target object state, and perform smoothing processing. At the same time, determine the change trends of position, attitude and temperature, judge the speed change state, detect whether there is a periodic change rule in the time series, and synchronously perform outlier detection on the time series data; Analysis in the space dimension: Build a three-dimensional coordinate system for the working area of the robotic arm, map the target object and the obstacles in the surrounding environment to the coordinate system, calculate the relative distance and azimuth angle between the target and the surrounding objects according to their spatial coordinate positions in the three-dimensional coordinate system, and extract the features of the spatial structure of the closed area and the motion channel by observing the distribution form of the environmental objects, and evaluate the influence mode and degree of the surrounding environmental factors on the target object attitude and temperature.

5. An intelligent robotic arm control system according to claim 1, characterized in that The environment and target prediction module makes a short-term prediction of the future state of the target object based on the analysis results through the target state prediction integration model. The calculation formula of the target state prediction integration model is as follows: , where is the state vector of the predicted target object at the next time point, is the fusion weight of the prediction results in the time dimension and the space dimension, is the state predicted by the time series model and obtained by training with historical time series data, is the th correction value of the target state by the spatial correlation object, obtained by statistically analyzing the change law of the target state under the spatial relationship in historical data, is the k th weight of the spatial correlation object, k is the index variable, is the number of surrounding objects included in the analysis.

6. The intelligent robotic arm control system according to claim 1, characterized in that, In the intelligent decision-making and control module, the control input sequence of the robotic arm is optimized according to the decision optimization algorithm, and the formula of the decision optimization algorithm is as follows: , where is the control input sequence of the robotic arm, is the motion energy consumption function, is the distance penalty term between the robotic arm and the obstacle, is the change amount of adjacent control inputs, and are weight coefficients dynamically adjusted according to the obstacle density and the environmental temperature change rate.

7. An intelligent robotic arm control system according to claim 1, wherein, The specific steps for planning the motion strategies of the mobile chassis and each joint of the robotic arm in the intelligent decision-making and control module are as follows: Analyze the received fused environmental information and prediction results to determine the type, priority, accuracy requirements of the current task executed by the robotic arm, as well as the position, attitude, motion trend of the target object and the information on the distribution of surrounding environmental obstacles. Based on the real-time distance between the target object and the robotic arm at present, and the parameters of the predicted motion speed and acceleration of the target, construct a multi-objective optimization function with the completion time, energy consumption, motion smoothness, and safety as the core objectives. According to the actual requirements of the task, set corresponding weight coefficients for each objective, combine the kinematic and dynamic constraint conditions of the robotic arm to form a multi-objective optimization model, and solve the multi-objective optimization model. Through continuous iterative calculations, obtain the control parameters of the traveling speed, steering angle of the mobile chassis of the robotic arm, as well as the rotation angle, angular velocity, and angular acceleration of each joint.

8. An intelligent robotic arm control system according to claim 7, wherein, The intelligent decision-making and control module constructs a multi-objective optimization function with the core objectives of task completion time, energy consumption, motion smoothness, and safety. Specifically, let be the optimization objective function, be the task completion time, be the reference completion time, be the energy consumption, be the reference energy consumption, be the motion smoothness index, be the safety assessment index. Then the formula of the multi-objective optimization function is: ; : ; ; ; ; , where , , , are the weight coefficients of the corresponding objectives respectively, and , which are dynamically adjusted according to the task type, represents the kinematic constraint function of the robotic arm, represents the dynamic constraint function of the robotic arm, is the joint angle vector of the robotic arm, is the joint angular velocity vector, is the joint angular acceleration vector, is the real-time distance between the target object and the robotic arm, measured through sensor data, is the maximum allowable distance value, is the target predicted speed, obtained by predicting the motion speed of the robotic arm through a prediction model, is the maximum allowable speed value, is the target predicted acceleration, calculated by the prediction model according to the data of the robotic arm motion state, is the maximum allowable acceleration value.

9. An intelligent robotic arm control system according to claim 1, characterized in that, In the intelligent decision-making and control module, the robotic arm motion optimization algorithm is used to adjust the instructions in real time. The formula of the robotic arm motion optimization algorithm is as follows: , where: is the control instruction of the robotic arm after being adjusted in real time by the optimization algorithm at time , is the control instruction received by the intelligent decision-making and control module, is the deviation vector between the actual state and the target state of the robotic arm, is the estimated value of the environmental interference intensity, is the feedback signal, is the time.

Images