Intelligent optimization method and system for loading and unloading arm mechanism based on reinforcement learning

Abstract

The invention relates to the field of mechanical engineering, and discloses an intelligent optimization method and system for a loading and unloading arm mechanism based on reinforcement learning, and the method comprises the steps: extracting environment data of an application scene, extracting physical parameters of a loading and unloading arm, analyzing a working process of the loading and unloading arm, and extracting operation data of the working process; analyzing joint torque and joint acceleration of the loading and unloading arm, calculating operation performance indexes of the loading and unloading arm, and analyzing working stability of the loading and unloading arm; analyzing an action sequence and an operation path of the loading and unloading arm, analyzing moment limitation and motion constraint of the loading and unloading arm, and determining an optimal operation path of the loading and unloading arm; constructing a sensor network of the loading and unloading arm, collecting working data of the loading and unloading arm in real time, analyzing risk factors in the working data, and calculating the risk probability of the loading and unloading arm; and a reinforcement learning optimization model of the loading and unloading arm is constructed, optimization parameters of the loading and unloading arm are analyzed, and intelligent optimization of the loading and unloading arm is executed. The operation efficiency and safety of the loading and unloading arm can be improved.

Description

Technical Field

[0001] The present invention relates to an intelligent optimization method and system for a loading and unloading arm mechanism based on reinforcement learning, belonging to the field of mechanical engineering. Background Art

[0002] A loading and unloading arm refers to a mechanical device for loading and unloading goods, which is usually installed on a fixed base or a moving vehicle and used to transfer goods from one place to another. The goal of intelligent optimization of the loading and unloading arm mechanism is to achieve higher operation efficiency, better operation performance, higher safety and lower energy consumption through the intelligent transformation of the loading and unloading arm, so as to improve the overall performance and economic benefits of the loading and unloading arm.

[0003] Currently, due to the usually complex design and programming of the loading and unloading arm, in the face of some complex tasks, the loading and unloading arm requires special customization and adjustment, and affected by external factors, the flexibility of the loading and unloading arm is poor and the accuracy is greatly affected, resulting in the work efficiency and safety of the loading and unloading arm being affected.

[0004] Therefore, there is an urgent need for a solution to improve the operation efficiency and safety of the loading and unloading arm. Summary of the Invention

[0005] The present invention provides an intelligent optimization method and system for a loading and unloading arm mechanism based on reinforcement learning, and its main purpose is to improve the operation efficiency and safety of the loading and unloading arm.

[0006] To achieve the above purpose, an intelligent optimization method for a loading and unloading arm mechanism based on reinforcement learning provided by the present invention includes:

[0007] Obtain the application scenarios of the loading and unloading arm, extract the environmental data of the application scenarios, extract the physical parameters of the loading and unloading arm, analyze the working process of the loading and unloading arm, and extract the operation data of the working process;

[0008] According to the environmental data, the physical parameters and the operation data, use the trained dynamic analysis model to analyze the joint torque and joint acceleration of the loading and unloading arm, calculate the operation performance index of the loading and unloading arm according to the joint torque and the joint acceleration, and analyze the working stability of the loading and unloading arm according to the operation performance index;

[0009] Analyze the action sequence and operation path of the loading and unloading arm, analyze the torque limit and motion constraint of the loading and unloading arm according to the action sequence and the operation path, and use the preset path planning algorithm to determine the optimal operation path of the loading and unloading arm according to the torque limit and the motion constraint;

[0010] Construct a sensor network for the loading and unloading arm. According to the sensor network, collect the working data of the loading and unloading arm in real time, analyze the risk factors in the working data, and calculate the risk probability of the loading and unloading arm using a preset risk analysis model based on the risk factors.

[0011] Construct a reinforcement learning optimization model for the loading and unloading arm based on the working stability, the optimal operation path, and the risk probability. According to the reinforcement learning optimization model, analyze the optimization parameters of the loading and unloading arm, and perform intelligent optimization of the loading and unloading arm based on the optimization parameters.

[0012] Optionally, the extraction of the physical parameters of the loading and unloading arm includes:

[0013] Analyze the geometric shape of the loading and unloading arm, and measure the dimensional parameters of the loading and unloading arm according to the geometric shape.

[0014] Identify the structural material of the loading and unloading arm, and analyze the material properties of the structural material.

[0015] Construct a geometric model of the loading and unloading arm based on the dimensional parameters and the material properties.

[0016] Conduct a simulation test on the geometric model of the loading and unloading arm to obtain test data.

[0017] Calculate the bending stress of the loading and unloading arm according to the test data.

[0018] Determine the physical parameters of the loading and unloading arm based on the bending stress, the dimensional parameters, and the material properties.

[0019] Optionally, the analysis of the joint torque and joint acceleration of the loading and unloading arm using the trained dynamic analysis model according to the environmental data, the physical parameters, and the operation data includes:

[0020] Extract the dynamic equations of the dynamic analysis model.

[0021] Analyze the external forces acting on the loading and unloading arm according to the environmental data.

[0022] Calculate the inertial force effect of the loading and unloading arm according to the physical parameters and the operation data.

[0023] Calculate the joint torque and joint acceleration of the loading and unloading arm using the dynamic equations based on the inertial force effect and the external forces.

[0024] Optionally, the calculation of the operation performance indicators of the loading and unloading arm according to the joint torque and the joint acceleration includes:

[0025] Calculate the joint power of the loading and unloading arm according to the joint torque and the joint acceleration;

[0026] Analyze the operation energy consumption of the loading and unloading arm according to the joint power;

[0027] Calculate the torque fluctuation of the loading and unloading arm according to the joint torque;

[0028] Determine the operation performance index of the loading and unloading arm according to the joint power, the operation energy consumption and the torque fluctuation.

[0029] Optionally, the analyzing the action sequence and operation path of the loading and unloading arm includes:

[0030] Determine the operation target and constraint conditions of the loading and unloading arm;

[0031] Construct a motion analysis model of the loading and unloading arm according to the operation target and the constraint conditions;

[0032] Analyze the action sequence and action logic relationship of the loading and unloading arm according to the motion analysis model;

[0033] Determine the action sequence of the loading and unloading arm based on the action sequence and the action logic relationship;

[0034] Analyze the operation path of the loading and unloading arm according to the action sequence.

[0035] Optionally, the determining the optimal operation path of the loading and unloading arm according to the torque limit and the motion constraint by using a preset path planning algorithm includes:

[0036] Perform a simulation test on the operation path corresponding to the loading and unloading arm according to the torque limit and the motion constraint to obtain simulation test data;

[0037] Calculate the path feasibility and path length of the operation path according to the simulation test data;

[0038] When the path feasibility does not meet the preset path feasibility threshold or the path length is higher than the preset path length threshold, optimize the operation path by using the path planning algorithm to obtain an optimized path;

[0039] Calculate the optimized path feasibility and optimized path length of the optimized path;

[0040] When the optimized path feasibility meets the path feasibility threshold and the optimized path length is lower than the path length threshold, use the optimized path as the optimal operation path of the loading and unloading arm.

[0041] Optionally, the constructing the sensor network of the loading and unloading arm includes:

[0042] Determine the monitoring objectives of the loading and unloading arm, and configure the sensors of the loading and unloading arm according to the monitoring objectives;

[0043] Determine the network topology and communication protocol of the loading and unloading arm;

[0044] Construct the network architecture of the loading and unloading arm according to the network topology and the communication protocol;

[0045] Construct the data acquisition unit of the sensor, and configure the network parameters of the network architecture;

[0046] Integrate the sensor network of the loading and unloading arm according to the sensor, the network architecture, the data acquisition unit, and the network parameters.

[0047] Optionally, the calculating the risk probability of the loading and unloading arm by using a preset risk analysis model according to the risk factor includes:

[0048] Calculate the risk coefficient of the risk factor by using the risk analysis model according to the working data corresponding to the loading and unloading arm;

[0049] Determine the initial risk probability of the loading and unloading arm based on the risk coefficient;

[0050] Calculate the risk probability of the loading and unloading arm according to the initial risk probability, the risk coefficient, and the risk factor.

[0051] Optionally, the constructing the reinforcement learning optimization model of the loading and unloading arm according to the working stability, the optimal operation path, and the risk probability includes:

[0052] Define the state space, action space, and reward function of the loading and unloading arm according to the working stability, the optimal operation path, and the risk probability;

[0053] Determine the reinforcement learning algorithm of the loading and unloading arm according to the state space, the action space, and the reward function;

[0054] Construct the initial optimization model of the loading and unloading arm;

[0055] Train the initial optimization model by using preset training data based on the reinforcement learning algorithm to obtain a trained model;

[0056] Identify the key parameters of the trained model, and verify the model performance of the trained model according to the key parameters;

[0057] When the model performance meets the preset model performance threshold, use the trained model as the reinforcement learning optimization model of the loading and unloading arm.

[0058] To solve the above problems, the present invention also provides an intelligent optimization system for a loading and unloading arm mechanism based on reinforcement learning. The system includes:

[0059] A data extraction module, configured to obtain the application scenarios of the loading and unloading arm, extract the environmental data of the application scenarios, extract the physical parameters of the loading and unloading arm, analyze the working process of the loading and unloading arm, and extract the operation data of the working process;

[0060] A stability analysis module, configured to analyze the joint torque and joint acceleration of the loading and unloading arm by using a trained dynamic analysis model according to the environmental data, the physical parameters, and the operation data, calculate the operation performance index of the loading and unloading arm according to the joint torque and the joint acceleration, and analyze the working stability of the loading and unloading arm according to the operation performance index;

[0061] A path planning module, configured to analyze the action sequence and operation path of the loading and unloading arm, analyze the torque limit and motion constraints of the loading and unloading arm according to the action sequence and the operation path, and determine the optimal operation path of the loading and unloading arm by using a preset path planning algorithm according to the torque limit and the motion constraints;

[0062] A risk analysis module, configured to construct a sensor network of the loading and unloading arm, collect the working data of the loading and unloading arm in real time according to the sensor network, analyze the risk factors in the working data, and calculate the risk probability of the loading and unloading arm by using a preset risk analysis model according to the risk factors;

[0063] An intelligent optimization module, configured to construct a reinforcement learning optimization model of the loading and unloading arm according to the working stability, the optimal operation path, and the risk probability, analyze the optimization parameters of the loading and unloading arm according to the reinforcement learning optimization model, and perform intelligent optimization of the loading and unloading arm based on the optimization parameters.

[0064] By obtaining the application scenarios of the loading and unloading arm, the embodiments of the present invention can ensure the stability and safety of the loading and unloading arm in complex or harsh environments, reduce the risk of accidents, and thus improve the efficiency of loading and unloading operations. Optionally, according to the environmental data, the physical parameters, and the operation data, the embodiments of the present invention can analyze the joint torque and joint acceleration of the loading and unloading arm by using a trained dynamic analysis model to identify the factors that may cause mechanism vibration, thereby optimizing the structure, reducing vibration, and improving the stability of the loading and unloading arm. By analyzing the torque limit and motion constraints of the loading and unloading arm according to the action sequence and the operation path, the embodiments of the invention can improve the performance of the loading and unloading arm, reduce the operation cost, and enhance the reliability of the operation on the premise of ensuring safety. By collecting the working data of the loading and unloading arm in real time according to the sensor network, the embodiments of the present invention can monitor the key components of the loading and unloading arm in real time. Once abnormal data is detected, measures can be taken immediately to prevent accidents and improve operation safety. By calculating the risk probability of the loading and unloading arm by using a preset risk analysis model according to the risk factors, the embodiments of the present invention can identify high-risk areas and optimize the design of the loading and unloading arm to improve its safety and reliability. Finally, by constructing a reinforcement learning optimization model of the loading and unloading arm according to the working stability, the optimal operation path, and the risk probability, the embodiments of the present invention can significantly improve the automation and intelligence level of loading and unloading operations, reduce human errors, improve operation efficiency and safety, and at the same time reduce maintenance costs and operation risks. Therefore, the intelligent optimization method and system for the loading and unloading arm mechanism based on reinforcement learning provided by the embodiments of the present invention can improve the operation efficiency and safety of the loading and unloading arm. Description of the Drawings

[0065] Figure 1 It is a schematic flowchart of an intelligent optimization method for a loading and unloading arm mechanism based on reinforcement learning provided by an embodiment of the present invention;

[0066] Figure 2 It is a schematic diagram of modules for implementing the intelligent optimization method for a loading and unloading arm mechanism based on reinforcement learning provided by an embodiment of the present invention.

[0067] The implementation, functional features, and advantages of the present invention will be further described with reference to the embodiments and the accompanying drawings. Detailed Embodiments

[0068] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0069] The embodiments of the present application provide an intelligent optimization method for a loading and unloading arm mechanism based on reinforcement learning. The execution subject of the intelligent optimization method for the loading and unloading arm mechanism based on reinforcement learning includes, but is not limited to, at least one of electronic devices such as a server, a terminal, etc. that can be configured to execute the method provided by the embodiments of the present application. In other words, the intelligent optimization method for the loading and unloading arm mechanism based on reinforcement learning can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to: a single server, a server cluster, a cloud server, or a cloud server cluster, etc.

[0070] Embodiment 1:

[0071] Referring to Figure 1 As shown, it is a schematic flowchart of an intelligent optimization method for a loading and unloading arm mechanism based on reinforcement learning provided by an embodiment of the present invention. In this embodiment, the intelligent optimization method for the loading and unloading arm mechanism based on reinforcement learning includes:

[0072] S1. Obtain the application scenarios of the loading and unloading arm, extract the environmental data of the application scenarios, extract the physical parameters of the loading and unloading arm, analyze the working process of the loading and unloading arm, and extract the operation data of the working process.

[0073] By obtaining the application scenarios of the loading and unloading arm, the embodiments of the present invention can ensure the stability and safety of the loading and unloading arm in complex or harsh environments, reduce the risk of accidents, and thus improve the efficiency of loading and unloading operations. Among them, the application scenarios refer to the environments and conditions in which the loading and unloading arm is specifically used, and these scenarios usually have specific requirements and demands, such as temperature, humidity, wind speed, etc.

[0074] By extracting the environmental data of the application scenarios, the embodiments of the present invention can optimize the operation parameters of the loading and unloading arm, thereby improving the operation efficiency and operation speed. Among them, the environmental data refers to the data of various natural and industrial conditions related to the application scenarios of the loading and unloading arm.

[0075] Optionally, as an embodiment of the present invention, the extraction of the environmental data of the application scenarios can be performed through sensor technology.

[0076] By extracting the physical parameters of the loading and unloading arm, the embodiments of the present invention can optimize the structural design of the loading and unloading arm, improve its load-bearing capacity, telescopic speed, and operation accuracy, and thus enhance the overall performance. Among them, the physical parameters refer to a series of measurement indexes involved in the design, manufacture, and operation of the loading and unloading arm, such as dimension parameters, material properties, etc.

[0077] As an embodiment of the present invention, the extraction of the physical parameters of the loading and unloading arm includes:

[0078] Analyze the geometric shape of the loading and unloading arm, and measure the dimension parameters of the loading and unloading arm according to the geometric shape;

[0079] Identify the structural material of the loading and unloading arm and analyze the material properties of the structural material;

[0080] Construct a geometric model of the loading and unloading arm according to the dimensional parameters and the material properties;

[0081] Conduct a simulation test on the geometric model of the loading and unloading arm to obtain test data;

[0082] Calculate the bending stress of the loading and unloading arm using the following formula according to the test data:

[0083]

[0084] where W y represents the bending stress, X represents the distance of the farthest fiber corresponding to the test data, H represents the acting load corresponding to the test data, B represents the acting distance from the load point corresponding to the test data to the fixed point, and E represents the cross-sectional diameter of the loading and unloading arm corresponding to the dimensional data;

[0085] Determine the physical parameters of the loading and unloading arm according to the bending stress, the dimensional parameters, and the material properties.

[0086] Among them, the geometric shape refers to the overall structure and appearance of the loading and unloading arm, such as the cross-sectional shape, the number of arm segments, etc. The dimensional parameters refer to the specific measurement indexes involved in the design and manufacture of the loading and unloading arm, such as the cross-sectional diameter, the length of the joint arm, etc. The structural material refers to various materials used to manufacture the loading and unloading arm. The material properties refer to the physical, chemical, and mechanical characteristics of the material itself. The geometric model of the loading and unloading arm refers to a virtual and digital representation of the loading and unloading arm, which includes the actual size, shape, structure, and other relevant geometric features of the loading and unloading arm. The test data refers to a series of data collected during the simulation test of the geometric model of the loading and unloading arm. The bending stress refers to the internal stress generated on the cross-section of the loading and unloading arm or other structural elements when they are subjected to a bending moment.

[0087] Optionally, the construction of the geometric model of the loading and unloading arm according to the dimensional parameters and the material properties can be constructed by means of parametric modeling.

[0088] In the embodiment of the present invention, by analyzing the working process of the loading and unloading arm, the bottlenecks and reasons for delays in the loading and unloading process can be identified, so as to optimize the action sequence and speed of the loading and unloading arm and improve the overall operation efficiency. Among them, the working process refers to a series of steps and operation sequences experienced by the loading and unloading arm when completing the loading and unloading task.

[0089] Optionally, as an embodiment of the present invention, the analysis of the working process of the loading and unloading arm can be monitored and analyzed through Internet of Things technology.

[0090] In the embodiment of the present invention, by extracting the operation data of the work process, delays and waiting times during the operation process can be identified, thereby optimizing the operation sequence and operation process, reducing unnecessary pauses, and improving the loading and unloading efficiency. Among them, the operation data refers to various information and records related to the operation of the loading and unloading arm.

[0091] Optionally, as an embodiment of the present invention, the operation data of the work process can be extracted by an automated data collection technology.

[0092] S2. According to the environmental data, the physical parameters, and the operation data, use the trained dynamic analysis model to analyze the joint torque and joint acceleration of the loading and unloading arm. According to the joint torque and the joint acceleration, calculate the operation performance index of the loading and unloading arm. According to the operation performance index, analyze the working stability of the loading and unloading arm.

[0093] In the embodiment of the present invention, by analyzing the joint torque and joint acceleration of the loading and unloading arm using the trained dynamic analysis model according to the environmental data, the physical parameters, and the operation data, factors that may cause mechanism vibration can be identified, thereby performing structural optimization, reducing vibration, and improving the stability of the loading and unloading arm. Among them, the trained dynamic analysis model analysis refers to a mathematical model that has been trained with a large amount of data and can simulate and predict the dynamic behavior of a mechanical system. The joint torque refers to the product of the force acting on the joint of the loading and unloading arm and the force arm. The joint acceleration refers to the angular acceleration of the joint of the loading and unloading arm during its movement.

[0094] As an embodiment of the present invention, the analyzing the joint torque and joint acceleration of the loading and unloading arm using the trained dynamic analysis model according to the environmental data, the physical parameters, and the operation data includes:

[0095] Extract the dynamic equation of the dynamic analysis model;

[0096] According to the environmental data, analyze the external force acting on the loading and unloading arm;

[0097] According to the physical parameters and the operation data, use the following formula to calculate the inertial force effect of the loading and unloading arm:

[0098]

[0099] Among them, G represents the inertial force effect, z i represents the link mass of the i-th link corresponding to the physical parameter, u i represents the position vector from the joint axis to the centroid of the i-th link corresponding to the physical parameter, s represents the joint angular velocity vector corresponding to the operation data, Represents the rate of change of the joint angular velocity vector over time;

[0100] According to the inertial force effect and the external acting force, use the dynamic equation to calculate the joint torque and joint acceleration of the handling arm.

[0101] Wherein, the dynamic equation refers to the equation describing the motion state of the mechanical system. The external acting force refers to various forces acting on the handling arm caused by the external environment or operating conditions during the operation of the handling arm. The inertial force effect refers to the force caused by the change of the motion state of the handling arm (such as acceleration, deceleration or change of motion direction). The position vector refers to a set of coordinate values from the joint axis to the center of mass of the connecting rod. The joint angular velocity vector refers to a set of vectors describing the rotational speed of each joint of the handling arm.

[0102] Optionally, analyzing the external acting force of the handling arm according to the environmental data can be analyzed by computational fluid dynamics simulation.

[0103] In the embodiment of the present invention, by calculating the operation performance index of the handling arm according to the joint torque and the joint acceleration, the precise control of the joint torque and acceleration can be optimized, thereby improving the positioning accuracy of the handling arm and the accuracy of repetitive operations. Wherein, the operation performance index refers to a series of quantitative indexes used to evaluate the performance of the robotic arm or handling arm.

[0104] As an embodiment of the present invention, calculating the operation performance index of the handling arm according to the joint torque and the joint acceleration includes:

[0105] Calculate the joint power of the handling arm according to the joint torque and the joint acceleration;

[0106] Analyze the operation energy consumption of the handling arm according to the joint power;

[0107] According to the joint torque, use the following formula to calculate the torque fluctuation of the handling arm:

[0108]

[0109] Where, L b Represents the torque fluctuation, m represents the total number of joints corresponding to the handling arm, R j Represents the joint torque of the jth joint corresponding to the handling arm;

[0110] Determine the operation performance index of the handling arm according to the joint power, the operation energy consumption and the torque fluctuation.

[0111] Among them, the joint power refers to the power generated by each joint of the robotic arm during movement. The operation energy consumption refers to the total energy consumed during the process of completing a specific operation task. The torque fluctuation refers to the degree of change of the loading and unloading arm joint torque over time during operation.

[0112] In the embodiment of the present invention, by analyzing the working stability of the loading and unloading arm according to the operation performance index, it can adapt to a wider range of working conditions and environments, and improve its adaptability in different application scenarios. Among them, the working stability refers to the ability of the loading and unloading arm to maintain stable operation when performing loading and unloading tasks.

[0113] Optionally, as an embodiment of the present invention, the analysis of the working stability of the loading and unloading arm according to the operation performance index can be analyzed by the finite element analysis method.

[0114] S3. Analyze the action sequence and operation path of the loading and unloading arm. According to the action sequence and the operation path, analyze the torque limit and motion constraint of the loading and unloading arm. According to the torque limit and the motion constraint, use a preset path planning algorithm to determine the optimal operation path of the loading and unloading arm.

[0115] In the embodiment of the present invention, by analyzing the action sequence and operation path of the loading and unloading arm, the degree of freedom and adaptability of the joint can be improved, so that the loading and unloading arm can complete more diverse operation tasks. Among them, the action sequence refers to a series of ordered actions required for the loading and unloading arm to complete a complete loading and unloading operation process. The operation path refers to the entire movement trajectory of the end effector of the loading and unloading arm from the initial position to the target position and back to the initial position when performing the loading and unloading operation.

[0116] As an embodiment of the present invention, the analysis of the action sequence and operation path of the loading and unloading arm includes:

[0117] Determine the operation target and constraint conditions of the loading and unloading arm;

[0118] According to the operation target and the constraint conditions, construct a motion analysis model of the loading and unloading arm;

[0119] According to the motion analysis model, analyze the action sequence and action logic relationship of the loading and unloading arm;

[0120] Based on the action sequence and the action logic relationship, determine the action sequence of the loading and unloading arm;

[0121] According to the action sequence, analyze the operation path of the loading and unloading arm.

[0122] Among them, the operation target refers to the specific goals and requirements that the loading and unloading arm needs to achieve when performing loading and unloading tasks. The constraint conditions refer to the limiting factors that must be considered in the design, planning, and execution of the loading and unloading arm operation. The motion analysis model refers to a mathematical model used to analyze and predict the motion behavior of the loading and unloading arm when performing specific tasks. The action sequence refers to the arrangement in which each action of the loading and unloading arm follows a certain logic and time sequence when completing a specific loading and unloading task. The action logical relationship refers to the dependency and sequential relationship existing between each action when the loading and unloading arm performs a series of actions.

[0123] Optionally, the motion analysis model of the loading and unloading arm constructed according to the operation target and the constraint conditions can be constructed by multi-body dynamics.

[0124] By analyzing the torque limit and motion constraints of the loading and unloading arm according to the action sequence and the operation path in the embodiments of the present invention, it is possible to improve the performance of the loading and unloading arm, reduce the operation cost, and enhance the reliability of the operation on the premise of ensuring safety. Among them, the torque limit refers to the maximum torque value that each joint or drive device of the loading and unloading arm can withstand during its movement. The motion constraint refers to the physical rules and conditions that limit the movement of the loading and unloading arm in actual operation.

[0125] Optionally, as an embodiment of the present invention, the analysis of the torque limit and motion constraints of the loading and unloading arm according to the action sequence and the operation path can be analyzed by machine learning methods.

[0126] By using a preset path planning algorithm to determine the optimal operation path of the loading and unloading arm according to the torque limit and the motion constraints in the embodiments of the present invention, unnecessary movements can be reduced, the operation cycle can be shortened, and the number of operations per unit time can be increased. Among them, the preset path planning algorithm refers to a series of calculation steps or rules designed to solve specific problems and used to automatically generate the optimal or optimized path of a robotic arm or other similar mechanical devices. The optimal operation path refers to the optimal path for the loading and unloading arm to reach the target position from the starting position and complete the specified task when performing loading and unloading operations under certain conditions.

[0127] As an embodiment of the present invention, the determination of the optimal operation path of the loading and unloading arm by using a preset path planning algorithm according to the torque limit and the motion constraints includes:

[0128] Performing a simulation test on the operation path corresponding to the loading and unloading arm according to the torque limit and the motion constraints to obtain simulation test data;

[0129] Calculating the path feasibility and path length of the operation path according to the simulation test data;

[0130] When the path feasibility does not meet the preset path feasibility threshold or the path length is higher than the preset path length threshold, the path planning algorithm is used to optimize the operation path to obtain an optimized path;

[0131] Calculate the optimized path feasibility and optimized path length of the optimized path;

[0132] When the optimized path feasibility meets the path feasibility threshold and the optimized path length is lower than the path length threshold, the optimized path is used as the best operation path of the loading and unloading arm.

[0133] Among them, the simulation test data refers to a series of data collected during the simulation of the operation path of the loading and unloading arm. The path feasibility refers to the actual executability of the planned path when the loading and unloading arm performs a specific operation task. The path length refers to the actual distance traveled by the loading and unloading arm from the starting position to the target position in path planning. The preset path feasibility threshold refers to a set of critical values or standards set to ensure that the loading and unloading arm can complete the operation task safely and effectively during path planning. The preset path length threshold refers to a maximum allowable path length value set to evaluate the path efficiency during the path planning of the loading and unloading arm. The optimized path refers to a new operation path obtained by adjusting and improving through the path planning algorithm. The optimized path feasibility refers to whether the optimized operation path can be successfully executed by the loading and unloading arm during actual operation, while meeting all preset constraint conditions and performance standards. The optimized path length refers to the total length of the path required to pass from the starting position to the target position after optimizing the operation path of the loading and unloading arm through the path planning algorithm.

[0134] S4. Construct a sensor network for the loading and unloading arm, and according to the sensor network, collect the working data of the loading and unloading arm in real time, analyze the risk factors in the working data, and calculate the risk probability of the loading and unloading arm according to the risk factors by using a preset risk analysis model.

[0135] In the embodiment of the present invention, potential safety hazards can be discovered in time by constructing the sensor network of the loading and unloading arm, and measures can be taken to prevent accidents. Among them, the sensor network refers to a system composed of multiple sensors, and these sensors are arranged at key parts of the loading and unloading arm to monitor and collect various physical parameters and data during the operation of the loading and unloading arm.

[0136] As an embodiment of the present invention, the construction of the sensor network for the loading and unloading arm includes:

[0137] Determine the monitoring target of the loading and unloading arm, and configure the sensors of the loading and unloading arm according to the monitoring target;

[0138] Determine the network topology structure and communication protocol of the loading and unloading arm;

[0139] Construct the network architecture of the loading and unloading arm according to the network topology structure and the communication protocol;

[0140] Construct the data acquisition unit of the sensor, and configure the network parameters of the network architecture;

[0141] Integrate the sensor network of the loading and unloading arm according to the sensor, the network architecture, the data acquisition unit and the network parameters.

[0142] Among them, the monitoring target refers to the key performance indicators and status parameters that need to be monitored during the operation of the loading and unloading arm. The sensor refers to an electronic device used to monitor and control various performance indicators and status parameters of the loading and unloading arm. The network topology structure refers to the connection method and organizational form among various sensors, data acquisition units, data processing centers and communication devices in the sensor network. The communication protocol refers to a set of rules and standards that define the data format, transmission sequence, error detection and correction methods, and data exchange mechanism for communication between devices in the sensor network. The network architecture refers to the overall structure design of the loading and unloading arm sensor network. The data acquisition unit refers to a device or system used to collect data from sensors and convert it into a usable digital format for further processing. The network parameters refer to the key parameters that need to be configured and optimized during the design and operation of the sensor network.

[0143] Optionally, the determination of the network topology structure and communication protocol of the loading and unloading arm can be determined by the method integrated by the Internet of Things platform.

[0144] In the embodiment of the present invention, by collecting the working data of the loading and unloading arm in real time according to the sensor network, the key components of the loading and unloading arm can be monitored in real time. Once abnormal data is detected, measures can be taken immediately to prevent accidents and improve operation safety. Among them, the working data refers to a series of data related to the operation and performance of the loading and unloading arm collected by the sensor network installed on the loading and unloading arm.

[0145] In the embodiment of the present invention, a risk database can be established by analyzing the risk factors in the working data, and historical risk events can be analyzed to better understand and manage risks. Among them, the risk factor refers to various factors that may cause safety risks, performance degradation or equipment failures during the operation of the loading and unloading arm.

[0146] Optionally, as an embodiment of the present invention, the analysis of the risk factors in the working data can be analyzed by the single-multi factor analysis method.

[0147] In the embodiments of the present invention, by calculating the risk probability of the loading and unloading arm using a preset risk analysis model according to the risk factors, high-risk areas can be identified to optimize the design of the loading and unloading arm, thereby improving its safety and reliability. Among them, the preset risk analysis model refers to a set of mathematical models or algorithms designed to evaluate and calculate the probability of potential risks during the operation of the loading and unloading arm. The risk probability refers to the likelihood of a specific risk event occurring for the loading and unloading arm under given conditions.

[0148] As an embodiment of the present invention, the calculating the risk probability of the loading and unloading arm using a preset risk analysis model according to the risk factors includes:

[0149] Calculating the risk coefficient of the risk factor using the risk analysis model according to the working data corresponding to the loading and unloading arm;

[0150] Determining the initial risk probability of the loading and unloading arm based on the risk coefficient;

[0151] Calculating the risk probability of the loading and unloading arm using the following formula according to the initial risk probability, the risk coefficient, and the risk factor:

[0152]

[0153] where F represents the risk probability, F0 represents the initial risk probability, k represents the number of factors of the risk factor, Y q represents the q-th risk factor, and θ q represents the risk coefficient of the q-th risk factor, and e represents the exponential function with base e.

[0154] Among them, the risk coefficient refers to the value used to quantify the influence degree of each risk factor on the risk probability of the loading and unloading arm in the risk analysis model. The initial risk probability refers to the inherent risk level of the loading and unloading arm without the influence of any risk factors.

[0155] Optionally, the determining the initial risk probability of the loading and unloading arm based on the risk coefficient can be determined through fault tree analysis.

[0156] S5. Constructing a reinforcement learning optimization model of the loading and unloading arm according to the working stability, the optimal operation path, and the risk probability, analyzing the optimization parameters of the loading and unloading arm according to the reinforcement learning optimization model, and performing intelligent optimization of the loading and unloading arm based on the optimization parameters.

[0157] In the embodiments of the present invention, by constructing an enhanced learning optimization model of the loading and unloading arm according to the working stability, the optimal operation path, and the risk probability, the automation and intelligent level of the loading and unloading operation can be significantly improved, human errors can be reduced, the operation efficiency and safety can be enhanced, and at the same time, the maintenance cost and operation risk can be lowered. Among them, the enhanced learning optimization model refers to a mathematical model based on the enhanced learning theory for optimizing the operation process of the loading and unloading arm.

[0158] As an embodiment of the present invention, the constructing of the enhanced learning optimization model of the loading and unloading arm according to the working stability, the optimal operation path, and the risk probability includes:

[0159] Defining the state space, action space, and reward function of the loading and unloading arm according to the working stability, the optimal operation path, and the risk probability;

[0160] Determining the enhanced learning algorithm of the loading and unloading arm according to the state space, the action space, and the reward function;

[0161] Constructing an initial optimization model of the loading and unloading arm;

[0162] Training the initial optimization model with preset training data based on the enhanced learning algorithm to obtain a trained model;

[0163] Identifying the key parameters of the trained model and verifying the model performance of the trained model according to the key parameters;

[0164] When the model performance meets the preset model performance threshold, using the trained model as the enhanced learning optimization model of the loading and unloading arm.

[0165] Among them, the state space refers to the set of all different environmental states that the loading and unloading arm may encounter when performing tasks. The action space refers to the set of all possible actions that the loading and unloading arm performs. The reward function refers to a function used to analyze the quality of the actions taken by the loading and unloading arm. The reinforcement learning algorithm refers to a computational method for solving reinforcement learning problems, which enables the agent to learn how to maximize the cumulative reward through exploration and exploitation in the environment. The initial optimization model refers to a model initially constructed according to the specific requirements of the loading and unloading arm and environmental characteristics before applying the reinforcement learning algorithm. The training model refers to a model in which the parameters have been adjusted and gradually converge after a certain number of iterative trainings during the reinforcement learning process. The key parameters refer to the parameters that have a significant impact on the model performance and training results in the reinforcement learning optimization model. The model performance refers to the performance metric of the reinforcement learning optimization model when completing a specific task. The preset model performance threshold refers to a set of criteria or boundaries set to determine whether the model has reached an acceptable or target performance level during the reinforcement learning training process.

[0166] In the embodiment of the present invention, by analyzing the optimization parameters of the loading and unloading arm according to the reinforcement learning optimization model, more accurate positioning and operation can be achieved, reducing the position error, improving the accuracy and efficiency of the loading and unloading operation, and at the same time enabling the loading and unloading arm to maintain stable operation in a complex or unstable environment, reducing faults and downtime. Among them, the optimization parameters refer to those parameters used to improve and optimize the operation performance of the loading and unloading arm.

[0167] In the embodiment of the present invention, by performing intelligent optimization of the loading and unloading arm based on the optimization parameters, it can better adapt to different operating environments and task requirements, and improve the flexibility and adaptability of the system.

[0168] By obtaining the application scenarios of the loading and unloading arm, the embodiments of the present invention can ensure the stability and safety of the loading and unloading arm in complex or harsh environments, reduce the risk of accidents, and thus improve the efficiency of loading and unloading operations; optionally, by analyzing the joint torque and joint acceleration of the loading and unloading arm using the trained dynamic analysis model according to the environmental data, the physical parameters, and the operation data, the embodiments of the present invention can identify the factors that may cause mechanism vibration, thereby optimizing the structure, reducing vibration, and improving the stability of the loading and unloading arm; by analyzing the torque limit and motion constraints of the loading and unloading arm according to the action sequence and the operation path, the embodiments of the invention can improve the performance of the loading and unloading arm, reduce the operation cost, and enhance the reliability of the operation on the premise of ensuring safety; by collecting the working data of the loading and unloading arm in real time according to the sensor network, the embodiments of the present invention can monitor the key components of the loading and unloading arm in real time. Once abnormal data is detected, measures can be taken immediately to prevent accidents and improve operation safety; by calculating the risk probability of the loading and unloading arm using the preset risk analysis model according to the risk factors, the embodiments of the present invention can identify high-risk areas and optimize the design of the loading and unloading arm to improve its safety and reliability. Finally, by constructing the reinforcement learning optimization model of the loading and unloading arm according to the working stability, the optimal operation path, and the risk probability, the embodiments of the present invention can significantly improve the automation and intelligence level of loading and unloading operations, reduce human errors, improve operation efficiency and safety, and at the same time reduce maintenance costs and operation risks. Therefore, the intelligent optimization method and system for the loading and unloading arm mechanism based on reinforcement learning provided by the embodiments of the present invention can improve the operation efficiency and safety of the loading and unloading arm.

[0169] Embodiment 2:

[0170] As Figure 2 shown, it is a functional module diagram of an intelligent optimization system for a loading and unloading arm mechanism based on reinforcement learning according to the present invention.

[0171] The intelligent optimization system 200 for a loading and unloading arm mechanism based on reinforcement learning according to the present invention can be installed in an electronic device. According to the functions achieved, the intelligent optimization system based on reinforcement learning can include a data extraction module 201, a stability analysis module 202, a path planning module 203, a risk analysis module 204, and an intelligent optimization module 205. The modules in the present invention can also be referred to as units, which refer to a series of computer program segments that can be executed by a processor of an electronic device and can complete fixed functions, and are stored in the memory of the electronic device.

[0172] In the embodiments of the present invention, the functions of each module / unit are as follows:

[0173] The data extraction module 201 is configured to obtain the application scenarios of the loading and unloading arm, extract the environmental data of the application scenarios, extract the physical parameters of the loading and unloading arm, analyze the working process of the loading and unloading arm, and extract the operation data of the working process;

[0174] The stability analysis module 202 is configured to analyze the joint torque and joint acceleration of the loading and unloading arm by using the trained dynamics analysis model according to the environmental data, the physical parameters, and the operation data, calculate the operation performance index of the loading and unloading arm according to the joint torque and the joint acceleration, and analyze the working stability of the loading and unloading arm according to the operation performance index;

[0175] The path planning module 203 is configured to analyze the action sequence and operation path of the loading and unloading arm, analyze the torque limit and motion constraints of the loading and unloading arm according to the action sequence and the operation path, and determine the optimal operation path of the loading and unloading arm by using a preset path planning algorithm according to the torque limit and the motion constraints;

[0176] The risk analysis module 204 is configured to construct a sensor network of the loading and unloading arm, collect the working data of the loading and unloading arm in real time according to the sensor network, analyze the risk factors in the working data, and calculate the risk probability of the loading and unloading arm by using a preset risk analysis model according to the risk factors;

[0177] The intelligent optimization module 205 is configured to construct a reinforcement learning optimization model of the loading and unloading arm according to the working stability, the optimal operation path, and the risk probability, analyze the optimization parameters of the loading and unloading arm according to the reinforcement learning optimization model, and perform intelligent optimization of the loading and unloading arm based on the optimization parameters.

[0178] Specifically, each module in the intelligent optimization system 200 of the loading and unloading arm mechanism based on reinforcement learning in the embodiment of the present invention adopts the same technical means as those in the Figure 1 intelligent optimization method of the loading and unloading arm mechanism based on reinforcement learning described above, and can produce the same technical effects, which will not be elaborated here.

[0179] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention.

[0180] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. An intelligent optimization method for loading and unloading arm mechanism based on reinforcement learning, characterized in that: The method comprises: Acquire an application scenario of the loading and unloading arm, extract environmental data of the application scenario, extract physical parameters of the loading and unloading arm, analyze the workflow of the loading and unloading arm, and extract operation data of the workflow; According to the environmental data, the physical parameters and the operation data, the joint torque and joint acceleration of the loading and unloading arm are analyzed using a trained dynamic analysis model, and according to the joint torque and the joint acceleration, an operation performance index of the loading and unloading arm is calculated, and according to the operation performance index, the working stability of the loading and unloading arm is analyzed; Analyze the action sequence and operation path of the loading and unloading arm, analyze the torque limit and motion constraint of the loading and unloading arm according to the action sequence and the operation path, and determine the optimal operation path of the loading and unloading arm by using a preset path planning algorithm according to the torque limit and the motion constraint; Constructing a sensor network of the loading and unloading arm, collecting working data of the loading and unloading arm in real time according to the sensor network, analyzing risk factors in the working data, and calculating the risk probability of the loading and unloading arm according to the risk factors using a preset risk analysis model; According to the working stability, the optimal working path and the risk probability, a reinforcement learning optimization model of the loading and unloading arm is constructed. According to the reinforcement learning optimization model, the optimization parameters of the loading and unloading arm are analyzed. Based on the optimization parameters, intelligent optimization of the loading and unloading arm is performed.

2. The intelligent optimization method for loading and unloading arm mechanism based on reinforcement learning according to claim 1, characterized in that: The extracting the physical parameters of the loading and unloading arm comprises: Analyzing the geometric shape of the loading and unloading arm, and measuring the dimensional parameters of the loading and unloading arm according to the geometric shape; identifying the structural material of the loading and unloading arm and analyzing the material properties of the structural material; constructing a loading arm geometric model of the loading arm according to the size parameters and the material properties; Performing simulation test on the loading and unloading arm geometric model to obtain test data; Calculating the bending stress of the loading and unloading arm according to the test data; A physical parameter of the loading and unloading arm is determined based on the bending stress, the dimensional parameter and the material property.

3. The intelligent optimization method for loading and unloading arm mechanism based on reinforcement learning according to claim 1, characterized in that: The analyzing the joint torque and joint acceleration of the loading and unloading arm using a trained dynamics analysis model according to the environmental data, the physical parameters and the operation data includes: Extracting a kinetic equation of the kinetic analysis model; analyzing the external force of the loading and unloading arm according to the environmental data; Calculating the inertial force effect of the loading and unloading arm according to the physical parameters and the operation data; The joint torque and joint acceleration of the loading and unloading arm are calculated according to the inertial force effect and the external force using the dynamic equation.

4. The intelligent optimization method for loading and unloading arm mechanism based on reinforcement learning according to claim 1, characterized in that: Calculating the operating performance index of the loading and unloading arm according to the joint torque and the joint acceleration includes: Calculating the joint power of the loading and unloading arm according to the joint torque and the joint acceleration; Analyzing the operating energy consumption of the loading and unloading arm according to the joint power; Calculating the torque fluctuation of the loading and unloading arm according to the joint torque; An operating performance index of the loading and unloading arm is determined according to the joint power, the operation energy consumption and the torque fluctuation.

5. The intelligent optimization method for loading and unloading arm mechanism based on reinforcement learning according to claim 1, characterized in that: The analyzing the action sequence and operation path of the loading and unloading arm includes: Determining the operation objectives and constraints of the loading and unloading arm; Constructing a motion analysis model of the loading and unloading arm according to the operation objectives and the constraints; Analyzing the action sequence and action logic relationship of the loading and unloading arm according to the motion analysis model; Determining the action sequence of the loading and unloading arm based on the action sequence and the action logic relationship; According to the action sequence, the operation path of the loading and unloading arm is analyzed.

6. The intelligent optimization method for loading and unloading arm mechanism based on reinforcement learning according to claim 1, characterized in that: Determining the optimal operation path of the loading and unloading arm by using a preset path planning algorithm according to the torque limit and the motion constraint includes: According to the torque limit and the motion constraint, a simulation test is performed on the operation path corresponding to the loading and unloading arm to obtain simulation test data; Calculating the path feasibility and path length of the operation path according to the simulation test data; When the path feasibility does not meet the preset path feasibility threshold or the path length is higher than the preset path length threshold, optimizing the operation path using the path planning algorithm to obtain an optimized path; Calculating the optimized path feasibility and the optimized path length of the optimized path; When the feasibility of the optimized path meets the path feasibility threshold and the optimized path length is lower than the path length threshold, the optimized path is used as the optimal operating path of the loading and unloading arm.

7. The intelligent optimization method for loading and unloading arm mechanism based on reinforcement learning according to claim 1, characterized in that: The sensor network of the loading and unloading arm is constructed, including: Determining a monitoring target of the loading and unloading arm, and configuring a sensor of the loading and unloading arm according to the monitoring target; Determining a network topology and communication protocol for the loading and unloading arm; Constructing a network architecture of the loading and unloading arm according to the network topology and the communication protocol; Constructing a data acquisition unit of the sensor and configuring network parameters of the network architecture; A sensor network of the loading and unloading arm is integrated according to the sensors, the network architecture, the data acquisition unit and the network parameters.

8. The intelligent optimization method for loading and unloading arm mechanism based on reinforcement learning according to claim 1, characterized in that: The step of calculating the risk probability of the loading and unloading arm according to the risk factor using a preset risk analysis model includes: Calculating the risk coefficient of the risk factor using the risk analysis model according to the working data corresponding to the loading and unloading arm; Based on the risk coefficient, determining an initial risk probability of the loading and unloading arm; The risk probability of the loading and unloading arm is calculated according to the initial risk probability, the risk coefficient and the risk factor.

9. The intelligent optimization method for loading and unloading arm mechanism based on reinforcement learning according to claim 1, characterized in that: The step of constructing a reinforcement learning optimization model of the loading and unloading arm according to the working stability, the optimal operation path and the risk probability includes: According to the working stability, the optimal working path and the risk probability, defining the state space, action space and reward function of the loading and unloading arm; Determining a reinforcement learning algorithm for the loading and unloading arm according to the state space, the action space, and the reward function; constructing an initial optimization model of the loading and unloading arm; Based on the reinforcement learning algorithm, the initial optimization model is trained using preset training data to obtain a training model; Identifying key parameters of the training model, and verifying model performance of the training model based on the key parameters; When the model performance meets a preset model performance threshold, the training model is used as a reinforcement learning optimization model for the loading and unloading arm.

10. An intelligent optimization system for loading and unloading arm mechanism based on reinforcement learning, characterized in that: The system comprises: A data extraction module, used to obtain application scenarios of the loading and unloading arm, extract environmental data of the application scenarios, extract physical parameters of the loading and unloading arm, analyze the workflow of the loading and unloading arm, and extract operation data of the workflow; a stability analysis module, for analyzing the joint torque and joint acceleration of the loading and unloading arm using a trained dynamic analysis model according to the environmental data, the physical parameters and the operation data, calculating an operating performance index of the loading and unloading arm according to the joint torque and the joint acceleration, and analyzing the working stability of the loading and unloading arm according to the operating performance index; A path planning module is used to analyze the action sequence and the working path of the loading and unloading arm, analyze the torque limit and motion constraint of the loading and unloading arm according to the action sequence and the working path, and determine the optimal working path of the loading and unloading arm according to the torque limit and the motion constraint by using a preset path planning algorithm; A risk analysis module is used to construct a sensor network of the loading and unloading arm, collect working data of the loading and unloading arm in real time according to the sensor network, analyze risk factors in the working data, and calculate the risk probability of the loading and unloading arm according to the risk factors using a preset risk analysis model; An intelligent optimization module is used to construct a reinforcement learning optimization model of the loading and unloading arm according to the working stability, the optimal working path and the risk probability, analyze the optimization parameters of the loading and unloading arm according to the reinforcement learning optimization model, and perform intelligent optimization of the loading and unloading arm based on the optimization parameters.

Images