An intelligent optimization method and system for a loading and unloading arm mechanism based on reinforcement learning

Abstract

The application relates to the field of mechanical engineering and discloses an intelligent optimization method and system for a loading and unloading arm mechanism based on reinforcement learning, which comprises the following steps: extracting environment data of an application scene, extracting physical parameters of the loading and unloading arm, analyzing a work flow of the loading and unloading arm, and extracting operation data of the work flow; analyzing joint torque and joint acceleration of the loading and unloading arm, calculating an operation performance index of the loading and unloading arm, and analyzing work stability of the loading and unloading arm; analyzing a motion sequence and an operation path of the loading and unloading arm, analyzing torque limitation and motion constraint of the loading and unloading arm, and determining an optimal operation path of the loading and unloading arm; constructing a sensor network of the loading and unloading arm, collecting work data of the loading and unloading arm in real time, analyzing risk factors in the work data, and calculating a risk probability of the loading and unloading arm; constructing a reinforcement learning optimization model of the loading and unloading arm, analyzing optimization parameters of the loading and unloading arm, and executing intelligent optimization of the loading and unloading arm. The application can improve operation efficiency and safety of the loading and unloading arm.

Description

TECHNICAL FIELD

[0001] The application relates to an intelligent optimization method and system for a loading and unloading arm mechanism based on reinforcement learning, and belongs to the field of mechanical engineering. BACKGROUND

[0002] The loading and unloading arm refers to a mechanical device for loading and unloading goods, which is usually installed on a fixed base or a moving vehicle and used for transferring goods from one place to another. The goal of intelligent optimization of the loading and unloading arm mechanism is to achieve higher work efficiency, better operation performance, higher safety and lower energy consumption through intelligent modification of the loading and unloading arm, so as to improve the overall performance and economic benefits of the loading and unloading arm.

[0003] At present, due to the complexity of the design and programming of the loading and unloading arm, the loading and unloading arm needs to be specially customized and adjusted in the face of some complex tasks, and is affected by external factors, so that the flexibility of the loading and unloading arm is poor and the precision is greatly affected, thereby affecting the work efficiency and safety of the loading and unloading arm.

[0004] Therefore, a solution is urgently needed to improve the work efficiency and safety of the loading and unloading arm. SUMMARY

[0005] The application provides an intelligent optimization method and system for a loading and unloading arm mechanism based on reinforcement learning, which mainly aims to improve the work efficiency and safety of the loading and unloading arm.

[0006] To achieve the above-mentioned purpose, the application provides an intelligent optimization method for a loading and unloading arm mechanism based on reinforcement learning, which comprises the following steps:

[0007] Obtaining the application scene of the loading and unloading arm, extracting the environmental data of the application scene, extracting the physical parameters of the loading and unloading arm, analyzing the work flow of the loading and unloading arm, and extracting the work data of the work flow;

[0008] According to the environmental data, the physical parameters and the work data, the joint torque and the joint acceleration of the loading and unloading arm are analyzed by using a trained dynamic analysis model, the operation performance index of the loading and unloading arm is calculated according to the joint torque and the joint acceleration, and the work stability of the loading and unloading arm is analyzed according to the operation performance index;

[0009] Analyzing the action sequence and the work path of the loading and unloading arm, analyzing the torque limit and the motion constraint of the loading and unloading arm according to the action sequence and the work path, and determining the optimal work path of the loading and unloading arm by using a preset path planning algorithm according to the torque limit and the motion constraint;

[0010] A sensor network of the handling arm is constructed, working data of the handling arm is collected in real time according to the sensor network, a risk factor in the working data is analyzed, and a risk probability of the handling arm is calculated according to the risk factor by using a preset risk analysis model;

[0011] According to the working stability, the optimal operation path and the risk probability, a reinforcement learning optimization model of the handling arm is constructed, an optimization parameter of the handling arm is analyzed according to the reinforcement learning optimization model, and intelligent optimization of the handling arm is performed based on the optimization parameter.

[0012] Optionally, the extracting the physical parameter of the handling arm comprises:

[0013] The geometric shape of the handling arm is analyzed, and a size parameter of the handling arm is measured according to the geometric shape;

[0014] The structural material of the handling arm is identified, and a material attribute of the structural material is analyzed;

[0015] A handling arm geometric model of the handling arm is constructed according to the size parameter and the material attribute;

[0016] The handling arm geometric model is simulated and tested to obtain test data;

[0017] The bending stress of the handling arm is calculated according to the test data;

[0018] The physical parameter of the handling arm is determined according to the bending stress, the size parameter and the material attribute.

[0019] Optionally, the analyzing the joint torque and the joint acceleration of the handling arm according to the environmental data, the physical parameter and the operation data by using the trained dynamic analysis model comprises:

[0020] A dynamic equation of the dynamic analysis model is extracted;

[0021] The external force of the handling arm is analyzed according to the environmental data;

[0022] The inertial force effect of the handling arm is calculated according to the physical parameter and the operation data;

[0023] The joint torque and the joint acceleration of the handling arm are calculated by using the dynamic equation according to the inertial force effect and the external force.

[0024] Optionally, the calculating the operation performance index of the handling arm according to the joint torque and the joint acceleration comprises:

[0025] According to the joint torque and the joint acceleration, a joint power of the handling arm is calculated;

[0026] According to the joint power, a work energy consumption of the handling arm is analyzed;

[0027] According to the joint torque, a torque fluctuation of the handling arm is calculated;

[0028] According to the joint power, the work energy consumption and the torque fluctuation, an operation performance index of the handling arm is determined.

[0029] Optionally, the analyzing the action sequence and the work path of the handling arm comprises:

[0030] A work target and a constraint condition of the handling arm are determined;

[0031] According to the work target and the constraint condition, a motion analysis model of the handling arm is constructed;

[0032] According to the motion analysis model, an action order and an action logical relationship of the handling arm are analyzed;

[0033] Based on the action order and the action logical relationship, an action sequence of the handling arm is determined;

[0034] According to the action sequence, a work path of the handling arm is analyzed.

[0035] Optionally, the determining the optimal work path of the handling arm according to the torque limit and the motion constraint by using a preset path planning algorithm comprises:

[0036] According to the torque limit and the motion constraint, a simulation test is performed on a work path corresponding to the handling arm, to obtain simulation test data;

[0037] According to the simulation test data, a path feasibility and a path length of the work path are calculated;

[0038] When the path feasibility does not meet a preset path feasibility threshold or the path length is higher than a preset path length threshold, the work path is optimized by using the path planning algorithm, to obtain an optimized path;

[0039] An optimized path feasibility and an optimized path length of the optimized path are calculated;

[0040] When the optimized path feasibility meets the path feasibility threshold and the optimized path length is lower than the path length threshold, the optimized path is taken as the optimal work path of the handling arm.

[0041] Optionally, the constructing the sensor network of the handling arm comprises:

[0042] determining a monitoring target of the handling arm, and configuring a sensor of the handling arm according to the monitoring target;

[0043] determining a network topology and a communication protocol of the handling arm;

[0044] constructing a network architecture of the handling arm according to the network topology and the communication protocol;

[0045] constructing a data acquisition unit of the sensor, and configuring a network parameter of the network architecture;

[0046] integrating a sensor network of the handling arm according to the sensor, the network architecture, the data acquisition unit, and the network parameter.

[0047] Optionally, the calculating the risk probability of the handling arm according to the risk factor by using a preset risk analysis model comprises:

[0048] calculating a risk coefficient of the risk factor by using the risk analysis model according to working data corresponding to the handling arm;

[0049] determining an initial risk probability of the handling arm based on the risk coefficient;

[0050] calculating the risk probability of the handling arm according to the initial risk probability, the risk coefficient, and the risk factor.

[0051] Optionally, the constructing the reinforcement learning optimization model of the handling arm according to the working stability, the optimal operation path, and the risk probability comprises:

[0052] defining a state space, an action space, and a reward function of the handling arm according to the working stability, the optimal operation path, and the risk probability;

[0053] determining a reinforcement learning algorithm of the handling arm according to the state space, the action space, and the reward function;

[0054] constructing an initial optimization model of the handling arm;

[0055] training the initial optimization model by using preset training data based on the reinforcement learning algorithm to obtain a trained model;

[0056] identifying a key parameter of the trained model, and verifying a model performance of the trained model according to the key parameter;

[0057] when the model performance meets a preset model performance threshold, taking the trained model as the reinforcement learning optimization model of the handling arm.

[0058] To solve the above problems, the application further provides a loading and unloading arm mechanism intelligent optimization system based on reinforcement learning, which comprises:

[0059] A data extraction module is configured to obtain an application scenario of the loading and unloading arm, extract environmental data of the application scenario, extract physical parameters of the loading and unloading arm, analyze a work flow of the loading and unloading arm, and extract job data of the work flow.

[0060] A stability analysis module is configured to analyze joint torque and joint acceleration of the loading and unloading arm according to the environmental data, the physical parameters and the job data by using a trained dynamic analysis model, calculate an operation performance index of the loading and unloading arm according to the joint torque and the joint acceleration, and analyze work stability of the loading and unloading arm according to the operation performance index.

[0061] A path planning module is configured to analyze a motion sequence and a job path of the loading and unloading arm, analyze torque limitation and motion constraint of the loading and unloading arm according to the motion sequence and the job path, and determine an optimal job path of the loading and unloading arm by using a preset path planning algorithm according to the torque limitation and the motion constraint.

[0062] A risk analysis module is configured to construct a sensor network of the loading and unloading arm, collect work data of the loading and unloading arm in real time according to the sensor network, analyze risk factors in the work data, and calculate a risk probability of the loading and unloading arm according to the risk factors by using a preset risk analysis model.

[0063] An intelligent optimization module is configured to construct a reinforcement learning optimization model of the loading and unloading arm according to the work stability, the optimal job path and the risk probability, analyze optimization parameters of the loading and unloading arm according to the reinforcement learning optimization model, and perform intelligent optimization of the loading and unloading arm based on the optimization parameters.

[0064] The embodiment of the application can ensure the stability and safety of the loading and unloading arm in complex or harsh environments by acquiring the application scene of the loading and unloading arm, reduce the risk of accidents, and thus improve the efficiency of loading and unloading operation; optionally, the embodiment of the application can identify the factors that may cause mechanism vibration by analyzing the joint torque and joint acceleration of the loading and unloading arm using the trained dynamic analysis model according to the environmental data, the physical parameters and the operation data, and thus optimize the structure, reduce vibration and improve the stability of the loading and unloading arm; the embodiment of the application can improve the performance of the loading and unloading arm, reduce operating costs and enhance the reliability of operation by analyzing the torque limit and motion constraint of the loading and unloading arm according to the action sequence and the operation path under the premise of ensuring safety; the embodiment of the application can monitor the key components of the loading and unloading arm in real time by collecting the working data of the loading and unloading arm in real time according to the sensor network, and once abnormal data is detected, measures can be taken immediately to prevent accidents and improve operation safety; the embodiment of the application can identify high-risk areas and optimize the design of the loading and unloading arm to improve its safety and reliability by calculating the risk probability of the loading and unloading arm using the preset risk analysis model according to the risk factor; finally, the embodiment of the application can significantly improve the automation and intelligent level of loading and unloading operation, reduce human errors, improve operation efficiency and safety, and reduce maintenance costs and operating risks by constructing the reinforcement learning optimization model of the loading and unloading arm according to the working stability, the optimal operation path and the risk probability. Therefore, the reinforcement learning-based intelligent optimization method and system for the loading and unloading arm mechanism provided by the embodiment of the application can improve the operation efficiency and safety of the loading and unloading arm. BRIEF DESCRIPTION OF DRAWINGS

[0065] Figure 1 A flowchart of the reinforcement learning-based intelligent optimization method for the loading and unloading arm mechanism provided by an embodiment of the application is shown.

[0066] Figure 2 A module diagram for implementing the reinforcement learning-based intelligent optimization method for the loading and unloading arm mechanism provided by an embodiment of the application is shown.

[0067] The object implementation, functional features and advantages of the application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0068] It should be understood that the specific embodiments described herein are only used to explain the application and not to limit the application.

[0069] The embodiment of the present application provides a reinforcement learning-based loading and unloading arm mechanism intelligent optimization method. The execution subject of the reinforcement learning-based loading and unloading arm mechanism intelligent optimization method includes but is not limited to at least one of electronic devices such as a server, a terminal and the like which can be configured to execute the method provided by the embodiment of the present application. In other words, the reinforcement learning-based loading and unloading arm mechanism intelligent optimization method can be executed by software or hardware installed in a terminal device or a server device. The server includes but is not limited to a single server, a server cluster, a cloud server or a cloud server cluster and the like.

[0070] Embodiment 1

[0071] Referring to Figure 1 FIG. 1 shows a flowchart of the reinforcement learning-based loading and unloading arm mechanism intelligent optimization method provided by an embodiment of the present application. In the embodiment, the reinforcement learning-based loading and unloading arm mechanism intelligent optimization method includes:

[0072] S1, obtaining an application scenario of a loading and unloading arm, extracting environmental data of the application scenario, extracting physical parameters of the loading and unloading arm, analyzing a work flow of the loading and unloading arm, and extracting job data of the work flow.

[0073] The embodiment of the present application can ensure the stability and safety of the loading and unloading arm in complex or harsh environments by obtaining the application scenario of the loading and unloading arm, thereby reducing the risk of accidents and improving the efficiency of loading and unloading operations. The application scenario refers to the environment and conditions in which the loading and unloading arm is used, which usually has specific requirements such as temperature, humidity, wind speed, etc.

[0074] The embodiment of the present application can optimize the operation parameters of the loading and unloading arm by extracting the environmental data of the application scenario, thereby improving the operation efficiency and job speed. The environmental data refers to the data of various natural and industrial conditions related to the application scenario of the loading and unloading arm.

[0075] Optionally, as an embodiment of the present application, the extraction of the environmental data of the application scenario can be extracted by sensor technology.

[0076] The embodiment of the present application can optimize the structural design of the loading and unloading arm by extracting the physical parameters of the loading and unloading arm, thereby improving its carrying capacity, stretching speed and operation precision, and thus improving the overall performance. The physical parameters refer to a series of measurement indexes involved in the design, manufacture and operation of the loading and unloading arm, such as size parameters, material properties, etc.

[0077] As an embodiment of the present application, the extraction of the physical parameters of the loading and unloading arm includes:

[0078] analyzing the geometric shape of the loading and unloading arm, and measuring the size parameters of the loading and unloading arm according to the geometric shape.

[0079] identifying a structural material of the handling arm, analyzing material properties of the structural material;

[0080] constructing a handling arm geometric model of the handling arm according to the size parameters and the material properties;

[0081] performing simulation testing on the handling arm geometric model to obtain testing data;

[0082] calculating a bending stress of the handling arm according to the testing data by using the following formula:

[0083]

[0084] wherein, the bending stress is represented by σ, the farthest fiber distance corresponding to the testing data is represented by L, the acting load corresponding to the testing data is represented by F, the acting distance from a load point to a fixed point corresponding to the testing data is represented by L, the cross-sectional diameter corresponding to the size data of the handling arm is represented by D;

[0085] determining physical parameters of the handling arm according to the bending stress, the size parameters and the material properties.

[0086] The geometric shape refers to the overall structure and appearance of the handling arm, such as the cross-sectional shape, the number of arm segments, etc. The size parameters refer to specific measurement indicators involved in the design and manufacture of the handling arm, such as the cross-sectional diameter, the length of the segment, etc. The structural material refers to various materials used to manufacture the handling arm. The material properties refer to the physical, chemical and mechanical properties of the material itself. The handling arm geometric model refers to a virtual, digitized representation of the handling arm, which contains the actual size, shape, structure and other related geometric features of the handling arm. The testing data refers to a series of data collected during the simulation testing of the handling arm geometric model. The bending stress refers to the internal stress generated on the cross section of the handling arm or other structural elements when subjected to bending moment.

[0087] Optionally, the construction of the handling arm geometric model of the handling arm according to the size parameters and the material properties can be constructed by the method of parameterized modeling.

[0088] The embodiment of the application can identify the bottlenecks and delay reasons in the handling process by analyzing the working process of the handling arm, thereby optimizing the action sequence and speed of the handling arm and improving the overall work efficiency. The working process refers to a series of steps and operation sequences experienced by the handling arm when completing the handling task.

[0089] Optionally, as an embodiment of the present application, the analysis of the work flow of the loading and unloading arm can be monitored and analyzed through Internet of Things technology.

[0090] The embodiment of the present application can identify the delay and waiting time in the work process by extracting the work data of the work flow, thereby optimizing the work sequence and operation process, reducing unnecessary pauses, and improving the loading and unloading efficiency. The work data refers to various information and records related to the operation of the loading and unloading arm.

[0091] Optionally, as an embodiment of the present application, the extraction of the work data of the work flow can be extracted through automatic data acquisition technology.

[0092] S2, according to the environmental data, the physical parameters and the work data, the trained dynamic analysis model is used to analyze the joint torque and joint acceleration of the loading and unloading arm, according to the joint torque and the joint acceleration, the operation performance index of the loading and unloading arm is calculated, and according to the operation performance index, the working stability of the loading and unloading arm is analyzed.

[0093] The embodiment of the present application can identify the factors that may cause mechanism vibration by analyzing the joint torque and joint acceleration of the loading and unloading arm according to the environmental data, the physical parameters and the work data, using the trained dynamic analysis model, thereby optimizing the structure, reducing the vibration and improving the stability of the loading and unloading arm. The trained dynamic analysis model refers to a mathematical model that has been trained with a large amount of data and can simulate and predict the dynamic behavior of mechanical systems. The joint torque refers to the product of the force acting on the joint of the loading and unloading arm and the force arm. The joint acceleration refers to the angular acceleration of the joint of the loading and unloading arm in its movement process.

[0094] As an embodiment of the present application, the analysis of the joint torque and joint acceleration of the loading and unloading arm according to the environmental data, the physical parameters and the work data, using the trained dynamic analysis model, includes:

[0095] extracting the dynamic equation of the dynamic analysis model;

[0096] According to the environmental data, the external force of the loading and unloading arm is analyzed;

[0097] According to the physical parameters and the work data, the inertial force effect of the loading and unloading arm is calculated by the following formula:

[0098]

[0099] wherein, represents the inertial force effect, represents the physical parameter corresponding to the first a link mass of the link, a position vector representing a physical parameter corresponding to a distance from a joint axis to a centroid of the link, a position vector representing a physical parameter corresponding to a distance from a joint axis to a centroid of the link, a joint angular velocity vector representing a joint angular velocity corresponding to the joint angle, a joint angular velocity vector representing a joint angular velocity corresponding to the joint angle,

[0100] a joint torque and a joint acceleration of the handling arm are calculated by using the dynamic equation according to the inertial force effect and the external force.

[0101] The dynamic equation refers to an equation describing a motion state of a mechanical system. The external force refers to various forces acting on the handling arm caused by external environments or operating conditions during handling arm operation. The inertial force effect refers to a force caused by a change in the motion state of the handling arm (such as acceleration, deceleration, or change in motion direction). The position vector refers to a set of coordinate values from a joint axis to a centroid of a link. The joint angular velocity vector refers to a set of vectors describing the rotational speed of each joint of the handling arm.

[0102] Optionally, the analyzing the external force of the handling arm according to the environment data can be analyzed by computational fluid dynamics simulation.

[0103] According to the joint torque and the joint acceleration, the operation performance index of the handling arm is calculated, which can optimize the accurate control of the joint torque and the acceleration, thereby improving the positioning accuracy and the accuracy of repeated operation of the handling arm. The operation performance index refers to a series of quantitative indexes for evaluating the performance of the mechanical arm or the handling arm.

[0104] As an embodiment of the present application, the operation performance index of the handling arm is calculated according to the joint torque and the joint acceleration, which includes:

[0105] The joint power of the handling arm is calculated according to the joint torque and the joint acceleration.

[0106] The operation energy consumption of the handling arm is analyzed according to the joint power.

[0107] The torque fluctuation of the handling arm is calculated according to the joint torque by using the following formula:

[0108]

[0109] wherein, the torque fluctuation is represented by T, the total number of joints of the handling arm is represented by N, the joint torque of the i th joint of the handling arm is represented by T i, ​

[0110] According to the joint power, the work energy consumption, and the torque fluctuation, an operation performance index of the loading and unloading arm is determined.

[0111] The joint power refers to the power generated by each joint of the mechanical arm during movement. The work energy consumption refers to the total energy consumed during the completion of a specific work task. The torque fluctuation refers to the degree of change of the joint torque of the loading and unloading arm over time during operation.

[0112] The embodiment of the present application can adapt to a wider range of working conditions and environments by analyzing the working stability of the loading and unloading arm according to the operation performance index, thereby improving its adaptability in different application scenarios. The working stability refers to the ability of the loading and unloading arm to maintain stable operation when performing loading and unloading tasks.

[0113] Optionally, as an embodiment of the present application, the analysis of the working stability of the loading and unloading arm according to the operation performance index can be analyzed by a finite element analysis method.

[0114] S3, analyze the action sequence and work path of the loading and unloading arm, analyze the torque limit and motion constraint of the loading and unloading arm according to the action sequence and the work path, and determine the optimal work path of the loading and unloading arm by using a preset path planning algorithm according to the torque limit and the motion constraint.

[0115] The embodiment of the present application can improve the degree of freedom and adaptability of the joint by analyzing the action sequence and work path of the loading and unloading arm, so that the loading and unloading arm can complete more diversified work tasks. The action sequence refers to a series of ordered actions required to be performed by the loading and unloading arm during the completion of a complete loading and unloading work. The work path refers to the entire motion trajectory of the end effector of the loading and unloading arm from the initial position to the target position during the execution of the loading and unloading work, and returns to the initial position after completing the loading and unloading task.

[0116] As an embodiment of the present application, the analysis of the action sequence and work path of the loading and unloading arm comprises:

[0117] determining the work target and constraint condition of the loading and unloading arm;

[0118] constructing a motion analysis model of the loading and unloading arm according to the work target and the constraint condition;

[0119] analyzing the action sequence and action logic relationship of the loading and unloading arm according to the motion analysis model;

[0120] determining the action sequence of the loading and unloading arm based on the action sequence and the action logic relationship;

[0121] According to the action sequence, a work path of the loading and unloading arm is analyzed.

[0122] The work target refers to specific targets and requirements that the loading and unloading arm needs to achieve when performing a loading and unloading task. The constraint condition refers to limiting factors that must be considered in the design, planning and execution of the loading and unloading arm work process. The motion analysis model refers to a mathematical model used to analyze and predict the motion behavior of the loading and unloading arm when performing a specific task. The action sequence refers to the arrangement of each action in a certain logic and time sequence when the loading and unloading arm completes a specific loading and unloading task. The action logic relationship refers to the dependency and sequence relationship between each action when the loading and unloading arm performs a series of actions.

[0123] Optionally, the motion analysis model of the loading and unloading arm is constructed according to the work target and the constraint condition, which can be constructed by multi-body dynamics.

[0124] The torque limit of the loading and unloading arm and the motion constraint of the loading and unloading arm are analyzed according to the action sequence and the work path in the embodiment of the application, which can improve the performance of the loading and unloading arm, reduce operating costs and enhance the reliability of operation under the premise of ensuring safety. The torque limit refers to the maximum torque value that each joint or driving device of the loading and unloading arm can withstand during its motion process. The motion constraint refers to the physical rules and conditions that limit the motion of the loading and unloading arm in actual operation.

[0125] Optionally, as one embodiment of the application, the torque limit of the loading and unloading arm and the motion constraint of the loading and unloading arm are analyzed according to the action sequence and the work path, which can be analyzed by a machine learning method.

[0126] The optimal work path of the loading and unloading arm is determined by using a preset path planning algorithm according to the torque limit and the motion constraint in the embodiment of the application, which can reduce unnecessary motion, shorten the work cycle and increase the number of work per unit time. The preset path planning algorithm refers to a series of calculation steps or rules designed to solve a specific problem, which is used to automatically generate the optimal or optimized path of a robotic arm or other similar mechanical device. The optimal work path refers to the optimal path of the loading and unloading arm from the starting position to the target position and completing the specified task when performing the loading and unloading work under certain conditions.

[0127] As one embodiment of the application, the optimal work path of the loading and unloading arm is determined according to the torque limit and the motion constraint by using a preset path planning algorithm, which includes:

[0128] According to the torque limit and the motion constraint, the simulation test data of the work path corresponding to the loading and unloading arm is obtained by simulating and testing the work path.

[0129] According to the simulation test data, the path feasibility and path length of the operation path are calculated;

[0130] When the path feasibility does not meet the preset path feasibility threshold or the path length is higher than the preset path length threshold, the operation path is optimized by using the path planning algorithm to obtain an optimized path;

[0131] The optimized path feasibility and optimized path length of the optimized path are calculated;

[0132] When the optimized path feasibility meets the path feasibility threshold and the optimized path length is lower than the path length threshold, the optimized path is taken as the optimal operation path of the loading and unloading arm.

[0133] The simulation test data refer to a series of data collected in the simulation process of the operation path of the loading and unloading arm. The path feasibility refers to the actual executability of the planned path of the loading and unloading arm when performing a specific operation task. The path length refers to the actual distance traveled by the loading and unloading arm from the starting position to the target position in path planning. The preset path feasibility threshold refers to a set of critical values or standards set to ensure that the loading and unloading arm can safely and effectively complete the operation task when performing path planning. The preset path length threshold refers to a maximum allowed path length value set to evaluate path efficiency when performing path planning of the loading and unloading arm. The optimized path refers to a new operation path obtained by adjusting and improving the path planning algorithm. The optimized path feasibility refers to whether the optimized operation path can be successfully executed by the loading and unloading arm in actual operation while meeting all preset constraint conditions and performance standards. The optimized path length refers to the total length of the path to be passed through from the starting position to the target position after optimizing the operation path of the loading and unloading arm by the path planning algorithm.

[0134] S4, a sensor network of the loading and unloading arm is constructed, working data of the loading and unloading arm are collected in real time according to the sensor network, risk factors in the working data are analyzed, and a risk probability of the loading and unloading arm is calculated by using a preset risk analysis model according to the risk factors.

[0135] The embodiment of the application can discover potential safety hazards in time by constructing the sensor network of the loading and unloading arm, and take measures to prevent accidents. The sensor network refers to a system composed of multiple sensors arranged at key positions of the loading and unloading arm to monitor and collect various physical parameters and data of the loading and unloading arm during operation.

[0136] As an embodiment of the application, the sensor network of the loading and unloading arm is constructed, including:

[0137] determining a monitoring target of the handling arm, and configuring a sensor of the handling arm according to the monitoring target;

[0138] determining a network topology and a communication protocol of the handling arm;

[0139] constructing a network architecture of the handling arm according to the network topology and the communication protocol;

[0140] constructing a data acquisition unit of the sensor, and configuring a network parameter of the network architecture;

[0141] integrating a sensor network of the handling arm according to the sensor, the network architecture, the data acquisition unit, and the network parameter.

[0142] The monitoring target refers to key performance indicators and state parameters of the handling arm that need to be monitored during operation. The sensor refers to an electronic device for monitoring and controlling various performance indicators and state parameters of the handling arm. The network topology refers to the connection mode and organizational form between various sensors, data acquisition units, data processing centers, and communication devices in the sensor network. The communication protocol refers to a set of rules and standards that define the data format, transmission sequence, error detection and correction methods, and data exchange mechanisms for communication between devices in the sensor network. The network architecture refers to the overall structure design of the handling arm sensor network. The data acquisition unit refers to a device or system for collecting data from sensors and converting it into a usable digital format for further processing. The network parameter refers to a key parameter that needs to be configured and optimized during the design and operation of the sensor network.

[0143] Optionally, the determination of the network topology and the communication protocol of the handling arm can be determined by the method of integrating the Internet of Things platform.

[0144] The embodiment of the application can monitor the key components of the handling arm in real time by collecting the working data of the handling arm in real time according to the sensor network. Once abnormal data is detected, measures can be taken immediately to prevent accidents and improve work safety. The working data refers to a series of data related to the operation and performance of the handling arm collected by the sensor network installed on the handling arm.

[0145] The embodiment of the application can establish a risk database by analyzing the risk factors in the working data, analyze historical risk events, and better understand and manage risks. The risk factors refer to various factors that may cause safety risks, performance degradation, or equipment failure during the operation of the handling arm.

[0146] Optionally, as an embodiment of the present application, the analysis of the risk factors in the working data can be analyzed by a single-multiple factor analysis method.

[0147] According to the risk factors, the risk probability of the loading and unloading arm can be calculated by using a preset risk analysis model, so that the high-risk area can be identified, the design of the loading and unloading arm can be optimized, and the safety and reliability thereof can be improved. The preset risk analysis model refers to a set of mathematical models or algorithms designed to evaluate and calculate the probability of potential risks in the operation of the loading and unloading arm. The risk probability refers to the possibility of a specific risk event occurring in the loading and unloading arm under given conditions.

[0148] As an embodiment of the present application, the calculation of the risk probability of the loading and unloading arm according to the risk factors by using a preset risk analysis model comprises:

[0149] According to the working data corresponding to the loading and unloading arm, the risk coefficient of the risk factor is calculated by using the risk analysis model;

[0150] Based on the risk coefficient, the initial risk probability of the loading and unloading arm is determined;

[0151] According to the initial risk probability, the risk coefficient, and the risk factor, the risk probability of the loading and unloading arm is calculated by using the following formula:

[0152]

[0153] wherein, represents the risk probability, represents the initial risk probability, represents the number of factors of the risk factor, represents the i-th risk factor, represents the risk coefficient of the i-th risk factor, represents the exponential function with base e. The risk coefficient refers to a numerical value in the risk analysis model for quantifying the influence degree of each risk factor on the risk probability of the loading and unloading arm. The initial risk probability refers to the inherent risk level of the loading and unloading arm without the influence of any risk factor. Optionally, the determination of the initial risk probability of the loading and unloading arm based on the risk coefficient can be determined by a fault tree analysis.

[0154]

[0155]

[0156] ​​​S5, constructing a reinforcement learning optimization model of the loading and unloading arm according to the working stability, the optimal operation path and the risk probability, analyzing optimization parameters of the loading and unloading arm according to the reinforcement learning optimization model, and performing intelligent optimization of the loading and unloading arm based on the optimization parameters.

[0157] The embodiment of the application can significantly improve the automation and intelligent level of loading and unloading operation, reduce human errors, improve operation efficiency and safety, and reduce maintenance cost and operation risk by constructing the reinforcement learning optimization model of the loading and unloading arm according to the working stability, the optimal operation path and the risk probability. The reinforcement learning optimization model refers to a mathematical model based on reinforcement learning theory, which is used to optimize the operation process of the loading and unloading arm.

[0158] As an embodiment of the application, constructing the reinforcement learning optimization model of the loading and unloading arm according to the working stability, the optimal operation path and the risk probability comprises:

[0159] defining a state space, an action space and a reward function of the loading and unloading arm according to the working stability, the optimal operation path and the risk probability;

[0160] determining a reinforcement learning algorithm of the loading and unloading arm according to the state space, the action space and the reward function;

[0161] constructing an initial optimization model of the loading and unloading arm;

[0162] training the initial optimization model by using preset training data based on the reinforcement learning algorithm to obtain a training model;

[0163] identifying key parameters of the training model, and verifying model performance of the training model according to the key parameters;

[0164] when the model performance meets a preset model performance threshold, taking the training model as the reinforcement learning optimization model of the loading and unloading arm.

[0165] The state space refers to a set of all different environmental states that the handling arm may encounter when performing tasks. The action space refers to a set of all possible actions performed by the handling arm. The reward function refers to a function for analyzing the good or bad of the actions taken by the handling arm. The reinforcement learning algorithm refers to a computational method for solving reinforcement learning problems, which enables the agent to learn how to maximize the cumulative reward in the environment through exploration and utilization. The initial optimization model refers to a model preliminarily constructed according to the specific requirements and environmental characteristics of the handling arm before applying the reinforcement learning algorithm. The trained model refers to a model whose parameters have been adjusted and gradually converged after a certain number of iterations of training in the reinforcement learning process. The key parameters refer to parameters that have a significant impact on the performance and training results of the reinforcement learning optimization model. The model performance refers to the performance measure of the reinforcement learning optimization model when completing a specific task. The preset model performance threshold is a set of standards or limits set to determine whether the model has reached an acceptable or target performance level during the reinforcement learning training process.

[0166] The embodiments of the present application can more accurately position and operate the handling arm by analyzing the optimization parameters of the handling arm based on the reinforcement learning optimization model, reduce position errors, improve the precision and efficiency of loading and unloading operations, and at the same time make the handling arm maintain stable operation in complex or unstable environments, reduce failures and downtime. The optimization parameters refer to those parameters for improving and optimizing the operation performance of the handling arm.

[0167] The embodiments of the present application can better adapt to different working environments and task requirements by performing intelligent optimization of the handling arm based on the optimization parameters, improving the flexibility and adaptability of the system.

[0168] The embodiment of the application can ensure the stability and safety of the loading and unloading arm in complex or harsh environments by acquiring the application scene of the loading and unloading arm, reduce the risk of accidents, and thus improve the efficiency of loading and unloading operation; optionally, the embodiment of the application can identify factors that may cause mechanism vibration by analyzing the joint torque and joint acceleration of the loading and unloading arm using a trained dynamic analysis model according to the environmental data, the physical parameters and the operation data, so as to optimize the structure, reduce vibration and improve the stability of the loading and unloading arm; the embodiment of the application can improve the performance of the loading and unloading arm, reduce operating costs and enhance the reliability of operation by analyzing the torque limit and motion constraint of the loading and unloading arm according to the action sequence and the operation path on the premise of ensuring safety; the embodiment of the application can monitor the key components of the loading and unloading arm in real time by collecting the working data of the loading and unloading arm in real time according to the sensor network, and once abnormal data is detected, measures can be taken immediately to prevent accidents and improve operation safety; the embodiment of the application can identify high-risk areas and optimize the design of the loading and unloading arm to improve its safety and reliability by calculating the risk probability of the loading and unloading arm using a preset risk analysis model according to the risk factor; finally, the embodiment of the application can significantly improve the automation and intelligent level of loading and unloading operation, reduce human errors, improve operation efficiency and safety, and reduce maintenance costs and operating risks by constructing a reinforcement learning optimization model of the loading and unloading arm according to the working stability, the optimal operation path and the risk probability. Therefore, the intelligent optimization method and system for the loading and unloading arm mechanism based on reinforcement learning provided by the embodiment of the application can improve the operation efficiency and safety of the loading and unloading arm.

[0169] Embodiment 2:

[0170] As Figure 2 shown in FIG. 1, it is a functional module diagram of an intelligent optimization system for a loading and unloading arm mechanism based on reinforcement learning.

[0171] The intelligent optimization system for the loading and unloading arm mechanism based on reinforcement learning can be installed in an electronic device. According to the functions to be implemented, the intelligent optimization system for the loading and unloading arm mechanism based on reinforcement learning can include a data extraction module 201, a stability analysis module 202, a path planning module 203, a risk analysis module 204 and an intelligent optimization module 205. The modules of the application can also be referred to as units, which refer to a series of computer program segments that can be executed by an electronic device processor and can complete a fixed function, and are stored in the memory of the electronic device.

[0172] In the embodiment of the application, the functions of each module / unit are as follows:

[0173] The data extraction module 201 is configured to obtain an application scenario of the loading and unloading arm, extract environmental data of the application scenario, extract physical parameters of the loading and unloading arm, analyze a work flow of the loading and unloading arm, and extract job data of the work flow.

[0174] The stability analysis module 202 is configured to analyze joint torque and joint acceleration of the loading and unloading arm according to the environmental data, the physical parameters and the job data by using a trained dynamic analysis model, calculate an operation performance index of the loading and unloading arm according to the joint torque and the joint acceleration, and analyze work stability of the loading and unloading arm according to the operation performance index.

[0175] The path planning module 203 is configured to analyze a motion sequence and a job path of the loading and unloading arm, analyze torque limitation and motion constraint of the loading and unloading arm according to the motion sequence and the job path, and determine an optimal job path of the loading and unloading arm by using a preset path planning algorithm according to the torque limitation and the motion constraint.

[0176] The risk analysis module 204 is configured to construct a sensor network of the loading and unloading arm, collect work data of the loading and unloading arm in real time according to the sensor network, analyze risk factors in the work data, and calculate a risk probability of the loading and unloading arm according to the risk factors by using a preset risk analysis model.

[0177] The intelligent optimization module 205 is configured to construct a reinforcement learning optimization model of the loading and unloading arm according to the work stability, the optimal job path and the risk probability, analyze optimization parameters of the loading and unloading arm according to the reinforcement learning optimization model, and perform intelligent optimization of the loading and unloading arm based on the optimization parameters.

[0178] In detail, the modules in the reinforcement learning-based intelligent optimization system 200 of the loading and unloading arm mechanism in the embodiments of the present application use the same technical means as the reinforcement learning-based intelligent optimization method of the loading and unloading arm mechanism in the above-mentioned Figure 1 embodiments, and can produce the same technical effects, which will not be described here again.

[0179] It is obvious for those skilled in the art that the present application is not limited to the details of the above-mentioned exemplary embodiments, and the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application.

[0180] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application but not limit the present application, and although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the present application.

Claims

1. A method for intelligent optimization of loading and unloading arm mechanisms based on reinforcement learning, characterized in that, The method includes: The application scenarios of the loading and unloading arm are obtained, the environmental data of the application scenarios are extracted, the physical parameters of the loading and unloading arm are extracted, the workflow of the loading and unloading arm is analyzed, and the operation data of the workflow is extracted. Based on the environmental data, physical parameters, and operational data, a trained dynamic analysis model is used to analyze the joint torques and accelerations of the loading arm. Based on the joint torques and accelerations, the operational performance indicators of the loading arm are calculated. Based on the operational performance indicators, the operational stability of the loading arm is analyzed. The step of using the trained dynamic analysis model to analyze the joint torques and accelerations of the loading arm includes: analyzing the external forces acting on the loading arm using the dynamic analysis model based on the environmental data; and calculating the inertial force effect of the loading arm using the following formula based on the physical parameters and operational data. ； in, Indicates the effect of inertial force. This indicates the number of links in the loading / unloading arm. Indicates the physical parameters corresponding to the first The mass of each link. This indicates that the physical parameters correspond to the joint axis from the 1st joint to the 2nd joint. The position vector of the centroid of each link. This represents the joint angular velocity vector corresponding to the operation data. The rate of change of the joint angular velocity vector with time is represented. Based on the inertial force effect and the external force, the joint torque and joint acceleration of the loading and unloading arm are calculated. The calculation of the operational performance indicators of the loading and unloading arm based on the joint torque and joint acceleration includes: calculating the joint power of the loading and unloading arm based on the joint torque and joint acceleration; analyzing the operational energy consumption of the loading and unloading arm based on the joint power; and calculating the torque fluctuation of the loading and unloading arm using the following formula based on the joint torque: ； in, Indicates torque fluctuation, This indicates the total number of joints corresponding to the loading / unloading arm. Indicates the loading / unloading arm corresponding to the first The joint torque of each joint is used to determine the operational performance indicators of the loading and unloading arm based on the joint power, the operational energy consumption, and the torque fluctuation. The action sequence and operation path of the loading and unloading arm are analyzed. Based on the action sequence and operation path, the torque limit and motion constraint of the loading and unloading arm are analyzed. Based on the torque limit and motion constraint, the optimal operation path of the loading and unloading arm is determined using a preset path planning algorithm. A sensor network for the loading / unloading arm is constructed. Based on this sensor network, real-time operating data of the loading / unloading arm is collected. Risk factors within the operating data are analyzed. Based on these risk factors, a preset risk analysis model is used to calculate the risk probability of the loading / unloading arm. The step of calculating the risk probability of the loading / unloading arm based on the risk factors and the preset risk analysis model includes: calculating the risk coefficient of the risk factor using the risk analysis model based on the operating data corresponding to the loading / unloading arm; determining the initial risk probability of the loading / unloading arm based on the risk coefficient; and calculating the risk probability of the loading / unloading arm using the following formula based on the initial risk probability, the risk coefficient, and the risk factor: ； in, Indicates the probability of risk. Indicates the initial risk probability. The number of factors representing risk factors. Indicates the first One risk factor, Indicates the first The risk coefficient of each risk factor Indicated by The base of the exponential function; Based on the operational stability, the optimal operation path, and the risk probability, a reinforcement learning optimization model for the loading and unloading arm is constructed. Based on the reinforcement learning optimization model, the optimization parameters of the loading and unloading arm are analyzed. Based on the optimization parameters, intelligent optimization of the loading and unloading arm is performed.

2. The intelligent optimization method for loading and unloading arm mechanism based on reinforcement learning as described in claim 1, characterized in that, The extraction of the physical parameters of the loading and unloading arm includes: Analyze the geometry of the loading and unloading arm, and measure the dimensional parameters of the loading and unloading arm based on the geometry; Identify the structural material of the loading and unloading arm and analyze the material properties of the structural material; Based on the dimensional parameters and material properties, construct the loading and unloading arm geometric model; The geometric model of the loading and unloading arm was simulated and tested to obtain test data; Based on the test data, calculate the bending stress of the loading and unloading arm; The physical parameters of the loading and unloading arm are determined based on the bending stress, the dimensional parameters, and the material properties.

3. The intelligent optimization method for loading and unloading arm mechanism based on reinforcement learning as described in claim 1, characterized in that, The analysis of the loading and unloading arm's motion sequence and operation path includes: Determine the operational objectives and constraints of the loading and unloading arm; Based on the operational objectives and constraints, a motion analysis model for the loading and unloading arm is constructed. Based on the motion analysis model, analyze the sequence of actions and the logical relationship of the actions of the loading and unloading arm; Based on the action sequence and the action logic relationship, the action sequence of the loading and unloading arm is determined; Based on the action sequence, analyze the working path of the loading and unloading arm.

4. The intelligent optimization method for loading and unloading arm mechanism based on reinforcement learning as described in claim 1, characterized in that, The step of determining the optimal operating path of the loading and unloading arm using a preset path planning algorithm based on the torque limit and the motion constraint includes: Based on the torque limit and the motion constraint, the operation path corresponding to the loading and unloading arm is simulated and tested to obtain simulation test data. Based on the simulation test data, calculate the path feasibility and path length of the operation path; When the feasibility of the path does not meet the preset path feasibility threshold or the path length is higher than the preset path length threshold, the path planning algorithm is used to optimize the operation path to obtain the optimized path. Calculate the feasibility and length of the optimized path; When the feasibility of the optimized path meets the path feasibility threshold and the length of the optimized path is lower than the path length threshold, the optimized path is taken as the optimal operating path for the loading and unloading arm.

5. The intelligent optimization method for loading and unloading arm mechanism based on reinforcement learning as described in claim 1, characterized in that, The sensor network for constructing the loading / unloading arm includes: Determine the monitoring target of the loading and unloading arm, and configure the sensors of the loading and unloading arm according to the monitoring target; Determine the network topology and communication protocol of the loading / unloading arm; Based on the network topology and the communication protocol, construct the network architecture of the loading and unloading arm; Construct the data acquisition unit of the sensor and configure the network parameters of the network architecture; The sensor network of the loading and unloading arm is integrated based on the sensors, the network architecture, the data acquisition unit, and the network parameters.

6. The intelligent optimization method for loading and unloading arm mechanism based on reinforcement learning as described in claim 1, characterized in that, The step of constructing a reinforcement learning optimization model for the loading and unloading arm based on the operational stability, the optimal operation path, and the risk probability includes: Based on the operational stability, the optimal operation path, and the risk probability, the state space, motion space, and reward function of the loading and unloading arm are defined. The reinforcement learning algorithm for the loading and unloading arm is determined based on the state space, the action space, and the reward function. Construct an initial optimization model for the loading and unloading arm; Based on the reinforcement learning algorithm, the initial optimization model is trained using preset training data to obtain a trained model; Identify the key parameters of the trained model, and verify the model performance based on the key parameters; When the model performance meets the preset model performance threshold, the trained model is used as the reinforcement learning optimization model for the loading and unloading arm.

7. An intelligent optimization system for a loading / unloading arm mechanism based on reinforcement learning, characterized in that, The system includes: The data extraction module is used to obtain the application scenario of the loading and unloading arm, extract the environmental data of the application scenario, extract the physical parameters of the loading and unloading arm, analyze the workflow of the loading and unloading arm, and extract the operation data of the workflow. The stability analysis module is used to analyze the joint torque and joint acceleration of the loading arm using a trained dynamic analysis model based on the environmental data, the physical parameters, and the operational data. Based on the joint torque and joint acceleration, it calculates the operational performance indicators of the loading arm and analyzes the operational stability of the loading arm based on the operational performance indicators. The step of analyzing the joint torque and joint acceleration of the loading arm using the trained dynamic analysis model includes: analyzing the external forces acting on the loading arm using the dynamic analysis model based on the environmental data; and calculating the inertial force effect of the loading arm using the following formula based on the physical parameters and the operational data. ； in, Indicates the effect of inertial force. This indicates the number of links in the loading / unloading arm. Indicates the physical parameters corresponding to the first The mass of each link. This indicates that the physical parameters correspond to the joint axis from the 1st joint to the 2nd joint. The position vector of the centroid of each link. This represents the joint angular velocity vector corresponding to the operation data. The rate of change of the joint angular velocity vector with time is represented. Based on the inertial force effect and the external force, the joint torque and joint acceleration of the loading and unloading arm are calculated. The calculation of the operational performance indicators of the loading and unloading arm based on the joint torque and joint acceleration includes: calculating the joint power of the loading and unloading arm based on the joint torque and joint acceleration; analyzing the operational energy consumption of the loading and unloading arm based on the joint power; and calculating the torque fluctuation of the loading and unloading arm using the following formula based on the joint torque: ； in, Indicates torque fluctuation, This indicates the total number of joints corresponding to the loading / unloading arm. Indicates the loading / unloading arm corresponding to the first The joint torque of each joint is used to determine the operational performance indicators of the loading and unloading arm based on the joint power, the operational energy consumption, and the torque fluctuation. The path planning module is used to analyze the action sequence and operation path of the loading and unloading arm, analyze the torque limit and motion constraint of the loading and unloading arm based on the action sequence and operation path, and determine the optimal operation path of the loading and unloading arm based on the torque limit and motion constraint using a preset path planning algorithm. A risk analysis module is used to construct a sensor network for the loading and unloading arm, collect real-time operating data of the loading and unloading arm based on the sensor network, analyze risk factors in the operating data, and calculate the risk probability of the loading and unloading arm using a preset risk analysis model based on the risk factors. The step of calculating the risk probability of the loading and unloading arm based on the risk factors and using the preset risk analysis model includes: calculating the risk coefficient of the risk factor using the risk analysis model based on the operating data corresponding to the loading and unloading arm; determining the initial risk probability of the loading and unloading arm based on the risk coefficient; and calculating the risk probability of the loading and unloading arm using the following formula based on the initial risk probability, the risk coefficient, and the risk factor: ； in, Indicates the probability of risk. Indicates the initial risk probability. The number of factors representing risk factors. Indicates the first One risk factor, Indicates the first The risk coefficient of each risk factor Indicated by The base of the exponential function; The intelligent optimization module is used to construct a reinforcement learning optimization model for the loading and unloading arm based on the operational stability, the optimal operation path, and the risk probability; analyze the optimization parameters of the loading and unloading arm based on the reinforcement learning optimization model; and perform intelligent optimization of the loading and unloading arm based on the optimization parameters.

Images