Forklift dynamic path planning method based on deep reinforcement learning

Abstract

The invention relates to the technical field of intelligent warehousing and logistics automation, in particular to a forklift dynamic path planning method based on deep reinforcement learning, and the method comprises the steps: deploying multi-view vision, geomagnetism and other multi-source sensors, achieving data calibration and fusion through employing an insect compound eye-imitating vision model, and constructing a high-dimensional state space vector; heuristic models such as migrant bird navigation and biological stress response are adopted, deep reinforcement learning is combined, decision instructions are generated from the three aspects of path planning, dynamic obstacle avoidance and energy efficiency management, actions are executed through a control system, and deviation is fed back; a reward function is used for evaluating the decision effect, rewards are formed by weighting path efficiency, obstacle avoidance success and energy consumption penalty, the weight can be updated in a self-adaptive mode, and therefore the deep reinforcement learning model is optimized. According to the method, the accuracy, safety and efficiency of forklift path planning are effectively improved, the method can adapt to complex dynamic environments, and the requirements of intelligent logistics and industrial automation for forklift intelligent operation are met.

Description

Technical Field

[0001] The present invention relates to the field of intelligent warehousing and logistics automation technology, and specifically to a forklift dynamic path planning method based on deep reinforcement learning. Background Art

[0002] In modern logistics, warehousing, and industrial production, forklifts are core handling equipment. The degree of intelligent path planning directly affects operational efficiency, safety, and operating costs. Currently, traditional forklift path planning and management technologies have many problems that need to be solved:

[0003] Limitations of path planning technology: Existing forklift path planning methods often use static maps combined with fixed rules, such as algorithms like A* and Dijkstra. These algorithms struggle to adapt to the dynamic working environment within warehouses or factories, such as temporary obstacles and personnel flow. When encountering complex scenarios, such as narrow passages and multiple vehicle intersections, traditional algorithms are prone to falling into local optimal solutions, resulting in inefficient path planning and even operational stagnation. Furthermore, in indoor environments, where GPS signals are limited, traditional navigation methods that rely on satellite positioning cannot provide accurate location information, significantly limiting the forklift's autonomous operation capabilities.

[0004] Inadequate environmental perception and data processing: A single sensor (such as a lidar or visual camera) cannot fully perceive the forklift's operating environment, resulting in blind spots and inaccurate data. While multi-sensor fusion technology has been applied, traditional methods face difficulties in time synchronization and complex data calibration when processing multi-source heterogeneous data (visual, geomagnetic, temperature, etc.), hindering the full utilization of sensor synergy. Furthermore, real-time processing of massive amounts of sensor data places extremely high demands on computing resources, and traditional data processing algorithms struggle to meet the forklift's need for rapid response to environmental information in dynamic operating scenarios.

[0005] Low level of intelligent decision-making: Traditional forklifts rely on preset rules for obstacle avoidance and path decision-making, lacking the ability to autonomously learn and dynamically make decisions in complex scenarios. For example, when encountering unexpected obstacles, they are unable to make optimal avoidance decisions based on the obstacle's type, speed, and direction of movement. In terms of energy efficiency management, they cannot rationally adjust operating modes based on cargo flow and equipment power status, resulting in energy waste and increased equipment wear.

[0006] Difficulties in equipment maintenance and collaborative operations: Existing forklift maintenance mostly adopts a passive mode of regular inspections or post-fault repairs. It is impossible to predict the wear and failure of equipment components in advance, which can easily cause unplanned downtime and affect production efficiency. In addition, in the scenario of multi-forklift collaborative operation, the traditional centralized scheduling system has problems such as high communication delay and unreasonable task allocation, which makes it difficult to achieve efficient collaboration and conflict resolution between multiple vehicles, limiting the development of large-scale automated operations.

[0007] Therefore, to address the above problems, a forklift dynamic path planning method based on deep reinforcement learning is proposed. Summary of the Invention

[0008] The purpose of the present invention is to provide a forklift dynamic path planning method based on deep reinforcement learning to solve the problems raised in the above background technology.

[0009] To achieve the above object, the present invention provides the following technical solutions:

[0010] A forklift dynamic path planning method based on deep reinforcement learning includes the following steps:

[0011] S1. Multi-source sensor deployment and calibration: Install multi-view visual sensors, geomagnetic sensors, sun angle sensors, temperature sensors, and wear monitoring sensors, and perform multi-view image spatial alignment and fusion using an insect-like compound eye vision model;

[0012] S2. Environmental state space construction: Real-time sensor data is collected and integrated with geomagnetic direction, solar angle, temperature, and equipment wear data. This data is combined with the forklift's operating speed, direction, load status, and current time information to generate a high-dimensional state space vector.

[0013] S3. Deep reinforcement learning decision generation:

[0014] Path planning decision: Using a migratory bird navigation heuristic model, it integrates GPS and environmental characteristics when the GPS signal is good. When GPS fails, it switches to geomagnetic information, solar angle, and historical path memory, and calculates the optimal path through a position estimation function.

[0015] Dynamic obstacle avoidance decision-making: Based on the biological stress response mechanism, it detects the type, speed, distance and movement direction of sudden obstacles, and generates avoidance direction and speed adjustment instructions based on the forklift's real-time movement status;

[0016] Energy efficiency management decisions: Mimicking biological circadian rhythms, the system triggers sleep mode during off-peak hours to shut down non-critical sensors based on cargo flow forecasts and battery status. It then automatically wakes up and performs a preheating self-test before peak hours.

[0017] S4, Action Execution and Feedback: The control system drives the actuator to complete path tracking and obstacle avoidance, and feedback the deviation between the actual state and the plan through sensors;

[0018] S5. Online model optimization: Based on the reward and punishment results of the reward function, the model weights are regularly fine-tuned. The reward function is composed of the weighted sum of path efficiency reward, obstacle avoidance success reward and energy consumption penalty, and the weight coefficient is adaptively adjusted.

[0019] As a preferred solution, data fusion in the construction of the environmental state space includes:

[0020] A temporal convolutional network is used to extract the time-dependent features of sensor data, and the weights of multi-source data are dynamically allocated through the attention mechanism.

[0021] The wear data is fed into a recursive jump component to predict the remaining service life, and a maintenance request is triggered when it falls below a threshold;

[0022] During fusion, the data from each sensor is multiplied by the dynamic weight and then summed up. The weight is calculated in real time by the attention mechanism.

[0023] As a preferred solution, the deep reinforcement learning model in step S3 adopts a dual network architecture:

[0024] Action Generation Network: Outputs steering angle and speed commands, embeds a Transformer encoding layer to expand the input dimension, and uses self-attention to focus on key state features;

[0025] Value Assessment Network: evaluates the action value, injects uncorrelated dynamic zero-mean Gaussian noise into the action layer, and dynamically adjusts the noise variance to balance exploration and exploitation;

[0026] In the self-attention mechanism, the scaled dot product of the query matrix and the key matrix is added to the mask matrix, and the weighted value matrix is normalized.

[0027] As a preferred solution, the model training process of the deep reinforcement learning model includes:

[0028] Adopting the importance sample replay mechanism, taking the time difference error as the sample priority;

[0029] When the reward of the new strategy increases by more than 5% for multiple consecutive training cycles, the old strategy is replaced;

[0030] The Gaussian noise variance decays exponentially with the number of training steps, and the initial variance is controlled by the decay coefficient.

[0031] As a preferred solution, the triggering conditions for dynamic obstacle avoidance decision-making are:

[0032] Obstacle avoidance is activated when the relative distance between the obstacle and the forklift is less than or equal to the forklift speed multiplied by the system response time plus the safety margin;

[0033] The obstacle avoidance trajectory uses cubic spline interpolation to generate a safety curve with a standard acceleration change rate;

[0034] The collision time is calculated by dividing the relative distance by the projection of the speed difference between the forklift and the obstacle in the direction of relative motion.

[0035] As a preferred option, energy efficiency management decisions include:

[0036] In sleep mode, only the geomagnetic sensor and low-power communication module are running, and energy consumption is reduced to 15% of normal state;

[0037] After waking up, the bearing temperature and motor wear data are pre-checked through the TCRSCANet network, and the startup is delayed in case of abnormality;

[0038] The remaining useful life prediction value is the sum of the constant term and the product of the weights of each life characteristic.

[0039] As a preferred solution, the reward function satisfies:

[0040] The path efficiency reward is inversely proportional to the degree to which the actual path duration deviates from the theoretical shortest path;

[0041] The reward for successful obstacle avoidance is a positive value when avoiding an obstacle, and a fixed penalty value is used when a collision occurs;

[0042] The energy penalty is proportional to the instantaneous power consumption and automatically returns to zero in sleep mode;

[0043] The weight coefficients are gradually updated according to the partial derivatives of the reward function with respect to each coefficient multiplied by the learning rate.

[0044] As a preferred solution, the remaining service life prediction of the recursive jump component includes:

[0045] When the wear characteristic data mutation amplitude exceeds three times the historical mean standard deviation, the emergency maintenance protocol is initiated;

[0046] The prediction results are corrected in real time through Kalman filtering.

[0047] As a preferred solution, the obstacle avoidance trajectory generated by cubic spline interpolation must meet the following requirements:

[0048] The acceleration rate of change is continuous and the peak value does not exceed the 5g safety threshold of ISO 3691-4 standard;

[0049] The curvature of the track matches the maximum steering angle rate of the forklift to avoid mechanical overload.

[0050] As a preferred solution, the updating rules of the weight coefficients include:

[0051] When the path tracking deviation exceeds the set threshold, the path efficiency reward weight increases by 20%;

[0052] Automatically reduce energy consumption penalty weight in high temperature environments to prioritize safety and obstacle avoidance.

[0053] It can be seen from the technical solutions provided by the present invention that the present invention provides a forklift dynamic path planning method based on deep reinforcement learning, which has the following beneficial effects:

[0054] High-precision environmental perception and data fusion: By deploying multiple sensors, including multi-perspective vision, geomagnetic sensors, and solar angle sensors, and applying technologies such as insect-like compound eye vision models and temporal convolutional networks, the system achieves precise calibration and deep fusion of multi-source data. This system can fully and real-timely perceive the forklift's operating environment and its own status, providing an accurate data foundation for path planning and effectively avoiding decision-making errors caused by missing or misjudgment of environmental information.

[0055] Intelligent Decision-Making and Efficient Path Planning: Integrating multiple biologically inspired models, such as migratory bird navigation, biological stress responses, and biological circadian rhythms, the deep reinforcement learning decision-making module can rapidly generate optimal path planning, dynamic obstacle avoidance, and energy efficiency management strategies for different scenarios. It can reliably locate and plan routes regardless of whether the GPS signal is good or bad, making quick avoidance decisions when faced with unexpected obstacles. It can also intelligently adjust the forklift's operating mode based on cargo flow and battery power, significantly improving forklift operating efficiency and the scientific nature of path planning.

[0056] Significantly improved safety performance: The obstacle avoidance trajectory generated by cubic spline interpolation strictly meets the following conditions: the acceleration rate of change is continuous and the peak value does not exceed the 5g safety threshold, and the trajectory curvature matches the forklift's steering angular rate. This ensures the forklift's smooth movement during obstacle avoidance, preventing damage to the mechanical structure from sudden acceleration, deceleration, or oversteering, as well as risks such as cargo falling and vehicle imbalance. At the same time, dynamic obstacle avoidance decisions based on the biological stress response mechanism significantly reduce the probability of collision accidents and ensure safe forklift operations.

[0057] Equipment maintenance and energy efficiency optimization: Recursive jump components are used to analyze equipment wear data to predict remaining service life, enabling preventive maintenance and reducing downtime and maintenance costs caused by equipment failures. Energy efficiency management decisions mimic biological circadian rhythms, intelligently controlling sensor switches and forklift operating modes based on actual needs, reducing unnecessary energy consumption, extending battery life, and improving energy efficiency.

[0058] Multi-vehicle collaboration and system adaptability: Based on a distributed training framework and improved auction algorithm based on federated learning, collaborative operations and conflict resolution between multiple forklifts are achieved, improving the efficiency and coordination of multi-vehicle parallel operations in logistics scenarios. At the same time, through adaptive weight update mechanisms, online model optimization and other technologies, the system can dynamically adjust decision-making strategies according to different operating environments and task requirements, enhancing the system's versatility and adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 This is a schematic flow chart of the steps of a forklift dynamic path planning method based on deep reinforcement learning in the present invention. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0061] In order to better understand the above technical solution, the above technical solution will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0062] like Figure 1 As shown, an embodiment of the present invention provides a forklift dynamic path planning method based on deep reinforcement learning, comprising the following steps:

[0063] S1. Multi-source sensor deployment and calibration: Install multi-view visual sensors, geomagnetic sensors, sun angle sensors, temperature sensors, and wear monitoring sensors, and perform multi-view image spatial alignment and fusion using an insect-like compound eye vision model;

[0064] S2. Environmental state space construction: Real-time sensor data is collected and integrated with geomagnetic direction, solar angle, temperature, and equipment wear data. This data is combined with the forklift's operating speed, direction, load status, and current time information to generate a high-dimensional state space vector.

[0065] S3. Deep reinforcement learning decision generation:

[0066] Path planning decision: Using a migratory bird navigation heuristic model, it integrates GPS and environmental characteristics when the GPS signal is good. When GPS fails, it switches to geomagnetic information, solar angle, and historical path memory, and calculates the optimal path through a position estimation function.

[0067] Dynamic obstacle avoidance decision-making: Based on the biological stress response mechanism, it detects the type, speed, distance and movement direction of sudden obstacles, and generates avoidance direction and speed adjustment instructions based on the forklift's real-time movement status;

[0068] Energy efficiency management decisions: Mimicking biological circadian rhythms, the system triggers sleep mode during off-peak hours to shut down non-critical sensors based on cargo flow forecasts and battery status. It then automatically wakes up and performs a preheating self-test before peak hours.

[0069] S4, Action Execution and Feedback: The control system drives the actuator to complete path tracking and obstacle avoidance, and feedback the deviation between the actual state and the plan through sensors;

[0070] S5. Online model optimization: Based on the reward and punishment results of the reward function, the model weights are regularly fine-tuned. The reward function is composed of the weighted sum of path efficiency reward, obstacle avoidance success reward and energy consumption penalty, and the weight coefficient is adaptively adjusted.

[0071] In this embodiment, step S1 is to provide a comprehensive and accurate environmental perception data foundation for forklift dynamic path planning through the scientific deployment and precise calibration of multiple sensor types, and to improve the fusion quality and reliability of image data using an insect-like compound eye vision model. The detailed steps are as follows:

[0072] Step S1-1: Multi-source sensor selection and installation planning:

[0073] Determine sensor type: Based on the forklift's operating environment and path planning requirements, select multi-view visual sensors (such as industrial-grade high-definition cameras to acquire images of the surrounding environment), geomagnetic sensors (to measure the Earth's magnetic field and assist in positioning), sun angle sensors (to monitor the sun's azimuth and altitude, providing a natural navigation reference), temperature sensors (to monitor the environment and equipment temperature in real time to prevent equipment failures caused by abnormal temperatures), and wear monitoring sensors (such as vibration sensors and infrared thermal imagers to detect wear on key forklift components).

[0074] Installation location design: Multi-view visual sensors are installed at the front, rear, and both sides of the forklift to provide all-round visual coverage and ensure no blind spots. The geomagnetic sensor is fixed at the center of the forklift chassis to reduce interference from metal components. The sun angle sensor is installed in an unobstructed area on the top of the forklift. Temperature sensors are distributed around key heat-generating components such as the forklift motor and battery, as well as representative locations of the working environment. Wear monitoring sensors are installed near easily worn parts such as the forklift's drive chain, tires, and hydraulic system to ensure accurate acquisition of wear data.

[0075] Step S1-2: Sensor hardware connection and preliminary debugging:

[0076] Hardware connection: Use dedicated data cables to connect various sensors to the forklift's onboard data acquisition system, ensuring a secure connection and good contact, and take waterproof and wear-resistant protective measures for the lines. At the same time, equip the sensors with independent voltage-stabilized power supply modules to ensure stable power supply and prevent voltage fluctuations from affecting data acquisition accuracy.

[0077] Preliminary debugging: Power on the connected sensors to test whether they are working properly. Use the data acquisition system to check whether the raw data output by the sensors is within a reasonable range, such as the magnetic field strength data of the geomagnetic sensor and the temperature reading of the temperature sensor. If abnormal data is found, promptly check for hardware failures or circuit problems and repair them.

[0078] Step S1-3: Sensor data collection and preprocessing:

[0079] Data collection: Set an appropriate data collection frequency, such as using a multi-view visual sensor to collect 15-30 frames of images per second, and geomagnetic sensors, sun angle sensors, temperature sensors, and wear monitoring sensors to collect data 5-10 times per second, to ensure real-time and accurate acquisition of environmental and equipment status information;

[0080] Preprocessing: Filter the collected raw data to remove noise interference. For example, use the median filter algorithm to process temperature sensor data to eliminate random temperature fluctuations. Perform preprocessing such as grayscale conversion and contrast enhancement on images collected by the visual sensor to improve image clarity and recognition, laying the foundation for subsequent image fusion.

[0081] Step S1-4: Construction of insect compound eye vision model and image fusion:

[0082] Model construction: Based on the principle that multiple ommatidia of insect compound eyes work together to perceive the environment, an insect-like compound eye vision model was established. This model contains multiple submodules, each corresponding to the image data processing collected by visual sensors at different viewpoints. By designing a specific neural network structure (such as a convolutional neural network (CNN)), feature extraction and matching of multi-view images are achieved.

[0083] Image spatial alignment: Utilize feature matching algorithms (such as SIFT and SURF) to extract feature points from multi-view images. By calculating the matching relationships between feature points, the geometric transformation parameters (translation, rotation, and scaling parameters) between different images are determined. The images are then spatially transformed to accurately align the multi-view images in space.

[0084] Image fusion: Algorithms such as weighted averaging and Laplace pyramid fusion are used to fuse aligned multi-view images. Weights are assigned based on factors such as image clarity and information content to generate a fused image containing multi-view information. This fused image can more comprehensively and clearly reflect the forklift's surrounding environment, providing higher-quality visual data support for subsequent path planning.

[0085] Step S1-5: Sensor calibration and error correction:

[0086] Calibration method selection: Different calibration methods are used for different sensor types. For example, geomagnetic sensors are measured in a standard environment with known magnetic field strength and direction to establish a calibration model between the measured value and the true value. Temperature sensors are calibrated by comparison with a high-precision standard thermometer. Multi-view vision sensors are calibrated with internal and external parameters using a checkerboard calibration plate to determine parameters such as the camera's focal length, optical center position, and relative position relationship.

[0087] Error correction: Based on the calibrated model or parameters, real-time error correction is performed on the data collected by the sensor to compensate for measurement errors caused by manufacturing processes, environmental factors, etc., to ensure the accuracy and reliability of the sensor output data and provide accurate environmental perception data for forklift dynamic path planning.

[0088] In this embodiment, data fusion in the construction of the environment state space includes:

[0089] A temporal convolutional network is used to extract the time-dependent features of sensor data, and the weights of multi-source data are dynamically allocated through the attention mechanism.

[0090] The wear data is fed into a recursive jump component to predict the remaining service life, and a maintenance request is triggered when it falls below a threshold;

[0091] During fusion, the data from each sensor is multiplied by the dynamic weight and then summed. The weight is calculated in real time by the attention mechanism;

[0092] Furthermore, step S2 is to construct a high-dimensional state space vector that fully reflects the forklift's operating environment and its own state through multi-dimensional data fusion and feature extraction, providing a rich and effective information basis for deep reinforcement learning decision-making. The following are the detailed steps:

[0093] Step S2-1: Data collection and integration:

[0094] Multi-source data aggregation: The geomagnetic direction, solar angle, temperature, equipment wear data collected by the multi-source sensors in step S1, as well as the forklift operating status data (speed, direction, load) and time information are centrally collected. The forklift operating status data is acquired in real time by onboard sensors, and the time information is recorded using a high-precision clock module to ensure the data timestamp is accurate.

[0095] Data format unification: Standardize the formats of data of different types and sampling frequencies, converting all data into a unified data structure and encoding format to facilitate subsequent data processing and fusion. For example, geomagnetic direction data is converted into angle values, solar angle data is converted into a combination of azimuth and altitude angles, and forklift speed data is unified into meters per second.

[0096] Step S2-2: Temporal Convolutional Network (TCN) feature extraction:

[0097] TCN model construction: Build a temporal convolutional network consisting of multiple layers of causal convolutional layers, residual connections, and activation functions. The causal convolutional layers ensure the temporal order of data processing, meaning that the output at the current moment depends only on the input at the past and current moments, which is consistent with the characteristics of time series data. For example, set 3-5 layers of causal convolutional layers, with a convolution kernel size of 3 in each layer and a gradually increasing number of channels to gradually extract the temporal dependency characteristics of the data.

[0098] Time series feature extraction: The integrated sensor data is input into the TCN model in time series. The model automatically extracts the data's temporal features through convolution operations, such as the temperature variation trend over time and the fluctuation pattern of forklift speed over a period of time. The output feature vector contains the data's time dependency information, providing temporal feature support for subsequent data fusion.

[0099] Step S2-3: Attention mechanism weighted processing:

[0100] Attention module design: A module based on a multi-head attention mechanism is constructed. This module receives feature vectors extracted by the TCN and raw data from each sensor as input. By calculating the similarity between the query, key, and value vectors, it assigns corresponding weights to data from different sources, highlighting the influence of important data and suppressing the interference of minor data.

[0101] Weight calculation and application: Dynamically calculate the attention weight of each data source based on factors such as the reliability of sensor data and its relevance to the current path planning task. For example, in complex environments, the weight of visual sensor data will be relatively higher; when making energy efficiency management decisions, the weight of temperature sensor and power data will be increased. The calculated weights are applied to the original data, and the data is weighted and summed to achieve dynamic adjustment of the importance of multi-source data.

[0102] Step S2-4: Equipment wear prediction and maintenance triggering:

[0103] Recursive Jump Component Application: Data collected by wear monitoring sensors (such as vibration signals and infrared thermal imaging data) is input into the recursive jump component. Based on a recursive neural network (RNN) structure, this component uses a jump connection mechanism to effectively capture long-term dependencies in the data, improving the accuracy of predicting equipment wear trends.

[0104] Remaining Useful Life (RUL) Prediction: Utilizing recursive jump components to analyze equipment wear data and combine historical wear data with equipment operating conditions to establish a RUL prediction model. This model calculates the equipment's RUL. When the predicted RUL falls below a preset threshold, a maintenance request is automatically triggered, alerting personnel to inspect and maintain the forklift, preventing equipment failures from impacting route planning and operational safety.

[0105] Step S2-5: Implementation of modified multi-source fusion algorithm:

[0106] Fusion algorithm calculation: using modified multi-source fusion algorithm (in, for Integrate features at all times; It is a temporal convolutional network; for Input data at the moment; Indicates the sensor type, from 1 to , is the number of sensor types; Sigmoid activation function is used to map the weighted results to a suitable range; Calculate the attention function; 、 is a trainable weight matrix used to calculate query and key vectors; 、 、 Respectively The algorithm combines the time-dependent features extracted by TCN with the attention-weighted multi-source data features to generate the final high-dimensional state space vector.

[0107] Vector output: The result of the fusion calculation is output as a high-dimensional state space vector. This vector comprehensively contains multi-dimensional information about the forklift's operating environment and its own state, providing rich and accurate input data for the deep reinforcement learning decision in the subsequent step S3.

[0108] In this embodiment, the deep reinforcement learning model in step S3 adopts a dual network architecture:

[0109] Action Generation Network: Outputs steering angle and speed commands, embeds a Transformer encoding layer to expand the input dimension, and uses self-attention to focus on key state features;

[0110] Value Assessment Network: evaluates the action value, injects uncorrelated dynamic zero-mean Gaussian noise into the action layer, and dynamically adjusts the noise variance to balance exploration and exploitation;

[0111] In the self-attention mechanism, the scaled dot product of the query matrix and the key matrix is added to the mask matrix, and the weighted value matrix is normalized;

[0112] The model training process of the deep reinforcement learning model includes:

[0113] Adopting the importance sample replay mechanism, taking the time difference error as the sample priority;

[0114] When the reward of the new strategy increases by more than 5% for multiple consecutive training cycles, the old strategy is replaced;

[0115] The Gaussian noise variance decays exponentially with the number of training steps, and the initial variance is controlled by the decay coefficient;

[0116] The triggering conditions for dynamic obstacle avoidance decision are:

[0117] Obstacle avoidance is activated when the relative distance between the obstacle and the forklift is less than or equal to the forklift speed multiplied by the system response time plus the safety margin;

[0118] The obstacle avoidance trajectory uses cubic spline interpolation to generate a safety curve with a standard acceleration change rate;

[0119] The collision time is calculated by dividing the relative distance by the projection of the speed difference between the forklift and the obstacle in the direction of relative motion;

[0120] Energy efficiency management decisions include:

[0121] In sleep mode, only the geomagnetic sensor and low-power communication module are running, and energy consumption is reduced to 15% of normal state;

[0122] After waking up, the bearing temperature and motor wear data are pre-checked through the TCRSCANet network, and the startup is delayed in case of abnormality;

[0123] The remaining useful life prediction value is the sum of the constant term and the product of the weight of each life characteristic;

[0124] Furthermore, step S3 uses deep reinforcement learning technology, combined with multiple biologically inspired models, to achieve intelligent decision-making in path planning, dynamic obstacle avoidance, and energy efficiency management for the forklift, providing strategic support for efficient and safe operation of the forklift. The detailed steps are as follows:

[0125] Step S3-1: Path planning decision implementation:

[0126] Construction of a migratory bird navigation heuristic model: This model simulates the principle that migratory birds use clues such as the Earth's magnetic field and the sun's position to navigate, and establishes a heuristic model for path planning. The model has two operating modes: When the GPS signal is good, the GPS positioning data is integrated with data such as the geomagnetic direction, the sun's angle, and the visual characteristics of the environment, and the global position coordinates of the forklift are constructed using this integrated information. When the GPS signal fails, the model switches to a positioning method based on the geomagnetic direction, the sun's angle, and historical path memory. The geomagnetic sensor obtains the current magnetic field direction information, and combines it with the sun's azimuth and altitude angle monitored by the solar angle sensor to estimate the forklift's relative position in the environment. At the same time, the historical driving path data is referenced to provide a reliable position basis for path planning.

[0127] Position estimation function design: Design a position estimation function based on the above data. Process the fused position information through algorithms such as weighted averaging and Kalman filtering to calculate the optimal path for the forklift in the current environment. For example, when using the weighted averaging algorithm, assign different weights to GPS data, geomagnetic data, and solar angle data based on the reliability of different data sources and environmental conditions. When environmental interference is small, increase the weight of GPS data; when the GPS signal is poor, increase the weight of geomagnetic and solar angle data, thereby accurately calculating the optimal driving path.

[0128] Step S3-2: Dynamic obstacle avoidance decision execution:

[0129] Biological stress response mechanism simulation: Drawing on the principle that organisms rapidly respond to sudden dangers, a dynamic obstacle avoidance decision-making mechanism for forklifts is established. Multi-view visual sensors, lidar, and other equipment are used to detect the type (fixed or mobile), speed, distance, and direction of sudden obstacles in real time. This is combined with the forklift's own real-time motion status (speed, direction, load, etc.) to quickly analyze potential collision risks.

[0130] Avoidance command generation: When an obstacle is detected and the obstacle avoidance trigger conditions are met (such as the relative distance between the obstacle and the forklift reaches a preset threshold), the avoidance direction and speed adjustment instructions are generated based on the biological stress response mechanism through a preset obstacle avoidance algorithm (such as the artificial potential field method, the rapid exploration random tree algorithm, etc.). For example, when using the artificial potential field method, the obstacle is regarded as the source of repulsive force and the target position is regarded as the source of attractive force. The avoidance direction of the forklift is determined according to the direction of the resultant force. The appropriate speed adjustment value is calculated based on the current speed and load of the forklift to ensure that the forklift can avoid the obstacle safely and quickly.

[0131] Step S3-3: Energy efficiency management decision making:

[0132] Application of the biological circadian rhythm model: By mimicking the biological circadian rhythm characteristics, energy efficiency management strategies are developed based on cargo flow forecast data and forklift battery status. By analyzing historical operation data and real-time order information, cargo flow conditions in different time periods are predicted, while also monitoring the remaining battery level of forklifts in real time.

[0133] Implementation of sleep and wake-up strategies: During off-peak hours, when cargo flow is low and the forklift's battery level is below a certain threshold, sleep mode is triggered, shutting down non-critical sensors (such as some visual sensors and sun angle sensors), leaving only the geomagnetic sensor and low-power communication module operational to reduce energy consumption. Before peak hours arrive, the forklift system automatically wakes up according to the preset wake-up time and performs a warm-up self-test. Using a TCRSCANet-based pre-test model, key forklift components (such as the motor, hydraulic system, and battery) are quickly tested to ensure normal operation during peak hours, achieving optimized energy efficiency management.

[0134] Step S3-4: Deep reinforcement learning model architecture construction:

[0135] Actor Network Construction: An actor network is constructed to output steering angle and speed commands in a continuous action space. The network is embedded in a Transformer encoding layer and uses a self-attention mechanism to process high-dimensional state space vectors, focusing on key state features relevant to the current decision (such as obstacle location and target path direction). For example, the self-attention mechanism calculates the correlation between various state features, assigning higher weights to important features, thereby more accurately generating steering angle and speed commands.

[0136] Critic Network Construction: A critic network is constructed to evaluate the action-value function. Dynamic Gaussian noise is introduced in the action selection layer to increase the exploratory nature of decision-making. The network receives the state vector and action vector as input, and calculates the value of the action in the current state through a multi-layer neural network. This provides feedback to the actor network, guiding it to optimize its action strategy.

[0137] Modified Transformer-Attention Module Implementation: Implement the Modified Transformer-Attention Module (in, is the modified attention output; is the query vector, used to query relevant information; is the key vector, used for matching information; is a value vector, used to provide actual information; is the dimension of the key vector; It is an element-by-element multiplication operation; It is a dynamic mask matrix that is adaptively adjusted according to the distance of the obstacle. m, shielding non-emergency features and highlighting emergency information, improving the timeliness and accuracy of decision-making, and enhancing the network's ability to capture key information in complex environments;

[0138] Steps S3-5: Deep reinforcement learning model training and optimization:

[0139] Training data preparation: During the actual operation of the forklift, data such as the state vector, action instructions, reward value, and next state vector are collected to construct a training dataset. The reward value is calculated based on the reward function in step S5 and reflects the quality of each decision.

[0140] Model training process: Using the importance sample replay mechanism, with the time series difference error as the sample priority, we prioritize the samples with larger errors for training to improve training efficiency; dynamically update the strategy network through paired t-test, evaluate the pros and cons of different strategies, and adjust the network parameters in time; use modified Gaussian to explore the noise attenuation mechanism (in, is the standard deviation of the exploration noise at time t; is the initial noise standard deviation; is the noise attenuation coefficient; is the training time step; is the noise compensation coefficient; For the current reward; For the average reward, when the current reward is lower than the average reward, enhance exploration to find a better strategy), balance the exploration and utilization capabilities of the model, and continuously optimize the deep reinforcement learning model so that it can make more efficient and safe decisions in complex environments.

[0141] In this embodiment, the function of step S4 is to convert the action instructions generated by deep reinforcement learning decisions into actual operation of the forklift, and to provide real-time feedback of the deviation between the actual state and the planned state through sensors, providing data basis for subsequent model optimization and forming a closed-loop control. The following are the detailed steps:

[0142] Step S4-1: Action instruction analysis and conversion:

[0143] Command reception: The forklift control system receives the action commands generated by the deep reinforcement learning model in step S3, including steering angle and speed commands for path planning decisions, and avoidance direction and speed adjustment commands for dynamic obstacle avoidance decisions. These commands are transmitted to the control system's central processing unit (CPU) in the form of digital signals.

[0144] Format parsing: The CPU parses the format of the received instructions and converts them into specific parameters that the control system can recognize. For example, it converts the steering angle instruction from an angle value into the control pulse signal parameters of the steering motor, and converts the speed instruction into the speed control parameters of the drive motor, ensuring that the instructions can be accurately transmitted to the actuator of the forklift.

[0145] Step S4-2: Actuator drive and action execution:

[0146] Drive system control: Based on the parsed action instructions, the control system sends control signals to the forklift's drive system. For the steering system, the steering motor is driven to rotate the corresponding angle according to the instructions, driving the forklift's steering mechanism to achieve steering action. For the drive system, the drive motor is controlled to adjust the speed and torque to make the forklift travel at the planned speed, ensuring that the forklift can follow the planned path.

[0147] Multi-actuator collaboration: During the execution of an action, multiple actuators are coordinated to work together. For example, during dynamic obstacle avoidance, not only the steering and drive motors must be controlled, but the hydraulic system must also be adjusted synchronously to ensure that the forks remain stable in their lifting and tilting positions while the forklift changes direction and speed, preventing cargo from falling or the forklift from becoming unbalanced, ensuring operational safety.

[0148] Step S4-3: Real-time collection of sensor data:

[0149] Condition monitoring sensors: Utilize various sensors installed on the forklift to collect real-time data on the forklift's actual operating status. Speed sensors monitor the forklift's actual speed, angle sensors detect the forklift's steering angle and fork tilt angle, acceleration sensors obtain acceleration and deceleration information, and position sensors (such as GPS, geomagnetic sensors, etc.) provide real-time feedback on the forklift's actual position coordinates.

[0150] Environmental perception sensors: Multi-view visual sensors, lidar and other environmental perception sensors continuously collect environmental data around the forklift, detecting whether new obstacles appear or whether the environment layout changes, providing a basis for determining whether the forklift deviates from the planned path or needs to re-plan the path;

[0151] Step S4-4: Calculation of deviation between actual state and plan:

[0152] Data comparison: Compare the actual state data collected by the sensor with the ideal state data corresponding to the action instruction planned in step S3; for example, compare the actual driving speed with the planned speed, the actual steering angle with the planned steering angle, the actual position with the target position on the planned path, etc.

[0153] Deviation calculation: Quantify the deviation between the actual state and the planned state through methods such as difference calculation and error percentage calculation. For example, the speed deviation is calculated as the difference between the actual speed and the planned speed. The position deviation can be measured by calculating the Euclidean distance between the actual position coordinates and the coordinates of the corresponding point on the planned path, providing specific quantitative indicators for subsequent model optimization.

[0154] Step S4-5: Deviation feedback and recording:

[0155] Feedback mechanism: The calculated deviation information is transmitted back to the input layer of the deep reinforcement learning model in real time through the feedback channel of the control system, serving as an important basis for model optimization. At the same time, the deviation data is intuitively displayed on the forklift monitoring interface, allowing operators to understand the forklift's operating status in real time and perform manual intervention when necessary.

[0156] Data recording: The deviation data obtained from each calculation, the corresponding action instructions, and the collected sensor data are classified and stored to establish an operation status record database; this data can be used for subsequent analysis of the forklift's operating performance, offline training and optimization of deep reinforcement learning models, as well as fault diagnosis and preventive maintenance, continuously improving the accuracy and reliability of the forklift's dynamic path planning system.

[0157] In this embodiment, the reward function satisfies:

[0158] The path efficiency reward is inversely proportional to the degree to which the actual path duration deviates from the theoretical shortest path;

[0159] The reward for successful obstacle avoidance is a positive value when avoiding an obstacle, and a fixed penalty value is used when a collision occurs;

[0160] The energy penalty is proportional to the instantaneous power consumption and automatically returns to zero in sleep mode;

[0161] The weight coefficients are gradually updated according to the partial derivatives of the reward function with respect to each coefficient multiplied by the learning rate;

[0162] Furthermore, step S5 is to build a reward feedback mechanism and model optimization strategy to adjust the deep reinforcement learning model online based on the actual performance of the forklift during operation, thereby continuously improving the model's decision-making ability. The detailed steps are as follows:

[0163] Step S5-1: Reward value calculation and distribution:

[0164] Path efficiency reward calculation: Calculate the path efficiency reward based on the actual path information fed back in step 54 Specifically, by calculating the time deviation between the actual route and the theoretical shortest route, if the actual driving time is closer to the theoretical shortest time, The higher the value, the lower the value, so that its value is inversely proportional to the time deviation between the two (where, is the path efficiency reward, which is used to measure the efficiency of forklift path planning);

[0165] Obstacle avoidance success reward determination: Determine the obstacle avoidance success reward based on the actual results of the dynamic obstacle avoidance process ; When the forklift successfully avoids the obstacle, Assign a positive value; if a collision occurs, a penalty value is given ( is a pre-set collision penalty coefficient) to motivate the model to generate better obstacle avoidance decisions (where, Reward for successful obstacle avoidance, reflecting the effectiveness of the obstacle avoidance operation);

[0166] Energy consumption penalty calculation: Real-time monitoring of the instantaneous power consumption during the operation of the forklift, based on the energy consumption and The energy consumption penalty is calculated in direct proportion to the forklift's power consumption. The larger the value, the more it will guide the model to optimize energy efficiency management decisions by punishing high energy consumption behaviors (where Energy consumption penalty, used to constrain forklift operation energy consumption);

[0167] Total Reward Synthesis: Utilizing Reward Functions Calculate the total reward value ;in, 、 、 is the adaptive weight coefficient, the initial value is set according to the task requirements, and is subsequently dynamically adjusted through the adaptive weight update mechanism (where is the total reward value, which comprehensively reflects the quality of a decision-making behavior; 、 、 are the weight ratios of path efficiency, obstacle avoidance success, and energy consumption penalty in the total reward respectively);

[0168] Step S5-2: Adaptive weight update:

[0169] Loss function definition: Using loss function Measures the difference between the current reward and the expected reward. This function can map the reward value to a loss value that is easy to optimize, and improve the model decision effect by minimizing the loss value (where, is the loss function used to evaluate the deviation of the model decision from the ideal result);

[0170] Gradient calculation and weight adjustment: according to the loss function Adaptive weight coefficient 、 、 Find the partial derivative and get the gradient 、 、 ; Using the formula ( 、 Similarly) weight update, where is the learning rate, which controls the step size of weight adjustment; the weights are continuously optimized through the gradient ascent algorithm, so that the model is more inclined to obtain high-reward behavior patterns in subsequent decisions (where, 、 、 is the weight coefficient of the next moment; 、 、 is the weight coefficient at the current moment; is the learning rate, which determines the magnitude of weight update);

[0171] Step S5-3: Model parameter fine-tuning:

[0172] Sample screening and replay: Using the importance sample replay mechanism, with temporal difference error (TD-error) as the sample priority evaluation criterion, we screen out samples that are more valuable for model optimization. Samples with larger TD-errors indicate a greater deviation between the current model's assessment of the state-action pair and the actual result. These samples are prioritized for training from the replay buffer to improve training efficiency.

[0173] Network parameter update: The filtered samples are input into the deep reinforcement learning model, based on the total reward value and loss function The back-propagation algorithm is used to calculate gradients and update the parameters of the actor and critic networks. A paired t-test is also used to dynamically evaluate the policy network, comparing the effectiveness of different policies and further optimizing the network structure and parameters to ensure that the model can adapt to environmental changes and task requirements.

[0174] Step S5-4: Regular model evaluation and optimization:

[0175] Performance indicator setting: Develop model performance evaluation indicators, including path planning success rate, obstacle avoidance success rate, average energy consumption reduction rate, etc., to measure the model decision-making effect from multiple dimensions;

[0176] Regular evaluation and optimization: The deep reinforcement learning model is comprehensively evaluated at preset intervals (e.g., every task cycle or every 8 hours of work). Based on the evaluation results, if it is found that the model does not meet expectations on certain indicators, the reward function parameters, network structure, or training strategy are readjusted to continuously optimize the model performance to ensure that the forklift dynamic path planning system always maintains efficient and reliable operation.

[0177] In this embodiment, the remaining useful life prediction of the recursive jump component includes:

[0178] When the wear characteristic data mutation amplitude exceeds three times the historical mean standard deviation, the emergency maintenance protocol is initiated;

[0179] The prediction results are corrected in real time through Kalman filtering;

[0180] The recursive jump component is the core module used in this invention to process equipment wear data and predict remaining useful life (RUL). Its design is inspired by the ability of recursive neural networks to process time series data, and it enhances the capture of long-term dependencies through a jump connection mechanism. The following is a detailed description:

[0181] Architectural design of recursive jump components:

[0182] 1. Basic recursive network layer:

[0183] Core structure: A multi-layer recurrent neural network is constructed using long short-term memory (LSTM) or gated recurrent units (GRU) as basic units. This network can effectively process the temporal characteristics of wear monitoring data and capture the dynamic changes in equipment wear through memory units.

[0184] Input processing: Receives time series data collected by wear monitoring sensors (such as vibration signals, temperature fluctuations, component deformation, etc.) and normalizes the data to eliminate the impact of dimensional differences on the model;

[0185] 2. Skip connection mechanism:

[0186] Cross-layer information transfer: Skip connections are introduced between different layers of a recurrent network, allowing low-level features to be directly transferred to higher layers. For example, a direct connection is established between the first and third layers of an LSTM network to avoid the vanishing gradient problem in deep network training while preserving key information about early wear features.

[0187] Feature fusion: Skip connections fuse wear features at different time scales. The bottom-level network captures short-term wear fluctuations (such as immediate vibration anomalies), while the upper-level network learns long-term wear trends (such as gradual aging of components). Skip connections enable the complementarity of multi-scale features.

[0188] 3. Attention Weighted Module:

[0189] Key feature screening: An attention mechanism is embedded in the skip connection path to dynamically assign weights based on the importance of wear data. For example, vibration peak signals with a high probability of causing equipment failure are given higher weights, while irrelevant fluctuations caused by environmental noise are ignored.

[0190] Weight calculation: By calculating the similarity between the query vector (Query) and the key vector (Key), attention weights for different wear characteristics are generated. The formula is: (in, is the query vector, is the key vector, is a value vector, is the key vector dimension, and this mechanism ensures that the model focuses on the wear characteristics that are most critical to the remaining life prediction);

[0191] The workflow of the recursive jump component:

[0192] 1. Wear data preprocessing:

[0193] Data cleaning: De-noising the raw wear data collected by vibration sensors, infrared thermal imagers and other equipment, using wavelet transform or median filtering to eliminate random noise;

[0194] Feature extraction: Convert the time-domain wear signal into frequency-domain features through short-time Fourier transform (STFT) or empirical mode decomposition (EMD), and extract key indicators such as peak frequency and energy distribution;

[0195] 2. Temporal feature learning:

[0196] Recursive network processing: The preprocessed feature sequence is input into the recursive network. LSTM or GRU units control the flow of information through forget gates, input gates, and output gates, memorizing the historical state of equipment wear. For example, when a forklift drive chain wears out, the network records the change pattern of the vibration signal over time.

[0197] Skip connection enhancement: Wear features extracted from early layers (such as slight deformation signals) are directly transferred to the deep network through skip connections, and combined with severe wear features extracted from higher layers (such as component crack signals). This prevents the deep network from losing initial features due to the increase in the number of layers.

[0198] 3. Remaining life prediction:

[0199] Multi-scale feature fusion: Skip connections are used to fuse wear features at different levels to form a comprehensive feature vector that includes short-term anomalies and long-term trends;

[0200] Lifetime mapping model: The comprehensive feature vector is mapped to the remaining useful life (RUL) prediction value through the fully connected layer. The formula is: (in, is the wear data series at time t, is the recursive network output, is the feature transferred by the skip connection, is the feature splicing operation, is the mapping function);

[0201] 4.Maintenance trigger mechanism:

[0202] Threshold comparison: The predicted RUL is compared with the preset maintenance threshold (such as 20% of the equipment design life). When the RUL is lower than the threshold, a maintenance request signal is automatically generated;

[0203] Dynamic threshold adjustment: Dynamically adjust the maintenance threshold based on the forklift's operating conditions (such as load and operating environment temperature). For example, in high-temperature environments, the threshold can be appropriately lowered to facilitate maintenance in advance.

[0204] Technical advantages of recursive jump components:

[0205] Long-distance dependency capture: Skip connections are used to address the problem of traditional recurrent networks' inability to capture long-term wear trends, thereby improving long-term prediction accuracy.

[0206] Noise immunity: The attention mechanism filters out irrelevant noise and focuses on key wear characteristics, making it suitable for strong noise scenarios in industrial environments;

[0207] Incremental learning capability: supports online updating of model parameters. When new wear data is input, it adapts to the dynamic changes of device aging by fine-tuning the jump connection weights.

[0208] Enhanced interpretability: Skip connections explicitly convey wear characteristics at different levels, facilitating analysis of key factors leading to equipment failure (e.g., abnormal vibration frequency of a component).

[0209] Application scenarios and examples:

[0210] Forklift drive chain wear prediction: By collecting chain vibration data, the recursive jump component can predict the risk of chain breakage due to wear in advance, improving prediction accuracy by over 30% compared to traditional threshold detection methods.

[0211] Tire wear monitoring: Combining tire tread images captured by vision sensors with pressure sensor data, the component can predict the remaining mileage of a tire, providing data support for fleet maintenance plans;

[0212] Through the innovative combination of recursive networks and jump connections, this component achieves accurate analysis of equipment wear status and remaining life prediction, providing core technical support for preventive maintenance and efficient operation of forklifts.

[0213] In this embodiment, the obstacle avoidance trajectory generated by cubic spline interpolation must meet the following requirements:

[0214] The acceleration rate of change is continuous and the peak value does not exceed the 5g safety threshold of ISO 3691-4 standard;

[0215] The curvature of the track matches the maximum steering angle rate of the forklift to avoid mechanical overload;

[0216] Furthermore, the core goal of using cubic spline interpolation to generate obstacle avoidance trajectories is to achieve smooth and efficient path planning while ensuring the safe operation of the forklift, thereby avoiding the risk of mechanical damage or collision caused by sudden trajectory changes. The following describes this in detail from the perspectives of principle, constraints, and practical application:

[0217] 1. Cubic spline interpolation principle:

[0218] Cubic spline interpolation is a mathematical method that fits discrete data points using a piecewise cubic polynomial function. Its advantage lies in ensuring the continuity and smoothness of the curve at the connection points. In forklift obstacle avoidance scenarios, this method uses the forklift's current position, obstacle position, target position, and motion state (speed, direction) as inputs. By solving specific boundary conditions and constraint equations, it generates a smooth obstacle avoidance trajectory.

[0219] Piecewise polynomial construction: The obstacle avoidance path is divided into multiple subintervals, and a cubic polynomial function is used in each subinterval. Describe the trajectory, (where, For the The sub-intervals are Trajectory position function at time; is a constant term; 、 、 are the coefficients of the linear, quadratic, and cubic terms, respectively, which together determine the shape of the trajectory);

[0220] Boundary condition constraints: By setting boundary conditions such as the position, speed, and acceleration of the starting and ending points, the trajectory is ensured to be seamlessly connected with the forklift's current motion state and target state. For example, the speed at the starting point must be consistent with the forklift's current speed, and the direction at the end point must point to the subsequent planned path.

[0221] 2. Analysis of security constraints:

[0222] 2.1 Acceleration rate of change is continuous and peak value limited:

[0223] Acceleration rate (jerk) continuity: Cubic spline interpolation ensures the continuity of the first-order derivative (velocity), second-order derivative (acceleration), and third-order derivative (jerk) of adjacent polynomials at the connection points, avoiding sudden changes in acceleration or deceleration in the trajectory, thereby ensuring the smooth movement of the forklift. For example, when steering to avoid obstacles, jerk continuity can prevent cargo from sliding due to inertia or the forklift from losing balance.

[0224] 5g safety threshold limits:

[0225] According to ISO 3691-4 (Industrial Truck Safety Specification), the peak acceleration of a forklift during emergency operation must not exceed ( is the acceleration due to gravity, approximately ), in order to protect the mechanical structure and operator safety; in the process of trajectory generation, the acceleration of any point on the trajectory is converted to Limited to a safe range, the formula is: (in, for The acceleration vector at the moment; is the vector norm, used to calculate the magnitude of the acceleration vector; 5 (This is the peak acceleration safety threshold during emergency operation of a forklift specified in ISO3691-4);

[0226] 2.2 Track curvature and steering angle rate matching:

[0227] Curvature calculation:

[0228] trajectory curvature Reflects the curvature of the curve. The calculation formula is: (in, is the curvature of the trajectory at time t; and are the first-order derivatives of the trajectory in the x and y directions, respectively, representing the velocity components in the x and y directions at time t; and are the second-order derivatives of the trajectory in the x and y directions, respectively, representing the acceleration components in the x and y directions at time t; the absolute value of the numerator is used to calculate the difference between the cross products of the velocity and acceleration components, and the denominator is used for normalization processing);

[0229] Steering angle rate constraint:

[0230] Maximum steering angle rate of forklift Determined by the mechanical structure (such as steering motor torque, transmission system limitations); to avoid overloading the steering mechanism, the trajectory curvature must be ensured and the current speed of the forklift Match, satisfy: (in, is the curvature of the trajectory at time t; is the maximum steering angular rate of the forklift, in radians per second; is the forklift's speed at time t, in meters per second; this formula ensures that the curvature of the trajectory is compatible with the forklift's steering capabilities to avoid mechanical overload);

[0231] 3. Practical application and optimization strategy:

[0232] Dynamically adjust interpolation nodes: The number and position of cubic spline interpolation nodes are adjusted in real time based on the obstacle distance and forklift speed. For example, when the obstacle is close, the node density is increased to generate a more accurate obstacle avoidance path.

[0233] Multi-objective optimization algorithm: Using methods such as genetic algorithms and particle swarm optimization, the optimal obstacle avoidance trajectory is generated while taking into account the shortest path and lowest energy consumption while meeting safety constraints.

[0234] Real-time verification and correction: Using actual motion data fed back by on-board sensors (such as inertial measurement units and encoders), trajectory execution is verified in real time, and the trajectory is dynamically corrected using model predictive control (MPC) technology to ensure safety and accuracy.

[0235] Through the above design, the obstacle avoidance trajectory generated by cubic spline interpolation can not only meet the mechanical performance and safety standards of the forklift, but also achieve efficient obstacle avoidance in complex environments, providing reliable path planning guarantee for forklift automated operations.

[0236] In this embodiment, the updating rules of the weight coefficients include:

[0237] When the path tracking deviation exceeds the set threshold, the path efficiency reward weight increases by 20%;

[0238] Automatically reduce energy consumption penalty weight in high temperature environments to prioritize safety and obstacle avoidance;

[0239] Furthermore, the weight coefficient update rule is the core of the adaptive reward mechanism in this invention. By dynamically adjusting the weight ratios of path efficiency, obstacle avoidance success, and energy consumption penalty, the deep reinforcement learning model can automatically optimize the decision-making strategy according to different scenarios. The following is a detailed description from four aspects: theoretical basis, mathematical model, practical application, and technical advantages:

[0240] 1. Theoretical significance of weight coefficient:

[0241] Weight coefficient Corresponding reward functions The path efficiency, obstacle avoidance success and energy consumption penalty in ; the dynamic update of these coefficients is based on the following principles:

[0242] Environmental adaptability: In areas with dense obstacles, improve To prioritize obstacle avoidance safety; in long-distance transportation scenarios, increase To optimize path efficiency;

[0243] Task priority: Lower for urgent tasks Sacrifice some energy consumption in exchange for fast response; improve To achieve energy-saving goals;

[0244] Learning stability: The gradient ascent algorithm is used to ensure that the weight update direction is consistent with the reward maximization, avoiding falling into local optimality;

[0245] 2. Mathematical model of weight update:

[0246] 2.1 Loss function design:

[0247] Using loss function Measures the difference between the current reward and the expected reward (where is the loss function, which is used to map the reward value to an optimizable loss value; is the total reward value, is a natural constant, the function is smooth and monotonic, so that negative rewards correspond to high losses and positive rewards correspond to low losses);

[0248] 2.2 Gradient calculation and weight adjustment:

[0249] According to the loss function Adaptive weight coefficient 、 、 Find the partial derivative and get the gradient 、 、 (in, is the loss function weight coefficient The partial derivative of the extent to which the change affects losses; 、 Similarly); using the formula ( 、 Similarly) perform weight update (where, is the weight coefficient for the next moment is the weight coefficient at the current moment ; is the learning rate, which controls the step size of weight adjustment; this formula is based on the principle of gradient ascent, so that the weight is adjusted in the direction of increasing reward);

[0250] 2.3 Dynamic adjustment of learning rate:

[0251] Learning rate According to the dynamic changes in task difficulty, the formula is (in, is the current learning rate; is the initial learning rate; is the attenuation coefficient; The difficulty of the task is comprehensively evaluated through factors such as environmental complexity and the number of obstacles; this mechanism ensures that weight updates are more cautious in complex tasks to avoid excessive fluctuations);

[0252] 3. Practical application process of weight update:

[0253] Initial weight setting:

[0254] According to the basic requirements of forklift operation, the initial weight is set to , , (i.e., path efficiency is prioritized, while taking into account obstacle avoidance and energy consumption);

[0255] Real-time weight adjustment:

[0256] Scene switching: When a forklift enters a narrow passage from an open area, Automatically increases to 0.6, giving priority to obstacle avoidance safety;

[0257] Task urgency: When receiving an expedited order, Increased to 0.7, Reduced to 0.1, sacrificing some energy consumption in exchange for fast response;

[0258] Abnormal energy consumption: When the battery power is less than 20%, Increasing it to 0.4 forces the model to choose a more energy-efficient path;

[0259] Gradient update calculation: based on weight For example, the gradient is calculated as (in, Reward for path efficiency; is the total reward value; this formula transforms the impact of the path efficiency reward on the total loss into an optimizable gradient through the exponential transformation of the reward value); if is positive and Larger, the gradient is positive, Increase and strengthen the weight of path efficiency;

[0260] Weight constraints and normalization:

[0261] After each update, make sure , and limit (Normalization ensures that the sum of the weight ratios is 1, maintaining the balance of the reward function; upper and lower bound constraints prevent a single weight from being too large, causing other factors to be ignored).

[0262] 4. Technical advantages of weight update:

[0263] Adaptability: The model can automatically adjust decision priorities based on the real-time environment and task requirements without human intervention;

[0264] Robustness enhancement: Dynamic adjustment of gradient ascent and learning rate can be used to avoid weight oscillation or falling into local optimality.

[0265] Improved interpretability: The weight coefficient directly reflects the importance of each decision factor, making it easier to analyze model behavior;

[0266] This update rule combines mathematical optimization with actual scenarios to achieve intelligent adaptation of forklift decision-making strategies, providing core technical support for efficient operations in complex industrial environments.

[0267] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A forklift dynamic path planning method based on deep reinforcement learning, characterized by: The following steps are involved: S1. Multi-source sensor deployment and calibration: Install multi-view visual sensors, geomagnetic sensors, sun angle sensors, temperature sensors, and wear monitoring sensors, and perform multi-view image spatial alignment and fusion using an insect-like compound eye vision model; S2. Environmental state space construction: Real-time sensor data is collected and integrated with geomagnetic direction, solar angle, temperature, and equipment wear data. This data is combined with the forklift's operating speed, direction, load status, and current time information to generate a high-dimensional state space vector. S3. Deep reinforcement learning decision generation: Path planning decision: Using a migratory bird navigation heuristic model, it integrates GPS and environmental characteristics when the GPS signal is good. When GPS fails, it switches to geomagnetic information, solar angle, and historical path memory, and calculates the optimal path through a position estimation function. Dynamic obstacle avoidance decision-making: Based on the biological stress response mechanism, it detects the type, speed, distance and movement direction of sudden obstacles, and generates avoidance direction and speed adjustment instructions based on the forklift's real-time movement status; Energy efficiency management decisions: Mimicking biological circadian rhythms, the system triggers sleep mode during off-peak hours to shut down non-critical sensors based on cargo flow forecasts and battery status. It then automatically wakes up and performs a preheating self-test before peak hours. The deep reinforcement learning model uses a dual network architecture: Action Generation Network: Outputs steering angle and speed commands, embeds a Transformer encoding layer to expand the input dimension, and uses self-attention to focus on key state features; Value Assessment Network: evaluates the action value, injects uncorrelated dynamic zero-mean Gaussian noise into the action layer, and dynamically adjusts the noise variance to balance exploration and exploitation; In the self-attention mechanism, the scaled dot product of the query matrix and the key matrix is added to the mask matrix, and the weighted value matrix is normalized; S4, Action Execution and Feedback: The control system drives the actuator to complete path tracking and obstacle avoidance, and feedback the deviation between the actual state and the plan through sensors; S5. Online model optimization: Regularly fine-tune the model weights based on the reward and punishment action results of the reward function. The reward function is composed of the weighted sum of path efficiency reward, obstacle avoidance success reward and energy consumption penalty, and the weight coefficient is adaptively adjusted.

2. A forklift dynamic path planning method based on deep reinforcement learning according to claim 1, characterized in that: The data fusion in the construction of the environment state space includes: A temporal convolutional network is used to extract the time-dependent features of sensor data, and the weights of multi-source data are dynamically allocated through the attention mechanism. The wear data is fed into a recursive jump component to predict the remaining service life, and a maintenance request is triggered when it falls below a threshold; During fusion, the data from each sensor is multiplied by the dynamic weight and then summed up. The weight is calculated in real time by the attention mechanism.

3. The forklift dynamic path planning method based on deep reinforcement learning according to claim 1, characterized in that: The model training process of the deep reinforcement learning model includes: Adopting the importance sample replay mechanism, taking the time difference error as the sample priority; When the reward of the new strategy increases by more than 5% for multiple consecutive training cycles, the old strategy is replaced; The Gaussian noise variance decays exponentially with the number of training steps, and the initial variance is controlled by the decay coefficient.

4. The forklift dynamic path planning method based on deep reinforcement learning according to claim 1, characterized in that: The triggering conditions for the dynamic obstacle avoidance decision are: Obstacle avoidance is activated when the relative distance between the obstacle and the forklift is less than or equal to the forklift speed multiplied by the system response time plus the safety margin; The obstacle avoidance trajectory uses cubic spline interpolation to generate a safety curve with a standard acceleration change rate; The collision time is calculated by dividing the relative distance by the projection of the speed difference between the forklift and the obstacle in the direction of relative motion.

5. The forklift dynamic path planning method based on deep reinforcement learning according to claim 1, characterized in that: The energy efficiency management decision includes: In sleep mode, only the geomagnetic sensor and low-power communication module are running, and energy consumption is reduced to 15% of normal state; After waking up, the bearing temperature and motor wear data are pre-checked through the TCRSCANet network, and the startup is delayed in case of abnormality; The remaining useful life prediction value is the sum of the constant term and the product of the weights of each life characteristic.

6. The forklift dynamic path planning method based on deep reinforcement learning according to claim 1, characterized in that: The reward function satisfies: The path efficiency reward is inversely proportional to the degree to which the actual path duration deviates from the theoretical shortest path; The reward for successful obstacle avoidance is a positive value when avoiding an obstacle, and a fixed penalty value is used when a collision occurs; The energy penalty is proportional to the instantaneous power consumption and automatically returns to zero in sleep mode; The weight coefficients are gradually updated according to the partial derivatives of the reward function with respect to each coefficient multiplied by the learning rate.

7. The forklift dynamic path planning method based on deep reinforcement learning according to claim 2, characterized in that: Remaining useful life prediction for recursive jump components includes: When the wear characteristic data mutation amplitude exceeds three times the historical mean standard deviation, the emergency maintenance protocol is initiated; The prediction results are corrected in real time through Kalman filtering.

8. The forklift dynamic path planning method based on deep reinforcement learning according to claim 4 is characterized in that: The obstacle avoidance trajectory generated by the cubic spline interpolation must meet the following requirements: The acceleration rate of change is continuous and the peak value does not exceed the 5g safety threshold of ISO 3691-4 standard; The curvature of the track matches the maximum steering angle rate of the forklift to avoid mechanical overload.

9. The forklift dynamic path planning method based on deep reinforcement learning according to claim 6, characterized in that: The updating rules of the weight coefficients include: When the path tracking deviation exceeds the set threshold, the path efficiency reward weight increases by 20%; Automatically reduce energy consumption penalty weight in high temperature environments to prioritize safety and obstacle avoidance.

Images