A forklift path planning and obstacle avoidance assistance method based on reinforcement learning

Abstract

The invention discloses a forklift path planning obstacle avoidance auxiliary method based on reinforcement learning, and relates to the technical field of forklift control. The auxiliary method has the following specific steps: S100, sensor deployment and calibration: installing a multi-view visual sensor, a geomagnetic sensor, a sun angle sensor, a temperature sensor and a wear monitoring sensor on a forklift, and calibrating various sensors; the invention configures a variety of sensors, such as a multi-view visual sensor, a geomagnetic sensor, and a sun angle sensor, and collects and fuses these data in real time to form a complete state space information vector. On this basis, an insect compound eye visual model and a biological stress response mechanism are used to respectively realize the alignment fusion of image information and the rapid obstacle avoidance response of sudden obstacles. The fusion application of this multi-source biological heuristic information not only improves the path planning accuracy of the forklift in a complex environment, but also enhances its ability to cope with sudden obstacles.

Description

Technical Field

[0001] The present invention relates to the technical field of forklift control, and in particular to a forklift path planning and obstacle avoidance assistance method based on reinforcement learning. Background Art

[0002] In logistics warehousing and manufacturing scenarios, forklifts are important material handling tools. Their operating efficiency and safety are directly related to the efficiency and safety of the entire production process. With the development of intelligent technology, path planning and obstacle avoidance technology based on reinforcement learning has provided new possibilities for the intelligent upgrade of forklifts.

[0003] Traditional forklift path planning and obstacle avoidance technology mainly relies on preset paths, fixed obstacle maps and simple sensor feedback. This method performs well in simple, static environments, but its flexibility and adaptability are limited in complex and changeable industrial scenarios. For example, when encountering sudden obstacles or path changes, traditional technology often cannot make adjustments quickly, which can easily lead to collisions or transportation interruptions. In addition, traditional technology lacks intelligent decision-making capabilities and cannot be dynamically optimized based on real-time cargo flow and forklift status information, thus affecting overall transportation efficiency.

[0004] Therefore, a forklift path planning and obstacle avoidance assistance method based on reinforcement learning is developed to provide efficient, safe and forklift operation solutions for logistics warehousing and production manufacturing scenarios. Summary of the invention

[0005] The purpose of the present invention is to make up for the shortcomings of the prior art and provide a forklift path planning and obstacle avoidance assistance method based on reinforcement learning. The method realizes real-time data acquisition and multi-source information fusion by installing multi-view visual sensors, geomagnetic sensors and multiple sensors, and applying insect compound eye visual models and wavelet transform technology. On this basis, the biological stress response mechanism and the bio-inspired model of migratory bird navigation ability are used for path planning and obstacle avoidance decisions. At the same time, the work-rest and maintenance decisions are combined to optimize the forklift operation. The model parameters are continuously optimized through reinforcement learning, which improves the path planning accuracy and obstacle avoidance ability of the forklift in complex environments and reduces the operating cost.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: a forklift path planning obstacle avoidance auxiliary method based on reinforcement learning, the specific steps of the auxiliary method are:

[0007] S100, sensor deployment and calibration: install multi-view visual sensors, geomagnetic sensors, sun angle sensors, temperature sensors and wear monitoring sensors on forklifts, and calibrate various sensors;

[0008] S200, data collection and fusion: During the operation of the forklift, various sensors continuously collect data, align and fuse images from different perspectives through the insect compound eye vision model, extract feature information from the environment, filter and calibrate the collected geomagnetic and solar angle information, and fuse the pre-processed data with the forklift's operating status data and current time information to form a complete state space information vector;

[0009] S300, comprehensive decision generation: including path planning decision, path planning decision, and work-rest and maintenance decision;

[0010] The path planning decision is based on the current state space information, and the path planning model of the migratory bird navigation capability is used to plan the path. When the GPS signal is good, the GPS information and environmental feature information are used first. When the GPS signal is interfered, the geomagnetic field, the sun angle and the traveled path memory are used, and the position estimation function is used to calculate the current position to plan the optimal path.

[0011] The path planning decision utilizes an obstacle avoidance strategy based on an obstacle avoidance reaction model of a biological stress response mechanism, continuously monitors the environment around the forklift, and when the sensor detects a sudden, fast-approaching obstacle, quickly evaluates the type, speed, distance, and direction of movement of the obstacle, and calculates the avoidance direction and speed adjustment method in combination with the current speed and direction of the forklift;

[0012] The work-rest and maintenance decision is based on a work arrangement and self-repair model that mimics biological circadian rhythms and rest mechanisms, and determines the work mode according to time and cargo flow forecast data. At night, during off-peak hours, and when there are no urgent tasks, it is determined whether to sleep based on power and key component status. When sleeping, some non-critical sensors and equipment are turned off to reduce power consumption. When approaching peak hours, it automatically wakes up according to the wake-up conditions and starts the preheating and self-test procedures. The temperature and wear data of key components are analyzed in real time. When exceeding the limit or reaching the maintenance threshold, a maintenance request is generated according to the self-repair mechanism to notify the maintenance personnel;

[0013] S400, action execution and feedback: Based on the generated path planning, obstacle avoidance, work-rest and maintenance decisions, the forklift control system sends corresponding action instructions to the actuator. While the forklift executes the action, the sensor continuously monitors the actual operating status of the forklift and feeds back the status information to the control system;

[0014] S500, model optimization: Based on the behavior results and status feedback of the forklift, rewards or penalties are given according to the reward function, and the parameters of the model are regularly fine-tuned according to the operation of the forklift in different environments.

[0015] Furthermore, in the data collection and fusion in S200, images from different perspectives are aligned and fused by simulating an insect compound eye vision model. Assume that there are n visual sensors, and the image collected by the i-th sensor is I i (x, y), where x, y are image coordinates, and the fused image is I fused (x,y), the calculation formula is: where w i is the weight of the i-th sensor image.

[0016] Furthermore, in the data collection and fusion step S200, feature information of the environment is extracted by wavelet transform, and the fused image I fused (x, y) is transformed by wavelet to obtain wavelet coefficients W of different scales and directions s,d (x,y), the calculation formula is: Where s is the scale parameter, d is the direction parameter, ψ s,d is a wavelet function, by calculating the energy E of the wavelet coefficients s,d To extract features, the formula is: E s,d =∑ x ∑ y |W s,d (x,y)| 2 .

[0017] Furthermore, in the S300, the potential field method is used to plan the path according to the current position in the comprehensive decision generation, assuming that the obstacle is at the position (x o ,y o ) is the repulsive potential field U rep (x,y), the calculation formula is: Where η is the repulsion coefficient, o is the gravitational potential field generated by the target after traversing all detected obstacles, and U att (x,y), the calculation formula is: Where ξ is the gravitational coefficient, the resultant force on the forklift is the negative gradient of the potential field, and the calculation formula is: The direction of movement of the forklift according to the direction of the resultant force and the current speed The adjustment calculation formula is: where v max is the maximum speed of the forklift, F th is the threshold force.

[0018] Furthermore, in the S300, the position estimation function is used to calculate the current position in the comprehensive decision generation, and the position coordinates of the forklift are (x, y) and the target position coordinates are (x g ,y g ), the geomagnetic intensity vector is The sun angle vector is α is the solar altitude angle, β is the solar azimuth angle, and the coordinates of the memory point on the traveled path are m is the number of memory points, and the position estimation function is constructed using geomagnetic and solar angle information. The formula is: in At the memory point (x k ,y k ) recorded at the geomagnetic and solar angle information, d((x,y),(x k ,y k )) is the Euclidean distance between the current position and the memory point, Z is the normalization constant, and σ is a parameter that controls the estimation accuracy.

[0019] Furthermore, in the S300, the potential danger is found by evaluating the threat level of obstacles in real time in the comprehensive decision generation, assuming that the speed of the forklift is v f , direction is θ f , the position of obstacle j is (x j ,y j ), with a velocity of v j , direction is θ j , the number of obstacles detected is n obs , calculate the threat level T of each obstacle to the forklift j , the formula is: in is the distance between the forklift and the obstacle, θ fj is the direction angle of the forklift pointing to the obstacle. Considering the threats of all obstacles, the formula for the overall danger index D is: where ω j is the weight determined by the obstacle type and size. th When you need to avoid, D th is the danger threshold, the avoidance direction θ avoid The calculation formula is: Adjust speed v new The calculation formula is: new =v f (1-kD), where k is the speed adjustment factor.

[0020] Furthermore, in the S300, the forklift is arranged to work according to the cargo flow and the component status through the work-rest decision formula in the comprehensive decision generation. Assuming the time is t, the cargo flow function is F(t), and the work suitability S(t) is calculated, the formula is: When S(t) low Determine that the forklift is in idle state, S low is the low traffic threshold, and the probability P of each forklift entering sleep mode is calculated. sleep ​, the formula is: Where S mid is the middle threshold, σ is the parameter that controls the slope of the curve. sleep >P th The forklift enters sleep mode, P th is the decision threshold.

[0021] Furthermore, in the S300, the maintenance plan of the forklift is arranged by calculating the maintenance demand index in the comprehensive decision generation, assuming that the temperature of the key component i of the forklift is T i , the wear amount is W i , the normal temperature range is [T i,min ,T i,max ], the wear threshold is W i,th , for each component i, calculate the maintenance requirement index M i The formula is: M i >W i,th Issue maintenance request and follow M i The value of determines the urgency of maintenance.

[0022] Furthermore, in the S400, the path planning decision in the action execution and feedback is made by controlling the steering motor and the drive motor of the forklift to adjust the driving direction and speed of the forklift so that the forklift can travel along the planned path. For obstacle avoidance operations, emergency obstacle avoidance action instructions are quickly executed. For sleep or wake-up operations, the power switches and startup sequences of related equipment are accurately controlled. For maintenance requests, detailed maintenance information is displayed on the display screen or the maintenance request is sent to the maintenance management system through the wireless communication module.

[0023] Furthermore, in S500, the updated Q-value reward function is used to give rewards or penalties in the model optimization, and the state vector is set to The action vector is The reward is r, and the Q value function is The learning rate is α, the discount factor is γ, and the calculation formula is: in Is to perform an action The next state after .

[0024] Compared with the prior art, this forklift path planning and obstacle avoidance assistance method based on reinforcement learning has the following beneficial effects:

[0025] 1. The present invention configures multiple sensors, such as multi-view visual sensors, geomagnetic sensors, and sun angle sensors, and collects and fuses these data in real time to form a complete state space information vector. On this basis, the insect compound eye visual model and biological stress response mechanism are used to respectively realize the alignment fusion of image information and the rapid obstacle avoidance response to sudden obstacles. This fusion application of multi-source biological heuristic information not only improves the path planning accuracy of the forklift in complex environments, but also enhances its ability to cope with sudden obstacles, thereby effectively improving the operating efficiency and safety of the forklift.

[0026] 2. The present invention introduces a work arrangement and self-repair model that imitates biological circadian rhythms and rest mechanisms, and intelligently determines the working mode of the forklift according to time and cargo flow forecast data. When there are no urgent tasks or low-peak periods, the forklift can automatically enter sleep mode based on the power and status of key components, thereby reducing energy consumption and extending service life. It also realizes intelligent maintenance management of the forklift by automatically generating maintenance requests when the temperature and wear data of key components are exceeded or the maintenance threshold is reached through real-time analysis. This method of imitating biological rhythms and maintenance mechanisms not only optimizes the work scheduling of the forklift, but also reduces maintenance costs and improves the overall operating efficiency of the forklift.

[0027] Other advantages, objectives and features of the present invention will be set forth in part in the following description and, in part, will be apparent to those skilled in the art based on an examination of the following or may be taught from the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0029] Figure 1 Flowchart of a forklift path planning and obstacle avoidance assistance method based on reinforcement learning. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] Embodiment 1:

[0032] Forklift operation in indoor logistics warehouse

[0033] Scenario description: A large indoor logistics warehouse with an area of ​​about 10,000 square meters has a complex layout and many shelves for storing goods. There are narrow passages between the shelves. Forklifts are responsible for transporting goods in the warehouse and need to frequently shuttle between different shelf areas while avoiding collisions with shelves, other forklifts, and staff walking in the passages. The GPS signal in some areas of the warehouse is weak, and the forklift needs to rely on its own sensors for positioning and navigation.

[0034] Sensor deployment and calibration: Multi-view visual sensors are installed at key locations on the front, rear, and both sides of the forklift. Geomagnetic sensors, sun angle sensors, temperature sensors, and wear monitoring sensors are also installed. Various sensors are calibrated in a relatively open area with stable signals in the warehouse to ensure the accuracy of the data collected by the sensors.

[0035] Data collection and fusion: When a forklift is running in the warehouse, various sensors continue to work. The visual sensor collects images from different perspectives. These images are aligned and fused by imitating the insect compound eye vision model. Suppose there are n visual sensors, and the image collected by the i-th sensor is I i (x, y), where x, y are image coordinates, and the fused image is I fused (x,y), the calculation formula is: where w i is the weight of the i-th sensor image, which is obtained according to the sensor's viewing angle distribution. The fused image is subjected to wavelet transform to obtain wavelet coefficients W of different scales and directions. s,d (x,y), the formula is W s,d (x,y)=∫∫I fused (x,y)ψ s,d (xx′,yy′)dx′dy′, where s is the scale parameter, d is the direction parameter, ψ s,d is a wavelet function, by calculating the energy E of the wavelet coefficients s,d To extract environmental feature information, the formula is E s,d =∑ x ∑ y |W s,d (x,y)| 2 ,The data collected by the geomagnetic sensor and the sun angle sensor are filtered and calibrated, and fused with the feature information extracted by the ,vision sensor, the forklift’s own operating status data and the current time ,information to form a complete state space information vector.

[0036] Comprehensive Decision Making:

[0037] Path planning decision: When a forklift receives a task to move goods in a warehouse and the target location is a certain shelf area, if the GPS signal is good in an open area, the GPS information is combined with the environmental feature information first, and the relative direction and distance between the forklift and the target location are calculated. The potential field method is used to plan the path, assuming that the obstacle is at the position (x o ,y o ) is the repulsive potential field U rep (x,y), the calculation formula is: The gravitational potential field generated by the target is U att (x,y), the calculation formula is: The resultant force on the forklift is the negative gradient of the potential field, and the calculation formula is: The calculation formula for adjusting the movement direction v of the forklift according to the direction of the resultant force and the current speed direction is: where v max is the maximum speed of the car, F th is the threshold force. If the forklift is in an area where the GPS signal is interfered, such as a corner of a warehouse, the forklift uses geomagnetic field, solar angle, and traveled path memory to calculate the current position using the position estimation function. Let the forklift position coordinates be (x, y) and the target position coordinates be (x g ,y g ), the geomagnetic intensity vector M=(M x ,M y ,M z ), the sun angle vector S = (α, β) and the coordinates of the memory point of the traveled path (x k ,y k ) to calculate the current position of the forklift, the position estimation function formula is: Among them, M k ,S k At the memory point (x k ,y k ) recorded at the geomagnetic and solar angle information, d((x,y),(x k ,y k )) is the Euclidean distance between the current position and the memory point, Z is the normalization constant, and σ is the parameter that controls the estimation accuracy. Combined with the distribution of obstacles in the environment, an optimal path is planned.

[0038] Obstacle avoidance decision: During the forklift's driving process, the sensor continuously monitors the surrounding environment. Suppose a worker is detected suddenly walking out from behind the shelf and into the forklift's driving path. Let its speed be v j , direction is θ j , and the speed of the car is v f , direction is θ f , the distance between the forklift and the obstacle The angle θ of the forklift pointing to the obstaclefj , calculate the threat level T of the obstacle to the forklift j , the formula is: Assume that n obs obstacles, and considering the threats of all obstacles, the formula for the overall danger index D is: When D>D th (D th is the danger threshold) and needs to be avoided. Avoidance direction: θ avoid The calculation formula is: Adjust speed v new The calculation formula is: new =v f (1-kD) (where k is the speed adjustment factor).

[0039] Work-rest and maintenance decision: Arrange forklift work according to cargo flow and component status. Let time be t, cargo flow function be F(t), and calculate work suitability: When S(t) low (S low is the low flow threshold), judge that the forklift is in idle state, and calculate the probability P of each forklift entering sleep mode sleep , the calculation formula is: (Where S mid is the middle threshold, σ is the parameter controlling the slope of the curve), when P sleep >P th (P th is the decision threshold), the forklift enters sleep mode, and at the same time, the temperature and wear data of key components are analyzed in real time. Suppose the temperature of key component i of the forklift is T i , the wear amount is W i , the normal temperature range is [T i,min ,T i,max ], the wear threshold is W i,th , for each component i, calculate the maintenance requirement index M i The formula is: When M i >W i,th Issue maintenance request and follow M i The value of the maintenance is used to determine the urgency of the maintenance, and detailed maintenance information is displayed on the display screen or the maintenance request is sent to the maintenance management system through the wireless communication module.

[0040] ​Action execution and feedback: Based on the generated path planning decision, the forklift's control system sends corresponding action instructions to the steering motor and drive motor, accurately controls the forklift's driving direction and speed, and enables the forklift to travel in the warehouse according to the planned path and accurately reach the target shelf area to transport goods. When the obstacle avoidance decision is executed, once a danger is detected, the emergency obstacle avoidance action instructions are quickly executed, such as immediately slowing down and turning to a safe direction. For sleep or wake-up operations, the power switch and startup sequence of related equipment are accurately controlled. When a maintenance request is received, the maintenance personnel will maintain the forklift in a timely manner according to the information on the display or the notification of the maintenance management system.

[0041] Model optimization: Based on the behavior results and status feedback of the forklift in the warehouse, such as whether it successfully avoids obstacles, whether it completes the cargo handling task on time, and whether it enters the sleep and wake-up states accurately, the updated Q-value reward function is used to give rewards or penalties. Let the state vector be The action vector is The reward is r, and the Q value function is The learning rate is α, the discount factor is γ, and the calculation formula is: in Is to perform an action The next state after the forklift is determined, and the parameters of the model are regularly fine-tuned according to the operation of the forklift in different environments.

[0042] In summary, in indoor logistics warehouses, forklifts collect data by installing and calibrating a variety of sensors, and process visual data using an insect-like compound eye vision model and wavelet transform. Based on fused information, path planning decisions use a migratory bird navigation ability model combined with a potential field method to plan the optimal path according to environmental characteristics and target locations. Obstacle avoidance decisions use a biological stress response mechanism model to assess obstacle threats in real time and adjust the driving direction and speed in a timely manner. Work-rest and maintenance decisions are based on biological circadian rhythms and rest mechanism models. The work mode is reasonably arranged according to cargo flow and component status, energy consumption is reduced, and maintenance is performed in a timely manner. In the action execution and feedback link, the forklift accurately executes various instructions to ensure the smooth progress of the operation. Finally, through model optimization, the model parameters are continuously adjusted according to the forklift behavior results and status feedback, which improves the operation efficiency and safety of the forklift in the indoor logistics warehouse environment.

[0043] Embodiment 2:

[0044] Forklift operation at port container terminal

[0045] Scenario description: A certain port container terminal has a large area and multiple container stacking areas. Forklifts are mainly responsible for moving containers between the yard and transport vehicles. The terminal environment is complex, with a large number of metal containers, which interfere with GPS signals. At the same time, there are other large equipment and many staff in the forklift operation area. Forklifts need to operate efficiently and safely in a busy environment and adapt to different weather and time conditions.

[0046] Sensor deployment and calibration: Multi-view visual sensors are installed on the top, front, back, left and right key parts of the forklift. At the same time, geomagnetic sensors, solar angle sensors, temperature sensors and wear monitoring sensors are installed. Sensor calibration is carried out in an open area of ​​the dock. For geomagnetic sensors, they are compared and adjusted with known standard values ​​of geomagnetic intensity to ensure that the geomagnetic sensors can accurately measure the local geomagnetic intensity.

[0047] Data collection and fusion When the forklift is running, the visual sensor continuously collects image data, and the image is fused by imitating the insect compound eye vision model. Suppose there are n visual sensors, and the image collected by the i-th sensor is I i (x,y), the fused image is I fused (x,y), the calculation formula is: Perform wavelet transform on the fused image to obtain the wavelet coefficient W s,d (x,y), the calculation formula is: By calculating the energy E of the wavelet coefficients s,d To extract features, the formula is: E s,d =∑ x ∑ y |W s,d (x,y)| 2 .

[0048] Comprehensive Decision Making:

[0049] Path planning decision: When a forklift receives a task to move a container from a storage yard to a designated transport vehicle, if the GPS signal is good, the path is planned by combining GPS and environmental characteristics, using the potential field method, taking into account obstacles at positions (x o ,y o ) is the repulsive potential field U rep (x,y), the calculation formula is: The gravitational potential field generated by the target is U att (x,y), the calculation formula is: The resultant force on the forklift is: Adjust the forklift's movement direction according to the combined force. The calculation formula is: If the GPS signal is not good, the current position is calculated by using the position estimation function based on geomagnetism, solar angle and the memory of the traveled path. Suppose the position coordinates of the forklift are (x, y) and the target position coordinates are (x g ,y g ), the geomagnetic intensity vector is The sun angle vector is α is the solar altitude angle, β is the solar azimuth angle, and the coordinates of the memory point on the traveled path are m is the number of memory points, and the position estimation function is constructed using geomagnetic and solar angle information. The formula is: in At the memory point (x k ,y k ) recorded at the geomagnetic and solar angle information, d((x,y),(x k ,y k )) is the Euclidean distance between the current position and the memory point, Z is the normalization constant, and σ is a parameter that controls the estimation accuracy.

[0050] Obstacle avoidance decision: During the operation, the sensor monitors in real time. Suppose a crane boom is detected to move suddenly into the forklift path, and its speed is set to v j , direction is θ j , and the speed of the car is v f , direction is θ f , the distance between the forklift and the obstacle The direction angle θ of the forklift pointing to the obstacle fj , calculate the threat level T of the obstacle to the forklift j , the formula is: If n is detected at the same time obs Other obstacles, the overall hazard index is calculated as follows: (ω j is the weight determined by the obstacle type and size), when D>D th (D th is the danger threshold), calculate the avoidance direction Adjust speed v new =v f (1-kD), so that the forklift can avoid obstacles in time.

[0051] Work-rest and maintenance decision: Arrange work according to cargo flow and component status. Assume that time is t and cargo flow function is F(t). Calculate work suitability using the formula: When S(t) low (S low ​is the low flow threshold), judge that the forklift is in idle state, and calculate the probability P of each forklift entering sleep mode sleep , the calculation formula is: (Where S mid is the middle threshold, σ is the parameter controlling the slope of the curve), when P sleep >P th (P th is the decision threshold), the forklift enters the sleep mode, and the temperature of the key component i of the forklift is T i , the wear amount is W i , the normal temperature range is [T i,min ,T i,max ], the wear threshold is W i,th , for each component i, calculate the maintenance requirement index M i The formula is: When M i >M th (M th is the maintenance threshold), a maintenance request is issued and detailed information is displayed, such as "hydraulic system components require maintenance, the temperature is high, and the wear is close to the threshold", and a request is sent to the maintenance system at the same time.

[0052] Action execution and feedback: Under the path planning decision, the control system controls the forklift motor to drive along the planned path, quickly executes action instructions when avoiding obstacles, accurately controls the power supply and startup sequence of the equipment when sleeping or waking up, and after the maintenance request is issued, the maintenance personnel will perform maintenance in time based on the information.

[0053] Model optimization: Based on the behavior and state feedback of the forklift in the terminal, such as the efficiency of completing the task and the success rate of obstacle avoidance, rewards or penalties are given according to the reward function. Let the state vector be The action vector is The reward is r, and the Q value function is The learning rate is α, the discount factor is γ, and the calculation formula is: in Is to perform an action The model parameters are fine-tuned according to the operation status under different cargo stacking layouts and weather conditions to improve the operating performance of forklifts at port terminals.

[0054] In summary, at the port container terminal, the forklift is equipped with a variety of sensors and calibrated. During data collection and fusion, the visual image is processed and combined with other sensor data. The position estimation function is used to achieve accurate positioning when the GPS signal is interfered. In terms of comprehensive decision-making, path planning uses a variety of information based on different GPS signals, obstacle avoidance can respond in real time, work-rest and maintenance are arranged according to cargo flow and component status, and the action is executed accurately. After model optimization, the forklift can operate efficiently and safely in complex environments, and improve performance to adapt to various operating conditions and tasks.

[0055] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A forklift path planning obstacle avoidance assistance method based on reinforcement learning, characterized in that: The specific steps of this auxiliary method are: S100, sensor deployment and calibration: install multi-view visual sensors, geomagnetic sensors, sun angle sensors, temperature sensors and wear monitoring sensors on forklifts, and calibrate various sensors; S200, data collection and fusion: During the operation of the forklift, various sensors continuously collect data, align and fuse images from different perspectives through the insect compound eye vision model, extract feature information from the environment, filter and calibrate the collected geomagnetic and solar angle information, and fuse the pre-processed data with the forklift's operating status data and current time information to form a complete state space information vector; S300, comprehensive decision generation: including path planning decision, path planning decision, and work-rest and maintenance decision; The path planning decision is based on the current state space information, and the path planning model of the migratory bird navigation capability is used to plan the path. When the GPS signal is good, the GPS information and environmental feature information are used first. When the GPS signal is interfered, the geomagnetic field, the sun angle and the traveled path memory are used, and the position estimation function is used to calculate the current position to plan the optimal path. The path planning decision utilizes an obstacle avoidance strategy based on an obstacle avoidance reaction model of a biological stress response mechanism, continuously monitors the environment around the forklift, and when the sensor detects a sudden, fast-approaching obstacle, quickly evaluates the type, speed, distance, and direction of movement of the obstacle, and calculates the avoidance direction and speed adjustment method in combination with the current speed and direction of the forklift; The work-rest and maintenance decision is based on a work arrangement and self-repair model that mimics biological circadian rhythms and rest mechanisms, and determines the work mode according to time and cargo flow forecast data. At night, during off-peak hours, and when there are no urgent tasks, it is determined whether to sleep based on power and key component status. When sleeping, some non-critical sensors and equipment are turned off to reduce power consumption. When approaching peak hours, it automatically wakes up according to the wake-up conditions and starts the preheating and self-test procedures. The temperature and wear data of key components are analyzed in real time. When exceeding the limit or reaching the maintenance threshold, a maintenance request is generated according to the self-repair mechanism to notify the maintenance personnel; S400, action execution and feedback: Based on the generated path planning, obstacle avoidance, work-rest and maintenance decisions, the forklift control system sends corresponding action instructions to the actuator. While the forklift executes the action, the sensor continuously monitors the actual operating status of the forklift and feeds back the status information to the control system; S500, model optimization: Based on the behavior results and status feedback of the forklift, rewards or penalties are given according to the reward function, and the parameters of the model are regularly fine-tuned according to the operation of the forklift in different environments.

2. The forklift path planning obstacle avoidance assistance method based on reinforcement learning according to claim 1 is characterized in that: In the data collection and fusion, images from different perspectives are aligned and fused by simulating an insect compound eye vision model. Assume that there are n visual sensors, and the image collected by the i-th sensor is I i (x, y), where x, y are image coordinates, and the fused image is I fused (x,y), the calculation formula is: where w i is the weight of the i-th sensor image.

3. The forklift path planning obstacle avoidance assistance method based on reinforcement learning according to claim 1, characterized in that: In the step S200, feature information of the environment is extracted by wavelet transform in data collection and fusion, and the fused image I fused (x, y) is transformed by wavelet to obtain wavelet coefficients W of different scales and directions s,d (x,y), the calculation formula is: Where s is the scale parameter, d is the direction parameter, ψ s,d is a wavelet function, by calculating the energy E of the wavelet coefficients s,d To extract features, the formula is: E s,d =∑ x ∑ y |W s,d (x,y)| 2 .

4. The forklift path planning obstacle avoidance assistance method based on reinforcement learning according to claim 1, characterized in that: In the above S300, the potential field method is used to plan the path according to the current position in the comprehensive decision generation, assuming that the obstacle is at the position (x o ,y o ) is the repulsive potential field U rep (x,y), the calculation formula is: Where η is the repulsion coefficient, o is the gravitational potential field generated by the target after traversing all detected obstacles, and U att (x,y), the calculation formula is: Where ξ is the gravitational coefficient, the resultant force on the forklift is the negative gradient of the potential field, and the calculation formula is: The direction of movement of the forklift according to the direction of the resultant force and the current speed The adjustment calculation formula is: where v max is the maximum speed of the forklift, F th is the threshold force.

5. The forklift path planning obstacle avoidance assistance method based on reinforcement learning according to claim 1, characterized in that: In the above S300, the current position is calculated by using the position estimation function in the comprehensive decision generation, and the position coordinates of the forklift are set to (x, y) and the target position coordinates are set to (x g ,y g ), the geomagnetic intensity vector is The sun angle vector is α is the solar altitude angle, β is the solar azimuth angle, and the coordinates of the memory point on the traveled path are m is the number of memory points, and the position estimation function is constructed using geomagnetic and solar angle information. The formula is: in At the memory point (x k ,y k ) recorded at the geomagnetic and solar angle information, d((x,y),(x k ,y k )) is the Euclidean distance between the current position and the memory point, Z is the normalization constant, and σ is a parameter that controls the estimation accuracy.

6. The forklift path planning obstacle avoidance assistance method based on reinforcement learning according to claim 1, characterized in that: In the S300, in the comprehensive decision making, the potential danger is found by evaluating the threat level of obstacles in real time. Assuming the speed of the forklift is v f , direction is θ f , the position of obstacle j is (x j ,y j ), with a velocity of v j , direction is θ j , the number of obstacles detected is n obs , calculate the threat level T of each obstacle to the forklift j , the formula is: in is the distance between the forklift and the obstacle, θ fj is the direction angle of the forklift pointing to the obstacle. Considering the threats of all obstacles, the formula for the overall danger index D is: where ω j is the weight determined by the obstacle type and size. th When you need to avoid, D th is the danger threshold, the avoidance direction θ avoid The calculation formula is: Adjust speed v new The calculation formula is: new =v f (1-kD), where k is the speed adjustment factor.

7. The forklift path planning obstacle avoidance assistance method based on reinforcement learning according to claim 1, characterized in that: In the above S300, the forklift work is arranged according to the cargo flow and the component status by the work-rest decision formula in the comprehensive decision generation. Assuming the time is t, the cargo flow function is F(t), and the work suitability S(t) is calculated by the formula: When S(t) low Determine that the forklift is in idle state, S low is the low traffic threshold, and the probability P of each forklift entering sleep mode is calculated. sleep , the formula is: Where S mid is the middle threshold, σ is the parameter that controls the slope of the curve. sleep >P th The forklift enters sleep mode, P th is the decision threshold.​ 8. The forklift path planning obstacle avoidance assistance method based on reinforcement learning according to claim 1, characterized in that: In the above S300, the maintenance plan of the forklift is arranged by calculating the maintenance demand index in the comprehensive decision generation. Assuming that the temperature of the key component i of the forklift is T i , the wear amount is W i , the normal temperature range is [T i,min ,T i,max ], the wear threshold is W i,th , for each component i, calculate the maintenance requirement index M i The formula is: M i >W i,th Issue maintenance request and follow M i The value of determines the urgency of maintenance.

9. The forklift path planning obstacle avoidance assistance method based on reinforcement learning according to claim 1, characterized in that: In the S400, the path planning decision in the action execution and feedback is to adjust the driving direction and speed of the forklift by controlling the steering motor and the drive motor of the forklift so that the forklift can travel along the planned path. For obstacle avoidance operations, the emergency obstacle avoidance action instructions are quickly executed. For sleep or wake-up operations, the power switches and startup sequences of related equipment are accurately controlled. For maintenance requests, detailed maintenance information is displayed on the display screen and the maintenance request is sent to the maintenance management system.

10. The forklift path planning obstacle avoidance assistance method based on reinforcement learning according to claim 1, characterized in that: In S500, the updated Q-value reward function is used to give rewards or penalties in the model optimization. Let the state vector be The action vector is The reward is r, and the Q value function is The learning rate is α, the discount factor is γ, and the calculation formula is: in Is to perform an action The next state after .

Images