A new energy engineering vehicle unmanned path planning method and system

Abstract

The application provides a new energy engineering vehicle unmanned driving path planning method and system, and belongs to the technical field of unmanned driving. First, the initial state and the current task list of the new energy engineering vehicle are acquired, the emergency degree is valued in the order from high to low according to the emergency degree of each task in the current task list, and then the initial state and the current task list of the new energy engineering vehicle are subjected to reinforcement learning based on an improved reinforcement learning algorithm. Finally, the unmanned driving planning path of the new energy engineering vehicle is acquired based on the reinforcement learning result. The application comprehensively considers multiple path planning dimensions such as the position, real-time power state and task emergency degree of the new energy engineering vehicle, improves the adaptability and scene adaptation ability of the planning strategy, and improves the practicability of the system. The search rate in the reinforcement learning is dynamically adjusted to balance the searchability and learnability of the algorithm, avoid falling into a local optimal solution, improve the convergence speed of the algorithm, and improve the efficiency of path planning.

Description

Technical Field

[0001] This invention belongs to the field of autonomous driving technology, and in particular relates to an autonomous driving path planning method and system for new energy engineering vehicles. Background Technology

[0002] In core operational scenarios for new energy engineering vehicles, such as mines, ports, and urban infrastructure construction sites, autonomous driving path planning technology not only addresses the pain points of traditional manual driving and singular path planning, but also achieves qualitative improvements in multiple dimensions, including operational efficiency, operating costs, safety control, and intelligent management, becoming a core driving force for the intelligent upgrading of the engineering field. However, the current mainstream autonomous driving path planning technology system still suffers from a significant lack of optimization dimensions, failing to systematically and comprehensively consider key optimization objectives such as task urgency, real-time vehicle battery status, and vehicle location. Specifically, both traditional graph search-based path planning algorithms and some lightweight intelligent planning schemes often focus solely on optimizing single metrics such as shortest travel distance or fastest arrival time. They fail to assign differentiated planning weights based on the urgency level of different tasks (e.g., expedited cargo transfer, emergency repairs in mining areas), nor do they dynamically adjust path strategies based on the battery SOC value and energy consumption characteristics of new energy engineering vehicles. Furthermore, they fail to incorporate the coupling relationship between driving efficiency, power consumption, and task timeliness into the planning model. This leads to problems such as path congestion and delays, breakdowns due to insufficient power, or excessive power consumption in pursuit of efficiency affecting subsequent operations when vehicles are performing high-priority tasks. These issues severely restrict the operational efficiency and intelligence level of unmanned driving systems for new energy engineering vehicles in complex engineering scenarios. In addition, existing unmanned driving path planning methods still face problems such as excessive iterations, susceptibility to local optima traps, and convergence difficulties when using reinforcement learning, affecting the efficiency of path planning. Summary of the Invention

[0003] To address the aforementioned problems in the existing technology, this invention proposes a path planning method and system for unmanned driving of new energy engineering vehicles, which improves the efficiency of path planning and thus enhances the intelligence level of the unmanned driving system for new energy engineering vehicles.

[0004] To achieve the above objectives, the present invention adopts the following technical solution: a method for unmanned driving path planning of new energy engineering vehicles, comprising the following steps: S1: obtaining the initial state and current task list of the new energy engineering vehicle, wherein the initial state of the new energy engineering vehicle includes the vehicle's location and current battery level, and the current task list includes the number of tasks, urgency level, task distance, and estimated completion time; S2: assigning urgency levels to the tasks in the current task list in descending order of urgency; S3: performing reinforcement learning on the initial state and current task list of the new energy engineering vehicle based on an improved reinforcement learning algorithm to obtain reinforcement learning results; S4: obtaining the unmanned driving planned path of the new energy engineering vehicle based on the reinforcement learning results in S3.

[0005] Furthermore, in S3, reinforcement learning is performed on the initial state and current task list of the new energy engineering vehicle based on an improved reinforcement learning algorithm, specifically including: S31: Initialize the learning rate, discount factor, and search rate of the improved reinforcement learning algorithm; initialize the Q-value table according to the number of tasks, and preset the maximum number of learning iterations; S32: Construct an outer loop with the preset maximum number of learning iterations as the maximum value for iteration, reset the cumulative reward before the start of each iteration, and create a task execution order list in the current task list based on the urgency level. S33: Construct an inner loop with a step size of step as the maximum number of tasks. For each step in the inner loop, take the task corresponding to the current step size as the current task according to the task execution order list created in S32. S34: Use a greedy strategy to select the next task index; S35: Calculate the real-time reward based on the current task and update the Q-value table; S36: After the outer loop iteration is completed, extract the optimal policy from the updated Q-value table; and use the optimal policy as the reinforcement learning result.

[0006] Furthermore, in step S34, a greedy strategy is used to select the next task index, specifically including: S341: Generate a random number less than 1 and greater than 0; S342: Determine the size of the random number and the search rate. When the random number is less than the search rate, randomly select a task from the task list as the next task and record the index as the next task index. S343: When the random number is greater than or equal to the search rate, select the task with the highest known Q value as the next task and record the index as the next task index; S344: Calculate the ratio of the current search rate to the preset maximum number of learning iterations, then multiply the ratio by the first coefficient to obtain the first ratio, and update the search rate to be the absolute difference between the current search rate and the first ratio.

[0007] Furthermore, the calculation of real-time rewards based on the current task in S35 specifically includes: S351: Calculate the difference between the x-coordinate of the current task position and the x-coordinate of the new energy engineering vehicle position; calculate the difference between the y-coordinate of the current task position and the y-coordinate of the new energy engineering vehicle position. S352: Calculate the Euclidean distance between the current task position and the position of the new energy engineering vehicle based on the difference between the x-coordinate of the current task position and the x-coordinate of the new energy engineering vehicle position and the difference between the y-coordinate of the current task position and the y-coordinate of the new energy engineering vehicle position. S353: Calculate the difference between the Euclidean distance and the preset Euclidean distance threshold as the first difference; calculate the product of the preset Euclidean distance threshold and the adjustment coefficient as the first product value; construct an exponential function by using the ratio of the first difference to the first product value as a power of the natural number e; sum the exponential function with 1 and take the reciprocal as the standardized Euclidean distance; S354: Calculate the product of the current task's urgency assignment and the preset urgency weight, and record it as the first reward product; calculate the product of the current task's standardized Euclidean distance and the preset standardized Euclidean distance weight, and record it as the second reward product; calculate the product of the new energy engineering vehicle's current battery level and the preset current battery level weight, and record it as the third reward product. S355: Calculate the sum of the first reward product, the second reward product, and the third reward product, and use the sum of the three as the real-time reward.

[0008] Further, the Q-value table update specifically includes: reading the values ​​in the current Q-value table with the current step size and the next task index as coordinates and recording them as the first intermediate value; reading the maximum value of the row containing the next task index in the current Q-value table and calculating the product of the maximum value and the discount factor to obtain the second intermediate value; calculating the sum of the real-time reward and the second intermediate value and then subtracting the first intermediate value to obtain the third intermediate value; adding the product of the third intermediate value and the learning rate to the first intermediate value, updating the first intermediate value, and using it as the basis for the update of the next inner loop.

[0009] Furthermore, in step S4, based on the reinforcement learning results in step S3, an unmanned driving planning path for the new energy engineering vehicle is obtained, specifically including: S41: Obtain the task execution order corresponding to the reinforcement learning results in S3; according to the task execution order, obtain the position of the first task, the real-time position of the new energy engineering vehicle, and the real-time battery level of the new energy engineering vehicle; S42: Generate a first autonomous driving path based on the engineering environment, and add the first intermediate waypoint to the first autonomous driving path; S43: Calculate the distance between the real-time location of the new energy engineering vehicle and the first path point and record it as the first distance. Multiply the first distance by the power consumption coefficient and the road condition coefficient, and take the absolute value of the difference between the first distance and the real-time power of the new energy engineering vehicle to obtain the current power of the new energy engineering vehicle. S44: Determine whether the current battery level of the new energy engineering vehicle is less than or equal to the battery threshold. If the current battery level of the new energy engineering vehicle is less than or equal to the battery threshold, terminate the first task and execute the charging station navigation strategy. If the current battery level of the new energy engineering vehicle is greater than the battery threshold, add a second intermediate waypoint on the first autonomous driving path and repeat the above S43-S44. S45: When all intermediate path points have been added and the battery level of the new energy engineering vehicle is still greater than the battery threshold, determine the first autonomous driving path as the autonomous driving planning path of the new energy engineering vehicle; otherwise, regenerate the second autonomous driving path and repeat the above S42-S44.

[0010] This invention also proposes an unmanned driving path planning system for new energy engineering vehicles, used to execute the aforementioned unmanned driving path planning method for new energy engineering vehicles. The system includes a data acquisition module, a data processing module, and a result output module. The data acquisition module is connected to the data processing module, and the data processing module is connected to the result output module. The data acquisition module acquires the initial state and current task list of the new energy engineering vehicle and inputs them into the data processing module. Based on the initial state and current task list of the new energy engineering vehicle, the data processing module generates an unmanned driving planned path for the new energy engineering vehicle and inputs it into the result output module. The result output module outputs the unmanned driving planned path of the new energy engineering vehicle.

[0011] The beneficial technical effects of this invention compared with the prior art are as follows: (1) By comprehensively considering multiple path planning dimensions such as the location of new energy engineering vehicles, real-time power status and task urgency, the adaptability of planning strategies and scenario adaptability can be improved, and emergency tasks can be responded to quickly to reduce production losses and improve system practicality. (2) Dynamically adjust the search rate in reinforcement learning to balance the search and learning properties of the algorithm, avoid getting stuck in local optima, improve the convergence speed of the algorithm, and enhance the efficiency of path planning. Attached Figure Description

[0012] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0013] Figure 1 This is a flowchart of the unmanned driving path planning method for new energy engineering vehicles in this invention; Figure 2 This is a simplified structural diagram of the unmanned driving path planning system for new energy engineering vehicles in this invention. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] Example Combination Figure 1 This invention proposes a method for unmanned driving path planning for new energy engineering vehicles, comprising the following steps: S1: Obtaining the initial state of the new energy engineering vehicle through a monitoring system composed of a station terminal, CAN bus, and remote monitoring platform; obtaining the current task list of the new energy engineering vehicle through a remote service platform and vehicle-to-everything (V2X) architecture. The initial state of the new energy engineering vehicle includes information such as its location and current battery level; the current task list includes information such as the number of tasks, urgency level, task distance, and estimated completion time. S2: Assigning urgency values ​​to tasks in the current task list in descending order of urgency. It is understood that tasks with higher urgency are assigned higher values. By assigning values ​​based on urgency before reinforcement learning iteration, high-urgency tasks are prioritized and executed during engineering scheduling, thus reducing the workload of multi-task scheduling scenarios for engineering vehicles. S3: Performing reinforcement learning on the initial state and current task list of the new energy engineering vehicle using an improved reinforcement learning algorithm to obtain reinforcement learning results. S4: Obtaining the unmanned driving planning path for the new energy engineering vehicle based on the reinforcement learning results in S3.

[0016] S3 involves performing reinforcement learning on the initial state and current task list of the new energy engineering vehicle based on an improved reinforcement learning algorithm. Specifically, this includes: S31: Initializing the learning rate, discount factor, and search rate of the improved reinforcement learning algorithm; initializing the Q-value table according to the number of tasks, and presetting a maximum number of learning iterations. This maximum number of learning iterations can adjust the number of learning rounds to ensure the learning level; S32: Constructing an outer loop with the preset maximum number of learning iterations as the maximum value for iteration, resetting the cumulative reward before each iteration; and creating a task execution order list in the current task list based on the urgency level; S33: Using the task... An inner loop with a step size of `step` is constructed based on the maximum number of tasks (the inner loop is constructed to simulate the execution process of a single task). For each step in the inner loop, the task corresponding to the current step size is taken as the current task according to the task execution order list created in S32; S34: a greedy strategy is used to select the index of the next task; S35: a real-time reward is calculated based on the current task, and the Q-value table is updated. Updating the Q-value table allows the Q-value to gradually approach the optimal true value; S36: after the outer loop iteration is completed, the optimal policy is extracted from the updated Q-value table; and the optimal policy is used as the reinforcement learning result. The extraction of the optimal policy can be achieved by finding the column containing the maximum value in each row of the Q-value table, and then storing the index corresponding to the column containing the maximum value in the optimal policy array as the optimal policy.

[0017] The S34 section uses a greedy strategy to select the next task index, specifically including: S341: Generate a random number less than 1 and greater than 0; S342: Determine the magnitude of the random number and the search rate. When the random number is less than the search rate, randomly select a task from the task list as the next task and record its index as the next task index; S343: When the random number is greater than or equal to the search rate, select the task with the highest known Q value as the next task and record its index as the next task index; S344: Calculate the ratio of the current search rate to the preset maximum number of learning iterations, multiply the ratio by a first coefficient to obtain the first ratio, and update the search rate as the absolute difference between the current search rate and the first ratio. By generating the random number, the search and learning effects can be balanced; the dynamic decay of the search rate ensures a smooth decline, avoiding drastic fluctuations and contributing to stable convergence of the learning process; the adjustment of the first coefficient precisely controls the decay rate and magnitude of the search rate throughout the training process. Setting the first coefficient ensures that the reinforcement learning algorithm can adapt to tasks of different complexities, improving the adaptability of the planning strategy and the scenario adaptability.

[0018] It's important to note that traditional greedy strategies in reinforcement learning typically employ a fixed search rate or simple linear decay to balance search and exploitation. If the search rate is set too high, the algorithm will maintain a high probability of random selection for an extended period, making it difficult to stably learn the optimal strategy, resulting in slow convergence and significant policy fluctuations. Conversely, if the search rate is set too low or decreased too early, the algorithm will over-rely on previously learned locally optimal behaviors in the early stages of reinforcement learning, easily getting trapped in local optima. This invention improves upon the application of traditional greedy strategies in reinforcement learning by dynamically and adaptively decaying the search rate. This ensures that the algorithm maintains a high search rate in the early stages of reinforcement learning, expanding the policy search range and avoiding erroneous local optima. As the number of reinforcement learning iterations increases, the search rate gradually and smoothly decreases according to a set rule, and the algorithm gradually shifts from a search mode to an exploitation mode, steadily converging to the optimal strategy while maintaining global search capability. Through this dynamic adjustment mechanism, this invention achieves a more reasonable and stable balance between sufficient search and efficient exploitation, avoiding convergence difficulties caused by over-search and effectively preventing local optima problems caused by over-exploitation, thereby significantly improving the algorithm's ability to obtain globally optimal paths and task scheduling strategies.S35, which calculates the real-time reward based on the current task, specifically includes: S351: Calculating the difference between the x-coordinate of the current task position and the x-coordinate of the new energy engineering vehicle position; calculating the difference between the y-coordinate of the current task position and the y-coordinate of the new energy engineering vehicle position; S352: Calculating the Euclidean distance between the current task position and the new energy engineering vehicle position based on the differences calculated in steps S351 and S352, thereby quantifying the spatial movement cost; S353: Calculating the difference between the Euclidean distance and a preset Euclidean distance threshold as a first difference; calculating the product of the preset Euclidean distance threshold and an adjustment coefficient as a first product value; constructing an exponential function by using the ratio of the first difference to the first product value as a power of the natural number e; summing the exponential function with 1 and taking the reciprocal as the standardized Euclidean distance; the standardized Euclidean distance is a nonlinear mapping of the Euclidean distance in S352, which can enhance the algorithm's sensitivity control to distance changes. Simultaneously, the adjustment coefficient can flexibly adjust the steepness of the mapping relationship, adapting to differences in distance sensitivity under different scenarios. S354: Calculate the product of the current task's urgency assignment and the preset urgency weight, denoted as the first reward product. The first reward product is used to quantify the task's urgency into a reward signal, guiding engineering vehicles to prioritize high-urgency tasks. Calculate the product of the current task's standardized Euclidean distance and the preset standardized Euclidean distance weight, denoted as the second reward product. The second reward product normalizes the driving distance into a reward signal, guiding engineering vehicles to choose shorter paths and reduce energy consumption. Calculate the product of the new energy engineering vehicle's current battery level and the preset current battery level weight, denoted as the third reward product. The third reward product normalizes the vehicle's battery status into a reward signal, guiding engineering vehicles to achieve more energy-efficient path planning, ensuring range and task continuity. S355: Calculate the sum of the first reward product, the second reward product, and the third reward product. Use the sum of the three as a real-time reward to form a multi-dimensional comprehensive reward, enabling the task order and path planning results output by reinforcement learning to prioritize more urgent, shorter, and more energy-efficient tasks. The above steps comprehensively consider multiple path planning dimensions such as the location of new energy engineering vehicles, real-time power status, and task urgency. This can improve the adaptability of the planning strategy and the scenario adaptability, quickly respond to urgent tasks, reduce production losses, and enhance system practicality.

[0019] In this invention, updating the Q-value table specifically includes: reading the values ​​in the current Q-value table with the current step size and the next task index as coordinates, and recording them as the first intermediate value; reading the maximum value of the row containing the next task index in the current Q-value table and calculating the product of the maximum value and the discount factor to obtain the second intermediate value; calculating the sum of the real-time reward and the second intermediate value, and then subtracting the first intermediate value to obtain the third intermediate value; adding the product of the third intermediate value and the learning rate to the first intermediate value, updating the first intermediate value, and using it as the basis for updating in the next inner loop. The above-mentioned learning rate setting can avoid drastic fluctuations in the Q-value, ensuring that the algorithm smoothly converges to the optimal policy.

[0020] Further, in step S4, based on the reinforcement learning results in step S3, an unmanned driving planning path for the new energy engineering vehicle is obtained. Specifically, this includes: S41: obtaining the task execution order corresponding to the reinforcement learning results in step S3; according to the task execution order, obtaining the position of the first task, the real-time position of the new energy engineering vehicle, and the real-time battery level of the new energy engineering vehicle; S42: generating a first unmanned driving path based on the engineering environment, and adding a first intermediate path point to the first unmanned driving path; by adding the intermediate path point, segmented battery level and segmented position monitoring of the new energy engineering vehicle are achieved, ensuring the smoothness of task execution when the vehicle status and environment change; S43: calculating the distance between the real-time position of the new energy engineering vehicle and the first path point and recording it as the first distance; multiplying the first distance by the power consumption coefficient (flexibly set according to different battery types) and the road condition coefficient (reflecting the road conditions of the working scene where the new energy engineering vehicle is located), and taking the absolute value of the difference between the first distance and the real-time battery level of the new energy engineering vehicle, to obtain the current battery level of the new energy engineering vehicle. Here, the power consumption coefficient can be considered as the working environment. The first distance is the basic power consumption per unit mileage of the vehicle, while the road condition coefficient is used to correct power consumption for special road conditions such as road slope, steepness, and obstacle detour. By multiplying the first distance, the power consumption coefficient, and the road condition coefficient, the comprehensive power consumption of the engineering vehicle traveling the first distance is reflected; S44: Determine whether the current power of the new energy engineering vehicle is less than or equal to the power threshold. If the current power of the new energy engineering vehicle is less than or equal to the power threshold, terminate the first task and execute the charging station navigation strategy (the above charging station navigation strategy is built into the new energy engineering vehicle and is used to navigate the new energy engineering vehicle to the nearest charging pile or charging station); If the current power of the new energy engineering vehicle is greater than the power threshold, add a second intermediate path point on the first unmanned driving path and repeat the above S43-S44; S45: When all intermediate path points have been added and the power of the new energy engineering vehicle is still greater than the power threshold, determine that the first unmanned driving path is the unmanned driving planning path of the new energy engineering vehicle; otherwise, regenerate the second unmanned driving path and repeat the above S42-S44.

[0021] In one possible implementation, a new energy engineering vehicle is located at coordinates (0, 0) with a current battery level of 80%. There are 3 tasks in the task list, with their locations and urgency levels as follows: Task 1: Location (10,20), High urgency (assigned value 3); Task 2: Location (30,40), Urgency level: Medium (assigned value 2); Task 3: Location (50, 60), low urgency (assigned value 1); Assign values ​​according to urgency from highest to lowest: Task 1 = 3, Task 2 = 2, Task 3 = 1.

[0022] In this embodiment, the learning rate is 0.1, the discount factor is 0.9, the initial search rate is 0.8, the maximum number of learning iterations is 1000, the first coefficient is 0.001, the preset Euclidean distance threshold is 50, the adjustment coefficient is 0.5, the preset urgency weight is 0.4, the preset standardized Euclidean distance weight is 0.3, the preset current battery weight is 0.3, the power consumption coefficient is 0.1 kWh / km, the road condition coefficient is 1.2, and the battery threshold is 20%. If the current search rate is 0.8 and the current iteration count is 100, then the first ratio is 8×10-7, and the updated search rate is 0.7999992. As the iteration progresses, the search rate gradually decreases, and the algorithm shifts from searching to exploiting.

[0023] Based on the vehicle position (0,0) and the mission 1 position (10,20), the Euclidean distance is calculated to be 22.36, the first difference is -27.64, the first product is 25, and the standardized Euclidean distance is 0.75. The first reward product is 1.2, the second reward product is 0.75 × 0.3 = 0.225, and the third reward product is 80 × 0.3 = 24. Therefore, the real-time reward is 1.2 + 0.225 + 24 = 25.425.

[0024] Combination Figure 2This invention also proposes an unmanned driving path planning system for new energy engineering vehicles, employing a hierarchical processing architecture or a linear processing architecture (pipeline architecture) to execute the aforementioned unmanned driving path planning method for new energy engineering vehicles. The system includes a data acquisition module, a data processing module, and a result output module. The data acquisition module is connected to the data processing module, and the data processing module is connected to the result output module. The data acquisition module (including sensors, a task management system interface, an environmental perception unit, etc.) acquires the initial state and current task list of the new energy engineering vehicle and inputs them to the data processing module. The data processing module (including a reinforcement learning engine, a path planning unit, a power monitoring unit, an integration processing unit, etc.) generates the unmanned driving planned path for the new energy engineering vehicle based on the initial state and current task list and inputs it to the result output module. The result output module outputs the unmanned driving planned path of the new energy engineering vehicle, realizing the visualization of the planned path, navigation instructions, and emergency strategy execution. This provides the management system with intuitive operational status monitoring and backtracking basis, adapting to the complex working conditions of engineering scenarios.

[0025] The embodiments and / or implementation methods described above are merely preferred embodiments and / or implementation methods for implementing the technology of the present invention, and are not intended to limit the implementation methods of the technology of the present invention in any way. Any person skilled in the art can make some modifications or alterations to other equivalent embodiments without departing from the scope of the technical means disclosed in the content of the present invention, but they should still be regarded as the technology or embodiments that are substantially the same as the present invention.

Claims

1. A method for unmanned driving path planning for new energy engineering vehicles, characterized in that, Includes the following steps: S1: Obtain the initial status and current task list of the new energy engineering vehicle. The initial status of the new energy engineering vehicle includes the vehicle's location and current battery level. The current task list includes the number of tasks, urgency level, task distance, and estimated completion time. S2: Assign urgency values ​​to tasks in the current task list in descending order of urgency. S3: Based on the improved reinforcement learning algorithm, reinforcement learning is performed on the initial state and current task list of the new energy engineering vehicle to obtain the reinforcement learning results; S4: Based on the reinforcement learning results in S3, obtain the unmanned driving planning path for new energy engineering vehicles.

2. The method for unmanned driving path planning of new energy engineering vehicles according to claim 1, characterized in that, In step S3, reinforcement learning is performed on the initial state and current task list of the new energy engineering vehicle based on an improved reinforcement learning algorithm, specifically including: S31: Initialize the learning rate, discount factor, and search rate of the improved reinforcement learning algorithm; initialize the Q-value table according to the number of tasks, and preset the maximum number of learning iterations; S32: Construct an outer loop with the preset maximum number of learning iterations as the maximum value for iteration, reset the cumulative reward before the start of each iteration, and create a task execution order list in the current task list based on the urgency level. S33: Construct an inner loop with a step size of step as the maximum number of tasks. For each step in the inner loop, take the task corresponding to the current step size as the current task according to the task execution order list created in S32. S34: Use a greedy strategy to select the next task index; S35: Calculate the real-time reward based on the current task and update the Q-value table; S36: After the outer loop iteration is completed, extract the optimal policy from the updated Q-value table; and use the optimal policy as the reinforcement learning result.

3. The method for unmanned driving path planning of new energy engineering vehicles according to claim 2, characterized in that, The S34 section uses a greedy strategy to select the next task index, specifically including: S341: Generate a random number less than 1 and greater than 0; S342: Determine the size of the random number and the search rate. When the random number is less than the search rate, randomly select a task from the task list as the next task and record the index as the next task index. S343: When the random number is greater than or equal to the search rate, select the task with the highest known Q value as the next task and record the index as the next task index; S344: Calculate the ratio of the current search rate to the preset maximum number of learning iterations, then multiply the ratio by the first coefficient to obtain the first ratio, and update the search rate to be the absolute difference between the current search rate and the first ratio.

4. The method for unmanned driving path planning of new energy engineering vehicles according to claim 2, characterized in that, The calculation of real-time rewards based on the current task in S35 specifically includes: S351: Calculate the difference between the x-coordinate of the current task position and the x-coordinate of the new energy engineering vehicle position; calculate the difference between the y-coordinate of the current task position and the y-coordinate of the new energy engineering vehicle position. S352: Calculate the Euclidean distance between the current task position and the position of the new energy engineering vehicle based on the difference between the x-coordinate of the current task position and the x-coordinate of the new energy engineering vehicle position and the difference between the y-coordinate of the current task position and the y-coordinate of the new energy engineering vehicle position. S353: Calculate the difference between the Euclidean distance and the preset Euclidean distance threshold as the first difference; calculate the product of the preset Euclidean distance threshold and the adjustment coefficient as the first product value; construct an exponential function by using the ratio of the first difference to the first product value as a power of the natural number e; sum the exponential function with 1 and take the reciprocal as the standardized Euclidean distance; S354: Calculate the product of the current task's urgency assignment and the preset urgency weight, and record it as the first reward product; calculate the product of the current task's standardized Euclidean distance and the preset standardized Euclidean distance weight, and record it as the second reward product; calculate the product of the new energy engineering vehicle's current battery level and the preset current battery level weight, and record it as the third reward product. S355: Calculate the sum of the first reward product, the second reward product, and the third reward product, and use the sum of the three as the real-time reward.

5. The unmanned driving path planning method for new energy engineering vehicles according to claim 4, characterized in that, The Q-value table update process specifically includes: reading the values ​​in the current Q-value table with the current step size and the next task index as coordinates and recording them as the first intermediate value; reading the maximum value of the row containing the next task index in the current Q-value table and calculating the product of the maximum value and the discount factor to obtain the second intermediate value; calculating the sum of the real-time reward and the second intermediate value and then subtracting the first intermediate value to obtain the third intermediate value; adding the product of the third intermediate value and the learning rate to the first intermediate value, updating the first intermediate value, and using it as the basis for the update in the next inner loop.

6. The method for unmanned driving path planning of new energy engineering vehicles according to claim 1, characterized in that, In step S4, based on the reinforcement learning results from step S3, the autonomous driving planning path for the new energy engineering vehicle is obtained, specifically including: S41: Obtain the task execution order corresponding to the reinforcement learning results in S3; according to the task execution order, obtain the position of the first task, the real-time position of the new energy engineering vehicle, and the real-time battery level of the new energy engineering vehicle; S42: Generate a first autonomous driving path based on the engineering environment, and add the first intermediate waypoint to the first autonomous driving path; S43: Calculate the distance between the real-time location of the new energy engineering vehicle and the first path point and record it as the first distance. Multiply the first distance by the power consumption coefficient and the road condition coefficient, and take the absolute value of the difference between the first distance and the real-time power of the new energy engineering vehicle to obtain the current power of the new energy engineering vehicle. S44: Determine whether the current battery level of the new energy engineering vehicle is less than or equal to the battery threshold. If the current battery level of the new energy engineering vehicle is less than or equal to the battery threshold, terminate the first task and execute the charging station navigation strategy. If the current battery level of the new energy engineering vehicle is greater than the battery threshold, add a second intermediate waypoint on the first autonomous driving path and repeat the above S43-S44. S45: When all intermediate path points have been added and the battery level of the new energy engineering vehicle is still greater than the battery threshold, determine the first autonomous driving path as the autonomous driving planning path of the new energy engineering vehicle; otherwise, regenerate the second autonomous driving path and repeat the above S42-S44.

7. A new energy engineering vehicle unmanned driving path planning system, used to execute the new energy engineering vehicle unmanned driving path planning method as described in any one of claims 1-6, characterized in that, It includes a data acquisition module, a data processing module, and a result output module, wherein the data acquisition module is connected to the data processing module, and the data processing module is connected to the result output module; The data acquisition module is used to acquire the initial state and current task list of the new energy engineering vehicle and input them into the data processing module. Based on the initial state and current task list of the new energy engineering vehicle, the data processing module generates an unmanned driving planning path for the new energy engineering vehicle and inputs it into the result output module. The result output module outputs the unmanned driving planning path of the new energy engineering vehicle.

Images