Intention-Driven Path Planning for Obstacle-Aware Data Collection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data collection path planning methods for UAVs, unmanned ships, and submarines in wireless sensor networks fail to consider the dynamic intentions of both data collectors and sensor nodes, leading to low efficiency and reliability due to limited energy and random obstacles.
Innovation Solution
An intention-driven reinforcement learning-based path planning method using Q-learning to optimize the trajectory of data collectors by expressing intentions as rewards and penalties, adjusting steering angles, and selecting actions based on real-time network states to ensure efficient and reliable data collection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If existing path planning methods are used that consider data collector and sensor node intentions separately, then the path planning is simpler, but data collection efficiency and reliability are low
Solution Approach 1:
The patent merges the intentions of the data collector and sensor nodes into a unified reinforcement learning framework. The reward function integrates multiple objectives including data collection completeness, energy consumption, collision avoidance, and time constraints, allowing simultaneous optimization of all factors rather than separate consideration.
Solution Approach 2:
The path planning method uses dynamic reinforcement learning where the Q-values and policies are continuously updated based on real-time environmental feedback. The data collector adapts its behavior dynamically by learning from interactions with the environment, adjusting its path planning based on changing conditions such as obstacle positions and sensor node states.
2Reliability
If existing path planning methods are used that do not consider dynamic obstacles, then the planning process is simpler, but reliability is reduced due to random obstacles
Solution Approach 1:
The reinforcement learning framework implements continuous feedback mechanisms where the data collector receives rewards or penalties based on its actions and the resulting state changes. This feedback loop enables the system to learn from collisions with obstacles or near-misses, progressively improving its ability to navigate around dynamic obstacles reliably.
Solution Approach 2:
The data collector autonomously learns navigation strategies through self-interaction with the environment. The reinforcement learning agent explores the environment, learns from its experiences, and develops collision avoidance capabilities without requiring explicit programming or external control, thereby improving reliability in dynamic obstacle environments.
3Productivity
If the data collector follows a fixed path, then energy consumption is easier to manage, but data collection efficiency decreases due to inability to adapt to changing conditions
Solution Approach 1:
The path planning method transitions from static fixed paths to dynamic adaptive paths using reinforcement learning. The data collector's trajectory is continuously optimized based on real-time environmental feedback, allowing it to adapt to changing conditions such as obstacle movements and sensor node locations, thereby improving data collection efficiency while managing energy consumption through intelligent decision-making.
Solution Approach 2:
The reinforcement learning framework dynamically adjusts multiple parameters including steering angle, speed, and target node selection based on the current state. The Q-values associated with different actions are updated continuously, allowing the system to find optimal parameter combinations that balance data collection efficiency with energy consumption constraints.
4Productivity
If the data collector explores new paths to improve data collection coverage, then data collection efficiency improves, but energy consumption increases
Solution Approach 1:
The reinforcement learning framework dynamically adjusts exploration parameters including the epsilon-greedy exploration rate, which balances exploitation of known good paths with exploration of new potentially better paths. As learning progresses, the system naturally shifts from extensive exploration to more efficient exploitation, optimizing the trade-off between data collection coverage and energy consumption.
Solution Approach 2:
The reinforcement learning process ensures continuous useful action by maintaining a policy that always selects actions with positive expected returns. The data collector continuously learns and refines its path planning, ensuring that exploration activities remain productive and contribute to both coverage improvement and energy efficiency rather than wasting energy on futile movements.
Data Source
AI summary
The present invention discloses an intention-driven reinforcement learning-based path planning method, including the following steps: 1: acquiring, by a data collector, a state of a monitoring network; 2: selecting a steering angle of the data collector according to positions of surrounding obstacles, sensor nodes, and the data collector; 3: selecting a speed of the data collector, a target node, and a next target node as an action of the data collector according to an ε greedy policy; 4: determining, by the data collector, the next time slot according to the selected steering angle and speed; 5: obtaining rewards and penalties according to intentions of the data collector and the sensor nodes, and updating a Q value; 6: repeating step 1 to step 5 until a termination state or a convergence condition is satisfied; and 7: selecting, by the data collector, an action in each time slot having the maximum Q value as a planning result, and generating an optimal path. The method provided in the present invention can complete the data collection path planning with a higher probability of success and performance closer to the intention.