A gradient-free reinforcement learning method based on acceleration / deceleration strategies

By constructing a low-dimensional parameter search space and introducing a gradient-free reinforcement learning method with a random embedding mechanism, the problem of difficulty in obtaining gradient information during autonomous navigation of unmanned surface vessels is solved, achieving efficient policy optimization and stability improvement while reducing computational costs.

CN121480599BActive Publication Date: 2026-09-01HUAZHONG UNIV OF SCI & TECH
2 Cites 0 Cited by

Patent Information

Application Number
CN202511486030.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-10-17
Publication Date
2026-09-01
Estimated Expiration
2045-10-17

AI Technical Summary

Technical Problem

Existing deep reinforcement learning methods suffer from stability and convergence issues when relying on gradient optimization in autonomous navigation and path planning of unmanned surface vessels. In particular, when the environmental model is unknown or the dynamics are uncertain, gradient information is difficult to obtain, resulting in low sampling efficiency and high computational cost.

Method used

A gradient-free reinforcement learning method based on acceleration and deceleration strategies is adopted. A low-dimensional parameter search space is constructed through dimensionality reduction and dimensionality increase strategies. The eigenvector with the minimum eigenvalue of the covariance matrix is ​​used as the parameter update direction and mapped to the high-dimensional policy space through a random matrix. The policy is updated by combining the cumulative total reward value. A mapping function is introduced to control the reward value and update rate to prevent local optima.

Benefits of technology

Without requiring explicit gradient calculation, it improves policy training efficiency, reduces computational resource consumption, enhances stability and generalization ability, is suitable for high-dimensional complex control scenarios, and solves the problem of high sampling cost in gradient-free reinforcement learning.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

This invention belongs to the technical field of intelligent control and reinforcement learning optimization for unmanned surface vessels (USVs), specifically involving a gradient-free reinforcement learning method based on acceleration / deceleration strategies. The method includes: reducing the original high-dimensional parameter search space to an effective low-dimensional parameter search space; evaluating the quality of the optimization results under this effective low-dimensional space; if the quality is not satisfactory, increasing the dimension of the effective low-dimensional space until a satisfactory effective low-dimensional parameter search space is determined; randomly generating a set of initial parameter solutions in this space, where each initial parameter solution consists of the values ​​of all parameters of the neural network to be learned; determining the parameter update direction based on the current set of initial parameter solutions; and mapping the current set of initial parameter solutions back to the original high-dimensional parameter search space as a set of policy parameter solutions; independently evaluating the cumulative total reward of each policy parameter solution; updating the initial parameter solution in conjunction with the parameter update direction; and repeating the above update operation. This invention has low sampling costs.
Need to check novelty before this filing date? Find Prior Art