The invention discloses a deep
reinforcement learning robot-assisted ultrasonic scanning
system and method based on security constraints. The invention discloses a deep
reinforcement learning mechanical arm ultrasonic scanning method and
system based on security constraints. A scanning
system comprising a mechanical arm, an ultrasonic probe, a joint torque and visual sensor and a
control unit is constructed. The environment
state space covers the
joint angle, the
tail end
pose, the
contact force, the scanning path and the safety
state variable of the mechanical arm, and the action space is a
tail end linear speed instruction of the mechanical arm. A deep
reinforcement learning model is built based on a near-end strategy optimization (PPO)
algorithm, and a reward function including
image quality,
task completion reward and multiple penalty terms is designed. A control
barrier function (CBFs) is used for implementing safety constraint on motion of the mechanical arm, and the safety constraint is fused into deep reinforcement learning training through a penalty
function method. A model is trained in a
simulation environment simulating an actual scanning scene and then deployed to an actual system, and a mechanical arm is controlled to scan and monitor
safety constraints according to a real-time environment state. The mechanical arm can efficiently execute the ultrasonic scanning task under the condition that the safety constraint is met, and a reliable scheme is provided for ultrasonic scanning
automation. Meanwhile, the system is provided with a
data acquisition module, a state construction module, a reward calculation module, a security constraint module, a control module and the like, and the model performance can be improved through
simulation training.