Human-machine fusion intelligent enhanced navigation method and device based on deep reinforcement learning

Abstract

The invention discloses a man-machine fusion intelligent enhanced navigation method based on deep reinforcement learning. The method comprises the following steps: carrying a laser radar sensor and image acquisition equipment on an intelligent agent; loading the trained intelligent navigation model into an intelligent agent; in the advancing process of the intelligent body, radar data and image data are collected in real time through a laser radar sensor and image collection equipment, the image data are transmitted to an auxiliary decision-making display, and an eye tracker is carried on the auxiliary decision-making display and used for obtaining eye movement data of an auxiliary operator; an auxiliary operator wears an electroencephalogram collection device to collect electroencephalogram data in real time and watches an image picture in an auxiliary decision display, and when an abnormal obstacle appears in the image picture, the auxiliary operator watches an intelligent agent to avoid the direction of the abnormal obstacle; and inputting the current environment state vector into the intelligent navigation model, and deciding and outputting a control instruction of the intelligent agent through the intelligent navigation model.

Description

Technical Field

[0001] The present invention belongs to the interdisciplinary research field of brain science, artificial intelligence and unmanned system control, and specifically relates to a human-machine fusion intelligent enhanced navigation method and device based on deep reinforcement learning. Background Art

[0002] With the rapid development of intelligent control technology, various intelligent robots or unmanned systems are playing an increasingly important role in many fields. For example, in the fields of medical care, catering and retail, intelligent robots can optimize service processes and improve customer experience; in the military and public safety, unmanned systems can enhance emergency response and security capabilities. Autonomous navigation, as a key technology in the field of intelligent robots, is an important guarantee for its completion of various functions. The goal of autonomous navigation is to enable the intelligent body to perceive the surrounding environment in real time, accurately locate its own position, and autonomously decide the optimal route according to preset goals or dynamic environmental changes by integrating high-precision sensors, advanced algorithm models (including SLAM (simultaneous localization and mapping), path planning and optimization algorithms) and powerful data processing capabilities.

[0003] In recent years, compared with traditional navigation methods, the advantages and importance of deep reinforcement learning in navigation applications have become increasingly prominent. Most traditional navigation methods require path planning based on static maps, which are often limited by environmental changes and data update lags, making it difficult to cope with dynamic changes in actual situations. Deep reinforcement learning, on the other hand, dynamically optimizes navigation strategies through continuous trial and error and learning through continuous interaction between the agent and the environment, and achieves perception, decision-making, and control in unknown environments. Despite this, deep reinforcement learning methods that currently rely solely on sensors such as lidar, cameras, and GPS also have some complex environments that are difficult to cope with, such as sudden obstacles, extreme weather or lighting conditions, sensor failures, and other special situations.

[0004] In summary, the existing navigation methods have the following problems:

[0005] Traditional navigation methods require path planning based on static maps, which makes it difficult to cope with dynamically changing complex environments; deep reinforcement learning navigation methods that rely solely on machine intelligence methods have difficulty dealing with some special abnormal obstacles, such as sensor failure, sudden emergencies, and the influence of extreme weather lighting and object appearance conditions. Summary of the invention

[0006] The purpose of the present invention is to provide a human-machine fusion intelligent enhanced navigation method and device based on deep reinforcement learning, so as to solve the problem that the existing intelligent body navigation model has insufficient coping ability for complex environments.

[0007] In order to achieve the above tasks, the present invention adopts the following technical solutions:

[0008] A first aspect of the present invention provides a human-machine fusion intelligent enhanced navigation method based on deep reinforcement learning, comprising:

[0009] The intelligent body is equipped with a lidar sensor and image acquisition equipment;

[0010] The trained intelligent navigation model is loaded into the intelligent agent; during the movement, the intelligent agent collects radar data and image data in real time through the lidar sensor and the image acquisition device, and transmits the image data to the auxiliary decision display. The auxiliary decision display is equipped with an eye tracker to obtain the eye movement data of the auxiliary operator;

[0011] The assistant operator wears an EEG acquisition device to collect EEG data in real time and looks at the image on the auxiliary decision display. When an abnormal obstacle appears in the image, the assistant operator looks at the direction in which the intelligent agent can avoid the abnormal obstacle.

[0012] The current environment state vector is input into the intelligent navigation model, and the intelligent navigation model decides and outputs the control instructions of the intelligent agent; wherein the current environment state vector includes: radar data, the current posture information of the intelligent agent, the distance information between the current position of the intelligent agent and the destination position, the detection results of abnormal obstacles in the environment by EEG data and eye movement data, and the human brain steering decision based on the eye movement data.

[0013] Furthermore, the training process of the intelligent navigation model is:

[0014] Build Actor network-Critic network;

[0015] The Actor network uses the current environment state vector as input and uses the linear velocity and angular velocity of the agent as output as control instructions to control the movement of the agent; the Critic network estimates the action value and state value of the agent after the movement according to the control instruction based on the preset reward function, adds the action value and state value to the loss function during the Actor network training, and feeds back to the Actor network as part of the training loss;

[0016] The environmental state vector at each moment is collected as a sample and input into the Actor network-Critic network, and the Actor network-Critic network is jointly trained; until the training stop condition is met, the trained Actor network is saved as the intelligent navigation model.

[0017] Optionally, both the Actor network and the Critic network are composed of four fully-connected layers, wherein the Critic network is four fully-connected layers connected in sequence, and the first three layers of the Actor network are three fully-connected layers connected in sequence, and the fourth layer is two parallel fully-connected layers and the sigmoid function and the hyperbolic tangent function are set as activation functions to output the linear velocity and angular velocity of the agent; the Relu activation function is used for the remaining fully-connected layers of the Actor network and the Critic network.

[0018] In one possible implementation, the Critic network is trained using the temporal difference algorithm, and the loss function uses the mean square error function; the Actor network is trained using the policy gradient algorithm to update the network parameters; the loss function is:

[0019] L=-lnπ(a|S)*A(S,a)

[0020] Among them, π(a|s) represents the probability that the Actor network selects the control instruction a under the environment state vector S; A(S,a) is the advantage function of the Actor network, which is defined as the difference between the action value function and the state value function; both are estimated by the Critic network.

[0021] Preferably, the preset reward function is expressed as follows:

[0022] Reward=R a +R b +R c +R d

[0023]

[0024] R d =-λ 5 |ω t -ω eye

[0025] In the above formula, Reward represents the preset reward function, R a represents the comprehensive reward, R b represents the safety reward, R c Represents abnormal obstacle reward, R d Represents eye movement information; r goal is the reward value of the agent when it reaches the destination, r collision is the penalty value for the agent encountering common obstacles; d t-1 and d t are the distances between the agent and the destination at the current time t and the previous time t-1, respectively. is the azimuth of the agent relative to the destination, is the minimum distance to common obstacles recorded by the lidar sensor at the current time t, d max is the threshold of the safe distance from common obstacles, d min is the threshold for determining whether a collision with a common obstacle occurs, v max is the maximum speed that can be achieved, v t ,ω t is the agent control instruction output by the Actor network at the current time t, that is, the linear velocity and angular velocity, h is the threshold for whether there is an abnormal obstacle, ω eye is the human brain turning decision based on eye movement data, s eeg Represents the detection results of abnormal obstacles in the environment using EEG data and eye movement data; 1 ,λ 2 ,λ 3 ,λ 4 ,λ 5 is a custom weight parameter.

[0026] Furthermore, the specific process of determining the detection result of abnormal obstacles in the environment by the EEG data and the eye movement data is as follows:

[0027] When the auxiliary operator observes the information in the image screen of the auxiliary decision display, the EEG acquisition device and the eye tracker collect EEG data and eye movement data in real time;

[0028] For EEG data, after preprocessing and STHCP feature extraction, the SVM classifier is used to give the result A1 of whether there is an abnormal obstacle in the current image;

[0029] For the eye movement data, preprocessing and eye movement feature extraction are performed in sequence, and the result A2 of whether there is an abnormal obstacle in the current image is given by the SVM classifier;

[0030] The decision layer fusion criterion is used to fuse the results A1 and A2 to obtain the detection results of abnormal obstacles in the environment using EEG data and eye movement data.

[0031] In a possible implementation, the eye movement features include pupil diameter and gaze point position; the decision layer fusion criteria include DS evidence theory and weighted average.

[0032] Furthermore, the specific determination process of the human brain turning decision based on the eye movement data is as follows:

[0033] The auxiliary operator's eye gaze point is obtained in real time, and the angle between the eye gaze point position and the midline of the auxiliary decision-making display screen is calculated. When the angle exceeds the preset threshold, it is judged that the auxiliary operator believes that the current intelligent agent needs to turn in the environment encountered, and the angle is used as the human brain turning decision made by the auxiliary operator based on the eye movement data; when it does not exceed the threshold, it is considered that no turning operation is required, and the human brain turning decision output is 0.

[0034] The second aspect of the present invention provides a human-machine fusion intelligent enhanced navigation device based on deep reinforcement learning, comprising: a laser radar sensor, an image acquisition device, an eye tracker, an electroencephalogram acquisition device, and a decision-making display, wherein:

[0035] The LiDAR sensor and image acquisition device are mounted on the intelligent agent; the intelligent agent is loaded with a trained intelligent navigation model;

[0036] During the movement, the intelligent body collects radar data and image data in real time through the lidar sensor and image acquisition equipment, and transmits the image data to the auxiliary decision display. The auxiliary decision display is equipped with an eye tracker to obtain the eye movement data of the auxiliary operator;

[0037] The assistant operator wears an EEG acquisition device to collect EEG data in real time and looks at the image on the auxiliary decision display. When an abnormal obstacle appears in the image, the assistant operator looks at the direction in which the intelligent agent can avoid the abnormal obstacle.

[0038] The current environment state vector is input into the intelligent navigation model, and the intelligent navigation model decides and outputs the control instructions of the intelligent agent; wherein the current environment state vector includes: radar data, the current posture information of the intelligent agent, the distance information between the current position of the intelligent agent and the destination position, the detection results of abnormal obstacles in the environment by EEG data and eye movement data, and the human brain steering decision based on the eye movement data.

[0039] A computer-readable storage medium stores a computer program; when the computer program is executed by a processor, the human-machine fusion intelligent enhanced navigation method based on deep reinforcement learning is implemented.

[0040] Compared with the prior art, the present invention has the following technical features:

[0041] The present invention decodes EEG and eye movement physiological information to obtain the human brain's cognition and decision-making of the current navigation environment in real time and integrates it with machine intelligence, further improving the ability of the intelligent body's navigation model to cope with complex environments. When applied, this method does not require scanning static maps in advance, can solve special abnormal conditions that are difficult to deal with relying solely on machine intelligence, avoid navigation errors, and will not cause excessive brain burden on operators. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a schematic diagram of the process of the present invention;

[0043] Figure 2 It is a schematic diagram of the principle of the method of the present invention;

[0044] Figure 3 This is a scene graph in one embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to solve the problems faced by existing technologies and improve the safety and flexibility of intelligent bodies, considering that humans have powerful rapid cognitive and decision-making capabilities, integrating human brain intelligence and machine intelligence to achieve intelligent enhanced navigation is a feasible solution.

[0046] See attached Figure 1 and Figure 2 The present invention discloses a human-machine fusion intelligent enhanced navigation method based on deep reinforcement learning, comprising the following steps:

[0047] The intelligent body is equipped with a lidar sensor and image acquisition equipment;

[0048] The trained intelligent navigation model is loaded into the intelligent agent. During the movement, the intelligent agent collects radar data and image data in real time through the lidar sensor and image acquisition equipment, and transmits the image data to the auxiliary decision display. The auxiliary decision display is equipped with an eye tracker to obtain the eye movement data of the auxiliary operator.

[0049] The assistant operator wears an EEG acquisition device to collect EEG data in real time and looks at the image on the auxiliary decision display. When an abnormal obstacle appears in the image, the assistant operator looks at the direction in which the intelligent agent can avoid the abnormal obstacle.

[0050] The current environment state vector is input into the intelligent navigation model, and the intelligent navigation model decides and outputs the control instructions of the intelligent agent; wherein the current environment state vector includes: radar data, the current posture information of the intelligent agent, the distance information between the current position of the intelligent agent and the destination position, the detection results of abnormal obstacles in the environment by EEG data and eye movement data, and the human brain steering decision based on the eye movement data.

[0051] 1. The intelligent agent described in this scheme can be, for example, a household robot; this scheme is applied to the path navigation process of the household robot; the auxiliary operator in this scheme is used to correct the autonomous navigation decision results of the intelligent agent through eye movement data and EEG data, so that the intelligent agent can better avoid obstacles; this scheme has good application prospects for emergencies, complex road conditions and other environments during the navigation process of the intelligent agent.

[0052] 2. Intelligent navigation model.

[0053] Actor-Critic is an important architecture in deep reinforcement learning, which cleverly combines the Actor network and the Critic network to jointly optimize the behavior of the intelligent agent.

[0054] During the training process of the intelligent navigation model, a neural network based on the actor-critic structure is adopted, in which the Actor network takes the current environment state vector as input and the linear velocity and angular velocity of the intelligent body as output as control instructions to control the motion of the intelligent body; the Critic network evaluates the action value and state value of the intelligent body after moving according to the control instruction through a preset reward function, adds the action value and state value to the loss function during the training of the Actor network, and feeds back to the Actor network as part of the training loss; through the continuous interaction and iterative update of the Actor network and the Critic network, the Actor network can gradually learn the optimal control instruction strategy under a given environment state vector.

[0055] In this embodiment, the Actor network and the Critic network are both composed of four fully connected layers, among which the Critic network is four fully connected layers connected in sequence, and the first three layers of the Actor network are three fully connected layers connected in sequence, and the fourth layer is two parallel fully connected layers and the sigmoid function and the hyperbolic tangent function are set as activation functions to output the linear velocity and angular velocity of the agent; the Relu activation function is used for the remaining fully connected layers of the Actor network and the Critic network.

[0056] When training the Critic network, the Temporal Difference (TD) algorithm can be used, and the loss function uses the mean square error function; when training the Actor network, the policy gradient algorithm can be used to update the network parameters; the loss function is:

[0057] L=-lnπ(a|S)*A(S,a)

[0058] Among them, π(a|s) represents the probability that the Actor network selects the control instruction a under the environment state vector S; A(S,a) is the advantage function of the Actor network, which is defined as the difference between the action value function and the state value function, both of which are estimated by the Critic network.

[0059] 3. Preset reward function.

[0060] Among them, the Critic network sets the reward function of human-machine fusion by comprehensively considering factors such as path length, collision avoidance, and time efficiency, among which:

[0061] R a R represents the comprehensive reward, which is a sparse reward when the agent reaches the destination or collides with a common obstacle, and a dense reward designed to accelerate the agent to reach the target point; b represents the safety reward, which aims to keep a safe distance from obstacles during the agent's path planning process. A penalty is imposed when the distance to a common obstacle is below a specified threshold, and the negative reward increases as the distance decreases; R c Represents the abnormal obstacle reward. The purpose is to slow down in time when an abnormal obstacle is detected based on eye movement data and EEG data. The slower the speed, the greater the reward. d Indicates the eye movement information penalty, the purpose is to make the intelligent agent consistent with the steering angle of the human brain decision;

[0062] Among them, ordinary obstacles refer to obstacles that can be detected by the lidar sensor during the movement of the intelligent agent; abnormal obstacles refer to obstacles that the intelligent agent suddenly encounters during the navigation process and obstacles that cannot be detected normally by the lidar sensor.

[0063] The preset reward function in this scheme is expressed as follows:

[0064] Reward=R a +R b +R c +R d

[0065]

[0066] In the above formula, Reward represents the preset reward function, r goal is the reward value of the agent when it reaches the destination, r collision is the penalty value for the agent encountering common obstacles; d t-1 and d t are the distances between the agent and the destination at the current time t and the previous time t-1, respectively. is the azimuth of the agent relative to the destination, is the minimum distance to common obstacles recorded by the lidar sensor at the current time t, d max is the threshold of the safe distance from common obstacles, d min is the threshold for determining whether a collision with a common obstacle occurs, v max is the maximum speed that can be achieved, v t ,ω t is the agent control instruction output by the Actor network at the current time t, that is, the linear velocity and angular velocity, h is the threshold for whether there is an abnormal obstacle, ω eyeis the human brain turning decision based on eye movement data, s eeg Represents the detection results of abnormal obstacles in the environment using EEG data and eye movement data; 1 ,λ 2 ,λ 3 ,λ 4 ,λ 5 is a custom weight parameter.

[0067] 4. Environmental state vector.

[0068] In navigation tasks, as the input of the Actor-Critic network, it is crucial to accurately and comprehensively set the state of the agent; methods that rely solely on artificial intelligence usually focus on the basic motion state of the agent, such as position, orientation, and radar data, but this is not enough to capture the full complexity of the environment when faced with special situations.

[0069] Therefore, the present invention introduces the environmental cognition and decision-making results obtained based on physiological information decoding into the state to enrich the state dimension; the environmental state vector S input by the intelligent navigation model in this solution is expressed as:

[0070] S=[s c ,s h ,s d ,s eeg ,s eye ]

[0071] Among them, s c is the radar data, s h is the current posture information of the agent (obtained through the sensor carried by the agent itself), s d is the distance information between the current position of the agent and the destination position, s eeg is the detection result of abnormal obstacles in the environment using EEG data and eye movement data, s eye The human brain turns to make decisions based on eye movement data.

[0072] 5. Detection results of abnormal obstacles in the environment using EEG data and eye movement data.

[0073] When the auxiliary operator observes the information in the image screen of the auxiliary decision display, the EEG acquisition device and the eye tracker collect EEG data and eye movement data in real time;

[0074] For EEG data, after preprocessing (filtering, segmentation by time) and STHCP feature extraction, the SVM classifier is used to give the result A1 of whether there is an abnormal obstacle in the current image;

[0075] For the eye movement data, preprocessing (denoising, segmentation by time) and eye movement feature extraction are performed in sequence, and a commonly used classifier such as an SVM classifier is used to give a result A2 on whether there are abnormal obstacles in the current image; wherein the eye movement features may be, for example, pupil diameter, gaze point position, etc.

[0076] The results A1 and A2 are fused by using decision-level fusion criteria such as DS evidence theory and weighted average to obtain the detection result s of abnormal obstacles in the environment by EEG data and eye movement data. eeg .

[0077] 6. Human brain turning decisions based on eye movement data.

[0078] The auxiliary operator's eye gaze point is obtained in real time, and the angle between the eye gaze point position and the midline of the auxiliary decision-making display screen is calculated. When the angle exceeds the preset threshold, it is judged that the auxiliary operator believes that the current intelligent agent needs to turn in the environment encountered. The angle is used as the human brain turning decision made by the auxiliary operator based on the eye movement data. When it does not exceed the threshold, it is considered that no turning operation is required, and the human brain turning decision output is 0.

[0079] The design idea of ​​this step is that it is difficult for the intelligent agent to detect or avoid abnormal obstacles through the lidar sensor. Therefore, when the auxiliary operator recognizes and observes the abnormal obstacle on the screen, the intelligent agent can avoid the abnormal obstacle by looking at the direction of the abnormal obstacle and turning the eyes to output the human brain turning decision and provide it to the intelligent navigation model, so that the intelligent navigation model considers the influence of the human brain turning decision when outputting the final decision.

[0080] 7. Training process of intelligent navigation model.

[0081] During training, the process of the intelligent agent performing the task is used as the training scene. LiDAR sensors and image acquisition devices are used to collect radar data and image data, as well as the EEG data and eye movement data of the auxiliary operator. Each moment t corresponds to an action instruction of the intelligent agent. The environment state vector S at each moment is collected as a sample and input into the Actor network-Critic network to perform joint training on the Actor network-Critic network. Until the training stop conditions are met, such as reaching the specified round or model convergence, the trained Actor network is saved as the final intelligent navigation model.

[0082] Through this process, the intelligent navigation model gradually learns to make the best decision in the environment to reach the destination. In actual application, large-scale training should be carried out according to the application scenario so that the performance of the intelligent navigation model can achieve relatively ideal results.

[0083] Example:

[0084] See attached Figure 3 , which is a simulation verification scenario of the present invention.

[0085] In this scenario, ROS is used as the agent operating system, a simulation environment is built based on gazebo, the agent model is Turtlebot3, the red square is the destination location, and two types of obstacles are set to simulate some sudden complex environments to verify the performance of the method of the present invention.

[0086] The white cylinders are ordinary obstacles, which are used to simulate pedestrians or vehicles in the scene and can be detected by the radar equipped by the smart car; the red spheres are set as abnormal obstacles that suddenly appear in front of the driving path of the smart body and cannot be detected by the lidar sensor.

[0087] The results of comparative training experiments show that the model trained solely by artificial intelligence cannot navigate correctly because the lidar sensor cannot detect abnormal obstacles. The navigation method proposed in the present invention can not only successfully avoid two types of obstacles, but also reduce the training time, which proves the effectiveness of the method proposed in the present invention.

[0088] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A human-machine fusion intelligent enhanced navigation method based on deep reinforcement learning, characterized in that: include: The intelligent body is equipped with a lidar sensor and image acquisition equipment; Load the trained intelligent navigation model into the agent; During the movement, the intelligent body collects radar data and image data in real time through the lidar sensor and image acquisition equipment, and transmits the image data to the auxiliary decision display. The auxiliary decision display is equipped with an eye tracker to obtain the eye movement data of the auxiliary operator; The assistant operator wears an EEG acquisition device to collect EEG data in real time and looks at the image on the auxiliary decision display. When an abnormal obstacle appears in the image, the assistant operator looks at the direction in which the intelligent agent can avoid the abnormal obstacle. The current environment state vector is input into the intelligent navigation model, and the intelligent navigation model decides and outputs the control instructions of the intelligent agent; wherein the current environment state vector includes: radar data, the current posture information of the intelligent agent, the distance information between the current position of the intelligent agent and the destination position, the detection results of abnormal obstacles in the environment by EEG data and eye movement data, and the human brain steering decision based on the eye movement data.

2. The human-machine fusion intelligent enhanced navigation method based on deep reinforcement learning according to claim 1 is characterized in that: The training process of the intelligent navigation model is: Build Actor network-Critic network; The Actor network uses the current environment state vector as input and uses the linear velocity and angular velocity of the agent as output as control instructions to control the movement of the agent; the Critic network estimates the action value and state value of the agent after the movement according to the control instruction based on the preset reward function, adds the action value and state value to the loss function during the Actor network training, and feeds back to the Actor network as part of the training loss; The environmental state vector at each moment is collected as a sample and input into the Actor network-Critic network, and the Actor network-Critic network is jointly trained; until the training stop condition is met, the trained Actor network is saved as the intelligent navigation model.

3. The human-machine fusion intelligent enhanced navigation method based on deep reinforcement learning according to claim 2 is characterized in that: Both the Actor network and the Critic network are composed of four fully connected layers, among which the Critic network is four fully connected layers connected in sequence, and the first three layers of the Actor network are three fully connected layers connected in sequence. The fourth layer is two parallel fully connected layers and the sigmoid function and hyperbolic tangent function are set as activation functions to output the linear velocity and angular velocity of the agent respectively; the Relu activation function is used after the remaining fully connected layers of the Actor network and the Critic network.

4. The human-machine fusion intelligent enhanced navigation method based on deep reinforcement learning according to claim 2 is characterized in that: The Critic network is trained using the temporal difference algorithm, and the loss function uses the mean square error function; the Actor network is trained using the policy gradient algorithm to update the network parameters; the loss function is: L=-lnπ(a|S)*A(S,a) Among them, π(a|s) represents the probability that the Actor network selects the control instruction a under the environment state vector S; A(S,a) is the advantage function of the Actor network, which is defined as the difference between the action value function and the state value function; both are estimated by the Critic network.

5. The human-machine fusion intelligent enhanced navigation method based on deep reinforcement learning according to claim 2 is characterized in that: The preset reward function Reward is expressed as follows: Reward=R a +R b +R c +R d R d =-λ5|ω t -oh eye In the above formula, R a represents the comprehensive reward, R b represents the safety reward, R c Represents abnormal obstacle reward, R d Indicates eye movement information reward; r goal is the reward value of the agent when it reaches the destination, r collision is the penalty value for the agent encountering common obstacles; d t-1 and d t are the distances between the agent and the destination at the current time t and the previous time t-1, respectively. is the azimuth of the agent relative to the destination, is the minimum distance to common obstacles recorded by the lidar sensor at the current time t, d max is the threshold of the safe distance from common obstacles, d min is the threshold for determining whether a collision with a common obstacle occurs, v max is the maximum speed that can be achieved, v t ,ω t is the agent control instruction output by the Actor network at the current time t, that is, the linear velocity and angular velocity, h is the threshold for whether there is an abnormal obstacle, ω eye is the human brain turning decision based on eye movement data, s eeg Represents the detection results of abnormal obstacles in the environment by EEG data and eye movement data; λ1, λ2, λ3, λ4, λ5 are custom weight parameters.

6. The human-machine fusion intelligent enhanced navigation method based on deep reinforcement learning according to claim 1 is characterized in that: The specific process of determining the detection result of abnormal obstacles in the environment using EEG data and eye movement data is as follows: When the auxiliary operator observes the information in the image screen of the auxiliary decision display, the EEG acquisition device and the eye tracker collect EEG data and eye movement data in real time; For EEG data, after preprocessing and STHCP feature extraction, the SVM classifier is used to give the result A1 of whether there is an abnormal obstacle in the current image; For the eye movement data, preprocessing and eye movement feature extraction are performed in sequence, and the result A2 of whether there is an abnormal obstacle in the current image is given by the SVM classifier; The decision layer fusion criterion is used to fuse the results A1 and A2 to obtain the detection results of abnormal obstacles in the environment using EEG data and eye movement data.

7. The human-machine fusion intelligent enhanced navigation method based on deep reinforcement learning according to claim 6 is characterized in that: The eye movement features include pupil diameter and gaze point position; the decision layer fusion criteria include DS evidence theory and weighted average.

8. The method for human-machine fusion intelligent enhanced navigation based on deep reinforcement learning according to claim 1 is characterized in that: The specific determination process of the human brain turning decision based on the eye movement data is as follows: The auxiliary operator's eye gaze point is obtained in real time, and the angle between the eye gaze point position and the midline of the auxiliary decision-making display screen is calculated. When the angle exceeds the preset threshold, it is judged that the auxiliary operator believes that the current intelligent agent needs to turn in the environment encountered, and the angle is used as the human brain turning decision made by the auxiliary operator based on the eye movement data; when it does not exceed the threshold, it is considered that no turning operation is required, and the human brain turning decision output is 0.

9. A human-machine fusion intelligent enhanced navigation device based on deep reinforcement learning, characterized in that: include: LiDAR sensor, image acquisition device, eye tracker, EEG acquisition device and decision-making display, including: The LiDAR sensor and image acquisition device are mounted on the intelligent agent; the intelligent agent is loaded with a trained intelligent navigation model; During the movement, the intelligent body collects radar data and image data in real time through the lidar sensor and image acquisition equipment, and transmits the image data to the auxiliary decision display. The auxiliary decision display is equipped with an eye tracker to obtain the eye movement data of the auxiliary operator; The assistant operator wears an EEG acquisition device to collect EEG data in real time and looks at the image on the auxiliary decision display. When an abnormal obstacle appears in the image, the assistant operator looks at the direction in which the intelligent agent can avoid the abnormal obstacle. The current environment state vector is input into the intelligent navigation model, and the intelligent navigation model decides and outputs the control instructions of the intelligent agent; wherein the current environment state vector includes: radar data, the current posture information of the intelligent agent, the distance information between the current position of the intelligent agent and the destination position, the detection results of abnormal obstacles in the environment by EEG data and eye movement data, and the human brain steering decision based on the eye movement data.

10. A computer-readable storage medium, wherein a computer program is stored in the medium; characterized in that: When the computer program is executed by a processor, the human-machine fusion intelligent enhanced navigation method based on deep reinforcement learning according to any one of claims 1-8 is implemented.

Images