An autonomous navigation method, device, equipment and medium for unmanned aerial vehicles (UAVs)

Abstract

This application relates to the field of unmanned aerial vehicle (UAV) technology and discloses an autonomous navigation method for UAVs. The method involves acquiring environmental perception data of the UAV's current environment; proposing actions based on the environmental perception data using an action decision network to obtain candidate actions; the reward function of the action decision network is derived from the characteristic differences between the current UAV environment and the predicted environment; determining the optimal action for the UAV at the current moment based on the future cumulative reward value of the candidate actions; and updating the action decision network and its reward function based on environmental perception data, the optimal action, and its corresponding future cumulative reward value. The beneficial effect is that by predicting the environment based on historical environmental data and comparing the differences with the current UAV environment, the optimal action for controlling the UAV can be determined, incentivizing the UAV to explore unknown environments and improving its autonomous navigation capability during mission execution.

Description

Technical Field

[0001] This application relates to the field of unmanned aerial vehicle (UAV) technology, and in particular to an autonomous navigation method, device, equipment, and medium for UAVs. Background Technology

[0002] With the development of intelligent vision inspection technology, Unmanned Aerial Vehicle (UAV) technology, with its flexibility, high maneuverability, and low cost, has been widely used in tasks such as UAV inspection, search and rescue, surveillance, and military reconnaissance. It enables precise real-time detection and tracking of ground or aerial targets and has broad applications in environmental monitoring, military reconnaissance, infrastructure inspection, and disaster relief. In practical scenarios, UAV navigation algorithms are typically trained using reward functions, such as by adding penalty terms related to collision risk, to improve the safety of UAVs during mission execution. However, these penalty terms may limit the UAV's exploration of the environment, leading to mission failure.

[0003] Among related technologies, the autonomous navigation capability of drones during mission execution still needs to be improved. Summary of the Invention

[0004] This application provides an autonomous navigation method, device, equipment, and medium for unmanned aerial vehicles (UAVs). It predicts the environment based on historical environmental data and compares the differences with the environment in which the UAV is located, thereby determining the optimal actions for controlling the UAV, encouraging the UAV to explore the unknown environment, and improving the autonomous navigation capability of the UAV during mission execution.

[0005] To achieve the above objectives, the main technical solutions adopted in this application include: In a first aspect, embodiments of this application provide an autonomous navigation method for unmanned aerial vehicles (UAVs), the method comprising: Obtain environmental perception data of the environment in which the drone is located at the current moment; The action decision network proposes actions based on the environmental perception data to obtain candidate actions for the UAV; wherein, the reward function of the action decision network is obtained based on the feature difference between the current environment of the UAV and the predicted environment, and the predicted environment is obtained based on the historical environmental data of the UAV. Based on the future cumulative reward value of the candidate actions, the optimal action of the drone at the current moment is determined for controlling the drone to perform the task; Based on the environmental perception data, the optimal action and its corresponding future cumulative reward value, the action decision network and its reward function are updated empirically to propose actions for the UAV and determine the optimal action in subsequent time moments of the current time.

[0006] The UAV autonomous navigation method proposed in this application acquires environmental perception data of the UAV's environment at each current moment and proposes candidate actions based on the environmental perception data. It then performs environmental prediction based on historical environmental data prior to the current moment, compares the predicted environment with the UAV's current environment to calculate the future cumulative reward value of the candidate actions, and thus determines the optimal action for the UAV at the current moment. After determining the optimal action, it updates the action decision network and reward function based on the current environmental perception data and action experience, for use in subsequent moments to propose actions and determine the optimal action for the UAV. Compared with related technologies, this application obtains a predicted environment based on historical environmental data and derives a reward function based on the characteristic differences between the predicted environment and the UAV's current environment to determine the optimal action for the UAV at the current moment. This incentivizes the UAV to explore unknown environments, reduces the probability of the UAV being confined to known environments, expands the UAV's exploration range, and effectively improves the UAV's autonomous navigation capability. Furthermore, the action policy network and reward function used for action proposals are continuously updated over time, enabling the UAV to autonomously adapt to changes in its environment and perform adaptive autonomous navigation, thus improving the UAV's environmental adaptability.

[0007] Optionally, the reward function can be obtained in the following way: Based on the environmental perception data, a fixed feature mapping is performed on the environment in which the UAV is located to obtain the mapped environmental features of the environment in which the UAV is located. Based on the historical environmental data of the UAV, the environmental characteristics of the environment in which the UAV is located are predicted to obtain the predicted environmental characteristics of the environment in which the UAV is located. Based on the difference calculation of the mapped environment features and the predicted environment features, an environment difference reward item is obtained to incentivize the UAV to explore the unknown environment; The reward function is obtained based on the environmental difference reward item.

[0008] Optionally, the mapped environment features and the predicted environment features are obtained in the following ways: The environmental perception data is input into a fixed mapping network, and the fixed mapping network performs fixed feature mapping based on the environmental perception data to output the mapped environmental features. The environment features of the UAV are predicted by an environment prediction network, and the predicted environment features are obtained; wherein, the environment prediction network is obtained by training the mapping output of the historical environment data based on the fixed mapping network.

[0009] Optionally, the drone corresponds to a mission objective; obtaining the reward function based on the environmental difference reward term includes: The proximity between the drone and the mission target is calculated to obtain a target proximity reward item used to incentivize the drone to approach the mission target. The flight status of the UAV is used to determine the UAV's capture result of the mission target, and a target-oriented reward item is obtained based on the determination result to incentivize the UAV to accurately capture the mission target. Based on the time taken by the drone to perform the task, a delay penalty is calculated to obtain a time cost penalty item used to suppress the drone from delaying behavior during the task. The reward function is obtained based on the environmental difference reward, the target proximity reward, the target orientation reward, and the time cost penalty.

[0010] Optionally, the step of updating the action decision network and its reward function based on the environmental perception data, the optimal action, and its corresponding future cumulative reward value includes: The parameters of the action decision network are updated based on the optimal action and its corresponding future cumulative reward value to obtain an updated action decision network, which is used to propose actions for the drone at the subsequent time. The environmental prediction network is updated with parameters based on the environmental perception data to obtain an updated environmental prediction network, which is then used to determine the optimal action for the UAV at the subsequent time.

[0011] Optionally, the environmental perception data is obtained based on multimodal data of the environment in which the UAV is located, including color images and depth images of the environment in which the UAV is located; the environmental perception data is obtained in the following ways: The color image is segmented and its features are embedded to obtain a color image embedding sequence; the depth image is segmented and its features are embedded to obtain a depth image embedding sequence. The color image embedding sequence and the depth image embedding sequence are cross-fused using modality, and the fusion result is encoded and output to obtain the environmental perception data.

[0012] Optionally, the environmental perception data includes a semantic map of the environment in which the UAV is located; determining the optimal action of the UAV at the current moment based on the future cumulative reward value of the candidate actions includes: Based on the future cumulative reward value of the candidate actions, a proposed action is determined, and a preset interference is applied to the proposed action to obtain multiple derivative actions; wherein, each of the multiple derivative actions corresponds to its own derivative trajectory; Based on the obstacle distribution in the environment where the UAV is located, a geometric risk assessment is performed on any derived trajectory to obtain the geometric risk data of any derived trajectory; Based on the distribution of semantic objects in the semantic map, a semantic risk assessment is performed on any of the derived trajectories to obtain the semantic risk data of any of the derived trajectories; Risk assessment data for any derived trajectory is obtained based on the geometric risk data and the semantic risk data, and the optimal trajectory is determined based on the risk assessment data of all derived trajectories; wherein, the derived action corresponding to the optimal trajectory is the optimal action.

[0013] Secondly, embodiments of this application provide an autonomous navigation device for unmanned aerial vehicles (UAVs), the device comprising: The environmental perception module is used to acquire environmental perception data of the environment in which the UAV is located at the current moment; The action proposal module is used to propose actions based on the environmental perception data through the action decision network to obtain candidate actions for the UAV; wherein, the reward function of the action decision network is obtained based on the feature difference between the current environment of the UAV and the predicted environment, and the predicted environment is obtained based on the historical environmental data of the UAV. The action decision module is used to determine the optimal action of the UAV at the current moment based on the future cumulative reward value of the candidate actions, so as to control the UAV to perform the task; The experience update module is used to update the action decision network and its reward function based on the environmental perception data, the optimal action and its corresponding future cumulative reward value, so as to propose actions for the UAV and determine the optimal action in subsequent time moments of the current time.

[0014] Thirdly, embodiments of this application provide a computer device, including: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the method described in any of the above embodiments.

[0015] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer instructions, which are used to cause a computer to perform the method described in any one of the above embodiments.

[0016] Fifthly, embodiments of this application provide a computer program product, including computer instructions, which are used to cause a computer to perform the method described in any of the above embodiments. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 A step diagram of the unmanned aerial vehicle autonomous navigation method provided in the embodiments of this application; Figure 2 This is a structural block diagram of the UAV in the embodiments of this application; Figure 3 This is a flowchart illustrating the steps involved in obtaining the reward function in an embodiment of this application. Figure 4 This is a flowchart illustrating the steps involved in obtaining the mapped environmental features and the predicted environmental features in an embodiment of this application. Figure 5 This is a flowchart illustrating the steps involved in obtaining the reward function in an embodiment of this application. Figure 6 This is a diagram illustrating the steps of experience updating in the embodiments of this application; Figure 7 This is a diagram illustrating the steps involved in obtaining environmental perception data in an embodiment of this application. Figure 8 This is a flowchart illustrating the steps for determining the optimal action in an embodiment of this application. Figure 9 This is a schematic diagram illustrating the process of determining the optimal action in an embodiment of this application; Figure 10 A block diagram of an autonomous navigation device for unmanned aerial vehicles provided in an embodiment of this application; Figure 11 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] With the development of intelligent vision inspection technology, Unmanned Aerial Vehicle (UAV) technology, due to its flexibility, high maneuverability, and low cost, has been widely used in tasks such as UAV inspection, search and rescue, surveillance, and military reconnaissance. It enables precise real-time detection and tracking of ground or aerial targets and has broad applications in environmental monitoring, military reconnaissance, infrastructure inspection, and disaster relief. In practical scenarios, UAV navigation algorithms are typically trained using reward functions, such as by adding penalty terms related to collision risk, to improve the safety of UAVs during mission execution. However, these penalty terms may limit the UAV's exploration of the environment, leading to mission failure. In related technologies, the autonomous navigation capability of UAVs during mission execution still needs improvement.

[0021] To address the aforementioned issues, this application provides an autonomous navigation method, apparatus, device, and medium for unmanned aerial vehicles (UAVs). The method involves acquiring environmental perception data of the current environment in which the UAV is located; proposing actions based on the environmental perception data using an action decision network to obtain candidate actions for the UAV; wherein the reward function of the action decision network is derived from the characteristic differences between the current environment and the predicted environment; determining the optimal action for the UAV at the current moment based on the future cumulative reward value of the candidate actions; and updating the action decision network and its reward function based on the environmental perception data, the optimal action, and its corresponding future cumulative reward value.

[0022] The UAV autonomous navigation method provided in this application acquires environmental perception data of the UAV's environment at each current moment and proposes candidate actions based on the environmental perception data; it performs environmental prediction based on historical environmental data before the current moment, compares the predicted environment with the environment in which the UAV is located, calculates the future cumulative reward value of the candidate actions, and then determines the optimal action of the UAV at the current moment; after determining the optimal action, it updates the action decision network and reward function based on the environmental perception data and action experience at the current moment, so as to propose actions and determine the optimal action for the UAV in subsequent moments.

[0023] Compared with related technologies, this application obtains a predicted environment based on historical environmental data and obtains a reward function based on the feature differences between the predicted environment and the environment in which the UAV is located, in order to determine the optimal action of the UAV at the current moment. This incentivizes the UAV to explore the unknown environment, reduces the probability of the UAV being limited to the known environment, expands the exploration range of the UAV, and effectively improves the autonomous navigation capability of the UAV.

[0024] Furthermore, the action policy network and reward function used to propose actions for the drone are continuously updated over time, enabling the drone to autonomously adapt to changes in its environment and perform adaptive autonomous navigation based on the environment, thereby improving the drone's adaptability to the environment.

[0025] According to an embodiment of this application, an embodiment of an autonomous navigation method for unmanned aerial vehicles is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0026] This embodiment provides an autonomous navigation method for unmanned aerial vehicles (UAVs), referring to... Figure 1 As shown, the method includes: S100. Obtain environmental perception data of the environment in which the drone is located at the current moment.

[0027] S200. Through the action decision network, action proposals are made based on environmental perception data to obtain candidate actions for the UAV; wherein, the reward function of the action decision network is obtained based on the feature difference between the current environment of the UAV and the predicted environment, and the predicted environment is obtained based on the historical environmental data of the UAV.

[0028] S300. Based on the future cumulative reward value of the candidate actions, determine the optimal action of the drone at the current moment to control the drone to perform the task.

[0029] S400. Based on environmental perception data, the optimal action and its corresponding future cumulative reward value, the action decision network and its reward function are updated empirically to propose actions for the UAV and determine the optimal action in subsequent time steps of the current time.

[0030] This embodiment can be applied to achieve autonomous navigation of unmanned aerial vehicles (UAVs) in complex environments. The UAV type can be a quadcopter or multi-rotor UAV platform, etc. The structure of the UAV can be referred to... Figure 2As shown, the drone includes a sensor module, a drone navigation system, a flight controller, and a power system connected in sequence. The sensor module includes a vision sensor and an inertial measurement unit (IMU). The vision sensor can be an integrated RGB-D (color and depth) camera, providing the drone with high-resolution color images and pixel-aligned depth images simultaneously to obtain environmental perception data of the drone's environment. For example, the RGB-D camera could be an Intel RealSense D435i. The IMU can be a high-frequency IMU built into the drone to measure the drone's three-axis acceleration and angular velocity. The drone navigation system can include a perception and understanding module, a decision-making module, and a safety verification module. The perception and understanding module acquires and processes environmental perception data of the drone's environment. The decision-making module proposes actions based on the environmental perception data and determines the optimal action for the drone at the current moment. The safety verification module ensures the safety of the drone during mission execution. The flight controller can be a low-level flight controller that receives the optimal action output from the drone navigation system and converts it into motor control signals to control the drone to perform the mission.

[0031] Specifically, at the current moment, data is collected on the environment in which the drone is located, and environmental perception data is obtained based on the collection results. Environmental perception data can include environmental characteristics of the drone's environment, the drone's current flight state, and the drone's actions from the previous moment. This allows the drone to grasp its own global state and that of its environment based on the environmental perception data, providing a comprehensive data foundation for subsequent action decisions. For example, the environmental characteristics of the drone's environment can be obtained through image acquisition and feature extraction, the drone's current flight state can be obtained through an inertial measurement unit, and the environmental perception data can be expressed in the following form: in, For environmental sensing data; The environmental characteristics of the environment in which the drone operates; The characteristics of the mission target of the UAV; This refers to the drone's actions in the previous moment of the current moment; This represents the current flight status of the drone. This is a vector concatenation operation.

[0032] Furthermore, environmental perception data is input into the action decision network. The action decision network proposes actions based on the environmental perception data, obtaining candidate actions for the UAV. There can be one or more candidate actions. After obtaining candidate actions, they are filtered based on their future cumulative reward values ​​in subsequent time steps. These future cumulative reward values ​​can be calculated using the reward function of the action decision network. The candidate action with the highest future cumulative reward value is determined as the optimal action for the UAV at the current time step, and the UAV is controlled to execute the task based on this optimal action.

[0033] It should be noted that the action decision network can be pre-trained in a virtual simulation environment using a reward function. The reward function of the action decision network is derived from the feature differences between the current environment of the drone and the predicted environment. The predicted environment is obtained based on the drone's historical environmental data. The current moment in the reward function can be either the moment the drone is actually performing a task or a moment during the training process of the action decision network. The historical environmental data can be environmental features acquired by the drone at a historical moment or pre-input environmental knowledge. Taking the training process of the action decision network as an example, the reward function of the action decision network is illustrated as follows: At the current moment, the drone predicts its environment based on historical environmental data, obtaining a predicted environment based on experience, and obtaining the predicted environment features. At the current moment, the drone acquires the current environmental features of the virtual simulation environment it is in, calculates the difference between the predicted environment features and the current environment features, determines the feature differences between the predicted environment and the virtual simulation environment in which the drone is in, and thus obtains the future cumulative reward value of the reward function at the current moment.

[0034] Understandably, the method for obtaining the cumulative reward value during the actual execution of a drone's mission is similar to the process described above. When the difference between the predicted environment and the actual environment is small, it indicates that the drone is in a known environment, which may have already been fully explored by the drone, and further exploration has low value. Conversely, when the difference between the predicted environment and the actual environment is large, it indicates that the drone is in an unknown environment, which has not yet been fully explored by the drone and has high exploration value. A reward function is derived based on the characteristic differences between the drone's current environment and the predicted environment. This function is used to train the action decision network and determine the optimal action, incentivizing the drone to explore unknown environments, reducing the probability of the drone being confined to known environments, expanding the drone's exploration range, and effectively improving the drone's autonomous navigation capabilities.

[0035] In some embodiments, the action decision network can be an Actor-Critic policy network trained with a reward function, including a policy network (Actor) and a value network (Critic). The policy network contains a policy function, which evaluates the state of the environmental perception data based on the policy function and outputs corresponding candidate actions. The process of the policy network outputting candidate actions can include: in the continuous action space, the output layer of the policy network parameterizes the environmental perception data to obtain an environmental perception probability distribution, which can be in the form of the mean or standard deviation of a Gaussian distribution, etc.; randomly sampling the environmental perception probability distribution, and generating candidate actions based on the sampling results. For example, the random sampling process can be represented by the following formula: in, The current moment; Candidate actions; This represents the probability distribution of environmental perception. This represents the mean of the probability distribution of environmental perception. Let be the standard deviation of the environmental perception probability distribution. It can be understood that by converting environmental perception data into an environmental perception probability distribution and generating candidate actions through random sampling, randomness is introduced into the action proposal process. This allows the policy network to make stable decisions while also exploring the UAV's environment, improving the UAV's autonomous navigation capabilities. After obtaining the candidate actions output by the policy network, the value network can calculate rewards for the candidate actions according to the reward function, obtaining the future cumulative reward value corresponding to the candidate action in subsequent time steps. The parameters of the policy network are then updated based on the candidate actions and the future cumulative reward value, enabling iterative optimization and improving the quality of the actions output by the policy network.

[0036] Furthermore, the environmental perception data at the current moment is used as historical environmental data to update the reward function of the action decision network, enabling the UAV to gradually learn about its unknown environment. The action decision network is then updated based on the optimal action and its corresponding future cumulative reward value, improving the quality of the actions output by the policy network for subsequent action proposals for the UAV.

[0037] The UAV autonomous navigation method provided in this embodiment acquires environmental perception data of the UAV's environment at each current moment and proposes candidate actions based on the environmental perception data; it performs environmental prediction based on historical environmental data before the current moment, compares the predicted environment with the environment in which the UAV is located, calculates the future cumulative reward value of the candidate actions, and then determines the optimal action of the UAV at the current moment; after determining the optimal action, it updates the action decision network and reward function based on the environmental perception data and action experience at the current moment, so as to propose actions and determine the optimal action for the UAV in subsequent moments.

[0038] Compared with related technologies, this application obtains a predicted environment based on historical environmental data and obtains a reward function based on the feature differences between the predicted environment and the environment in which the UAV is located, in order to determine the optimal action of the UAV at the current moment. This incentivizes the UAV to explore the unknown environment, reduces the probability of the UAV being limited to the known environment, expands the exploration range of the UAV, and effectively improves the autonomous navigation capability of the UAV.

[0039] Furthermore, the action policy network and reward function used to propose actions for the drone are continuously updated over time, enabling the drone to autonomously adapt to changes in its environment and perform adaptive autonomous navigation based on the environment, thereby improving the drone's adaptability to the environment.

[0040] Reference Figure 3 As shown, in one embodiment of this application, the reward function is obtained in the following manner: S210. Based on the environmental perception data, perform fixed feature mapping on the environment in which the UAV is located to obtain the mapped environmental features of the environment in which the UAV is located.

[0041] S220. Based on the historical environmental data of the UAV, predict the environmental characteristics of the environment in which the UAV is located, and obtain the predicted environmental characteristics of the environment in which the UAV is located.

[0042] S230. Based on the differences between the mapped environmental characteristics and the predicted environmental characteristics, an environmental difference reward item is obtained to incentivize UAVs to explore the unknown environment.

[0043] S240. Obtain the reward function based on the environmental difference reward item.

[0044] Specifically, based on the environmental characteristics of the environment in which the UAV is located in the environmental perception data, a fixed feature mapping is performed on the environment in which the UAV is located to obtain the mapped environmental characteristics of the environment in which the UAV is located, so as to represent the true characteristics of the environment in which the UAV is located.

[0045] Furthermore, based on the historical environmental data of the UAV, environmental characteristics are predicted at the current moment to obtain the environmental prediction characteristics of the environment in which the UAV is located, so as to represent the UAV's judgment result on the environment based on experience.

[0046] Furthermore, based on the difference between the mapped environmental features and the predicted environmental features, the feature error between the mapped environmental features and the predicted environmental features is calculated, thereby determining the difference between the predicted environment and the environment in which the UAV is located, and obtaining the environmental difference reward item.

[0047] Understandably, when the difference between the predicted environment and the actual environment is small, it indicates that the drone is in a known environment, which may have already been fully explored, making further exploration less valuable. Conversely, when the difference between the predicted and actual environments is large, it indicates that the drone is in an unknown environment, which has not yet been fully explored and has higher exploration value. An environmental difference reward is derived based on the characteristic differences between the drone's current environment and the predicted environment. Higher reward values ​​are given to the drone when the characteristic differences are large, incentivizing it to explore unknown environments, reducing the probability of the drone being confined to known environments, expanding its exploration range, and effectively improving its autonomous navigation capabilities.

[0048] Reference Figure 4 As shown, in one embodiment of this application, the mapped environment features and the predicted environment features are obtained in the following manner: S212. Input the environmental perception data into the fixed mapping network, perform fixed feature mapping based on the environmental perception data through the fixed mapping network, and output the mapped environmental features.

[0049] S222. The environmental characteristics of the environment in which the UAV is located are predicted by an environmental prediction network to obtain environmental prediction characteristics; wherein, the environmental prediction network is obtained by training on the mapping output of historical environmental data based on a fixed mapping network.

[0050] Specifically, this embodiment employs a Random Network Distillation (RND) mechanism to obtain the mapped environment features and environment prediction features, thereby deriving the reward function of the action decision network. The RND mechanism includes a fixed-map network and an environment prediction network, both of which are identical feedforward neural networks, consisting of a sequentially connected input layer, a linear transformation layer, a ReLU nonlinear activation function, another linear transformation layer, another ReLU nonlinear activation function, and a fully connected layer. The fixed-map network can be a neural network with fixed parameters. During initialization, the parameters of the fixed-map network are randomly set and then frozen to ensure that the parameters are not updated during the training of the action decision network, thus maintaining the correspondence between the output and input. The environment prediction network can be a trainable neural network that learns from historical environment parameters and adjusts its parameters based on these parameters, making its output close to the output of the fixed-map network.

[0051] Understandably, when the drone is in a known environment, the output of the environment prediction network is similar to that of the fixed mapping network; however, when the drone is in an unknown environment, due to the lack of knowledge generalization ability, the output of the environment prediction network for the unknown environment differs significantly from that of the fixed mapping network. Therefore, based on the output difference between the environment prediction network and the fixed mapping network, the novelty of the drone's environment can be evaluated to obtain an environment difference reward term to incentivize the drone to explore the unknown environment.

[0052] In some embodiments, the environmental difference reward item can be represented as in, For environmental differences, a reward item is given. Represents a fixed mapping network; Represents an environmental prediction network. These are the trainable parameters for the environmental prediction network.

[0053] Reference Figure 5 As shown, in one embodiment of this application, the drone corresponds to a mission objective; a reward function is obtained based on environmental difference reward terms, including: S242. Calculate the proximity between the UAV and the mission target to obtain a target proximity reward item used to incentivize the UAV to approach the mission target.

[0054] S244. Judge the drone's capture result of the mission target based on the drone's flight status, and obtain a target-oriented reward item to incentivize the drone to accurately capture the mission target based on the judgment result.

[0055] S246. Calculate the delay penalty based on the time taken by the drone to perform the mission, and obtain the time cost penalty item used to suppress the drone from delaying behavior during the mission.

[0056] S248. Obtain the reward function based on the environmental difference reward, the goal proximity reward, the goal-oriented reward, and the time cost penalty.

[0057] Specifically, the drone's task may be to track or detect a target. In these tasks, the drone typically needs to approach the target and bring it within its field of view to accurately capture it. In this embodiment, the proximity between the drone and the target is calculated to obtain the physical distance between them at the current moment. This distance is then combined with the physical distance at the previous moment to obtain a target proximity reward. This reward incentivizes the drone to gradually approach the target over time. For example, the target proximity reward can be represented as: in, To get closer to the target reward; The distance between the drone and the mission target at the previous moment. This represents the physical distance between the drone and the mission target at the current moment.

[0058] Furthermore, based on the physical distance between the UAV and the mission target, the UAV's pose is calculated according to its current flight state to determine its field of view orientation. Based on the UAV's field of view orientation and the physical distance between the UAV and the mission target, the relative positional relationship between the two is determined, resulting in a target-oriented reward item. This reward incentivizes the UAV to approach and accurately capture the mission target. In some embodiments, the target-oriented reward item includes an indicator function for physical distance and an indicator function for field of view orientation. The indicator function for physical distance is set to 1 if the physical distance between the UAV and the mission target is less than a distance threshold, and 0 otherwise. The indicator function for field of view orientation is set to 1 if the angular difference between the UAV's field of view orientation and the relative direction between the UAV and the mission target is less than an angle threshold, and 0 otherwise. When both the physical distance between the UAV and the mission target is less than the distance threshold, and the angular difference between the UAV's field of view orientation and the relative direction between the UAV and the mission target is less than an angle threshold, both indicator functions are set to 1, and a reward is given to the UAV.

[0059] Furthermore, based on the time taken by the drone to perform the mission, a negative penalty is applied to each time step to obtain a time cost penalty term, which is used to punish the drone for failing to complete the mission after the time has elapsed, thereby suppressing the drone from delaying the mission.

[0060] For example, the external reward for the drone, derived from the target proximity reward, the target orientation reward, and the time cost penalty, can be expressed in the following form: in, External rewards; For goal-oriented reward items, For the indication function of physical distance, This is an indicator function for the orientation of the field of view. Rewards are based on objectives; This represents the time cost penalty. Based on external rewards and environmental difference rewards, the reward function can be obtained, which can be expressed as: in, For the reward function; and These are the balancing weighting coefficients.

[0061] Reference Figure 6 The example shown is an embodiment of this application. Based on environmental perception data, the optimal action, and its corresponding future cumulative reward value, the action decision network and its reward function are updated empirically, including: S410. Update the parameters of the action decision network based on the optimal action and its corresponding future cumulative reward value to obtain the updated action decision network, which can be used to propose actions for the drone in subsequent time steps.

[0062] S420. Update the parameters of the environmental prediction network based on the environmental perception data to obtain the updated environmental prediction network, which can be used to determine the optimal actions of the UAV at subsequent time points.

[0063] Specifically, the action decision network is updated with parameter guidance based on the optimal action and its corresponding future cumulative reward value, resulting in an updated action decision network. This improves the quality of the actions output by the policy network, which can then be used to propose actions for the drone in subsequent time steps.

[0064] Furthermore, by using the current environmental perception data as historical environmental data, the parameters of the environmental prediction network are updated based on the updated historical environmental data, enabling the UAV to autonomously adapt to changes in its environment and perform adaptive autonomous navigation, thereby improving the UAV's adaptability to the environment.

[0065] Reference Figure 7 As shown, in one embodiment of this application, the environmental perception data is obtained based on multimodal data of the environment in which the UAV is located. The multimodal data includes color images and depth images of the environment in which the UAV is located. The environmental perception data is obtained in the following manner: S110. Perform image segmentation and feature embedding on the color image to obtain a color image embedding sequence; perform image segmentation and feature embedding on the depth image to obtain a depth image embedding sequence.

[0066] S120. Perform modal cross-fusion on the color image embedding sequence and the depth image embedding sequence, and encode the fusion result to obtain environmental perception data.

[0067] Specifically, the environmental features of the environment in which the UAV is located in the environmental perception data can be obtained from color and depth images of the environment in which the UAV is located. For the color images of the environment collected by the UAV, the color images are segmented into multiple non-overlapping image patches, and each image patch is converted into a feature embedding vector through linear projection to obtain a color image embedding sequence. Let N represent the color information of the environment in which the UAV is located, where N is the total number of image patches and D is the vector embedding dimension. Similarly, for the depth image of the environment acquired by the UAV, the depth image is divided into multiple non-overlapping image patches, and each image patch is converted into a feature embedding vector through linear projection, resulting in a depth image embedding sequence. This is used to represent the depth information of the environment in which the drone is located.

[0068] Furthermore, modal cross-fusion is performed on the color image embedding sequence and the depth image embedding sequence to fuse the color and depth information of the UAV's environment, obtaining fused information about the UAV's environment. Based on this, the fused information of the UAV's environment is encoded and output to obtain the environmental features of the UAV's environment, thus obtaining environmental perception data. For example, the process of fusing color information into depth information through modal cross-fusion can be represented as follows: Among them, query vector Key vector Value vector ; , and These are the learnable weight matrices; is the dimension of the key vector.

[0069] Reference Figure 8As shown in one embodiment of this application, the environmental perception data includes a semantic map of the environment in which the UAV is located; determining the optimal action of the UAV at the current moment based on the future cumulative reward value of candidate actions includes: S310. Determine the proposed action based on the future cumulative reward value of the candidate action, and apply a preset interference to the proposed action to obtain multiple derivative actions; wherein, each of the multiple derivative actions corresponds to its own derivative trajectory.

[0070] S320. Based on the obstacle distribution in the environment where the UAV is located, perform a geometric risk assessment on any derived trajectory to obtain the geometric risk data of any derived trajectory.

[0071] S330. Based on the distribution of semantic objects in the semantic map, perform semantic risk assessment on any derived trajectory to obtain semantic risk data for any derived trajectory.

[0072] S340. Obtain risk assessment data for any derived trajectory based on geometric risk data and semantic risk data, and determine the optimal trajectory based on the risk assessment data of all derived trajectories; wherein, the derived action corresponding to the optimal trajectory is the optimal action.

[0073] The semantic map of the drone's environment can be obtained from a color image of the drone's environment. By performing real-time semantic segmentation on the color image of the drone's environment, semantic segmentation and labeling are performed on each pixel in the color image, thereby obtaining a semantic map of the same size as the color image, which represents the distribution of different semantic objects in the drone's environment.

[0074] Reference Figure 9 As shown, Figure 9 This is a top-down view of the scene where the drone is located. With the objective of maximizing the future cumulative reward value, proposed actions are determined based on the future cumulative reward values ​​of candidate actions. Preset perturbations are then applied to these proposed actions to obtain multiple derivative actions. The preset perturbations can be small disturbances applied to the speed or angular velocity set by the proposed actions, allowing the drone to fly along different trajectories. After obtaining the derivative actions, kinematic forward integration is performed on them within a preset time window to generate a derivative trajectory corresponding to each derivative action.

[0075] Furthermore, the obstacle distribution in the drone's environment can be obtained from a depth image of that environment. For any derived trajectory, the distance between any position on that trajectory and the nearest obstacle in the drone's environment is calculated in the depth image. Based on this distance, a geometric risk assessment is performed on that derived trajectory to obtain its geometric risk data. The geometric risk data can be represented as: in, Geometric risk data; For any derived trajectory, For any position on any of the derived trajectories; The distance between any location and the nearest obstacle in the drone's environment; This is the safe distance threshold between the drone and the obstacle.

[0076] Furthermore, for any derived trajectory, it is projected onto the semantic map to obtain the projected trajectory. Based on the projected trajectory, the overlap between the UAV and semantic objects in the semantic map is estimated, thereby performing a semantic risk assessment on any derived trajectory to obtain its semantic risk data. The semantic risk data can be represented as: in, This is semantic risk data; The semantic object label corresponding to any position on the derived trajectory; This refers to the risk weight corresponding to the semantic object label. For example, when the semantic object label is "person," the risk weight is... When the semantic object label is "wall", the risk weight is... When the semantic object label is "floor", the risk weight is... .

[0077] Furthermore, risk fusion is performed on the geometric risk data and semantic risk data to obtain the risk assessment data for any derived trajectory. For example, the risk fusion can be achieved by weighted summation of the geometric risk data and semantic risk data, in which case the risk assessment data can be represented as: in, For risk assessment data; and where is the weighting coefficient. After obtaining the risk assessment data for all derived trajectories, the derived trajectories are filtered based on the risk assessment data, and the derived trajectory with the lowest risk assessment data is selected as the optimal trajectory. The optimal trajectory can be expressed as: in, The optimal trajectory is determined. The derived actions corresponding to the optimal trajectory are output as the optimal actions to control the drone in performing its mission.

[0078] In some embodiments, a risk safety threshold can be set for the risk assessment data to compare the risk assessment data of all derived trajectories with the risk safety threshold. If the risk assessment data of all derived trajectories is higher than the risk safety threshold, it indicates that all derived trajectories pose a high risk and the optimal action cannot be obtained. In this case, the drone can be forced to output an emergency maneuver of hovering and rotating in place, causing the drone to move away from its surroundings and avoid safety risks.

[0079] Accordingly, please refer to Figure 10 This application provides an autonomous navigation device for unmanned aerial vehicles (UAVs), the device comprising: The environmental perception module 1010 is used to acquire environmental perception data of the environment in which the UAV is located at the current moment.

[0080] The action proposal module 1020 is used to propose actions based on environmental perception data through the action decision network to obtain candidate actions for the UAV. The reward function of the action decision network is obtained based on the feature difference between the current environment of the UAV and the predicted environment, and the predicted environment is obtained based on the historical environmental data of the UAV.

[0081] The action decision module 1030 is used to determine the optimal action of the drone at the current moment based on the future cumulative reward value of the candidate actions, so as to control the drone to perform the task.

[0082] The experience update module 1040 is used to update the action decision network and its reward function based on environmental perception data, optimal actions and their corresponding future cumulative reward values, so as to propose actions for the UAV and determine the optimal actions in subsequent time steps of the current time.

[0083] In some alternative implementations, the action proposal module 1020 includes: The fixed feature mapping unit is used to perform fixed feature mapping on the environment in which the UAV is located based on environmental perception data, so as to obtain the mapped environmental features of the environment in which the UAV is located.

[0084] The environmental feature prediction unit is used to predict the environmental features of the environment in which the UAV is located based on the UAV's historical environmental data, and obtain the predicted environmental features of the environment in which the UAV is located.

[0085] The feature difference calculation unit is used to calculate the difference based on the mapped environment features and the predicted environment features to obtain an environment difference reward item to incentivize UAVs to explore the unknown environment.

[0086] The reward function acquisition unit is used to obtain the reward function based on the environmental difference reward items.

[0087] In some alternative implementations, the fixed feature mapping unit includes: The fixed mapping network subunit is used to input environmental perception data into the fixed mapping network, and then perform fixed feature mapping based on the environmental perception data through the fixed mapping network to output mapped environmental features.

[0088] The environmental feature prediction unit includes: The environmental prediction network subunit is used to predict the environmental characteristics of the environment in which the UAV is located through the environmental prediction network, and obtain the environmental prediction characteristics; wherein, the environmental prediction network is obtained by training on the mapping output of historical environmental data based on the fixed mapping network.

[0089] In some optional implementations, the reward function acquisition unit includes: The proximity calculation subunit is used to calculate the proximity between the UAV and the mission target, and to obtain a target proximity reward item to incentivize the UAV to approach the mission target.

[0090] The capture result judgment subunit is used to judge the capture result of the UAV on the mission target based on the flight status of the UAV, and to obtain a target-oriented reward item to incentivize the UAV to accurately capture the mission target based on the judgment result.

[0091] The delay penalty calculation subunit is used to calculate the delay penalty based on the time taken by the drone to perform the task, and obtain the time cost penalty item used to suppress the drone's delay behavior during the task.

[0092] The reward function acquisition sub-unit is used to obtain the reward function based on environmental difference reward items, target proximity reward items, target-oriented reward items, and time cost penalty items.

[0093] In some alternative implementations, the experience update module 1040 includes: The decision network update unit is used to update the parameters of the action decision network based on the optimal action and its corresponding future cumulative reward value, so as to obtain the updated action decision network for making action proposals for the drone in subsequent time steps.

[0094] The prediction network update unit is used to update the parameters of the environmental prediction network based on the environmental perception data, so as to obtain the updated environmental prediction network for determining the optimal action of the UAV in subsequent time steps.

[0095] In some alternative implementations, the environment sensing module 1010 includes: The image processing unit is used to perform image segmentation and feature embedding on color images to obtain color image embedding sequences; and to perform image segmentation and feature embedding on depth images to obtain depth image embedding sequences.

[0096] The cross-fusion unit is used to perform modal cross-fusion of color image embedding sequences and depth image embedding sequences, and encodes the fusion result to obtain environmental perception data.

[0097] In some alternative implementations, the action decision module 1030 includes: A preset interference application unit is used to determine the proposed action based on the future cumulative reward value of the candidate action, and apply preset interference to the proposed action to obtain multiple derivative actions; wherein, each of the multiple derivative actions corresponds to its own derivative trajectory.

[0098] The geometric risk assessment unit is used to assess the geometric risk of any derived trajectory based on the obstacle distribution in the environment in which the UAV is located, and to obtain the geometric risk data of any derived trajectory.

[0099] The semantic risk assessment unit is used to assess the semantic risk of any derived trajectory based on the distribution of semantic objects in the semantic map, and obtain the semantic risk data of any derived trajectory.

[0100] The optimal trajectory determination unit is used to obtain risk assessment data for any derived trajectory based on geometric risk data and semantic risk data, and to determine the optimal trajectory based on the risk assessment data of all derived trajectories; wherein, the derived action corresponding to the optimal trajectory is the optimal action.

[0101] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0102] In this embodiment, the UAV autonomous navigation device is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0103] Please see Figure 11 , Figure 11This is a schematic diagram of a computer device according to an embodiment of this application. As shown in the figure, the computer device includes one or more processors 10, a memory 20, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components communicate with each other using different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some alternative embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 11 Take a processor 10 as an example.

[0104] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0105] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0106] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, which can be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0107] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0108] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0109] This application also provides a computer-readable storage medium. The methods described in this application can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the methods shown in the above embodiments are implemented.

[0110] This application provides a computer program product including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method of any embodiment of this application.

[0111] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and all such modifications and variations fall within the scope defined by the appended claims.

[0112] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0113] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0114] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0115] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0116] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0117] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0118] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0119] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0120] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

[0121] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. An autonomous navigation method for unmanned aerial vehicles (UAVs), characterized in that, The method includes: Obtain environmental perception data of the environment in which the drone is located at the current moment; The action decision network proposes actions based on the environmental perception data to obtain candidate actions for the UAV. The reward function of the action decision network is obtained based on the feature differences between the current environment of the UAV and the predicted environment, where the predicted environment is obtained based on the UAV's historical environmental data. The reward function is obtained as follows: Based on the environmental perception data, a fixed feature mapping is performed on the environment of the UAV to obtain the mapped environmental features; based on the UAV's historical environmental data, environmental feature prediction is performed on the environment of the UAV to obtain the predicted environmental features; the difference between the mapped environmental features and the predicted environmental features is calculated to obtain an environmental difference reward term to incentivize the UAV to explore the unknown environment; and the reward function is obtained based on the environmental difference reward term. Based on the future cumulative reward value of the candidate actions, the optimal action of the drone at the current moment is determined for controlling the drone to perform the task; Based on the environmental perception data, the optimal action and its corresponding future cumulative reward value, the action decision network and its reward function are updated empirically to propose actions for the UAV and determine the optimal action in subsequent time moments of the current time.

2. The method according to claim 1, characterized in that, The mapped environment features and the predicted environment features are obtained in the following ways: The environmental perception data is input into a fixed mapping network, and the fixed mapping network performs fixed feature mapping based on the environmental perception data to output the mapped environmental features. The environment features of the UAV are predicted by an environment prediction network, and the predicted environment features are obtained; wherein, the environment prediction network is obtained by training the mapping output of the historical environment data based on the fixed mapping network.

3. The method according to claim 1, characterized in that, The drone corresponds to a mission objective; the process of obtaining the reward function based on the environmental difference reward item includes: The proximity between the drone and the mission target is calculated to obtain a target proximity reward item used to incentivize the drone to approach the mission target. The flight status of the UAV is used to determine the UAV's capture result of the mission target, and a target-oriented reward item is obtained based on the determination result to incentivize the UAV to accurately capture the mission target. Based on the time taken by the drone to perform the task, a delay penalty is calculated to obtain a time cost penalty item used to suppress the drone from delaying behavior during the task. The reward function is obtained based on the environmental difference reward, the target proximity reward, the target orientation reward, and the time cost penalty.

4. The method according to claim 2, characterized in that, The step of updating the action decision network and its reward function based on the environmental perception data, the optimal action, and its corresponding future cumulative reward value includes: The parameters of the action decision network are updated based on the optimal action and its corresponding future cumulative reward value to obtain an updated action decision network, which is used to propose actions for the drone at the subsequent time. The environmental prediction network is updated with parameters based on the environmental perception data to obtain an updated environmental prediction network, which is then used to determine the optimal action for the UAV at the subsequent time.

5. The method according to claim 1, characterized in that, The environmental perception data is obtained based on the multimodal data of the environment in which the UAV is located, including color images and depth images of the environment in which the UAV is located. The environmental perception data is obtained through the following methods: The color image is segmented and its features are embedded to obtain a color image embedding sequence; the depth image is segmented and its features are embedded to obtain a depth image embedding sequence. The color image embedding sequence and the depth image embedding sequence are cross-fused using modality, and the fusion result is encoded and output to obtain the environmental perception data.

6. The method according to claim 1, characterized in that, The environmental perception data includes a semantic map of the environment in which the UAV is located; determining the optimal action of the UAV at the current moment based on the future cumulative reward value of the candidate actions includes: Based on the future cumulative reward value of the candidate actions, a proposed action is determined, and a preset interference is applied to the proposed action to obtain multiple derivative actions; wherein, each of the multiple derivative actions corresponds to its own derivative trajectory; Based on the obstacle distribution in the environment where the UAV is located, a geometric risk assessment is performed on any derived trajectory to obtain the geometric risk data of any derived trajectory; Based on the distribution of semantic objects in the semantic map, a semantic risk assessment is performed on any of the derived trajectories to obtain the semantic risk data of any of the derived trajectories; Risk assessment data for any derived trajectory is obtained based on the geometric risk data and the semantic risk data, and the optimal trajectory is determined based on the risk assessment data of all derived trajectories; wherein, the derived action corresponding to the optimal trajectory is the optimal action.

7. An autonomous navigation device for unmanned aerial vehicles (UAVs), characterized in that, The device includes: The environmental perception module is used to acquire environmental perception data of the environment in which the UAV is located at the current moment; An action proposal module is used to propose actions based on the environmental perception data through an action decision network to obtain candidate actions for the UAV. The reward function of the action decision network is obtained based on the feature differences between the current environment of the UAV and the predicted environment, where the predicted environment is obtained based on the UAV's historical environmental data. The reward function is obtained through the following methods: mapping fixed features to the environment of the UAV based on the environmental perception data to obtain mapped environmental features; predicting environmental features of the environment based on the UAV's historical environmental data to obtain predicted environmental features; calculating the difference between the mapped environmental features and the predicted environmental features to obtain an environmental difference reward term to incentivize the UAV to explore the unknown environment; and obtaining the reward function based on the environmental difference reward term. The action decision module is used to determine the optimal action of the UAV at the current moment based on the future cumulative reward value of the candidate actions, so as to control the UAV to perform the task; The experience update module is used to update the action decision network and its reward function based on the environmental perception data, the optimal action and its corresponding future cumulative reward value, so as to propose actions for the UAV and determine the optimal action in subsequent time moments of the current time.

8. A computer device, characterized in that, include: A memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, the processor executing the computer instructions to perform the method of any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the method of any one of claims 1 to 6.

Images