Unmanned aerial vehicle simulation system, method and device and electronic equipment

Abstract

The invention provides an unmanned aerial vehicle simulation system, method and device and electronic equipment, and the system comprises a simulation control module, an environment perception module and a decision learning module. The simulation control module is used for initializing an initial environment of the to-be-simulated unmanned aerial vehicle and performing motion control on the to-be-simulated unmanned aerial vehicle; the environment sensing module is used for collecting a plurality of images in the flight process of the to-be-simulated unmanned aerial vehicle and fusing image features in the plurality of images to obtain visual feature information; and the decision learning module is used for training a decision control action model according to the initial environment and the visual feature information, and the decision control action model is used for identifying the current environment and the corresponding visual features of the to-be-simulated unmanned aerial vehicle to obtain decision control actions. Through the system, simulation acceleration of the unmanned aerial vehicle can be promoted, and the autonomous exploration navigation effect of the unmanned aerial vehicle is realized.

Description

Technical Field

[0001] The present application relates to the field of drone simulation, and more specifically, to a drone simulation system, method, device and electronic equipment. Background Art

[0002] With the widespread application of drone technology in military, civil and scientific research fields, the demand for simulation training and testing of drones is growing. The traditional drone simulation training system, with its unique advantages, has brought revolutionary changes to the simulation training and testing of drone technology. It provides a highly realistic simulation environment based on a game engine, covering all possible flight scenarios from city streets to complex mountains, from extreme weather to special environments. In terms of real-time interactivity, by building an efficient data transmission and synchronization mechanism, it ensures seamless connection between the simulation environment and the actual drone control system. By interacting with the drone's sensors and control system, it can achieve real-time simulation of the drone's flight behavior.

[0003] However, although the traditional method provides such a powerful simulation scenario, it also limits the simulation speed of the UAV. When in a complex environment, it seriously affects the path exploration of the UAV and delays the simulation speed.

[0004] Therefore, how to promote the acceleration of UAV simulation and realize autonomous exploration and navigation of UAV is a technical problem that needs to be solved. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a drone simulation system. Through the technical solutions of the embodiments of the present application, it is possible to promote the acceleration of drone simulation and realize autonomous exploration and navigation of the drone.

[0006] In the first aspect, an embodiment of the present application provides a drone simulation system, including a simulation control module, an environment perception module and a decision learning module; the simulation control module is used to initialize the initial environment of the drone to be simulated and the motion control of the drone to be simulated; the environment perception module is used to collect multiple images of the drone to be simulated during flight, and fuse the image features in the multiple images to obtain visual feature information; the decision learning module is used to train a decision control action model based on the initial environment and visual feature information, wherein the decision control action model is used to identify the current environment of the drone to be simulated and the corresponding visual features to obtain a decision control action.

[0007] In the above system, the present application obtains environmental-related information and visual feature information of the drone in different environments and corresponding scene graphs, continuously trains the decision-making control action model in the drone simulation system, realizes the autonomous exploration and navigation tasks of the drone, and at the same time can promote the acceleration of drone simulation.

[0008] In some embodiments, it also includes:

[0009] The decision module is used to identify the current environment and corresponding visual features of the UAV to be simulated through the decision control action model, and obtain the decision control action.

[0010] In the above embodiments of the present application, the drone obtains current environmental information and image visual features, makes autonomous decisions, and achieves the task of autonomous navigation.

[0011] In some embodiments, the simulation control module is specifically used to:

[0012] Defining an initial environment of the drone to be simulated, wherein the initial environment includes: at least one of an initial position, a state space, an action space, and an observation space;

[0013] Control the movement and state of the drone to be simulated.

[0014] In the above embodiments of the present application, the initial environment of the drone can be reset and the drone simulation process can be controlled through the simulation control module, ensuring that each drone simulation can be in the initial environment.

[0015] In some embodiments, the environment perception module is specifically used to:

[0016] Collect multiple images during the flight of the UAV to be simulated, wherein the multiple images include a color image of the region of interest, a target segmentation image, a depth image, and a surface method image;

[0017] The image features corresponding to multiple images are extracted and fused through the visual fusion network to obtain visual feature information.

[0018] In the above embodiments of the present application, by extracting and fusing the features of images of different scenes, the drone can autonomously make corresponding decisions and perform autonomous navigation according to different scenes.

[0019] In some embodiments, the decision learning module is specifically used to:

[0020] Input the initial environment and visual features into the main network and the target network to obtain the probabilities of different decision-making control actions;

[0021] Determine the model loss corresponding to the probability of the target decision control action, update the main network and the target network, and obtain the decision control action model.

[0022] In the above embodiments of the present application, by adjusting the parameters of the main network and the target network, the decision control action model can be converged so that the decision control action model can guide the autonomous navigation of the drone in different scenarios and environments.

[0023] In the second aspect, an embodiment of the present application provides a drone simulation method, comprising: obtaining the current environment of the drone to be simulated and multiple images collected during the flight, wherein the multiple images include: a color image of the area of ​​interest, a target segmentation image, a depth image, and a surface method image; identifying the current environment and the multiple images through a preset decision control action model to obtain a decision control action; and controlling the flight of the drone to be simulated according to the decision control action.

[0024] In the above-mentioned embodiments of the present application, the current environment and the corresponding scene graph are identified through the decision-making control action model, so that the UAV can make decisions according to the current scene, realize the autonomous exploration and navigation tasks of the UAV, and promote the acceleration of UAV simulation.

[0025] In some embodiments, before obtaining the current environment of the UAV to be simulated and multiple images collected during the flight, it also includes: initializing the initial environment of the UAV to be simulated; collecting images of the UAV to be simulated during the flight, and fusing the image features in the collected images to obtain visual feature information; training a decision control action model based on the initial environment and visual feature information.

[0026] In the above-mentioned embodiments of the present application, environmental-related information and visual feature information of the UAV are obtained in different environments and corresponding scene graphs, and the decision-making control action model in the UAV simulation system is continuously trained to realize the autonomous exploration and navigation tasks of the UAV, while promoting the acceleration of UAV simulation.

[0027] In some embodiments, the current environment and multiple images are identified through a preset decision control action model to obtain a decision control action, including: extracting and fusing image features corresponding to multiple images to obtain visual features; identifying visual features and the current environment through a decision control action model to obtain a decision control action, wherein the current environment includes the current position information of the drone to be simulated.

[0028] In the above embodiments of the present application, through the visual features of the scene image, the decision can be directly made through the decision control action model to achieve the autonomous navigation task of the drone.

[0029] In a third aspect, an embodiment of the present application provides a drone simulation device, comprising:

[0030] An acquisition module is used to acquire the current environment of the UAV to be simulated and multiple images collected during the flight process, wherein the multiple images include: a color image of the region of interest, a target segmentation image, a depth image, and a surface method image;

[0031] A decision module, used to identify the current environment and multiple images through a preset decision control action model to obtain a decision control action;

[0032] The control module is used to control the actions according to the decisions and control the flight of the UAV to be simulated.

[0033] Optionally, the device further comprises:

[0034] A training module, used for initializing the initial environment of the UAV to be simulated before the acquisition module acquires the current environment of the UAV to be simulated and a plurality of images collected during the flight;

[0035] Collect images of the simulated UAV during flight, and fuse the image features in the collected images to obtain visual feature information;

[0036] The decision-making control action model is trained based on the initial environment and visual feature information.

[0037] Optionally, the decision module is specifically used for:

[0038] Extract and fuse the image features corresponding to multiple images to obtain visual features;

[0039] The decision control action model is used to identify visual features and the current environment to obtain the decision control action, wherein the current environment includes the current position information of the UAV to be simulated.

[0040] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps in the method provided in the second aspect are executed.

[0041] In a fifth aspect, an embodiment of the present application provides a readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in the method provided in the second aspect are executed.

[0042] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the embodiments of the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1A flowchart of a drone simulation system provided in an embodiment of the present application;

[0045] Figure 2 A decision learning logic diagram of a decision control action model provided in an embodiment of the present application;

[0046] Figure 3 A flowchart of a drone simulation method provided in an embodiment of the present application;

[0047] Figure 4 A schematic block diagram of a drone simulation device provided in an embodiment of the present application;

[0048] Figure 5 A schematic block diagram of the structure of a drone simulation device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0050] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0051] First, some terms involved in the embodiments of the present application are explained to facilitate understanding by those skilled in the art.

[0052] AirSim is an open source cross-platform simulator based on a game engine, mainly used for physical and visual simulation of robots such as drones and unmanned vehicles.

[0053] DQN (Deep Q-Network) is a method that combines deep learning and reinforcement learning, mainly used to solve problems with high-dimensional observation space. DQN uses neural networks to approximate the Q-value function, so that it can learn the optimal strategy in high-dimensional state space and action space.

[0054] The VF-DQN algorithm is a reinforcement learning algorithm that combines visual fusion and DQN (Deep Q-Network). The core of the VF-DQN algorithm is to use neural networks to approximate the Q-value function, thereby processing high-dimensional state spaces, and improve the stability and efficiency of learning and training through technologies such as experience replay and fixed target networks.

[0055] The Unreal Engine rendering engine is a highly modular and extensible game development framework, mainly used to create high-quality 3D games, virtual reality (VR), augmented reality (AR) applications, simulations and visualization projects.

[0056] API (Application Programming Interface) is a set of predefined functions that aims to provide applications and developers with the ability to access a set of routines based on certain software or hardware without having to access the source code or understand the details of the internal working mechanism.

[0057] The Gym interface is a set of functions and methods defined in the OpenAI Gym library for interacting with reinforcement learning environments. The core of the Gym interface is the Env class, which is the base class for all environments and provides a unified operation interface, allowing developers to interact with different environments in a standardized way.

[0058] The present application is applied to the scenario of drone simulation. The specific scenario is to train the decision model through the environmental information and scene graph information of the drone in different scenarios. The decision model can make decisions according to the drone in different scenarios and events, thereby achieving the effect of autonomous decision-making and navigation of the drone.

[0059] With the widespread application of drone technology in military, civil and scientific research fields, the demand for simulation training and testing of drones is growing. The traditional drone simulation training system (Airsim), with its unique advantages, has brought revolutionary changes to the simulation training and testing of drone technology. It provides a highly realistic simulation environment based on a game engine, covering various possible flight scenarios from urban streets to complex mountains, from extreme weather to special environments. In terms of real-time interactivity, by building an efficient data transmission and synchronization mechanism, it ensures seamless connection between the simulation environment and the actual drone control system. By interacting with the drone's sensors and control system, it can achieve real-time simulation of the drone's flight behavior. However, although the traditional method provides such a powerful simulation scenario, it also limits the simulation speed of the drone. When in a complex environment, it seriously affects the drone's path exploration and delays the simulation speed.

[0060] To this end, the present application provides a UAV simulation system, including: a simulation control module, an environment perception module and a decision learning module; the simulation control module is used to initialize the initial environment of the UAV to be simulated and the motion control of the UAV to be simulated; the environment perception module is used to collect multiple images of the UAV to be simulated during flight, and fuse the image features in the multiple images to obtain visual feature information; the decision learning module is used to train a decision control action model based on the initial environment and visual feature information, wherein the decision control action model is used to identify the current environment and corresponding visual features of the UAV to be simulated, and obtain a decision control action. The environment-related information and visual feature information of the UAV are obtained in different environments and corresponding scene graphs, and the decision control action model in the UAV simulation system is continuously trained to realize the autonomous exploration and navigation tasks of the UAV, and at the same time, the UAV simulation acceleration can be promoted.

[0061] In the embodiment of the present application, the execution entity may be a drone simulation device in a drone simulation system. In actual applications, the drone simulation device may be electronic devices such as terminal devices and servers, which are not limited here.

[0062] Combine the following Figure 1 The drone simulation system of an embodiment of the present application is described in detail.

[0063] Please see Figure 1 , Figure 1 A schematic block diagram of a drone simulation system 100 provided in an embodiment of the present application is shown in FIG. Figure 1 The UAV simulation system shown includes:

[0064] The simulation control module 110 is used to initialize the initial environment of the UAV to be simulated and the motion control of the UAV to be simulated; the environment perception module 120 is used to collect multiple images of the UAV to be simulated during flight, and fuse the image features in the multiple images to obtain visual feature information; the decision learning module 130 is used to train a decision control action model based on the initial environment and visual feature information, wherein the decision control action model is used to identify the current environment of the UAV to be simulated and the corresponding visual features to obtain a decision control action.

[0065] Among them, the multiple images can be images acquired by the front middle camera during the flight of the drone in different scenes, including RGB color images, target segmentation images, depth images and surface normal maps, etc. The simulation control module 110 is used to realize the control of the drone, including: obtaining status, position, reset, environment shutdown, etc.; the environmental perception module 120 mainly performs visual feature fusion, including multiple views in the drone motion scene, and the multi-dimensional features output through the feature fusion VF network are used to specify the decision process; the decision learning module 130 includes the main network, the target network and the experience playback, which improves the efficiency of autonomous learning training and improves the decision performance of the network model.

[0066] In some embodiments of the present application, Figure 1 The system shown also includes: a decision module, which is used to identify the current environment and corresponding visual features of the drone to be simulated through a decision control action model to obtain a decision control action.

[0067] In the above process, the present application obtains current environmental information and image visual features through the drone, can make autonomous decisions, and achieve the task of autonomous navigation.

[0068] In some embodiments of the present application, the simulation control module 110 is specifically used to: define the initial environment of the drone to be simulated, wherein the initial environment includes: at least one of an initial position, a state space, an action space, and an observation space; and control the movement and state of the drone to be simulated.

[0069] In the above process, the simulation control module 110 can be used to reset the initial environment of the drone and control the drone simulation process, ensuring that each drone simulation can be performed in the initial environment.

[0070] Among them, the construction of the initial environment can select the three-dimensional simulation environment for drone flight through the Airsim simulation platform. The urban environment built with the Unreal Engine rendering engine is used. The control and environmental information of the drone is obtained through the API provided by Airsim, and the drone environment construction is realized in combination with the Gym interface.

[0071] Optionally, initialize the environment and define the initial position, state space, action space, and observation space of the drone. The purpose of the initial position is to set the starting point of the drone's movement, so that it can return to the initial position after colliding with a building in subsequent movements and restart the exploration of path planning. The action space is a three-dimensional coordinate system with 6 actions: front and back, left and right, and up and down. The state space includes the position, speed, and direction of the drone. The observation space is to collect visual information obtained during the flight of the drone, which is used for decision-making and control action model training.

[0072] Optionally, the motion and state of the drone to be simulated can be controlled by the following control function:

[0073] F next =F(V x , V y , V z , D);

[0074] Among them, V x Used to control the speed of the drone in the X-axis direction. y and V z Control the speed of the drone in the Y and Z directions. The speed value can be set to 10m / s. D is the duration of the drone in a single movement. The time can be 0.5s.

[0075] Optionally, when controlling the motion and state of the simulated drone, the drone can return a reward value, which is calculated based on the reward function and is used to regulate the behavior of the drone. t Drones are encouraged to approach mission targets, while collisions during flight are punished. In order to prevent drones from exploring the movement path without limit before reaching the destination, the survival cost concept is adopted to limit the movement of drones. The distance between the current drone position and the destination is calculated. A larger reward is obtained when the destination is reached. The closer the distance to the destination is, the greater the reward is. A negative reward is obtained when an obstacle is encountered. Please refer to the following formula for reward calculation:

[0076]

[0077] Optionally, after initializing the initial environment of the simulated drone and the motion control of the simulated drone, it is also necessary to set the drone state to the initial state and return the state information. This usually happens at the beginning of each new experiment round. After the environment is reset, it returns to the initial state and usually resets the environment information, such as accumulated rewards, action history, etc.

[0078] In some embodiments of the present application, the environmental perception module 120 is specifically used to: collect multiple images during the flight of the drone to be simulated, wherein the multiple images include a color image of the area of ​​interest, a target segmentation image, a depth image, and a surface method image; extract and fuse image features corresponding to the multiple images through a visual fusion network to obtain visual feature information.

[0079] In the above process, the present application extracts and fuses the features of images of different scenes, so that the drone can autonomously make corresponding decisions and autonomously navigate according to different scenes.

[0080] Among them, RGB (region of interest) color images are widely used in various display fields to simulate the human visual system. They are also a common data input format in model training. The Airsim simulation environment also provides segmentation maps, depth maps, and surface normal maps. For segmentation maps, different colors in the map represent different objects, which are divided into different regions to identify different objects, understand the structure and characteristics of the scene, and help drones better understand the surrounding environment, so as to make more reasonable flight decisions and avoid collisions. Depth maps can better show the distance. For example, 100 meters corresponds to white and 0 meters corresponds to black. It can help identify the occlusion relationship between objects. The supplementary information such as structure and distance provided can become an important auxiliary content for drones in exploring scenes with low contrast and complex backgrounds. Surface normal images are an important content in three-dimensional graphics. The surface normal of an object is the direction of the outer normal pointing to the surface, that is, the direction perpendicular to the surface. It can better understand the geometric characteristics of the surface of an object and provide relatively rich detailed information.

[0081] Optionally, the image perceived by the drone during flight is used to obtain high-dimensional feature information through a feature fusion VF network, which is used as important feature data in the decision learning module 130 network. The feature fusion VF network structure includes four dimensional feature views (F Scene 、F Seg 、F Dep 、F Surf ) are collectively referred to as F i As input, a channel dimension concatenation strategy is adopted to form a new global feature map F All This strategy enables the concatenated features to retain rich information in each view, taking into account both details and perceptual range, which helps improve the performance of the model, as follows:

[0082] F All =Concat(F Scene 、F Seg 、F Dep 、F Surf );

[0083] Global feature map F All The extracted feature map F can be obtained by 5×5 convolution, ReLU activation function and two 3×3 convolution functions respectively. local , through convolution and activation function calculation, learn the important local feature information in the global feature map, enhance the nonlinear expression ability of the global feature, reduce the computing resource consumption of the model, and accelerate training, as follows:

[0084] F local =Conv 3×3 (ReLU(Conv 3×3 (ReLU(Conv5×5 (F All ))_));

[0085] Among them, when the scene graphs and environmental information in different scenarios and corresponding UAV flight missions are accelerated and optimized by the above method, the decision-making control action model can make decisions according to the current environment and scene graph to achieve autonomous navigation.

[0086] In some embodiments of the present application, the decision learning module 130 is specifically used to: input the initial environment and visual features into the main network and the target network to obtain the probabilities of different decision control actions; determine the model loss corresponding to the probability of the target decision control action, update the main network and the target network, and obtain the decision control action model.

[0087] In the above process, the present application can converge the decision-making control action model by adjusting the parameters of the main network and the target network, so that the decision-making control action model can guide the autonomous navigation of the drone in different scenarios and environments.

[0088] Optionally, the training process of the decision control action model can be obtained through the decision learning logic diagram of the decision control action model shown in 2.

[0089] Please see Figure 2 , Figure 2 A decision control action model decision learning logic diagram provided for this application includes:

[0090] Loss function, environment, main network Q ) , target network Q t and experience recycling.

[0091] Specifically, the main network Q m and the target network Q t , which is used to estimate the Q value of the current state and calculate the target Q value. The initial part of the network structure is the VF perception feature fusion module, which is used to calculate the feature map F local , then through three convolution calculations, and then through two fully connected layers, the Q value of all possible actions predicted by the current state is obtained. t Its network structure is similar to the main network Q m The same as the main network, but the parameter update lags behind the main network, which is used to calculate the target Q value and improve the training stability.

[0092] In addition, experience replay is used to store the experience data generated by the interaction between the model and the environment, including the current feature view, action, reward, and next feature view (F i ,a,r,F i'), during training, a certain amount of experience data is randomly extracted from the experience pool for training, so that the network model can be continuously optimized.

[0093] The decision learning process is as follows: First, initialize the model and initialize the main network Q m and the target network Q t The parameters of Q m The parameters are copied to Q t At the beginning of the training, the simulated UAV is designated to fly to the initial setting position (D x , D y , D z ), and obtain the visual feature information F of the current position i This step is to ensure exploration, so that the algorithm starts from a unified position in each experimental round and gradually learns to achieve convergence. i Through the main network Q m Calculate the probability Q of all actions m (F i , a, θ), θ represents the network parameters, and the action a is obtained through the argmax function m . Execute the current action a m , update one step to reach the new position and obtain the new feature view F o ' and reward value r, and get an experience data (F i ,a,r,F i '), put it into the experience replay pool; extract experience data from the experience pool (F i ,a,r,F i '), F i 'Input to the target network Q t The probability Q of all actions is calculated t (F j ', a', θ), take the maximum Q value and reward value r as the calculation data of the real action value, Q = r + γ * max (Q t (F i ', a′, θ)), the coefficient γ represents the discount factor. Finally, Smooth L1 is used as the loss function to calculate Q m (F i , a, θ) and the loss of Q, and the gradient information calculated according to the loss function guides the main network Q m and the target network Q t Parameter update, where Q is performed every preset number of training steps (e.g. 500 steps) tThe replication of network parameters. After the intensive training of the decision learning module 130, the decision control action model of the simulated UAV can learn the decision control actions that should be taken in different scenarios and different events, and realize the autonomous exploration capability.

[0094] In the above Figure 1 In the process shown, the present application provides a UAV simulation system, including: a simulation control module 110, an environment perception module 120 and a decision learning module 130; the simulation control module 110 is used to initialize the initial environment of the UAV to be simulated and the motion control of the UAV to be simulated; the environment perception module 120 is used to collect multiple images during the flight of the UAV to be simulated, and fuse the image features in the multiple images to obtain visual feature information; the decision learning module 130 is used to train a decision control action model according to the initial environment and visual feature information, wherein the decision control action model is used to identify the current environment and corresponding visual features of the UAV to be simulated, and obtain a decision control action. The environment-related information and visual feature information of the UAV are obtained in different environments and corresponding scene graphs, and the decision control action model in the UAV simulation system is continuously trained to realize the autonomous exploration and navigation tasks of the UAV, and at the same time, the UAV simulation acceleration can be promoted.

[0095] Combine the following Figure 3 The drone simulation method of the embodiment of the present application is described in detail.

[0096] Please see Figure 3 , Figure 3 A flowchart of a drone simulation method provided in an embodiment of the present application is shown in FIG. Figure 3 The drone simulation method shown includes:

[0097] Step 310: Acquire the current environment of the UAV to be simulated and multiple images collected during the flight.

[0098] The multiple images include: a color image of an area of ​​interest, a target segmentation image, a depth image and a surface method image.

[0099] In some embodiments of the present application, before acquiring the current environment of the simulated drone and multiple images collected during the flight, Figure 3 The method shown also includes: initializing the initial environment of the UAV to be simulated; collecting images of the UAV to be simulated during flight, and fusing the image features in the collected images to obtain visual feature information; and training a decision control action model based on the initial environment and visual feature information.

[0100] In the above process, the present application obtains environmental-related information and visual feature information of the UAV in different environments and corresponding scene graphs, continuously trains the decision-making control action model in the UAV simulation system, realizes the autonomous exploration and navigation tasks of the UAV, and at the same time can promote the acceleration of UAV simulation.

[0101] Step 320: Identify the current environment and multiple images through a preset decision control action model to obtain a decision control action.

[0102] In some embodiments of the present application, the current environment and multiple images are identified through a preset decision control action model to obtain a decision control action, including: extracting and fusing image features corresponding to multiple images to obtain visual features; identifying visual features and the current environment through a decision control action model to obtain a decision control action, wherein the current environment includes the current position information of the drone to be simulated.

[0103] In the above process, the present application can realize the autonomous navigation task of the UAV by directly making decisions through the decision-making control action model through the visual features of the scene image.

[0104] Step 330: Control the flight of the simulated UAV according to the decision control action.

[0105] also, Figure 3 The specific methods and steps shown can be found in Figure 1 The system shown will not be described in detail here.

[0106] Previous article Figure 3 Describes the UAV simulation method, and combines it with Figure 4-Figure 5 Describe the drone simulation device.

[0107] Please refer to Figure 4 , is a schematic block diagram of a drone simulation device 400 provided in an embodiment of the present application, and the device 400 may be a module, program segment or code on an electronic device. Figure 2 The method embodiment corresponds to and can be executed Figure 2 The various steps involved in the method embodiment and the specific functions of the device 400 can be found in the description below. To avoid repetition, the detailed description is appropriately omitted here.

[0108] Optionally, the device 400 includes:

[0109] The acquisition module 410 is used to acquire the current environment of the UAV to be simulated and multiple images collected during the flight process, wherein the multiple images include: a color image of the region of interest, a target segmentation image, a depth image, and a surface method image;

[0110] A decision module 420, used to identify the current environment and multiple images through a preset decision control action model to obtain a decision control action;

[0111] The control module 430 is used to control the flight of the UAV to be simulated according to the decision control action.

[0112] Optionally, the device further comprises:

[0113] The training module is used for initializing the initial environment of the UAV to be simulated before the acquisition module acquires the current environment of the UAV to be simulated and multiple images collected during the flight; collecting images of the UAV to be simulated during the flight, and fusing the image features in the collected images to obtain visual feature information; and training the decision control action model according to the initial environment and the visual feature information.

[0114] Optionally, the decision module is specifically used for:

[0115] The image features corresponding to the multiple images are extracted and fused to obtain the visual features; the visual features and the current environment are identified through the decision control action model to obtain the decision control action, wherein the current environment includes the current position information of the UAV to be simulated.

[0116] Please refer to Figure 5 5 is a schematic block diagram of a UAV simulation device provided in an embodiment of the present application. The device may include a memory 510 and a processor 520. Optionally, the device may also include: a communication interface 530 and a communication bus 540. The device is similar to the above Figure 2 The method embodiment corresponds to and can be executed Figure 2 The various steps involved in the method embodiment and the specific functions of the device can be found in the description below.

[0117] Specifically, the memory 510 is used to store computer-readable instructions.

[0118] Processor 520, configured to process readable instructions stored in the memory, capable of executing Figure 2 The steps in the method.

[0119] The communication interface 530 is used for signaling or data communication with other node devices, for example, for communication with a server or a terminal, or for communication with other device nodes, but the embodiments of the present application are not limited thereto.

[0120] The communication bus 540 is used to realize direct connection and communication among the above components.

[0121] The communication interface 530 of the device in the embodiment of the present application is used to communicate signals or data with other node devices. The memory 510 can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The memory 510 can also be at least one storage device located away from the aforementioned processor. The memory 510 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 520, the electronic device executes the aforementioned Figure 2 The method process shown. The processor 520 can be used on the device 400 and used to perform the functions in the present application. Exemplarily, the above-mentioned processor 520 can be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and the embodiments of the present application are not limited thereto.

[0122] The embodiment of the present application also provides a readable storage medium, when the computer program is executed by a processor, Figure 2 The system process executed by the electronic device in the method embodiment is shown.

[0123] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method, and will not be described in detail here.

[0124] In summary, the embodiments of the present application provide a UAV simulation system, method, device and electronic device, the system comprising a simulation control module, an environment perception module and a decision learning module; the simulation control module is used to initialize the initial environment of the UAV to be simulated and the motion control of the UAV to be simulated; the environment perception module is used to collect multiple images of the UAV to be simulated during flight, and fuse the image features in the multiple images to obtain visual feature information; the decision learning module is used to train a decision control action model according to the initial environment and visual feature information, wherein the decision control action model is used to identify the current environment of the UAV to be simulated and the corresponding visual features to obtain a decision control action. This system can achieve the effect of promoting the acceleration of UAV simulation and realizing autonomous exploration and navigation of UAV.

[0125] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0126] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0127] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0128] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0129] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0130] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A drone simulation system, characterized in that: include: Simulation control module, environment perception module and decision learning module; The simulation control module is used to initialize the initial environment of the UAV to be simulated and the motion control of the UAV to be simulated; The environment perception module is used to collect multiple images of the simulated UAV during flight, and fuse the image features in the multiple images to obtain visual feature information; The decision learning module is used to train a decision control action model according to the initial environment and the visual feature information, wherein the decision control action model is used to identify the current environment of the simulated drone and the corresponding visual features to obtain a decision control action.

2. The system according to claim 1, characterized in that: Also includes: The decision module is used to identify the current environment and corresponding visual features of the unmanned aerial vehicle to be simulated through the decision control action model to obtain the decision control action.

3. The system according to claim 1 or 2, characterized in that: The simulation control module is specifically used for: Defining an initial environment of the drone to be simulated, wherein the initial environment includes: at least one of an initial position, a state space, an action space, and an observation space; Control the movement and state of the drone to be simulated.

4. The system according to claim 1 or 2, characterized in that: The environment perception module is specifically used for: Collecting multiple images of the simulated UAV during flight, wherein the multiple images include a color image of an area of ​​interest, a target segmentation image, a depth image, and a surface method image; The image features corresponding to the multiple images are extracted and fused through a visual fusion network to obtain the visual feature information.

5. The system according to claim 1 or 2, characterized in that: The decision learning module is specifically used for: Inputting the initial environment and the visual features into the main network and the target network to obtain the probabilities of different decision-making control actions; Determine the model loss corresponding to the probability of the target decision control action, update the main network and the target network, and obtain the decision control action model.

6. A drone simulation method, characterized in that: include: Acquire the current environment of the UAV to be simulated and multiple images collected during the flight, wherein the multiple images include: a color image of the region of interest, a target segmentation image, a depth image, and a surface method image; Identifying the current environment and the multiple images through a preset decision control action model to obtain a decision control action; According to the decision control action, the flight of the unmanned aerial vehicle to be simulated is controlled.

7. The method according to claim 6, characterized in that Before acquiring the current environment of the UAV to be simulated and a plurality of images collected during the flight, the method further includes: Initializing the initial environment of the unmanned aerial vehicle to be simulated; Collecting images of the unmanned aerial vehicle to be simulated during flight, and fusing image features in the collected images to obtain visual feature information; A decision-making control action model is trained according to the initial environment and the visual feature information.

8. The method according to claim 6 or 7, characterized in that: The identifying the current environment and the multiple images by using a preset decision control action model to obtain a decision control action includes: Extracting and fusing image features corresponding to the multiple images to obtain visual features; The decision control action is obtained by identifying the visual features and the current environment through the decision control action model, wherein the current environment includes the current position information of the unmanned aerial vehicle to be simulated.

9. A drone simulation device, characterized in that: include: An acquisition module is used to acquire the current environment of the UAV to be simulated and multiple images collected during the flight process, wherein the multiple images include: a color image of the region of interest, a target segmentation image, a depth image, and a surface method image; A decision module, used to identify the current environment and the multiple images through a preset decision control action model to obtain a decision control action; The control module is used to control the flight of the unmanned aerial vehicle to be simulated according to the decision control action.

10. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps in the method according to any one of claims 6 to 8 are executed.

Images