Unmanned aerial vehicle path planning method and system based on digital twinning

Abstract

The invention discloses an unmanned aerial vehicle path planning method and system based on digital twinning. The method comprises the following steps: constructing a three-dimensional environment model based on first target environment data in first target multi-source environment perception data; constructing a digital twin based on the geometric model, the physical model, the control model, the target physical data hybrid dynamic model and the three-dimensional environment model; inputting the second target multi-source environment sensing data into the digital twin, and dynamically simulating the flight behavior of the physical unmanned aerial vehicle based on the second target multi-source environment sensing data to obtain simulated flight environment state data; and based on the simulated flight environment state data, performing path planning on the physical unmanned aerial vehicle in combination with a prediction control algorithm and a path planning algorithm to obtain a target flight path, transmitting the target flight path to the physical unmanned aerial vehicle, and controlling the flight of the physical unmanned aerial vehicle. The method can dynamically simulate the real-time change of the physical unmanned aerial vehicle, plan the optimal flight path and effectively avoid obstacles, and can be widely applied to the technical field of unmanned aerial vehicle path control.

Description

Technical Field

[0001] The present application relates to the field of UAV path control technology, and in particular to a UAV path planning method and system based on digital twins. Background Art

[0002] In the field of drone technology, the application of digital twins has great potential. The autonomous navigation and obstacle avoidance capabilities of traditional drone systems in complex environments are often limited by factors such as sensor accuracy, computing resources, and environmental dynamics. Especially in scenarios that require refined operations such as disaster monitoring, smart logistics, and agricultural plant protection, related technical means are difficult to cope with highly uncertain environmental changes. Digital twins provide a new solution for drones through real-time simulation and virtual environment reconstruction.

[0003] Although digital twin technology brings significant advantages to drone systems, there are still many challenges to achieve this goal. For example, it is difficult to process and fuse massive data from multiple sensors in real time, resulting in delayed system response; the generated three-dimensional models are often not accurate enough, affecting the drone's environmental perception ability; and the synchronization between the physical entity and the virtual twin is low.

[0004] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the invention

[0005] The embodiments of the present application aim to solve at least one of the technical problems in the related art to a certain extent. To this end, the main purpose of the embodiments of the present application is to propose a drone path planning method and system based on digital twins, which can dynamically simulate the real-time changes of physical drones, plan the optimal flight path and effectively avoid obstacles.

[0006] To achieve the above objectives, one aspect of an embodiment of the present application proposes a drone path planning method based on digital twins, the method comprising the following steps:

[0007] Acquire first initial multi-source environmental perception data corresponding to the physical UAV in the target flight area;

[0008] Preprocessing the first initial multi-source environment perception data to obtain first target multi-source environment perception data; the first target multi-source environment perception data includes first target environment data;

[0009] constructing a three-dimensional environment model based on the first target environment data;

[0010] Constructing a digital twin based on the geometric model, physical model, control model, target physical data hybrid dynamic model and the three-dimensional environment model corresponding to the physical UAV;

[0011] Inputting the second target multi-source environmental perception data into the digital twin, dynamically simulating the flight behavior of the physical UAV in a virtual space based on the second target multi-source environmental perception data by the digital twin, and obtaining simulated flight environment state data for the physical UAV;

[0012] Based on the simulated flight environment state data, the path of the physical UAV is planned in combination with a target prediction control algorithm and a target path planning algorithm to obtain a target flight path, and the target flight path is transmitted to the physical UAV to control the physical UAV to fly along the target flight path.

[0013] In some embodiments, the method further comprises the step of pre-building the target physical data hybrid dynamic model, wherein the pre-building the target physical data hybrid dynamic model specifically comprises the following steps:

[0014] Acquire a data-driven model and the physical model corresponding to the physical drone; the data-driven model is used to supplement the nonlinear characteristics of the physical model;

[0015] Combining the physical model and the data-driven model, modeling the physical drone and the operating environment corresponding to the physical drone to obtain an initial physical data hybrid dynamic model;

[0016] The initial physical data hybrid dynamic model is trained by objectively minimizing the loss function to obtain the target physical data hybrid dynamic model.

[0017] In some embodiments, the expression of the target physical data hybrid dynamic model is:

[0018]

[0019] in, represents the derivative of the state vector x(t) with respect to time t, x(t) represents the state vector of the physical drone, t represents time, u(t) represents the control input, f(x(t),u(t)0 represents the function of x(t) and u(t), η(t) represents the noise term, N(x(t),u(t); θ) represents the prediction result of the data-driven model, and θ represents the parameters of the data-driven model.

[0020] In some embodiments, the preprocessing of the first initial multi-source environmental perception data to obtain first target multi-source environmental perception data includes:

[0021] Performing feature extraction processing on the first initial multi-source environment perception data to obtain first multi-source environment perception feature data;

[0022] Performing time synchronization processing on the first multi-source environment perception feature data to obtain first time synchronization feature data;

[0023] Performing spatial alignment processing on the first time synchronization feature data to obtain first time-space synchronization feature data;

[0024] Performing noise filtering on the first spatiotemporal synchronization feature data to obtain first spatiotemporal clean feature data;

[0025] The first spatiotemporal clean feature data is filtered through a particle filter to obtain the first target multi-source environment perception data; wherein the state prediction of each particle in the particle filter is implemented based on the target physical data hybrid dynamic model.

[0026] In some embodiments, constructing a three-dimensional environment model based on the first target environment data includes:

[0027] Modeling the first target environment data by using a target surface reconstruction algorithm to obtain a three-dimensional smooth surface model;

[0028] The three-dimensional smooth surface model is subjected to triangulation meshing processing by a target triangulation algorithm to obtain the three-dimensional environment model.

[0029] In some embodiments, the second target multi-source environmental perception data includes second target multi-source sensor data and second target environmental data, and the digital twin dynamically simulates the flight behavior of the physical drone in a virtual space based on the second target multi-source environmental perception data to obtain simulated flight environment state data for the physical drone, including:

[0030] Dynamically updating virtual environment information in the digital twin based on the second target multi-source sensor data;

[0031] Dynamically updating the three-dimensional environment model in the digital twin based on the second target environment data;

[0032] Dynamically updating the digital twin based on the updated virtual environment information and the updated three-dimensional environment model;

[0033] The flight behavior of the physical UAV is dynamically simulated in the virtual space based on the updated digital twin to obtain the simulated flight environment state data for the physical UAV.

[0034] In some embodiments, the path planning of the physical UAV is performed based on the simulated flight environment state data in combination with a target prediction control algorithm and a target path planning algorithm to obtain a target flight path, including:

[0035] Performing behavior prediction on the simulated flight environment state data according to the target prediction control algorithm to obtain a target flight behavior control sequence;

[0036] The target flight behavior control sequence is path planned according to the target path planning algorithm and the target collision avoidance strategy to obtain the target flight path.

[0037] To achieve the above objectives, another aspect of the embodiment of the present application proposes a drone path planning system based on digital twins, and the system includes the following modules:

[0038] A data acquisition module, used to acquire first initial multi-source environmental perception data corresponding to the physical UAV in the target flight area;

[0039] A data processing module, configured to preprocess the first initial multi-source environment perception data to obtain first target multi-source environment perception data; the first target multi-source environment perception data includes first target environment data;

[0040] A three-dimensional environment model building module, used to build a three-dimensional environment model based on the first target environment data;

[0041] A digital twin construction module, used to construct a digital twin based on a geometric model, a physical model, a control model, a target physical data hybrid dynamic model and the three-dimensional environment model corresponding to the physical UAV;

[0042] A flight behavior simulation module, used to input the second target multi-source environmental perception data into the digital twin, and dynamically simulate the flight behavior of the physical UAV in a virtual space based on the second target multi-source environmental perception data through the digital twin to obtain simulated flight environment state data for the physical UAV;

[0043] The flight path planning module is used to plan the path of the physical UAV based on the simulated flight environment state data in combination with the target prediction control algorithm and the target path planning algorithm to obtain the target flight path, and transmit the target flight path to the physical UAV to control the physical UAV to fly along the target flight path.

[0044] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application provides an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the above-mentioned method when executing the computer program.

[0045] To achieve the above objective, another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described above is implemented.

[0046] The embodiments of the present application include at least the following beneficial effects: The present application provides a method and system for UAV path planning based on digital twins, which obtains first initial multi-source environmental perception data corresponding to a physical UAV in a target flight area; pre-processes the first initial multi-source environmental perception data to obtain first target multi-source environmental perception data; the first target multi-source environmental perception data includes first target environmental data; constructs a three-dimensional environmental model based on the first target environmental data; constructs a digital twin based on a geometric model, a physical model, a control model, a target physical data hybrid dynamic model, and a three-dimensional environmental model corresponding to the physical UAV; inputs second target multi-source environmental perception data into the digital twin, and dynamically simulates the flight behavior of the physical UAV in a virtual space based on the second target multi-source environmental perception data through the digital twin to obtain simulated flight environment state data for the physical UAV; based on the simulated flight environment state data, the physical UAV is path planned in combination with a target prediction control algorithm and a target path planning algorithm to obtain a target flight path, and the target flight path is transmitted to the physical UAV to control the physical UAV to fly along the target flight path. The embodiment of the present application can remove noise, redundant and erroneous data, improve data quality, and provide a clean and effective data source for subsequent processing by preprocessing the first initial multi-source environmental perception data. At the same time, by fusing data from different sources to form a unified data representation, it is convenient for subsequent analysis and modeling, so that the system can respond quickly and reduce the delay caused by data fusion processing; the three-dimensional environmental model constructed based on the first target environmental data is the basis for the interaction between digital twins and real-time data. The virtual environment constructed by the three-dimensional environmental model provides a reliable simulation scene for digital twins and path planning, and is the basis of digital twins. At the same time, the three-dimensional environmental model provides drones with more sophisticated environmental perception capabilities. , making the navigation and obstacle avoidance of drones in complex environments more accurate and reliable; through the real-time input of the second target multi-source environmental perception data, the simulation process of the digital twin can dynamically adapt to the changes in the actual environment, and the physical drone and its virtual twin can be closely synchronized, thereby improving the real-time and accuracy of the simulation; based on the simulated flight environment state data, combined with the predictive control algorithm and the path planning algorithm, the path planning of the physical drone can be carried out, and the optimal flight path can be intelligently planned. At the same time, the path planning algorithm can effectively avoid obstacles, improve the safety and stability of the physical drone flight, and, by optimizing the flight path, can reduce the flight time and energy consumption, and improve the operation efficiency of the physical drone. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a flowchart of the steps of the drone path planning method based on digital twins provided in an embodiment of the present application;

[0048] Figure 2 It is a structural schematic diagram of a drone path planning system based on digital twins provided in an embodiment of the present application;

[0049] Figure 3 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are only examples of systems and methods consistent with some aspects of the embodiments of the present application as detailed in the attached claims.

[0051] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".

[0052] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0054] In the field of drone technology, the application of digital twins has great potential. The autonomous navigation and obstacle avoidance capabilities of traditional drone systems in complex environments are often limited by factors such as sensor accuracy, computing resources, and environmental dynamics. Especially in scenarios that require refined operations such as disaster monitoring, smart logistics, and agricultural plant protection, related technical means are difficult to cope with highly uncertain environmental changes. Digital twins provide a new solution for drones through real-time simulation and virtual environment reconstruction. Although digital twin technology has brought significant advantages to drone systems, there are still many challenges to achieve this goal. For example, there are difficulties in real-time processing and fusion of massive data from multiple sensors, resulting in delayed system response; the generated three-dimensional models are often not accurate enough, affecting the drone's environmental perception ability; and the synchronization between physical entities and virtual twins is low.

[0055] In view of this, a method and system for UAV path planning based on digital twins are provided in an embodiment of the present application. The scheme obtains first initial multi-source environmental perception data corresponding to a physical UAV in a target flight area; preprocesses the first initial multi-source environmental perception data to obtain first target multi-source environmental perception data; the first target multi-source environmental perception data includes first target environmental data; constructs a three-dimensional environmental model based on the first target environmental data; constructs a digital twin based on a geometric model, a physical model, a control model, a target physical data hybrid dynamic model, and a three-dimensional environmental model corresponding to the physical UAV; inputs the second target multi-source environmental perception data into the digital twin, and dynamically simulates the flight behavior of the physical UAV in a virtual space based on the second target multi-source environmental perception data through the digital twin to obtain simulated flight environment state data for the physical UAV; based on the simulated flight environment state data, the physical UAV is path planned in combination with a target prediction control algorithm and a target path planning algorithm to obtain a target flight path, and the target flight path is transmitted to the physical UAV to control the physical UAV to fly along the target flight path. The embodiment of the present application can remove noise, redundant and erroneous data, improve data quality, and provide a clean and effective data source for subsequent processing by preprocessing the first initial multi-source environmental perception data. At the same time, by fusing data from different sources to form a unified data representation, it is convenient for subsequent analysis and modeling, so that the system can respond quickly and reduce the delay caused by data fusion processing; the three-dimensional environmental model constructed based on the first target environmental data is the basis for the interaction between digital twins and real-time data. The virtual environment constructed by the three-dimensional environmental model provides a reliable simulation scene for digital twins and path planning, and is the basis of digital twins. At the same time, the three-dimensional environmental model provides drones with more sophisticated environmental perception capabilities. , making the navigation and obstacle avoidance of drones in complex environments more accurate and reliable; through the real-time input of the second target multi-source environmental perception data, the simulation process of the digital twin can dynamically adapt to the changes in the actual environment, and the physical drone and its virtual twin can be closely synchronized, thereby improving the real-time and accuracy of the simulation; based on the simulated flight environment state data, combined with the predictive control algorithm and the path planning algorithm, the path planning of the physical drone can be carried out, and the optimal flight path can be intelligently planned. At the same time, the path planning algorithm can effectively avoid obstacles, improve the safety and stability of the physical drone flight, and, by optimizing the flight path, can reduce the flight time and energy consumption, and improve the operation efficiency of the physical drone.

[0056] The unmanned aerial vehicle path planning method based on digital twins provided in the embodiment of the present application relates to the field of unmanned aerial vehicle path control technology. The unmanned aerial vehicle path planning method based on digital twins provided in the embodiment of the present application can be applied to a terminal, can also be applied to a server, and can also be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a car terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or it can be configured as a server cluster or a distributed system composed of multiple physical servers, and can also be configured to provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network, content distribution network) and big data and artificial intelligence platforms and other basic cloud computing services. The cloud server, the server can also be a node server in a blockchain network; the software can be an application that implements a unmanned aerial vehicle path planning method based on digital twins, etc., but is not limited to the above forms.

[0057] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs (Personal Computers, personal computers), minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0058] See also Figure 1 , Figure 1 is an optional step flow chart of the UAV path planning method based on digital twins provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S106.

[0059] Step S101, obtaining first initial multi-source environmental perception data corresponding to a physical UAV in a target flight area;

[0060] The target flight area is a specific geographic space in which the physical drone is planning or currently performing flight operations. It should be noted that the determination of the target flight area needs to be set based on actual conditions.

[0061] For physical drones, they are actual unmanned aerial vehicles that can fly autonomously or remotely. Unlike virtual drones, physical drones have real flying capabilities and can complete complex flight and operation tasks through the various sensors, cameras, communication equipment, etc. that they carry. Virtual drones do not have the actual flight capabilities of physical drones, but can simulate the flight behavior, environment, and tasks of physical drones in a virtual environment.

[0062] The first initial multi-source environment perception data includes first initial multi-source sensor data and first initial environment data. The first initial environment data is used for subsequent construction of a three-dimensional environment model, and the first initial multi-source sensor data is used for subsequent real-time data interaction.

[0063] Among them, multi-source sensor data is used for real-time navigation and obstacle avoidance of drones. Specifically, the initial multi-source sensor data will obtain the target multi-source sensor data after preprocessing, and the target multi-source sensor data will be transmitted to the digital twin and real-time data interaction module in the digital twin system to achieve data synchronization and dynamic behavior simulation, and provide a data basis for the drone's path planning and collision avoidance strategy. Multi-source sensor data mainly comes from cameras, sonar, GPS (Global Positioning System), IMU (Inertial Measurement Unit) and other devices. Multi-source sensor data includes but is not limited to the following data:

[0064] (1) Camera data: including high-resolution color and depth images, used to capture visual information of the environment;

[0065] (2) Sonar or Lidar data: used to obtain precise distance and shape information of the environment and generate point cloud data;

[0066] (3) Global Positioning System (GPS) data: provides the location information of the drone;

[0067] (4) Inertial Measurement Unit (IMU) data: includes data from the accelerometer and gyroscope, which is used to estimate the attitude changes of the drone.

[0068] Environmental data is mainly used to build a three-dimensional model of the environment to support the simulation scene of the digital twin system. Specifically, the initial environmental data can be obtained through lidar, cameras or other sensors. The initial environmental data is collected and processed by the multi-source data fusion and environmental perception module in the digital twin system to obtain the target environmental data, which provides accurate input for the high-precision three-dimensional model and virtualization module in the digital twin system. The high-precision three-dimensional model and virtualization module use a three-dimensional reconstruction algorithm to reconstruct the processed target environmental data to generate a three-dimensional model of the environment, that is, the three-dimensional environmental model constructed subsequently. The three-dimensional environmental model includes external factors that affect the flight of drones, such as terrain, obstacles, and meteorological conditions, and can provide accurate virtual scenes for digital twins. Environmental data includes but is not limited to the following data:

[0069] (1) Terrain data: used to describe the height and slope of the ground, which is of great significance for flight path planning;

[0070] (2) Obstacle data: location and motion status of fixed or moving obstacles such as buildings and trees;

[0071] (3) Meteorological conditions data: including wind speed, wind direction, temperature, humidity and other information, which affect the flight stability of the drone.

[0072] It should be noted that the initial multi-source environmental perception data also needs to be obtained in real time when real-time data interaction is performed in the digital twin later. The initial multi-source environmental perception data obtained when real-time data interaction is performed in the digital twin later can be called the second initial multi-source environmental perception data. The second initial multi-source environmental perception data is used to input into the digital twin for real-time data interaction after the same preprocessing operation. Before obtaining the second initial multi-source environmental perception data, the first initial multi-source sensor data obtained can be transmitted to the digital twin and real-time data interaction module in the digital twin system after preprocessing to achieve data synchronization and dynamic behavior simulation.

[0073] In the embodiment of the present application, the architecture of the digital twin system mainly includes six modules: a physical data hybrid drive integration module, a multi-source data fusion and environment perception module, a high-precision three-dimensional model and virtualization module, a digital twin and real-time data interaction module, a model predictive control and collision avoidance strategy module, and a system verification and optimization module. The physical data hybrid drive integration module provides basic dynamic modeling capabilities, laying the foundation for subsequent multi-source data fusion, three-dimensional environment reconstruction and control strategies. The multi-source data fusion and environment perception module is used to collect and process multi-source environment perception data, providing high-precision input for three-dimensional modeling and real-time simulation. The high-precision three-dimensional model and virtualization module uses the data fused by the multi-source data fusion and environment perception modules to build a real three-dimensional environment, providing accurate virtual scenes for digital twins and model predictive control. The digital twin and real-time data interaction module can reflect the state of the physical drone in real time, and perform real-time path planning and collision avoidance through model predictive control and path planning strategies. The model predictive control and collision avoidance strategy module uses digital twins and environmental data to generate optimal control and collision avoidance strategies, so that the drone can operate safely and efficiently. The system verification and optimization module verifies and provides feedback on the entire system, promoting closer collaboration between modules and continuously improving system performance. Through this structured technical framework, the embodiments of the present application can achieve independent and coordinated operation of each module of the entire drone digital twin system, ultimately achieving autonomous navigation, obstacle avoidance, and mission planning of drones in complex environments.

[0074] In step S101 shown in the embodiment of the present application, by acquiring the initial multi-source environmental perception data corresponding to the physical UAV in the target flight area, it is possible to more comprehensively and accurately reflect the real-time environmental conditions of the physical UAV in the target flight area, and provide a rich and accurate data basis for subsequent modeling, flight decision-making and path planning.

[0075] Step S102, preprocessing the first initial multi-source environment perception data to obtain first target multi-source environment perception data; the first target multi-source environment perception data includes first target environment data;

[0076] In some embodiments, the step of pre-building a target physical data hybrid dynamic model is also included. Pre-building the target physical data hybrid dynamic model specifically includes the following steps: obtaining the data-driven model and the physical model corresponding to the physical UAV; the data-driven model is used to supplement the nonlinear characteristics of the physical model; combining the physical model and the data-driven model to model the physical UAV and the operating environment corresponding to the physical UAV to obtain an initial physical data hybrid dynamic model; training the initial physical data hybrid dynamic model by objectively minimizing the loss function to obtain the target physical data hybrid dynamic model.

[0077] Among them, the target physical data hybrid dynamic model is a dynamic model obtained by optimizing and training the initial physical data hybrid dynamic model using the target minimization loss function, and the initial physical data hybrid dynamic model is obtained by integrating the data-driven model and the physical model corresponding to the physical UAV. Among them, the trained target physical data hybrid dynamic model is obtained by combining the physical model with the data-driven model in the physical data hybrid drive integration module of the UAV digital twin system to dynamically model the UAV and its operating environment, and optimizing the model by minimizing the loss function. The loss function includes state prediction error and physical constraints.

[0078] Specifically, according to the dynamic characteristics of the physical UAV, its physical model is constructed. The physical model may include a rigid body dynamics model and an aerodynamics model based on Newton's law. The physical model describes the state change of the UAV, and the state equation of the physical model can be expressed as:

[0079]

[0080] in, represents the derivative of the state vector x(t) with respect to time t, that is, the rate of change of the state, x(t) represents the state vector of the drone (such as position, velocity, acceleration), t represents time, u(t) represents the control input (such as thrust and torque), f(x(t),u(t)0 represents a function related to x(t) and u(t), and η(t) represents the noise term.

[0081] The data-driven model is used to supplement the nonlinear characteristics of the physical model. The prediction result of the data-driven model is N(x(t),u(t);θ), where θ represents the parameters of the data-driven model.

[0082] In the embodiment of the present application, the physical data hybrid drive module in the UAV digital twin system uses a physical model and a data-driven model to describe the state change of the UAV. The physical data hybrid drive state equation (i.e., the expression of the target physical data hybrid dynamic model) after integrating the physical model and the data-driven model is:

[0083]

[0084] in, represents the derivative of the state vector x(t) with respect to time t, x(t) represents the state vector of the physical drone, t represents time, u(t) represents the control input, f(x(t), u(t)) represents the function of x(t) and u(t), η(t) represents the noise term, N(x(t), u(t); θ) represents the prediction result of the data-driven model, and θ represents the parameters of the data-driven model.

[0085] The training of the state equation driven by the integrated physical data hybrid is achieved by minimizing the loss function L(θ). The calculation formula for minimizing the loss function L(θ) is as follows:

[0086]

[0087] Among them, x true (t i ) represents the real state, represents the predicted state, λR(θ) represents the regularization term, and N represents the total number of training samples.

[0088] In the specific implementation, the physical model and the data-driven model are combined into a target physical data hybrid dynamic model, which can achieve accurate simulation of the UAV and the environment, enabling the system to handle complex dynamic relationships and nonlinear behaviors to provide more accurate predictions and decision support.

[0089] In the embodiment of the present application, the physical data hybrid drive integration module is the basis of the entire UAV digital twin system. It is responsible for generating a dynamic model of the UAV and its operating environment, providing real-time state prediction, and providing basic dynamic modeling capabilities for the system. It can combine the physical model with the data-driven model to achieve efficient autonomous navigation and obstacle avoidance of the UAV digital twin in a virtual and real environment.

[0090] In some embodiments, step S102 may include: performing feature extraction processing on the first initial multi-source environmental perception data to obtain first multi-source environmental perception feature data; performing time synchronization processing on the first multi-source environmental perception feature data to obtain first time synchronization feature data; performing spatial alignment processing on the first time synchronization feature data to obtain first space-time synchronization feature data; performing noise filtering processing on the first space-time synchronization feature data to obtain first space-time clean feature data; filtering processing on the first space-time clean feature data through a particle filter to obtain first target multi-source environmental perception data; wherein, the state prediction of each particle in the particle filter is implemented based on the target physical data hybrid dynamic model.

[0091] Among them, the step of preprocessing the first initial multi-source environmental perception data includes feature extraction processing, time synchronization processing, space alignment processing, noise filtering processing and filtering processing.

[0092] Among them, the first target multi-source environmental perception data includes the first target environmental data and the first target multi-source sensor data. The first target environmental data is mainly used to build a three-dimensional model of the environment to support the simulation scenario of the digital twin system. The first target multi-source sensor data will be transmitted to the digital twin and real-time data interaction module in the digital twin system to achieve data synchronization and dynamic behavior simulation, providing a data basis for the path planning and collision avoidance strategy of the drone.

[0093] The implementation process of feature extraction processing includes: first, using Hough transform and edge detection to extract feature points, edges and other information, aligning images from different perspectives, and determining the relative position and posture relationship between the images; then using the convolutional data-driven model (Convolutional Neural Networks, referred to as CNN) to identify and classify the drone and its surrounding objects, and extract information such as size, shape, and position; among them, Hough transform and edge detection extraction are mainly aimed at the initial multi-source environmental perception data, while the convolutional data-driven model focuses on identifying and classifying specific objects in the drone and its surroundings from the extracted data. The data after feature extraction processing is used in 3D modeling software and physics engines to build a 3D model of the environment and estimate the state of the drone (such as mass, center of mass, rotational inertia), improve the accuracy and comprehensiveness of perception, and support navigation and decision-making in complex environments.

[0094] The first multi-source environmental perception feature data is key feature data obtained after feature extraction processing.

[0095] Since the data collection time of different sensors may be different, time inconsistency may occur. In order to solve this problem, the sensor data can be aligned to the same timestamp by interpolation or extrapolation. In an embodiment of the present application, the first multi-source environment perception feature data is time synchronized by interpolation to obtain the first time-synchronized feature data. The implementation process of the time synchronization processing is as follows: Assuming that the data time of the drone sensor S1 is t1 and the data time of the drone sensor S2 is t2, the data of the drone sensor S1 at time t2 can be estimated by linear interpolation. The calculation formula for the actual data value of the drone sensor S1 at time t2 is as follows:

[0096]

[0097] in, represents the actual data value of the drone sensor S1 at time t2, represents the actual data value of the drone sensor S1 at time t1, Δt is the sampling time interval of the drone sensor S1, Indicates the measurement data of the unmanned sensor S1.

[0098] For spatial alignment processing, the implementation process is as follows: using the transformation matrix Aligning the coordinate systems of different sensors can obtain time-space synchronous feature data. Spatial inconsistency usually comes from the different installation positions of sensors. Through the relative position and posture between sensors, the transformation matrix can be used Convert data from one sensor's coordinate system to another sensor's coordinate system. The calculation formula for coordinate system conversion is as follows:

[0099]

[0100] in, represents a homogeneous transformation matrix including rotation and translation, Represents the measurement data of unmanned sensor S1, Indicates the measurement data of unmanned sensor S1 Equivalent data in the coordinate system of drone sensor S2.

[0101] For noise filtering processing, it includes statistical outlier removal (SOR) processing and voxel grid filtering (Voxel Grid Filtering) processing. Specifically, statistical outlier removal is first performed on the spatiotemporal synchronization data, and then voxel grid filtering is performed on the data after the outliers are removed. After the noise filtering processing, spatiotemporal clean feature data can be obtained. Among them, the specific implementation process of noise filtering processing is: using the statistical outlier removal method, the outliers are removed by calculating the average distance between each processed data point and its neighboring points. The calculation formula for the average distance between each processed data point and its neighboring points is as follows:

[0102]

[0103] Among them, p i Represents the point in the current data, p j Represents point p i neighborhood points, i represents the index of the current point, j represents the index of the neighborhood point, and k represents the total number of points in the neighborhood.

[0104] Then, voxel grid filtering is performed on the data after removing outliers. Specifically, voxel grid filtering is used to reduce the point cloud density. The specific calculation formula is as follows:

[0105]

[0106] Among them, Voxel Size represents the size of voxels, which determines the area covered by each voxel in space. The larger the voxel, the fewer points each voxel contains, and the lower the density of the point cloud. Voxel Resolution represents the voxel resolution, which is used to indicate the resolution of the voxel grid. The higher the resolution, the smaller the voxel size and the higher the density of the point cloud. The inverse of the resolution is the size of the voxel.

[0107] After filtering the data for noise, a particle filter is used to integrate multiple sensor data to ensure data consistency and perception accuracy, and obtain unified target multi-source environmental perception data. Among them, the state prediction of each particle in the particle filter is realized through the target physical data hybrid dynamic model. The state prediction expression of the particle expressed by the target physical data hybrid dynamic model is as follows:

[0108]

[0109] in, Represents the state vector The derivative at time t-1, that is, the rate of change of the state, represents the state of particle i at the previous time t-1, u t-1 represents the control input at time t-1, represents process noise.

[0110] Among them, the weight update of particles is based on the observation model The expression for weight update is as follows:

[0111]

[0112] in, represents the weight of particle i at time t, represents the weight of particle i at the previous time t-1, z t represents the observed value, R represents the covariance matrix of the observation noise, and T represents the transpose of the matrix.

[0113] Specifically, the process of using particle filter to perform filtering fusion processing is as follows:

[0114] (1) Initialization:

[0115] Initialize particles: Randomly generate N particles in the state space. These particles are sampled from the prior distribution. Each particle represents a possible state.

[0116] Initialization weight: All particles have the same initial weight, set to

[0117] (2) Prediction:

[0118] For each particle i (assuming ), predicting the next state of the system based on its dynamic model (i.e., the target physical data hybrid dynamic model) This can be done by adding noise to the dynamic model of the system, that is, the state prediction expression of the particle represented by the above target physical data mixed dynamic model is as follows:

[0119]

[0120] (3) Update weights:

[0121] According to the observed value z t Calculate the weight of each particle. The calculation of weights usually depends on the observation model The expression for weight update is as follows:

[0122]

[0123] (4) Resampling:

[0124] According to weight Resample the particles. The purpose of this step is to reduce the number of particles with smaller weights, thereby increasing the proportion of particles with larger weights. Resampling methods include system resampling, roulette resampling, etc., which are not limited in the embodiments of the present application.

[0125] (5) Estimated status:

[0126] Estimate the current state of the system by weighted average or other methods:

[0127]

[0128] in, represents the state estimate at time step t, represents the state of the ith particle at time step t, represents the weight of the i-th particle at time step t.

[0129] It should be noted that the embodiments of the present application do not limit other methods for estimating the current state of the system.

[0130] Among them, the process of preprocessing the initial multi-source environmental perception data to obtain the target multi-source environmental perception data is implemented in the multi-source data fusion and environmental perception module. The multi-source data fusion and environmental perception module ensures the temporal and spatial consistency of all sensor data and improves data quality through noise filtering. The high-precision environmental perception information output by the multi-source data fusion and environmental perception module is used to input the high-precision three-dimensional model and virtualization module. The high-precision three-dimensional environment model in the virtual environment is generated by the high-precision three-dimensional model and virtualization module based on the three-dimensional reconstruction algorithm.

[0131] In step S102 shown in the embodiment of the present application, the multi-source data fusion and environmental perception module processes the multi-source data of the drone through time synchronization and space alignment technology to ensure the consistency of data from different sensors in time and space, and uses statistical outlier removal and voxel grid filtering methods for noise filtering, which can remove noise, redundant and erroneous data, improve data quality, and provide a clean and effective data source for subsequent processing. Finally, the noise-filtered data is filtered and fused to output high-precision environmental perception information for subsequent analysis and modeling.

[0132] It should be noted that the acquisition and preprocessing of multi-source data are both realized by the multi-source data fusion and environment perception module.

[0133] Step S103, constructing a three-dimensional environment model based on the first target environment data;

[0134] Among them, the three-dimensional environment model is constructed based on the first target environment data, and the virtual environment constructed by the three-dimensional environment model can provide a reliable simulation scene for digital twins and path planning. It is the basis of digital twins and can reflect the UAV and its surrounding environment in the physical environment. The three-dimensional environment model includes external factors that affect the flight of the UAV, such as terrain, obstacles, and weather conditions. Among them, the first target environment data refers to the environment data obtained for the first time to build the three-dimensional environment model. The target environment data in the subsequent digital real-time interaction can be called the second target environment data. The second target environment data is used to represent the real-time environment data, and the real-time environment data is used to input into the digital twin to update the three-dimensional environment model.

[0135] In some embodiments, step S103 may include: modeling the first target environment data by a target surface reconstruction algorithm to obtain a three-dimensional smooth surface model; and triangulating the three-dimensional smooth surface model by a target triangulation algorithm to obtain a three-dimensional environment model.

[0136] Among them, the target surface reconstruction algorithm is a Poisson surface reconstruction algorithm, which can be used to model the first target environment data to obtain a three-dimensional smooth surface model, that is, the Poisson surface reconstruction algorithm is used to construct a smooth three-dimensional surface from noise and sparse data. The target triangulation algorithm is a Delaunay triangulation algorithm, which can be used to triangulate the three-dimensional smooth surface model, that is, the Delaunay triangulation algorithm is used to optimize the triangular mesh to obtain a three-dimensional environment model.

[0137] The process of constructing a three-dimensional environment model based on the first target environment data is implemented in the high-precision three-dimensional model and virtualization module of the UAV digital twin system.

[0138] Among them, the Poisson surface reconstruction algorithm is a surface reconstruction method based on point cloud normals. By solving the Poisson equation, a smooth three-dimensional surface is constructed from noise and sparse data. The calculation formula for generating a smooth surface by solving the Poisson equation is:

[0139]

[0140] Among them, u represents the scalar field; N represents the point cloud normal; represents the Laplace operator, which is used to calculate the derivatives of scalar fields; The gradient of a scalar field u is a vector field that represents the direction in which the scalar field changes fastest in space and its rate of change.

[0141] The high-precision 3D model and virtualization module uses Delaunay triangulation to generate a triangular mesh of a 3D point cloud, ensuring that the circumscribed circle of a triangle does not contain other points. Delaunay triangulation generates a triangular mesh on a point cloud so that the circumscribed circle of any triangle does not contain other points, ensuring the uniformity and stability of the triangle, and is particularly suitable for generating smooth surfaces. The expression for constructing triangulation using the Delaunay condition is:

[0142] ||O-p1_||>r

[0143] Among them, O represents the center of the circumscribed circle, r represents the radius of the circumscribed circle, and p1 represents other points in the point set.

[0144] In an embodiment of the present application, the high-precision three-dimensional model and virtualization module can effectively generate a high-precision three-dimensional surface model, i.e., a three-dimensional environment model, through a Poisson surface reconstruction algorithm and a Delaunay triangulation algorithm. The generated three-dimensional environment model is the basis of the digital twin and real-time data interaction module. The virtual environment constructed by the three-dimensional environment model provides a reliable simulation scene for digital twins and path planning, and is the basis of digital twins.

[0145] In the specific implementation, by using image processing and depth estimation technology, the system can reconstruct the 3D model of the operating environment in real time. The 3D model of the environment reflects the dynamic changes of the drone and its surroundings for simulation and visual analysis. That is, through multi-view stereo technology and optimization algorithms, the system builds high-precision 3D point cloud and surface models, which are not only used to simulate the movement of the drone, but also for interaction and testing in the virtual environment.

[0146] Step S103 shown in the embodiment of the present application is to construct a three-dimensional environment model based on the first target environment data. The three-dimensional environment model can intuitively display the complex environment information such as geography, obstacles, and weather of the flight area, which is easy to understand and analyze, and can provide a high-precision environmental background for UAV flight simulation, thereby improving the authenticity and reliability of the simulation. It is understandable that the three-dimensional environment model is the basis for the interaction between the digital twin and real-time data. The virtual environment constructed by the three-dimensional environment model provides a reliable simulation scene for the digital twin and path planning, and is the basis of the digital twin.

[0147] Step S104, constructing a digital twin based on the geometric model, physical model, control model, target physical data hybrid dynamic model and the three-dimensional environment model corresponding to the physical UAV;

[0148] Among them, the process of building a digital twin is based on the digital twin and real-time data interaction module in the UAV digital twin system. The digital twin and real-time data interaction module is used to create a real-time digital twin. By receiving and processing the data of the physical UAV, it interacts with the physical UAV in real time to achieve data synchronization and dynamic behavior simulation, and update the virtual environment in real time.

[0149] The digital twin of the drone and its operating environment created by the drone digital twin system constructed in the embodiment of the present application can receive and reflect the data and status of the physical drone in real time, providing a real-time virtual platform for control and optimization. The digital twin is combined with the power system model of the drone, and the digital twin system can simulate the dynamic behavior of the drone under different environmental conditions, including force, motion path, etc.

[0150] The digital twin combines multi-source data fusion with multi-source environmental sensor data measured in the environmental perception module to obtain the physical information of the drone, and realizes the dynamic behavior simulation of the drone in the virtual environment. In addition, the geometric model of the drone is combined with the generated three-dimensional environmental model to construct a complete virtual space. Physical model of the drone Control model u(t) and target physical data hybrid dynamic model All will be displayed in the virtual space. The 3D environment model in the virtual space will be updated in real time according to the movement of the drone and the dynamic changes of the environment. This update process continuously adjusts the 3D environment model by continuously inputting new multi-view images using structural motion estimation technology.

[0151] In the specific implementation, data exchange between the physical drone and the digital twin is achieved through a high-speed communication link such as 5G (5th Generation Mobile Communication Technology). The physical drone transmits real-time status data to the digital twin, and the digital twin feeds back the predicted or optimized control instructions to the physical drone to achieve real-time two-way data flow. At the same time, it is also necessary to set up control strategy feedback. Specifically, based on the status of the digital twin and the changes in the virtual environment, the control strategy is adjusted, and the optimized control instructions are sent back to the physical drone to ensure that it can operate safely and efficiently in a dynamic environment. Through the construction of the digital twin, the drone status in the virtual environment is synchronized with the real-time status of the physical drone, providing a reliable data basis for subsequent path planning and collision avoidance strategies.

[0152] For the basic construction of the digital twin, it is first necessary to create a geometric model for the physical drone. The geometric model can be generated by high-precision three-dimensional scanning, CAD (Computer Aided Design) modeling and other methods, which are not limited in the embodiments of the present application. The geometric model can reflect the shape and structure of the drone. At the same time, it is also necessary to build a physical model, which can be built according to the dynamic characteristics of the drone. In addition, environmental data is obtained through lidar, cameras or other sensors, and a three-dimensional reconstruction algorithm is used to reconstruct the processed environmental data into a three-dimensional model of the environment, that is, a three-dimensional environmental model. The three-dimensional environmental model includes external factors that affect the flight of the drone, such as terrain, obstacles, and meteorological conditions.

[0153] In the specific implementation, the digital twin can simulate the dynamic behavior of the physical drone. Specifically, the physical model of the drone is used to simulate the dynamic behavior, and the real-time control strategy of the model predictive control is combined to optimize the flight path. For dynamic environments (such as moving obstacles), the high-precision three-dimensional model in the virtual environment needs to be updated in real time, and updated through real-time input environmental data. Using multi-source data fusion technology, the virtual environment can be dynamically updated through data input from different sensors, that is, updated through multi-source sensor data.

[0154] Step S104 shown in the embodiment of the present application constructs a digital twin based on the geometric model, physical model, control model, target physical data hybrid dynamic model and three-dimensional environment model corresponding to the physical UAV. The digital twin can map the state and environmental changes of the physical UAV in real time to achieve virtual and real synchronization. At the same time, through simulation and prediction, it can discover potential problems and risks in advance, provide early warning and solutions for the operation of the physical UAV, and based on the simulation results of the digital twin, the flight strategy and control parameters of the UAV can be optimized to improve flight efficiency and safety.

[0155] Step S105, inputting the second target multi-source environmental perception data into the digital twin, dynamically simulating the flight behavior of the physical UAV in a virtual space based on the second target multi-source environmental perception data by the digital twin, and obtaining simulated flight environment state data for the physical UAV;

[0156] In some embodiments, dynamically simulating the flight behavior of a physical UAV in a virtual space based on second target multi-source environmental perception data through a digital twin to obtain simulated flight environment status data for the physical UAV may include: dynamically updating virtual environment information in the digital twin based on the second target multi-source sensor data; dynamically updating the three-dimensional environment model in the digital twin based on the second target environment data; dynamically updating the digital twin based on the updated virtual environment information and the updated three-dimensional environment model; dynamically simulating the flight behavior of a physical UAV in a virtual space based on the updated digital twin to obtain simulated flight environment status data for the physical UAV.

[0157] Among them, the second target multi-source environmental perception data includes second target multi-source sensor data and second target environmental data. The second target multi-source environmental perception data is obtained after preprocessing the second initial multi-source environmental perception data. The processing process is the same as that of obtaining the first target multi-source environmental perception data after preprocessing the first initial multi-source environmental perception data. The composition of the data is also the same, both containing multi-sensor data and environmental data, and is also implemented in the source data fusion and environmental perception module. It should be noted that the difference between the first target multi-source environmental perception data and the second target multi-source environmental perception data is that the first target environmental data of the first target multi-source environmental perception data is used to construct a three-dimensional environmental model, while the second target environmental data of the second target multi-source environmental perception data is used to input the digital twin to update the three-dimensional environmental model. In addition, the first multi-source sensor data of the first target multi-source environmental perception data and the second multi-source sensor data of the second target multi-source environmental perception data can both be used to input the digital twin for real-time data interaction, realize data synchronization and dynamic behavior simulation, and provide a data basis for the path planning and collision avoidance strategy of the drone. However, the first multi-source sensor data of the first target multi-source environmental perception data mainly refers to the real-time data corresponding to the data acquired for the first time, and the environmental data in the data acquired for the first time is used to build a three-dimensional environmental model, while the second target multi-source environmental perception data is used to characterize the real-time data transmitted to the digital twin, and does not only refer to the real-time data of a certain time, but refers to the real-time data that is continuously updated, which is used to be input into the digital twin for real-time data interaction.

[0158] Among them, the simulated flight environment status data is the simulation data obtained by the digital twin in the virtual space through dynamic simulation based on the real-time target multi-source environment perception data. The optimal flight path can be planned based on the simulation data.

[0159] In the specific implementation, relevant simulation software can be selected to provide support for specific scenarios (such as cities, forests, indoors, etc.) that need to be tested, and the corresponding virtual environment can be built in the simulation software. The virtual environment includes adding elements such as buildings, terrain, and vegetation, and setting various obstacles in the virtual environment, including static (such as buildings) and dynamic (such as moving vehicles or other drones) obstacles to simulate the complex situations in the real world. Next, the drone model and collision avoidance algorithm are integrated into the simulation platform, and the corresponding sensors (such as radar, laser radar (Lidar) and visual sensors) and data fusion technology are configured to improve the accuracy and robustness of obstacle detection and environmental perception. After performing a large number of automatic tests (including different flight routes, obstacle configurations and environmental conditions), performance data (including key performance indicators such as collision avoidance success rate, number of collisions, and path deviation) are collected and analyzed to identify problems and make strategic adjustments.

[0160] In step S105 shown in the embodiment of the present application, the second target multi-source environmental perception data is input into the digital twin, and the flight behavior of the physical drone is dynamically simulated in the virtual space based on the second target multi-source environmental perception data through the digital twin. The multi-source data input in real time enables the simulation process to dynamically adapt to changes in the actual environment, improves the real-time and accuracy of the simulation, provides a basis for flight decision-making, and reduces the risk of actual flight.

[0161] Step S106, based on the simulated flight environment state data, combining the target prediction control algorithm and the target path planning algorithm, the path planning of the physical UAV is performed to obtain the target flight path, and the target flight path is transmitted to the physical UAV to control the physical UAV to fly along the target flight path.

[0162] In some embodiments, based on the simulated flight environment state data, the path of the physical UAV is planned in combination with the target prediction control algorithm and the target path planning algorithm to obtain the target flight path, which may include: performing behavior prediction on the simulated flight environment state data according to the target prediction control algorithm to obtain the target flight behavior control sequence; performing path planning on the target flight behavior control sequence according to the target path planning algorithm and the target collision avoidance strategy to obtain the target flight path.

[0163] Among them, based on the simulated flight environment state data, the path planning of the physical UAV is combined with the target prediction control algorithm and the target path planning algorithm. The process of obtaining the target flight path is realized in the model prediction control and collision avoidance strategy module. The model prediction control and collision avoidance strategy module performs path planning based on the MPC-RRT algorithm and adjusts the path in real time through dynamic obstacle modeling and safety constraint optimization. By predicting the future state, the optimal control strategy is generated to ensure the safety and efficient flight of the UAV in complex environments.

[0164] Among them, the target predictive control algorithm is the MPC (Model Predictive Control) algorithm, and the model predictive control and collision avoidance strategy module performs real-time path optimization through the MPC algorithm. The state prediction of the UAV digital twin system is implemented using the state equation of the target physical data hybrid dynamic model, which can be called the state space model here. The expression of the state space model (that is, the state equation of the target physical data hybrid dynamic model) is as follows:

[0165]

[0166] in, represents the derivative of the state vector x(t) with respect to time t, x(t) represents the state vector of the physical drone, t represents time, u(t) represents the control input, f(x(t), u(t)) represents the function of x(t) and u(t), ηη(t) represents the noise term, N(x(t), u(t); θ) represents the prediction result of the data-driven model, and θ represents the parameters of the data-driven model.

[0167] At each control moment, the MPC algorithm starts from the current state and uses the state space model (i.e., the target physical data hybrid dynamic model) to predict the system state for a period of time in the future (called the prediction time domain), which is conducive to formulating the optimal control strategy and ensuring the safe and efficient operation of the UAV in a dynamic environment. The number of predicted time steps is called the prediction time domain, which is usually N discrete time steps. The predicted state sequence is:

[0168] x k+1 , x k+2 , …, x k+N

[0169] Among them, each state is based on the current state x k and the future control input sequence u k ,u k+1 , ..., u k+N-1 .

[0170] In the embodiment of the present application, an objective function may be used to optimize the state space model, and the expression of the optimization objective function is as follows:

[0171]

[0172] Among them, x ref Indicates the expected state input; u ref represents the desired control input; Q represents the weight matrix of the state vector, R represents the weight matrix of the control vector, Q and R are used to adjust the relative importance of the state error and the control cost; represents the weighted quadratic norm, which is used to measure the deviation; J represents the total optimization cost function; N is the length of the prediction time domain; x t+i represents the actual state vector, which is used to represent the actual state of the system at time t+i; u t+i represents the actual control input vector, represents the control signal applied at time t+i, and t represents the current time step of the system.

[0173] By optimizing the objective function J, the optimal control sequence can be found while satisfying the constraints. The expression of the optimal control sequence is as follows:

[0174]

[0175] It should be noted that finding the optimal control sequence is a constrained quadratic programming or nonlinear programming problem. The specific solution depends on the form of the system model and the characteristics of the objective function, which will not be elaborated in detail in the embodiments of the present application.

[0176] Among them, the target path planning algorithm is the RRT (Rapidly-exploring Random Trees) algorithm. The RRT algorithm incrementally constructs a tree structure, explores the state space, and finds a feasible path from the starting point to the target. In the embodiment of the present application, the state space is first randomly sampled to generate a path, and the tree is expanded by the RRT algorithm to search for the optimal path. Combined with dynamic obstacle modeling and safety constraints, the path is adjusted in real time to ensure the effectiveness of the collision avoidance strategy. Specifically:

[0177] Based on the optimal control sequence generated by the MPC algorithm, the RRT algorithm is combined for path planning, and the path is optimized by extending the tree of the RRT algorithm. The expression of the extended tree is:

[0178]

[0179] Among them, x new represents the new path node, ∈ represents the step length, x near Represents the newly generated node x rand is a randomly generated point used to guide the expansion direction of the tree.

[0180] Among them, the algorithm flow of the RRT algorithm is as follows:

[0181] (1) Initialization: Set the starting point q start and the target point q goal ; Create a file containing only q start The RRT tree T.

[0182] (2) Iteration: Repeat the following steps (3)-(10) until a path is found or the maximum number of iterations is reached.

[0183] (3) Random sampling: Randomly sample a point q in the environment space rand .

[0184] (4) Expand the tree: Find the nearest node q from the tree T near , with q near As the starting point, extend a certain distance in the direction to get a new node q new .

[0185] (5) Collision detection: Detection path q near to q new If there is a collision with an obstacle, return to step (2) to continue the next iteration.

[0186] (6) Find the optimal connection: Find the connection with q in tree T new The nearest node q min , and calculate from q min to q new The cost cost(qnew min .

[0187] (7) Reconnection: For the tree T with q new Nodes q within a certain distance near_neighbors , calculated by q near Connect to q new The cost cost(qnew min ), choose the connection method with the lowest cost.

[0188] (8) Update the tree: set q new Add tree T and update the connection relationship and cost information between nodes.

[0189] (9) Optimize the path: For some nodes in the tree T, the target point q goal Optimize the target and improve the quality of the path by adjusting the connection relationship and cost information.

[0190] (10) Determine the termination condition: If the newly added node q new Approaching the target point q goal , a feasible path is found and the algorithm ends.

[0191] The specific implementation process of the RRT algorithm is as follows:

[0192] (1) Cubic Bezier curve trajectory smoothing: Use cubic Bezier curves to make discrete control points continuous, generate smooth flight trajectories, and improve flight stability and efficiency. To connect a series of discrete control points (such as P0, P1, P2, P3, ...) into a continuous curve, the steps are as follows:

[0193] 1) Selection of segmented control points: Group the discrete control points, and every four control points are used as control points of a cubic Bezier curve. For example, given the control points {P0, P1, P2, P3, P4, P5, P5, P7}, two cubic Bezier curves can be obtained: the first is defined by P0, P1, P2, P3, and the second is defined by P3, P4, P5, P6. Given four control points P0, P1, P2, P3, the formula for calculating the point B(t) on the cubic Bezier curve is as follows:

[0194] B(t)=(1-t) 3 P0+3(1-t) 2 tP1+3(1-t)t 2 P2+t 3 P3

[0195] Among them, t represents a parameter, ranging from [0, 1]; P0 represents the starting control point of the curve; P1 and P2 represent intermediate control points, which are used to control the shape and curvature of the curve; P3 represents the ending control point of the curve.

[0196] 2) Curve connection: Ensure that every two cubic Bezier curves are smoothly connected at the connection point. In order to achieve smooth connection, it is usually necessary to align control points P3 and P4 and ensure that the tangent direction at the connection point is consistent.

[0197] 3) Calculate the curve: For each segment of the cubic Bezier curve, use the cubic Bezier curve formula to calculate the points on the curve. If t changes from 0 to 1, all the points on the curve in this segment can be obtained.

[0198] (2) Collision avoidance strategy: Through dynamic obstacle modeling, the path is adjusted in real time to ensure the safe flight of the drone in complex environments. near to x new Whether the path collides with an obstacle. If no collision occurs, x new Add to the tree: in, Indicates the current set or state. If x new Close enough to the target point x goal, then it is considered that a feasible path has been found. In order to ensure the safety of UAV collision avoidance in a dynamic environment, collision avoidance constraints can be introduced in the path planning process of RRT, specifically:

[0199] Treat obstacles as time-dependent states O(t) and ensure a safe distance d between the path and the obstacles safe The following relationship is satisfied:

[0200] ||x new -O(t)||>d safe

[0201] When optimizing the path, dynamic collision avoidance constraints are added to ensure that the generated path does not collide with obstacles in the entire time domain. During execution, the tree structure and path are updated in real time to adapt to environmental changes. MPC can be combined to adjust the path in real time to ensure the effectiveness of the collision avoidance strategy.

[0202] In the embodiment of the present application, the model predictive control and collision avoidance strategy module is based on the environmental data collected and processed by the multi-source data fusion and environmental perception module, as well as the data reflecting the real-time status of the physical drone by the digital twin and real-time data interaction module, and generates the optimal path and implements the collision avoidance strategy through the combination of the MPC algorithm and the RRT algorithm. MPC provides the basic control strategy for short-term path optimization, while RRT is responsible for path fine-tuning in a dynamic environment to ensure the safety of the drone.

[0203] In an embodiment of the present application, the drone digital twin system also includes a system verification and optimization module, which verifies the system functions through virtual simulation and actual testing. The simulation environment is based on the modeling of the actual physical environment. The system is continuously optimized during the test process. By analyzing the feedback data, the control strategy and model parameters are adjusted to improve the system's adaptability in complex environments. Specifically, the system verification and optimization module passes the feedback data back to the aforementioned modules to promote continuous optimization of the system, especially in terms of control strategies, path planning, and collision avoidance strategies, to ensure the efficient operation of the system in different complex environments. The implementation process of the system verification and optimization module includes the following:

[0204] (1) Virtual simulation test: Perform various scenario tests in a virtual environment to verify the UAV’s collision avoidance strategy and path planning effect;

[0205] (2) Continuous optimization: Based on feedback data and performance indicators, optimize control strategies and models to improve the system's adaptability in complex environments;

[0206] (3) Virtual space verification: Simulate and test in a virtual environment to verify the functions and strategies of the system and ensure its reliability and security;

[0207] (4) Continuous optimization and adaptability: Through feedback and data analysis, the system continuously optimizes its control strategies and models to improve its ability to adapt to complex environments.

[0208] In summary, the UAV path planning method based on digital twins provided in the embodiment of the present application realizes efficient autonomous navigation and obstacle avoidance of the UAV digital twin in a virtual-real environment by combining the physical model with the data-driven model. The UAV path planning method based on digital twins is applied to the UAV digital twin system, which includes six core modules: a physical data hybrid drive integration module, a multi-source data fusion and environment perception module, a high-precision three-dimensional model and virtualization module, a digital twin and real-time data interaction module, a model predictive control and collision avoidance strategy module, and a system verification and optimization module. Physical data hybrid drive is used to accurately model and predict the dynamic behavior of drones and their operating environment, that is, to combine physical models with data-driven models to achieve dynamic modeling and prediction of drones and their operating environment; multi-source data fusion processes multiple sensor data through particle filters to ensure the perception accuracy and consistency of data; high-precision three-dimensional model reconstruction combines Delaunay triangulation algorithm and Poisson surface reconstruction algorithm to generate high-precision three-dimensional models in virtual environments; digital twin and real-time data interaction module is used to create and maintain real-time digital twins of drones and their environments; model predictive control and collision avoidance strategy module uses MPC algorithm and RRT path planning algorithm for real-time path optimization, and introduces collision avoidance strategy to ensure real-time path planning and collision avoidance strategy in dynamic environments. The system verification and optimization module verifies and continuously optimizes the system through virtual simulation and actual testing. The various modules of the entire drone digital twin system provided in the embodiment of the present application operate independently and work together to ultimately achieve autonomous navigation, obstacle avoidance and mission planning of drones in complex environments.

[0209] Step S106 shown in the embodiment of the present application can intelligently plan the optimal flight path by performing path planning on the physical drone based on simulated flight environment state data in combination with a predictive control algorithm and a path planning algorithm. At the same time, the path planning algorithm can effectively avoid obstacles, improve the safety and stability of the drone flight, and by optimizing the flight path, it can reduce flight time and energy consumption, and improve the operating efficiency of the drone. Finally, the target flight path is transmitted to the physical drone to control it to fly along the target flight path, so that the physical drone can fly along the optimal path, improving the efficiency and success rate of mission execution, while reducing operational complexity and potential safety risks.

[0210] Steps S101 to S106 shown in the embodiment of the present application are as follows: obtaining the first initial multi-source environmental perception data corresponding to the physical UAV in the target flight area; preprocessing the first initial multi-source environmental perception data to obtain the first target multi-source environmental perception data; the first target multi-source environmental perception data includes the first target environmental data; constructing a three-dimensional environmental model based on the first target environmental data; constructing a digital twin based on the geometric model, physical model, control model, target physical data hybrid dynamic model and three-dimensional environmental model corresponding to the physical UAV; inputting the second target multi-source environmental perception data into the digital twin, dynamically simulating the flight behavior of the physical UAV in the virtual space based on the second target multi-source environmental perception data through the digital twin, and obtaining the simulated flight environment state data for the physical UAV; based on the simulated flight environment state data, combining the target prediction control algorithm and the target path planning algorithm to plan the path of the physical UAV to obtain the target flight path, and transmitting the target flight path to the physical UAV to control the physical UAV to fly along the target flight path. The embodiment of the present application can remove noise, redundant and erroneous data, improve data quality, and provide a clean and effective data source for subsequent processing by preprocessing the first initial multi-source environmental perception data. At the same time, by fusing data from different sources to form a unified data representation, it is convenient for subsequent analysis and modeling, so that the system can respond quickly and reduce the delay caused by data fusion processing; the three-dimensional environmental model constructed based on the first target environmental data is the basis for the interaction between digital twins and real-time data. The virtual environment constructed by the three-dimensional environmental model provides a reliable simulation scene for digital twins and path planning, and is the basis of digital twins. At the same time, the three-dimensional environmental model provides drones with more sophisticated environmental perception capabilities. , making the navigation and obstacle avoidance of drones in complex environments more accurate and reliable; through the real-time input of the second target multi-source environmental perception data, the simulation process of the digital twin can dynamically adapt to the changes in the actual environment, and the physical drone and its virtual twin can be closely synchronized, thereby improving the real-time and accuracy of the simulation; based on the simulated flight environment state data, combined with the predictive control algorithm and the path planning algorithm, the path planning of the physical drone can be carried out, and the optimal flight path can be intelligently planned. At the same time, the path planning algorithm can effectively avoid obstacles, improve the safety and stability of the physical drone flight, and, by optimizing the flight path, can reduce the flight time and energy consumption, and improve the operation efficiency of the physical drone.

[0211] See also Figure 2 The embodiment of the present application also provides a UAV path planning system 200 based on digital twins, which can implement the above-mentioned UAV path planning method based on digital twins. The system 200 includes the following modules:

[0212] The data acquisition module 201 is used to acquire the first initial multi-source environment perception data corresponding to the physical UAV in the target flight area;

[0213] The data processing module 202 is used to pre-process the first initial multi-source environment perception data to obtain first target multi-source environment perception data; the first target multi-source environment perception data includes first target environment data;

[0214] A three-dimensional environment model building module 203, used to build a three-dimensional environment model based on the first target environment data;

[0215] A digital twin construction module 204 is used to construct a digital twin based on a geometric model, a physical model, a control model, a target physical data hybrid dynamic model and the three-dimensional environment model corresponding to the physical UAV;

[0216] The flight behavior simulation module 205 is used to input the second target multi-source environment perception data into the digital twin, and dynamically simulate the flight behavior of the physical UAV in the virtual space based on the second target multi-source environment perception data through the digital twin to obtain simulated flight environment state data for the physical UAV;

[0217] The flight path planning module 206 is used to plan the path of the physical UAV based on the simulated flight environment state data in combination with the target prediction control algorithm and the target path planning algorithm to obtain the target flight path, and transmit the target flight path to the physical UAV to control the physical UAV to fly along the target flight path.

[0218] It can be understood that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0219] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned drone path planning method based on digital twins when executing the computer program. The electronic device can be any smart terminal including a tablet computer, a car computer, etc.

[0220] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0221] See also Figure 3 , Figure 3The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0222] The processor 301 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC for short), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0223] The memory 302 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 302 can store an operating system and other applications. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 302, and the processor 301 calls and executes the drone path planning method based on digital twins in the embodiment of this application;

[0224] Input / output interface 303, used to implement information input and output;

[0225] The communication interface 304 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0226] A bus 305 that transmits information between the various components of the device (e.g., the processor 301, the memory 302, the input / output interface 303, and the communication interface 304);

[0227] The processor 301 , the memory 302 , the input / output interface 303 and the communication interface 304 are connected to each other in communication within the device via the bus 305 .

[0228] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned drone path planning method based on digital twins.

[0229] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0230] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0231] The UAV path planning method based on digital twin and the UAV path planning system based on digital twin provided in the embodiments of the present application obtain the first initial multi-source environmental perception data corresponding to the physical UAV in the target flight area; pre-process the first initial multi-source environmental perception data to obtain the first target multi-source environmental perception data; the first target multi-source environmental perception data includes the first target environmental data; construct a three-dimensional environmental model based on the first target environmental data; construct a digital twin based on the geometric model, physical model, control model, target physical data hybrid dynamic model and three-dimensional environmental model corresponding to the physical UAV; input the second target multi-source environmental perception data into the digital twin, dynamically simulate the flight behavior of the physical UAV in the virtual space based on the second target multi-source environmental perception data through the digital twin, and obtain the simulated flight environment state data for the physical UAV; based on the simulated flight environment state data, the physical UAV is path planned in combination with the target prediction control algorithm and the target path planning algorithm to obtain the target flight path, and the target flight path is transmitted to the physical UAV to control the physical UAV to fly along the target flight path. The embodiment of the present application can remove noise, redundant and erroneous data, improve data quality, and provide a clean and effective data source for subsequent processing by preprocessing the first initial multi-source environmental perception data. At the same time, by fusing data from different sources to form a unified data representation, it is convenient for subsequent analysis and modeling, so that the system can respond quickly and reduce the delay caused by data fusion processing; the three-dimensional environmental model constructed based on the first target environmental data is the basis for the interaction between digital twins and real-time data. The virtual environment constructed by the three-dimensional environmental model provides a reliable simulation scene for digital twins and path planning, and is the basis of digital twins. At the same time, the three-dimensional environmental model provides drones with more sophisticated environmental perception capabilities. , making the navigation and obstacle avoidance of drones in complex environments more accurate and reliable; through the real-time input of the second target multi-source environmental perception data, the simulation process of the digital twin can dynamically adapt to the changes in the actual environment, and the physical drone and its virtual twin can be closely synchronized, thereby improving the real-time and accuracy of the simulation; based on the simulated flight environment state data, combined with the predictive control algorithm and the path planning algorithm, the path planning of the physical drone can be carried out, and the optimal flight path can be intelligently planned. At the same time, the path planning algorithm can effectively avoid obstacles, improve the safety and stability of the physical drone flight, and, by optimizing the flight path, can reduce the flight time and energy consumption, and improve the operation efficiency of the physical drone.

[0232] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0233] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0234] The system embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0235] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0236] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0237] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0238] In the several embodiments provided in the present application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or units, which can be electrical, mechanical or other forms.

[0239] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0240] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0241] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0242] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. The UAV path planning method based on digital twin is characterized by: The method comprises the following steps: Acquire first initial multi-source environmental perception data corresponding to the physical UAV in the target flight area; Preprocessing the first initial multi-source environmental perception data to obtain first target multi-source environmental perception data; the first target multi-source environmental perception data includes first target environmental data; Building a three-dimensional environment model based on the first target environment data; Building a digital twin based on the geometric model, physical model, control model, target physical data hybrid dynamic model and the three-dimensional environment model corresponding to the physical UAV; Inputting the second target multi-source environmental perception data into the digital twin, dynamically simulating the flight behavior of the physical UAV in a virtual space based on the second target multi-source environmental perception data by the digital twin, and obtaining simulated flight environment state data for the physical UAV; Based on the simulated flight environment state data, the path of the physical UAV is planned in combination with a target prediction control algorithm and a target path planning algorithm to obtain a target flight path, and the target flight path is transmitted to the physical UAV to control the physical UAV to fly along the target flight path.

2. The method according to claim 1, characterized in that The method further comprises the step of pre-building the target physical data hybrid dynamic model, wherein the pre-building the target physical data hybrid dynamic model specifically comprises the following steps: Acquire a data-driven model and the physical model corresponding to the physical drone; the data-driven model is used to supplement the nonlinear characteristics of the physical model; Combining the physical model and the data-driven model, modeling the physical drone and the operating environment corresponding to the physical drone to obtain an initial physical data hybrid dynamic model; The initial physical data hybrid dynamic model is trained by objectively minimizing the loss function to obtain the target physical data hybrid dynamic model.

3. The method according to claim 1, characterized in that The expression of the target physical data hybrid dynamic model is: in, represents the derivative of the state vector x(t) with respect to time t, x(t) represents the state vector of the physical drone, t represents time, u(t) represents the control input, f(x(t),u(t)) represents the function of x(t) and u(t), η(t) represents the noise term, N(x(t),u(t); θ) represents the prediction result of the data-driven model, and θ represents the parameters of the data-driven model.

4. The method according to claim 1, characterized in that: The preprocessing of the first initial multi-source environment perception data to obtain first target multi-source environment perception data includes: Performing feature extraction processing on the first initial multi-source environment perception data to obtain first multi-source environment perception feature data; Performing time synchronization processing on the first multi-source environment perception feature data to obtain first time synchronization feature data; Performing spatial alignment processing on the first time synchronization feature data to obtain first time-space synchronization feature data; Performing noise filtering on the first spatiotemporal synchronization feature data to obtain first spatiotemporal clean feature data; The first spatiotemporal clean feature data is filtered through a particle filter to obtain the first target multi-source environment perception data; wherein the state prediction of each particle in the particle filter is implemented based on the target physical data hybrid dynamic model.

5. The method according to claim 1, characterized in that The constructing of a three-dimensional environment model based on the first target environment data includes: Modeling the first target environment data by using a target surface reconstruction algorithm to obtain a three-dimensional smooth surface model; The three-dimensional smooth surface model is subjected to triangulation meshing processing by a target triangulation algorithm to obtain the three-dimensional environment model.

6. The method according to claim 1, characterized in that The second target multi-source environmental perception data includes second target multi-source sensor data and second target environmental data, and the digital twin dynamically simulates the flight behavior of the physical UAV in a virtual space based on the second target multi-source environmental perception data to obtain simulated flight environment state data for the physical UAV, including: Dynamically updating virtual environment information in the digital twin based on the second target multi-source sensor data; Dynamically updating the three-dimensional environment model in the digital twin based on the second target environment data; Dynamically updating the digital twin based on the updated virtual environment information and the updated three-dimensional environment model; The flight behavior of the physical UAV is dynamically simulated in the virtual space based on the updated digital twin to obtain the simulated flight environment state data for the physical UAV.

7. The method according to claim 1, characterized in that The method of performing path planning on the physical UAV based on the simulated flight environment state data and combining a target prediction control algorithm and a target path planning algorithm to obtain a target flight path includes: Performing behavior prediction on the simulated flight environment state data according to the target prediction control algorithm to obtain a target flight behavior control sequence; The target flight behavior control sequence is path planned according to the target path planning algorithm and the target collision avoidance strategy to obtain the target flight path.

8. The UAV path planning system based on digital twin is characterized by: The system includes the following modules: A data acquisition module, used to acquire first initial multi-source environmental perception data corresponding to the physical UAV in the target flight area; A data processing module, configured to preprocess the first initial multi-source environment perception data to obtain first target multi-source environment perception data; the first target multi-source environment perception data includes first target environment data; A three-dimensional environment model building module, used to build a three-dimensional environment model based on the first target environment data; A digital twin construction module, used to construct a digital twin based on a geometric model, a physical model, a control model, a target physical data hybrid dynamic model and the three-dimensional environment model corresponding to the physical UAV; A flight behavior simulation module, used to input the second target multi-source environmental perception data into the digital twin, and dynamically simulate the flight behavior of the physical UAV in a virtual space based on the second target multi-source environmental perception data through the digital twin to obtain simulated flight environment state data for the physical UAV; The flight path planning module is used to plan the path of the physical UAV based on the simulated flight environment state data in combination with the target prediction control algorithm and the target path planning algorithm to obtain the target flight path, and transmit the target flight path to the physical UAV to control the physical UAV to fly along the target flight path.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images