Unmanned aerial vehicle scene interaction method and device based on digital twinning and storage medium

Abstract

The invention discloses an unmanned aerial vehicle scene interaction method and device based on digital twinning and a storage medium, and relates to the technical field of digital twinning and simulation, and the method comprises the steps: obtaining environment data of a target scene, and constructing a scene twinning model of the target scene based on the environment data; the unmanned aerial vehicle simulation model is imported into the scene twin model, a target scene interaction model is obtained, and the unmanned aerial vehicle simulation model comprises a flight dynamics module, a flight control module and a multi-sensor simulation module; in the target scene interaction model, interaction simulation of the unmanned aerial vehicle and the twin scene is carried out according to flight data in a target module, and the target module comprises at least one of a flight dynamics module, a flight control module and a multi-sensor simulation module. According to the invention, accurate simulation and interactive training of the unmanned aerial vehicle in a highly real virtual environment are realized, complexity and diversity of the real world can be reflected in simulation, and the accuracy and effectiveness of performance evaluation and training of the unmanned aerial vehicle are improved.

Description

Technical Field

[0001] The present application relates to the field of digital twin and simulation technology, and in particular to a drone scene interaction method, device and storage medium based on digital twin. Background Art

[0002] At present, the simulation of UAV scene interaction is mainly carried out through traditional simulation software or simulators. These tools are usually based on preset models and algorithms to simulate the flight process of UAVs in specific environments. These simulators can simulate some basic flight conditions, such as wind speed, terrain undulations, etc. However, when building UAV and environmental models, existing simulators are often limited by data acquisition and processing capabilities, resulting in insufficient model accuracy and difficulty in accurately reflecting the complexity and diversity of the real world. As a result, when simulating the UAV flight environment, only some simple scenes can be simulated, such as flat ground, etc., and it is impossible to fully simulate the complex environment in the real world, such as urban buildings. This limitation has greatly affected the performance evaluation and training effects of UAVs in practical applications.

[0003] Therefore, how to realize a drone interaction simulation that fits the real scene is a problem that needs to be solved urgently. Summary of the invention

[0004] The main purpose of this application is to provide a drone scene interaction method, device and storage medium based on digital twins, aiming to solve the technical problem of how to achieve a drone interaction simulation that fits the real scene.

[0005] To achieve the above objectives, the present application proposes a drone scene interaction method based on digital twins, and the drone scene interaction method based on digital twins includes: Acquire environmental data of a target scene, and construct a scene twin model of the target scene based on the environmental data; Importing a UAV simulation model into the scene twin model to obtain a target scene interaction model, wherein the UAV simulation model includes a flight dynamics module, a flight control module, and a multi-sensor simulation module; In the target scene interaction model, an interaction simulation between the UAV and the twin scene is performed based on the flight data in the target module, wherein the target module includes at least one of the flight dynamics module, the flight control module and the multi-sensor simulation module.

[0006] In one embodiment, the step of constructing a scene twin model of the target scene based on the environmental data includes: Performing data cleaning on the environmental data, and performing coordinate conversion and time mapping on the cleaned environmental data to obtain target environmental data, wherein the target environmental data is environmental data under a unified time and space reference; A scene twin model of the target scene is constructed based on the target environment data.

[0007] In one embodiment, the target environment data includes target geographic information data and target building information data, and the step of constructing the scene twin model of the target scene based on the target environment data includes: Acquire the spatiotemporal information of the target environment data, and fuse the target geographic information data and the target building information data based on the spatiotemporal information to form a digital base plate of the target scene; A scene twin model of the target scene is constructed based on the digital baseplate.

[0008] In one embodiment, in the target scene interaction model, the step of performing interactive simulation between the drone and the twin scene based on the flight data in the target module includes: Acquire weather data of the target scene, and import the weather data into the target scene interaction model to obtain a new target scene interaction model; in the new target scene interaction model, perform interaction simulation between the drone and the twin scene according to the weather data and the flight data in the target module; and / or The flight area restriction information of the target scene is obtained, and the flight area restriction information is imported into the target scene interaction model to obtain a new target scene interaction model; in the new target scene interaction model, the interaction simulation between the UAV and the twin scene is performed according to the flight area restriction information and the flight data in the target module.

[0009] In one embodiment, a weather model is integrated into the new target scene interaction model, and the weather model is constructed based on the weather data; The step of performing interactive simulation between the UAV and the twin scene according to the weather data and the flight data in the target module includes: During the interactive simulation process between the UAV and the twin scene, the weather model is called to apply environmental pressure to the UAV and monitor the sensor data; The sensor data is transmitted to the flight dynamics module to adjust the simulated flight attitude of the UAV in the twin scene.

[0010] In one embodiment, the flight data includes sensor data in the multi-sensor simulation module, and the new target scene interaction model integrates a flight restriction area, and the flight restriction area is constructed based on the flight area restriction information; The step of performing interactive simulation between the UAV and the twin scene according to the flight area restriction information and the flight data in the target module includes: During the interactive simulation process between the drone and the twin scene, monitoring the sensor data; If the drone position in the sensor data is monitored to be within the buffer range of the flight restricted area, the control instructions in the flight control module are adjusted according to the restriction conditions of the flight restricted area, and the drone is controlled to perform simulated flight in the twin scene based on the adjusted control instructions.

[0011] In one embodiment, the drone scene interaction method based on digital twins further includes: During the interactive simulation between the UAV and the twin scene, based on the distribution of buildings in the target scene interaction model, the transmission attenuation of the radio signal between the control signal transmission source in the flight control module and the UAV is calculated to obtain the attenuated radio signal; The flight control module is called to control the UAV to perform interactive simulation with the twin scene according to the attenuated radio signal.

[0012] In one embodiment, the drone simulation model further includes a fault simulation module, and the drone scene interaction method based on digital twins further includes: During the interactive simulation of the UAV and the twin scene, the fault simulation module is called to generate a fault in the UAV, so as to simulate the flight of the UAV in the twin scene after the fault occurs.

[0013] In addition, to achieve the above-mentioned purpose, the present application also proposes an electronic device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the drone scene interaction method based on digital twins as described above.

[0014] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the drone scene interaction method based on digital twins as described above are implemented.

[0015] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the drone scene interaction method based on digital twins as described above.

[0016] One or more technical solutions proposed in this application have at least the following technical effects: This application first uses environmental data of real scenes to perform scene modeling based on digital twins, realizes accurate digital reconstruction of the real-world environment, and thus achieves high-precision and high-fidelity simulation of the target scene; then the UAV simulation model is imported into the scene twin model to obtain the target scene interaction model, so as to integrate advanced simulation technology to combine the dynamic characteristics, control logic and perception capabilities of the UAV with the digital twin environment, thereby realizing a true simulation of the behavior and reaction of the UAV in the virtual environment, and achieving the preliminary technical effect of performance testing and training of UAVs in complex environments; finally, in the target scene interaction model, the interaction between the UAV and the twin scene is simulated according to the flight data in the target module, so as to realize dynamic interaction and instant response of the UAV in the virtual scene through real-time data processing and simulation feedback mechanism, so as to simulate the complex interactive behavior of the UAV in the real environment, and improve the simulation authenticity and training effectiveness.

[0017] In summary, by collecting and processing real-scene data and building digital twin modeling, building a scene twin model and importing a UAV model that includes flight dynamics, flight control and multi-sensor simulation for real-time interactive simulation, the problems of low model accuracy and inability to simulate complex environments caused by limited data acquisition and processing capabilities of traditional simulation software are avoided. Accurate simulation and interactive training of UAVs are achieved in a highly realistic virtual environment, which can reflect the complexity and diversity of the real world in the simulation, thereby improving the accuracy and effectiveness of UAV performance evaluation and training. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 A flowchart diagram of the first embodiment of the drone scene interaction method based on digital twins of this application is provided; Figure 2 A flow chart of the second embodiment of the drone scene interaction method based on digital twins provided in this application; Figure 3 A brief flowchart of the drone scene interaction method based on digital twins provided in Example 2 of the present application; Figure 4A schematic diagram of the twin model construction process of the drone scene interaction method based on digital twins provided in Example 2 of the present application; Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the drone scene interaction method based on digital twins in an embodiment of the present application.

[0021] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0022] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0023] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0024] The main solution of the embodiment of the present application is: obtaining the environmental data of the target scene, and constructing a scene twin model of the target scene based on the environmental data; importing the UAV simulation model into the scene twin model to obtain a target scene interaction model, wherein the UAV simulation model includes a flight dynamics module, a flight control module and a multi-sensor simulation module; in the target scene interaction model, performing an interaction simulation between the UAV and the twin scene according to the flight data in the target module, wherein the target module includes at least one of the flight dynamics module, the flight control module and the multi-sensor simulation module.

[0025] At present, the simulation of UAV scene interaction is mainly carried out through traditional simulation software or simulators. These tools are usually based on preset models and algorithms to simulate the flight process of UAVs in specific environments. These simulators can simulate some basic flight conditions, such as wind speed, terrain undulations, etc. However, when constructing UAV and environmental models, existing simulators are often limited by data acquisition and processing capabilities, resulting in insufficient model accuracy and difficulty in accurately reflecting the complexity and diversity of the real world. As a result, when simulating the UAV flight environment, only some simple scenes, such as flat ground, can be simulated, and complex environments in the real world, such as urban buildings, cannot be fully simulated. This limitation greatly affects the performance evaluation and training effect of UAVs in practical applications. Therefore, how to achieve a UAV interaction simulation that fits the real scene is a problem that needs to be solved urgently.

[0026] The present application provides a solution, which avoids the problems of low model accuracy and inability to simulate complex environments caused by limited data acquisition and processing capabilities of traditional simulation software by collecting and processing data of real scenes, as well as digital twin modeling, and by building a scene twin model and importing a drone model that includes flight dynamics, flight control and multi-sensor simulation for real-time interactive simulation. It realizes precise simulation and interactive training of drones in a highly realistic virtual environment, thereby reflecting the complexity and diversity of the real world in the simulation, thereby improving the accuracy and effectiveness of drone performance evaluation and training.

[0027] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device that can realize the above functions. The following takes an electronic device as an example to illustrate this embodiment and the following embodiments.

[0028] Based on this, the embodiment of the present application provides a drone scene interaction method based on digital twins, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the drone scene interaction method based on digital twins of the present application.

[0029] In this embodiment, the drone scene interaction method based on digital twins includes steps S10 to S30: Step S10, acquiring environmental data of a target scene, and constructing a scene twin model of the target scene based on the environmental data; It should be noted that the target environment refers to the real-world environment in which the UAV will perform tasks, such as cities, forests, mountains, etc.; environmental data refers to various physical and geographic information of the target scene, including data from multiple fields such as urban entities (such as buildings, roads, bridges, etc.), traffic operations, meteorological environment, and public services; the scene twin model refers to a digital virtual model corresponding to the real-world environment, which can simulate various characteristics and dynamic behaviors of the real scene.

[0030] It is understandable that existing simulators are often limited by data acquisition and processing capabilities when constructing environmental models, resulting in insufficient model accuracy and difficulty in accurately reflecting the complexity and diversity of the real world. Therefore, step S10 is performed to use real-scene environmental data for scene modeling based on digital twins. This can avoid the problem of inaccurate simulation results due to lack of accurate environmental data, thereby achieving high-precision simulation of the complex environment in which drones operate.

[0031] For example, first, various data on urban construction and operation are collected, including data from multiple fields such as urban entities (such as buildings, roads, bridges, etc.), traffic operations, meteorological environment, public services, etc., and the data are used as the environmental data of the target scene; then, these data are imported into professional 3D modeling software, and through data processing and model construction, a high-precision, interactive scene twin model is generated. This model can accurately reflect the geometric structure and physical properties of the target scene, providing a real environmental foundation for subsequent drone simulation.

[0032] In a feasible implementation manner, the step of constructing the scene twin model of the target scene based on the environmental data in step S10 may further include steps S11-S12: Step S11, performing data cleaning on the environmental data, and performing coordinate conversion and time mapping on the cleaned environmental data to obtain target environmental data, wherein the target environmental data is environmental data under a unified time and space reference; It should be noted that data cleaning refers to the process of screening, correcting and removing errors from the collected environmental data, including removing noise, duplicate data, outliers, etc., to ensure the accuracy and reliability of the data; coordinate transformation refers to converting the coordinate points in different coordinate systems in the environmental data into a unified coordinate system to facilitate data integration and analysis. It can be a conversion from a geographic coordinate system to a projected coordinate system, or it can be a conversion from one projected coordinate system to another projected coordinate system. The specific method of coordinate conversion is not limited in this implementation; time mapping refers to adjusting the time information in the environmental data to a unified time reference to ensure the consistency of all data on the timeline; target environmental data refers to environmental data after data cleaning, coordinate transformation and time mapping. The environmental data is environmental data in a unified spatial coordinate system and a unified time coordinate axis. The data is an accurate and reliable data set under a unified space-time reference that is suitable for building a scene twin model.

[0033] In addition, it should be noted that the environmental data includes geographic information data and building information data. In the process of coordinate conversion and time mapping, if the corresponding opposite data of the geographic information data or building information data is missing, the opposite data is supplemented, and coordinate conversion and time mapping are performed based on the supplemented opposite data, wherein the opposite data is the building information data corresponding to the geographic information data, or the geographic information data corresponding to the building information data. Specifically, if the geographic information data refers to information about natural geographical features such as topography, landforms, and vegetation, then the opposite data refers to the building information data at the same geographical location, such as the location and structure of the building; conversely, if the building information data refers to various information about the building, then the opposite data refers to the geographic information data of the geographical location where these buildings are located. For the missing building information data, it can be supplemented in the following ways: a. Use historical archives, urban planning drawings or building information models (BIM, Building Information Modeling) to obtain the missing building information data. b. Collect detailed information about buildings through on-site surveys or remote sensing technologies (such as laser scanning and drone photography); missing geographic information data can be supplemented in the following ways: a. Use satellite remote sensing images, topographic mapping or GIS (Geographic Information System) databases to supplement geographic information such as terrain and vegetation. b. Use GIS interpolation technology to infer the characteristics of the missing area based on the geographic information of the surrounding area, and then integrate the supplemented opposite data with the original environmental data.

[0034] It is understandable that since the original environmental data often has error problems, and the environmental data may come from different data sources and use different coordinate systems and time standards, resulting in inconsistencies in time and space, performing step S11 can avoid the problem of inaccurate scene twin models due to data quality issues and inaccurate environmental modeling caused by inconsistent data time and space benchmarks, thereby achieving the effect of improving data accuracy and reliability in the process of environmental model construction.

[0035] For example, the collected raw environmental data may contain noise, outliers and incomplete information. Through the data cleaning step, these noise and outliers in the environmental data are removed, the missing values ​​in the data are filled, and the accuracy and completeness of the data are ensured. Then, the cleaned environmental data is subjected to coordinate transformation, for example, the data collected by different sensors are converted from their independent coordinate systems to a unified geographic coordinate system (such as WGS84) to facilitate data consistency and subsequent processing. At the same time, time mapping is performed to synchronize the timestamps of different data records to the same time reference, such as UTC time. Finally, the target environmental data is obtained, which can accurately represent the flight environment and status of the drone under a unified time and space reference, providing a reliable basis for subsequent flight simulation and decision-making. For example, the terrain, weather and obstacle information collected at different time points are converted into three-dimensional maps and time series data in a unified coordinate system so that the drone can accurately identify and avoid obstacles in the simulated environment.

[0036] Step S12: construct a scene twin model of the target scene based on the target environment data.

[0037] For example, using 3D modeling software and GIS technology, the cleaned and standardized target environment data is imported into the modeling platform: first, a terrain model is created based on the terrain elevation data, and then layer information such as buildings, roads, and vegetation is superimposed to construct the geometric structure of the target scene; then, surface details such as color and material are added to the model through texture mapping technology to enhance the realism of the model; finally, dynamic environmental factors such as weather and traffic flow are integrated into the model to create a scene twin model that can simulate the dynamic behavior of the real world. This model can be run in simulation software to provide an accurate environmental background for drone simulation.

[0038] In this implementation, by performing data cleaning, coordinate conversion, and time mapping, the inaccurate scene twin model caused by data quality problems, as well as the difficulties in data integration and simulation errors caused by inconsistent coordinate systems and time standards are avoided. This enables the construction of an accurate and reliable scene twin model of the target scene under a unified space-time reference, thereby ensuring that the model can truly reflect the actual conditions of the target environment, providing an accurate environmental basis for UAV simulation, and thus improving the accuracy and practicality of the simulation results.

[0039] Step S20, importing the UAV simulation model into the scene twin model to obtain a target scene interaction model, wherein the UAV simulation model includes a flight dynamics module, a flight control module and a multi-sensor simulation module; It should be noted that the UAV simulation model refers to a computer-simulated UAV system, which includes a flight dynamics module, a flight control module and a multi-sensor simulation module, which is used to simulate the flight behavior and perception capabilities of the UAV; the target scene interaction model refers to embedding the UAV simulation model into the scene twin model to form a composite model that can simulate the flight and interaction of the UAV in the target scene.

[0040] It is understandable that since the performance and response of the UAV need to be verified in a high-precision virtual environment, step S20 is performed. By integrating advanced simulation technology, the dynamic characteristics, control logic and perception capabilities of the UAV are combined with the digital twin environment, which can avoid the problem that the traditional single simulation environment cannot comprehensively evaluate the performance of the UAV, and provide an effective simulation platform for comprehensively evaluating the flight performance and operational reliability of the UAV in a specific environment.

[0041] Exemplarily, first, a UAV simulation model is developed or selected that includes a flight dynamics module, a flight control module, and a multi-sensor simulation module, wherein the flight dynamics module is used to simulate the aerodynamic characteristics and gravity influence of the UAV, the flight control module is used to simulate the navigation, guidance, and flight control logic of the UAV, and the multi-sensor simulation module is used to simulate the camera, radar, GPS and other sensing devices of the UAV; then, this UAV simulation model is embedded into the previously constructed scene twin model through an API interface or software integration to form a complete interactive model that can simulate the flight of the UAV in the target scene.

[0042] Step S30, in the target scene interaction model, performing an interaction simulation between the UAV and the twin scene according to the flight data in the target module, wherein the target module includes at least one of the flight dynamics module, the flight control module and the multi-sensor simulation module.

[0043] It should be noted that the target module refers to a specific part of the UAV simulation model, namely, at least one of the flight dynamics module, flight control module or multi-sensor simulation module, which is used for specific simulation tasks; flight data refers to various flight-related information generated by the UAV during the simulation process, such as position, speed, attitude, etc.

[0044] It is understandable that since it is necessary to simulate the interaction between the UAV and the environment, step S30 is performed. Through real-time data processing and simulation feedback mechanism, it is possible to avoid the problem of being unable to simulate various dynamic interactions that the UAV may encounter in the real environment, and realize the simulation of the UAV's perception, decision-making and execution of actions in a complex environment, thereby improving the authenticity and effectiveness of UAV operation training and mission rehearsal.

[0045] For example, in the constructed target scene interactive model, the simulation process is started, and the flight dynamics module of the UAV simulation model begins to generate flight data according to the preset flight path and flight parameters. These data are transmitted to the scene twin model in real time, and the latter updates the position, speed and attitude of the UAV based on these data. At the same time, the multi-sensor simulation module simulates the environmental information received by the UAV sensor, such as obstacle location, ground features, etc., and the flight control module adjusts the flight instructions of the UAV based on this information. Through this interactive simulation, the flight performance and response of the UAV in the target scene can be observed and analyzed in real time, thereby verifying the flight performance and mission execution capability of the UAV.

[0046] In a feasible implementation, the step of performing interactive simulation between the UAV and the twin scene based on the flight data in the target module in the target scene interaction model in step S30 may include steps A31-A32: Step A31, obtaining weather data of the target scene, and importing the weather data into the target scene interaction model to obtain a new target scene interaction model; It should be noted that weather data refers to the meteorological conditions information of the target scene within a specific time, including temperature, humidity, wind speed, wind direction, precipitation, visibility, etc.

[0047] It is understandable that since UAV flight is greatly affected by weather conditions, weather factors need to be considered to ensure the authenticity of the simulation. Therefore, performing step A31 can avoid the problem of inaccurate UAV simulation results due to ignoring weather factors, thereby improving the adaptability and predictability of the simulation model to the actual flight environment.

[0048] For example, by accessing the weather service API or using weather station data, the weather data of the target scene, such as cloud thickness, rainfall intensity, wind direction and speed, etc., is obtained in real time, and then the weather data is formatted and imported into the drone simulation software, which already contains the interactive model of the target scene. In the software, the weather data is used to update the model's environmental parameters, such as wind speed, precipitation, and visibility, thereby creating a new interactive model of the target scene that takes into account real-time weather conditions.

[0049] Step A32: In the new target scene interaction model, an interaction simulation between the UAV and the twin scene is performed based on the weather data and the flight data in the target module.

[0050] It is understandable that in order to evaluate and train the flight performance and response strategies of the UAV under specific weather conditions, step A32 is performed to avoid the problem of being unable to simulate the flight behavior of the UAV under complex weather conditions, thereby improving the UAV's ability to fly safely and perform tasks in a changeable weather environment.

[0051] Exemplarily, in the updated target scene interaction model, the simulation software starts to simulate the flight of the drone. The software calculates the dynamic response of the drone in real time based on the imported weather data and the flight instructions in the drone flight control module. For example, if strong winds are encountered in the simulation, the software will adjust its flight path and attitude according to the drone's flight dynamics model. At the same time, the multi-sensor simulation module simulates the performance of drone sensors under specific weather conditions, such as the reduced visibility of the camera in fog. Through this interactive simulation, the flight performance of the drone under different weather conditions can be observed, and the flight strategy can be adjusted accordingly or the autonomous response capability of the drone can be trained.

[0052] In this implementation, by importing weather data into the target scene interaction model, the UAV can test and optimize its flight strategy for different weather conditions in a simulated environment, avoiding the problem of distortion of UAV simulation results caused by ignoring real-time weather conditions. It realizes the interactive simulation of the UAV and the twin scene while considering actual meteorological factors, thereby improving the flight safety and mission execution efficiency of the UAV under complex meteorological conditions.

[0053] In another feasible implementation, in the target scene interaction model described in step S30, the step of performing interactive simulation between the drone and the twin scene based on the flight data in the target module may include steps B31-B32: B31, obtaining flight area restriction information of the target scene, and importing the flight area restriction information into the target scene interaction model to obtain a new target scene interaction model; It should be noted that flight area restriction information refers to various spatial restrictions set for drone flight operations, including but not limited to no-fly zones, restricted altitude zones, danger zones, air traffic control zones, etc.

[0054] It is understandable that since drones must comply with relevant laws and safety regulations when performing missions in real scenarios, performing step B31 can avoid the problem of drones violating regulations in simulation training due to ignoring flight area restrictions, thereby ensuring that drone simulation training complies with actual flight regulations and safety management requirements.

[0055] For example, by accessing the database provided by the official aviation management department or using GIS tools, the flight area restriction information of the target scene is collected, including no-fly zones, restricted altitude zones, air corridors, etc. This information is then integrated into the drone simulation software in a digital form to update the environmental parameters of the target scene interaction model. In the software, this restriction information is converted into virtual barriers or rules, thereby creating a new target scene interaction model containing all flight restrictions in the simulation environment.

[0056] B32, in the new target scene interaction model, the interaction simulation between the UAV and the twin scene is performed according to the flight area restriction information and the flight data in the target module.

[0057] It is understandable that in order to evaluate and train the UAV's flight capabilities and response strategies under flight restrictions, step B32 is performed to avoid the problem of being unable to simulate the UAV's flight behavior in restricted airspace in a simulation environment, thereby improving the UAV's operational safety and mission execution efficiency in a complex airspace environment.

[0058] Exemplarily, in the updated target scenario interactive model, the simulation software starts to perform the UAV flight simulation. The software monitors the position and status of the UAV in real time based on the imported flight area restriction information and the UAV's flight data, such as the predetermined flight path, speed, and altitude. If the UAV approaches or attempts to violate any flight restrictions, the software will trigger a corresponding response, such as automatically adjusting the flight path, slowing down, or climbing to comply with the restrictions. Through this interactive simulation, the operator can observe the performance of the UAV while complying with the flight restrictions and adjust and train the flight strategy for potential conflicts.

[0059] In this embodiment, by integrating the flight area restriction information into the target scene interaction model and performing UAV simulation based on it, the problem that the UAV may violate actual flight rules and restrictions during simulation training is avoided, and the actual flight safety and management requirements are fully considered in the simulation environment, thereby ensuring that the UAV can accurately comply with airspace restrictions during simulation training, improving the UAV operator's ability to adapt to complex airspace environments and the UAV's flight safety in the real world.

[0060] It should be noted that steps A31~A32 and steps B31~B32 can be combined to obtain steps C31~C32: Step C31, obtain the weather data and flight area restriction information of the target scene, and import the weather data and the flight area restriction information into the target scene interaction model to obtain a new target scene interaction model; Step C32, in the new target scene interaction model, perform an interactive simulation between the drone and the twin scene based on the weather data, the flight area restriction information and the flight data in the target module. This step is the same or similar to the above steps A31~A32 and steps B31~B32, and can be referred to the above introduction, which will not be repeated.

[0061] This embodiment provides a UAV scene interaction method based on digital twins, which collects and processes real scene data and models digital twins, builds a scene twin model and imports a UAV model that includes flight dynamics, flight control, and multi-sensor simulation for real-time interactive simulation. It avoids the problems of low model accuracy and inability to simulate complex environments caused by limited data acquisition and processing capabilities of traditional simulation software, and achieves accurate simulation and interactive training of UAVs in a highly realistic virtual environment, so that the complexity and diversity of the real world can be reflected in the simulation, thereby improving the accuracy and effectiveness of UAV performance evaluation and training.

[0062] In a feasible implementation manner, the target environment data includes target geographic information data and target building information data, and step S12 may include steps S121-S122: Step S121, acquiring the spatiotemporal information of the target environment data, and fusing the target geographic information data and the target building information data based on the spatiotemporal information to form a digital base plate of the target scene; It should be noted that spatiotemporal information refers to the spatial dimension (such as location, shape, size) information and temporal dimension (such as change history, update timestamp) information of the geographic information in the target environment, which can reflect the temporal and spatial benchmarks of environmental elements; data fusion refers to the process of integrating data from different sources, formats or types (such as geographic information data and building information data) through certain algorithms and processes to form a unified and coordinated data set; digital baseplate refers to a comprehensive, multi-level, dynamically updated digital map, which contains detailed information such as the geographic information, building information, infrastructure, etc. of the target scene, providing basic data support for the construction of scene twin models.

[0063] It is understandable that, since it is necessary to create an accurate and comprehensive scene description to support subsequent simulation and simulation, performing step S121 can avoid the problem of incomplete or inconsistent environmental information caused by data islands, thereby providing a unified data foundation for better understanding and simulation of the target scene.

[0064] Exemplarily, GIS is used to collect geographic information data of the target environment, including terrain, landforms, vegetation, etc., and the three-dimensional structure and attribute information of the building are obtained through BIM software. Then, the timestamps and spatial coordinate information of these data are matched, and data fusion algorithms (such as spatial overlay analysis, attribute matching, etc.) are used to integrate the geographic information data and building information data into a unified coordinate system. For example, the vegetation coverage data is combined with the terrain model, and the vegetation model is added to the corresponding area according to the vegetation coverage data. Different materials and maps are used to distinguish different types of vegetation, such as forests, grasslands, bushes, etc., thereby creating a digital base containing spatiotemporal information.

[0065] Step S122: construct a scene twin model of the target scene based on the digital baseplate.

[0066] For example, after obtaining the digital baseplate, use professional simulation software or a custom development platform to import the data in the digital baseplate into the scene construction module of the software platform. Through the three-dimensional modeling technology used in the module, the geographic information and building information are converted into a visual three-dimensional model, and the model is given corresponding physical properties and behavior rules. Then, combined with environmental simulation technology, such as weather systems and traffic flow simulation, dynamic environmental factors are integrated into the scene twin model. Finally, a digital twin model that can reflect the state and behavior of the target scene in real time is constructed for simulation training and mission planning of drones.

[0067] In this implementation, by integrating spatiotemporal information and fusion of data, the problem of inaccurate modeling caused by data dispersion, incompatibility and inaccuracy is avoided, and a high-precision, dynamically updated digital base plate of the target scene is created in a digital environment. Based on this, a scene twin model that can truly reflect the real-world scene is constructed, which not only ensures the consistency and accuracy of the model, but also provides a reliable foundation for UAV simulation, training and mission planning, thereby improving the safety and efficiency of UAV operations.

[0068] In a feasible implementation manner, a weather model is integrated into the new target scene interaction model, and the weather model is constructed based on the weather data; The step of performing interactive simulation between the UAV and the twin scene according to the weather data and the flight data in the target module in step A32 may include steps A321 to A322: Step A321, during the interactive simulation process between the UAV and the twin scene, calling the weather model to apply environmental pressure to the UAV and monitoring the sensor data; It should be noted that the weather model refers to a mathematical model or calculation program that can simulate weather conditions in a specific environment. The model includes the simulation of meteorological factors such as wind speed, temperature, humidity, precipitation, and snowfall; sensor data refers to the information collected by various sensors on the drone (such as GPS, IMU, camera, radar, etc.) during operation. These data reflect the status of the drone, the surrounding environment, and various parameters during the flight. In addition, the sensor data simulates the influence of factors such as noise and drift to improve the authenticity of the simulation.

[0069] It is understandable that since it is necessary to simulate various weather conditions that may be encountered in actual flight in a simulation environment to test the performance of the UAV in a complex meteorological environment, performing step A321 can avoid the problem of inaccurate UAV simulation training results caused by ignoring the impact of weather, thereby achieving the effect of evaluating and improving the UAV's ability to cope with severe weather conditions in simulation.

[0070] Exemplarily, in the simulation system, the built-in weather model is first activated. The model can simulate corresponding environmental conditions such as wind speed, precipitation, and temperature according to preset weather scenarios or real-time weather data; these environmental conditions are then applied to the UAV model to simulate the flight state of the UAV under the influence of real weather. At the same time, the simulation system monitors the data output of each sensor on the UAV model in real time, such as wind speed changes displayed by the wind speed sensor and fuselage posture changes displayed by the IMU, to ensure that the UAV's response to environmental changes can be accurately recorded and analyzed.

[0071] Step A322, transmitting the sensor data to the flight dynamics module to adjust the simulated flight posture of the UAV in the twin scene.

[0072] It is understandable that since it is necessary to dynamically adjust the flight attitude of the UAV according to the real-time status of the UAV and the surrounding environment information to ensure the authenticity of the simulation and the stable flight of the UAV in the simulated environment, performing step A322 can avoid the problem of inaccurate UAV simulation flight control due to lack of real-time feedback, thereby realizing real-time simulation and optimization of UAV flight control strategies in a simulation environment.

[0073] For example, in the simulation system, sensor data is transmitted to the flight dynamics module in real time through the data interface. The module calculates the dynamic response of the drone under the current environmental conditions based on the received data, such as GPS position, IMU attitude and acceleration information. Then, the flight dynamics module sends adjustment instructions to the control module of the drone based on these calculation results to change the thrust and rudder deflection of the drone, thereby adjusting the flight attitude of the drone in the simulation environment. This ensures that the behavior of the drone in the simulation is as close to the actual situation as possible, and improves the fidelity and effectiveness of the simulation training. For example: 1) The interactive effect of rain on drone flight: by simulating the real rain (rainfall, raindrop size, etc.) model, the impact on the drone's flight attitude, performance, sensor accuracy, stability, and flight line of sight. First, rain directly acts on the surface of the drone, increasing the air resistance during flight, causing the drone to need more thrust to maintain speed and altitude. In the simulation of the interaction between the drone and rain, it is necessary to present the impact of the amount and intensity of rain on the speed and altitude of the drone. Secondly, rain hitting the drone will cause the drone to vibrate. The amount of rainfall and the size of raindrops affect the magnitude of the vibration of the drone, which in turn affects the stability of the drone. Thirdly, rain will block the line of sight of optical sensors (such as cameras, lidar, etc.), reducing their performance or completely failing. The blurring of the drone's line of sight can be simulated according to the density of rain, etc. Simulating the impact of rain on drone flight can more accurately evaluate the impact of rain on drone flight performance and provide valuable reference for the design and flight control strategy of drones; 2) The interactive impact of wind on the flight of drone models: wind field simulation includes wind speed, wind direction, wind force, etc. In the constructed twin model, the wind field model is superimposed to evaluate the impact of wind on drone performance. The interactive effects of wind on the flight attitude of drones include: pitch angle, when the wind blows from above or below, the pitch angle of the drone may change. The drone needs to adjust the thrust or the angle of the control surface; horizontal angle, when the wind blows from the side, the yaw angle of the drone will be affected, causing the drone to deviate from the predetermined heading; roll angle, in strong wind or gusty conditions, the drone may experience rolling motion, depending on the wind strength and the stability of the drone; wind affects the ground speed and airspeed of the drone, as well as the thrust required to maintain altitude. In strong wind conditions, the altitude and speed of the drone will be affected; 3) The interactive effect of snow on the flight of the drone: In the simulated snow environment, in addition to the size and density of the snow particles, the temperature is also included. The impact of simulated snow particles and snow accumulation may cause the drone to vibrate, and it is also necessary to consider the additional pitch moment of the drone, the change of its pitch angle, etc., which affects its flight stability.In addition, if the temperature affects the shape and melting speed of snow, the impact of temperature must be fully considered when simulating snowy environmental conditions. If the ambient temperature is too low, the drone will freeze, which will affect the sensitivity, speed or power of the drone's sensors. Weather conditions often do not appear alone, and various factors must be considered when simulating the environment. For example, snowy weather is often accompanied by wind. In interactive simulations, the drone's impact on environmental perception is also multifaceted, including flight heading, flight horizontal angle, pitch angle, flight speed, aircraft vibration, etc.

[0074] In this implementation, the weather model is called during the interactive simulation process to apply environmental pressure to the UAV, and sensor data is monitored in real time. These data are then transmitted to the flight dynamics module to adjust the simulated flight attitude of the UAV. This avoids the problem of ignoring the impact of weather and sensor feedback in actual flight during UAV simulation training, and achieves a realistic simulation of the flight performance and dynamic response of the UAV under complex weather conditions in a simulation environment, thereby improving the accuracy and practicality of UAV simulation training, ensuring that the UAV can better adapt to various environmental challenges in actual operations, and improving flight safety and mission success rate.

[0075] In a feasible implementation manner, the flight data includes sensor data in the multi-sensor simulation module, and the new target scene interaction model integrates a flight restriction area, and the flight restriction area is constructed based on the flight area restriction information; The step of performing interactive simulation between the UAV and the twin scene according to the flight area restriction information and the flight data in the target module in step B32 may include steps B321 to B322: Step B321, monitoring the sensor data during the interactive simulation process between the drone and the twin scene; It is understandable that since it is necessary to track the status and surrounding environment of the drone in real time to ensure the accuracy and safety of the simulation training, performing step B321 can avoid the problem of the drone exceeding the safe operating range in the simulation due to lack of real-time monitoring, thereby achieving the effect of real-time monitoring and feedback of the drone status.

[0076] For example, during the operation of the simulation system, data is collected in real time from various virtual sensors (such as virtual GPS, virtual IMU, etc.) on the drone model, including information such as the drone's position, speed, acceleration, and attitude. The simulation system receives these data streams through a dedicated monitoring module and displays them on the operator's monitoring interface, while storing them in the system's log file for subsequent analysis and evaluation.

[0077] Step B322: If it is monitored that the position of the UAV in the sensor data is within the buffer range of the flight restricted area, the control instructions in the flight control module are adjusted according to the restriction conditions of the flight restricted area, and the UAV is controlled to perform simulated flight in the twin scene based on the adjusted control instructions.

[0078] It should be noted that the drone position refers to the specific coordinate point of the drone in three-dimensional space, which can be provided by GPS or other positioning systems; the buffer range refers to an area of ​​a certain range set around the flight restricted area, which is used to give early warning and prevent the drone from entering the actual flight restricted area; the control instruction refers to the command in the drone flight control system used to guide the drone to perform specific actions, such as climbing, descending, turning, etc.

[0079] It is understandable that since it is necessary to ensure that the UAV complies with flight restrictions in the simulation and prevent it from entering prohibited flight areas, performing step B322 can avoid the problem of the UAV violating flight regulations in simulation training, thereby achieving the effect of training the UAV to comply with flight restrictions and automatic obstacle avoidance in a simulation environment. Through this adjustment, it can be ensured that the UAV can perform its tasks safely and compliantly in real flight.

[0080] Exemplarily, the geo-fence module in the simulation system continuously monitors the virtual location data of the drone. Once the drone is found to enter the preset flight restriction zone buffer, the system immediately activates the adjustment program of the flight control module. The program automatically generates new control instructions based on the specific conditions of the flight restriction zone (such as maximum flight altitude, no-fly zone, etc.), such as instructing the drone to climb or change course to avoid violating flight restrictions. These adjusted instructions are sent to the virtual flight control system of the drone, so as to adjust the simulated flight path and attitude of the drone in real time in the twin scene. For example: in the simulated no-fly zone, the drone will not be able to take off. In the twin model of the urban scene, the no-fly zone is enclosed by an electronic fence. Once the drone attempts to take off in the no-fly zone, the drone will not be able to take off. If the drone encounters the boundary of the simulated no-fly zone during flight, the drone will have a hovering effect or change its flight path to avoid entering the no-fly zone; in the height-restricted zone simulated in the twin model, the maximum flight altitude of the drone will be strictly limited. This means that the drone cannot fly above the set height limit. For example, if the height limit is set to 120 meters, then the drone's flight altitude will not exceed this value when flying in the restricted area. The height limit of a drone usually refers to the relative height, that is, the height from the take-off point or current position to the vertical height above the drone. When a drone encounters a restricted altitude area during flight, it will not be able to climb and will automatically lower its flight altitude. When the flight altitude is lowered, it needs to perceive the surrounding environment in real time and respond to environmental factors in a timely manner. For example, if it encounters buildings, mountains, trees, etc. during the descent, it is necessary to change the flight path to avoid collision.

[0081] In this embodiment, by monitoring sensor data and automatically adjusting flight control instructions according to flight restricted area conditions, the problem of violation of flight restrictions and potential safety risks that may occur in the UAV during simulated flight is avoided, and precise control of the UAV and flight restriction compliance training are achieved in a simulation environment, thereby ensuring that the UAV can effectively rehearse and adapt to safety rules in actual flight during simulation training, thereby improving the safety awareness of UAV operators and the autonomous obstacle avoidance capabilities of the UAV system.

[0082] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 2 , the drone scene interaction method based on digital twins also includes steps S01~S02: Step S01, during the interactive simulation between the UAV and the twin scene, based on the building distribution in the target scene interaction model, the transmission attenuation of the radio signal between the control signal transmission source in the flight control module and the UAV is calculated to obtain the attenuated radio signal; It should be noted that building distribution refers to the collection of spatial information such as the position, size, shape and height of all buildings in the target scene. This information is used to simulate the impact of buildings on radio signal propagation in the real world; the control signal transmitter refers to the virtual device responsible for sending control instructions in the UAV simulation model, usually the ground control station or remote control of the UAV; transmission attenuation refers to the degree of signal strength weakening caused by factors such as building obstruction and increased distance during the propagation of radio signals.

[0083] It is understandable that since the propagation characteristics of radio signals in complex environments such as cities in real environments will be affected by the external environment, step S01 is performed to simulate the propagation characteristics of radio signals in real environments, thereby avoiding the problem of ignoring the impact of buildings on radio signal propagation in simulation training, thereby achieving accurate simulation of radio signal attenuation in the simulation environment.

[0084] For example, the simulation system uses a ray tracing algorithm or a similar signal propagation model to calculate the transmission path of the control signal from the transmission source to the drone receiver based on the distribution of buildings. In this process, the algorithm takes into account the shielding and reflection effects of the building on the signal, thereby calculating the attenuation of the signal at each path point, and finally obtaining an attenuated radio signal strength value.

[0085] Step S02, calling the flight control module to control the UAV to perform interactive simulation with the twin scene according to the attenuated radio signal.

[0086] It is understandable that since it is necessary to adjust the control response of the UAV according to the signal attenuation to simulate the signal reception situation in actual flight, performing step S02 can avoid the problem that the UAV cannot accurately reflect the impact of signal attenuation on flight control in simulation training, thereby improving the reliability of UAV flight control in complex environments in a simulation environment.

[0087] Exemplarily, after receiving the attenuated radio signal strength value, the flight control module in the simulation system will adjust the control logic of the UAV according to this value. For example, if the signal attenuation is severe, the module may trigger a safety protocol to limit the flight speed of the UAV or change its flight path to maintain reliable communication with the control source. Specific operations may include adjusting the flight attitude, speed and direction of the UAV. These adjustments are reflected in real time in the interactive simulation of the UAV and the twin scene through the dynamic model of the simulation system, thereby simulating the control environment that the UAV may encounter in a real environment.

[0088] In this embodiment, by calculating the radio signal transmission attenuation based on the building distribution in the target scene interaction model, the problem of ignoring the impact of urban buildings on radio signal propagation in UAV simulation training is avoided, and the communication conditions of UAVs in complex urban environments are more realistically simulated in the simulation environment to ensure that the flight control response of the UAV in the interactive simulation matches the conditions that may occur due to signal attenuation in actual flight, thereby improving the safety and reliability of UAVs when flying in urban areas.

[0089] In a feasible implementation, the drone simulation model further includes a fault simulation module, and the drone scene interaction method based on digital twins further includes step S100: Step S100, during the interactive simulation of the UAV and the twin scene, the fault simulation module is called to generate a fault in the UAV, so as to simulate the flight of the UAV in the twin scene after the fault occurs.

[0090] It should be noted that fault generation refers to the artificial introduction of various preset fault conditions, such as engine failure, sensor failure, communication interruption, etc., through the fault simulation module during the UAV simulation process to simulate the problems that the UAV may encounter in actual flight.

[0091] It is understandable that in order to test and verify the response and processing capabilities of the UAV system when encountering a fault in a simulation environment, so as to ensure that the UAV can safely deal with emergencies in actual operations, step S100 is performed. This can avoid the problem of ignoring the importance of fault handling in UAV design and training, thereby reducing flight accidents and losses caused by faults, and realizing fault response training for UAVs in a safe simulation environment, thereby improving the reliability and robustness of the UAV system, and also providing UAV operators with experience and skills in dealing with emergencies.

[0092] For example, during the operation of the simulation system, the operator or the system automatically triggers the fault simulation module, which selects a specific fault type and occurrence time based on a preset fault library or a randomly generated fault scenario. For example, the module may simulate a sudden engine failure. At this time, the module will send a signal to the virtual power system of the drone to simulate a sudden drop in engine output power or complete shutdown. Then the drone model in the simulation system will immediately respond to this fault and show corresponding changes in flight characteristics, such as speed drop, altitude drop, or attitude loss. Subsequently, other systems of the drone will also respond to this fault, such as automatically switching to backup power or initiating an emergency landing procedure. During the entire process, the simulated flight status of the drone will be updated in real time and displayed in the twin scene, so that the operator or R&D personnel can observe and analyze the behavior of the drone in the event of a fault and the emergency response effect of the system.

[0093] In this implementation, by calling the fault simulation module in a simulation environment to generate faults for the UAV, safety risks and equipment losses caused by faults in actual flight tests are avoided, and the response strategy of the UAV system to sudden faults is evaluated and optimized in a safe virtual environment.

[0094] For example, in order to help understand the implementation process of the drone scene interaction method based on digital twins obtained by combining this embodiment with the above-mentioned embodiment 1, please refer to Figure 3 , Figure 3 A brief flow chart of the UAV scene interaction method based on digital twin is provided, specifically: First, a city twin model (i.e., a target scene twin model) is generated based on the city GIS data and the three-dimensional model data. Then, the model is imported and combined with the UAV simulation model to obtain a target scene interaction model, so as to simulate the dynamic interaction between the UAV and the environment in the target scene interaction model. The UAV simulation model includes a flight dynamics module, a control system (i.e., a flight control module), a sensor module (i.e., a multi-sensor module), and a fault effect simulation (i.e., a fault simulation module). In addition, weather data and no-fly (height-restricted) area information (i.e., flight area restriction information) can also be imported into the target scene interaction model. Then, the flight screen of the interactive simulation process is rendered, and the performance of the simulated interaction is evaluated and optimized, that is, the flight efficiency (such as energy consumption ratio), stability (such as attitude fluctuation range), safety (such as obstacle avoidance capability), etc. of the UAV are evaluated by analyzing the various parameters collected in real time during the flight process. Based on the evaluation results, the design parameters, control algorithms, or flight strategies of the UAV are adjusted to improve the performance of the UAV and enhance the market competitiveness of the product.

[0095] For further information, please refer to Figure 4 , Figure 4 A schematic diagram of the twin model construction process of the UAV scene interaction method based on digital twin is provided, specifically: Collect GIS data, BIM data, 3D models, etc. of the target scene, and clean the collected data to remove noise, duplicate data, outliers, etc. to ensure the accuracy and reliability of the data. Then unify the cleaned data in terms of time and space, unifying them into data under the same coordinate system and time reference. Then fuse the unified data, fuse all types of data, and form a unified city digital base, so as to build a city twin model based on the digital base, that is, the target scene twin model.

[0096] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the drone scene interaction method based on digital twins of the present application. More forms of simple transformations based on this technical concept are all within the scope of protection of the present application.

[0097] The present application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the drone scene interaction method based on digital twins in the above-mentioned embodiment one.

[0098] Reference below Figure 5 , which shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present application. The electronic devices in the embodiments of the present application may include but are not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions: tablet computers), PMPs (Portable Media Players: portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0099] like Figure 5As shown, the electronic device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the electronic device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows an electronic device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0100] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0101] The electronic device provided by this application adopts the drone scene interaction method based on digital twin in the above embodiment, which can solve the technical problem of how to realize a drone interaction simulation that fits the real scene. Compared with the prior art, the beneficial effects of the electronic device provided by this application are the same as the beneficial effects of the drone scene interaction method based on digital twin provided in the above embodiment, and the other technical features in the electronic device are the same as the features disclosed in the previous embodiment method, which will not be repeated here.

[0102] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0103] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0104] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the drone scene interaction method based on digital twins in the above-mentioned embodiment.

[0105] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0106] The computer-readable storage medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

[0107] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by an electronic device, the electronic device: obtains environmental data of the target scene, and constructs a scene twin model of the target scene based on the environmental data; imports the UAV simulation model into the scene twin model to obtain a target scene interaction model, wherein the UAV simulation model includes a flight dynamics module, a flight control module and a multi-sensor simulation module; in the target scene interaction model, the interaction between the UAV and the twin scene is simulated according to the flight data in the target module, wherein the target module includes at least one of the flight dynamics module, the flight control module and the multi-sensor simulation module.

[0108] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0109] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0110] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0111] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned drone scene interaction method based on digital twins, and can solve the technical problem of how to realize a drone interaction simulation that fits the real scene. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the drone scene interaction method based on digital twins provided in the above-mentioned embodiments, and will not be repeated here.

[0112] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned digital twin-based drone scene interaction method.

[0113] The computer program product provided by this application can solve the technical problem of how to realize a drone interactive simulation that fits the real scene. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the drone scene interaction method based on digital twins provided in the above embodiment, which will not be repeated here.

[0114] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A drone scene interaction method based on digital twins, characterized in that: The drone scene interaction method based on digital twins includes: Acquire environmental data of a target scene, and construct a scene twin model of the target scene based on the environmental data; Importing a UAV simulation model into the scene twin model to obtain a target scene interaction model, wherein the UAV simulation model includes a flight dynamics module, a flight control module, and a multi-sensor simulation module; In the target scene interaction model, an interaction simulation between the UAV and the twin scene is performed based on the flight data in the target module, wherein the target module includes at least one of the flight dynamics module, the flight control module and the multi-sensor simulation module.

2. The method for drone scene interaction based on digital twins according to claim 1, characterized in that: The step of constructing the scene twin model of the target scene based on the environmental data comprises: Performing data cleaning on the environmental data, and performing coordinate conversion and time mapping on the cleaned environmental data to obtain target environmental data, wherein the target environmental data is environmental data under a unified time and space reference; A scene twin model of the target scene is constructed based on the target environment data.

3. The drone scene interaction method based on digital twins according to claim 2 is characterized in that: The target environment data includes target geographic information data and target building information data, and the step of constructing a scene twin model of the target scene based on the target environment data includes: Acquire the spatiotemporal information of the target environment data, and fuse the target geographic information data and the target building information data based on the spatiotemporal information to form a digital base plate of the target scene; A scene twin model of the target scene is constructed based on the digital baseplate.

4. The drone scene interaction method based on digital twins according to claim 1, characterized in that: In the target scene interaction model, the step of performing interactive simulation between the UAV and the twin scene based on the flight data in the target module includes: Acquire weather data of the target scene, and import the weather data into the target scene interaction model to obtain a new target scene interaction model; in the new target scene interaction model, perform interaction simulation between the drone and the twin scene according to the weather data and the flight data in the target module; and / or The flight area restriction information of the target scene is obtained, and the flight area restriction information is imported into the target scene interaction model to obtain a new target scene interaction model; in the new target scene interaction model, the interaction simulation between the UAV and the twin scene is performed according to the flight area restriction information and the flight data in the target module.

5. The drone scene interaction method based on digital twins according to claim 4 is characterized in that: The flight data includes sensor data in the multi-sensor simulation module, and a weather model is integrated into the new target scene interaction model, and the weather model is constructed based on the weather data; The step of performing interactive simulation between the UAV and the twin scene according to the weather data and the flight data in the target module includes: During the interactive simulation process between the UAV and the twin scene, the weather model is called to apply environmental pressure to the UAV and monitor the sensor data; The sensor data is transmitted to the flight dynamics module to adjust the simulated flight attitude of the UAV in the twin scene.

6. The method for drone scene interaction based on digital twins according to claim 4, characterized in that: The flight data includes sensor data in the multi-sensor simulation module, and the new target scene interaction model integrates a flight restriction area, and the flight restriction area is constructed based on the flight area restriction information; The step of performing interactive simulation between the UAV and the twin scene according to the flight area restriction information and the flight data in the target module includes: During the interactive simulation process between the drone and the twin scene, monitoring the sensor data; If the drone position in the sensor data is monitored to be within the buffer range of the flight restricted area, the control instructions in the flight control module are adjusted according to the restriction conditions of the flight restricted area, and the drone is controlled to perform simulated flight in the twin scene based on the adjusted control instructions.

7. The method for drone scene interaction based on digital twins according to claim 1, characterized in that: The digital twin-based drone scene interaction method also includes: During the interactive simulation between the UAV and the twin scene, based on the distribution of buildings in the target scene interaction model, the transmission attenuation of the radio signal between the control signal transmission source in the flight control module and the UAV is calculated to obtain the attenuated radio signal; The flight control module is called to control the UAV to perform interactive simulation with the twin scene according to the attenuated radio signal.

8. The method for drone scene interaction based on digital twins according to claim 1, characterized in that: The drone simulation model also includes a fault simulation module, and the drone scene interaction method based on digital twins also includes: During the interactive simulation of the UAV and the twin scene, the fault simulation module is called to generate a fault in the UAV, so as to simulate the flight of the UAV in the twin scene after the fault occurs.

9. An electronic device, characterized in that: The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the digital twin-based drone scene interaction method as described in any one of claims 1 to 8.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the drone scene interaction method based on digital twins are implemented as described in any one of claims 1 to 8.

Images