Virtual reality remote sensing breeding science popularization device and method

Abstract

The invention provides a virtual reality remote sensing breeding science popularization device and method, and relates to the technical field of science popularization, and the device comprises a virtual reality VR unmanned aerial vehicle training module which is used for providing a virtual environment model and evaluating a driving skill proficiency value of a user driving a virtual unmanned aerial vehicle; the data acquisition module is used for acquiring unmanned aerial vehicle remote sensing breeding data from a real unmanned aerial vehicle driven by the user whose driving skill proficiency value reaches a preset threshold value; the data preprocessing module is used for preprocessing the unmanned aerial vehicle remote sensing breeding data obtained by the data acquisition module; and the data analysis module is used for extracting breeding parameters in combination with the preprocessed unmanned aerial vehicle remote sensing breeding data and ground breeding data. Therefore, the training risk in the remote sensing breeding science popularization process is reduced, and science popularization display of the remote sensing breeding process is performed more visually.

Description

Technical Field

[0001] The present invention relates to the field of popular science technology, and in particular to a virtual reality remote sensing breeding popular science device and method. Background Art

[0002] Currently, the acquisition of phenotypic information of field crops in high-precision agricultural base breeding is generally obtained by manual investigation, which requires a large amount of manpower, material resources and time, and has hysteresis, and cannot meet the application requirements of rapid and large-scale acquisition of breeding information. Remote sensing technology has the characteristics of quickly and non-destructively obtaining ground object information, and its rapid development can provide necessary information for the process of agricultural production management and promote the rapid development of agricultural modernization.

[0003] However, the scientific popularization rate of remote sensing breeding technology among the public, especially among teenagers, is very low. On the one hand, it is risky to train people with no experience in remote sensing breeding. On the other hand, common popular science methods cannot intuitively display the process of remote sensing breeding, and it is difficult for the public to understand. Summary of the Invention

[0004] The present invention provides a virtual reality remote sensing breeding popular science device and method to solve the defects in the prior art that it is risky to train people with no experience in remote sensing breeding and common popular science methods cannot intuitively display the process of remote sensing breeding, and to achieve reducing the training risk and more intuitively popularizing and displaying the process of remote sensing breeding.

[0005] The present invention provides a virtual reality remote sensing breeding popular science device, including the following modules: A virtual reality (VR) drone training module for providing a virtual environment model and evaluating the proficiency value of a user's driving skills of a virtual drone; A data acquisition module for acquiring drone remote sensing breeding data from a real drone driven by a user whose driving skill proficiency value reaches a preset threshold; A data preprocessing module for preprocessing the drone remote sensing breeding data obtained by the data acquisition module; A data analysis module for extracting breeding parameters by combining the preprocessed drone remote sensing breeding data and ground breeding data.

[0006] According to a virtual reality remote sensing breeding popular science device provided by the present invention, the virtual reality (VR) drone training module includes: A flight area modeling sub-module for constructing a digital surface model of original terrain data based on the Unity3D engine and optimizing the digital surface model based on the vertex deletion method and the level of detail method to obtain a virtual environment model; A data transmission analysis sub-module, which is used to synchronize the flight-related data of a real unmanned aerial vehicle (UAV) to a MySQL database in real time through serial communication; A driving interaction training sub-module, which is used to present the virtual environment model and the data in the MySQL database through a visual three-dimensional mode, and is used to evaluate the proficiency value of the user's driving skills of the virtual UAV based on the data of the user driving the virtual UAV.

[0007] According to a virtual reality remote sensing breeding popular science device provided by the present invention, the driving interaction training sub-module is specifically used for: Determining the kinematic variable similarity between the kinematic variables of the real UAV and the kinematic variables of the virtual UAV driven by the user through the Euclidean distance; Determining the proficiency value of the user's driving skills of the virtual UAV based on the variance value of the kinematic variable similarity.

[0008] According to a virtual reality remote sensing breeding popular science device provided by the present invention, the data preprocessing module includes: A panoramic stitching sub-module, which is used to perform regional panoramic stitching processing on multiple pictures in the UAV remote sensing breeding data; An orthorectification mosaicking sub-module, which is used to perform interior orientation, aerial triangulation, digital elevation model extraction, and orthorectification processing on the UAV remote sensing breeding data; An image preprocessing sub-module, which is used to perform contrast enhancement, grayscale conversion, filtering enhancement, and denoising processing on the color images in the UAV remote sensing breeding data.

[0009] According to a virtual reality remote sensing breeding popular science device provided by the present invention, the data analysis module includes: A data analysis sub-module, which is used to analyze the preprocessed UAV remote sensing breeding data and ground breeding data, extract the spectral data of the preprocessed UAV remote sensing breeding data and ground breeding data, and analyze the spectral data; An information extraction sub-module, which is used to extract breeding parameters based on the data obtained by the data analysis sub-module and the expert database model.

[0010] The present invention also provides a virtual reality remote sensing breeding popular science method, which is applied to any one of the virtual reality remote sensing breeding popular science devices as described above, and includes the following steps: Evaluating the proficiency value of the user's driving skills of the virtual UAV based on the data of the user driving the virtual UAV; Obtaining UAV remote sensing breeding data from the real UAV driven by the user whose driving skill proficiency value reaches a preset threshold; Preprocessing the UAV remote sensing breeding data obtained by the data acquisition module; Extract breeding parameters by combining the pre - processed UAV remote - sensing breeding data and the ground breeding data.

[0011] According to a virtual reality remote - sensing breeding popularization method provided by the present invention, the method further includes: Construct a digital surface model of the original terrain data based on the Unity3D engine, and optimize the digital surface model based on the vertex deletion method and the level - of - detail method to obtain a virtual environment model; Real - time synchronize the flight - related data of the real UAV to the MySQL database through serial communication; Present the data in the virtual environment model and the MySQL database through a visual three - dimensional mode.

[0012] According to a virtual reality remote - sensing breeding popularization method provided by the present invention, evaluate the proficiency value of the user's driving skills of the virtual UAV based on the data of the user driving the virtual UAV, including: Determine the kinematic variable similarity between the kinematic variables of the real UAV and the kinematic variables of the virtual UAV driven by the user through the Euclidean distance; Determine the proficiency value of the user's driving skills of the virtual UAV based on the variance value of the kinematic variable similarity.

[0013] According to a virtual reality remote - sensing breeding popularization method provided by the present invention, pre - process the UAV remote - sensing breeding data obtained by the data acquisition module, including: Perform regional panoramic stitching on multiple pictures in the UAV remote - sensing breeding data; Perform internal orientation, aerial triangulation, digital elevation model extraction, and orthorectification on the UAV remote - sensing breeding data; Perform contrast enhancement, grayscale conversion, filter enhancement, and denoising on the color images in the UAV remote - sensing breeding data.

[0014] According to a virtual reality remote - sensing breeding popularization method provided by the present invention, extract breeding parameters by combining the pre - processed UAV remote - sensing breeding data and the ground breeding data, including: Analyze the pre - processed UAV remote - sensing breeding data and the ground breeding data, extract the spectral data of the pre - processed UAV remote - sensing breeding data and the ground breeding data, and analyze the spectral data; Extract breeding parameters based on the data obtained from analyzing the pre - processed UAV remote - sensing breeding data and the ground breeding data, the data obtained from analyzing the spectral data, and the expert database model.

[0015] The virtual reality remote sensing breeding popular science device and method provided by the present invention provide a virtual environment model through the VR drone training module, and evaluate the proficiency value of the user's driving skills of the virtual drone, which can ensure that the user masters the drone control skills and screen qualified operators. The data acquisition module obtains the drone remote sensing breeding data from the real drone driven by the user whose driving skill proficiency value reaches the preset threshold. The data preprocessing module preprocesses the drone remote sensing breeding data obtained by the data acquisition module. The data analysis module extracts breeding parameters by combining the preprocessed drone remote sensing breeding data and the ground breeding data, so that during the process of the user driving the real drone, the popular science display of the remote sensing breeding process can be intuitively carried out. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0017] Figure 1 It is a schematic structural diagram of the virtual reality remote sensing breeding popular science device provided by the present invention.

[0018] Figure 2 It is a schematic flow chart of the virtual reality remote sensing breeding popular science method provided by the present invention.

[0019] Figure 3 It is a schematic flow chart of Module 3 and Module 4 provided by the present invention.

[0020] Figure 4 It is a schematic flow chart of orthophoto mosaicking provided by the present invention.

[0021] Figure 5 It is a schematic flow chart of image preprocessing provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] To make the objectives, technical solutions and advantages of the present invention clearer, the following will clearly and completely describe the technical solutions in the present invention with reference to the drawings in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments of the present invention belong to the scope of protection of the present invention.

[0023] Figure 1 It is a schematic structural diagram of the virtual reality remote sensing breeding popular science device provided by the present invention. As Figure 1 shown, the device includes the following modules: The virtual reality (VR) drone training module 100 is used to provide a virtual environment model and evaluate the proficiency value of the user's driving skills for virtual drones. The data acquisition module 110 is used to obtain drone remote sensing breeding data from a real drone driven by a user whose driving skill proficiency value reaches a preset threshold. The data preprocessing module 120 is used to preprocess the drone remote sensing breeding data obtained by the data acquisition module. The data analysis module 130 is used to extract breeding parameters by combining the preprocessed drone remote sensing breeding data and ground breeding data.

[0024] Specifically, the virtual reality remote sensing breeding popular science device provided by the embodiments of the present invention realizes a complete closed-loop from skill training to scientific practice through the coordinated operation of four functional modules, reduces the practical operation risk through virtual simulation, realizes precise popular science through data-driven, and reveals scientific laws through multi-source data fusion.

[0025] Among them, the virtual reality (VR) drone training module is the entry link of the entire device. The VR drone training module can construct a three-dimensional virtual scene of the real terrain environment through high-precision three-dimensional modeling technology. After the user wears the VR device, they can operate through a controller in the physical space to train to drive a virtual drone.

[0026] During the process of the user training to drive a virtual drone, the VR drone training module can evaluate the proficiency value of the user's driving skills through a variety of indicators and using an evaluation algorithm. After the proficiency value of the user's driving skills reaches a preset threshold set in advance, the permission to drive a real drone can be opened for the user.

[0027] The data acquisition module can rely on the general sensor technology carried by existing drones, such as multi-spectral cameras, global positioning system (GPS) positioning modules, and environmental sensors, etc., to automatically collect data when the user operates a real drone, so as to obtain drone remote sensing breeding data. The data transmission process can use a general wireless communication protocol to ensure the real-time transmission of remote sensing breeding data.

[0028] After obtaining the drone remote sensing breeding data, the data preprocessing module can use data cleaning technologies, such as denoising processing, coordinate calibration, and data format standardization, etc., to standardize and organize the drone remote sensing breeding data obtained by the data acquisition module to obtain the preprocessed drone remote sensing breeding data, thereby eliminating the influence of equipment errors and environmental interference.

[0029] The data analysis module can integrate the pre - processed UAV remote - sensing breeding data and the ground breeding data collected by ground sensors, and use certain algorithms (such as deep - learning models, etc.) for feature extraction and correlation analysis, and output basic breeding parameters such as crop growth trends and pest and disease risks.

[0030] It should be noted that in the embodiments of the present invention, users must pass the training assessment of driving a virtual UAV to activate the operation permission of driving a real UAV, which can ensure the standardization of data collection; through the data acquisition module, data pre - processing module and data analysis module, the process of obtaining UAV remote - sensing breeding data until finally generating breeding parameters can be demonstrated in the virtual - reality remote - sensing breeding popular science device.

[0031] The virtual - reality remote - sensing breeding popular science device provided by the present invention provides a virtual environment model through the VR UAV training module and evaluates the proficiency value of the user's driving skills of the virtual UAV, which can ensure that the user masters the UAV control skills and screen qualified operators. The data acquisition module obtains UAV remote - sensing breeding data from the real UAV driven by a user whose driving skill proficiency value reaches a preset threshold. The data pre - processing module pre - processes the UAV remote - sensing breeding data obtained by the data acquisition module, and the data analysis module combines the pre - processed UAV remote - sensing breeding data and the ground breeding data to extract breeding parameters, so that during the process of the user driving a real UAV, the popular science display of the remote - sensing breeding process can be intuitively carried out.

[0032] According to a virtual - reality remote - sensing breeding popular science device provided by the present invention, the virtual - reality VR UAV training module includes: The flight area modeling sub - module is used to build a digital surface model of the original terrain data based on the Unity3D engine, and optimize the digital surface model based on the vertex deletion method and the level - of - detail method to obtain a virtual environment model; The data transmission and analysis sub - module is used to synchronize the flight - related data of the real UAV to the MySQL database in real time through serial communication; The driving interaction training sub - module is used to present the virtual environment model and the data in the MySQL database in a visual three - dimensional mode, and is used to evaluate the proficiency value of the user's driving skills of the virtual UAV based on the data of the user driving the virtual UAV.

[0033] Specifically, in the virtual - reality remote - sensing breeding popular science device provided by the present invention, the VR UAV training module realizes the training function through three sub - modules: the flight area modeling sub - module, the data transmission and analysis sub - module, and the driving interaction training sub - module.

[0034] The flight area modeling sub-module can build a virtual environment based on 3D modeling technology, import the original terrain data obtained by satellite remote sensing or surveying and mapping through the Unity3D engine, and generate an initial Digital Surface Model (DSM).

[0035] In some embodiments, the oblique photography data is the main source of DSM data. Oblique photography uses an aircraft equipped with one vertical and multiple oblique synchronous image acquisition devices to obtain images of the ground terrain and buildings. After collecting the data, a real 3D model is generated through 3D reconstruction and other methods.

[0036] The vertex deletion method is used to generate 3D models with different resolutions of the same object, and then the data is organized through the 3D model organization method of the Levels of Detail (LOD) technology to achieve high-fidelity and high-efficiency basic scene rendering in the virtual reality scenario, and finally form a virtual scene model that can run smoothly on VR devices. This process provides a basis for the skill training of users to drive drones.

[0037] In some embodiments, during the scene generation process, the realistic rendering of real-time weather and environmental changes greatly affects the user's evaluation of the realism of the system environment. In the embodiments of the present invention, the weather conditions can be realistically simulated through the particle system inside Unity, and the display of multiple weathers is supported.

[0038] The data transmission and analysis sub-module can establish a real-time connection between the real drone and the MySQL database through the serial communication protocol, and synchronize the flight-related data of the real drone (such as drone status data, route data, self-check data, and takeoff point setting feedback data) to the database for storage. This data can be applied in the driving interaction training sub-module. The data stored in the MySQL database can drive the synchronous movement of the virtual drone to ensure that the movement states of the virtual drone and the real drone are consistent.

[0039] The driving interaction training sub-module can build an immersive operation interface for the user by calling the virtual environment model obtained by the flight area modeling sub-module and reading the flight-related data of the real drone from the MySQL database.

[0040] It should be noted that the driving interaction training sub-module can cover two working modes: one is the training application mode; the other is the flight application mode.

[0041] In the training application mode, the user simulates a complete flight process through online or offline simulation to learn how to drive a drone. The MySQL database stores the standard flight path and standard attitude information of the real drone.

[0042] In the virtual scene model presented by the VR device, the standard flight path and standard attitude information of the real drone are presented in a visual three-dimensional mode, and users conduct virtual drone driving training and experience according to the standard flight path and standard attitude information of the real drone. When the user manipulates the virtual drone to interact with the environment, the driving interaction training sub-module can record the data of the user driving the virtual drone and calculate the driving skill proficiency value of the user driving the virtual drone based on a preset evaluation algorithm.

[0043] In the flight application mode, during the flight process of the user operating the real drone, the driving interaction training sub-module can access the real-time data of the real drone through the MySQL database, and use the virtual scene model to realistically present the special effects such as the scene, appearance, flight state, position, and attitude of the real drone during flight, so that flight operators and commanders can immediately and intuitively observe the working state, mode, and effect of the drone to assist on-site command, dispatching, and decision-making.

[0044] According to a virtual reality remote sensing breeding popular science device provided by the present invention, the driving interaction training sub-module is specifically used for: Determining the kinematic variable similarity between the kinematic variables of the real drone and the kinematic variables of the virtual drone driven by the user through the Euclidean distance; Determining the driving skill proficiency value of the user driving the virtual drone based on the variance value of the kinematic variable similarity.

[0045] Specifically, the driving interaction training sub-module can accurately evaluate the driving skills of the user by quantitatively analyzing the kinematic differences between the virtual drone driven by the user and the real drone.

[0046] This sub-module first calculates the similarity between the kinematic variable data of the virtual drone driven by the user and the standard kinematic variable data of the real drone through the Euclidean distance.

[0047] Specifically, in a three-dimensional space coordinate system, the kinematic variables such as linear velocity, angular velocity, and acceleration synchronously collected by the real drone and the virtual drone can be vectorized, and the variable combination at each moment is regarded as a point in a multi-dimensional space. The Euclidean distance formula is used to quantify the real-time deviation degree of the two trajectories, and the calculation result reflects the closeness between the operation of the user driving the virtual drone and the ideal operation.

[0048] In the embodiments of the present invention, the kinematic variables of the drone can include five variables: attitude (position and orientation), linear velocity, linear acceleration, angular velocity, and angular acceleration.

[0049] After obtaining the similarity data of the continuous time series, the driving interaction training sub-module can evaluate the operation stability of the user driving the virtual drone based on the analysis of variance.

[0050] Specifically, by statistically analyzing the dispersion degree of all similarity values within a specific training period (such as a single flight mission), the variance calculation module can identify the fluctuation characteristics of the user's operation of driving the virtual drone: a lower variance indicates a high consistency in the user's operation of driving the virtual drone, corresponding to a higher skill level; a higher variance reflects insufficient control ability of the user in driving the virtual drone.

[0051] Therefore, the driving skill proficiency value of the user driving the virtual drone can be determined according to the variance value of the kinematic variable similarity. For example, the variance threshold interval can be matched with the preset scoring standard to determine the driving skill proficiency value of the user driving the virtual drone; or, the driving skill proficiency value of the user driving the virtual drone can be jointly determined according to the variance values of multiple kinematic variables.

[0052] According to a virtual reality remote sensing breeding popular science device provided by the present invention, the data preprocessing module includes: A panoramic stitching sub-module for performing regional panoramic stitching processing on multiple pictures in the drone remote sensing breeding data; An orthorectification mosaicking sub-module for performing internal orientation, aerial triangulation, digital elevation model extraction, and orthorectification processing on the drone remote sensing breeding data; An image preprocessing sub-module for performing contrast enhancement, grayscale conversion, filter enhancement, and denoising processing on the color images in the drone remote sensing breeding data.

[0053] Specifically, in the embodiment of the present invention, the data preprocessing module can process the drone remote sensing breeding data (i.e., the original data) collected by the real drone through the panoramic stitching sub-module, the orthorectification mosaicking sub-module, and the image preprocessing sub-module.

[0054] The panoramic stitching sub-module can perform spatial alignment and fusion on multiple local images in the drone remote sensing breeding data, and implement the panoramic stitching of the image sequence by referring to the OpenCV graphics and image library; for a sequence of pictures with only an overlap degree, information such as the camera focal length and picture exposure can be automatically estimated, and multiple feature matching methods such as SIFT / ORB / SURF can be set to achieve image feature recognition and matching, and finally generate a panoramic stitching map.

[0055] The orthorectification mosaicking sub-module can perform geometric precise processing on UAV remote sensing breeding data. First, it restores the internal parameters of the camera through interior orientation to establish an image coordinate system, and then combines the data of the UAV's position and orientation system (POS) to implement aerial triangulation. The bundle adjustment method is used to optimize the spatial positioning accuracy. Subsequently, terrain elevation information is extracted based on the Digital Elevation Model (DEM) data. Finally, the orthorectification algorithm is used to eliminate terrain undulation and perspective distortion to generate an orthophoto map with real geographical coordinates.

[0056] The image preprocessing sub-module can perform radiation enhancement processing on the color images in the UAV remote sensing breeding data. First, automatic contrast enhancement can be carried out through methods such as histogram equalization or gamma correction, and then the color image is converted into a grayscale image to simplify subsequent analysis. Then, methods such as Wallis filtering are used to suppress noise while retaining edge details. Finally, the image quality is further optimized through denoising algorithms such as wavelet denoising.

[0057] According to a virtual reality remote sensing breeding popular science device provided by the present invention, the data analysis module includes: The data analysis sub-module is used to analyze the preprocessed UAV remote sensing breeding data and ground breeding data, extract the spectral data of the preprocessed UAV remote sensing breeding data and ground breeding data, and analyze the spectral data. The information extraction sub-module is used to extract breeding parameters based on the data obtained by the data analysis sub-module and the expert library model.

[0058] Specifically, the data analysis module completes the extraction of breeding parameters through the data analysis sub-module and the information extraction sub-module.

[0059] The data analysis sub-module can first analyze the preprocessed UAV remote sensing breeding data and ground breeding data, extract the spectral data of the preprocessed UAV remote sensing breeding data and ground breeding data, and then use existing multi-spectral or hyper-spectral analysis techniques to analyze the spectral data.

[0060] In some embodiments, the data analysis sub-module can use standard vegetation indices, such as the Normalized Difference Vegetation Index (NDVI), Enhanced Vegetation Index (EVI), etc., calculation methods to quantify the crop growth status, and at the same time use time series analysis methods to track the change law of spectral characteristics. For example, the key turning points in the crop growth period are identified through the sliding window statistical method. The data analysis sub-module can separate the reflectance data of different bands from the spectral data, and combine parameters such as soil nutrients, temperature and humidity collected in the ground breeding data for sensitivity analysis to construct a model.

[0061] After the data analysis sub-module analyzes various data, the information extraction sub-module can, based on the output results of the data analysis sub-module, access a preset expert knowledge base model for parameter derivation. Among them, the expert knowledge base model can be implemented through a rule engine or a machine learning model (such as a decision tree, random forest), and establish an associated mapping between various data features and breeding parameters such as crop varieties, stress resistance, and yield potential.

[0062] For example, a classification model trained by combining a specific band reflectance threshold and historical breeding data can automatically identify plant populations with high photosynthetic efficiency characteristics; while a regression model based on growth curve fitting can predict the phenotypic performance of hybrid offspring.

[0063] The virtual reality remote sensing breeding popular science method provided by the present invention will be described below. The virtual reality remote sensing breeding popular science method described below can be mutually referenced with the virtual reality remote sensing breeding popular science device described above.

[0064] Figure 2 It is a schematic flowchart of the virtual reality remote sensing breeding popular science method provided by the present invention. As Figure 2 shown, this method is applied to any one of the virtual reality remote sensing breeding popular science devices as described above, and includes the following steps: Step 200: Evaluate the proficiency value of the user's driving skills of the virtual drone based on the data of the user driving the virtual drone.

[0065] Step 201: Obtain unmanned aerial vehicle (UAV) remote sensing breeding data from the real UAV driven by a user whose driving skill proficiency value reaches a preset threshold.

[0066] Step 202: Preprocess the UAV remote sensing breeding data obtained by the data acquisition module.

[0067] Step 203: Extract breeding parameters by combining the preprocessed UAV remote sensing breeding data and ground breeding data.

[0068] Specifically, the virtual reality remote sensing breeding popular science method provided by the embodiments of the present invention reduces the practical operation risk through virtual simulation, realizes precise popular science through data driving, and reveals scientific laws through multi-source data fusion, constructing a complete closed loop from skill training to scientific practice.

[0069] In the implementation of the present invention, a three-dimensional virtual scene of the real terrain environment can be constructed through high-precision three-dimensional modeling technology. After the user wears the VR device, they can operate through the controller in the physical space to train for driving the virtual drone.

[0070] During the training process of a user driving a virtual drone, the proficiency value of the user's driving skills can be evaluated through various metrics using an evaluation algorithm. After the proficiency value of the user's driving skills reaches a preset threshold, the permission to drive a real drone can be granted to the user.

[0071] Relying on the general sensor technology equipped on existing drones, such as multispectral cameras, GPS positioning modules, and environmental sensors, etc., data can be automatically collected when the user operates a real drone, thereby obtaining drone remote sensing breeding data. The data transmission process can use a general wireless communication protocol to ensure the real-time transmission of remote sensing breeding data.

[0072] After obtaining the drone remote sensing breeding data, data cleaning techniques, such as denoising processing, coordinate calibration, and data format standardization, etc., can be used to normalize the drone remote sensing breeding data obtained by the data acquisition module, obtaining the preprocessed drone remote sensing breeding data, thereby eliminating the influence of equipment errors and environmental interference.

[0073] Then, the preprocessed drone remote sensing breeding data can be integrated with the ground breeding data collected by ground sensors, and certain algorithms (such as deep learning models, etc.) can be used for feature extraction and correlation analysis to output basic breeding parameters such as crop growth trends and pest and disease risks.

[0074] It should be noted that in the embodiments of the present invention, the user must pass the training assessment of driving a virtual drone to activate the operation permission of driving a real drone, which can ensure the standardization of data collection; through the processes of data acquisition, data preprocessing, and data analysis, the process of obtaining drone remote sensing breeding data until finally generating breeding parameters can be displayed on the virtual reality remote sensing breeding popular science device.

[0075] The virtual reality remote sensing breeding popular science method provided by the present invention evaluates the proficiency value of the user's driving skills of a virtual drone by providing a virtual environment model, which can ensure that the user masters the drone operation skills and screens qualified operators, obtains drone remote sensing breeding data from the real drone driven by the user whose driving skill proficiency value reaches the preset threshold, preprocesses the drone remote sensing breeding data obtained by the data acquisition module, combines the preprocessed drone remote sensing breeding data and the ground breeding data, and extracts breeding parameters, so that during the process of the user driving a real drone, the popular science display of the remote sensing breeding process can be intuitively carried out.

[0076] According to a virtual reality remote sensing breeding popular science method provided by the present invention, the method further includes: Constructing a digital surface model of the original terrain data based on the Unity3D engine, and optimizing the digital surface model based on the vertex deletion method and the level of detail method to obtain a virtual environment model; Real-time synchronize the flight-related data of the real drone to the MySQL database through serial communication; Present the virtual environment model and the data in the MySQL database through a visual three-dimensional mode.

[0077] Specifically, in the virtual reality remote sensing breeding popularization method provided by the present invention, a virtual environment can be constructed based on three-dimensional modeling technology, and the original terrain data obtained by satellite remote sensing or surveying and mapping is imported through the Unity3D engine to generate an initial DSM.

[0078] In some embodiments, the oblique photography data is the main source of DSM data. Oblique photography uses an aircraft to carry a vertical and multiple oblique synchronous image acquisition devices to obtain images of the ground terrain and buildings, and generates a real three-dimensional model through three-dimensional reconstruction and other methods.

[0079] Use the vertex deletion method to generate three-dimensional models with different resolutions of the same object, and then organize the data through the three-dimensional model organization method of the LOD technology to achieve high-fidelity and high-efficiency basic scene rendering and drawing in the virtual reality scene, and finally form a virtual scene model that can run smoothly in VR devices. This process provides a basis for the skill training of users to drive drones.

[0080] In some embodiments, during the scene generation process, the realistic rendering of real-time meteorological and environmental changes greatly affects the user's evaluation of the realism of the system environment. In the embodiments of the present invention, the weather conditions can be realistically simulated through the particle system inside Unity, and the display of multiple weather conditions is supported.

[0081] In the embodiments of the present invention, a real-time connection between the real drone and the MySQL database can be established through the serial communication protocol, and the flight-related data of the real drone (such as drone status data, route data, self-check data, and takeoff point setting return data) is synchronized to the database for storage. The data stored in the MySQL database can drive the synchronous movement of the virtual drone to ensure that the movement states of the virtual drone and the real drone are consistent.

[0082] In the embodiments of the present invention, the virtual environment model obtained by calling the foregoing steps can be used, and the flight-related data of the real drone is read from the MySQL database, thereby constructing an immersive operation interface for the user.

[0083] It should be noted that the embodiments of this method can cover two working modes: one is the training application mode; the other is the flight application mode.

[0084] In the training application mode, the user simulates a complete flight process through online or offline simulation to learn how to operate a drone. The MySQL database stores the standard flight path and standard attitude information of a real drone.

[0085] In the virtual scene model presented by the VR device, the standard flight path and standard attitude information of the real drone are presented in a visual three-dimensional mode. The user conducts virtual drone driving training and experience according to the standard flight path and standard attitude information of the real drone, etc. When the user manipulates the virtual drone to interact with the environment, the data of the user driving the virtual drone can be recorded, and the driving skill proficiency value of the user driving the virtual drone can be calculated based on a preset evaluation algorithm.

[0086] In the flight application mode, during the flight process of the user operating the real drone, the real-time data of the real drone can be accessed through the MySQL database, and the virtual scene model is used to realistically present the special effects such as the scene, appearance, flight state, position, and attitude of the real drone during flight in real time, so that flight operators and commanders can immediately and intuitively observe the working state, mode, and effect of the drone to assist on-site command, dispatching, and decision-making.

[0087] According to a virtual reality remote sensing breeding popular science method provided by the present invention, based on the data of the user driving the virtual drone, the driving skill proficiency value of the user driving the virtual drone is evaluated, including: Determine the kinematic variable similarity between the kinematic variables of the real drone and the kinematic variables of the virtual drone driven by the user through the Euclidean distance; Determine the driving skill proficiency value of the user driving the virtual drone based on the variance value of the kinematic variable similarity.

[0088] Specifically, the accurate evaluation of the user's driving skill can be realized by quantitatively analyzing the kinematic differences between the virtual drone driven by the user and the real drone.

[0089] In the embodiment of the present invention, first, the similarity between the kinematic variable data of the virtual drone driven by the user and the standard kinematic variable data of the real drone is calculated through the Euclidean distance.

[0090] Specifically, in a three-dimensional space coordinate system, the kinematic variables such as linear velocity, angular velocity, and acceleration synchronously collected by the real drone and the virtual drone are vectorized, and the variable combination at each moment is regarded as a point in a multi-dimensional space. The real-time deviation degree of the two trajectories is quantified through the Euclidean distance formula, and the calculation result reflects the degree of closeness between the operation of the user driving the virtual drone and the ideal operation.

[0091] In the embodiments of the present invention, the kinematic variables of the unmanned aerial vehicle may include five variables: attitude (position and orientation), linear velocity, linear acceleration, angular velocity, and angular acceleration.

[0092] After obtaining the similarity data of the continuous time series, the operation stability of the user driving the virtual unmanned aerial vehicle can be evaluated based on the analysis of variance.

[0093] Specifically, by statistically analyzing the dispersion degree of all similarity values within a specific training period (such as a single flight mission), the variance calculation module can identify the fluctuation characteristics of the user's operation of driving the virtual unmanned aerial vehicle: a lower variance indicates a higher consistency in the user's operation of driving the virtual unmanned aerial vehicle, corresponding to a higher skill level; a higher variance reflects insufficient control ability of the user in driving the virtual unmanned aerial vehicle.

[0094] Therefore, the proficiency value of the user's driving skills of the virtual unmanned aerial vehicle can be determined according to the variance value of the kinematic variable similarity. For example, the variance threshold interval can be matched with the preset scoring standard to determine the proficiency value of the user's driving skills of the virtual unmanned aerial vehicle; or, the proficiency value of the user's driving skills of the virtual unmanned aerial vehicle can be jointly determined according to the variance values of multiple kinematic variables.

[0095] According to a virtual reality remote sensing breeding popular science method provided by the present invention, the unmanned aerial vehicle remote sensing breeding data obtained by the data acquisition module is preprocessed, including: Performing regional panoramic stitching on multiple pictures in the unmanned aerial vehicle remote sensing breeding data; Performing interior orientation, aerial triangulation, digital elevation model extraction, and orthorectification processing on the unmanned aerial vehicle remote sensing breeding data; Performing contrast enhancement, grayscale conversion, filter enhancement, and denoising processing on the color images in the unmanned aerial vehicle remote sensing breeding data.

[0096] Specifically, in the embodiments of the present invention, spatial alignment and fusion can be performed on multiple local images in the unmanned aerial vehicle remote sensing breeding data, and panoramic stitching of the picture sequence can be realized by referring to the OpenCV graphics image library; for a picture sequence with only overlapping degrees, information such as the camera focal length and picture exposure can be automatically estimated, and multiple feature matching methods such as SIFT / ORB / SURF can be set to realize image feature recognition and matching, and finally a panoramic stitching map is generated.

[0097] In the embodiments of the present invention, geometric precise processing can be performed on the unmanned aerial vehicle remote sensing breeding data. First, the internal parameters of the camera are restored through interior orientation to establish an image coordinate system, and then aerial triangulation is implemented in combination with the POS data of the unmanned aerial vehicle, and the beam method adjustment is used to optimize the spatial positioning accuracy; subsequently, terrain elevation information is extracted based on the DEM data; finally, the orthorectification algorithm is used to eliminate terrain undulation and perspective distortion to generate an orthophoto map with real geographic coordinates.

[0098] In the embodiments of the present invention, radiation enhancement processing can be performed on the color images in the UAV remote sensing breeding data. First, automatic contrast enhancement can be performed through methods such as histogram equalization or gamma correction, and then the color image can be converted into a grayscale image to simplify subsequent analysis. Then, methods such as Wallis filtering are used to suppress noise while retaining edge details. Finally, the image quality is further optimized through denoising algorithms such as wavelet denoising.

[0099] According to a virtual reality remote sensing breeding popularization method provided by the present invention, breeding parameters are extracted by combining the preprocessed UAV remote sensing breeding data and ground breeding data, including: Analyze the preprocessed UAV remote sensing breeding data and ground breeding data, extract the spectral data of the preprocessed UAV remote sensing breeding data and ground breeding data, and analyze the spectral data. Based on the data obtained from analyzing the preprocessed UAV remote sensing breeding data and ground breeding data, the data obtained from analyzing the spectral data, and the expert database model, breeding parameters are extracted.

[0100] Specifically, in the process of extracting breeding parameters by combining the preprocessed UAV remote sensing breeding data and ground breeding data, first, the preprocessed UAV remote sensing breeding data and ground breeding data can be analyzed to extract the spectral data of the preprocessed UAV remote sensing breeding data and ground breeding data, and then the existing multispectral or hyperspectral analysis techniques are used to analyze the spectral data.

[0101] In some embodiments, standard vegetation indices, such as the calculation methods of the normalized difference vegetation index (NDVI), enhanced vegetation index (EVI), etc., can be used to quantify the crop growth status. At the same time, time series analysis methods are used to track the change rules of spectral characteristics. For example, the key turning points in the crop growth period are identified through the sliding window statistical method. The reflectance data of different bands can be separated from the spectral data, and sensitivity analysis is performed in combination with parameters such as soil nutrients, temperature and humidity collected in the ground breeding data to construct a model.

[0102] After analyzing various data (including the data obtained from analyzing the preprocessed UAV remote sensing breeding data and ground breeding data, and the data obtained from analyzing the spectral data), based on these data, parameter derivation can be performed by accessing a preset expert knowledge base model. Among them, the expert database model can be implemented through a rule engine or a machine learning model (such as a decision tree, random forest), and various data features are associated and mapped with breeding parameters such as crop varieties, stress resistance, and yield potential.

[0103] For example, a classification model trained by combining the reflectance threshold of a specific wavelength band with historical breeding data can automatically identify plant populations with high photosynthetic efficiency characteristics; while a regression model based on growth curve fitting can predict the phenotypic performance of hybrid offspring.

[0104] The following introduces the virtual reality remote sensing breeding popular science device and method provided by the present invention through embodiments in specific application scenarios.

[0105] The implementation solution of this embodiment includes four modules.

[0106] Through Module 1, the UAV flight module based on digital twin, users can understand and master the skills of controlling UAVs for low-altitude flight. After mastering the flight skills, users fly a real UAV for low-altitude flight, and obtain the original remote sensing breeding data through Module 2. The obtained original data is input into Module 3 for preprocessing to realize the preprocessing of digital pictures, multi-spectral, thermal infrared and hyperspectral data. Finally, it is input into Module 4 for the analysis of remote sensing data, and efficient, fast and accurate information extraction of target breeding parameters is carried out.

[0107] Module 1: Data-driven VR UAV Training Module The data-driven VR UAV training module includes three sub-modules, namely (1) UAV VR flight area modeling; (2) UAV data transmission analysis; (3) UAV VR driving interaction training.

[0108] First, construct a UAV VR virtual flight area through sub-module (1), which simulates the on-site environment of real UAV flight; secondly, transmit, analyze and store the flight data of the real UAV through sub-module (2), and apply the obtained flight data to sub-module (3) for the UAV VR driving interaction training of users.

[0109] (1) UAV VR Flight Area Modeling Select the Unity3D virtual simulation engine to establish the UAV flight area scene, and display the three-dimensional scene of the twin UAV flight area by three-dimensional modeling of the UAV flight space scene. Import the original terrain data recording real geographical information, set the terrain conversion algorithm, apply terrain textures, map terrain feature data, and then batch process to generate a terrain model database.

[0110] In this embodiment, the DSM model is used to simulate the virtual terrain environment. The oblique photography data is the main source of DSM data. Oblique photography uses an aircraft equipped with one vertical and multiple oblique synchronous image acquisition devices to obtain images of the ground terrain and buildings. After collecting the data, a real three-dimensional model is generated through three-dimensional reconstruction and other methods. In order to smoothly render the oblique photography three-dimensional model in virtual reality, it is necessary to optimize the oblique photography model. The vertex deletion method and the LOD method are combined. Different resolution three-dimensional models of the same object are generated through the vertex deletion method, and then the data is organized in the LOD three-dimensional model organization method to achieve high-fidelity and high-efficiency basic scene rendering in the virtual reality scene.

[0111] In scene generation, the realistic rendering of real-time meteorological and environmental changes greatly affects the user's evaluation of the realism of the system environment. Through the particle system inside Unity, the weather conditions can be realistically simulated and the display of multiple weather conditions is supported.

[0112] (2)Data transmission analysis In the virtual scene of the UAV flight area, the actions of the UAV twin model are simulated.

[0113] The data transmission analysis module mainly makes a serial port connection according to the communication serial port setting parameters, and receives the UAV status data, route data, self-check data and takeoff point setting return data sent from the UAV. And the received downlink data is parsed and processed according to the data communication protocol.

[0114] In the data transmission analysis module, the flight data of the UAV is stored in the MySQL database, and this data is applied in sub-module (3) the UAV driving training interaction module. Through the stored data, the synchronous movement of the virtual UAV can be driven to ensure that the movement states of the virtual UAV and the real UAV are consistent.

[0115] (3)UAV VR driving training interaction The VR driving training interaction module covers two working modes: (i) training application mode; (ii) flight application mode.

[0116] (i)Training application mode In the training application mode, the user simulates the complete flight process through online or offline simulation to learn how to drive the UAV. In this embodiment, a scoring mechanism for the proficiency of driving the UAV is added. Through the scoring mechanism, the proficiency of the user's driving skills can be intuitively judged. It can not only improve the training effect, but also provide a basis for actual assessment.

[0117] The user sits on a VR aircraft and, in a virtual scenario, undergoes the first training and experience according to the flight route trajectory and offset angle of a real drone. This process presents the standard flight path and attitude information of the real drone in a visual three-dimensional mode. Through this process, problems such as the danger of drone driving training and the inability to conduct in-depth popular science training anytime and anywhere can be solved.

[0118] The core link of the training application mode is the evaluation module. In this embodiment, evaluation criteria are set to score the user's virtual driving skills, and through the scoring mechanism, the proficiency value of the user's driving skills is visually presented.

[0119] The kinematic model of a drone can be defined by five physical quantities, including attitude (position and orientation), linear velocity, linear acceleration, angular velocity, and angular acceleration. In this embodiment, the flight time of the real drone is set as T, and the time T is evenly divided into N segments, with each time segment t = 1,..., N.

[0120] Among them, for each time segment t, the kinematic variables of the real drone are attitude Rp, linear velocity Rv, linear acceleration Ra, angular velocity Rw, and angular acceleration Rg.

[0121] The kinematic variables of the virtual drone are attitude Vp, linear velocity Vv, linear acceleration Va, angular velocity Vw, and angular acceleration Vg.

[0122] In this embodiment, the 5 kinematic variables of the real drone and the virtual drone in each time segment t are compared for similarity. The Euclidean distance is used to measure the path distance of the corresponding motion variables, and 5 groups of data similarity values are obtained.

[0123] The Euclidean distance is a measure of the "ordinary" distance between two points in Euclidean space. In mathematics, the Euclidean distance refers to the straight-line distance between two points in Euclidean space. Its definition formula is:

[0124] Among them, is the Euclidean distance between point and point , representing the distance between two points.

[0125] Finally, the variance of each of the 5 groups of data similarity values is calculated respectively to obtain the deviation degree of each group of data.

[0126] Given a set of data with an average value of M, the variance is expressed by the formula:

[0127] Among them, if the data is relatively concentrated, the variance value is smaller.

[0128] By calculating the variances of 5 groups of data, according to the variance values, the consideration criteria for the scoring mechanism are as follows: If the variance values of all 5 groups of data are small, the score is 100 points; If the variance values of 4 groups of data are small, the score is 90 points; If the variance values of 3 groups of data are small, the score is 80 points; If the variance values of 2 groups of data are small, the score is 70 points; If the variance value of 1 group of data is small, the score is 60 points; If the variance values of 0 groups of data are small, the score is 50 points; For users with scores of 100 points and 90 points, they can directly drive a real drone and, under the guidance of a professional teacher, conduct short-distance and short-time drone driving. For users with a score of 80 points, they need to continue learning the driving skills of virtual drones in a virtual environment. For users with scores of 70 points, 60 points and below 60 points, they need to conduct targeted driving skill training to improve the users' comprehensive driving ability.

[0129] (2) Flight application mode The flight application mode is mainly during the actual flight of a real drone. The system accesses the real data of the drone and uses three-dimensional visual scenes to realistically reproduce special effects such as the scene, appearance, flight state, position, and attitude of the drone during flight, enabling flight operators and commanders to immediately and intuitively observe the working state, mode, and effect of the drone to assist on-site command, dispatching, and decision-making.

[0130] Module 2: Acquisition of original data for real drone remote sensing breeding Users experience and obtain relevant skills for driving drones through the data-driven VR popular science communication module in Module 1. Under the guidance of a professional teacher, they operate the real machine. Select a specific real plot, and users fly a real drone at low altitude through the joystick. During the flight, use the high-definition digital camera, multi-spectral camera, thermal infrared imager, hyperspectral imager, and canopy growth monitor carried by the low-altitude remote sensing drone to obtain data for drone remote sensing breeding, and obtain different spectral and spatial resolution data.

[0131] Through the two stages of Module 1 and Module 2, users have initially mastered the skills of driving real drones and can initially obtain original data for remote sensing breeding through drones. Next, through Module 3 and Module 4, the analysis and processing of remote sensing breeding data are carried out. Figure 3It is a schematic flow chart of Module 3 and Module 4 provided by the present invention. The operations of the two processes of Module 3 and Module 4 are carried out on a personal computer (PC) under the guidance of professional technical teachers. Through the training of the two modules, users can master the analysis of breeding data through professional analysis software, providing support for breeding decision-making analysis.

[0132] Module 3: UAV Remote Sensing Data Preprocessing Data with different spectral and spatial resolutions need to undergo data preprocessing (Module 3: UAV Remote Sensing Data Preprocessing) before subsequent analysis and data prediction can be carried out.

[0133] It mainly includes seamless panoramic stitching of digital photos, multispectral, thermal infrared, etc., aerial triangulation, DEM extraction and ortho-mosaicking with POS or ground control points (GCP), as well as stitching and post-processing of hyperspectral data. Module 3 can process a large amount of data and perform parallel computing.

[0134] As Figure 3 shown, Module 3 includes panoramic stitching, ortho-mosaicking, and hyperspectral stitching.

[0135] Among them, panoramic stitching can input multiple pictures to achieve regional panoramic stitching.

[0136] The input is continuous pictures, and the flight strip needs to be manually input, and the flight strip matching strategy needs to be manually set.

[0137] Among them, sub-flight strip stitching aligns and fuses adjacent images within the same flight route of the UAV to solve the matching problem of local overlapping areas; global stitching integrates all block results across UAV flight strips to form a complete regional panoramic map.

[0138] The panoramic stitching of the picture sequence is realized by referring to the OpenCV graphics library; for a picture sequence with only overlapping degrees, information such as the camera focal length and picture exposure can be automatically estimated, and multiple feature matching methods such as SIFT / ORB / SURF can be set to realize image feature recognition and matching, and finally a panoramic stitching map is generated.

[0139] Figure 4 It is a schematic flow chart of the ortho-mosaicking provided by the present invention. As Figure 4As shown in the figure, ortho-mosaicking can achieve the internal orientation, relative orientation, aerial triangulation (3D point cloud and absolute orientation), DEM extraction, ortho-rectification, and mosaicking of the Charge Coupled Device (CCD) image data of unmanned aerial vehicles. During this process, full matching refers to the automatic matching of homologous points within the global range, which is used to verify the stitching accuracy of the sub-strip processing and trigger local re-rectification when residual errors are found; multiple consecutive overlapping image sequences need to be input, the strip needs to be set manually, the POS data, GCP data, survey area, and camera parameters, etc. are input, and Module 3 can output the flight track line, aerial triangulation results, DEM map, and ortho-mosaic map.

[0140] During the stitching process of hyperspectral data, sensor errors and atmospheric interference can be eliminated through radiometric calibration to ensure the physical accuracy of spectral data. Then, geometric calibration and image alignment are performed through accurate POS data, so as to perform spectral image stitching and ensure the spatial consistency of multi-view images.

[0141] After the above processes such as stitching, calibration, and alignment, image preprocessing can be carried out.

[0142] Figure 5 is the schematic diagram of the image preprocessing process provided by the present invention. As Figure 5 shown, in image preprocessing, the automatic contrast enhancement of the color original image is first performed. Since performing Wallis filtering on the color image will damage the visual effect of the image, the purpose of performing Wallis filtering on the image is to enhance the texture of the image and extract more feature points. During the process of extracting feature points, first, the 24-bit color image needs to be converted into an 8-bit grayscale image, then Wallis filtering enhancement and wavelet transform denoising are performed on the converted 8-bit image, and then feature extraction is carried out.

[0143] Module 4: Analysis of UAV-borne Breeding Remote Sensing Data Based on the UAV remote sensing data processed by the UAV remote sensing data preprocessing module, the remote sensing data analysis of breeding information is carried out, which can realize the collaborative analysis and modeling of ground data and UAV-borne remote sensing data, integrate the expert model library to realize the monitoring of crop growth parameters, the monitoring and prediction of quality and pests and diseases, and yield prediction, etc., providing technical service support for subsequent breeding decision-making analysis.

[0144] As Figure 3 shown, the main functions of Module 4 are introduced as follows: The relevant system of the file: It can recognize remote sensing data in common formats such as.jpg,.tif,.img, etc. obtained by digital cameras, multispectral cameras, and imaging hyperspectral sensors. It can recognize and load vector files in.evf and.shp formats, and the vector files can be overlaid and displayed with raster files in the view window. It can load ground record data in excel format and parameters parsed and exported later, and can be loaded and displayed in tabular form. The loaded raster and vector data can be converted into tif and img data formats. Operations such as opening raster files, opening vector files, opening text files, saving files as, and closing can be achieved.

[0145] The basic tools for data analysis can include: radiometric correction, geometric correction, image cropping, spatial interpolation, image classification, length and area statistics, etc.

[0146] The sampling data analysis can include the following: Reading and analyzing the reflectance data of the Analytical Spectral Devices (ASD): The ASD hyperspectral sensor obtains the ground object reflectance data through continuous narrow bands (hundreds).

[0147] Adding ground measured data: Integrating UAV remote sensing data with ground measured data (such as soil moisture, crop plant height) to improve the model accuracy.

[0148] Overall statistical analysis and comparison of data: Conducting statistics on multi-source data (such as mean, standard deviation) and comparing the differences between UAV and ground data.

[0149] Data screening for variability analysis: Using analysis of variance or stepwise regression to screen high-variability samples.

[0150] Spectral resampling variability analysis: Resampling hyperspectral data (such as resampling from 5nm interval to 10nm) to unify the resolution of multi-source data. This process needs to be combined with the spectral response function to avoid information loss.

[0151] Band & vegetation index screening: Selecting sensitive bands (such as near-infrared) or vegetation indices (such as NDVI) to build models.

[0152] Modeling & validation: Conducting parameter inversion through integrated machine learning (such as random forest) or expert models.

[0153] The multi / hyperspectral analysis can include the following: Spectral extraction: Extracting the multi / hyperspectrum of the corresponding plot according to the vector layer.

[0154] Data statistics: Statistically analyzing the spectral reflectance distribution in the entire image, specifying a point or area, and displaying the average spectral curve.

[0155] Reliability analysis: According to the cell number, compare the ground hyperspectral and airborne hyperspectral curves, compare the characteristic bands and the specified vegetation indices, and calculate the fitting degree between the two sets of data.

[0156] Band operation: Input the formula, perform band operations, and calculate the vegetation index.

[0157] Sensitivity analysis modeling: Conduct sensitivity analysis with the ground sampling data and build a model.

[0158] Information extraction may include the following: Plant height extraction: For example, based on the three-dimensional point cloud data generated by unmanned aerial vehicle lidar (LiDAR) or stereo images, calculate the average height of individual plants or populations by the difference between the surface elevation and the crop canopy height.

[0159] Lodging monitoring: For example, use multi-temporal visible light and radar image collaborative analysis: The visible light image identifies the lodging area through the change of texture features (such as local color anomalies caused by stem breakage); the radar data detects the change of plant structure through the difference in backscattering coefficients. Fusing the two types of data can improve the monitoring reliability and mark the lodging degree (mild / severe) and distribution range.

[0160] Leaf color and pest / disease classification: For example, extract vegetation indices (such as NDVI, red edge band index) from multi-spectral / hyperspectral images to map the chlorophyll content and quantify the leaf color depth; combine supervised classification (support vector machine, random forest) or deep learning models (convolutional neural network) to identify pest / disease types (such as rust, aphid) based on spectral feature differences, and output the infection level and spatial distribution heat map.

[0161] Leaf Area Index (LAI) & biomass analysis: For example, LAI is estimated through empirical models (such as NDVI-LAI conversion formula) or physical models (inversion of radiative transfer model); biomass constructs a multiple regression model by combining plant height, coverage and spectral index.

[0162] Yield estimation & yield grading: For example, integrate LAI, biomass, plant height and environmental parameters (accumulated temperature, precipitation), and use machine learning (such as gradient boosting tree) or mechanism models (such as WOFOST) to predict the single yield; divide the yield grades (high / medium / low) based on historical data, and combine the spatial distribution map to identify potential areas and risk areas to assist in the optimal allocation of breeding resources.

[0163] Import / Customize Model: The system of this embodiment can support importing third-party models (such as open-source vegetation index libraries) or user-defined algorithms (Python scripts), so as to adapt to different crops (rice, wheat) and regional conditions through the parameter configuration interface. The modular design allows for flexible expansion, such as adding new pest and disease identification models or adjusting the yield prediction formula.

[0164] Analysis Parameter Export: The above analysis results of this embodiment can be output in a standardized format.

[0165] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative effort.

[0166] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0167] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or equivalently replace some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of the present invention.

Claims

1. A virtual reality remote sensing breeding science popularization device, characterized in that: include: Virtual reality (VR) drone training module, which is used to provide a virtual environment model and evaluate the user's proficiency in driving a virtual drone; A data acquisition module, used to acquire drone remote sensing breeding data from a real drone piloted by a user whose driving skill proficiency value reaches a preset threshold; A data preprocessing module, used for preprocessing the UAV remote sensing breeding data obtained by the data acquisition module; The data analysis module is used to extract breeding parameters by combining the pre-processed UAV remote sensing breeding data and ground breeding data.

2. The virtual reality remote sensing breeding science popularization device according to claim 1, characterized in that: The VR drone training modules include: The flight area modeling submodule is used to construct a digital surface model of the original terrain data based on the Unity3D engine, and optimize the digital surface model based on the vertex deletion method and the detail level method to obtain a virtual environment model; The data transmission and analysis submodule is used to synchronize the flight-related data of the real drone to the MySQL database in real time through serial communication; The driving interaction training submodule is used to present the virtual environment model and the data in the MySQL database in a visualized three-dimensional mode, and to evaluate the user's driving skill proficiency value of driving the virtual drone based on the data of the user driving the virtual drone.

3. The virtual reality remote sensing breeding science popularization device according to claim 2, characterized in that: The driving interaction training submodule is specifically used for: Determine the similarity of kinematic variables between the kinematic variables of the real drone and the kinematic variables of the virtual drone piloted by the user through the Euclidean distance; Based on the variance value of the kinematic variable similarity, a driving skill proficiency value of the user in driving the virtual drone is determined.

4. The virtual reality remote sensing breeding science popularization device according to claim 1, characterized in that: The data preprocessing module comprises: A panoramic stitching submodule, used for performing regional panoramic stitching processing on multiple pictures in the UAV remote sensing breeding data; An orthorectification mosaic submodule is used to perform interior orientation, aerial triangulation, digital elevation model extraction and orthorectification processing on the UAV remote sensing breeding data; The image preprocessing submodule is used to perform contrast enhancement, grayscale conversion, filter enhancement and denoising on the color images in the UAV remote sensing breeding data.

5. The virtual reality remote sensing breeding science popularization device according to claim 1, characterized in that: The data analysis module comprises: A data analysis submodule is used to analyze the pre-processed UAV remote sensing breeding data and the ground breeding data, extract spectral data of the pre-processed UAV remote sensing breeding data and the ground breeding data, and analyze the spectral data; The information extraction submodule is used to extract breeding parameters based on the data obtained by the data analysis submodule and the expert library model.

6. A virtual reality remote sensing breeding science popularization method, characterized in that: The virtual reality remote sensing breeding science popularization device as claimed in any one of claims 1 to 5 comprises: Based on the data of the user flying the virtual drone, the user's driving skill proficiency value of the virtual drone is evaluated; Obtaining drone remote sensing breeding data from real drones piloted by users whose piloting skill proficiency values ​​reach a preset threshold; Preprocessing the UAV remote sensing breeding data obtained by the data acquisition module; Breeding parameters were extracted by combining preprocessed UAV remote sensing breeding data and ground breeding data.

7. The virtual reality remote sensing breeding science popularization method according to claim 6 is characterized in that: The method further comprises: Building a digital surface model of the original terrain data based on the Unity3D engine, and optimizing the digital surface model based on a vertex deletion method and a detail level method to obtain a virtual environment model; The flight-related data of the real drone is synchronized to the MySQL database in real time through serial communication; The virtual environment model and the data in the MySQL database are presented in a visual three-dimensional mode.

8. The virtual reality remote sensing breeding science popularization method according to claim 7, characterized in that: Based on the data of the user flying the virtual drone, the user's driving skill proficiency value of the virtual drone is evaluated, including: Determine the similarity of kinematic variables between the kinematic variables of the real drone and the kinematic variables of the virtual drone piloted by the user through the Euclidean distance; Based on the variance value of the kinematic variable similarity, a driving skill proficiency value of the user in driving the virtual drone is determined.

9. The virtual reality remote sensing breeding science popularization method according to claim 6, characterized in that: Preprocessing the UAV remote sensing breeding data obtained by the data acquisition module includes: Performing regional panoramic stitching processing on multiple pictures in the UAV remote sensing breeding data; Performing interior orientation, aerial triangulation, digital elevation model extraction and orthorectification processing on the UAV remote sensing breeding data; The color images in the UAV remote sensing breeding data are subjected to contrast enhancement, grayscale conversion, filtering enhancement and denoising.

10. The virtual reality remote sensing breeding science popularization method according to claim 6, characterized in that: Combine the pre-processed UAV remote sensing breeding data with ground breeding data to extract breeding parameters, including: Analyzing the pre-processed UAV remote sensing breeding data and the ground breeding data, extracting spectral data of the pre-processed UAV remote sensing breeding data and the ground breeding data, and analyzing the spectral data; Breeding parameters are extracted based on data obtained by analyzing the preprocessed UAV remote sensing breeding data and ground breeding data, data obtained by analyzing the spectral data, and an expert library model.

Images