Low-altitude unmanned aerial vehicle live video real-time rapid mapping method and rapid mapping system

Abstract

The application discloses a low-altitude unmanned aerial vehicle live video real-time rapid mapping method and a rapid mapping system, belongs to the technical field of low-altitude unmanned aerial vehicle rapid mapping production, and comprises the following steps: planning and optimizing the flight path of the unmanned aerial vehicle based on real-time meteorological data and flight environment, and adjusting the flight attitude of the unmanned aerial vehicle in real time; correcting the photographed image in real time based on the flight attitude data of the unmanned aerial vehicle, and splicing the photographed image into a coherent image; fusing the image and sensor data to generate a real-time image; repairing the missing part in the real-time image and adjusting the real-time image in real time; generating a panoramic image by splicing the real-time image and transmitting the panoramic image to a ground receiving end; optimizing the data transmission speed and feeding back the image quality in real time; and improving the overall quality of the image through the repair of the missing part, and reducing the visual incoherence or distortion phenomenon caused by the loss or interference in the image acquisition process.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of low-altitude unmanned aerial vehicle rapid mapping production, more particularly, it relates to a low-altitude unmanned aerial vehicle live video real-time rapid mapping method and a rapid mapping system. BACKGROUND

[0002] Low-altitude unmanned aerial vehicles have been widely used in recent years, especially in live video, city monitoring, environmental monitoring and other fields;

[0003] Currently, unmanned aerial vehicle image stitching technology mainly focuses on using image stitching algorithms to stitch multiple adjacent images into a panoramic image, or through real-time data streaming, optimizing image and sensor data quality real-time image generation technology. The development of these technologies has significantly improved the accuracy and real-time performance of low-altitude unmanned aerial vehicles;

[0004] However, despite the progress made in various aspects of the prior art, there are still many deficiencies, specifically, the existing low-altitude unmanned aerial vehicle real-time image generation method mostly relies on a single data source and traditional image stitching algorithms, which results in poor image quality stability in dynamic environments;

[0005] Especially in the flight process, there are big problems in the stitching effect of image edges and the recovery of image details, such as unclear details, white edges in generated images, etc.

[0006] In addition, the current image transmission and real-time feedback mechanism is not sufficient to ensure that the image quality in various flight environments meets the final expectations;

[0007] And when the unmanned aerial vehicle flies to a complex environment area, the image quality often has problems such as loss, blur or inaccurate stitching;

[0008] Therefore, we designed a low-altitude unmanned aerial vehicle live video real-time rapid mapping method and a rapid mapping system that can effectively improve the accuracy and real-time performance of image generation, and optimize the image quality during data transmission. SUMMARY

[0009] To achieve the above purpose, the present application provides the following technical scheme:

[0010] The low-altitude unmanned aerial vehicle live video real-time rapid mapping method comprises the following steps:

[0011] Based on the flight attitude data of the unmanned aerial vehicle, the captured image is corrected in real time, and the captured image is stitched into a continuous image;

[0012] Fusing the image data with real-time data collected from flight attitude sensors, meteorological sensors, and environmental perception sensors to generate real-time images;

[0013] Repairing missing parts in the real-time images and adjusting the real-time images in real time;

[0014] Generating panoramic images by splicing the real-time images and transmitting them to the ground receiving end;

[0015] Optimizing data transmission speed and feeding back image quality in real time;

[0016] Further, the flight path of the unmanned aerial vehicle is planned and optimized based on real-time meteorological data and flight environment, and the flight attitude of the unmanned aerial vehicle is adjusted in real time, including:

[0017] Obtaining real-time meteorological data, including real-time meteorological data and remote meteorological data;

[0018] The real-time meteorological data is collected by the real-time sensors carried by the unmanned aerial vehicle, and the meteorological data includes wind speed data, air pressure data, and temperature and humidity data.

[0019] The remote meteorological data obtains real-time weather forecast data through the cloud and the ground station, including regional wind field distribution, rainfall information, and air temperature change;

[0020] Obtaining real-time environmental data, including terrain data and obstacle data;

[0021] The terrain data is obtained by the ground station and the cloud server;

[0022] The obstacle data is obtained by real-time scanning of the laser radar carried by the unmanned aerial vehicle;

[0023] Synchronizing different data sources, as follows:

[0024] ;

[0025] Wherein, is the data synchronization value at time , wherein ; is the starting time point; is the data value collected by the th sensor at time ; is the data weight of the th sensor, and the calculation process is , wherein is the data reliability score of the th sensor, which is obtained by evaluating the stability and accuracy of long-term running data, and the specific calculation process is , wherein is the The standard deviation of data collected by each sensor; For the first The correlation score of each sensor represents the degree to which the sensor data contributes to the target environment model. The correlation score is... ,in For sensor data Integrated environmental model The correlation coefficient; The sum of all sensor weights is used for normalization. For the first The time deviation function of each sensor is defined as follows: ,in For sensors Synchronization reference time;

[0026] The synchronized data is then subjected to noise correction, as shown in the following formula:

[0027] ;

[0028] in, For sensors The corrected data values; This is the width parameter for Gaussian filtering, which controls the filtering strength; For sensors The mean of the data; For sensors Outlier penalty coefficient; This represents data deviation; a larger deviation indicates a higher likelihood of it deviating from the normal range.

[0029] The corrected data is then unified to a global coordinate system, as shown in the following formula:

[0030] ;

[0031] in, To unify spatial coordinates to a global coordinate system, where ; The radius of the Earth; The three-dimensional coordinates of the reference point; These are the local coordinates acquired by the sensor. This is a function for calculating the angle between two points, used to determine the distance between the two points on the Earth's surface. Here, is the coordinate translation correction function, used to correct errors caused by local sensor offset; is ; is ; is ; is ;

[0032] By combining the corrected meteorological and environmental data, a comprehensive flight environment model is constructed, as shown in the following formula:

[0033] ;

[0034] wherein, is the static function data, used to describe the terrain height, obstacle distribution and terrain slope of the flight area, and the influence of the environment, defined as wherein, is the terrain height value, collected by DEM and DTM data, is the obstacle density, calculated by combining LiDAR and SLAM technology in obstacle data, is the terrain slope, calculated by elevation difference; is the dynamic function data, indicating the dynamic influence of wind speed, air pressure and temperature and humidity on the flight environment, defined as , is the wind speed, measured by the anemometer carried by the unmanned aerial vehicle, is the air pressure value, measured by the air pressure sensor, is the temperature and humidity, collected by the temperature and humidity sensor, represents the base of the natural logarithm; wherein respectively represent the current position coordinates of the unmanned aerial vehicle in three-dimensional space; is the time decay function, describing the weight decreasing effect of dynamic data over time , wherein is the time decay factor, determined by experiment; is the flight environment comprehensive model value at time and spatial position ; is the starting time point of the start of the collection of environmental data; is the current time; is the total number of sensor data, including dynamic meteorological and static environmental data; is the sensor index, used to traverse all collected data; wherein represents the comprehensive value of the current environment model, the lower limit of the value range is , and the higher the value, the more complex the flight environment in the region, and the lower the value, the simpler the environment in the region and the more suitable for the flight of the unmanned aerial vehicle;

[0035] Based on the output value of the flight environment comprehensive model and the current flight gimbal of the unmanned aerial vehicle, the flight attitude of the unmanned aerial vehicle is dynamically adjusted, including the pitch angle, the heading angle and the roll angle.

[0036] Further, based on the flight attitude data of the unmanned aerial vehicle, the photographed image is corrected in real time, and the photographed image is spliced into a coherent image, including:

[0037] Input the real-time attitude data of the unmanned aerial vehicle, including the heading angle , the pitch angle and roll angle Real-time pose data is acquired in real time through an inertial measurement unit;

[0038] Raw image data is collected from a camera ;

[0039] A dynamic correction model is established using real-time pose data, and a rotation matrix is used to correct the raw image, and the correction formula is:

[0040] ;

[0041] wherein, is the value of the corrected image in the pixel coordinates , is the rotation matrix of the dynamic adjustment of the pose angle, and is defined as:

[0042] ;

[0043] The dynamic correction model is used to dynamically adjust the image coordinates to ensure geometric consistency;

[0044] A dynamic weight adjustment mechanism based on image features is introduced, and the correction parameters are optimized in combination with the image texture complexity, as follows:

[0045] ;

[0046] wherein, is the correction optimization weight function, which is automatically adjusted according to the feature point density of the image, and the calculation steps are to calculate the gradient intensity of each pixel point, then calculate the feature point confidence and normalize to calculate; is the optimized rotation matrix, which combines the dynamic adjustment parameters of the image feature complexity;

[0047] The output image of the dynamic correction model has set consistency, providing a basis for subsequent feature extraction and stitching;

[0048] A feature extraction method based on deep learning is used to extract multi-scale feature points from the corrected image;

[0049] A robust matching algorithm based on feature descriptors is used to match the feature points of adjacent images;

[0050] A RANSAC-based false matching elimination mechanism is introduced to eliminate false matching points;

[0051] Finally, a set of feature matching point pairs between adjacent images is generated;

[0052] ​Based on the set of matching point pairs, estimate the homography transformation matrix between adjacent images.

[0053] The Laplacian pyramid fusion algorithm is then used to process the stitched area to improve the smoothness of the stitched edges;

[0054] Edge continuity detection and distortion correction are performed on the stitched images to ensure seamless stitching of the final images;

[0055] The corrected images and preliminary stitching results provide geometric consistency and feature integrity, laying a high-quality foundation for the fusion of image and sensor data. Feature matching point pairs can be used as input for subsequent data fusion and correction.

[0056] Furthermore, the process of fusing image and sensor data to generate a real-time image includes:

[0057] Corrected image data from drones It integrates and calculates data uploaded from all sensors;

[0058] The fusion calculation uses a weighted fusion method, and the fused image information is defined as follows:

[0059] ;

[0060] in, To combine the final image from multiple sensor data sources; For the first The feature images from each data source represent the image data corresponding to each sensor or data source. The coordinates in the image; For the first This measure of data correlation from a single sensor reflects the reliability and stability of the sensor's data. It is defined based on factors such as the reliability of historical data and the sensor's accuracy.

[0061] A deep learning model is built using a convolutional neural network to process the fused image. Enhance;

[0062] Set up the input layer to receive the merged image. ;

[0063] Set up the convolutional layer as follows:

[0064] ;

[0065] in, For the first The weights of the convolutional kernels control the filtering strength and feature extraction capability of that layer for image information. For the first Layer bias term, used to adjust the result of the convolution kernel output; is an activation function; is the output of the first layer, representing the image information processed by the first layer in the convolutional neural network; is the output of the first layer, representing the image information processed by the previous layer, multiplied by the weight through convolution operation, extracting the features of the image;

[0066] Set the pooling layer, and use maximum pooling or mean pooling to reduce the dimension of the feature map;

[0067] Set the output layer to generate the enhanced image;

[0068] Where MSE is used as the loss function to optimize the network parameters:

[0069] ;

[0070] Where, is the loss function, which measures the difference between the output of the convolutional neural network and the target image , used to optimize the parameters of the network; is the number of images in the training set, used for averaging; is the target image, i.e. the high-quality target image we want the network to output; is the enhanced image, output by the network, which is obtained through the deep learning enhancement process;

[0071] Based on the deep learning enhanced image and the correction data, generate high-resolution real-time images , and use the generated high-resolution real-time images as input for subsequent stitching and transmission.

[0072] Further, the missing part in the real-time image is repaired and the real-time image is adjusted in real time, which includes:

[0073] Based on image data fusion and enhancement technology, the loss area is repaired using convolutional neural network, and the features extracted from real-time image and environmental data are fused by weighting, to fill the missing area;

[0074] Edge smoothing and repair technology is adopted to ensure the smoothness and consistency of the image edge, by accurately adjusting the boundary of the repair area and ensuring its seamless integration with the surrounding image, unnatural fracture after image recovery can be effectively avoided;

[0075] Using an optimization method based on image color adjustment, the color of the restored part is consistent with the color of the overall image, as follows:

[0076] ;

[0077] wherein, is the adjusted color; is the color of the original image; is the target color; is the adjustment coefficient;

[0078] Dynamic contrast enhancement and noise removal techniques are used to further improve image quality, wherein the contrast enhancement formula is as follows:

[0079] ;

[0080] wherein, is the enhanced contrast; and are the minimum and maximum pixel values of the adjusted image, respectively;

[0081] The repaired image is compared with the previous image source to ensure the integrity and naturalness of the repaired area, and no visual artifacts or discontinuous areas are introduced. After the above steps, the final image not only restores the original content, but also improves the overall quality of the image. The final output image formula is as follows:

[0082] ;

[0083] wherein, is the final repaired image, and all repair processes are optimized by convolutional neural networks and other image optimization algorithms to ensure the final improvement of image quality.

[0084] Further, the panoramic image is generated by splicing the real-time image and transmitted to the ground receiving end, comprising:

[0085] Using panoramic image splicing technology, and combining image feature matching and geometric correction optimization technology, the repaired image generates the final panoramic image;

[0086] After the image is generated, a layered transmission method is used to layer the panoramic image and transmit it to the ground end layer by layer through the transmission network, wherein each layer of data transmission transmits data through data compression algorithm and encrypted transmission;

[0087] After the ground end receives each layer of data, the accuracy and integrity of the splicing result are determined by comparing the error between the spliced image and the ground true value image, as follows:

[0088] ;

[0089] wherein, is the splicing quality evaluation value; is the spliced image; is the ground truth image; is the standard deviation of the image area, indicating the degree of change in image quality in different areas;

[0090] The evaluation results are fed back to the flight control system, and the subsequent flight task is adjusted in a timely manner according to the changes in the flight environment;

[0091] The ground end and the flight end form a closed-loop feedback mechanism, and the sensor settings, flight path and transmission strategy of the flight end are adjusted according to the splicing effect and image quality fed back by the ground end; ensure that high-quality image data can be obtained at all times during flight.

[0092] Further, the optimization data transmission speed and real-time feedback image quality includes:

[0093] During data transmission, the ground end receives real-time image data transmitted back from the unmanned aerial vehicle and performs quality evaluation, and the image quality feedback includes:

[0094] By comparing the original image and the target image, the clarity, detail fidelity and color accuracy of the image during transmission are evaluated;

[0095] When performing image splicing, the image splicing effect is evaluated in real time, wherein the splicing effect standard is that there is no obvious distortion or misplacement at the splicing position;

[0096] The network bandwidth, delay and packet loss rate are evaluated, and the evaluation standard is image data transmission stability;

[0097] Based on the feedback of image quality and network state information, the data transmission rate is adjusted, and the transmission process of image data is optimized, and the optimization step includes:

[0098] According to the feedback information of image quality and network state, the rate of data transmission is automatically adjusted to maximize the efficiency of image data transmission;

[0099] According to the real-time network condition and image quality, a compression strategy is selected to reduce the amount of transmission data and improve the transmission speed;

[0100] When the network bandwidth is insufficient, the resolution of the image is reduced to ensure stable transmission of image data;

[0101] According to the feedback of image quality information and network state, the flight path and shooting angle of the unmanned aerial vehicle are dynamically adjusted, and the specific adjustment method includes:

[0102] According to the image quality feedback, including blurring, brightness, contrast, the unmanned aerial vehicle shooting angle is adjusted in real time, and the best quality image is ensured to be shot each time;

[0103] According to the flight path optimization result, the flight route is adjusted.

[0104] The real-time fast mapping system comprises an image acquisition module for real-time image acquisition of the unmanned aerial vehicle, and provides flight state and environmental information in combination with various sensors;

[0105] The image quality evaluation and feedback module is used for evaluating the image quality in real time and feeding back the evaluation result to the data transmission optimization module;

[0106] The data transmission optimization module is used for optimizing the data transmission rate, and dynamically adjusting the image transmission strategy according to the image quality evaluation and network condition feedback;

[0107] The flight path and attitude optimization module is used for adjusting the flight path and attitude of the unmanned aerial vehicle according to the image quality feedback and flight data, and ensuring the image capturing effect;

[0108] The image stitching and panoramic image generation module is used for stitching the images to generate panoramic images;

[0109] The image transmission and real-time feedback module is used for adjusting the image quality and data transmission strategy according to the feedback of real-time image transmission.

[0110] In summary, the present application has the following beneficial effects:

[0111] Through flight path optimization and attitude adjustment, the unmanned aerial vehicle can dynamically adjust the flight route and attitude according to real-time meteorological and flight environment data, thereby avoiding the influence of adverse weather or complex terrain on flight and image acquisition, ensuring that the unmanned aerial vehicle can fly stably during flight, and adjusting the shooting angle in real time to maintain image stability and accuracy;

[0112] By reducing image distortion, each frame of image is ensured to be accurate and stable, so that multiple images can be seamlessly connected to generate a complete panoramic image, thereby effectively avoiding the splicing error problem caused by the movement of the unmanned aerial vehicle during image acquisition or other factors;

[0113] By generating panoramic images in real time, the image information can present a wider viewing angle, wherein the transmission of panoramic images improves the utilization rate of data, so that the ground end can obtain high-quality images in real time, providing reliable support for subsequent analysis, processing and display;

[0114] Through the repair of the missing part, the overall quality of the image is improved, the visual discontinuity or distortion phenomenon caused by the loss or interference in the image acquisition process is reduced, and through the process of real-time adjustment of the image, the image quality is always in the best state, avoiding the decline of the image quality caused by unstable factors, improving the reliability of the image acquisition of the unmanned aerial vehicle in complex environment, especially in the case of large-scale interference or missing in the image, still being able to provide complete and high-quality images. BRIEF DESCRIPTION OF DRAWINGS

[0115] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0116] Figure 1 The flow chart of the low-altitude unmanned aerial vehicle live video real-time fast mapping method of the present application;

[0117] Figure 2 The real-time fast mapping system diagram of the present application. DETAILED DESCRIPTION

[0118] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0119] Embodiment:

[0120] The following will be combined with the drawings Figures 1-2 The present application will be further described in detail.

[0121] Please refer to Figures 1-2 The present application provides a technical solution: a low-altitude unmanned aerial vehicle live video real-time fast mapping method, as shown in Figures 1-2 , comprising:

[0122] Based on real-time meteorological data and flight environment, the flight path of the unmanned aerial vehicle is planned and optimized, and the flight attitude of the unmanned aerial vehicle is adjusted in real time;

[0123] Based on the flight attitude data of the unmanned aerial vehicle, the captured image is corrected in real time, and the captured image is spliced into a coherent image;

[0124] Fusing the image data with real-time data collected from flight attitude sensors, meteorological sensors, and environmental perception sensors to generate real-time images;

[0125] Repairing missing parts in the real-time images and adjusting the real-time images in real time;

[0126] Generating panoramic images by splicing the real-time images and transmitting them to the ground receiving end;

[0127] Optimizing data transmission speed and feeding back image quality in real time;

[0128] As shown in Figures 1-2 , the flight path of the unmanned aerial vehicle is planned and optimized based on real-time meteorological data and flight environment, and the flight attitude of the unmanned aerial vehicle is adjusted in real time, which includes:

[0129] Obtaining real-time meteorological data, including real-time meteorological data and remote meteorological data;

[0130] Among them, the real-time meteorological data is collected by the real-time sensors carried by the unmanned aerial vehicle, and the meteorological data includes wind speed data, air pressure data and temperature and humidity data;

[0131] Remote meteorological data obtains real-time weather forecast data through cloud and ground station, including regional wind field distribution, rainfall information and air temperature change;

[0132] Obtaining real-time environmental data, including terrain data and obstacle data;

[0133] Among them, the terrain data is obtained by the ground station and cloud server;

[0134] Obstacle data is obtained by real-time scanning of laser radar carried by unmanned aerial vehicle;

[0135] Synchronize different data sources, as follows:

[0136] ;

[0137] Among them, is the data synchronization value at time , wherein ; is the starting time point; is the data value collected by the th sensor at time ; is the data weight of the th sensor, and the calculation process is , wherein is the data reliability score of the th sensor, which is obtained by evaluating the stability and accuracy of long-term running data, and the specific calculation process is , wherein the standard deviation of data collected by the i-th sensor; the correlation score of the i-th sensor, indicating the degree of contribution of the sensor data to the target environment model, the correlation score is wherein is the correlation coefficient of the sensor data and the environment comprehensive model ; is the sum of the weight numerators of all sensors, used for normalization; is the time deviation function of the i-th sensor, defined as wherein is the synchronization reference time of the sensor ; noise correction is performed on the synchronized data, as follows:

[0138] ;

[0139] ;

[0140] wherein, is the corrected data value of the sensor ; is the width parameter of Gaussian filtering, controlling the filtering strength; is the mean value of the sensor data; is the outlier penalty coefficient of the sensor ; is the data deviation, the larger the value, the higher the possibility of deviating from the normal range;

[0141] the corrected data is unified to the global coordinate system, as follows:

[0142] ;

[0143] wherein, is the spatial coordinate unified to the global coordinate system, wherein ; is the radius of the earth; is the three-dimensional coordinate of the reference point; is the local coordinate obtained by the sensor; is the angle calculation function between two points, used to determine the spherical distance between two points on the earth; is the coordinate translation correction function, which corrects the error caused by the local sensor offset; is; is; is; is;

[0144] the corrected weather data and environmental data are combined to construct a flight environment comprehensive model, as follows:

[0145] ​​ ;

[0146] wherein, is static function data, used to describe the terrain height, obstacle distribution and terrain slope of the flight area, and the influence of the environment, defined as wherein, is the terrain height value, collected by DEM and DTM data, is the obstacle density, calculated by combining LiDAR and SLAM technology in obstacle data, is the terrain slope, calculated by elevation difference; is dynamic function data, representing the dynamic influence of wind speed, air pressure and temperature and humidity on the flight environment, defined as , is the wind speed, measured by the anemometer carried by the UAV, is the air pressure value, measured by the air pressure sensor, is the temperature and humidity, collected by the temperature and humidity sensor, represents the base of the natural logarithm; wherein respectively represent the current position coordinates of the UAV in three-dimensional space; is a time decay function, describing the weight decreasing effect of dynamic data over time wherein, is the time decay factor, determined by experiment; is the flight environment comprehensive model value at time and spatial position ; is the starting time point of the beginning of the collection of environmental data; is the current time; is the total number of sensor data, including dynamic meteorological and static environmental data; is the sensor index, used to traverse all collected data; wherein represents the comprehensive value of the current environment model, the lower limit of the value range is , the higher the value, the more complex the flight environment in the region, and the lower the value, the simpler the environment in the region and the more suitable for UAV flight;

[0147] Based on the output value of the flight environment comprehensive model and the current flight platform of the UAV, the flight attitude of the UAV is dynamically adjusted, including pitch angle, heading angle and roll angle.

[0148] In this embodiment: design experiment, for the prior art scheme, and for the optimization scheme proposed in this embodiment, the experiment first collects meteorological data in the actual flight environment through the unmanned aerial vehicle, and adjusts the flight path and state in real time. For meteorological data, information including meteorological sensor data, temperature and humidity data, wind speed data, etc. is obtained. This data is collected by sensor equipment and further transmitted to the ground receiving station for data analysis and processing. According to the processed data, the flight path is optimized, and by comparing with the existing scheme, the advantages of this embodiment in real-time image generation, data transmission efficiency and flight path adjustment are shown. In this experiment, this embodiment optimizes the flight path while dynamically adjusting according to meteorological data, and combines real-time feedback technology of image transmission to ensure the quality and stability of flight images. At the same time, the ground receiving module and the transmission system of the flight equipment are highly integrated, the settings of the flight end sensor are adjusted to improve the data transmission rate and reduce the delay in data transmission. Specifically, meteorological data and flight state information are obtained through the ground transmission station to dynamically adjust the flight path and shooting angle to optimize the flight image quality. This embodiment also optimizes the flight route and flight attitude adjustment process through the data analysis feedback mechanism to ensure stable output of image quality during flight,

[0149] In the implementation process, flight data and image quality data are transmitted to the ground receiving station in real time for image processing and quality feedback. Through real-time feedback and flight path adjustment, the embodiment significantly improves the clarity and stability of images in the unmanned aerial vehicle image acquisition process, and also optimizes the data transmission efficiency. For specific content, refer to the following table:

[0150] Comparison of flight path optimization and image processing technology experimental data (# Table 1)

[0151] ;

[0152] From the data in the table, it can be seen that the scheme of this embodiment is obviously better than the existing technology scheme in many key parameters;

[0153] First of all, in terms of flight path optimization, the optimization scheme of this embodiment reduces the flight time by adjusting the flight path in real time, from 20.1 kilometers to 25.4 kilometers, and the optimized flight distance is increased by 25%. This means that after optimizing the flight path, the flight distance of the aircraft is effectively reduced, thereby reducing unnecessary time waste during flight and improving the execution efficiency of the task;

[0154] Secondly, in terms of image clarity, the image clarity of the optimization scheme of the embodiment reaches 43.5dB, which is 13.3% higher than the existing technology of 38.4dB. This is because the embodiment adopts a real-time optimization algorithm for meteorological data and flight environment, dynamically adjusts the flight route and shooting angle, and ensures clear and stable image quality in various flight states.

[0155] In terms of data transmission delay, the scheme of the embodiment reduces the transmission delay to 32 milliseconds while optimizing data transmission, which is 36% less than the existing technology of 50 milliseconds. This improvement means that the transmission efficiency of real-time video and data is higher, and faster feedback can be achieved in task execution, especially in real-time video acquisition and flight control. The reduction of delay greatly improves the response speed of the task.

[0156] In addition, the embodiment also has obvious advantages in data transmission rate, which is improved to 8.2Mbps, which is 64% higher than the existing technology of 5.0Mbps. Through optimization of flight path and processing of flight environment data, the scheme of the embodiment can provide higher bandwidth, effectively improve the stability of image transmission, and reduce interference and packet loss phenomenon in the transmission process.

[0157] Finally, in terms of flight stability score, the flight stability score of the embodiment is 9, while the existing technology is 7, indicating that through the combination of image processing and flight path optimization, the stability of flight has been significantly improved.

[0158] In summary, the embodiment significantly improves the deficiencies of the existing technology in flight path, image clarity and transmission efficiency through optimization of flight path, improvement of data transmission rate and improvement of real-time image quality.

[0159] As shown in Figures 1-2 , the flight attitude data based on the unmanned aerial vehicle corrects the captured images in real time and splices the captured images into coherent images, including:

[0160] Input the real-time attitude data of the unmanned aerial vehicle, including the heading angle , the pitch angle and the roll angle , the real-time attitude data is obtained in real time by an inertial measurement unit;

[0161] Collect raw image data from the camera ;

[0162] Use the real-time attitude data to establish a dynamic correction model, and use a rotation matrix to correct the raw image, and the correction formula is:

[0163] ;

[0164] wherein, is the value of the corrected image in pixel coordinates , is the rotation matrix of the dynamic adjustment of the attitude angle, defined as:

[0165] ;

[0166] The dynamic correction model is used for dynamically adjusting the image coordinates to ensure geometric consistency.

[0167] A dynamic weight adjustment mechanism based on image features is introduced to optimize the correction parameters in combination with the image texture complexity, as follows:

[0168] ;

[0169] wherein, is the correction optimization weight function, which automatically adjusts according to the feature point density of the image. The calculation steps are as follows: first, calculate the gradient intensity of each pixel point, then calculate the feature point confidence and normalize to calculate; is the optimized rotation matrix, which combines the dynamic adjustment parameters of the image feature complexity;

[0170] The output image of the dynamic correction model has set consistency, providing a basis for subsequent feature extraction and stitching;

[0171] A deep learning-based feature extraction method is used to extract multi-scale feature points from the corrected image.

[0172] A robust matching algorithm based on feature descriptors is used to match the feature points of adjacent images.

[0173] A RANSAC-based false matching elimination mechanism is introduced to eliminate false matching points.

[0174] Finally, a set of feature matching point pairs between adjacent images is generated.

[0175] Based on the matching point pair set, the homography matrix between adjacent images is estimated,

[0176] Then, a Laplacian pyramid fusion algorithm is used to process the stitching area to improve the smoothness of the stitching edge.

[0177] The stitched image is subjected to edge continuity detection and distortion correction to ensure seamless connection of the final image.

[0178] The corrected image and the preliminary stitching result provide geometric consistency and feature integrity, laying a high-quality foundation for image and sensor data fusion, and the feature matching point pairs can be used as input for subsequent data fusion and correction.

[0179] The experimental design in this embodiment is as follows:

[0180] A low-altitude unmanned aerial vehicle platform is selected, model: M30, and a camera system model: Matrice30Camera;

[0181] The unmanned aerial vehicle collects external data through the real-time meteorological sensor and aircraft sensor mounted, and the experimental design includes two schemes: scheme A based on existing image restoration technology, and scheme B using the real-time rapid mapping and optimized data transmission technology of the above embodiment. The experimental process mainly includes the following steps:

[0182] In the experimental area, the unmanned aerial vehicle obtains relevant meteorological data and aircraft state data through flight path planning and flight environment model. For each flight period, real-time meteorological data is obtained using a meteorological sensor, including wind speed, air temperature, air pressure, etc., and real-time attitude data of the aircraft, including heading angle, pitch angle, and roll angle, are also obtained;

[0183] During the flight, the camera continuously captures real-time video, and the video data is transmitted to the ground end through wireless transmission. According to different flight altitudes and weather conditions, the flight route and flight attitude of the unmanned aerial vehicle are constantly adjusted in real time during the experiment;

[0184] Using the existing scheme A, the video data is directly processed for image stitching. However, due to the lack of optimization for dynamic environment adaptation, the image restoration effect is relatively poor, especially when there is a large attitude change during flight, the image deformation and incomplete restoration phenomenon is more obvious;

[0185] Using the image restoration and data transmission optimization method of the above embodiment, the real-time acquired video data is processed. First, the real-time flight data and meteorological data are used for dynamic optimization of the flight trajectory, and then the real-time image restoration model is applied for image restoration. Through the optimization of image stitching technology, the image quality is maintained stable and improved during the flight;

[0186] Low-altitude unmanned aerial vehicle image restoration and data transmission efficiency comparison experiment table (# Table 2)

[0187] ;

[0188] In the above table, scheme A uses traditional image stitching technology, which can generate images when the weather conditions are stable, but the quality of the restored images is greatly reduced due to the inability to dynamically adjust the flight state, with an evaluation of 3. Scheme B, by combining real-time flight data and meteorological data to optimize the flight path, and using optimized image restoration technology, significantly improves the quality of image restoration, with an evaluation of 5;

[0189] Among them, the splicing time of scheme A is longer, which needs 5 seconds, while scheme B optimizes the splicing process, significantly reduces the splicing time to 3 seconds through the joint optimization of data transmission and repair algorithm, which not only improves the image transmission efficiency, but also improves the real-time feedback effect;

[0190] Through comparison, it is found that scheme A has lower image transmission efficiency under fixed bandwidth, and occupies larger bandwidth, while scheme B effectively optimizes the data transmission process, improves the bandwidth utilization rate through improved transmission technology, dynamically adjusts the transmission path and flight trajectory, so that the network bandwidth is used more efficiently, and the transmission speed is improved by about 150%;

[0191] Therefore, the above-mentioned embodiments can significantly improve the repaired image quality, shorten the image splicing time, and optimize the use of network bandwidth in the low-altitude unmanned aerial vehicle live video data transmission and image repair.

[0192] As shown in Figures 1-2 , the fusion of the image and the sensor data to generate a real-time image includes:

[0193] fusing and calculating the corrected image data shot from the unmanned aerial vehicle and the uploaded data of all sensors;

[0194] The fusion calculation adopts a weighted fusion method, and the fused image information is defined as follows:

[0195] ;

[0196] Among them, is the final image combined with multiple sensor data sources; is the feature image of the th data source, which represents the image data corresponding to each sensor or data source, is the coordinate in the image; is the data correlation measure of the th sensor, which reflects the credibility and stability of the sensor data. This item is defined by factors such as the reliability of historical data and the accuracy of the sensor;

[0197] A deep learning model is established using a convolutional neural network to enhance the fused image ;

[0198] An input layer is set to receive the fused image ;

[0199] A convolutional layer is set as follows:

[0200] ;

[0201] wherein, is the weight of the first layer convolution kernel, which controls the filtering strength and feature extraction capability of the image information by the layer; is the bias of the first layer, which is used to adjust the result of the convolution kernel output; is the activation function; is the output feature map of the first layer, which represents the image information processed by the first layer in the convolutional neural network; is the output of the first layer, which represents the image information processed by the previous layer, and the features of the image are extracted by multiplying the image information by the weight ;

[0202] The pooling layer is set, and the feature map is processed by dimension reduction using maximum pooling or mean pooling;

[0203] The output layer is set to generate the enhanced image;

[0204] wherein, the MSE is used as the loss function to optimize the network parameters:

[0205] ;

[0206] wherein, is the loss function, which measures the difference between the output of the convolutional neural network and the target image , and is used to optimize the parameters of the network; is the number of images in the training set, which is used to calculate the average; is the target image, which is the high-quality target image that we hope the network to output; is the enhanced image, which is the image output by the network, and the image is obtained through the deep learning enhancement process;

[0207] Based on the deep learning enhanced image and the correction data, a high-resolution real-time image is generated, and the generated high-resolution real-time image is used as the input for subsequent stitching and transmission;

[0208] In this embodiment, the real-time image generation and transmission optimization method based on the fusion of unmanned aerial vehicle flight data and environmental data is used, and the purpose of the test is to compare the differences in image processing quality, transmission efficiency and real-time performance between the prior art and the above-mentioned embodiment method;

[0209] wherein, the tested unmanned aerial vehicle is an A-type unmanned aerial vehicle, which is equipped with a high-resolution camera, a temperature and humidity sensor, a weather data collector, a flight attitude sensor and a LiDAR sensor;

[0210] The flight data of the UAV includes flight route, flight height, meteorological parameters such as temperature, humidity, wind speed, wind direction, etc., and the data of the ground station includes received image quality, transmission speed and feedback information, etc.

[0211] The experimental content in the embodiment is as follows:

[0212] Raw data including meteorological data and flight state are collected, including environmental data such as air temperature, air pressure, wind speed collected by meteorological sensors, and flight attitude data such as heading angle, pitch angle and roll angle;

[0213] On the ground station, the flight data and image data collected by the UAV are preprocessed, including denoising, filtering, etc.

[0214] By calculating the fusion of image and sensor data, a fused image is generated , the image is enhanced using a deep learning model , and a target image is generated ;

[0215] The generated image is dynamically optimized, the image quality is transmitted and fed back in real time, and the transmission path is adjusted to reduce transmission delay;

[0216] The image quality is evaluated in real time, and the image transmission parameters and flight path are adjusted according to the feedback;

[0217] Image quality, transmission speed and delay comparison table (#Table 3)

[0218] ;

[0219] From the above table, it can be seen that in terms of image quality, the image quality generated by the embodiment is significantly improved through real-time optimization and deep learning enhancement, the image quality of the prior art is about 50dB, while the method of the present application can reach 70dB, the test value is between 93% and 98%, which shows that the method of the present application can generate higher quality images, reduce image distortion and blur, especially in complex flight environment, the image quality has been significantly improved;

[0220] Secondly, in terms of transmission speed, the embodiment can significantly improve the data transmission efficiency, the transmission speed of the prior art is only 15Mbps, while the method of the present application realizes a transmission speed of 30Mbps by optimizing the data transmission path and reducing redundant data transmission, the test value is from 28Mbps to 35Mbps, which shows that the scheme in the embodiment not only improves the image transmission rate, but also ensures the real-time and efficiency, thereby reducing the decline of image quality caused by transmission delay;

[0221] In terms of delay, the scheme in this embodiment effectively reduces the delay of image transmission. The delay of the prior art is 100 ms, while the present application successfully reduces the delay to 50 ms through real-time feedback and transmission optimization, with a test value of 52 ms to 48 ms. This shows that the scheme of this embodiment can significantly reduce the delay in real-time transmission, and is suitable for the rapid response requirements of unmanned aerial vehicles, especially for application scenarios that require real-time monitoring and image processing.

[0222] As Figures 1-2 shown, the scheme for repairing the missing part in the real-time image and adjusting the real-time image in real time includes:

[0223] Based on image data fusion and enhancement technology, the missing area is repaired using a convolutional neural network, and the features extracted from the real-time image and environmental data are fused by weighting to fill the missing area;

[0224] Edge smoothing and repair technology is used to ensure the smoothness and consistency of the image edge. By accurately adjusting the boundary of the repair area and ensuring seamless integration with the surrounding image, unnatural breaks after image recovery can be effectively avoided;

[0225] An optimization method based on image color adjustment is used to keep the color of the recovered part consistent with the color of the overall image, as follows:

[0226] ;

[0227] Wherein, is the adjusted color; is the color of the original image; is the target color; is the adjustment coefficient;

[0228] Dynamic contrast enhancement and noise removal techniques are used to further improve image quality. The contrast enhancement formula is as follows:

[0229] ;

[0230] Wherein, is the enhanced contrast; and are the minimum and maximum pixel values of the adjusted image, respectively;

[0231] The repaired image is compared with the previous image source to ensure the integrity and naturalness of the repair area, and no visual artifacts or discontinuous areas are introduced. After the above steps, the final image not only restores the original content, but also improves the overall quality of the image. The final output image formula is as follows:

[0232] ;

[0233] wherein, is the final repaired image, all repair processes are optimized through convolutional neural networks and other image optimization algorithms to ensure the final improvement of image quality;

[0234] In this embodiment, through color optimization, the color of the image is more balanced, avoiding the common color distortion or color difference problem in traditional repair technology, and the details in the image are more naturally presented, especially for images taken in complex environments, color adjustment can effectively improve the quality of the image, making it more realistic and ornamental, suitable for fine image display and high-quality video processing;

[0235] Through the technology of deep learning, the image repair is not limited to simple image splicing, but through learning the pattern of the original image to perform complex repair work, especially when dealing with severely damaged images, CNN can restore more details and improve the overall quality of the image. Compared with traditional image repair technology, the CNN method provides higher flexibility and repair capability, and can more accurately preserve the real details of the image during image processing, so that this embodiment can also be applied to high-demand real-time image processing tasks;

[0236] And this step enhances the visual impact of the image by strengthening the edges of the image, especially in the outline, texture or other small changes of the object in the image, through edge enhancement, the performance of the image in details and accuracy has been significantly improved, so that the image taken in complex environment can be more clear and fine in vision, suitable for high-precision application scenarios.

[0237] As shown in Figures 1-2 the panoramic image is generated by splicing the real-time image, and transmitted to the ground receiving end, which includes:

[0238] The panoramic image is generated by splicing the real-time image, and transmitted to the ground receiving end, which includes: generating the final panoramic image;

[0239] After the image is generated, a layered transmission method is adopted to layer the panoramic image and transmit it to the ground end layer by layer through the transmission network, wherein the data transmission of each layer is through data compression algorithm and encrypted transmission to transmit data;

[0240] After the ground end receives each layer of data, the accuracy and integrity of the splicing result are determined by comparing the error between the spliced image and the ground true value image, as follows:

[0241] ;

[0242] wherein, is the splicing quality evaluation value; is a stitched image; is a ground truth image; is a standard deviation of the image region, indicating the degree of variation in image quality of different regions;

[0243] The evaluation results are fed back to the flight control system, and the subsequent flight task is adjusted in a timely manner according to the changes in the flight environment;

[0244] The ground end and the flight end form a closed-loop feedback mechanism, and the sensor settings, flight path and transmission strategy of the flight end are adjusted according to the stitching effect and image quality fed back by the ground end, so as to ensure that high-quality image data can be obtained at all times during flight;

[0245] The experimental design in this embodiment is as follows:

[0246] Two sets of experimental data are compared, representing the prior art and the technology in this embodiment respectively. In this experiment, the superiority of the technology is evaluated by comparing key parameters such as image restoration quality, image stitching effect and real-time data transmission speed;

[0247] The experimental process is as follows:

[0248] Obtain image data, wherein the image data used in the experiment is derived from real-time image data captured by a drone, and real-time environmental data including meteorological data and flight environment information is obtained through corresponding sensors;

[0249] Use convolutional neural networks combined with image restoration technology to repair missing parts of the image, and use image edge detection technology to optimize image quality. In this process, our technology adjusts the flight route and state of the drone in real time to avoid unnatural breaks after image restoration, further improving the visual quality of the image;

[0250] Fuse the repaired image with sensor data to generate the final stitched image, and improve image clarity through image enhancement and smoothing processing;

[0251] Use optimized data transmission algorithms to dynamically adjust the flight state and real-time flight path to ensure efficient transmission of image data, and compare it with the prior art to evaluate the differences in data transmission speed and data quality;

[0252] ;

[0253] It can be seen that the PSNR value is improved from 30.2 to 35.5, which indicates that the embodiment technology can maintain higher details and visual quality when repairing the missing part of the image, reduce the distortion generated in the image reconstruction process, and the improvement of the SSIM value from 0.85 to 0.92 proves that the embodiment technology effectively improves the consistency of the spliced image by optimizing the image splicing algorithm, making the transition of the image more smooth and natural, avoiding obvious seams. In terms of data transmission speed, the transmission rate in the embodiment technology is increased to 3.8MB / S, while the prior art is 32.5MB / s, which reflects that the optimization of the data transmission algorithm in the embodiment technology significantly improves the efficiency and reduces the delay while ensuring the quality of data transmission. Through comparison data, it is shown that the embodiment significantly improves the image recovery quality, splicing quality, data transmission efficiency and stability by combining flight environment, real-time path adjustment and image repair and splicing technology.

[0254] As Figures 1-2 shown, the optimized data transmission speed and real-time image quality feedback includes:

[0255] During data transmission, the ground end receives real-time image data transmitted back from the unmanned aerial vehicle and performs quality assessment. The image quality feedback includes:

[0256] By comparing the original image and the target image, the clarity, detail fidelity and color accuracy of the image during transmission are evaluated;

[0257] When performing image splicing, the image splicing effect is evaluated in real time, and the splicing effect standard is that there is no obvious distortion or misplacement at the splicing position;

[0258] The network bandwidth, delay and packet loss rate are evaluated, and the evaluation standard is image data transmission stability;

[0259] Based on the feedback of image quality and network state information, the data transmission rate is adjusted to optimize the image data transmission process. The optimization steps include:

[0260] According to the feedback information of image quality and network state, the data transmission rate is automatically adjusted to maximize the efficiency of image data transmission;

[0261] According to the real-time network condition and image quality, a compression strategy is selected to reduce the amount of transmission data and improve the transmission speed;

[0262] When the network bandwidth is insufficient, the resolution of the image is reduced to ensure stable transmission of image data;

[0263] According to the feedback of image quality information and network state, the flight path and shooting angle of the unmanned aerial vehicle are dynamically adjusted. The specific adjustment method includes:

[0264] According to the image quality feedback, including blurriness, brightness, contrast, the shooting angle of the unmanned aerial vehicle is adjusted in real time to ensure that the best quality image is captured each time;

[0265] According to the flight path optimization result, the flight route is adjusted;

[0266] In this embodiment, by optimizing the transmission rate of image data, real-time feedback of image quality, and dynamically adjusting the flight path, the performance and stability of the unmanned aerial vehicle system are significantly improved. When the unmanned aerial vehicle performs a task, the data transmission speed can be dynamically adjusted according to the real-time state of the network, avoiding transmission delays when the network is congested or bandwidth is insufficient. When the network conditions change or the flight environment changes, the system will update the flight path and flight strategy in real time, avoiding the impact of network bottlenecks. In combination with image repair and color enhancement technology, seamless panoramic images can be generated in different perspectives and environments. Image stitching not only considers the alignment between images, but also ensures color consistency, avoiding obvious color differences or unnatural boundaries after stitching. Furthermore, by dynamically adjusting the amount of data transmitted, the resolution and data volume of the image are reduced, effectively reducing the network burden and ensuring stable data transmission.

[0267] The real-time fast mapping system comprises:

[0268] The image acquisition module is used for real-time image acquisition of the unmanned aerial vehicle, and provides flight state and environmental information in combination with various sensors;

[0269] It can obtain aerial images through the camera on the unmanned aerial vehicle, simultaneously collect meteorological data and flight state data, and based on a high-resolution camera, can acquire image data in real time, and assist flight control and image quality evaluation in combination with meteorological, temperature, humidity and other sensor data;

[0270] The image quality evaluation and feedback module is used for real-time evaluation of image quality and feedback of the evaluation results to the data transmission optimization module;

[0271] It uses an image quality evaluation algorithm to evaluate image quality and timely adjust data transmission and image processing strategies;

[0272] The data transmission optimization module is used for optimizing data transmission rate and dynamically adjusting image transmission strategies according to image quality evaluation and network condition feedback;

[0273] It adaptively adjusts the data transmission rate according to network feedback and image quality feedback to optimize image transmission effect.

[0274] The flight path and attitude optimization module is used for adjusting the flight path and attitude of the unmanned aerial vehicle according to image quality feedback and flight data to ensure image capture effect;

[0275] The flight path and attitude of the unmanned aerial vehicle are adjusted by the flight control system to optimize the image capturing effect and ensure the image quality.

[0276] The image stitching and panoramic image generation module is configured to stitch the images and generate a panoramic image.

[0277] The image stitching algorithm, such as planar image stitching and geometric transformation, is used to realize seamless image stitching and ensure the overall quality of the stitched image.

[0278] The image transmission and real-time feedback module is configured to adjust the image quality and data transmission strategy according to the feedback of real-time image transmission.

[0279] The image compression rate and transmission strategy are adjusted through real-time feedback of network quality and image quality to ensure the clarity and integrity of the image.

[0280] In the description of the present specification, the description of the terms "one embodiment", "example", "specific example" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0281] The basic principles, main features and advantages of the present application are shown and described above. It should be understood by those skilled in the art that the present application is not limited by the above embodiments, and the above embodiments and descriptions in the specification are only illustrative of the principles of the present application. Without departing from the spirit and scope of the present application, various changes and improvements can be made to the present application, and these changes and improvements all fall within the scope of the claimed present application.

Claims

1. A low-altitude unmanned aerial vehicle live video real-time fast mapping method, characterized in that, The method comprises the following steps: S1: based on real-time weather data and flight environment, planning and optimizing the flight path of the unmanned aerial vehicle, and adjusting the flight attitude of the unmanned aerial vehicle in real time, comprising: Obtain real-time weather data, including real-time weather data and remote weather data; Obtain real-time environment data, including terrain data and obstacle data; Time synchronization processing is performed on data from different data sources, and each source data is corresponded in time according to a predetermined sensor weight and a time deviation function; The synchronized data is corrected for noise, and the corrected data is obtained by using Gaussian filtering and outlier suppression; The corrected data is unified to a global coordinate system; The corrected weather data and environment data are fused; S2: based on the flight attitude data of the unmanned aerial vehicle, correcting the photographed image in real time, and splicing the photographed image into a continuous image, comprising: inputting real-time attitude data of the unmanned aerial vehicle, including a heading angle , a pitch angle , and a roll angle ; collecting raw image data from the camera; Constructing an attitude correction model, using the rotation matrix determined by the above-mentioned triangle to perform geometric correction on the original image to obtain a corrected image; Using a feature extraction method based on deep learning, multi-scale feature points are extracted from the corrected image; Based on the robust matching algorithm of feature descriptors, the feature points of adjacent images are matched; the false matching elimination mechanism based on RANSAC is introduced to eliminate the false matching points; Finally, a set of feature matching point pairs between adjacent images is generated; Based on the matching point pair set, the homographic transformation matrix between adjacent images is estimated; then the Laplacian pyramid fusion algorithm is used to process the splicing area; The spliced image is subjected to edge continuity detection and distortion correction; the feature matching point pairs are used as the input for subsequent data fusion and correction; S3: fusing the image data and real-time data collected from the flight attitude sensor, the weather sensor and the environment perception sensor to generate real-time images, comprising: Fusing and calculating the corrected image data photographed by the unmanned aerial vehicle and the data uploaded by each sensor, wherein the fusion calculation adopts a weighted fusion method, and the image information from different data sources is weighted and summarized according to a preset weight to obtain a fused image; Using a convolutional neural network to enhance the fused image; Using mean square error as a loss function to train and optimize network parameters; Based on the enhanced image and the corrected data, a high-resolution real-time image is generated, and the high-resolution real-time image is used as the input for subsequent splicing and transmission; S4: repairing missing parts in the real-time image and adjusting the real-time image in real time; S5: generating a panoramic image by splicing the real-time image, and transmitting the panoramic image to a ground receiving end, comprising: The panorama image stitching technology is adopted, and the optimization technology of combining image feature matching and geometric correction is adopted to repair the image generate a final panorama image; After the image is generated, a layered transmission method is adopted to layer the panoramic image and transmit each layer to the ground end through a transmission network, wherein the data transmission of each layer is performed by using a data compression algorithm and an encrypted transmission to transmit data; After receiving each layer of data, the ground end evaluates the error of the spliced image and the ground true value image, determines a splicing quality evaluation value, and judges the accuracy and integrity of the splicing result according to the evaluation value; The evaluation result is fed back to the flight control system, and the subsequent flight task is adjusted in time according to the change of the flight environment. The ground end and the flight end form a closed-loop feedback mechanism, which adjusts the sensor settings, flight path and transmission strategy of the flight end according to the splicing effect and image quality feedback by the ground end; S6: Optimize data transmission speed and real-time feedback image quality.

2. The method of claim 1, wherein, The missing part in the real-time image is repaired, and the real-time image is adjusted in real time, including: based on image data fusion and enhancement technology, using convolutional neural network to repair the loss area, extracting features from real-time image and environmental data by weighted fusion, filling the missing area; Edge smoothing and repair technology is adopted to accurately adjust the boundary of the repair area; An optimization method based on image color adjustment is used to keep the color of the restored part consistent with the color of the whole image; The restored image is taken as input, and convolutional neural network enhancement, edge detection constraint and color consistency correction are performed in turn to output the final repair image.

3. A real-time fast mapping system based on the low-altitude unmanned aerial vehicle live video real-time fast mapping method of any one of claims 1-2, characterized in that, It includes: An image acquisition module for real-time image acquisition of the unmanned aerial vehicle; Combined with flight attitude sensor, meteorological sensor and environmental perception sensor to provide flight state and environmental information, the meteorological sensor is used to obtain wind speed, air pressure and temperature and humidity; An image quality evaluation and feedback module for real-time evaluation of image quality and feedback of evaluation results to the data transmission optimization module; A data transmission optimization module for optimizing data transmission rate and dynamically adjusting image transmission strategy according to image quality evaluation and network condition feedback; A flight path and attitude optimization module for adjusting the flight path and attitude of the unmanned aerial vehicle according to image quality feedback and flight data to ensure image capture effect; An image stitching and panoramic image generation module for stitching images to generate panoramic images; An image transmission and real-time feedback module for adjusting image quality and data transmission strategy according to real-time image transmission feedback.

Images