Unmanned aerial vehicle control method and system, electronic device, and storage medium

Abstract

The application belongs to the technical field of unmanned aerial vehicles, and discloses an unmanned aerial vehicle control method and system, an electronic device and a storage medium, which comprise: optimizing a reflection component by using a generative adversarial network, optimizing an illumination component according to a comprehensive illumination intensity, combining a high-quality reflection component image and a high-quality illumination component image into an enhanced image, determining an unmanned aerial vehicle pose by using a visual inertial odometer in combination with the enhanced image, constructing a map in real time by using a simultaneous localization and mapping method in combination with the enhanced image, realizing unmanned aerial vehicle positioning by matching the pose and the map, and planning a flight path according to a current position, a target position and map information of the unmanned aerial vehicle by using a path planning algorithm. The image enhancement algorithm based on the GAN and Retinex theories effectively solves the problems of image blurring, detail loss, overexposure or underexposure under complex illumination, and improves the reliability of unmanned aerial vehicle inspection.

Description

Technical Field

[0001] This invention belongs to the field of unmanned aerial vehicle (UAV) technology, and more specifically, relates to a UAV control method, system, electronic device, and storage medium. Background Technology

[0002] With the continuous advancement of technology, drones, as a new type of aviation equipment, are able to efficiently inspect large-area energy facilities, such as photovoltaic power plants. During the inspection of photovoltaic power plants, they can quickly and comprehensively check the working status of photovoltaic modules, promptly detect faults such as hot spots and microcracks, and provide data support for precise power supply.

[0003] However, in practical applications, complex lighting conditions have become a key factor restricting the further development of UAVs. The intensity, angle, and spectral distribution of light vary drastically under different times, weather, and terrain conditions, posing a severe challenge to UAV image acquisition and autonomous navigation. Under strong direct sunlight, acquired images may be overexposed, resulting in the loss of some details; while in low-light environments, such as early morning, evening, or cloudy days, images become blurry and noisy, making accurate analysis and identification difficult. Complex lighting conditions can also interfere with the UAV's visual navigation system, reducing navigation accuracy and increasing the risk of collisions with obstacles.

[0004] To improve image sharpness, common image enhancement algorithms include histogram equalization (HEM) and the Retinex algorithm. Histogram equalization is a statistical image enhancement method that adjusts the image's gray-level histogram to make the gray-level distribution more uniform, thereby enhancing image contrast. Histogram equalization applies the same transformation function to the entire image, which can enhance images with significant differences in brightness between the background and foreground, improving overall image sharpness. However, it has significant limitations under complex lighting conditions. When there are localized lighting variations in the image, histogram equalization may over-enhance certain areas, leading to image distortion and loss of detail.

[0005] The Retinex algorithm, based on the characteristics of the human visual system, decomposes an image into reflection and illumination components, and enhances the image by adjusting the illumination component. This algorithm can improve the quality of low-light images to a certain extent, making the brightness and contrast more natural, and it is effective in removing shadows and uneven lighting. However, the traditional Retinex algorithm has high computational complexity and poor real-time performance, making it difficult to meet the needs of real-time image enhancement for UAVs. When dealing with complex lighting scenes, such as mixed scenes with both strong and weak light, the Retinex algorithm may not be able to accurately separate the illumination and reflection components, resulting in unsatisfactory enhancement effects.

[0006] Therefore, researching and improving UAV image enhancement and autonomous navigation systems that adapt to complex lighting conditions is of great practical significance and is crucial for expanding the application scope of UAVs and improving their operational efficiency and safety. Summary of the Invention

[0007] This invention aims to solve key technical problems such as inaccurate inspection tasks and poor navigation accuracy of UAVs under complex lighting conditions, so as to improve their performance and reliability in practical applications.

[0008] According to a first aspect of this application, a method for controlling an unmanned aerial vehicle (UAV) is provided, comprising: Generative Adversarial Network (GAN) training steps: Obtain the first acquired image, decompose the first acquired image into a first reflection component map and a first illumination component map using the retinal cortex network, optimize the first reflection component map to obtain a first high-quality reflection component map, and use the first reflection component map and the first high-quality reflection map to build a dataset to train the GAN. Image enhancement steps: Obtain a second acquired image captured during the flight of the UAV; decompose the second acquired image into a second reflection component map and a second illumination component map using a retinal cortex network; optimize the second reflection component map using a trained generative adversarial network to generate a second high-quality reflection component map; optimize the second illumination component map based on the comprehensive illumination intensity to generate a second high-quality illumination component map; and combine the second high-quality reflection component map and the second high-quality illumination component map into an enhanced image using a retinal cortex network. Navigation steps: The UAV pose is determined by combining visual inertial odometry with the enhanced image. A flight environment map of the UAV is constructed in real time using a simultaneous localization and mapping (SMR) method combined with the enhanced image. The UAV is located by matching the pose with the map. A flight path is planned based on the UAV's current position, target position, and map information using a path planning algorithm.

[0009] Optionally, optimizing the first reflection component map to obtain the first high-quality reflection component map includes: The first reflection component image is subjected to discrete wavelet transform, with a decomposition of no less than 3 layers. Each layer yields a set of sub-band images, each containing a low-frequency sub-band and a high-frequency sub-band. The low-frequency sub-band is input into a smoothing filter to smooth the illumination, and the high-frequency sub-band is denoised by removing wavelet coefficients whose absolute values ​​are less than the noise threshold and retaining wavelet coefficients whose absolute values ​​are greater than or equal to the noise threshold. The processed low-frequency and high-frequency sub-bands are then reconstructed using inverse discrete wavelet transform to obtain the first high-quality reflection component image.

[0010] Optionally, the visual inertial odometry includes an inertial measurement unit and a vision unit. The inertial measurement unit measures the acceleration and angular velocity information of the UAV and obtains the UAV's pose by integrating the acceleration and angular velocity. The vision unit extracts feature points from the enhanced image using the ORB algorithm and calculates the UAV's pose by matching feature points in different frames of enhanced images. The pose obtained by the inertial measurement unit and the pose obtained by the vision unit are fused to determine the UAV's pose.

[0011] Optionally, landmark objects in the UAV flight environment can also be identified from the enhanced images using deep learning methods, and the 3D coordinates of the landmark objects can be determined from the enhanced images of different frames, thereby updating the map based on the 3D coordinates of the landmark objects.

[0012] Optionally, performing discrete wavelet transform on the first reflection component map means using the Bior 3.5 wavelet as the wavelet basis for discrete wavelet transform.

[0013] Optionally, the smoothing filter is a weighted least squares filter.

[0014] Optionally, the high-frequency subband denoising process refers to using soft threshold denoising.

[0015] Optionally, the step of optimizing the second illumination component map based on the comprehensive illumination intensity to generate the second high-quality illumination component map includes measuring the illumination intensity in the environment in real time using a light sensor, obtaining the illumination intensity of the acquired image using an image analysis method, fusing the illumination intensity obtained by the light sensor and the image analysis method as a comprehensive illumination intensity, optimizing the second illumination component map based on the comprehensive illumination intensity, and generating the second high-quality illumination component map.

[0016] Optionally, the step of obtaining the illumination intensity of the acquired image using image analysis methods includes: converting the acquired image to HSV space, calculating the average pixel brightness of the V channel, and determining the converted illumination value of the acquired image based on a pre-established mapping relationship between the average pixel brightness and the illumination value.

[0017] Optionally, fusing the light intensity obtained by the light sensor and the image analysis method into a comprehensive light intensity includes: taking a weighted average of the light intensity measured by the light sensor and the converted illuminance value of the acquired image analyzed by the image analysis method to obtain the comprehensive light intensity.

[0018] Optionally, the step of taking a weighted average of the light intensity measured by the light sensor and the light intensity of the acquired image analyzed by the image analysis method to obtain the comprehensive light intensity includes: Steps for evaluating the confidence level of the light sensor: If the light sensor measurement value is within the set range, then the light sensor data is taken as the main factor, and the confidence level of the light sensor is set to 0.8; otherwise, the confidence level of the light sensor is set to 0.2. The steps for evaluating image confidence are as follows: Brightness uniformity = 1 - (image brightness standard deviation / image brightness mean). If brightness uniformity > 0.7, it indicates that the image brightness value is reliable, and the image confidence is set to 0.7; otherwise, the image confidence is set to 0.3. Then, the weights are normalized to obtain the comprehensive illumination intensity c= , Where a is the light intensity measured by the light sensor; It is the illumination intensity obtained from image analysis; a and All of this data has been normalized. w1 is the normalized confidence level of the illumination sensor; w2 is the normalized image confidence score.

[0019] According to a second aspect of this application, a drone control system is provided, comprising: Generative Adversarial Network Training Module: It is configured to obtain a first acquired image, decompose the first acquired image into a first reflection component map and a first illumination component map using the retinal cortex network, optimize the first reflection component map to obtain a first high-quality reflection component map, and use the first reflection component map and the first high-quality reflection map to build a dataset to train the generative adversarial network. Image enhancement module: configured to acquire a second captured image during the flight of the UAV, decompose the second captured image into a second reflection component map and a second illumination component map using a retinal cortex network, optimize the second reflection component map using a trained generative adversarial network to generate a second high-quality reflection component map, optimize the second illumination component map according to the comprehensive illumination intensity to generate a second high-quality illumination component map, and synthesize the second high-quality reflection component map and the second high-quality illumination component map into an enhanced image using a retinal cortex network; The navigation module is configured to determine the UAV's pose using a visual inertial odometry system combined with the enhanced image, construct a real-time UAV flight environment map using a simultaneous localization and mapping (SMR) method combined with the enhanced image, achieve UAV positioning by matching the pose with the map, and plan a flight path using a path planning algorithm based on the UAV's current position, target position, and map information.

[0020] According to a third aspect of this application, a computing device is provided, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the UAV control method described above.

[0021] According to a fourth aspect of this application, a computer-readable storage medium is provided that stores computer-executable instructions, which, when executed by a processor, implement the steps of the unmanned aerial vehicle control method described above.

[0022] The UAV control method, system, electronic device, and storage medium of the present invention have the following beneficial effects: (1) In terms of image quality improvement, the image enhancement algorithm based on GAN and Retinex theory effectively solves problems such as image blurring, loss of detail, overexposure, or underexposure under complex lighting conditions. The enhanced image has higher clarity, contrast, and detail, and can accurately present the characteristics and state of the target object. In the inspection of photovoltaic power stations, it can clearly detect the subtle defects of photovoltaic modules, such as hot spots and microcracks, providing an accurate basis for timely maintenance and repair; in topographic mapping, the enhanced image can accurately reflect the detailed information of the topography and improve the accuracy and reliability of the mapping data.

[0023] (2) Regarding improved navigation accuracy, the autonomous navigation technology integrating visual inertial odometry and deep learning target detection significantly enhances the navigation accuracy and stability of UAVs in complex environments. Visual inertial odometry effectively reduces pose estimation errors and improves flight attitude stability by fusing visual and inertial information; deep learning target detection technology can accurately identify navigation landmarks, providing reliable positioning and navigation information for UAVs. In complex terrains such as mountainous areas and under varying lighting conditions, UAVs can accurately fly along predetermined paths, avoid obstacles, achieve high-precision navigation, and greatly reduce flight risks.

[0024] (3) It realizes efficient collaborative work among various modules, improving the overall working efficiency of the UAV. The illumination perception and compensation unit can perceive changes in illumination conditions in real time and automatically adjust the parameters of the image enhancement algorithm to keep the image enhancement effect at its best. The image enhancement module quickly processes the acquired images to provide high-quality image information for the autonomous navigation module. The navigation module efficiently plans the flight path and executes the task based on the image information and navigation algorithm. In photovoltaic power station inspection, the UAV can quickly complete the inspection of a large area of ​​power station, promptly discover and deal with problems, and improve the operation and maintenance efficiency of the power station. In emergency rescue scenarios, the UAV can quickly reach the rescue site, saving valuable time for rescue work.

[0025] (4) In terms of expanding the scope of application, the technical solution of this invention enables photovoltaic drones to adapt to more complex environments and mission requirements. Whether in desert areas with strong direct sunlight, forest environments with low light, or in smoke-filled fire scenes or mountainous areas with complex terrain, the drone can work stably and complete image acquisition and autonomous navigation tasks. This provides technical support for the widespread application of photovoltaic drones in fields such as agricultural monitoring, power line inspection, geological exploration, and urban planning, and expands its application scenarios and market prospects. Attached Figure Description

[0026] Figure 1 This is a flowchart illustrating the unmanned aerial vehicle (UAV) control method according to an embodiment of the present invention; Figure 2 This is a structural diagram of the unmanned aerial vehicle control system according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the image enhancement process according to an embodiment of the present invention; Figure 4 This is a schematic block diagram of one embodiment of the computing device described in this invention; Figure 5 This is a schematic block diagram of another embodiment of the computing device described in this invention. Detailed Implementation

[0027] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] During drone inspections, complex lighting conditions can severely degrade image quality. Under direct sunlight, images are prone to overexposure, causing some areas to exceed the camera's dynamic range and resulting in the loss of significant detail. For example, when inspecting photovoltaic power plants, overexposed images of photovoltaic modules fail to clearly display subtle surface defects. In low-light environments, such as early morning, evening, or cloudy days, image noise increases significantly, contrast decreases, and images become blurry. This poses a major challenge for tasks requiring precise image analysis and identification, such as identifying microcracks and hot spots in photovoltaic modules, and detecting dust cover.

[0029] Secondly, in terms of navigation, complex lighting conditions severely interfere with the visual navigation system of UAVs. Changes in lighting make the extraction of feature points in acquired images unstable, causing visual navigation algorithms to struggle to accurately identify environmental features and thus reducing navigation accuracy. When flying in mountainous areas, rapid changes in sunlight and the influence of terrain shadows can cause acquired images to lose detailed information, potentially leading the UAV's visual navigation system to misjudge its flight direction and increasing the risk of collisions with obstacles. In urban environments, reflected light and shadows from buildings can also cause acquired images to lose detailed information, interfering with the UAV's navigation signals and making it difficult for it to fly along the intended path.

[0030] This is because existing UAV systems lack an effective collaborative mechanism between modules when dealing with complex lighting conditions. The image enhancement module cannot adjust parameters in real time according to changes in lighting conditions to achieve the best image enhancement effect; the navigation module also cannot make full use of the information after image enhancement to achieve more stable and accurate navigation.

[0031] Therefore, this invention aims to develop a UAV control method and system that can effectively improve adaptability to complex lighting conditions. Through innovative algorithms and system architecture design, it solves the aforementioned technical challenges and enhances the UAV's working capabilities and application range in complex environments. This invention establishes a data sharing mechanism to optimize and enhance acquired images, and utilizes the recognition of landmarks in the enhanced images to update maps for navigation. This enables timely acquisition of high-quality enhanced image information, improving navigation accuracy. Furthermore, by adjusting image enhancement parameters through lighting perception and compensation, it allows for faster and more accurate response to changes in lighting conditions, providing reliable support for image enhancement and autonomous navigation.

[0032] A method for controlling an unmanned aerial vehicle (UAV) according to the present invention includes: Generative Adversarial Network (GAN) training steps: Obtain the first acquired image, decompose the first acquired image into a first reflection component map and a first illumination component map using the Retinex network, optimize the first reflection component map to obtain a first high-quality reflection component map, and use the first reflection component map and the first high-quality reflection component map to build a dataset to train the GAN.

[0033] The initial image is acquired via the drone's camera. The objects being inspected (such as power lines, wind turbines, pipes, and bridges) are typically located below or to the side of the drone. The camera is mounted on a gyro-stabilized gimbal on the underside of the drone, providing an unobstructed, directly facing optimal viewing angle. The gyro-stabilized gimbal counteracts all shaking and vibration during drone flight, ensuring extremely stable and clear footage. The drone's camera captures images during flight; for example, when inspecting a photovoltaic power station, the camera captures images of the photovoltaic modules' operational status.

[0034] The retinal cortex network, based on the human visual system's perception of color constancy, decomposes the acquired image into a first reflection component map and a first illumination component map. The first reflection component map reflects the object's inherent characteristics, such as texture and edges, and is independent of lighting conditions. The first illumination component map reflects the influence of ambient light on the acquired image. Although the first reflection component map generated by the retinal cortex network separates most of the illumination, there may still be residual light, causing variations in brightness in uniform color areas, making the texture and edges of the first reflection component map less clear. To address this, a Generative Adversarial Network (GAN) is trained to optimize the reflection component map decomposed by Retinex, generating a high-quality reflection component map that removes the influence of illumination, thereby improving texture and edge clarity.

[0035] A dataset composed of multiple first reflection component maps and first high-quality reflection component maps is used to train a generative adversarial network (GAN). The GAN consists of a generator and a discriminator. Through adversarial training between the two, it learns the characteristics and distribution patterns of the first reflection component maps and the first high-quality reflection component maps. The generator generates reflection component maps and inputs them to the discriminator, attempting to deceive the discriminator into believing they are high-quality reflection component maps. The discriminator then judges whether the reflection component maps generated by the generator are high-quality or defective, continuously optimizing the discriminator's performance to encourage the generator to produce high-quality reflection component maps. During training, the generator and discriminator compete against each other, constantly improving their respective capabilities. Ultimately, this enables the generator to optimize the reflection component maps decomposed from images captured under complex lighting conditions into high-quality reflection component maps with clear details and good visual effects.

[0036] In some embodiments, optimizing the first reflection component map to obtain a first high-quality reflection component map includes: performing a discrete wavelet transform on the first reflection component map, decomposing it into at least 3 layers, with each layer yielding a set of sub-band images, each set of sub-band images containing a low-frequency sub-band and a high-frequency sub-band; inputting the low-frequency sub-band into a smoothing filter to smooth the illumination; denoising the high-frequency sub-band by removing wavelet coefficients whose absolute values ​​are less than a noise threshold and retaining wavelet coefficients whose absolute values ​​are greater than or equal to a noise threshold; and reconstructing the processed low-frequency and high-frequency sub-bands using an inverse discrete wavelet transform to obtain the first high-quality reflection component map.

[0037] Specifically, the first reflection component image undergoes discrete wavelet transform. Since the UAV is primarily used for inspection, the Bior 3.5 wavelet can be used as the wavelet basis for discrete wavelet transform, which can reveal the clarity of object edges and textures. The decomposition layer is, for example, three layers: the first layer contains fine details of the image, the second layer contains medium-scale details, and the third layer contains coarse details. Each layer yields a set of sub-band images, each containing low-frequency and high-frequency sub-bands. The low-frequency sub-band (LL) contains approximate information and the main structure of the normal reflection component image, but also retains most of the uneven illumination; the high-frequency sub-bands (LH, HL, HH) contain edge and texture details in horizontal, vertical, and diagonal directions, and also contain noise. The low-frequency sub-band is input into a smoothing filter to smooth illumination variations, such as weighted least squares filtering or anisotropic diffusion, which can smooth illumination while preserving the structural boundaries between objects as much as possible, further smoothing and suppressing residual illumination variations.

[0038] Denoising the high-frequency subband can be achieved using soft thresholding. Wavelet coefficients with absolute values ​​less than the noise threshold are likely noise and should be removed. Wavelet coefficients with absolute values ​​greater than or equal to the noise threshold may correspond to important edges and should be preserved or moderately enhanced. This enhances true details and suppresses noise. The processed low-frequency and high-frequency subbands are then reconstructed using inverse discrete wavelet transform to obtain the first high-quality reflection component image. The noise threshold can be selected using, for example, a specific noise threshold value. , Where is the noise standard deviation, and N is the number of wavelet coefficients. Alternatively, the SURE criterion can be used to determine the noise threshold.

[0039] Image enhancement steps: Obtain a second acquired image captured during the drone's flight. Use a retinal cortex network to decompose the second acquired image into a second reflectance component image and a second illumination component image. Then, use a trained generative adversarial network to optimize the second reflectance component image, generating a second high-quality reflectance component image. The retinal cortex network can then synthesize the second high-quality reflectance component image and the second illumination component image into an enhanced image.

[0040] Because the generative adversarial network has been trained, it can optimize the second reflection component map, which still contains variations in brightness and darkness, to remove the influence of these variations as much as possible, thereby making the edges and textures of the optimized second high-quality reflection component map clearer.

[0041] In this invention, Retinex theory is integrated into a GAN network. First, wavelet transform is used to optimize the first reflection component map into a first high-quality reflection component map, removing the influence of illumination variations on the reflection component. Whether the influence is local or global illumination variation, it is minimized to reduce the impact of illumination on image edge and texture sharpness. The first reflection component map and the first high-quality reflection component map are used to train the GAN network, enabling it to optimize the reflection component map under poor illumination conditions from Retinex decomposition into a high-quality reflection component map, thus eliminating potential noise and texture loss issues in the separated reflection component map. This approach fully leverages the advantages of GAN and Retinex theory to achieve efficient image sharpening under complex lighting conditions. Then, the retinal cortex network unit fuses the illumination component map and the generator-optimized high-quality reflection component map to obtain the enhanced image.

[0042] Even when images are overexposed under direct sunlight, causing some areas to exceed the camera's dynamic range and resulting in the loss of significant detail, or when images become significantly noisy, have reduced contrast, and are blurry in low-light conditions, these issues can be addressed by inputting the reflection component map into a GAN for optimization. This enhances the edge and texture features of objects, enabling drones to perform precise analysis and recognition tasks based on the enhanced images. Specifically, it allows for relatively accurate identification of features such as microcracks and hot spots on photovoltaic modules.

[0043] In this system, the generator in the Generative Adversarial Network (GAN) unit learns a large number of mappings between defective and high-quality reflection component images during training, enabling it to optimize the reflection component images and generate clearer and more accurate ones. The discriminator in the trained GAN unit then distinguishes between high-quality and defective reflection component images, prompting the generator to produce reflection component images that are closer to high-quality ones. This approach fully leverages the advantages of GAN and Retinex theory to achieve efficient image sharpening under complex lighting conditions. Analysis of the enhanced images can then identify the condition of inspected objects, such as micro-cracks, stains, and loose connections on the surface of photovoltaic modules.

[0044] While wavelet transform can optimize acquired images, optimizing the image for each acquisition requires adjusting wavelet transform parameters (such as noise threshold and decomposition level) for each image, lacking adaptability and the ability to automatically learn global features. This invention trains a GAN using high-quality reflectance component maps, enabling the network to learn the mapping pattern from defective reflectance component maps with brightness variations to high-quality reflectance component maps. The trained GAN can handle unseen defective reflectance component maps without redesigning the optimization process, exhibiting strong generalization ability and thus being applicable to image enhancement under various poor lighting conditions.

[0045] In one feasible embodiment, a high-precision light sensor is also used to measure parameters such as ambient light intensity and color temperature in real time. The light sensor operates based on the photoelectric effect, converting light signals into electrical signals. Accurate lighting information is obtained through the processing and analysis of these electrical signals. Under different lighting conditions, such as strong sunlight on a sunny day, diffused light on a cloudy day, and low light in the early morning and evening, the light sensor can respond quickly and provide reliable lighting data.

[0046] The ambient light intensity is measured in real time using a light sensor, and the light intensity of the acquired images is obtained using image analysis methods. The light intensity obtained from the light sensor and the image analysis methods is fused into a comprehensive light intensity. The second light component map is then optimized based on the comprehensive light intensity to generate a second high-quality light component map. The retinal cortex network can then synthesize the second high-quality reflectance component map and the second high-quality light component map into an enhanced image.

[0047] Image analysis methods, by analyzing features such as brightness and color distribution in the acquired images, can further accurately determine the current lighting conditions. For example, the presence of large areas of highlight in the acquired image indicates strong light intensity; conversely, an overall dark image with low color saturation suggests a low-light environment. By combining data from the light sensor and the analysis results of the acquired images, the system can comprehensively and accurately perceive the current lighting conditions.

[0048] The step of obtaining the illumination intensity of the acquired image using image analysis methods includes: converting the acquired image to HSV space, calculating the average pixel brightness of the V channel, and determining the converted illumination value of the acquired image based on a pre-established mapping relationship between the average pixel brightness and the illumination value.

[0049] The step of fusing the light intensity obtained by the light sensor and the image analysis method into a comprehensive light intensity includes: taking a weighted average of the light intensity measured by the light sensor and the converted illuminance value of the acquired image analyzed by the image analysis method to obtain the comprehensive light intensity.

[0050] Steps for evaluating the confidence level of the light sensor: If the light sensor measurement value is within the set range (e.g., 10-10000 Lux), then the light sensor data is the primary factor, and the confidence level of the light sensor is set to 0.8; otherwise, the confidence level of the light sensor is set to 0.2. The steps for evaluating image confidence are as follows: Brightness uniformity = 1 - (image brightness standard deviation / image brightness mean). If brightness uniformity > 0.7, it indicates that the image brightness value is reliable, and the image confidence is set to 0.7; otherwise, the image confidence is set to 0.3. Then the weights are normalized, i.e., total confidence = illumination sensor confidence + image confidence. w1 = Confidence level of light sensor / Total confidence level w2 = Image confidence score / Total confidence score Comprehensive light intensity c= , Where a is the light intensity measured by the light sensor; It is the illumination intensity obtained from image analysis; a and All of this data has been normalized. w1 is the normalized confidence level of the illumination sensor; w2 is the normalized image confidence score.

[0051] The second illumination component image is dynamically adjusted based on the overall illumination intensity. Under high overall illumination intensity (i.e., strong light conditions), the brightness gain of the second illumination component image is reduced to avoid overexposure. Under low overall illumination intensity (i.e., low light environments), the brightness and contrast parameters of the second illumination component image are increased to improve its sharpness, thereby generating a high-quality second illumination component image. This dynamic adjustment mechanism enables the enhancement of the image quality and usability of the combined high-quality second illumination component image and the high-quality second reflection component image.

[0052] In some embodiments, histogram specification can also be used to enhance the acquired images. Histogram specification adjusts the pixel value distribution of an image to conform to a specified histogram distribution, thereby enhancing image contrast and brightness. It first calculates the probability distribution of gray levels in the original and target images. Then, for each pixel value in the original image, it finds the pixel value in the target image whose gray level probability is closest to its original value and maps them, ultimately obtaining the enhanced image. The advantage of this method is that it can enhance image contrast and brightness to a certain extent while preserving the texture and features of the original image. When processing images with relatively uniform lighting but low contrast, histogram specification can effectively improve image clarity and make image details more apparent. However, this method also has significant drawbacks. The calculation of the probability distribution involves a large computational load, which may result in slow processing speeds for photovoltaic drones that need to process large numbers of images in real time, failing to meet the needs of practical applications. For images in complex lighting scenarios, such as images with both strong light and shadow, the enhancement effect of histogram specification may not be ideal, and may even introduce unpredictable changes, leading to image distortion or unnatural effects.

[0053] Navigation steps: The UAV pose is determined by combining the visual inertial odometry with the enhanced image. The UAV flight environment map is constructed in real time by combining the Simultaneous Localization and Mapping (SLAM) method with the enhanced image. The UAV is located by matching the pose with the map. The flight path is planned by the path planning algorithm based on the UAV's current position, target position and map information.

[0054] Visual inertial odometry (VIO) combines information from visual sensors (such as cameras) and inertial measurement units (IMUs). By fusing enhanced image processing from the image enhancement module with inertial data, it can estimate the UAV's pose information in real time. In terms of visual processing, feature extraction and matching algorithms, such as the ORB (Oriented Fast and Rotated BRIEF) algorithm, are used to extract feature points from the enhanced image. By matching feature points in different frames of enhanced images, the relative motion of the UAV is calculated. The inertial measurement unit provides the UAV's acceleration and angular velocity information. By integrating the acceleration and angular velocity information, the attitude and position changes of the UAV can be obtained. The fusion of visual and inertial information effectively improves the accuracy and stability of pose estimation and reduces error accumulation.

[0055] In this process, simultaneous localization and mapping (SMR) technology is used to build a map of the surrounding environment in real time using augmented images during flight. The augmented images can accurately reflect the details of the terrain and landforms, improving the accuracy and reliability of the constructed map data. The pose information determined by visual inertial odometry (VIO) is matched with the map to achieve precise positioning.

[0056] Furthermore, by employing deep learning methods, such as object detection models based on convolutional neural networks (CNNs), landmark objects in the UAV flight environment are identified from the enhanced images. The 3D coordinates of the landmark objects are determined through enhanced images of different frames, and then updated in the map constructed by the simultaneous localization and mapping method based on their 3D coordinates.

[0057] It also utilizes path planning algorithms to plan a safe and efficient flight path based on the drone's current location, target location, and map information. Path planning algorithms can be, for example, D Lite. Quick Exploration Random Tree Series (Quick Exploration Random Tree Series RRT) Informed RRT Path planning is performed using RRT-Connect.

[0058] The object detection model is used to identify navigation landmarks in the UAV's flight environment. A deep neural network model can be trained beforehand using images of landmark objects. This training enables the UAV to quickly and accurately identify specific landmark objects, such as buildings, road signs, and mountains, from the enhanced images output by the image enhancement module. The 3D coordinates of these landmark objects can be determined from different frames of the enhanced images, and these coordinates are then used to update the SLAM-built map. These landmark objects serve as navigation reference points, providing the UAV with environmental information and positioning guidance. During flight, the UAV continuously detects landmark objects in its surrounding environment and adjusts its flight path based on their position and attitude information to ensure accurate arrival at the target location.

[0059] Inertial navigation has the advantages of strong autonomy and immunity to external signal interference. However, its positioning error gradually increases over time, leading to a decrease in navigation accuracy. During long-duration flight missions, inertial navigation errors may cause the UAV to deviate from its planned flight path and fail to accurately reach the target location.

[0060] Visual navigation utilizes cameras mounted on drones to acquire image information of the surrounding environment, achieving autonomous navigation through image processing and computer vision techniques. Visual navigation boasts advantages such as strong environmental perception and the ability to acquire rich scene information. However, it also faces numerous challenges under complex lighting conditions. Changes in lighting make image feature extraction difficult, hindering visual navigation algorithms from accurately identifying environmental features and thus affecting navigation accuracy. In low-light environments, increased image noise and decreased image quality further reduce the reliability of the visual navigation system. This application addresses this issue by enhancing images to construct clearer maps and identifying landmarks within the enhanced images as navigation guides, thereby improving the reliability of drone navigation and enabling the drone to accurately reach its target location.

[0061] The application of the UAV control method of the present invention will be described below according to different usage scenarios.

[0062] Example 1: Inspection of a photovoltaic power station

[0063] This embodiment describes an inspection experiment conducted at a large photovoltaic power station. The power station covers a vast area with densely distributed photovoltaic modules. The experiment included various complex lighting conditions, such as direct sunlight on sunny days, shadows cast by buildings or other objects blocking some modules, and diffused light on cloudy days.

[0064] When the drone flies under direct sunlight, the image enhancement module quickly activates, and the algorithm based on GAN and Retinex theories begins to work. Since the generator has learned the characteristics of photovoltaic module images under strong light based on a large amount of training data, it can process the acquired overexposed images, suppressing the brightness of highlight areas and restoring lost details, making the outline and surface texture of the photovoltaic modules clearer. The enhanced image clearly shows anomalies such as tiny cracks, stains, and loose connections on the surface of the photovoltaic modules. Furthermore, the light perception and compensation module can obtain the comprehensive light intensity based on the light sensor and image recognition technology, and adjust the parameters of the second illumination component map according to the comprehensive light intensity. Because the comprehensive light intensity is high, the brightness gain of the second illumination component map is reduced, thereby avoiding overexposure. Through this dynamic adjustment mechanism, the enhanced image synthesized from the second high-quality illumination component map and the second high-quality reflection component map has higher quality and usability, and can reliably identify defects in the photovoltaic modules.

[0065] Visual inertial odometry (VIO) calculates the UAV's pose in real time by fusing enhanced image information with acceleration and angular velocity data provided by the inertial measurement unit (IMU), ensuring stable flight attitude. A deep learning target detection model identifies navigation landmarks such as iconic buildings, roads, and specific component arrangement patterns within the photovoltaic power station, providing the UAV with accurate positioning and navigation information. Combined with map building and path planning algorithms, the UAV efficiently completes the inspection of the entire photovoltaic power station according to a pre-set inspection route, accurately detecting problematic photovoltaic modules and transmitting relevant images and data back to the ground control center in real time.

[0066] Example 2: Mountain Topographic Mapping

[0067] This embodiment involves a topographic mapping experiment conducted in a mountainous area with complex terrain and variable lighting. The mountainous terrain is highly undulating, with numerous valleys, peaks, and ravines. Furthermore, due to variations in the solar altitude angle and the obstruction and reflection of sunlight by the terrain, the lighting conditions are extremely complex.

[0068] During flight, the illumination sensing and compensation unit plays a crucial role. Illumination sensors monitor changes in ambient light intensity in real time, while image analysis methods extract illumination features from the acquired images. The combination of these two methods accurately determines the current overall illumination intensity. Based on different illumination conditions, the system automatically adjusts the parameters of the second illumination component map. In sunlit mountain areas, the brightness gain of the second illumination component map is reduced to avoid overexposure; in low-light areas such as valleys, the brightness and contrast enhancement parameters of the second illumination component map are increased to make terrain features more clearly visible.

[0069] The image enhancement module processes the acquired images, effectively removing the effects of uneven lighting and noise, and enhancing the detailed information of the terrain. The navigation module utilizes visual inertial odometry and deep learning object detection technology to achieve stable flight in complex mountainous terrain. Based on enhanced images and inertial data, the visual inertial odometry can accurately estimate the UAV's pose, thus overcoming the impact of unstable airflow on flight attitude in mountainous areas. The deep learning object detection model identifies landmarks such as ridges, rivers, and roads, providing accurate navigation references for the UAV. Combined with map building and path planning algorithms, the enhanced images enable the UAV to plan the optimal flight path in complex mountainous terrain, avoiding obstacles, achieving high-precision terrain mapping, and acquiring detailed terrain data, providing a reliable basis for subsequent geographic information analysis and engineering construction.

[0070] Example 3: Emergency Rescue Scenario Simulation

[0071] In the emergency rescue simulation scenario, complex lighting conditions such as smoke and low light were set up to simulate the actual situation at disaster sites such as fires and earthquakes. In a smoke environment, light is scattered and absorbed by smoke particles, making the image blurry and the visibility extremely low; the low light environment further increases the difficulty of image acquisition and analysis.

[0072] The image enhancement module pre-programs images based on the characteristics of smoke and low-light conditions. Using a generative adversarial network, it learns the features of the reflectance component maps decomposed in smoke and low-light environments, as well as the corresponding high-quality reflectance component maps. This allows it to generate a clear second reflectance component map in smoke and low-light conditions. Furthermore, considering the relatively low overall illumination intensity, it enhances the contrast and clarity of the second illumination component map, making rescue targets stand out in blurry images. By enhancing the image, key information such as the location of trapped personnel, the extent of building collapse, and road congestion can be clearly displayed.

[0073] In complex emergency rescue scenarios, the navigation module relies on visual-inertial odometry (VIO) and deep learning target detection technology to quickly and accurately reach the location of rescue targets. Under low visibility and complex terrain conditions, VIOS stably estimates the drone's pose by fusing enhanced imagery and inertial information. The deep learning target detection model identifies key landmarks such as the location of fire sources at fire scenes, signs of life in earthquake rubble, and rescue routes, providing navigation guidance for the drone. Enhanced imagery, combined with map building and path planning algorithms, enables the drone to plan safe and efficient flight paths in smoke and low-light environments, avoiding dangerous areas and delivering rescue supplies to the scene in a timely manner, providing strong support for rescue operations.

[0074] Furthermore, this invention does not preclude the use of a fusion of satellite navigation and lidar navigation as an alternative navigation module. Satellite navigation, such as GPS and BeiDou, provides absolute positioning information globally, offering advantages such as high precision and all-weather capability. LiDAR, on the other hand, acquires distance information by emitting laser pulses towards a target and measuring the time of reflected light, thereby achieving high-precision three-dimensional environmental perception. It is not limited by lighting conditions, can provide stable navigation information in various complex environments, and has strong anti-interference capabilities and high reliability. By fusing satellite navigation and lidar navigation, satellite navigation can provide the UAV with approximate position information, solving positioning problems during long-distance flights; lidar, utilizing its high-precision ranging and environmental perception capabilities, provides the UAV with precise local environmental information, enabling obstacle avoidance and accurate navigation. In urban environments, lidar can perceive the position and distance of surrounding buildings in real time, helping the UAV avoid obstacles, while satellite navigation ensures that the UAV flies along a predetermined route. However, this fusion approach also faces some challenges. Satellite navigation signals are susceptible to interference in certain environments, such as urban canyons with tall buildings or in severe weather conditions, where signals may be interrupted or errors may occur. LiDAR equipment is expensive and consumes a lot of power, which increases the weight and energy consumption of drones, affecting their endurance. Furthermore, LiDAR has high requirements for the surface of the target object; for rough objects with low reflectivity, the measurement accuracy may be affected.

[0075] According to a second aspect of this application, a drone control system is also provided, comprising: Generative Adversarial Network Training Module 10: is configured to obtain a first acquired image, decompose the first acquired image into a first reflection component map and a first illumination component map using the retinal cortex network unit 101, optimize the first reflection component map to obtain a first high-quality reflection component map, and use the first reflection component map and the first high-quality reflection component map to establish a dataset to train the generative adversarial network.

[0076] A dataset consisting of multiple first reflection component maps and first high-quality reflection component maps is used to train the Generative Adversarial Network (GAN) unit 102. The generator of the GAN unit 102 generates reflection component maps as input to the discriminator, attempting to deceive the discriminator into believing they are high-quality reflection component maps. The discriminator of the GAN unit 102 then determines whether the reflection component maps generated by the generator are high-quality or defective, continuously optimizing the discriminator's performance to encourage the generator to produce high-quality reflection component maps. During training, the generator and discriminator compete against each other, continuously improving their respective capabilities, ultimately enabling the generator to produce high-quality reflection component maps with clear details and good visual effects under complex lighting conditions.

[0077] In some embodiments, the generative adversarial network training module 10 further includes a reflection component optimization unit 103, which is configured to optimize the first reflection component map to obtain a first high-quality reflection component map, and the specific optimization process is the same as described above.

[0078] In this invention, Retinex theory is integrated into a GAN network. First, wavelet transform is used to optimize the first reflection component map into a first high-quality reflection component map. The GAN network is then trained using this first reflection component map and the high-quality reflection component map. This allows the GAN network to optimize the reflection component map under poor lighting conditions decomposed by Retinex into a high-quality reflection component map, thus removing potential noise and texture loss issues from the separated reflection component map. This approach fully leverages the advantages of GAN and Retinex theory to achieve efficient image sharpening under complex lighting conditions. Then, the retinal cortex network unit fuses the lighting component map and the generator-optimized high-quality reflection component map to obtain the enhanced image.

[0079] Image enhancement module 20: Configured to acquire a second captured image during UAV flight, decompose the second captured image into a second reflectance component map and a second illumination component map using a retinal cortex network unit 101, and optimize the second reflectance component map using a trained generative adversarial network unit 102 to generate a second high-quality reflectance component map. The retinal cortex network unit 101 can then synthesize the second high-quality reflectance component map and the second illumination component map into an enhanced image.

[0080] Since the generative adversarial network unit 102 has been trained, it can optimize the second reflection component map, which still contains brightness variations, to remove the influence of brightness variations as much as possible, thereby making the edges and textures of the optimized second high-quality reflection component map clearer.

[0081] In one feasible embodiment, the image enhancement module 20 further includes a light sensing and compensation unit 201, configured to obtain high-precision real-time measurements of parameters such as light intensity and color temperature in the environment from a light sensor. The light sensor operates based on the photoelectric effect, converting light signals into electrical signals. Through processing and analysis of these electrical signals, accurate light information is obtained. Under different lighting conditions, such as strong sunlight on a sunny day, diffused light on a cloudy day, and low light in the early morning and evening, the light sensor can respond quickly and provide reliable light data.

[0082] The illumination sensing and compensation unit 201 measures the ambient light intensity in real time using a light sensor and acquires the light intensity of the captured image using image analysis methods. It then fuses the light intensities obtained from the light sensor and image analysis methods to obtain a comprehensive light intensity. Based on this comprehensive light intensity, it optimizes the second illumination component map to generate a second high-quality illumination component map. The retinal cortex network can then synthesize the second high-quality reflectance component map and the second high-quality illumination component map into an enhanced image.

[0083] Image analysis methods, by analyzing features such as brightness and color distribution in the acquired images, can further accurately determine the current lighting conditions. For example, the presence of large areas of highlight in the acquired image indicates strong light intensity; conversely, an overall dark image with low color saturation suggests a low-light environment. By combining data from the light sensor and the analysis results of the acquired images, the system can comprehensively and accurately perceive the current lighting conditions.

[0084] The step of obtaining the illumination intensity of the acquired image using image analysis methods includes: converting the acquired image to HSV space, calculating the average pixel brightness of the V channel, and determining the converted illumination value of the acquired image based on a pre-established mapping relationship between the average pixel brightness and the illumination value.

[0085] The step of fusing the light intensity obtained by the light sensor and the image analysis method into a comprehensive light intensity includes: taking a weighted average of the light intensity measured by the light sensor and the converted illuminance value of the acquired image analyzed by the image analysis method to obtain the comprehensive light intensity.

[0086] Steps for evaluating the confidence level of the light sensor: If the light sensor measurement value is within the set range (e.g., 10-10000 Lux), then the light sensor data is the primary factor, and the confidence level of the light sensor is set to 0.8; otherwise, the confidence level of the light sensor is set to 0.2. The steps for evaluating image confidence are as follows: Brightness uniformity = 1 - (image brightness standard deviation / image brightness mean). If brightness uniformity > 0.7, it indicates that the image brightness value is reliable, and the image confidence is set to 0.7; otherwise, the image confidence is set to 0.3. Then the weights are normalized, i.e., total confidence = illumination sensor confidence + image confidence. w1 = Confidence level of light sensor / Total confidence level w2 = Image confidence score / Total confidence score Comprehensive light intensity c= , Where a is the light intensity measured by the light sensor; It is the illumination intensity obtained from image analysis; a and All of this data has been normalized. w1 is the normalized confidence level of the illumination sensor; w2 is the normalized image confidence score.

[0087] The second illumination component image is dynamically adjusted based on the overall illumination intensity. Under high overall illumination intensity (i.e., strong light conditions), the brightness gain of the second illumination component image is reduced to avoid overexposure. Under low overall illumination intensity (i.e., low light environments), the brightness and contrast parameters of the second illumination component image are increased to improve its sharpness, thereby generating a high-quality second illumination component image. This dynamic adjustment mechanism enables the enhancement of the image quality and usability of the combined high-quality second illumination component image and the high-quality second reflection component image.

[0088] Navigation module 30 is configured to determine the UAV pose using visual inertial odometry 301 in conjunction with the enhanced image, construct a UAV flight environment map in real time using simultaneous localization and mapping unit (SLAM) 302 in conjunction with the enhanced image, achieve UAV positioning by matching the pose with the map, and plan a flight path using a path planning algorithm based on the UAV's current position, target position, and map information.

[0089] Visual inertial odometry (VIO) combines information from visual sensors (such as cameras) and inertial measurement units (IMUs). By fusing enhanced image processing from the image enhancement module with inertial data, it can estimate the UAV's pose information in real time. In terms of visual processing, feature extraction and matching algorithms, such as the ORB (Oriented Fast and Rotated BRIEF) algorithm, are used to extract feature points from the enhanced image. By matching feature points in different frames of enhanced images, the relative motion of the UAV is calculated. The inertial measurement unit provides the UAV's acceleration and angular velocity information. By integrating the acceleration and angular velocity information, the attitude and position changes of the UAV can be obtained. The fusion of visual and inertial information effectively improves the accuracy and stability of pose estimation and reduces error accumulation.

[0090] In this process, simultaneous localization and mapping (SMR) technology is used to build a map of the surrounding environment in real time using augmented images during flight. The augmented images can accurately reflect the details of the terrain and landforms, improving the accuracy and reliability of the constructed map data. The pose information determined by visual inertial odometry (VIO) is matched with the map to achieve precise positioning.

[0091] The system also uses a deep learning unit 303, such as a target detection model based on a convolutional neural network (CNN), to identify landmark objects in the UAV flight environment from the enhanced images, determine the 3D coordinates of the landmark objects from the enhanced images of different frames, and update them in the map constructed by the simultaneous localization and mapping method based on their 3D coordinates.

[0092] It also utilizes path planning algorithms to plan a safe and efficient flight path based on the drone's current location, target location, and map information.

[0093] The deep learning unit is used to identify navigation landmarks in the drone's flight environment. A deep neural network model can be trained beforehand using images of landmark objects; the deep learning unit can be, for example, a convolutional neural network (CNN)-based object detection model. This training enables the drone to quickly and accurately identify specific landmark objects, such as buildings, road signs, and mountains, from the enhanced images output by the image enhancement module. The 3D coordinates of these landmark objects can be determined from different frames of enhanced images, and these coordinates are then used to update the SLAM-built map. These landmark objects serve as navigation reference points, providing the drone with environmental information and positioning guidance. During flight, the drone continuously detects landmark objects in its surrounding environment and adjusts its flight path based on their position and attitude information to ensure accurate arrival at the target location.

[0094] Inertial navigation has the advantages of strong autonomy and immunity to external signal interference. However, its positioning error gradually increases over time, leading to a decrease in navigation accuracy. During long-duration flight missions, inertial navigation errors may cause the UAV to deviate from its planned flight path and fail to accurately reach the target location.

[0095] Visual navigation utilizes cameras mounted on drones to acquire image information of the surrounding environment, achieving autonomous navigation through image processing and computer vision techniques. Visual navigation boasts advantages such as strong environmental perception and the ability to acquire rich scene information. However, it also faces numerous challenges under complex lighting conditions. Changes in lighting make image feature extraction difficult, hindering visual navigation algorithms from accurately identifying environmental features and thus affecting navigation accuracy. In low-light environments, increased image noise and decreased image quality further reduce the reliability of the visual navigation system. This application addresses this issue by enhancing images to construct clearer maps and identifying landmarks within the enhanced images as navigation guides, thereby improving the reliability of drone navigation and enabling the drone to accurately reach its target location.

[0096] Figure 4 A schematic diagram of an application scenario of the UAV control method described in this invention is shown.

[0097] exist Figure 4 In the application scenario, computing device 100 can acquire images. Then, computing device 100 can use Retinex to decompose the first acquired image into a first reflection component map and a first illumination component map, and use the reflection component optimization unit 103 to optimize the first reflection component map to obtain a first high-quality reflection component map. Afterwards, computing device 100 can train a GAN network using the first reflection component map and the first high-quality reflection component map. During UAV inspection, computing device 100 can use Retinex to decompose a second real-time captured image into a second reflection component map and a second illumination component map. The trained GAN network optimizes the second reflection component map into a second high-quality reflection component map. Computing device 100 also uses the illumination sensing and compensation unit 201 to adjust the parameters of the second illumination component map to obtain a second high-quality illumination component map. The second high-quality illumination component map and the second high-quality reflection component map are synthesized into an enhanced image for use in the navigation module.

[0098] The computing device 100 uses visual inertial odometry 301 combined with enhanced images to obtain the UAV pose, and uses SLAM 302 to construct a map during flight. The deep learning unit 303 identifies landmark objects in the enhanced images, and updates the map constructed by the simultaneous localization and mapping method according to their 3D coordinates.

[0099] It should be noted that the aforementioned computing device 100 can be either hardware or software. When the computing device 100 is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or as a single server or a single terminal device. When the computing device 100 is software, it can be installed in the hardware devices listed above. It can be implemented as, for example, multiple software programs or software modules used to provide distributed services, or as a single software program or software module. No specific limitations are made here.

[0100] Figure 5 A schematic diagram illustrating another application scenario of the UAV control method described in this invention is shown.

[0101] exist Figure 5 In this application scenario, the components of the computing device 100 include, but are not limited to, a memory 110 and a processor 120. The processor 120 is connected to the memory 110 via a bus 130, and the database 150 is used to store data.

[0102] The computing device 100 also includes an access device 140, which enables the computing device 100 to communicate via one or more networks 160. Examples of such networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 140 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0103] In one embodiment of the present invention, the above-mentioned components of the computing device 100 and Figure 5 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 5 The illustrated block diagram of the computing device is for illustrative purposes only and is not intended to limit the scope of the invention. Those skilled in the art can add or replace other components as needed.

[0104] The computing device 100 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 100 can also be a mobile or stationary server.

[0105] The processor 120 is configured to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the aforementioned drone control method. The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the aforementioned drone control method belong to the same concept; details not described in detail in the technical solution of the computing device can be found in the description of the technical solution of the aforementioned drone control method.

[0106] The present invention also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the above-described UAV control method.

[0107] The above is an illustrative embodiment of the computer-readable storage medium described in this invention. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solution of the aforementioned UAV control method. Details not described in detail in the technical solution of the storage medium can be found in the description of the technical solution of the aforementioned UAV control method.

[0108] The present invention also provides a computer program, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the above-described unmanned aerial vehicle (UAV) control method.

[0109] The above is an illustrative scheme of the computer program described in this invention. It should be noted that the technical solution of this computer program and the technical solution of the aforementioned UAV control method belong to the same concept. Details not described in detail in the computer program's technical solution can be found in the description of the aforementioned UAV control method's technical solution.

[0110] The foregoing has described specific embodiments of the invention. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0111] Of course, the present invention may have other various embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art can make various corresponding changes and modifications according to the present invention, but these corresponding changes and modifications are all within the protection scope of the claims of the present invention.

Claims

1. A method for controlling an unmanned aerial vehicle (UAV), characterized in that, include: Generative Adversarial Network (GAN) training steps: Obtain the first acquired image, decompose the first acquired image into a first reflection component map and a first illumination component map using the retinal cortex network, optimize the first reflection component map to obtain a first high-quality reflection component map, and use the first reflection component map and the first high-quality reflection map to build a dataset to train the GAN. Image enhancement steps: Obtain a second acquired image captured during the flight of the UAV; decompose the second acquired image into a second reflection component map and a second illumination component map using a retinal cortex network; optimize the second reflection component map using a trained generative adversarial network to generate a second high-quality reflection component map; optimize the second illumination component map based on the comprehensive illumination intensity to generate a second high-quality illumination component map; and combine the second high-quality reflection component map and the second high-quality illumination component map into an enhanced image using a retinal cortex network. Navigation steps: The UAV pose is determined by combining visual inertial odometry with the enhanced image. A flight environment map of the UAV is constructed in real time using a simultaneous localization and mapping (SMR) method combined with the enhanced image. The UAV is located by matching the pose with the map. A flight path is planned based on the UAV's current position, target position, and map information using a path planning algorithm.

2. The UAV control method according to claim 1, characterized in that, The step of optimizing the first reflection component map to obtain the first high-quality reflection component map includes: The first reflection component image is subjected to discrete wavelet transform, with a decomposition of no less than 3 layers. Each layer yields a set of sub-band images, each containing a low-frequency sub-band and a high-frequency sub-band. The low-frequency sub-band is input into a smoothing filter to smooth the illumination, and the high-frequency sub-band is denoised by removing wavelet coefficients whose absolute values ​​are less than the noise threshold and retaining wavelet coefficients whose absolute values ​​are greater than or equal to the noise threshold. The processed low-frequency and high-frequency sub-bands are then reconstructed using inverse discrete wavelet transform to obtain the first high-quality reflection component image.

3. The UAV control method according to claim 1, characterized in that, The visual inertial odometry includes an inertial measurement unit (IMU) and a vision unit. The IMU measures the acceleration and angular velocity information of the UAV and obtains the UAV's pose by integrating the acceleration and angular velocity. The vision unit uses the ORB algorithm to extract feature points from the enhanced image and calculates the UAV's pose by matching feature points in different frames of enhanced images. The pose obtained by the IMU and the pose obtained by the vision unit are fused to determine the UAV's pose.

4. The UAV control method according to claim 3, characterized in that, The system also uses deep learning methods to identify landmark objects in the UAV flight environment from the enhanced images, determines the 3D coordinates of the landmark objects using enhanced images of different frames, and updates the map based on the 3D coordinates of the landmark objects.

5. The UAV control method according to claim 2, characterized in that, The step of performing discrete wavelet transform on the first reflection component map refers to using the Bior 3.5 wavelet as the wavelet basis for discrete wavelet transform.

6. The UAV control method according to claim 2, characterized in that, The smoothing filter is a weighted least squares filter.

7. The UAV control method according to claim 2, characterized in that, The aforementioned high-frequency subband denoising process refers to the use of soft threshold denoising.

8. The UAV control method according to claim 1, characterized in that, The step of optimizing the second illumination component map based on the comprehensive illumination intensity to generate the second high-quality illumination component map includes measuring the illumination intensity in the environment in real time using a light sensor, obtaining the illumination intensity of the acquired image using an image analysis method, fusing the illumination intensity obtained by the light sensor and the image analysis method as a comprehensive illumination intensity, optimizing the second illumination component map based on the comprehensive illumination intensity, and generating the second high-quality illumination component map.

9. The UAV control method according to claim 8, characterized in that, The method of obtaining the illumination intensity of the acquired image using image analysis includes: converting the acquired image to HSV space, calculating the average pixel brightness of the V channel, and determining the converted illumination value of the acquired image based on the pre-established mapping relationship between the average pixel brightness and the illumination value.

10. The UAV control method according to claim 9, characterized in that, The method of fusing the light intensity obtained by the light sensor and the image analysis method into a comprehensive light intensity includes: taking a weighted average of the light intensity measured by the light sensor and the converted illuminance value of the acquired image analyzed by the image analysis method to obtain the comprehensive light intensity.

11. The UAV control method according to claim 10, characterized in that, The step of taking a weighted average of the light intensity measured by the light sensor and the light intensity of the acquired image analyzed by the image analysis method to obtain the comprehensive light intensity includes: Steps for evaluating the confidence level of the light sensor: If the light sensor measurement value is within the set range, then the light sensor data is taken as the main factor, and the confidence level of the light sensor is set to 0.8; otherwise, the confidence level of the light sensor is set to 0.

2. The steps for evaluating image confidence are as follows: Brightness uniformity = 1 - (image brightness standard deviation / image brightness mean). If brightness uniformity > 0.7, it indicates that the image brightness value is reliable, and the image confidence is set to 0.7; otherwise, the image confidence is set to 0.

3. Then, the weights are normalized to obtain the comprehensive illumination intensity c= , Where a is the light intensity measured by the light sensor; It is the illumination intensity obtained from image analysis; a and All of this data has been normalized. w1 is the normalized confidence level of the illumination sensor; w2 is the normalized image confidence score.

12. A drone control system, characterized in that, include: Generative Adversarial Network Training Module: It is configured to obtain a first acquired image, decompose the first acquired image into a first reflection component map and a first illumination component map using the retinal cortex network, optimize the first reflection component map to obtain a first high-quality reflection component map, and use the first reflection component map and the first high-quality reflection map to build a dataset to train the generative adversarial network. Image enhancement module: configured to acquire a second captured image during the flight of the UAV, decompose the second captured image into a second reflection component map and a second illumination component map using a retinal cortex network, optimize the second reflection component map using a trained generative adversarial network to generate a second high-quality reflection component map, optimize the second illumination component map according to the comprehensive illumination intensity to generate a second high-quality illumination component map, and synthesize the second high-quality reflection component map and the second high-quality illumination component map into an enhanced image using a retinal cortex network; The navigation module is configured to determine the UAV's pose using a visual inertial odometry system combined with the enhanced image, construct a real-time UAV flight environment map using a simultaneous localization and mapping (SMR) method combined with the enhanced image, achieve UAV positioning by matching the pose with the map, and plan a flight path using a path planning algorithm based on the UAV's current position, target position, and map information.

13. A computing device, characterized in that, include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the UAV control method according to any one of claims 1 to 11.

14. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the unmanned aerial vehicle control method according to any one of claims 1 to 11.

Images