Unattended full-automatic unmanned aerial vehicle observation system

By using a blind zone self-discovery and filling algorithm based on a multi-stage hybrid generation framework, the problem of data incompleteness caused by dynamic blind zones in UAV observation systems is solved, achieving data integrity restoration and improving system robustness, while optimizing energy consumption and autonomy.

CN120877155AActive Publication Date: 2025-10-31DI RUI TIANCHENG INFORMATION TECH (BEIJING) CO LTD

Patent Information

Application Number
CN202511031644.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-07-25
Publication Date
2025-10-31
Estimated Expiration
2045-07-25

AI Technical Summary

Technical Problem

Existing unmanned fully automated drone observation systems suffer from incomplete data due to dynamic blind spots during long-term observation. Traditional methods increase energy consumption and reduce observation efficiency, and lack quantitative assessment of blind spot uncertainties and adaptive decision-making mechanisms, affecting the system's autonomy and robustness.

Method used

The observation blind zone self-discovery and filling algorithm adopts a multi-stage hybrid generative framework, including a blind zone detection module, a conditional generative adversarial network variant module, a Bayesian uncertainty quantification submodule, and a reinforcement learning agent module. It realizes blind zone self-discovery and filling through multi-sensor data streams, dynamically adjusts the confidence threshold, and makes decisions on fusion or reflight verification paths.

Benefits of technology

It improves the integrity and accuracy of observation data, reduces the frequency of flyback verification, optimizes energy consumption and flight efficiency, enhances the system's robustness and autonomous adaptability in dynamic environments, and is suitable for long-term observation processes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120877155A_ABST
    Figure CN120877155A_ABST
Patent Text Reader

Abstract

The invention discloses an unattended full-automatic unmanned aerial vehicle observation system, and relates to the technical field of unmanned aerial vehicles, and the system comprises a multi-sensor group which is used for collecting real-time multi-sensor data streams; an observation blind area self-discovery filling algorithm is stored in the memory, and the processor executes the observation blind area self-discovery filling algorithm and comprises a blind area detection module, a conditional generative adversarial network variant module, a Bayesian uncertainty quantization sub-module and a reinforcement learning agent module; the blind area detection module extracts a blind area boundary mask through a convolutional layer and quantizes a geometric shape; a conditional generative adversarial network variant module generates candidate filling data; the Bayesian uncertainty quantization sub-module deduces and calculates the evidence lower bound of each generated pixel or point; and the reinforcement learning agent module decides and fuses candidate filling data or triggers a reflight verification path. According to the invention, through the multi-stage hybrid generation framework of the observation blind area self-discovery filling algorithm, self-discovery and filling of the blind area of the unmanned aerial vehicle are realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) technology, and more specifically, to an unattended, fully automated UAV observation system. Background Technology

[0002] In existing technologies, unmanned fully automated drone observation systems are widely used in environmental monitoring, geological exploration, and ecological tracking. These systems typically rely on multi-sensor groups to collect real-time data streams, such as camera images, LiDAR point clouds, and acoustic wave sensor reflections, to achieve autonomous observation. However, during long-term observation, the dynamic blind zone problem has become a key bottleneck restricting system performance. These dynamic blind zones are often caused by factors such as fog obstruction, vegetation obstruction, or terrain depressions, preventing sensors from directly acquiring complete data. Traditional methods mainly employ passive avoidance strategies, such as path detours or post-hoc data interpolation, but these methods have significant shortcomings: on the one hand, path detours increase energy consumption and flight time, reducing observation efficiency; on the other hand, simple interpolation ignores the weak multimodal signals at the edge of the blind zone, failing to accurately reconstruct complex data distributions, easily introducing errors and illusions, resulting in incomplete overall observation data, which in turn affects the reliability of subsequent analysis and the accuracy of decision-making. Furthermore, existing systems lack quantitative assessment and adaptive decision-making mechanisms for blind zone uncertainties, and cannot achieve end-to-end real-time processing, further limiting the system's autonomy and robustness in complex environments. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide an unattended, fully automatic unmanned aerial vehicle (UAV) observation system to address the problems mentioned in the background art.

[0004] To achieve the above objectives, the present invention adopts the following technical solution: An unattended, fully automated drone observation system includes the drone itself, a multi-sensor group, a processor, and a memory; The multi-sensor group is used to collect real-time multi-sensor data streams; The memory stores an observation blind zone self-discovery and filling algorithm, and the processor executes the observation blind zone self-discovery and filling algorithm to process the real-time multi-sensor data stream, thereby realizing blind zone self-discovery and filling. The self-discovery and filling algorithm for observation blind spots is a multi-stage hybrid generation framework, including a blind spot detection module, a conditional generative adversarial network variant module, a Bayesian uncertainty quantification sub-module, and a reinforcement learning agent module. The blind zone detection module extracts the blind zone boundary mask and quantizes the geometry through a convolutional layer; the conditional generative adversarial network variant module incorporates a self-attention mechanism, using multimodal feature vectors extracted from the blind zone edge as conditional inputs to generate candidate filling data; the Bayesian uncertainty quantification submodule calculates the lower bound of evidence for each generated pixel or point based on variational inference and dynamically adjusts the confidence threshold. The reinforcement learning agent module evaluates the consistency reward of the filled data based on the deep Q network, and decides whether to fuse the candidate filled data or trigger a re-flight verification path, thereby realizing an end-to-end self-discovery, generation and verification loop.

[0005] More specifically, the multi-sensor group includes a camera, a LiDAR sensor, and an acoustic sensor, used to acquire real-time multi-sensor data streams, including camera images, LiDAR point clouds, and acoustic sensor reflections.

[0006] More specifically, the blind spot detection module analyzes the real-time multi-sensor data stream, extracts the blind spot boundary mask from the camera image using the convolutional layer, extracts the point cloud boundary mask from the LiDAR point cloud, extracts the reflection boundary mask from the acoustic sensor reflection, and fuses the blind spot boundary mask, the point cloud boundary mask, and the reflection boundary mask to quantize the geometry, which includes a convex hull boundary.

[0007] More specifically, the multimodal feature vector includes the Sobel gradient of image edges, the point density gradient of LiDAR, and the spectral features of sound waves.

[0008] More specifically, the process of processing the multimodal feature vector includes calculating the attention weights of the Sobel gradient of the image edge and the point density gradient of LiDAR, and applying the attention weights to generate pixel values ​​or point cloud values ​​in the candidate filling data.

[0009] More specifically, the dynamic adjustment of the confidence threshold is applied to the historical fill accuracy via gradient descent, with the initial value of the confidence threshold being 0.75.

[0010] More specifically, the historical fill accuracy is obtained from a comparison of previously fused candidate fill data with actual verification data, the comparison including calculating the mean square error at the pixel level or point cloud level.

[0011] More specifically, the trigger go-around verification path includes generating an alternative flight path to bypass the blind zone or to reapproach the edge of the blind zone to collect additional data.

[0012] More specifically, the blind spot detection module is based on the U-Net architecture.

[0013] More specifically, the self-discovery and filling algorithm for observation blind spots includes sensor blind spots caused by fog, vegetation obstruction, or terrain depressions.

[0014] The advantages of this invention compared to existing technologies lie in its multi-stage hybrid generation framework for blind zone self-discovery and filling algorithms, achieving blind zone self-discovery and filling. This framework includes a blind zone detection module, a conditional generative adversarial network variant module, a Bayesian uncertainty quantification submodule, and a reinforcement learning agent module. It can extract multimodal feature vectors from the blind zone edge to generate candidate filling data, and dynamically adjust the confidence threshold through evidence lower bound calculation and consistency reward evaluation, deciding whether to fuse the filling data or trigger a re-flight verification path, thereby ensuring an end-to-end self-discovery, generation, and verification loop. This invention improves the integrity and accuracy of observation data, reduces data loss and analytical bias caused by blind zones, lowers the frequency of re-flight verification, optimizes energy consumption and flight efficiency, and enhances the system's robustness and autonomous adaptability in dynamic environments. It is suitable for long-term observation processes, such as those with fog obstruction or terrain depressions, achieving data integrity recovery. Attached Figure Description

[0015] Figure 1 This is a flowchart of the invention; Figure 2 This is a system structure diagram of the present invention; Figure 3 This is a flowchart of the blind zone detection module of the present invention; Figure 4 This is a flowchart of the process for generating fill data in this invention. Detailed Implementation

[0016] The specific embodiments of the present invention will now be described with reference to the accompanying drawings.

[0017] like Figure 1 and Figure 2 As shown, the unattended fully automatic drone observation system of the present invention consists of a drone body, a multi-sensor group, a processor, and a memory.

[0018] The multi-sensor group specifically includes a camera, a LiDAR sensor, and an acoustic sensor, which work together to collect real-time multi-sensor data streams; The camera is responsible for acquiring image data within the visible light range; LiDAR sensors generate three-dimensional point cloud data by measuring distance using laser pulses, while acoustic sensors emit sound waves and receive reflected signals to detect changes on the surface or medium of an object.

[0019] This data stream is generated in real time during the observation process. For example, in foggy conditions, camera images may show blurry areas, LiDAR point clouds may show sparse areas, and acoustic sensor reflections may cause signal attenuation and interruption, thus forming dynamic blind spots.

[0020] The memory pre-stores the self-discovery and filling algorithm for observation blind spots. The processor executes this algorithm to process the real-time multi-sensor data stream, thereby realizing the automatic discovery and data filling of blind spots.

[0021] The algorithm of this invention adopts a multi-stage hybrid generation framework design, which organically integrates multiple modules to ensure full automation from blind spot detection to data verification.

[0022] Specifically, the framework of this invention includes a blind spot detection module, a conditional generative adversarial network variant module, a Bayesian uncertainty quantification submodule, and a reinforcement learning agent module. These modules cooperate sequentially, first detecting the blind spot location, then generating the filling content, quantifying the uncertainty, and finally deciding whether to adopt the filling result, forming a closed-loop processing mechanism.

[0023] like Figure 3 As shown, the blind zone detection module is built on the U-Net architecture, which is a common semantic segmentation network consisting of an encoder and a decoder. The encoder extracts features step by step through multiple convolutions, while the decoder restores the resolution through upsampling and fuses low-level and high-level features in skip connections to accurately locate the blind zone boundary.

[0024] In practical implementation, this module analyzes real-time multi-sensor data streams, using convolutional layers to extract blind zone boundary masks from camera images (e.g., identifying regions with abrupt pixel value changes using edge detection operators); extracting point cloud boundary masks from LiDAR point clouds (e.g., calculating threshold regions where point density decreases); and extracting reflection boundary masks from acoustic sensor reflections (e.g., analyzing the starting point of waveform amplitude attenuation). These masks are then fused, integrated into a unified boundary representation through weighted averaging or logical operations, and their geometry is quantified. Specifically, the geometry includes a convex hull boundary, obtained by calculating the minimum convex polygon of the mask point set, used to describe the size and shape of the blind zone's outer contour. For example, in a river monitoring scenario, if fog covers the middle section of the water surface, the module extracts a fog edge mask from the image, a sparse shoreline point mask from the point cloud, and a reflection interruption mask from the acoustic wave. After fusion, an elliptical convex hull boundary with an area of ​​approximately 100 square meters is generated, helping subsequent modules focus on the processing range.

[0025] like Figure 4As shown, the Conditional Generative Adversarial Network variant module is responsible for generating candidate padding data within the framework. This module is an improved form of the standard generative adversarial network, containing a generator and a discriminator. The generator learns to produce data from noisy or conditional inputs, and the discriminator evaluates the realism.

[0026] To adapt to multimodal data, this module incorporates a self-attention mechanism, which allows the model to dynamically allocate attention weights when processing features, emphasizing relevant parts while ignoring noise. In its implementation, the self-attention mechanism processes multimodal feature vectors extracted from blind zone edges. These vectors include the Sobel gradient of image edges (pixel gradient vectors calculated using the Sobel operator to capture texture changes); the point density gradient of LiDAR (local density differences in the point cloud to represent 3D structural transitions); and the spectral features of sound waves (frequency components obtained through Fourier transform to reflect medium properties). The processing first calculates the attention weights for the Sobel gradient of image edges and the point density gradient of LiDAR, for example, by normalizing the dot product using a softmax function. These weights are then applied to generate pixel values ​​or point cloud values ​​in the candidate filling data.

[0027] Specifically, the generator takes these weighted vectors as conditional inputs and outputs reconstructed blind zone data, such as filled image pixels or supplemented point cloud locations. During training, this module uses adversarial loss and reconstruction loss for joint optimization; the generator attempts to deceive the discriminator while minimizing the L1 distance from the real data. Training data comes from labeled observation datasets, such as synthetic samples simulating fog blind zones. The number of iterations is typically set to 10,000, and the learning rate starts at 0.001 and gradually decays. Through this design, the module can generate more context-appropriate fill content. For example, in forest observations with vegetation occlusion, features extracted from edge gradients are used to generate ground point clouds hidden under leaves, avoiding the distortion of simple interpolation.

[0028] The Bayesian uncertainty quantification submodule runs in parallel with the conditional generative adversarial network variant module. This parallelism is achieved through multithreading or GPU acceleration to ensure that the generation speed is not affected. This submodule computes the lower bound of evidence for each generated pixel or point based on the variational inference method. Variational inference is a technique that approximates the Bayesian posterior by introducing a variational distribution q to minimize the KL divergence with the true posterior, thereby estimating the uncertainty.

[0029] In the computation, pixels or points are generated by sampling from the candidate imputation data, for example, by randomly selecting 100 samples. Then, a lower bound of evidence is calculated for each sample. This value is based on the expectation, log-likelihood, and KL divergence under the variational distribution. Specifically, the lower bound of evidence equals the log-likelihood under the variational expectation minus the KL divergence between the variational distribution and the prior distribution. This setting is because it provides a lower bound approximation, which can efficiently quantify the model's confidence in the generated data without requiring precise integration of the high-dimensional posterior.

[0030] A higher lower bound on the evidence indicates lower uncertainty. The value can be set between -10 and 0, with negative values ​​representing approximation errors. After uncertainty quantification, the submodule dynamically adjusts the confidence threshold, applying it to historical fill accuracy via gradient descent. The initial confidence threshold is 0.75, ranging from 0.5 to 0.95, adjusted according to the actual environment to balance accuracy and efficiency. Historical fill accuracy is obtained by comparing previously fused candidate fill data with actual validation data, for example, by calculating pixel-level mean squared error. If the error is less than 0.1, it is considered accurate, and the accuracy is accumulated. Gradient descent uses the Adam optimizer with a step size of 0.01 and 5 to 10 iterations. This mechanism ensures that only reliable fills are used. For example, in observations of mountainous areas with terrain depressions, if the lower bound on the evidence for generating point clouds is below -5, the submodule lowers the confidence threshold to 0.6 to prevent the introduction of high-uncertainty data.

[0031] The reinforcement learning agent module is built upon a Deep Q-network, a value function approximation reinforcement learning model composed of multiple fully connected layers. The input state is padded data, and the output is the action Q-value used for decision-making. The Deep Q-network architecture includes an input layer receiving consistent reward features, a three-layer hidden layer with 128 neurons, and an output layer corresponding to actions such as fusion or re-flight. The training process uses an experience replay buffer to store state transitions, and the target network is periodically updated to stabilize learning. A discount factor of 0.99 is used, and the exploration rate decays from 1 to 0.1. Training data comes from simulated observation episodes, with each episode simulating a blind zone event. The reward is designed as a negative KL divergence plus a positive confidence score.

[0032] This module receives a consistency reward for the filled data as input. This reward is based on the KL divergence, confidence weight factor, and confidence score between the filled data distribution and the distribution of surrounding data. The KL divergence measures the distribution difference and ranges from 0 to infinity, with smaller values ​​indicating greater consistency. The confidence weight factor is set to 0.4 to 0.6 and adjusted according to importance. The confidence score is obtained directly from the Bayesian submodule.

[0033] After reward calculation, it is used to evaluate consistency. If the reward is higher than 0, the decision is made to fuse candidate data, that is, to insert the generated content into the real-time multi-sensor data stream to fill the blind spot. This insertion is performed smoothly through linear interpolation and only occurs when the confidence level exceeds a preset value, such as 0.7. Otherwise, a re-flight verification path is triggered, for example, generating an alternative flight path to bypass the blind spot or to re-approach the edge to collect additional data. The path is integrated into the path planning module using the A* algorithm and is seamlessly combined with the observation algorithm. The planning module uses a grid map with a resolution of 0.1 meters, taking energy costs into account. This decision-making process realizes an end-to-end self-discovery, generation, and verification loop, which is executed in real time on the processor, continuously inputting the real-time multi-sensor data stream, outputting the filled observation data, and feeding it back to the system path control to adjust the flight attitude.

[0034] The self-discovery and filling algorithm for observation blind spots is particularly suitable for long-term observation processes, such as ecological monitoring lasting several hours. During this process, it addresses dynamic blind spots such as image blurring due to fog, missing point clouds due to vegetation obstruction, or sensor blind spots caused by terrain depressions. Through the collaboration of a multi-stage hybrid generation framework, these blind spots are detected, filled, their uncertainties quantified, and verified, ultimately restoring data integrity. For example, in a long-term river pollution tracking task, when a drone encounters a fog blind spot after flying for 8 hours, the detection module quantizes the convex hull boundary as a rectangular region; the generation module generates a pollutant concentration map from Sobel gradients and spectral features; the uncertainty submodule calculates the lower bound of evidence at an average of -1.5 and adjusts the threshold to 0.8; the surrogate module calculates a reward of 0.3 and fuses the data to complete the distribution map, avoiding the omission of pollution peaks caused by missing data. The entire loop executes every second, ensuring data continuity.

[0035] In terms of system hardware configuration, the drone body adopts a quadcopter structure, and the processor uses an NVIDIA Jetson series chip to support parallel computing; the memory capacity is no less than 64GB to accommodate algorithm models and data buffers. The sampling rate of the multi-sensor group is set to 30Hz to ensure data stream resolution, the camera resolution is 1920x1080, the LiDAR range is 500 meters with centimeter-level accuracy, and the acoustic sensor frequency is 50-200kHz.

[0036] In the software implementation, the algorithm framework is developed using PyTorch. The U-Net model has approximately 5 million parameters, and training is performed on a GPU with a batch size of 16. The loss function is binary cross-entropy. During training, the conditional generative adversarial network variant uses an adversarial loss weight of 0.5 and a reconstruction loss of 0.5. The training set includes 10,000 pairs of real observations and GAN-enhanced samples. The variational distribution of the Bayesian submodule adopts a Gaussian distribution with a mean of 0 and a variance of 1, and the prior is a standard normal distribution. The Deep Q-network is trained with 5,000 episodes and a buffer size of 20,000, with the target updated every 100 steps. This detailed implementation ensures the robustness of the algorithm; for example, in simulation tests, the imputation accuracy reaches 88%, which is 20% higher than the baseline interpolation.

[0037] To further refine the blind zone fusion process, in the detection module, the mask fusion adopts a pixel-level voting mechanism. If at least two of the three masks overlap, the boundary point is confirmed. When quantizing the convex hull, the Graham scan algorithm is used, with a time complexity of O(nlog n), where n is the number of points, which is usually less than 1000.

[0038] In the self-attention calculation of the generation module, the weight matrix dimension is set to 64 for the feature vector length, and the softmax temperature parameter is 1.0 to avoid gradient explosion. In uncertainty calculation, the sampling number can be dynamically set to 10%-50% of the generated pixels. ELBO maximization is used for evidence lower bound optimization, and gradients are updated through backpropagation of variational parameters. The confidence-adjusted gradient descent learning rate is 0.005, and the stopping condition is an accuracy change of less than 0.01. The input state of the reinforcement agent includes a reward vector dimension of 5, an action space of binary choice, and Huber loss is used for Q-value updates to handle outliers.

[0039] In one specific embodiment, consider an unattended, fully automated drone observation system deployed in a dense forest area for long-term monitoring of wildlife migration routes. The system is equipped with a drone body, a multi-sensor array, a processor, and memory. The multi-sensor array integrates high-resolution images acquired by a camera, detailed point cloud mapping of forest structure generated by a LiDAR sensor, and acoustic wave sensors to capture reflected waveforms caused by animal activity.

[0040] In a typical vegetation obstruction observation mission, the drone flew along a preset path along a forest trail, collecting multi-sensor data streams in real time: camera images showed blurred ground traces in dense foliage, LiDAR point clouds showed gaps in the leaf layer, and acoustic sensor reflections were interrupted by vegetation absorption, forming a dynamic blind zone of about 30 meters in diameter, obscuring potential animal paths.

[0041] The multi-stage hybrid generation framework of the blind zone self-discovery and filling algorithm is immediately activated. First, the blind zone detection module, based on the U-Net architecture, processes the data stream through convolutional layers: extracting blind zone boundary masks from camera images to identify pixel interruption regions at leaf edges; extracting point cloud boundary masks from LiDAR point clouds to mark boundaries where branch point density sharply decreases; and extracting reflection boundary masks from acoustic sensor reflections to detect the starting segments of waveform amplitude attenuation. These masks are fused and their geometry is quantized, such as calculating the convex hull boundary, forming a polygon surrounding the blind zone with a total side length of approximately 100 meters, helping the framework locate the processing focus.

[0042] Subsequently, a variant module of the conditional generative adversarial network incorporates a self-attention mechanism to extract multimodal feature vectors from the edges of blind zones: Sobel gradients at image edges capture leaf vein gradations, LiDAR point density gradients analyze branch and leaf transitions, and spectral features of sound waves reflect ground echo offsets. The self-attention mechanism calculates attention weights for the Sobel gradient and point density gradients, such as weighted emphasis on structural relevance, and then applies these weights to generate candidate filling data, such as reconstructing animal footprint pixels within blind zones or supplementing point cloud path locations, ensuring that the generated content is consistent with the surrounding forest data.

[0043] Meanwhile, the Bayesian uncertainty quantification submodule operates in parallel based on variational inference, sampling pixels or points from the candidate filling data, for example, selecting 200 samples. For each sample, it calculates a lower bound of evidence (the calculation of which is within existing technology). This value is obtained by subtracting the log-likelihood and KL divergence from the expectation under the variational distribution, representing the generation reliability. The initial confidence threshold is 0.75, dynamically adjusted using gradient descent based on historical filling accuracy. For example, from the pixel-level mean squared error of 0.08 in previous animal path filling, an accuracy of 0.85 is obtained, and the threshold is fine-tuned to 0.77.

[0044] Finally, the reinforcement learning agent module evaluates the data consistency reward after filling using a deep Q-network. The reward value is calculated to be approximately 0.55, based on the KL divergence (0.15) between the filled data distribution and its surrounding distributions, a confidence weight factor of 0.5, and a confidence score of 0.8. Based on this, the agent decision fuses the candidate filling data and inserts the reconstructed path into the data stream to fill the blind spot. Because the confidence score exceeds the preset value of 0.7, it does not trigger a re-flight verification path to save energy. The entire end-to-end self-discovery, generation, and verification loop is completed on the processor, outputting complete migration path data for base station analysis of animal behavior patterns.

[0045] Through this process, the system of this invention successfully reconstructs migration traces in vegetation blind spots, avoids path breaks caused by incomplete data, and ultimately tracks the hidden migration route of a herd of deer, thus optimizing forest protection measures.

[0046] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. An unattended, fully automated unmanned aerial vehicle (UAV) observation system, characterized in that, This includes the drone itself, a multi-sensor array, a processor, and memory. The multi-sensor group is used to collect real-time multi-sensor data streams; The memory stores an observation blind zone self-discovery and filling algorithm, and the processor executes the observation blind zone self-discovery and filling algorithm to process the real-time multi-sensor data stream, thereby realizing blind zone self-discovery and filling. The self-discovery and filling algorithm for observation blind spots is a multi-stage hybrid generation framework, including a blind spot detection module, a conditional generative adversarial network variant module, a Bayesian uncertainty quantification sub-module, and a reinforcement learning agent module. The blind zone detection module extracts the blind zone boundary mask and quantizes the geometry through a convolutional layer; The conditional generative adversarial network variant module incorporates a self-attention mechanism, using multimodal feature vectors extracted from the edge of the blind zone as conditional inputs to generate candidate filling data; the Bayesian uncertainty quantification submodule calculates the lower bound of evidence for each generated pixel or point based on variational inference and dynamically adjusts the confidence threshold. The reinforcement learning agent module evaluates the consistency reward of the filled data based on the deep Q network, and decides whether to fuse the candidate filled data or trigger a re-flight verification path, thereby realizing an end-to-end self-discovery, generation and verification loop.

2. The unattended fully automatic UAV observation system according to claim 1, characterized in that, The multi-sensor group includes a camera, a LiDAR sensor, and an acoustic sensor, used to acquire real-time multi-sensor data streams, including camera images, LiDAR point clouds, and acoustic sensor reflections.

3. The unattended fully automatic UAV observation system according to claim 2, characterized in that, The blind spot detection module analyzes the real-time multi-sensor data stream, extracts the blind spot boundary mask from the camera image using the convolutional layer, extracts the point cloud boundary mask from the LiDAR point cloud, extracts the reflection boundary mask from the acoustic sensor reflection, and fuses the blind spot boundary mask, the point cloud boundary mask, and the reflection boundary mask to quantize the geometry, which includes a convex hull boundary.

4. The unattended fully automatic UAV observation system according to claim 3, characterized in that, The multimodal feature vector includes the Sobel gradient of image edges, the point density gradient of LiDAR, and the spectral features of sound waves.

5. The unattended fully automatic UAV observation system according to claim 4, characterized in that, The process of processing the multimodal feature vectors includes calculating the attention weights of the Sobel gradient of the image edge and the point density gradient of LiDAR, and applying the attention weights to generate pixel values ​​or point cloud values ​​in the candidate filling data.

6. The unattended fully automatic UAV observation system according to claim 1, characterized in that, The dynamic adjustment of the confidence threshold is applied to the historical fill accuracy through gradient descent, and the initial value of the confidence threshold is 0.

75.

7. The unattended fully automatic UAV observation system according to claim 6, characterized in that, The historical fill accuracy is obtained by comparing the previously fused candidate fill data with the actual verification data, and the comparison includes calculating the mean square error at the pixel level or point cloud level.

8. The unattended fully automatic UAV observation system according to claim 1, characterized in that, The trigger go-back verification path includes generating an alternative flight path to bypass the blind zone or to reapproach the edge of the blind zone to collect additional data.

9. The unattended fully automatic UAV observation system according to claim 1, characterized in that, The blind spot detection module is based on the U-Net architecture.

10. The unattended fully automatic UAV observation system according to claim 1, characterized in that, The self-discovery and filling algorithm for observation blind spots includes sensor blind spots caused by fog, vegetation obstruction, or terrain depressions.

Citation Information

Patent Citations

  • Virtual power plant collaborative optimization scheduling method and system based on deep reinforcement learning

    CN119494521A

  • Systems and methods for a bayesian spatiotemporal graph transformer network for multi-aircraft trajectory prediction

    US20240054329A1

Cited By

  • A target-driven unmanned aerial vehicle adaptive observation route planning method

    CN122505272A