A method and system for defect screening by unmanned aerial vehicle inspection

Abstract

The application discloses a kind of unmanned aerial vehicle inspection defect screening method and system, by constructing observation state vector, the value of view point utility function is calculated.The calculation process includes obtaining geometric coverage score, texture definition score and motion cost.The above score is multiplied by preset weighting coefficient respectively, and then algebraic summation operation is carried out to obtain the value of view point utility function.The alignment feature set and defect category global prior feature prototype are obtained using dynamic fusion weight.The dot product value is calculated by querying mapping function and key-value mapping function, and the dynamic fusion weight is obtained by exponential operation and normalization processing.Based on the comparison result of defect determination probability and preset high determination threshold and preset low determination threshold, the hazard level is determined as confirmed defect, suspected hazard or normal.The application realizes autonomous view point selection under multi-objective optimization, considers detection quality and operation efficiency, effectively reduces the miss rate and false alarm rate, and improves the intelligent level of inspection.

Description

Technical Field

[0001] This invention belongs to the field of unmanned aerial vehicle (UAV) intelligent inspection and computer vision technology, and relates to a method and system for screening defects in UAV inspection. Background Technology

[0002] Unmanned aerial vehicle (UAV) inspection technology has been widely applied in the detection of defects in power lines, oil pipelines, and infrastructure. Existing detection methods largely rely on manual review of video footage or automated screening based on simple image recognition algorithms. Traditional manual methods are inefficient, susceptible to subjective biases, and struggle to handle massive amounts of inspection data. Early automated detection algorithms typically processed the entire image at a fixed resolution, resulting in inefficient allocation of computational resources. In scenarios with complex backgrounds or small targets, these algorithms are prone to missing detections or generating numerous false alarms.

[0003] Existing technologies often lack effective multi-view observation mechanisms when dealing with minor defects. Single-image captures frequently suffer from uneven lighting, angular obstruction, or motion blur, leading to insufficient feature extraction. Some solutions attempt multi-angle shooting via preset trajectories, but lack strategies for dynamically adjusting the observation viewpoint based on real-time detection results. This open-loop control approach cannot perform targeted re-examination of suspected defect areas, resulting in the loss of crucial information. Furthermore, in the multi-view image fusion process, existing technologies often employ simple averaging or max-pooling operations, failing to effectively distinguish the quality differences between images from different perspectives. Feature noise from low-quality perspectives often interferes with the final judgment result, reducing the accuracy of defect identification.

[0004] Existing screening systems typically set only a single threshold in the judgment stage, failing to distinguish between confirmed defects and suspected potential risks. This coarse-grained classification method results in a huge workload for subsequent manual review and makes it difficult to manage risk levels in a refined manner.

[0005] As the complexity of inspection tasks increases, there is an urgent need for a defect screening solution that can dynamically allocate computing resources, adaptively plan observation viewpoints, and accurately integrate multi-view features to solve the aforementioned technical problems. Summary of the Invention

[0006] To address the problems existing in the background technology, this invention proposes a method and system for defect screening during unmanned aerial vehicle (UAV) inspections.

[0007] The first aspect of this application provides a method for defect screening during unmanned aerial vehicle (UAV) inspections, including:

[0008] The system acquires real-time video streams during the flight of a drone, uses a pre-built lightweight attention-guided detection model to identify image regions in the real-time video stream that contain suspected defects, and outputs a set of candidate boxes containing location coordinates, preliminary categories, and initial confidence scores.

[0009] When the initial confidence level is greater than a preset trigger threshold, an observation state vector containing the gimbal angle and the relative position of the UAV is constructed. The viewpoint utility function value is calculated based on the geometric coverage score, texture clarity score, and motion cost. The optimal observation viewpoint sequence is generated based on the viewpoint utility function value. The motion cost is used to measure the energy or time consumed by the UAV to move from its current position to the candidate viewpoint.

[0010] The drone gimbal and flight controller acquire a set of multi-view images based on the optimal observation viewpoint sequence, and the multi-view image set is encoded using a shared weight feature extraction network to obtain a feature map set;

[0011] The feature map is mapped to a unified virtual frontal plane to obtain an aligned feature set. The spatiotemporal cross-attention fusion module is used to calculate dynamic fusion weights in combination with the global prior feature prototype of the defect category. Based on the dynamic fusion weights, the aligned feature set is weighted and summed to generate fusion features. The global prior feature prototype of the defect category is a set of feature vectors that are obtained in advance by extracting and clustering features from historical defect samples and are used to characterize the standard semantic information of various defects.

[0012] The fused features are input into the classification head network to output the defect determination probability. The defect determination probability is then compared with a preset high determination threshold and a preset low determination threshold to determine the hazard level.

[0013] Optionally, the pre-lightweight attention-guided detection model adopts a dynamic sparse window attention mechanism, which dynamically allocates computing resources according to the texture complexity of the input image, performs full-resolution feature extraction on regions with high texture complexity, and performs downsampling processing on background regions.

[0014] Optionally, the observation state vector includes the gimbal yaw angle, gimbal pitch angle, relative distance between the UAV and the target, azimuth angle of the UAV relative to the target, and altitude angle of the UAV relative to the target.

[0015] Optionally, the process of calculating the viewpoint utility function value includes: obtaining the geometric coverage score under the current observation state vector; obtaining the texture sharpness score under the current observation state vector; obtaining the motion cost under the current observation state vector; and performing algebraic summation on the geometric coverage score, the texture sharpness score, and the motion cost by multiplying them by a preset geometric coverage weighting coefficient and a preset texture sharpness weighting coefficient, respectively, to obtain the viewpoint utility function value.

[0016] Optionally, calculating the dynamic fusion weights includes:

[0017] The query vector is obtained by processing each feature in the aligned feature set using a query mapping function.

[0018] The key-value vector is obtained by processing the global prior feature prototype of the defect category using a key-value mapping function.

[0019] The dot product of the query vector and the key vector is calculated to obtain the dot product value; an exponentiation operation is performed on each dot product value to obtain the exponent value;

[0020] The sum of the exponent values ​​corresponding to all features in the alignment feature set is calculated to obtain the exponent sum;

[0021] The dynamic fusion weight is obtained by dividing each of the index values ​​by the sum of the indexes.

[0022] Optionally, determining the hazard level includes:

[0023] If the defect determination probability is greater than the preset high determination threshold, it is marked as a confirmed defect.

[0024] If the defect determination probability is less than or equal to the preset high determination threshold and greater than the preset low determination threshold, it is marked as a suspected hidden danger.

[0025] If the probability of defect determination is less than or equal to the preset low determination threshold, it is determined to be normal.

[0026] A second aspect of this application provides a drone inspection defect screening system, comprising:

[0027] The pre-detection module is used to acquire real-time video streams during the flight of the UAV, and uses a pre-lightweight attention-guided detection model to identify image regions containing suspected defects in the real-time video stream, and outputs a set of candidate boxes containing location coordinates, preliminary categories and initial confidence scores.

[0028] The viewpoint planning module is used to construct an observation state vector containing the gimbal angle and the relative position of the UAV when the initial confidence is greater than a preset trigger threshold, calculate the viewpoint utility function value based on the geometric coverage score, texture clarity score and motion cost, and generate the optimal observation viewpoint sequence according to the viewpoint utility function value;

[0029] The data acquisition module is used to control the UAV gimbal and flight controller to acquire a set of multi-view images according to the optimal observation viewpoint sequence, and to encode the set of multi-view images using a shared weight feature extraction network to obtain a feature map set;

[0030] The feature fusion module is used to map the feature map to a unified virtual frontal plane to obtain an aligned feature set. The spatiotemporal cross-attention fusion module is used to calculate dynamic fusion weights in combination with global prior feature prototypes of defect categories. Based on the dynamic fusion weights, the aligned feature set is weighted and summed to generate fused features.

[0031] The screening decision module is used to input the fused features into the classification head network to output the defect judgment probability, and compare the defect judgment probability with a preset high judgment threshold and a preset low judgment threshold to determine the hazard level.

[0032] Optionally, the pre-detection module integrates a dynamic sparse window attention mechanism unit, which is configured to dynamically adjust the size of the attention window according to the texture complexity of the input image and perform full-resolution calculation only on regions with high texture complexity.

[0033] Optionally, the viewpoint planning module is configured to solve for the maximum cumulative utility over multiple future steps, and the output optimal observation viewpoint sequence includes gimbal yaw angle commands, gimbal pitch angle commands, and UAV displacement commands arranged in chronological order.

[0034] Optionally, the spatiotemporal cross-attention fusion unit in the feature fusion module is configured to calculate the correlation degree between each feature in the aligned feature set and the global prior feature prototype of the defect category, generate dynamic fusion weights based on the correlation degree, and use the dynamic fusion weights to perform weighted processing on the aligned feature set.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] This invention provides a method and system for defect screening during UAV inspection. It employs a pre-built lightweight attention-guided detection model that dynamically allocates computational resources based on the texture complexity of the input image. This model performs full-resolution feature extraction on regions with high texture complexity and downsamples background regions. This mechanism effectively reduces the computational overhead of invalid backgrounds and significantly improves the detection efficiency of minute defects.

[0037] This invention constructs an observation state vector including the gimbal angle and the relative position of the UAV, and calculates the viewpoint utility function value based on geometric coverage score, texture clarity score, and motion cost. The system generates an optimal observation viewpoint sequence based on the viewpoint utility function value, realizing the transformation from open-loop preset trajectory to closed-loop dynamic programming. When the initial confidence level is greater than a preset trigger threshold, the UAV can automatically adjust its attitude to re-examine suspected areas from multiple angles. This process overcomes the problem of information loss caused by uneven lighting or angle occlusion in a single shot, ensuring the effective acquisition of key features.

[0038] This invention utilizes a spatiotemporal cross-attention fusion module combined with global prior feature prototypes of defect categories to calculate dynamic fusion weights. The system generates dynamic fusion weights based on the correlation between each feature in the aligned feature set and the global prior feature prototype of the defect category, and then performs a weighted summation on the aligned feature set. This method adaptively suppresses interference from low-quality viewpoint features and highlights the contribution of high-confidence viewpoints, thereby generating more robust fusion features. This solves the technical problem that traditional average pooling operations cannot distinguish differences in viewpoint quality, significantly improving the accuracy of defect identification.

[0039] This invention compares the probability of defect detection with preset high and low detection thresholds to determine the level of potential hazards. The system classifies hazards into three levels: confirmed defects, suspected hazards, and normal. This multi-level classification strategy enables refined management of inspection results, ensuring timely alarms for serious defects and providing clear guidance for manual review of suspected hazards, effectively reducing operation and maintenance costs and improving the intelligence level of the inspection system. Attached Figure Description

[0040] Figure 1 This is a flowchart of a method for screening defects in unmanned aerial vehicle (UAV) inspections according to an embodiment of the present invention;

[0041] Figure 2 This is a schematic diagram of a drone inspection defect screening system according to an embodiment of the present invention. Detailed Implementation

[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] In one embodiment, such as Figure 1 As shown, a method for defect screening in UAV inspection is provided, which is applied to... Figure 1 Taking China as an example, the following specific steps will be used:

[0044] S10: Acquire real-time video stream during the flight of the UAV, use a pre-built lightweight attention-guided detection model to identify image regions containing suspected defects in the real-time video stream, and output a set of candidate boxes containing location coordinates, preliminary category and initial confidence.

[0045] Specifically, in this invention, the step of acquiring real-time video streams during the drone's flight is achieved by continuously capturing image frames using an onboard high-definition camera. The camera outputs raw video data at a frequency of thirty frames per second and transmits the data stream to the onboard computing unit in real time. The computing unit preprocesses each frame of the image, including color space conversion and size normalization, to adapt to the input requirements of subsequent models.

[0046] The process of identifying image regions containing suspected defects using a pre-lightweight attention-guided detection model comprises two stages: feature extraction and attention allocation. The pre-lightweight attention-guided detection model first extracts global low-order features of the image through a shallow convolutional network and estimates the texture complexity of each local region. For regions with high texture complexity, the model dynamically allocates larger computational weights while preserving full-resolution information. For background regions with simple textures, the model performs downsampling to reduce the number of parameters. This dynamic sparse window attention mechanism ensures that computational resources are concentrated on regions that may contain defects.

[0047] The model then uses the weighted feature map for object detection, processing it in parallel through anchor box regression and classification branches. The regression branch predicts the position coordinates of candidate boxes, including the x-coordinate and y-coordinate of the center point, as well as width and height. The classification branch outputs preliminary category labels and corresponding initial confidence scores. The preliminary category labels cover specific defect types such as insulator damage, conductor strand breakage, and hardware corrosion. The initial confidence score represents the model's assessment of the reliability of the current detection results, and its value ranges from zero to one.

[0048] The final output set of candidate boxes consists of multiple structured data units, each containing three elements: location coordinates, preliminary category, and initial confidence level. This set serves as the input for the subsequent viewpoint planning module. When the initial confidence level of a candidate box exceeds a preset trigger threshold, the system will initiate a dynamic observation process.

[0049] The technical advantage of this embodiment lies in its significantly improved balance between detection speed and accuracy. By dynamically allocating computing resources, the model maintains high detection accuracy while drastically reducing computational latency. Full-resolution feature extraction ensures that details of even minute defects are not lost, while downsampling of the background area effectively avoids wasting computing power. This mechanism enables the UAV to perform defect screening in complex backgrounds in real time while flying at high speed, providing a reliable data foundation for the subsequent generation of the optimal observation viewpoint sequence.

[0050] In one embodiment, in step S10, the pre-lightweight attention-guided detection model adopts a dynamic sparse window attention mechanism, dynamically allocates computing resources according to the texture complexity of the input image, performs full-resolution feature extraction on regions with high texture complexity, and performs downsampling processing on background regions.

[0051] Specifically, the pre-processor lightweight attention-guided detection model integrates a dynamic sparse window attention mechanism, which is a core component for achieving efficient inference. After receiving the input image, the system first performs a global scan of the image through a lightweight texture evaluation sub-network. This sub-network calculates the gradient variance or frequency domain energy distribution of local regions of the image to quantify the texture complexity of each region. Texture complexity is an indicator of the richness of image detail; more complex textures mean that the region contains more edges, corners, or fine structures.

[0052] Based on the calculated texture complexity, the system dynamically allocates computational resources. The system divides the input image into several non-overlapping candidate windows. For candidate windows with texture complexity exceeding a preset complexity threshold, the system determines them as image regions containing suspected defects. These regions typically contain suspected defects, device edges, or critical components. The system enables full-resolution feature extraction for these high-texture-complexity regions. In this mode, the dynamic sparse window attention mechanism maintains the original pixel resolution and does not perform any spatial downsampling operations. The attention head performs query, key-value mapping calculations for each pixel within the window, ensuring that subtle defect features are not lost. This approach guarantees that the model can capture crucial information such as tiny cracks, rust, or foreign objects.

[0053] For candidate windows with texture complexity below a preset complexity threshold, the system classifies them as background regions. These regions typically resemble skies, grass, or uniformly colored walls, lacking effective discriminative features. The system performs downsampling on these background regions. Specifically, the system reduces the resolution of the feature maps of the background regions through pooling operations or strided convolutions. For example, the feature map size of the background regions is reduced to half or a quarter of its original size. On the downsampled low-resolution feature map, the dynamic sparse window attention mechanism only performs attention calculations on sparse keypoints. This strategy significantly reduces the computational cost and memory usage of background regions, allowing the model to concentrate limited computing power on high-value, textured, complex regions.

[0054] The technical advantage of this embodiment lies in significantly improving the balance between real-time performance and detection accuracy of the UAV-borne detection model. By employing a dynamic sparse window attention mechanism, the model can adaptively perceive image content, breaking away from the traditional convolutional neural network's indiscriminate computational model that treats all regions equally. Full-resolution feature extraction is performed on regions with high texture complexity, effectively avoiding the blurring or disappearance of minute defect features caused by downsampling, greatly improving the detection rate of small target defects. Downsampling of background regions significantly reduces the model's floating-point operations and memory bandwidth requirements without affecting overall detection performance. This differentiated resource allocation strategy enables the lightweight model to run smoothly on computationally limited embedded devices, extending the single-operation time of the UAV. Simultaneously, this mechanism enhances the model's resistance to interference from complex backgrounds, reducing the probability of misjudging background noise as defects and ensuring the efficient and accurate execution of inspection tasks.

[0055] S20: When the initial confidence level is greater than the preset trigger threshold, construct an observation state vector containing the gimbal angle and the relative position of the UAV, calculate the viewpoint utility function value based on the geometric coverage score, texture clarity score and motion cost, and generate the optimal observation viewpoint sequence according to the viewpoint utility function value.

[0056] Specifically, when the initial confidence level of a detected candidate bounding box exceeds a preset trigger threshold, the system determines that a suspected defect exists in the current area and then initiates a dynamic viewpoint planning process. The system first constructs an observation state vector, which collects and integrates in real-time gimbal yaw angle, gimbal pitch angle, relative distance between the UAV and the target, UAV azimuth angle relative to the target, and UAV altitude angle relative to the target. These parameters together constitute a complete state space describing the UAV's current observation attitude, providing precise geometric basis for subsequent decision-making.

[0057] The system calculates the viewpoint utility function value based on the observed state vector, taking into account three key indicators. The first indicator is the geometric coverage score, which assesses the proportion of the projected area of ​​the defect region in the image from the current viewpoint, ensuring the target subject is fully presented. The second indicator is the texture sharpness score, which quantifies the clarity of the defect texture from the current viewpoint by analyzing local gradient information or frequency domain features of the image, avoiding motion blur or defocus effects. The third indicator is the motion cost, which measures the energy or time required for the UAV to move from its current position to the candidate viewpoint, typically proportional to the flight distance and gimbal rotation angle. The system multiplies the geometric coverage score by a preset geometric coverage weighting coefficient, the texture sharpness score by a preset texture sharpness weighting coefficient, and the motion cost by a preset motion cost weighting coefficient. Finally, it performs an algebraic summation of the three values ​​to obtain the final viewpoint utility function value.

[0058] The system generates an optimal observation viewpoint sequence based on the calculated viewpoint utility function value. This process employs a heuristic search algorithm or model predictive control strategy, traversing possible combinations of gimbal angle commands and UAV displacement commands within multiple future time windows. The algorithm aims to find a trajectory path that maximizes the cumulative viewpoint utility function value. The output optimal observation viewpoint sequence contains a series of control commands arranged chronologically, specifically including gimbal yaw angle commands, gimbal pitch angle commands, and UAV displacement commands in three-dimensional space for each time step. The UAV flight controller and gimbal controller coordinate their actions according to this sequence, automatically adjusting the flight attitude to obtain the optimal observation angle.

[0059] The technical advantage of this embodiment lies in achieving closed-loop adaptive control of the inspection process. By constructing a multi-dimensional observation state vector, the system can accurately perceive its relative pose to the target, overcoming the shortcomings of traditional fixed-track inspection in dealing with unexpected situations. The introduction of a comprehensive evaluation mechanism combining geometric coverage score, texture clarity score, and motion cost enables the system to optimize flight energy consumption while ensuring image quality, avoiding resource waste caused by blind shooting. The generated optimal observation viewpoint sequence guides the UAV to actively avoid obstructions and eliminate illumination blind spots, ensuring the acquisition of high-quality defect images from multiple optimal angles. This dynamic programming strategy significantly improves the reproducibility rate of minute defects in complex environments, providing valuable raw data for subsequent multi-view feature fusion, thereby effectively reducing the risk of missed detections.

[0060] In one embodiment, in step S20, the observation state vector includes the gimbal yaw angle, the gimbal pitch angle, the relative distance between the UAV and the target, the UAV's azimuth angle relative to the target, and the UAV's altitude angle relative to the target.

[0061] Specifically, the observation state vector is a core set of data characterizing the relative spatial attitude and positional relationship between the UAV and the target. This vector consists of five key physical quantities: gimbal yaw angle, gimbal pitch angle, relative distance between the UAV and the target, UAV azimuth angle relative to the target, and UAV altitude angle relative to the target. These five components together constitute the complete state input for the reinforcement learning agent to perceive the environment, guiding the UAV to adjust its flight attitude to obtain the optimal observation perspective.

[0062] The gimbal yaw angle refers to the angle at which the gimbal lens mounted on the UAV rotates around its vertical axis. This angle defines the lens's orientation in the horizontal direction, typically with the UAV's longitudinal axis or true north as the reference zero degree. The system reads this angle value in real time through an encoder inside the gimbal, normalizes it, and fills it into the first dimension of the observation state vector. The gimbal pitch angle refers to the angle at which the gimbal lens rotates around its horizontal axis. This angle defines the lens's tilt in the vertical direction; a positive value indicates the lens is tilted upwards, and a negative value indicates the lens is tilted downwards. The system also obtains this real-time angle through a sensor, serving as the second dimension of the observation state vector. These two angles directly determine the line-of-sight direction when the image sensor acquires the image, ensuring that the lens is always pointed at the target area to be detected.

[0063] The relative distance between the UAV and the target refers to the straight-line Euclidean distance between the UAV's current position and the center point of the target defect. The system calculates this distance value in real time using an airborne LiDAR or binocular vision ranging module, with the unit being meters. This distance information reflects the proximity of the UAV to the target and is a key basis for determining whether the shooting resolution meets the detection requirements. The UAV's azimuth angle relative to the target refers to the horizontal angle between the UAV and the target's front or geographic north when viewed from the target point. This angle describes the UAV's position distribution around the target on the horizontal plane, helping the agent determine whether all sides of the target have been covered. The UAV's elevation angle relative to the target refers to the angle between the line of sight and the horizontal plane when viewed from the target point. This angle describes the UAV's vertical position relative to the target and is used to assess whether there is a risk of overhead or upward occlusion. These three spatial position parameters together construct a three-dimensional spherical coordinate system description of the UAV relative to the target.

[0064] The system arranges the five physical quantities in a fixed order to form a multidimensional floating-point array, i.e., the observation state vector. At each control time step, the flight control system updates the values ​​in this vector and inputs them into the policy network. The policy network outputs the next action command based on the current observation state vector, including the gimbal rotation speed and the UAV flight speed. By continuously monitoring the changes in the observation state vector, the system can dynamically adjust the UAV's flight trajectory and gimbal angle, gradually bringing it closer to the preset optimal observation viewpoint.

[0065] The technical advantage of this embodiment lies in providing precise and comprehensive state perception capabilities for autonomous UAV inspection. By including gimbal yaw and pitch angles, the system can accurately determine the lens's direction, avoiding blind spots caused by gimbal limitations or mechanical errors. Introducing the relative distance between the UAV and the target, the UAV's azimuth angle relative to the target, and the UAV's altitude angle relative to the target allows the agent to fully understand its position in three-dimensional space, thereby planning a collision-free and comprehensive flight path. This high-dimensional state representation method enhances the convergence speed of reinforcement learning algorithms, enabling the UAV to quickly learn the optimal observation strategy for targets of different shapes and positions. Simultaneously, explicit spatial geometric parameters help eliminate ambiguity in the state space, improve the accuracy of control commands, ensure stability and consistency during multi-view image acquisition, and lay a solid data foundation for subsequent high-precision defect identification.

[0066] In one embodiment, step S20, i.e., the process of calculating the viewpoint utility function value, includes:

[0067] S21: Obtain the geometric coverage score under the current observation state vector.

[0068] S22: Obtain the texture sharpness score under the current observation state vector.

[0069] S23: Obtain the motion cost under the current observed state vector.

[0070] S24: Multiply the geometric coverage score by a preset geometric coverage weighting coefficient, the texture sharpness score by a preset texture sharpness weighting coefficient, and the motion cost by a preset motion cost weighting coefficient, and then perform an algebraic summation operation to obtain the viewpoint utility function value.

[0071] Specifically, the process of obtaining the viewpoint utility function value in this invention is a key step in transforming multi-dimensional observation indicators into a single evaluation scalar. This process first involves obtaining the geometric coverage score under the current observation state vector. The system constructs a three-dimensional spatial projection model based on the current gimbal yaw angle, gimbal pitch angle, relative distance between the UAV and the target, UAV azimuth angle relative to the target, and UAV altitude angle relative to the target. This model calculates the ratio of the effective projected area of ​​the target region on the camera imaging plane to the actual surface area of ​​the target. When the UAV is at its optimal viewing angle, capable of completely covering the target without obstruction, this ratio approaches 1, and the corresponding geometric coverage score reaches its maximum. If part of the target region exceeds the field of view or is obstructed by its own structure, this ratio decreases, and the geometric coverage score decreases accordingly. This score directly reflects the completeness of the current viewpoint's coverage of the detected object at the spatial geometric level.

[0072] Next, the system executes the operation of obtaining the texture sharpness score under the current observation state vector. The system calls the onboard image processing module to perform frequency domain analysis or gradient calculation on the image data acquired at the current moment. By extracting high-frequency component information from the image or calculating the variance of the Laplacian operator, the system quantifies the sharpness of the image. When the UAV flies smoothly, is accurately focused, and has good lighting conditions, the image is rich in detail and has clear edges, resulting in a high texture sharpness score. Conversely, if motion blur occurs due to the high-speed movement of the UAV, or if the resolution is insufficient due to excessive distance, high-frequency information in the image is lost, and the texture sharpness score is significantly reduced. This score directly represents the amount of effective information in the image acquired at the current viewpoint for subsequent defect identification, ensuring that only high-quality images can contribute a high utility value.

[0073] Subsequently, the system executes the step of obtaining the motion cost under the current observation state vector. Motion cost measures the resources consumed by the UAV in moving from the previous pose to the current observation pose. The system calculates the spatial displacement between the current position and the previous position, as well as the change in gimbal angle. If the UAV needs to perform long-distance flight or large-angle turns to reach the current viewpoint, it is considered high-energy and high-time consumption, resulting in a large motion cost. If the current viewpoint is adjacent to the previous viewpoint and the transition is smooth, the motion cost is small. Introducing motion cost aims to suppress frequent oscillations or meaningless maneuvers by the UAV, encouraging the system to select the shortest path and lowest-energy observation sequence while meeting detection quality requirements.

[0074] After acquiring the three independent components, the system performs a weighted fusion operation to obtain the viewpoint utility function value. The system reads preset geometric coverage weighting coefficients, preset texture sharpness weighting coefficients, and preset motion cost weighting coefficients pre-stored in the controller. These three coefficients define the relative importance of geometric integrity, image quality, and motion efficiency in the overall evaluation system, respectively. The system multiplies the geometric coverage score by the preset geometric coverage weighting coefficient to generate the geometric contribution value. The system multiplies the texture sharpness score by the preset texture sharpness weighting coefficient to generate the texture contribution value. The system multiplies the motion cost by the preset motion cost weighting coefficient to generate the cost contribution value. Since motion cost is a negative indicator, the preset motion cost weighting coefficient is usually set to a negative value or treated as a subtraction term in subsequent calculations. Finally, the system performs an algebraic summation operation on the geometric contribution value, texture contribution value, and cost contribution value. This summation result is the final viewpoint utility function value. The larger this value, the better the current viewpoint is in terms of coverage, clarity, and cost-effectiveness, and it is the core basis for the reinforcement learning agent to choose the next action.

[0075] The technical advantage of this embodiment lies in achieving autonomous optimal viewpoint planning under multi-objective constraints. By constructing a composite utility function that includes geometric coverage score, texture sharpness score, and motion cost, the system can simultaneously consider detection coverage, image recognition rate, and operational energy efficiency. The introduction of geometric coverage score ensures that the UAV can automatically search for and lock onto the best observation angle without blind spots, effectively avoiding the risk of missed defects due to a single viewpoint. The introduction of texture sharpness score gives the system real-time perception of image quality, enabling it to actively avoid blurry or low-resolution shooting positions, ensuring the high reliability of input data for subsequent defect recognition algorithms. The introduction of motion cost effectively smooths the flight trajectory, reduces unnecessary energy consumption and time waste, and extends the UAV's single-operation endurance. By unifying multi-dimensional heterogeneous indicators into a single viewpoint utility function value through algebraic summation operations, the design complexity of the reinforcement learning reward function is greatly simplified, accelerating the training and convergence process of the policy network. This method enables the UAV to autonomously plan a clear, comprehensive, energy-efficient inspection trajectory in complex dynamic environments, significantly improving the overall operational efficiency and adaptability of the intelligent inspection system.

[0076] S30: Control the UAV gimbal and flight controller to acquire a set of multi-view images based on the optimal observation viewpoint sequence, and use a shared weight feature extraction network to encode the set of multi-view images to obtain a feature map set.

[0077] Specifically, the step of controlling the UAV gimbal and flight controller to acquire a multi-view image set based on the optimal observation viewpoint sequence is achieved through the coordinated operation of the airborne flight control system and the gimbal servo system. The system reads each control command from the optimal observation viewpoint sequence, which includes the gimbal yaw angle, gimbal pitch angle, and the UAV's three-dimensional spatial displacement at a specific time step. The flight control system drives the UAV to the designated coordinate position, while the gimbal servo motor adjusts the lens orientation to the designated angle. Once the UAV's attitude is stable and the gimbal reaches the target angle, the image sensor triggers the shutter to capture a single high-resolution image. The system executes all commands in the sequence sequentially, thereby acquiring a set of images of the same suspected defect area taken from different spatial positions and angles; this set of images together constitutes a multi-view image set.

[0078] The process of encoding a multi-view image set using a shared-weight feature extraction network to obtain a feature map aims to extract deep features with semantic consistency. The shared-weight feature extraction network employs a convolutional neural network architecture, where the parameters of all convolutional layers remain fixed throughout the multi-view image processing. The system inputs each image from the multi-view image set sequentially into the network or in batches in parallel. Due to the shared network weights, the network uses the same set of feature extraction operators regardless of the input image's perspective. The network gradually abstracts the texture, edge, and shape information of the image through multi-layer convolution operations, non-linear activation, and pooling operations. For each image in the multi-view image set, the network ultimately outputs a high-dimensional feature matrix, which represents the deep semantic information of the defect region under the corresponding perspective. The feature matrices corresponding to all perspectives together form the feature map set, where each feature matrix corresponds one-to-one with an image in the input multi-view image set.

[0079] The technical advantage of this embodiment lies in ensuring the alignment and comparability of multi-view features in the semantic space. By employing a feature extraction network with shared weights, the feature distribution bias caused by differences in network parameters is eliminated, ensuring that features extracted from different perspectives reflect the essential attributes of the same target rather than perspective-specific noise. This processing method provides standardized input data for the subsequent spatiotemporal cross-attention fusion module, enabling the system to accurately calculate the correlation between features from different perspectives. Simultaneously, the shared weight mechanism significantly reduces the number of model parameters, lowers the memory footprint and inference latency of the onboard computing unit, and allows the UAV to complete multi-view image encoding processing in real time during flight. This step effectively overcomes the fusion difficulties caused by inconsistent feature spaces in traditional multi-model processing schemes, improves the robustness and accuracy of multi-view information fusion, and lays a solid data foundation for generating high-precision defect judgment results.

[0080] S40: Map the feature map set to a unified virtual frontal plane to obtain an aligned feature set. Calculate dynamic fusion weights using a spatiotemporal cross-attention fusion module combined with global prior feature prototypes of defect categories. Generate fused features by weighted summation of the aligned feature set based on the dynamic fusion weights.

[0081] Specifically, the process of mapping the feature map set to a unified virtual frontal view plane to obtain an aligned feature set aims to eliminate geometric distortions caused by different shooting angles in multi-view images. The system first calculates a homography transformation matrix based on the gimbal angle and the relative position of the UAV recorded when acquiring each image. This matrix describes the geometric mapping relationship between the current view plane and the preset unified virtual frontal view plane. The unified virtual frontal view plane is a standard reference plane perpendicular to the normal of the target defect surface. The system uses the homography transformation matrix to perform spatial resampling on each feature matrix in the feature map set. Through bilinear interpolation, the coordinates of feature points under the original view are mapped to the corresponding coordinates on the unified virtual frontal view plane. After this processing, the feature textures that were originally stretched or compressed due to view tilt are corrected to a frontal view shape. All geometrically corrected feature matrices together constitute the aligned feature set, in which the features are strictly aligned in spatial position, eliminating spatial misalignment caused by view differences.

[0082] The system employs a spatiotemporal cross-attention fusion module to calculate dynamic fusion weights in conjunction with global prior feature prototypes for defect categories, enabling adaptive selection of features from multiple perspectives. The system pre-builds and stores global prior feature prototypes for defect categories. These prototypes are standard feature vectors of various defects trained on a large number of historical defect samples, representing the core semantic information of each defect type under ideal conditions. The spatiotemporal cross-attention fusion module performs similarity matching between each feature vector in the aligned feature set and the global prior feature prototype. The module calculates an attention score between each perspective feature vector and the global prior feature prototype, reflecting the degree of fit between the current perspective feature and the standard defect pattern. For perspectives with clear texture and no occlusion, the correlation between their features and the global prior feature prototype is high, resulting in a larger attention score. For low-quality perspectives with motion blur, insufficient lighting, or partial occlusion, the correlation between their features and the global prior feature prototype is low, resulting in a smaller attention score. The system uses the normalized attention score as the dynamic fusion weight, which reflects the contribution of each perspective to the final judgment result in real time.

[0083] The process of generating fused features by weighting and summing the aligned feature set based on dynamic fusion weights effectively integrates multi-source information. The system iterates through each feature vector in the aligned feature set and multiplies it by the corresponding dynamic fusion weight. Then, the system performs element-wise summation of all weighted feature vectors to generate a single fused feature vector. This fused feature vector aggregates effective information from all perspectives while suppressing noise interference introduced by low-quality perspectives. Because the dynamic fusion weights are adaptively generated based on feature quality, the fused feature highlights detailed information from high-confidence perspectives while retaining key defect-discriminating features. The generated fused feature is then fed into a classifier for final hazard level determination.

[0084] The technical advantage of this embodiment lies in its significantly improved robustness and accuracy of multi-view feature fusion. By mapping the feature map set to a unified virtual frontal plane, the system effectively overcomes the inconsistency in feature space caused by the varying angles of drone shooting, enabling features from different perspectives to be compared and fused under the same geometric benchmark. Introducing a global prior feature prototype of the defect category to guide the calculation of dynamic fusion weights gives the system the ability to identify viewpoint quality. This method can automatically distinguish between high-quality and low-quality views, avoiding the problem of low-quality features diluting high-quality features in traditional average pooling methods. The generated fused features have stronger semantic expressive power and anti-interference ability, accurately restoring the essential features of defects even when some views are severely occluded or blurred. This mechanism significantly improves the accuracy of defect identification in complex environments, reduces the risk of misjudgment due to missing information from a single perspective, and provides reliable data support for subsequent refined hazard classification.

[0085] In one embodiment, step S40, i.e., calculating the dynamic fusion weights, further includes the following steps:

[0086] S41: Process each feature in the aligned feature set using the query mapping function to obtain the query vector;

[0087] S42: Use the key-value mapping function to process the global prior feature prototype of the defect category to obtain the key-value vector;

[0088] S43: Calculate the dot product of the query vector and the key value vector to obtain the dot product value;

[0089] S44: Perform an exponential operation on each of the dot product values ​​to obtain the exponential value;

[0090] S45: Calculate the sum of the exponent values ​​corresponding to all features in the alignment feature set to obtain the exponent sum;

[0091] S46: Divide each of the index values ​​by the sum of the indices to obtain the dynamic fusion weight.

[0092] Specifically, the process of calculating dynamic fusion weights in this invention is based on an adaptive feature selection and weighting strategy implemented using an attention mechanism. This process aims to dynamically adjust the importance of each feature channel in the alignment feature set according to the specific content of the current input image, ensuring optimal matching with the global prior feature prototype of the defect category. First, the system processes each feature in the alignment feature set using a query mapping function. The alignment feature set contains multi-scale feature maps extracted from the input image and spatially aligned. The query mapping function typically consists of a convolutional layer or a fully connected layer, which linearly projects the high-dimensional alignment features onto a specific query subspace. After processing by this function, each spatial location or channel feature in the alignment feature set is transformed into a query vector. This query vector contains semantic information about the current local image region, representing the content currently "focused" on by the model.

[0093] Subsequently, the system uses a key-value mapping function to process the global prior feature prototypes of the defect categories to obtain key-value vectors. The global prior feature prototypes of the defect categories are a set of vectors pre-trained on a large-scale defect dataset, representing the essential features of various typical defects. These prototype vectors store common patterns for different defect categories, such as the linear texture of cracks or the mottled color of rust. The key-value mapping function also consists of a learnable linear transformation layer, which projects the fixed global prior feature prototypes of the defect categories onto a key-value subspace with the same dimension as the query vector, generating the corresponding key-value vector. This step ensures that the query vector and the key-value vector are similarly measured within the same feature space, laying the foundation for subsequent matching calculations.

[0094] Next, the system calculates the dot product of the query vector and the key-value vector to obtain the dot product value. The dot product operation is an effective mathematical tool for measuring the consistency and similarity of the directions of two vectors. The system performs element-wise multiplication of each query vector with each key-value vector and sums the results. The output dot product value characterizes the degree of matching between the current local features of the image and a specific defect prototype. The larger the dot product value, the higher the similarity between the current alignment feature and a certain type of defect prototype, and the greater the probability that the region belongs to that type of defect. Conversely, the smaller the dot product value, the weaker the correlation between the two. In this way, the model can initially identify which regions in the image are highly correlated with known defect patterns.

[0095] To transform the dot product values ​​into weights in a probability distribution form, the system performs an exponential operation on each dot product value to obtain an exponential value. The exponential function amplifies differences, further amplifying larger dot product values ​​while compressing smaller ones closer to zero. This step enhances the discriminative power of salient features, allowing high-matching features to dominate in subsequent fusion, while effectively suppressing low-matching background noise or irrelevant features. After the exponential operation, all matching scores are converted to positive numbers, preparing for normalization processing.

[0096] Subsequently, the system calculates the sum of the exponent values ​​corresponding to all features in the alignment feature set to obtain the exponent sum. The system iterates through all positions or channels in the alignment feature set, accumulating all the exponent values ​​generated in the previous steps. This exponent sum serves as a normalization factor to balance the numerical scale differences between different samples or different feature maps, ensuring that the final weight sum is always 1. This global normalization strategy guarantees the relativity and stability of the weight allocation, avoiding weight imbalance caused by individual extreme values.

[0097] Finally, the system divides each index value by the sum of the indices to obtain the dynamic fusion weight. Through this division operation, the index value at each position is converted into a scaling factor between 0 and 1. These factors are the dynamic fusion weights, forming a probability distribution map. For regions that highly match the global prior feature prototype of the defect category, the dynamic fusion weight is close to 1; for background or irrelevant regions, the dynamic fusion weight is close to 0. The system uses these dynamic fusion weights to perform a weighted summation of the original aligned feature set, thereby generating the final fused feature. This process enables the model to automatically focus on key defect regions while ignoring invalid background information, providing an intelligent filtering function.

[0098] The technical advantage of this embodiment lies in its significantly improved targeting and robustness of feature fusion. By utilizing an attention mechanism built with query mapping and key-value mapping functions, the system can dynamically adjust weights based on the real-time content of the input image, rather than using fixed static weights. This dynamism allows the model to adapt to changes in defects under different lighting conditions, angles, and shapes, effectively solving the problem of missed detections caused by fixed templates in traditional methods. Calculating the dot product of the query vector and key-value vector, followed by exponential operations and normalization, greatly enhances the model's sensitivity to subtle defect features, enabling it to accurately extract high-value defect information from complex background noise. Introducing global prior feature prototypes of defect categories as keys is equivalent to injecting prior knowledge into the model, allowing it to maintain high recognition accuracy even in scenarios with scarce samples or small target detection. The resulting dynamic fusion weights not only optimize the quality of feature representation but also accelerate the convergence speed of subsequent classification networks, improving the intelligence and generalization ability of the entire defect detection system and ensuring stable and reliable detection results under various complex conditions.

[0099] S50: Input the fused features into the classification head network to output the defect determination probability, and compare the defect determination probability with the preset high determination threshold and the preset low determination threshold to determine the hidden danger level.

[0100] Specifically, the step of inputting the fused features into the classification head network to output the defect determination probability is achieved through the collaborative work of a fully connected layer and an activation function. The classification head network receives the fused feature vector generated in the previous step as input data. This network contains at least one fully connected layer, used to map the high-dimensional fused features to the spatial dimension of the defect category. The output of the fully connected layer is connected to a normalized exponential function activation layer. This activation layer transforms the output value of the fully connected layer into a probability distribution. The output defect determination probability is a value between 0 and 1, representing the likelihood that the currently detected target belongs to a specific defect category. The closer the probability value is to 1, the higher the certainty that the model judges the target to have a defect. The closer the probability value is to 0, the higher the certainty that the model judges the target to be a normal background or without defects.

[0101] The process of comparing the defect determination probability with preset high and low determination thresholds to determine the hazard level constructs a three-level decision logic. The system first reads two fixed parameters: the preset high and low determination thresholds. The value of the preset high determination threshold is greater than the value of the preset low determination threshold. The system then performs a first comparison between the output defect determination probability and the preset high determination threshold. If the defect determination probability is greater than or equal to the preset high determination threshold, the system determines the current target's hazard level as a serious hazard. This level indicates that the defect characteristics are significant and the confidence level is extremely high, requiring immediate repair. If the defect determination probability is less than the preset high determination threshold, the system performs a second comparison, comparing the defect determination probability with the preset low determination threshold. If the defect determination probability is less than the preset high determination threshold but greater than or equal to the preset low determination threshold, the system determines the current target's hazard level as a general hazard. This level indicates that there are suspected defect characteristics but the confidence level is in the intermediate range, requiring inclusion in the key monitoring plan or manual review. If the defect determination probability is less than the preset low determination threshold, the system determines the current target's hazard level as no hazard. This level indicates that no obvious defect characteristics were found in the target area, and the system marks it as normal and archives it.

[0102] The technical advantage of this embodiment lies in achieving hierarchical management and risk quantification for defect assessment. By introducing a dual-threshold comparison mechanism, the system can effectively distinguish between highly certain severe defects and uncertain suspected defects, avoiding missed detections or false alarms caused by single-threshold assessments. For serious hidden dangers, the system can trigger immediate alarms, ensuring that high-risk defects are prioritized for handling, thereby guaranteeing the safe operation of the power grid. For general hidden dangers, the system marks them as pending review, preserving potential risk information while avoiding the waste of maintenance resources caused by overreaction. This hierarchical strategy makes inspection results more refined, facilitating maintenance personnel to formulate differentiated handling plans based on the level of hidden dangers. At the same time, the defect assessment probability based on fused feature output has high stability and can effectively resist interference from changes in lighting and background noise, ensuring the reliability and consistency of hidden danger level assessment results, and significantly improving the practical value and intelligence level of the UAV autonomous inspection system.

[0103] In one embodiment, step S50, i.e., determining the hazard level, further includes the following steps:

[0104] S51: If the defect determination probability is greater than the preset high determination threshold, it is marked as a confirmed defect;

[0105] S52: If the defect determination probability is less than or equal to the preset high determination threshold and greater than the preset low determination threshold, it is marked as a suspected hidden danger;

[0106] S53: If the defect determination probability is less than or equal to the preset low determination threshold, it is determined to be normal.

[0107] Specifically, the process of determining the hazard level in this invention is a logical judgment flow based on the comparison of defect judgment probability and multi-level thresholds. This process aims to transform the continuous probability values ​​output by the model into discrete level labels with clear business meaning, so as to take differentiated handling measures subsequently. The system first obtains the defect judgment probability output by the defect detection network. This probability value is a floating-point number between 0 and 1, representing the likelihood of a specific type of defect existing in the current detection area. The closer the value is to 1, the higher the confidence level of the presence of a defect; the closer the value is to 0, the greater the likelihood that the area is a normal background.

[0108] The system compares the defect determination probability with a preset high determination threshold. This preset high determination threshold is a high-standard critical value set through extensive experimental verification, representing an extremely high confidence level. If the defect determination probability is greater than the preset high determination threshold, the system marks the detected area as a confirmed defect. This means the model confirms the existence of a real defect in the area with a very high degree of confidence, and can directly generate a repair work order or alarm signal without requiring secondary manual verification. Such markings typically correspond to severe defects with obvious features, typical morphology, and no background interference, such as obvious structural fractures or large-area corrosion.

[0109] If the defect determination probability does not exceed a preset high determination threshold, the system will continue to compare it with a preset low determination threshold. The preset low determination threshold is a critical value lower than the preset high determination threshold, used to distinguish between obvious normal areas and ambiguous suspicious areas. If the defect determination probability is less than or equal to the preset high determination threshold but greater than the preset low determination threshold, the system marks the detected area as a suspected hidden danger. This level indicates that the model has detected some abnormal features, but the significance of these features is insufficient to meet the diagnostic criteria, or the features are in the transition zone between the defect and the background. Areas marked as suspected hidden dangers will be pushed to a manual review terminal for final confirmation by professional inspectors, or a drone will be triggered for close-up re-photographing to obtain more detailed information. This mechanism effectively balances the missed detection rate and the false alarm rate, avoiding the omission of potential risks due to excessively high threshold settings, and also preventing a massive number of invalid alarms due to excessively low thresholds.

[0110] If the defect determination probability is less than or equal to the preset low determination threshold, the system classifies the detected area as normal. This indicates that the image features of the current area are highly consistent with the background model, and there are no identifiable defect signs. The system will ignore these normal areas, thus avoiding the consumption of subsequent storage resources and manual review efforts, thereby significantly reducing the data processing load. By dividing the system into these two thresholds, the system clearly divides the continuous probability space into three mutually exclusive intervals: confirmed defects, suspected hidden dangers, and normal, achieving a smooth mapping from algorithm output to business decisions.

[0111] The technical advantage of this embodiment lies in constructing a hierarchical intelligent inspection decision-making mechanism. By introducing a dual threshold strategy with preset high and low judgment thresholds, the system can automatically match different processing procedures based on the confidence level of defect judgment. For confirmed defects, the system achieves immediate response and automated handling, greatly shortening the time cycle from defect discovery to maintenance initiation and effectively preventing the further expansion of potential hazards. For suspected hazards, the system adopts a human-machine collaborative review mode, utilizing both the efficient screening capabilities of machines and retaining the experience and judgment advantages of human experts, significantly reducing the false alarm interference that may be generated by a single automatic detection system and improving the credibility of the final report. For normal areas, the system's rapid filtering mechanism saves valuable communication bandwidth and storage space, making the management of inspection data more efficient and orderly. This hierarchical judgment method enhances the system's ability to handle uncertainty, enabling it to maintain a robust operating state even in the face of complex and ever-changing field environments, ensuring the safety and economy of power line or infrastructure inspection tasks.

[0112] In one embodiment, such as Figure 2 As shown, a UAV inspection defect screening system is provided, which corresponds one-to-one with the UAV inspection defect screening method in the above embodiments. The UAV inspection defect screening system includes: a pre-detection module, a viewpoint planning module, a data acquisition module, a feature fusion module, and a screening decision module. The detailed description of each functional module is as follows:

[0113] The pre-detection module is used to acquire real-time video streams during the flight of the UAV, and uses a pre-lightweight attention-guided detection model to identify image regions containing suspected defects in the real-time video stream, and outputs a set of candidate boxes containing location coordinates, preliminary categories and initial confidence scores.

[0114] The viewpoint planning module is used to construct an observation state vector containing the gimbal angle and the relative position of the UAV when the initial confidence is greater than a preset trigger threshold, calculate the viewpoint utility function value based on the geometric coverage score, texture clarity score and motion cost, and generate the optimal observation viewpoint sequence according to the viewpoint utility function value;

[0115] The data acquisition module is used to control the UAV gimbal and flight controller to acquire a set of multi-view images according to the optimal observation viewpoint sequence, and to encode the set of multi-view images using a shared weight feature extraction network to obtain a feature map set;

[0116] The feature fusion module is used to map the feature map to a unified virtual frontal plane to obtain an aligned feature set. The spatiotemporal cross-attention fusion module is used to calculate dynamic fusion weights in combination with global prior feature prototypes of defect categories. Based on the dynamic fusion weights, the aligned feature set is weighted and summed to generate fused features.

[0117] The screening decision module is used to input the fused features into the classification head network to output the defect judgment probability, and compare the defect judgment probability with a preset high judgment threshold and a preset low judgment threshold to determine the hazard level.

[0118] Furthermore, the pre-detection module integrates a dynamic sparse window attention mechanism unit, which is configured to dynamically adjust the size of the attention window according to the texture complexity of the input image, and perform full-resolution calculation only on regions with high texture complexity.

[0119] Furthermore, the viewpoint planning module is configured to solve for the maximum cumulative utility over multiple future steps, and the output optimal observation viewpoint sequence includes gimbal yaw angle commands, gimbal pitch angle commands, and UAV displacement commands arranged in chronological order.

[0120] Furthermore, the spatiotemporal cross-attention fusion unit in the feature fusion module is configured to calculate the correlation degree between each feature in the aligned feature set and the global prior feature prototype of the defect category, generate dynamic fusion weights based on the correlation degree, and use the dynamic fusion weights to perform weighted processing on the aligned feature set.

[0121] Specific limitations regarding the UAV inspection defect screening system can be found in the limitations of the UAV inspection defect screening method described above, and will not be repeated here. Each module in the aforementioned UAV inspection defect screening system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0122] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0123] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for defect screening during unmanned aerial vehicle (UAV) inspections, characterized in that, include: The system acquires real-time video streams during the flight of a drone, uses a pre-built lightweight attention-guided detection model to identify image regions in the real-time video stream that contain suspected defects, and outputs a set of candidate boxes containing location coordinates, preliminary categories, and initial confidence scores. When the initial confidence level is greater than a preset trigger threshold, an observation state vector containing the gimbal angle and the relative position of the UAV is constructed. The viewpoint utility function value is calculated based on the geometric coverage score, texture clarity score, and motion cost. The optimal observation viewpoint sequence is generated based on the viewpoint utility function value. The motion cost is used to measure the energy or time consumed by the UAV to move from its current position to the candidate viewpoint. The drone gimbal and flight controller acquire a set of multi-view images based on the optimal observation viewpoint sequence, and the multi-view image set is encoded using a shared weight feature extraction network to obtain a feature map set; The feature map is mapped to a unified virtual frontal plane to obtain an aligned feature set. The spatiotemporal cross-attention fusion module is used to calculate dynamic fusion weights in combination with global prior feature prototypes of defect categories. Based on the dynamic fusion weights, the aligned feature set is weighted and summed to generate fused features. The global prior feature prototype of the defect category is a set of feature vectors that are obtained in advance by extracting features and clustering historical defect samples, and are used to characterize the standard semantic information of various defects. The fused features are input into the classification head network to output the defect determination probability. The defect determination probability is then compared with a preset high determination threshold and a preset low determination threshold to determine the hazard level.

2. The method for screening defects in unmanned aerial vehicle (UAV) inspections according to claim 1, characterized in that, The aforementioned lightweight attention-guided detection model employs a dynamic sparse window attention mechanism, which dynamically allocates computational resources based on the texture complexity of the input image, extracts full-resolution features from regions with high texture complexity, and performs downsampling on background regions.

3. The method for screening defects in UAV inspections according to claim 1, characterized in that, The observation state vector includes the gimbal yaw angle, gimbal pitch angle, relative distance between the UAV and the target, azimuth angle of the UAV relative to the target, and altitude angle of the UAV relative to the target.

4. The method for screening defects in UAV inspections according to claim 1, characterized in that, The process of calculating the viewpoint utility function value includes: Obtain the geometric coverage score under the current observation state vector; Obtain the texture sharpness score under the current observation state vector; Obtain the motion cost under the current observed state vector; The viewpoint utility function value is obtained by multiplying the geometric coverage score by a preset geometric coverage weighting coefficient, the texture sharpness score by a preset texture sharpness weighting coefficient, and the motion cost by a preset motion cost weighting coefficient, and then performing an algebraic summation operation.

5. The method for defect screening during UAV inspection according to claim 1, characterized in that, Calculating the dynamic fusion weights includes: The query vector is obtained by processing each feature in the aligned feature set using a query mapping function. The key-value vector is obtained by processing the global prior feature prototype of the defect category using a key-value mapping function. The dot product of the query vector and the key vector is calculated to obtain the dot product value; The exponent value is obtained by performing an exponentiation operation on each of the dot product values; The sum of the exponent values ​​corresponding to all features in the alignment feature set is calculated to obtain the exponent sum; The dynamic fusion weight is obtained by dividing each of the index values ​​by the sum of the indexes.

6. The method for screening defects in unmanned aerial vehicle (UAV) inspections according to claim 1, characterized in that, Determining the level of the hazard includes: If the defect determination probability is greater than the preset high determination threshold, it is marked as a confirmed defect; If the defect determination probability is less than or equal to the preset high determination threshold and greater than the preset low determination threshold, it is marked as a suspected hidden danger; If the defect determination probability is less than or equal to the preset low determination threshold, it is determined to be normal.

7. A drone inspection defect screening system, characterized in that, include: The pre-detection module is used to acquire real-time video streams during the flight of the UAV, and uses a pre-lightweight attention-guided detection model to identify image regions containing suspected defects in the real-time video stream, and outputs a set of candidate boxes containing location coordinates, preliminary categories and initial confidence scores. The viewpoint planning module is used to construct an observation state vector containing the gimbal angle and the relative position of the UAV when the initial confidence is greater than a preset trigger threshold. It calculates the viewpoint utility function value based on the geometric coverage score, texture clarity score and motion cost, and generates the optimal observation viewpoint sequence according to the viewpoint utility function value. The motion cost is used to measure the energy or time consumed by the UAV to move from the current position to the candidate viewpoint. The data acquisition module is used to control the UAV gimbal and flight controller to acquire a set of multi-view images according to the optimal observation viewpoint sequence, and to encode the set of multi-view images using a shared weight feature extraction network to obtain a feature map set; The feature fusion module is used to map the feature map to a unified virtual frontal plane to obtain an aligned feature set. The spatiotemporal cross-attention fusion module is used to calculate dynamic fusion weights in combination with global prior feature prototypes of defect categories. Based on the dynamic fusion weights, the aligned feature set is weighted and summed to generate fused features. The global prior feature prototype of the defect category is a set of feature vectors that are obtained in advance by extracting features and clustering historical defect samples, and are used to characterize the standard semantic information of various defects. The screening decision module is used to input the fused features into the classification head network to output the defect judgment probability, and compare the defect judgment probability with a preset high judgment threshold and a preset low judgment threshold to determine the hazard level.

8. The UAV inspection defect screening system according to claim 7, characterized in that, The pre-detection module integrates a dynamic sparse window attention mechanism unit, which is configured to dynamically adjust the size of the attention window according to the texture complexity of the input image and perform full-resolution calculations only on regions with high texture complexity.

9. The UAV inspection defect screening system according to claim 7, characterized in that, The viewpoint planning module is configured to solve for the maximum cumulative utility over multiple future steps, and the output optimal observation viewpoint sequence includes gimbal yaw angle commands, gimbal pitch angle commands, and UAV displacement commands arranged in chronological order.

10. The UAV inspection defect screening system according to claim 7, characterized in that, The spatiotemporal cross-attention fusion unit in the feature fusion module is configured to calculate the correlation between each feature in the aligned feature set and the global prior feature prototype of the defect category, generate dynamic fusion weights based on the correlation, and use the dynamic fusion weights to perform weighted processing on the aligned feature set.

Images