A method for rapid processing and feature extraction of low-altitude inspection images of a UAV
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHEJIANG FENCE NETWORK TECH CO LTD
- Filing Date
- 2026-03-05
- Publication Date
- 2026-05-29
Smart Images
Figure CN122116207A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of UAV image processing technology, specifically to a method for rapid processing and feature extraction of low-altitude inspection images from UAVs. Background Technology
[0002] Low-altitude automated inspection by drones is an important means of intelligent operation and maintenance in industries such as power and new energy. However, during outdoor inspections, the relative positions and angles between the drone, the inspected equipment, and the sun change rapidly, resulting in rapidly moving and changing local shadows and highlights in the acquired images. For example, there are strip-shaped dark shadows created by insulator strings due to their own cylindrical structure, which change with the aerial shooting angle; and highly reflective areas that flicker as the drone flies on smooth surfaces such as composite crossarms and metal clamps. Therefore, the rapid processing and feature extraction of low-altitude drone inspection images are prone to the following technical problems:
[0003] 1) The edges of dynamically generated shadows are quite similar to the edge features of real defects, which makes it easy for subsequent feature extraction to misidentify the shadows as actual structural defects, thus affecting the feature extraction effect;
[0004] 2) To cope with complex lighting conditions, existing methods or use computationally intensive image enhancement algorithms (such as multi-scale retinal enhancement algorithms) for preprocessing, which cannot meet the real-time requirements of inspection, thus creating a contradiction between processing speed and accuracy. Summary of the Invention
[0005] To address this issue, the present invention provides a rapid processing and feature extraction method for low-altitude inspection images from unmanned aerial vehicles (UAVs), which solves the problem in the prior art that shadows are easily misidentified as actual structural defects, affecting the feature extraction effect, and also presents a contradiction between processing speed and accuracy.
[0006] To achieve the above objectives, the present invention provides the following technical solution:
[0007] A method for rapid processing and feature extraction of low-altitude inspection images from unmanned aerial vehicles (UAVs) includes the following steps:
[0008] S1: Receive the original UAV inspection image and divide it into multiple rectangular sub-blocks. Based on a preset threshold, rectangles with low standard deviation and gradient average of brightness components are identified as suspected interference blocks. All marked suspected interference blocks are merged to generate light and shadow interference areas.
[0009] S2: Extract basic feature maps with different characteristics from the original UAV inspection image through two parallel feature extraction paths and a shared basic convolution module, and output corrected feature maps by combining the light and shadow interference areas.
[0010] S3: The basic feature map and the corrected feature map are concatenated along the channel dimension to generate a composite feature tensor. The composite feature tensor is then processed based on a pre-set micro-network model to obtain an adaptive compensation weight map. The basic feature map and the corrected feature map are then weighted and fused at each spatial location using the adaptive compensation weight map to output a robust feature map.
[0011] S4: Input the robust feature map into the lightweight region generation network to generate defect candidate regions and their location offsets. Then, classify and regress each candidate region through a multi-task recognition mechanism to generate a list of defect instances.
[0012] S5: Original UAV inspection images and a list of defect instances, generating inspection report data including image information, defect statistics, defect details, and areas of light and shadow interference.
[0013] Furthermore, the specific steps of S1 are as follows:
[0014] S1.1: Receive the original UAV inspection image and convert it from the red-green-blue color space to the HSV color space using color space conversion technology;
[0015] S1.2: Divide the separated brightness component image into a regular grid to obtain several rectangular sub-blocks, and calculate the standard deviation of the brightness values of all pixels in each rectangular sub-block. and the average gradient magnitude of all pixels ;
[0016] S1.3: Determine light and shadow interference based on preset thresholds, standard deviations, and average values;
[0017] S1.4: Merge all marked suspected interference blocks to form a light and shadow interference area.
[0018] Furthermore, in step S1.3, when judging light and shadow interference, if the standard deviation and average value of the rectangular sub-block are both lower than the threshold, it is determined that the area corresponding to the rectangular sub-block is covered by uniform shadows or highlights, the texture information is suppressed, and it is marked as a suspected interference block.
[0019] Furthermore, the two parallel feature extraction pathways include an original information pathway and a fast correction pathway; the original information pathway is used to directly input the uncorrected raw UAV inspection image into the basic convolutional module.
[0020] Furthermore, the fast correction path can fuse the original UAV inspection image and the light and shadow interference area, process the pixel area corresponding to the light and shadow interference area through a smoothing filter, output a locally smoothed corrected image, and then input the locally smoothed corrected image into the basic convolution module for feature extraction to obtain a corrected feature map.
[0021] Furthermore, the specific steps of S3 are as follows:
[0022] S3.1: The basic feature map and the corrected feature map are concatenated along the channel dimension to form a composite feature tensor with original information and local corrected information;
[0023] S3.2: Based on a pre-set micro-network model, the composite feature tensor is processed to obtain an adaptive compensation weight map;
[0024] S3.3: After obtaining the adaptive compensation weight map, use the preset weight values for each spatial location in the weight map. The representative value is obtained by weighted summation of the feature vectors of the corrected feature map and the basic feature map at that location. The process is performed in parallel across all spatial locations to generate robust feature maps. The calculation formula is as follows:
[0025]
[0026] in, Spatial location coordinates, Indicates row index, Indicates column index, In order to be in At this point, the second feature vector extracted from the fast correction pathway, In order to be in The basic feature vector extracted from the original information path.
[0027] Furthermore, the specific steps of S4 are as follows:
[0028] S4.1: Input the robust feature map into the preset lightweight region generation network, and generate defect candidate regions including the probability of defect existence and the offset of candidate regions through convolution operation;
[0029] S4.2: For each defect candidate region, extract the features of the corresponding region from the robust feature map and input them into the multi-task recognition mechanism;
[0030] S4.3: Combine the outputs of the two branches to obtain a list of defect instances.
[0031] Furthermore, the multi-task recognition mechanism includes a classification branch and an edge region regression branch; the classification branch classifies the region features through a fully connected layer, outputs a defect category probability distribution, which is used to determine whether the candidate region contains a defect and to identify the defect type;
[0032] The edge region regression branch predicts the offset between the candidate region and the real defect region through a fully connected layer.
[0033] This invention has the following advantages: By analyzing the uniformity and edge richness of the brightness component, it quickly locates the light and shadow interference area, providing a clear target for subsequent processing. Simultaneously, it uses a rapid correction path to specifically smooth this area, suppressing abnormal gradients and generating a corrected feature map. Then, it dynamically fuses the basic feature map and the corrected feature map using an adaptive compensation weight map. The resulting robust feature map relies more on the corrected structural features in the light and shadow area, thereby weakening the response of shadow edges at the feature level. This effectively reduces the probability of misidentifying light and shadow as defects, allowing subsequent defect detection to focus more on true defects.
[0034] Meanwhile, a lightweight region generation network (RPN) is used to directly generate candidate regions on the dimensionality-reduced robust feature map, and basic convolutional modules are used for fast classification and regression. This avoids redundant global computation in the overall process, achieving both fast interference detection and efficient feature extraction with speed optimization.
[0035] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. Attached Figure Description
[0036] To more intuitively illustrate the prior art and this application, exemplary drawings are provided below. It should be understood that the specific shapes and structures shown in the drawings should not generally be regarded as limiting conditions for implementing this application; for example, based on the technical concept disclosed in this application and the exemplary drawings, those skilled in the art are able to easily make conventional adjustments or further optimizations to the addition / reduction / classification, specific shapes, positional relationships, connection methods, size ratios, etc. of certain units (components).
[0037] Figure 1 This is a flowchart illustrating the implementation of a rapid processing and feature extraction method for low-altitude inspection images from unmanned aerial vehicles (UAVs) according to the present invention. Detailed Implementation
[0038] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. It should be understood that these embodiments are merely for further explanation of the present invention and should not be construed as limiting the scope of protection of the present invention. Technical engineers in the field can make some non-essential improvements and adjustments to the present invention based on the above-described content. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0039] Please see Figure 1A method for rapid processing and feature extraction of low-altitude inspection images from unmanned aerial vehicles (UAVs) includes the following steps:
[0040] S1: Receive the original UAV inspection image and divide it into multiple rectangular sub-blocks. Based on a preset threshold, rectangles with excessively low standard deviations and average gradient values for the brightness components are identified as suspected interference blocks. All marked suspected interference blocks are merged to generate the final light and shadow interference region. This allows for quick and rough location of areas in the image severely affected by light and shadow interference, providing a clear processing target for subsequent targeted compensation at the feature level.
[0041] The specific steps of S1 are as follows:
[0042] S1.1: Receive the original UAV inspection image and convert it from the red-green-blue color space to the HSV color space (hue, saturation, and lightness color space) using color space conversion technology. The purpose of this conversion is to separate the brightness information and color information of the image, thereby facilitating the direct analysis and processing of the lightness component representing the light intensity and avoiding interference from color information in the judgment of light and shadow.
[0043] The aforementioned color space conversion technology refers to a class of well-known algorithms in the field of digital image processing that map pixel color information from a device-dependent representation of the three primary colors (red, green, and blue) to a representation of three components: hue, saturation, and brightness. This technology is mainly used to effectively separate the illumination brightness information (brightness) in an image from the color attribute information (hue and saturation) of objects, thereby providing an independent and direct brightness component input for subsequent processing steps that focus on illumination change analysis, shadow detection, or brightness adjustment.
[0044] S1.2: Divide the separated brightness component image into a regular grid, dividing it into multiple non-overlapping, fixed-size rectangular sub-blocks. Calculate the standard deviation of the brightness values of all pixels within each rectangular sub-block. and the average gradient magnitude of all pixels This is used to quantify the uniformity of brightness and the richness of the edges in the area. The calculation formula is as follows:
[0045]
[0046] in, This represents the total number of pixels within the rectangular sub-block. The first one within the rectangular sub-block The brightness value of each pixel. This is the average brightness value of all pixels within the rectangular sub-block. The average gradient magnitude. The calculation formula is as follows:
[0047]
[0048] in, The height of the sub-block. The width of the sub-block. For the first sub-block line, number The gradient magnitude of each column pixel. The calculation formula is as follows:
[0049]
[0050] in, Calculate pixel points for gradient operators (such as the Sobel operator). The gradient value in the horizontal direction, Calculate pixel points for gradient operator The gradient value in the vertical direction.
[0051] S1.3: Determine light and shadow interference based on preset thresholds, standard deviations, and average values;
[0052] If the standard deviation and average value of a rectangular sub-block are both below the threshold, it is determined that the corresponding area of the rectangular sub-block is covered by uniform shadows or highlights, and the texture information is suppressed, and it is marked as a suspected interference block.
[0053] S1.4: Merge all marked suspected interference blocks to form a light and shadow interference area.
[0054] S2: The basic feature maps with different characteristics in the original UAV inspection images are extracted by two parallel feature extraction paths and a shared basic convolution module. Combined with the light and shadow interference area, the corrected feature map is output to provide a basis for subsequent feature fusion.
[0055] The basic convolutional module consists of multiple consecutive convolutional layers, activation functions, and pooling layers, which are responsible for extracting common primary visual features from the input raw UAV inspection images.
[0056] The system employs two parallel feature extraction pathways: a raw information pathway and a fast correction pathway. The raw information pathway directly inputs the uncorrected, raw UAV inspection image into the basic convolutional module. After layer-by-layer processing by the module, a basic feature map is output. This basic feature map retains all the information from the original image.
[0057] The fast correction path can fuse the original UAV inspection image with areas of light and shadow interference. A smoothing filter processes the pixel regions corresponding to these interference areas. This filter suppresses drastic grayscale abrupt changes and abnormal gradients caused by strong shadows or highlights in local areas while preserving object edge information as much as possible, outputting a locally smoothed corrected image. For example, an anisotropic diffusion filter diffuses at different speeds in different directions based on the image's gradient information, thus smoothing noise while preserving edges.
[0058] The locally smoothed image is then input into the basic convolutional module, where it passes through multiple consecutive convolutional layers, activation functions, and pooling layers for feature extraction. However, since the interference regions in the locally smoothed image have been suppressed beforehand, the features extracted by this module in this path will include less information about light and shadow disturbances, thus outputting a corrected feature map. The corrected feature map represents more of the object's inherent structure and texture, reducing the impact of light and shadow interference on feature extraction.
[0059] S3: The base feature map and the corrected feature map are concatenated along the channel dimension to generate a composite feature tensor. This composite feature tensor is then processed based on a pre-built micro-network model to obtain an adaptive compensation weight map. This adaptive compensation weight map is then used to weight and fuse the base feature map and the corrected feature map at each spatial location, outputting a robust feature map. This allows for dynamic fusion of the original and corrected features based on the degree of light and shadow interference in local areas, enhancing the robustness of the features to changes in illumination.
[0060] The specific steps for S3 are as follows:
[0061] S3.1: The basic feature map and the corrected feature map are concatenated along the channel dimension to form a composite feature tensor with original information and local corrected information;
[0062] S3.2: Based on a pre-set micro-network model, the composite feature tensor is processed to obtain an adaptive compensation weight map.
[0063] The miniature network model consists of two consecutive 1×1 convolutional layers. The first 1×1 convolutional layer is used to fuse cross-channel information and perform dimensionality reduction, with its main function focused on the channel dimension. When the composite feature tensor enters the first 1×1 convolutional layer, this layer learns different convolutional kernel weights and linearly combines the various channels of the input feature tensor to achieve the fusion of cross-channel information, allowing the feature information carried by different channels to communicate and integrate with each other.
[0064] Simultaneously, this layer can also reduce the dimensionality of features by appropriately setting the number of output channels. Since the input composite feature tensor may have a large number of channels, too many channels increase the number of parameters and computational complexity in subsequent calculations. Dimensionality reduction reduces the dimensionality of the data while preserving the main feature information, thus improving computational efficiency. After processing by the first 1×1 convolutional layer, a composite feature map with a reduced number of channels will be output.
[0065] The composite feature map output from the first 1×1 convolutional layer is then processed by a second 1×1 convolutional layer. The main task of this layer is to further compress the number of channels to 1, outputting a single-channel composite feature map. After obtaining the single-channel composite feature map, a sigmoid activation function is applied, which maps the output value to the [0,1] interval. For each spatial location on the single-channel composite feature map (i.e., the feature vector corresponding to each pixel), this mini-network can independently learn and predict a scalar weight value between 0 and 1. The weight value reflects the importance of the original information pathway and the fast correction pathway features at that spatial location. All weight values together form an adaptive compensation weight map with the same spatial size as the input feature map.
[0066] S3.3: After obtaining the adaptive compensation weight map, use the preset weight values for each spatial location in the weight map. The representative value is obtained by weighted summation of the feature vectors of the corrected feature map and the basic feature map at that location. The process is performed in parallel across all spatial locations, and the resulting robust feature map is calculated using the following formula:
[0067]
[0068] in, Spatial location coordinates, Indicates row index, Indicates column index, In order to be in This allows for rapid correction of the second feature vector extracted from the pathway. In order to be in The basic feature vector extracted from the original information path.
[0069] S4: Input the robust feature map into the lightweight region generation network (RPN) to generate defect candidate regions and their location offsets. Then, classify and regress each candidate region through a multi-task recognition mechanism to generate a list of defect instances, which is used to convert the robust feature map into a structured defect recognition result.
[0070] The specific steps for S4 are as follows:
[0071] S4.1: The robust feature map is input into a pre-built lightweight Region Proposal Network (RPN). The RPN slides a fixed-size window across the robust feature map and generates candidate regions for defects through convolution operations, including the probability of defect presence and the candidate region offset. The probability of defect presence is used to determine whether the current window region is likely to contain a defect. The candidate region offset is used to predict the offset of the window relative to the actual defect location. The candidate regions generated by the RPN tend to cover the actual defect region rather than lighting artifacts (such as shadow edges). This reduces the number of invalid candidate regions in subsequent processing.
[0072] S4.2: For each defect candidate region, extract the features of the corresponding region from the robust feature map and input them into the multi-task recognition mechanism. The multi-task recognition mechanism includes a classification branch and an edge region regression branch; the classification branch classifies the region features through the fully connected layer of the RPN, outputs the defect category probability distribution, which is used to determine whether the candidate region contains a defect, and further identify the defect type (such as cracks, corrosion, dirt, etc.).
[0073] The edge region regression branch predicts the offset between the candidate region and the real defect region (such as the difference in center point coordinates and the difference in aspect ratio) through the fully connected layer of the RPN, and applies the offset to correct the candidate region. This is used to fine-tune the position and size of the candidate region so that it fits the defect target more accurately.
[0074] S4.3: Merge the outputs of the two branches to obtain a list of defect instances. Each defect instance includes a defect category and candidate regions, providing a complete data foundation for subsequent analysis and visualization.
[0075] S5: The system generates a structured inspection report containing raw UAV inspection images and a list of defect instances. This report includes image information, defect statistics, defect details, and areas affected by light and shadow interference. This ensures the generation of intuitive and machine-readable final results.
[0076] This invention rapidly locates areas of light and shadow interference by analyzing the uniformity and edge richness of the brightness component, providing a clear target for subsequent processing. Simultaneously, a rapid correction pathway is used to specifically smooth these areas, suppressing abnormal gradients and generating a corrected feature map. This corrected feature map is then dynamically fused with an adaptive compensation weight map. The resulting robust feature map relies more heavily on the corrected structural features in the light and shadow regions, thereby weakening the response at the shadow edges at the feature level. This effectively reduces the probability of misidentifying light and shadow as defects, allowing subsequent defect detection to focus more on genuine defects.
[0077] Meanwhile, a lightweight region generation network (RPN) is used to directly generate candidate regions on the dimensionality-reduced robust feature map, and basic convolutional modules are used for fast classification and regression. This avoids redundant global computation in the overall process, achieving both fast interference detection and efficient feature extraction with speed optimization.
[0078] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for rapid processing and feature extraction of low-altitude inspection images from unmanned aerial vehicles (UAVs), characterized in that, Includes the following steps: S1: Receive the original UAV inspection image and divide it into multiple rectangular sub-blocks. Based on a preset threshold, rectangles with low standard deviation and gradient average of brightness components are identified as suspected interference blocks. All marked suspected interference blocks are merged to generate light and shadow interference areas. S2: Extract basic feature maps with different characteristics from the original UAV inspection image through two parallel feature extraction paths and a shared basic convolution module, and output corrected feature maps by combining the light and shadow interference areas. S3: The basic feature map and the corrected feature map are concatenated along the channel dimension to generate a composite feature tensor. The composite feature tensor is then processed based on a pre-set micro-network model to obtain an adaptive compensation weight map. The basic feature map and the corrected feature map are then weighted and fused at each spatial location using the adaptive compensation weight map to output a robust feature map. S4: Input the robust feature map into the lightweight region generation network to generate defect candidate regions and their location offsets. Then, classify and regress each candidate region through a multi-task recognition mechanism to generate a list of defect instances. S5: Original UAV inspection images and a list of defect instances, generating inspection report data including image information, defect statistics, defect details, and areas of light and shadow interference.
2. The method for rapid processing and feature extraction of low-altitude UAV inspection images according to claim 1, characterized in that, The specific steps of S1 are as follows: S1.1: Receive the original UAV inspection image and convert it from the red-green-blue color space to the HSV color space using color space conversion technology; S1.2: Divide the separated brightness component image into a regular grid to obtain several rectangular sub-blocks, and calculate the standard deviation of the brightness values of all pixels in each rectangular sub-block. and the average gradient magnitude of all pixels ; S1.3: Determine light and shadow interference based on preset thresholds, standard deviations, and average values; S1.4: Merge all marked suspected interference blocks to form a light and shadow interference area.
3. The method for rapid processing and feature extraction of low-altitude UAV inspection images according to claim 2, characterized in that, When judging light and shadow interference in S1.3, if the standard deviation and average value of the rectangular sub-block are both lower than the threshold, it is determined that the area corresponding to the rectangular sub-block is covered by uniform shadow or highlight, the texture information is suppressed, and it is marked as a suspected interference block.
4. The method for rapid processing and feature extraction of low-altitude UAV inspection images according to claim 1, characterized in that, The two parallel feature extraction pathways include the original information pathway and the fast correction pathway; the original information pathway is used to directly input the raw UAV inspection images without any correction into the basic convolutional module.
5. The method for rapid processing and feature extraction of low-altitude UAV inspection images according to claim 4, characterized in that, The fast correction path can fuse the original UAV inspection image and the light and shadow interference area, process the pixel area corresponding to the light and shadow interference area through a smoothing filter, output a locally smoothed corrected image, and then input the locally smoothed corrected image into the basic convolution module for feature extraction to obtain the corrected feature map.
6. The method for rapid processing and feature extraction of low-altitude UAV inspection images according to claim 1, characterized in that, The specific steps of S3 are as follows: S3.1: The basic feature map and the corrected feature map are concatenated along the channel dimension to form a composite feature tensor with original information and local corrected information; S3.2: Based on a pre-set micro-network model, the composite feature tensor is processed to obtain an adaptive compensation weight map; S3.3: After obtaining the adaptive compensation weight map, use the preset weight values for each spatial location in the weight map. The representative value is obtained by weighted summation of the feature vectors of the corrected feature map and the basic feature map at that location. The process is performed in parallel across all spatial locations to generate robust feature maps. The calculation formula is as follows: in, Spatial location coordinates, Indicates row index, Indicates column index, In order to be in This allows for rapid correction of the second feature vector extracted from the pathway. In order to be in The basic feature vector extracted from the original information path.
7. The method for rapid processing and feature extraction of low-altitude inspection images from unmanned aerial vehicles according to claim 1, characterized in that, The specific steps of S4 are as follows: S4.1: Input the robust feature map into the preset lightweight region generation network, and generate defect candidate regions including the probability of defect existence and the offset of candidate regions through convolution operation; S4.2: For each defect candidate region, extract the features of the corresponding region from the robust feature map and input them into the multi-task recognition mechanism; S4.3: Combine the outputs of the two branches to obtain a list of defect instances.
8. The method for rapid processing and feature extraction of low-altitude UAV inspection images according to claim 7, characterized in that, The multi-task recognition mechanism includes a classification branch and a border region regression branch; The classification branch classifies the regional features through a fully connected layer and outputs a probability distribution of defect categories, which is used to determine whether the candidate region contains a defect and to identify the defect type. The edge region regression branch predicts the offset between the candidate region and the real defect region through a fully connected layer.