Unmanned aerial vehicle inspection defect detection method and system based on double-end cooperative technology

Abstract

The application relates to the technical field of image detection, in particular to a UAV inspection defect detection method and system based on a double-end cooperative technology, which specifically comprises the following steps: collecting inspection images through a front-end device of a UAV, and uploading the inspection images to a UAV inspection defect detection model deployed in the cloud end to perform defect detection; the model is improved by adding a visual Mamba module after each encoder in the down-sampling stage of a convolutional neural network model, and by increasing the weight of an image block where a suspected defect area is located in the state updating process of the visual Mamba, so that the perception ability of the model to small defects and fuzzy boundary defects is enhanced, the missing detection rate is effectively reduced, and the problem that small defect features are difficult to accurately identify in the power inspection scene of "large background and small target" is avoided.

Description

Technical Field

[0001] This application relates to the field of image detection technology, specifically to a method and system for detecting defects in unmanned aerial vehicle (UAV) inspections based on dual-end collaborative technology. Background Technology

[0002] In recent years, drones have been used more maturely in civilian sectors, including agricultural plant protection, power line inspection, and emergency rescue. In power line inspection, high-performance drones equipped with various sensors acquire image data of equipment. Deep learning-based neural network image recognition technology is then used to rapidly process the collected data, enabling accurate detection of power line defects. This integrated dual-end (front-end data acquisition and cloud data processing) collaborative operation significantly improves the efficiency of power line inspection.

[0003] To avoid interference with communication and control from high-voltage power lines during power line inspections, drones typically maintain a safe operating distance from the target. This results in power line inspection images characterized by large scenes and small targets. Defects such as cracks, gaps, and corrosion in power equipment constitute a small proportion of the overall image, with weak overall texture and shape features. Deep learning-based neural network models prioritize preserving high-level semantic features of salient targets during image convolution and downsampling, leading to the compression or even loss of local details of small-scale defects during the gradual feature extraction process. Therefore, uploading real-time inspection image data collected by the drone's front-end device to a cloud-based model for defect detection under these circumstances reduces the accuracy of small-scale defect identification, resulting in lower timeliness and accuracy for power safety hazard inspections using dual-end collaborative drones. Summary of the Invention

[0004] To address the aforementioned technical problems, the purpose of this application is to provide a method and system for detecting defects in unmanned aerial vehicle (UAV) inspections based on dual-end collaborative technology. The specific technical solution adopted is as follows:

[0005] In a first aspect, embodiments of this application provide a method for detecting defects in unmanned aerial vehicle (UAV) inspections based on dual-end collaborative technology, the method comprising the following steps:

[0006] Images are collected during inspections using the front-end equipment of drones;

[0007] In the downsampling stage of the convolutional neural network model, a visual Mamba module is added after each encoder to obtain the UAV inspection defect detection model.

[0008] The real-time collected inspection images are uploaded to the cloud, and the trained UAV inspection defect detection model is used to detect defects in the uploaded inspection images.

[0009] The detection process of the UAV inspection defect detection model includes:

[0010] The inspection image is used as the input to the model. After passing through the encoders at each stage of the model, feature maps for each channel at that stage are obtained. Each feature map is divided into multiple image blocks in a fixed manner by the visual Mamba module at that stage. For non-pure background image blocks, the gain weight of the non-pure background image blocks is determined based on the discreteness of the pixel gradient distribution and the complexity of the pixel value distribution in the central region under the multi-scale window, so as to adjust the state update formula in the visual Mamba module. Then, the updated feature map output by the visual Mamba module is obtained as the input to the encoder of the next stage. Downsampling stops after passing through the encoder and visual Mamba module of the last stage. Finally, the defect detection result output by the UAV inspection defect detection model is obtained through feature extraction and decoding.

[0011] Preferably, when training the UAV inspection defect detection model, the images in the UAV inspection image sample set include defect-free images and defective images of the inspection object, as well as pure background images.

[0012] Preferably, the process of obtaining the non-pure background image patch includes:

[0013] Before training the UAV inspection defect detection model, a preset number of pure background images in the training set are used as input to the UAV inspection defect detection model. Image blocks are obtained by dividing the images of each channel under the corresponding stage of each encoder through the visual Mamba module, which serve as pure background control image blocks.

[0014] The image block to be identified is compared with the pure background control image block in the corresponding channel of the corresponding stage to determine whether the image block to be identified is a non-pure background image block.

[0015] Preferably, the step of comparing the image block to be identified with a pure background control image block in the corresponding channel of the corresponding stage to determine whether the image block to be identified is a non-pure background image block includes:

[0016] For an image block to be identified, the average pixel value information entropy of all pure background comparison image blocks in the corresponding channel of the corresponding stage is calculated as the background threshold. If the pixel value information entropy of the image block to be identified is greater than the background threshold, the image block to be identified is regarded as a non-pure background image block.

[0017] Preferably, determining the gain weight of the non-pure background image patch based on the discreteness of the pixel gradient distribution and the complexity of the pixel value distribution within the central region under the multi-scale window includes:

[0018] The discreteness of the global and local gradient distributions in the image patches is analyzed separately to obtain the global and local structural continuity of each image patch, and the first visual features of each image patch are constructed.

[0019] The second visual feature of each image patch is constructed based on the complexity of the pixel value distribution in the central region under the multi-scale window in each image patch.

[0020] Gain weights for each image block are constructed based on the first visual feature and the second visual feature.

[0021] Preferably, the first visual feature is specifically:

[0022] The horizontal and vertical gradients of each pixel in the image patch are obtained through a gradient detection algorithm. The gradient direction angle of each pixel is calculated by combining the arctangent function. The sine and cosine values ​​of the gradient direction angle of each pixel are obtained. The average value of the sine function value and the average value of the cosine function value of all pixels in the image patch are calculated and the sum of squares is obtained. The square root of the sum of squares is taken to obtain the global structural continuity of the image patch.

[0023] Set a sliding window in the image block; calculate the sum of squares of the average of the sine function value and the average of the cosine function value of all pixels in each sliding window, and denot it as the first sum of squares; obtain the minimum value of the first sum of squares of all sliding windows in the image block; take the square root of the minimum value as the local structural continuity of the image block;

[0024] Calculate the difference between the global structural continuity and the local structural continuity, and take the maximum value between the difference and the natural number 0 as the first visual feature of the image patch.

[0025] Preferably, the second visual feature is specifically:

[0026] The central region of an image patch is determined using windows of different sizes, and the variance of all pixels within the central region under different sizes is calculated.

[0027] Calculate the absolute value of the difference between the variances of the central regions at two adjacent sizes; use the sum of all the absolute values ​​of the differences as the second visual feature of the image patch.

[0028] Preferably, the gain weight is specifically the average of the normalized value of the first visual feature and the normalized value of the second visual feature.

[0029] Preferably, the state update formula in the adjusted visual Mamba module is:

[0030] In the formula, Let be the updated feature value of the nth image patch. Let be the updated feature value of the (n-1)th image patch. The mapping value for the nth image patch. The gain weight for the nth image patch; , These are the state transition matrix and the input matrix, which are automatically calculated by visual Mamba based on the input, respectively. It is an exponential function with the natural constant e as the base; where, if the nth image patch is a pure background image patch, then the gain weight of the nth image patch is 0; the original state update formula in the visual Mamba module is: .

[0031] Secondly, embodiments of this application also provide a UAV inspection defect detection system based on dual-end collaborative technology, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of any of the methods described above.

[0032] The embodiments of this application have at least the following beneficial effects:

[0033] This application acquires inspection images using a UAV front-end device and performs defect detection using a UAV inspection defect detection model. This model distinguishes between pure background and non-pure background image blocks for each image block divided by the Mamba module, avoiding equal-intensity feature calculations on irrelevant background areas. This allows the model to concentrate limited computational resources and representation capabilities on truly valuable image areas, reducing the interference of background noise on feature extraction. By introducing a gain weight mechanism based on the discreteness of pixel gradient distribution and the complexity analysis of the central region under multi-scale windows, it can intelligently identify non-pure background image blocks containing key defect features. Higher weights are assigned to suspected defect areas with drastic gradient changes and complex textures, making the state update process focus more on these key areas. This significantly enhances the model's ability to perceive minute defects and defects with ambiguous boundaries, effectively reducing the false negative rate and avoiding the problem of inaccurate identification of minute defect features in power inspection scenarios with "large background, small target". Attached Figure Description

[0034] Figure 1 A flowchart illustrating the steps of a UAV inspection defect detection method based on dual-end collaborative technology provided in one embodiment of this application;

[0035] Figure 2 This is a schematic diagram of a defect detection model for drone inspection.

[0036] Figure 3 Schematic diagram of center areas of different sizes. Detailed Implementation

[0037] The following description, in conjunction with the accompanying drawings, details the specific scheme of the UAV inspection defect detection method and system based on dual-end collaborative technology provided in this application.

[0038] Please see Figure 1 The diagram illustrates a flowchart of a UAV inspection defect detection method based on dual-end collaborative technology according to an embodiment of this application. The method includes the following steps:

[0039] Step 1: Collect inspection images using the drone's front-end equipment.

[0040] For scenarios involving front-end and cloud-based collaborative drone inspections of power grid facilities, a high-definition camera mounted on the drone serves as the front-end device. This camera captures real-time images of the inspected objects and uploads them to a cloud-based analysis platform for defect detection via a 5G communication link. The inspected objects include power equipment such as transmission lines, substations, and power poles.

[0041] Step 2: Construct a sample set of drone inspection images.

[0042] A sample image set for UAV inspections was constructed by selecting images from historical inspection images. The selected images included historical inspection images of each type of inspection object, which were then manually labeled. Each type of inspection object's historical inspection images contained defect-free images and defective images, as well as images with a pure background. All images were manually labeled to determine the defect type of the inspection object, such as insulator normal, insulator cracked, transformer bushing normal, transformer bushing damaged, etc.

[0043] The acquired historical inspection images need to be preprocessed. The preprocessing methods include image cropping and image enhancement. Specifically, all image data is cropped to the same size; in this embodiment, the image size is 1024×1024. A filtering algorithm is then used to denoise the image, enhancing image quality and reducing noise interference with the image content. Many existing filtering algorithms exist, such as the Retinex image enhancement algorithm and bilateral filtering denoising algorithms; implementers can choose their preferred method, and this application does not impose specific limitations. This embodiment specifically uses the Retinex image enhancement algorithm for image denoising.

[0044] Furthermore, in this embodiment, the ratio of the training set to the test set of the UAV inspection image sample set is set to 7:3. Implementers can set the ratio of the training set to the test set according to the actual situation, and this application does not impose specific restrictions.

[0045] Step 3: In the downsampling stage of the convolutional neural network model, a visual Mamba module is added after each encoder to obtain the UAV inspection defect detection model. The UAV inspection defect detection model is trained using a preset UAV inspection image sample set.

[0046] Vision Mamba, a state-space model-based visual backbone network, can effectively capture long-distance dependencies in feature maps with linear computational complexity. This characteristic makes it highly promising for improving the real-time performance of defect detection in front-end-cloud collaborative UAV inspection scenarios where communication bandwidth is limited. Integrating Vision Mamba with Convolutional Neural Networks (CNNs) can compensate for the insufficient global feature modeling capabilities of CNNs due to the limitations of local receptive fields, thereby enhancing the network model's representational capabilities in complex inspection scenarios.

[0047] However, power line inspection images typically exhibit the characteristic of "large background, small target." Existing visual Mamba models employ a fixed block-segmentation strategy during processing: the input image is uniformly divided into several image blocks, and a homogeneous state update mechanism is applied to all image blocks. This "one-size-fits-all" approach ignores the differences in the microstructure within different image blocks. Specifically, image blocks containing minute defects are given equal contribution weights in the state space as image blocks containing pure background. This causes the subtle defect features to be submerged by strong background features during state propagation, severely limiting the model's ability to perceive and identify detailed information, ultimately affecting the accuracy and reliability of target detection.

[0048] To address the aforementioned issues, this application proposes a defect detection model for UAV inspection that integrates visual Mamba and convolutional neural networks. For example... Figure 2 As shown, the model employs a U-Net encoder-decoder architecture, with each downsampling stage consisting of a cascaded encoder and a visual Mamba module. Specifically, the first convolutional layer downsamples the input image to half its original size; the subsequent four encoder stages sequentially downsample the feature map by a factor of 1. The visual Mamba modules corresponding to encoders 1 to 4 use block sizes of 16, 16, 8, and 8, respectively, achieving multi-scale feature modeling from coarse to fine. The decoder section corresponds to four upsampling stages, which, through step-by-step feature recovery, ultimately reconstruct the output feature map to the original resolution of the input image.

[0049] Regarding the model training strategy, the loss function is a weighted combination of GIoU bounding box regression loss and multi-class cross-entropy loss; the optimizer used is Adam, and the initial learning rate is set to 1×10⁻⁶. -4 The model output consists of defect location bounding boxes and their category predictions overlaid on the original inspection image.

[0050] The model is trained using the training set constructed above, and the training process follows standard deep learning methods, which will not be elaborated here. After training, the model performance is validated and evaluated using the test set constructed above. Finally, the converged optimal model is deployed to the cloud inference service to support real-time defect detection tasks in collaboration between the UAV front-end and the cloud.

[0051] Step 4: Upload the real-time collected inspection images to the cloud, and use the trained UAV inspection defect detection model to perform defect detection on the uploaded inspection images.

[0052] The inspection images of the target area collected in real time by the drone during flight are uploaded to the cloud in real time via a 5G low-latency, high-bandwidth communication link. The cloud server deploys a drone inspection defect detection model that has been trained on a large scale. The inspection images are input into the deep learning defect detection model for defect detection, and the model outputs the defect type and location in the image. The detection results are promptly transmitted back to the ground control center via the 5G downlink, facilitating timely inspection and repair by maintenance personnel. The detection process of the drone inspection defect detection model includes:

[0053] (1) The inspection image is input into the UAV inspection defect detection model. After processing by the encoder at any stage, a multi-channel feature map is output. Subsequently, the visual Mamba module divides each channel feature map into N image blocks of size × size. Among them, size is usually an integer multiple of 4, and the maximum is no more than 64. The specific value can be flexibly set by the implementer according to the feature map size. This application does not limit this. The size value corresponding to each encoder in this embodiment is detailed in the above content.

[0054] Taking the feature map X of any channel output by any encoder as an example, let its nth image block be denoted as .

[0055] (2) Further, background recognition is performed on each image block to determine whether it is a pure background image block or a non-pure background image block containing electrical equipment. Specifically:

[0056] First, a linear normalization method is used to scale and map the continuous floating-point pixel values ​​within each image block to a numerical range of 0 to 255. Then, the scaled pixel values ​​are divided into equidistant segments within this 0-255 range (in this embodiment, this is divided into 256 equidistant value buckets), and the frequency of values ​​within each segment is counted to obtain a discrete probability distribution. Based on this discrete probability distribution, the information entropy of the pixel value distribution in the image block is calculated. The calculation of information entropy is a well-known technique, and the specific process will not be elaborated further.

[0057] Then, a background threshold is set. In a preferred embodiment of this application, the background threshold is obtained as follows:

[0058] Before training the UAV inspection defect detection model, several historical inspection images containing only background areas are randomly selected from the training set. The number of images selected can be set by the implementer according to the actual situation; this application does not impose a specific limit. In this embodiment, the number of images selected is 10. The selected images are input into the UAV inspection defect detection model to obtain the feature maps of each channel output after processing by the encoder at each stage. Then, the visual Mamba module of that stage is used to divide the feature maps of each channel into image blocks, and the resulting image blocks are recorded as pure background control image blocks. The average pixel value information entropy of all pure background control image blocks in the same channel is calculated as the background threshold.

[0059] Next, taking the feature map X of any channel output by any encoder as an example, its nth image block is denoted as... If image block If the information entropy of all pixel values ​​in the image is less than or equal to the corresponding background threshold, then the image patch is... Marked as a pure background image block; if the image block If the information entropy of all pixel values ​​in the image is greater than the corresponding background threshold, then the image patch is... Marked as a non-pure background image block.

[0060] The design basis of the above background recognition mechanism is that in power inspection images, the overall texture distribution of the background, such as the sky and woodland, is relatively smooth, while the structure of power equipment has obvious texture structure features. Information entropy can reflect the complexity and information content of the distribution of each element in the image block. Therefore, the background is relatively smooth, its gray-scale distribution complexity is low, and its information entropy is small, while the information entropy in the image block containing power equipment is large.

[0061] (3) Further, for non-pure background image patches, the gain weights of the non-pure background image patches are determined based on the degree of dispersion of the gradient direction of the pixels and the complexity of the pixel value distribution in the central region under the multi-scale window. The process includes:

[0062] (3.1) By analyzing the structural regularity and spatial continuity features within different image patches in historical inspection images, a first visual feature is constructed to reflect the degree of structural difference between the local structural features and the global structural features in a single image patch. The specific process is as follows:

[0063] In power line inspection scenarios using drones, the objects being inspected (insulators, cables, etc.) typically exhibit strong structural continuity, with relatively fixed geometric contours and edge morphologies. Under normal, defect-free conditions, image features within the same equipment area maintain a high degree of consistency in texture distribution and edge direction characteristics. However, when defects occur (such as cracks or gaps in insulators, broken strands or damage in cables), although these defective areas may appear small in terms of area proportion, they disrupt the consistency of texture distribution and edge structure in localized regions of the equipment.

[0064] Therefore, in image patches For example, the Sobel operator is used to compute... The horizontal and vertical gradients of each pixel are calculated, and then the gradient direction angle of each pixel can be obtained using the arctangent function. The gradient direction angle of the i-th pixel is represented as: Then, the gradient direction vector for each pixel is constructed by mapping sine and cosine functions, so as to... Taking the i-th pixel as an example, its direction vector is The purpose of using sine and cosine functions is to scale the gradient direction angle, which represents the directional features of a pixel, to the range [-1, 1], facilitating subsequent processing. Statistical analysis of the overall directional characteristics.

[0065] Furthermore, calculate image patches global structural continuity The expression is: ,in, express The number of pixels in the image.

[0066] Known When image blocks When the gradient directions of all pixels in an image are exactly the same, the result calculated by the above formula is 1. The more severe the gradient direction angle dispersion of all pixels, the higher the degree of dispersion, the smaller the result obtained by the above formula.

[0067] Furthermore, a sliding window is used to calculate image patches. The local structural continuity in the sliding window is... , The size can be set by the implementer according to the implementation scenario; this application does not impose any special restrictions. In this embodiment... Set the size to 3, and use a sliding window in the image patch. First, slide the slide window from left to right, then from top to bottom, with a step size of 1. Calculate the structural continuity within each sliding window using the same method as described above. The method is similar: calculate the sum of the squares of the mean sine and cosine values ​​of the gradient direction angles of all pixels in the sliding window, denoted as the first sum of squares, and then take the square root of this sum to obtain the structural continuity of the sliding window. (Image patch acquisition) The minimum value of the structural continuity of all sliding windows in the image is used as the image patch. Local structural continuity .

[0068] Furthermore, calculate image patches First visual features The expression is: ,in, To maximize the function, the value of the first visual feature should not be less than 0.

[0069] When image blocks When local power equipment in a system approaches a defect-free state, due to the significant structural regularity inherent in the equipment itself, its structural orientation remains relatively stable regardless of whether statistical analysis is performed at the global or local scale. At this point, the structural continuity calculated from the local window and the global level shows a high degree of consistency; that is, the global structural continuity is quite close to the local structural continuity. It approaches 0.

[0070] Conversely, when image blocks When minute defects exist, since the defective region typically occupies only a few pixels, the overall structural consistency remains relatively strong from a global perspective. The gradient direction deviations caused by a very small number of defects are easily masked in the global statistics. However, if a sliding window is used for small-scale statistical analysis at a local scale, the structural inconsistencies caused by minute defects will be significantly amplified. The structural continuity index within a particular sliding window will decrease significantly, leading to a minimum structural continuity across all sliding windows. Significantly reduced. At this point, A larger value indicates a larger image patch. It contains fine-grained feature information similar to that of minute defects.

[0071] (3.2) Further, the differences in feature complexity of image patches under multi-scale windows are analyzed to construct a second visual feature, which is used to reflect the differences in feature information under different scale windows during the process of image patches changing from small scale to large scale. The specific process is as follows:

[0072] First, build Considering that the number of pixels on the side of each image block in the visual Mamba image partitioning is even, a minimum center region is set in this embodiment. Size 2 2. Then, based on the center area of ​​the previous size, expand outwards to construct a new center area, such as... Figure 3 As shown, the second central region The size is 4 4. The third central area The size is 6 6. The fourth central area The size is 8 8. And so on, the size of the largest central region is the size of the image block.

[0073] Calculate the variance of all pixel values ​​within the central region of each size; a larger variance indicates a higher level of image information complexity. Then, the image patch... Second visual features The expression is: Where M is an image patch The number of central regions of all sizes in the middle. , Image blocks The variance of all pixel values ​​within the m-th and m+1-th central regions.

[0074] Because minute defects are characterized by significant local anomalies and global stability, as the statistical region expands, more and more structures are included in the statistical scope, and the texture composition relationship between the local defect region and the surrounding normal region continues to change. Therefore, the variance calculated at different scales will fluctuate significantly. This fluctuation may manifest as either an increase or decrease in variance, but it essentially reflects the degree of change in the region's texture complexity during scale expansion. At this point, if... A larger value indicates that the texture complexity within the image patch continuously changes with scale expansion, meaning that the region contains local detail defects that disrupt the original normal texture structure of the image patch; conversely, when... When the value is small or close to 0, it indicates that there are no obvious defects in the central region of the image patch, and the regions covered by each expansion scale have high texture consistency. As the statistical range continues to increase, the variance difference between adjacent scales is small.

[0075] (3.3) Further, calculate image patches Gain weights The expression is:

[0076] in, For image blocks The normalized value of the first visual feature, For image blocks The normalized value of the second visual feature. This application normalizes the first and second visual features of all image blocks separately using the maximum value normalization method. Implementers may also use other normalization methods to normalize the first and second visual features, and this application does not impose specific restrictions.

[0077] when When the gain weight is large, it indicates that the image patch may contain minor defect features. In the subsequent process of updating the state of the entire feature map using the visual Mamba module, the strength of the input mapping of the image patch should be enhanced to increase its feature writing weight in the state space model, so that the local detail information of the image patch can be more fully preserved in the state transfer process.

[0078] (4) Further, in the Visual Mamba module, after the feature map is segmented, it is mapped and flattened to flatten the two-dimensional feature map into a one-dimensional sequence. The calculation operation of mapping and flattening the feature map in the Visual Mamba module belongs to the fixed model architecture of Visual Mamba, and the specific processing process will not be described in detail. The resulting one-dimensional sequence contains The elements of each time step correspond to the mapping value of an image patch.

[0079] The original state update formula for visual Mamba is: ;

[0080] The state update formula for visual Mamba based on the gain weight improvement in this application is as follows: ,in, The updated feature value is the state value at the nth time step, which is the updated feature value of the nth image patch. The updated feature value is the state value at the (n-1)th time step, which is the updated feature value of the (n-1)th image patch. This represents the input at the nth time step, i.e. The mapping value; , These are the state transition matrix and the input matrix, which are automatically calculated by visual Mamba based on the input. An exponential function with the natural constant as its base, used for quantization. and The non-linear relationship between them is represented here by an exponential function. The mapping is performed taking into account the characteristic that the growth rate of the exponential function gradually accelerates.

[0081] when Or when smaller When the value approaches 1, the write weight adjustment to the output is minimal, and then as... The increase, The increase is quite significant, allowing image patches containing fine-grained defect features to occupy a larger influence weight. Compared to linear mapping, exponential mapping can further highlight the differences between high-value image patches and ordinary image patches, improving the ability of visual Mamba to learn and represent fine-grained defect features. Specifically, for background image patches, their gain weight is 0, meaning they are calculated and processed according to the original state update formula of visual Mamba.

[0082] Furthermore, in the visual Mamba module, the state update formula improved by the aforementioned gain weight is used to update the state of all image blocks to obtain an output sequence. The output sequence is then inversely mapped into a two-dimensional feature map to obtain a new feature map. The inverse mapping is a well-known step in visual Mamba, and the specific process will not be elaborated further.

[0083] The new feature map output by the visual Mamba module is used as the input to the encoder in the next stage. This process is iterated step by step. After the feature extraction of the encoder and visual Mamba module in the last stage is completed, the downsampling operation is terminated. Then, through subsequent feature fusion, upsampling, and detection processing, the defect detection result output by the UAV inspection defect detection model is obtained. This achieves dual-end collaborative UAV inspection defect detection, effectively improving inspection efficiency and defect recognition accuracy.

[0084] Based on the same inventive concept as the above methods, this application also provides a UAV inspection defect detection system based on dual-end collaborative technology, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of any one of the above-described UAV inspection defect detection methods based on dual-end collaborative technology.

[0085] In summary, this application provides a UAV inspection defect detection method based on dual-end collaborative technology. Inspection images are acquired by the UAV's front-end device and then processed by a UAV inspection defect detection model. This model distinguishes between pure background and non-pure background image blocks for each image block divided by the Mamba module, avoiding equal-intensity feature calculations on irrelevant background areas. This allows the model to concentrate limited computational resources and representation capabilities on truly valuable image areas, reducing the interference of background noise on feature extraction. By introducing a gain weight mechanism based on the discreteness of pixel gradient distribution and the complexity analysis of the central region under multi-scale windows, the model can intelligently identify non-pure background image blocks containing key defect features. Higher weights are assigned to suspected defect areas with drastic gradient changes and complex textures, making the state update process focus more on these key areas. This significantly enhances the model's ability to perceive minute defects and defects with ambiguous boundaries, effectively reducing the false negative rate and avoiding the problem of inaccurate identification of minute defect features in power inspection scenarios with "large background, small target".

Claims

1. A method for defect detection in UAV inspection based on dual-end collaborative technology, characterized in that, The method includes the following steps: Images are collected during inspections using the front-end equipment of drones; In the downsampling stage of the convolutional neural network model, a visual Mamba module is added after each encoder to obtain the UAV inspection defect detection model. The real-time collected inspection images are uploaded to the cloud, and the trained UAV inspection defect detection model is used to detect defects in the uploaded inspection images. The detection process of the UAV inspection defect detection model includes: The inspection image is used as the input to the model. After passing through the encoders of each stage of the model, feature maps of each channel in that stage are obtained. Each feature map is divided into multiple image blocks in a fixed manner in the visual Mamba module of that stage. For non-pure background image blocks, the gain weight of the non-pure background image blocks is determined according to the discreteness of the pixel gradient distribution and the complexity of the pixel value distribution in the central region under the multi-scale window, so as to adjust the state update formula in the visual Mamba module. Then, the updated feature map output by the visual Mamba module is obtained as the input to the encoder of the next stage. Downsampling stops after passing through the encoder and visual Mamba module of the last stage. Finally, the defect detection result output by the UAV inspection defect detection model is obtained through feature extraction and decoding.

2. The UAV inspection defect detection method based on dual-end collaborative technology as described in claim 1, characterized in that, When training the UAV inspection defect detection model, the images in the UAV inspection image sample set include defect-free images and defective images of the inspection object, as well as pure background images.

3. The UAV inspection defect detection method based on dual-end collaborative technology as described in claim 1, characterized in that, The process of obtaining the non-pure background image patch includes: Before training the UAV inspection defect detection model, a preset number of pure background images in the training set are used as input to the UAV inspection defect detection model. Image blocks are obtained by dividing the images of each channel under the corresponding stage of each encoder through the visual Mamba module, which serve as pure background control image blocks. The image block to be identified is compared with the pure background control image block in the corresponding channel of the corresponding stage to determine whether the image block to be identified is a non-pure background image block.

4. The UAV inspection defect detection method based on dual-end collaborative technology as described in claim 3, characterized in that, The step of comparing the image block to be identified with a pure background control image block in the corresponding channel of the corresponding stage to determine whether the image block to be identified is a non-pure background image block includes: For an image block to be identified, the average pixel value information entropy of all pure background comparison image blocks in the corresponding channel of the corresponding stage is calculated as the background threshold. If the pixel value information entropy of the image block to be identified is greater than the background threshold, the image block to be identified is regarded as a non-pure background image block.

5. The UAV inspection defect detection method based on dual-end collaborative technology as described in claim 1, characterized in that, The determination of gain weights for non-pure background image patches based on the discreteness of pixel gradient distribution and the complexity of pixel value distribution within the central region under a multi-scale window includes: The discreteness of the global and local gradient distributions in the image patches is analyzed separately to obtain the global and local structural continuity of each image patch, and the first visual features of each image patch are constructed. The second visual feature of each image patch is constructed based on the complexity of the pixel value distribution in the central region under the multi-scale window in each image patch. Gain weights for each image block are constructed based on the first visual feature and the second visual feature.

6. The UAV inspection defect detection method based on dual-end collaborative technology as described in claim 5, characterized in that, The first visual feature is specifically: The horizontal and vertical gradients of each pixel in the image patch are obtained through a gradient detection algorithm. The gradient direction angle of each pixel is calculated by combining the arctangent function. The sine and cosine values ​​of the gradient direction angle of each pixel are obtained. The average value of the sine function value and the average value of the cosine function value of all pixels in the image patch are calculated and the sum of squares is obtained. The square root of the sum of squares is taken to obtain the global structural continuity of the image patch. Set a sliding window within the image block; Calculate the sum of squares of the average of the sine function values ​​and the average of the cosine function values ​​of all pixels in each sliding window, and denote it as the first sum of squares; obtain the minimum value of the first sum of squares of all sliding windows in the image patch; The square root of the minimum value is used as the local structural continuity of the image patch; Calculate the difference between the global structural continuity and the local structural continuity, and take the maximum value between the difference and the natural number 0 as the first visual feature of the image patch.

7. The UAV inspection defect detection method based on dual-end collaborative technology as described in claim 5, characterized in that, The second visual feature is specifically: The central region of an image patch is determined using windows of different sizes, and the variance of all pixels within the central region under different sizes is calculated. Calculate the absolute value of the difference between the variances of the central regions at two adjacent sizes; use the sum of all the absolute values ​​of the differences as the second visual feature of the image patch.

8. The UAV inspection defect detection method based on dual-end collaborative technology as described in claim 5, characterized in that, The gain weight is specifically the average of the normalized values ​​of the first visual feature and the normalized values ​​of the second visual feature.

9. The UAV inspection defect detection method based on dual-end collaborative technology as described in claim 1, characterized in that, The adjusted state update formula in the visual Mamba module is as follows: In the formula, Let be the updated feature value of the nth image patch. Let be the updated feature value of the (n-1)th image patch. The mapping value for the nth image patch. The gain weight for the nth image patch; , These are the state transition matrix and the input matrix, which are automatically calculated by visual Mamba based on the input, respectively. It is an exponential function with the natural constant e as the base; where, if the nth image patch is a pure background image patch, then the gain weight of the nth image patch is 0; the original state update formula in the visual Mamba module is: .

10. A UAV inspection defect detection system based on dual-end collaborative technology, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1-9.

Images