Unmanned aerial vehicle inspection compressed image quality evaluation method and system based on transformer

Abstract

This application discloses a Transformer-based method and system for evaluating the quality of compressed images used in UAV inspections, relating to the field of image processing technology. The method includes: loading an original high-resolution image and a compressed image to be evaluated; inputting the compressed image to be evaluated into a task agent network to obtain predicted detection results; loading the actual detection results of the original high-resolution image through a benchmark detection model; calculating a task performance loss vector based on the predicted and actual detection results; weighted fusion of the task performance loss vector to obtain a comprehensive task performance score; mapping the comprehensive task performance score to a preset quality level; and outputting the quality level of the compressed image to be evaluated. This method solves the technical problem of existing compressed image quality evaluation methods being unable to adaptively adjust the compression strategy according to task requirements, leading to wasted computing power.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to a method and system for evaluating the quality of compressed images from UAV inspections based on Transformer. Background Technology

[0002] With the development of UAV inspection technology for power and oil and gas lines, the inspection images captured by UAVs need to be wirelessly transmitted back to the ground processing platform. Due to the limited wireless transmission bandwidth, the original inspection images usually need to be compressed at a high ratio before transmission. The compressed images will have varying degrees of distortion, affecting the accuracy of downstream defect detection tasks. Therefore, it is necessary to evaluate the quality of the compressed UAV inspection images, select compressed images that meet the requirements of downstream tasks, and take corresponding processing measures such as retransmission and manual review for compressed images that do not meet the quality standards.

[0003] Traditional image quality assessment methods are mostly based on pixel-level error calculations, such as mean squared error (MSE) and peak signal-to-noise ratio (PSNR). These methods obtain evaluation results by comparing the pixel differences between compressed images and original high-resolution images. However, pixel-level errors have low correlation with the performance of downstream defect detection tasks. Some compressed images with high pixel errors can still support accurate results in defect detection tasks, while compressed images with low pixel errors may have the problem of losing key target features. This makes them unsuitable for the quality assessment needs of UAV inspection scenarios, which are centered on task efficiency. Some task-oriented image quality assessment methods rely on deep networks to extract features. However, for specific scenarios such as UAV inspection, these methods have large model parameters, high computational requirements, and are difficult to deploy at the edge. Furthermore, they do not design targeted evaluation dimensions that take into account the characteristics of defect target size and location in the inspection scenario, and the evaluation results cannot directly support subsequent inspection process decisions. Summary of the Invention

[0004] This application provides a Transformer-based method and system for evaluating the quality of compressed images during UAV inspections, which solves the technical problem that existing compressed image quality evaluation methods cannot adaptively adjust compression strategies according to task requirements, resulting in wasted computing power.

[0005] The technical solution to the above-mentioned technical problems in this application is as follows: In a first aspect, this application provides a Transformer-based method for evaluating the quality of compressed images used in UAV inspections, the method comprising: Load the original high-resolution image and the corresponding compressed image to be evaluated; The compressed image to be evaluated is input into the task agent network to obtain the predicted detection result. The task agent network is a lightweight target detection network used to simulate downstream defect detection tasks. Load the actual detection results of the original high-definition image, which are obtained by inference from the original high-definition image using a benchmark detection model; Calculate the task performance loss vector based on the predicted detection results and the actual detection results; The task performance loss vector is weighted and fused according to preset weight coefficients to obtain a comprehensive task performance score. The overall performance score of the task is mapped to a preset quality level, and the quality level of the compressed image to be evaluated is output.

[0006] Secondly, this application provides a Transformer-based UAV inspection compressed image quality evaluation system, including: The initial image acquisition module is used to load the original high-definition image and the corresponding compressed image to be evaluated. The detection result prediction module is used to input the compressed image to be evaluated into the task agent network to obtain the predicted detection result. The task agent network is a lightweight target detection network used to simulate downstream defect detection tasks. The detection result loading module is used to load the real detection results of the original high-definition image, which are obtained by inference from the original high-definition image by the benchmark detection model. The loss vector calculation module is used to calculate the task performance loss vector based on the predicted detection results and the actual detection results. The comprehensive score acquisition module is used to perform weighted fusion of the task performance loss vector according to preset weight coefficients to obtain a comprehensive task performance score. The quality level output module is used to map the comprehensive score of task performance to a preset quality level and output the quality level of the compressed image to be evaluated.

[0007] This application provides one or more technical solutions, which have at least the following technical effects or advantages: This application provides a Transformer-based method and system for evaluating the quality of compressed images used in UAV inspections. First, a lightweight task agent network is used to simulate the performance changes of downstream defect detection tasks, measuring the impact of compressed images on inspection effectiveness from a task performance perspective, thus obtaining evaluation results that better reflect the actual task requirements of UAV inspections. Second, a lightweight task agent network is obtained through knowledge distillation, with a parameter count far smaller than the benchmark detection model, effectively reducing the computational load of the evaluation process and making it more suitable for deployment on edge devices such as UAVs. Finally, the interaction features between the original and compressed images are extracted using a dual-stream Transformer structure, and task-aware attention is introduced to enhance the weight of task-related features, improving the accuracy of the evaluation results. Furthermore, the obtained quality level can directly support automated decision-making in subsequent inspection processes, adapting to the compressed image transmission and processing needs under different bandwidth scenarios.

[0008] Through the above technical solution, this application solves the problem of low correlation between traditional pixel-level quality evaluation results and downstream defect detection tasks, overcomes the shortcomings of existing task-oriented evaluation methods with large model parameters and difficulty in adapting to edge deployment, and can output compressed image quality evaluation results that can directly support process decisions for UAV inspection scenarios. It balances compression ratio and downstream defect detection accuracy under limited bandwidth conditions, and improves the overall work efficiency of UAV inspection. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a flowchart illustrating the method for evaluating the quality of compressed images from unmanned aerial vehicle (UAV) inspections based on Transformer, as provided in this application embodiment. Figure 2 This is a schematic diagram of the structure of the UAV inspection compressed image quality evaluation system based on Transformer provided in the embodiments of this application.

[0011] Appendix Figure 2 The components represented by each number are explained below: The system includes an initial image acquisition module 11, a detection result prediction module 12, a detection result loading module 13, a loss vector calculation module 14, a comprehensive score acquisition module 15, and a quality level output module 16. Detailed Implementation

[0012] This application provides a Transformer-based method and system for evaluating the quality of compressed images during UAV inspections, addressing the technical problem that existing compressed image quality evaluation methods cannot adaptively adjust compression strategies according to task requirements, leading to wasted computing power.

[0013] Example 1, as Figure 1 As shown in the embodiments of this application, a method for evaluating the quality of compressed images from UAV inspections based on Transformer is provided, including: S10: Load the original high-resolution image and the corresponding compressed image to be evaluated; In this embodiment of the application, an original high-definition image and a corresponding compressed image to be evaluated are first acquired. The original high-definition image is the original image of the area to be inspected acquired by the UAV during the inspection process, and the compressed image to be evaluated is the image obtained by encoding and compressing the original high-definition image, which is used to be transmitted back to the ground processing platform for defect detection.

[0014] S20: Input the compressed image to be evaluated into the task agent network to obtain the predicted detection result. The task agent network is a lightweight target detection network used to simulate downstream defect detection tasks. In this embodiment, a benchmark detection model is used to perform defect detection on all training samples in advance to obtain a true labeled detection result as the benchmark ground truth. Then, the output of the benchmark detection model is used as a soft label to train the target detection network built on a lightweight backbone through knowledge distillation. The distillation yields a task agent network, which can simulate the output of the downstream defect detection task with a smaller number of parameters, reducing the computational overhead in the image quality evaluation stage and adapting to the edge deployment requirements of UAVs. Specifically, the predicted detection result includes the number of detected defective targets, the bounding box coordinates of each defective target, and the detection confidence of each defective target.

[0015] Specifically, the task proxy network is a lightweight copy of the benchmark detection model, typically with only 10% to 20% of the parameters. It is trained through knowledge distillation to make its output distribution approximate the benchmark. During online evaluation, the compressed image is input into the task proxy network to obtain predicted detection results, replacing the cumbersome benchmark model to reduce computational overhead. Both are used together to calculate the task performance loss.

[0016] Further, the compressed image to be evaluated is input into the task agent network to obtain the prediction detection result, which includes the following steps: Collect a set of original inspection images in the scenario of drone inspection. The set of original inspection images contains multiple original high-definition images and corresponding defect target annotations. For each original high-resolution image, multiple lossy compression algorithms and multiple compression quality parameters are used to generate multiple compressed images to be evaluated, forming an image pair training set; The original high-definition images in the original inspection image set are input into the benchmark detection model to obtain the true detection result of each original high-definition image; The image pairs are input into the benchmark detection model along with the compressed images to be evaluated in the training set to obtain the benchmark detection result for each compressed image to be evaluated. Calculate the task performance loss label between the benchmark detection result and the corresponding real detection result for each compressed image to be evaluated; Using the original high-resolution images and the compressed images to be evaluated in the image pair training set as input, and the corresponding task performance loss labels as supervision, a two-stream Transformer feature extraction network is trained, and the training loss function is the mean square error between the predicted task performance loss vector and the task performance loss label.

[0017] In this embodiment, firstly, a set of original inspection images from drone inspection scenarios is collected. This set consists of real-world drone images taken under different lines and scenarios, covering various common defect types to ensure the trained model can adapt to the diverse changes in inspection scenarios. Defect target annotation refers to manually or semi-automatically annotating the inspection targets of interest in the original high-definition images, such as insulator damage, missing pins, and foreign objects on the line. The location information and category label of each defect target are recorded and used as a supervisory signal for training the target detection model.

[0018] Secondly, various compression schemes are used to generate compressed images of different distortion levels to be evaluated, and sufficient training sample pairs are constructed to ensure that the model learns the impact of different compression distortion levels on the performance of downstream tasks. Among them, the lossy compression algorithm is at least one of JPEG, WebP, and HEVC, and the compression quality parameters cover multiple levels from low compression ratio and high fidelity to high compression ratio and low fidelity, thereby generating compressed images of different sizes and different distortion levels to meet the diversity requirements of training samples.

[0019] Subsequently, the benchmark detection model is used to obtain the true detection results of the original high-definition image and the benchmark detection results of the compressed image to be evaluated. This is used to calculate the task performance loss label, reflecting the actual impact of compression distortion on task performance. This label serves as a supervisory signal to train a dual-stream Transformer feature extraction network, allowing the model to learn the correspondence between the feature differences between the original and compressed images and the task loss. The benchmark detection model is a high-precision defect detection model actually deployed in UAV inspection systems, such as YOLOv8 or Faster R-CNN. It has a large number of parameters and high accuracy, and is used to infer from the original high-definition image to generate true detection results, serving as a reference standard for quality evaluation.

[0020] For example, to train the benchmark detection model, a training dataset is first constructed, consisting of 8,000 images captured by drone inspections. These images are labeled by annotators according to six defect categories: bolt corrosion, insulator damage, hardware cracking, and broken conductor strands. Each image is labeled with the coordinates and category of the defect bounding box. The dataset is divided into training, validation, and test sets in a 7:2:1 ratio. YOLOv8x is selected as the basic architecture of the benchmark detection model. The input image size is set to 1280×1280, the training batch size is set to 16, the initial learning rate is set to 1×10^-4, and the weight decay coefficient is set to 5×10^-4. The AdamW optimizer is used, and the model is trained for 100 epochs. A cosine annealing learning rate decay strategy is employed, and the model is validated on the validation set every 10 training epochs. Finally, the model weights with the highest mAP on the validation set are selected as the weights of the trained benchmark detection model.

[0021] Specifically, the task performance loss label is calculated between the benchmark detection result and the corresponding real detection result of each compressed image to be evaluated. The task performance loss label is a three-dimensional vector, which is generated by comparing the detection output of the original image and the compressed image, provided that the benchmark detection model has been trained. It is used to train the quality evaluation model, such as the two-stream Transformer feature extraction network.

[0022] Furthermore, training the two-stream Transformer feature extraction network also includes training a task agent network: A lightweight object detection network is constructed as the task proxy network; The benchmark detection model is used to generate pseudo-labels for the original high-definition images in the original inspection image set; Using the original high-definition image as input and the pseudo-label as supervision, the task agent network is trained through knowledge distillation. The training aims to minimize the divergence between the output distribution of the task agent network and the output distribution of the benchmark detection model, and the number of parameters of the task agent network is less than a preset ratio threshold of the number of parameters of the benchmark detection model. The parameters of the pre-trained task agent network are fixed.

[0023] In this embodiment, a lightweight object detection network is first constructed as a task proxy network. For example, the backbone network can be a lightweight MobileNet or ShuffleNet to ensure that the overall model's parameter count and inference speed meet the requirements for edge deployment.

[0024] Then, the trained benchmark detection model is used to infer and generate detection pseudo-labels for all the original high-definition images in the original inspection image set. Pseudo-labels refer to the detection results automatically generated by the trained high-performance model, such as the benchmark detection model, on the unlabeled original images. Although the automatically generated results are not manually labeled, they can be used as supervision signals to train another model, such as a task agent network.

[0025] Subsequently, the original high-resolution images are input into the task agent network to be trained. The pseudo-labels output by the benchmark detection model are used as supervision signals. The task agent network is trained using knowledge distillation. The training objective is to minimize the KL divergence between the detection probability distribution output by the task agent network and the detection probability distribution output by the benchmark detection model. During the training process, the number of parameters of the task agent network is constrained not to exceed a preset proportional threshold of the number of parameters of the benchmark detection model to ensure that the model meets the lightweight requirements. After training, all parameters of the task agent network are fixed and reserved for use in the subsequent image quality evaluation stage.

[0026] Here, KL divergence is used to measure the difference between two probability distributions. In this embodiment, it is used to quantify and minimize the difference between the output distribution of the task agent network and the output distribution of the benchmark detection model. The smaller the KL divergence, the closer the output of the task agent network is to the output of the benchmark detection model, and the better the simulation effect of downstream defect detection tasks. For example... Where P(i) is the detection probability of the i-th type of defect target output by the benchmark detection model, and Q(i) is the detection probability of the i-th type of defect target output by the task agent network. The parameters of the task agent network are continuously updated by gradient descent, so that the KL divergence gradually decreases until the model converges and the training is completed.

[0027] For example, the training of the task agent network is carried out through the following steps: First, the parameters of the lightweight object detection network are initialized, with an initial learning rate of 0.001, a batch size of 32, and a total training epoch of 50. 32 original high-resolution images are randomly selected from the original inspection image set to form a training batch, which is then input into the task agent network to be trained, obtaining the detection probability distribution output by the task agent network. Then, the original high-resolution images from the same batch are input into the benchmark detection model, obtaining the detection probability distribution output by the benchmark detection model. The KL divergence between the two distributions is calculated as the distillation loss, and the gradient is calculated through backpropagation. The parameters of the task agent network are updated using the Adam optimizer. After each training epoch, the learning rate is decayed to 0.9 times the current learning rate. The above steps are repeated until all 50 training epochs are completed. Training is stopped early when the KL divergence on the validation set no longer decreases for 5 consecutive epochs, finally fixing all the parameters of the task agent network.

[0028] Specifically, the dual-stream Transformer feature extraction network has an asymmetric structure, including a raw image stream branch and a compressed image stream branch; The original image stream branch contains L1 Transformer coding blocks, and the compressed image stream branch contains L2 Transformer coding blocks, where L1 is greater than L2. The original image stream branch and the compressed image stream branch are equipped with a cross-attention module after the K-th layer coding block to realize information interaction between the feature map of the original image stream and the feature map of the image stream to be evaluated, where K is less than or equal to L2.

[0029] In this embodiment, the dual-stream Transformer feature extraction network adopts an asymmetric structure design. The original image stream uses a deeper encoder to extract accurate reference features, while the compressed image stream uses a shallower encoder to reduce computation.

[0030] The cross-attention module enables feature interaction between the two branches in the corresponding coding layer, allowing shallow features to be enhanced with deep reference features. This achieves lightweight design while maintaining evaluation accuracy, making it suitable for edge deployment of drones. Through asymmetric design, it further reduces the overall computational overhead of the model while ensuring feature extraction capabilities, making it more suitable for the deployment needs of edge devices.

[0031] Specifically, the original image stream branch contains L1 Transformer coding blocks, and the compressed image stream branch contains L2 Transformer coding blocks. For example, L1 is set to 6 and L2 is set to 3. A cross-attention module is set after the Kth layer coding block, where K is less than or equal to L2. For example, K can be set to 3. This allows the deep reference features extracted from the original image to interact with the shallow features extracted from the compressed image, enabling the features of the compressed image stream to align with the feature distribution of the original image and more accurately capture the feature differences caused by compression distortion.

[0032] Furthermore, the Transformer-based method for evaluating the quality of compressed images used in UAV inspections also includes: On the deep feature map of the dual-stream Transformer feature extraction network, the feature response map output by the intermediate layer of the task agent network is loaded as the task-aware attention weight. The task-aware attention weights are multiplied element-wise with the original image stream feature map and the image stream feature map to be evaluated to obtain the weighted original feature map and the weighted feature map to be evaluated. The weighted original feature map and the weighted feature map to be evaluated are used as inputs to subsequent hierarchical coding blocks.

[0033] In this embodiment, firstly, on the deep feature map of the dual-stream Transformer feature extraction network, the feature response map corresponding to the detection target region is extracted from the intermediate layer of the task agent network. The response value of the region where the defect target is located in the feature response map is higher, and the response value of the background region is lower, which can reflect the spatial distribution of the defect target. This is used as the task perception attention weight and loaded onto the corresponding deep feature map of the dual-stream Transformer feature extraction network. Then, it is multiplied element-wise with the feature maps of the original image stream and the compressed image stream, respectively. The weighted feature map will automatically enhance the feature contribution of the defect target region and suppress the feature interference of irrelevant background regions, so that the subsequent model can focus more on the impact of compression distortion on the feature of the defect target region, improve the correlation between the quality evaluation result and the downstream defect detection task, and then input the weighted feature map into the subsequent coding block to continue to extract features, so that the dual-stream Transformer feature extraction network focuses on the semantic region that has a key impact on the downstream defect detection task.

[0034] For example, the output feature map is extracted from the Xth convolutional layer of the task agent network backbone. The dimensionality is reduced by global average pooling to obtain a feature response map with the same size as the deep feature map of the two-stream Transformer feature extraction network. After normalization, the response value is mapped to the 0-1 interval and directly used as the task-aware attention weight. It is then multiplied element-wise with the feature maps of the two branches of the two streams and input into the subsequent encoding block for further processing. The normalization is achieved by max-min normalization.

[0035] S30: Load the actual detection results of the original high-definition image, which are obtained by inference from the original high-definition image using a benchmark detection model; In this embodiment, during the actual online image quality evaluation stage, the original high-definition image to be evaluated is first input into a pre-trained benchmark detection model to obtain the real detection result corresponding to the original high-definition image, which serves as a benchmark reference for subsequent calculation of the performance loss of the prediction task. This step only needs to be performed once for a single original high-definition image, and after obtaining the reference information, it can be used for the quality evaluation calculation of all corresponding compressed version images.

[0036] S40: Calculate the task performance loss vector based on the predicted detection results and the actual detection results. The task performance loss vector includes the target loss rate, the positioning offset, and the confidence decrease. Wherein, the target loss rate is the proportion of targets in the actual detection results that are not detected in the predicted detection results, the positioning offset is the average offset between the bounding box position and size of each matched detection target pair, and the confidence decrease is the average decrease in the confidence score of each matched detection target pair.

[0037] In this embodiment of the application, firstly, the predicted detection result of the compressed image to be evaluated output by the task agent network is matched with the real detection result of the original high-definition image output by the benchmark detection model to obtain the corresponding detection target pair. The matching detection target pair is a correspondence consisting of a target in the real detection result and a target in the predicted detection result that is successfully matched with it. Each corresponding detection target pair represents the detection result of the same physical target in different images.

[0038] Subsequently, the task performance loss vector is calculated from three dimensions: First, the number of targets that did not match the predicted detection results in the actual detection results is counted, and the target loss rate is obtained by dividing it by the total number of targets in the actual detection results; then, the intersection-union ratio (IUU) difference between the actual bounding box and the predicted bounding box in all successfully matched target pairs is calculated, and the average of all differences is taken to obtain the localization offset; finally, the difference between the actual confidence score and the predicted confidence score in all matched target pairs is calculated, and the average of all differences is taken to obtain the confidence reduction. Finally, a three-dimensional task performance loss vector is obtained, which quantifies the performance loss caused by compression distortion to downstream defect detection tasks.

[0039] S50: The task performance loss vector is weighted and fused according to the preset weight coefficients to obtain a comprehensive task performance score; In this embodiment, based on the requirements of UAV inspection defect detection tasks, different weight coefficients are pre-set for target loss rate, positioning offset, and confidence reduction, and the loss values ​​of the three dimensions are weighted and summed to obtain the comprehensive task performance score corresponding to the compressed image to be evaluated. The higher the comprehensive score, the greater the performance loss of the compressed image to the downstream detection task and the worse the image quality; conversely, the lower the comprehensive score, the better the image quality. This realizes a targeted no-reference image quality evaluation for UAV inspection defect detection tasks.

[0040] Specifically, step S50 in the method includes: The task performance loss vector is weighted and fused according to preset weight coefficients to obtain a comprehensive task performance score, including: Set a first weight coefficient, a second weight coefficient, and a third weight coefficient, and the sum of the first weight coefficient, the second weight coefficient, and the third weight coefficient is 1; Calculate the weighted sum, which is equal to the first weight coefficient multiplied by the target loss rate, plus the second weight coefficient multiplied by the positioning offset, plus the third weight coefficient multiplied by the confidence decrease. Calculate the overall task performance score, which is equal to 1 minus the weighted sum.

[0041] In this embodiment, firstly, a first weight coefficient, a second weight coefficient, and a third weight coefficient are preset, and the sum of the three is 1. The weights can be adjusted according to the importance of the impact of different loss dimensions in the defect detection task. For example, the first weight coefficient, i.e. the weight corresponding to the target loss rate, can be appropriately increased in scenarios where the impact of missed defect detection is greater, so as to more accurately reflect the task requirements.

[0042] Subsequently, the total loss weighted sum is calculated according to the weighted sum formula. Finally, by subtracting the weighted sum from 1, a comprehensive task performance score ranging from 0 to 1 is obtained. The closer the score is to 1, the smaller the task performance loss and the better the compressed image quality. The closer the score is to 0, the greater the task performance loss and the worse the compressed image quality. This intuitively realizes the quantitative evaluation of task-oriented compressed image quality.

[0043] For example, the first, second, and third weighting coefficients are preset to 0.5, 0.3, and 0.2, respectively. At this setting, the target loss rate has the highest weight on the mission performance, which aligns with the mission requirement that missed defects in UAV inspections can lead to serious safety hazards. The corresponding formula for calculating the comprehensive mission performance score is: If the target loss rate of the compressed image to be evaluated is 0.1, the localization offset is 0.08, and the confidence decrease is 0.12, then the final comprehensive task performance score is: This indicates that the compressed image has minimal performance loss for downstream defect detection tasks, and the compressed image quality is high.

[0044] S60: Map the overall task performance score to a preset quality level and output the quality level of the compressed image to be evaluated.

[0045] In this embodiment, multiple threshold ranges are preset, with different threshold ranges corresponding to different quality levels. By matching the calculated comprehensive task performance score to the corresponding threshold range, the quality level of the compressed image to be evaluated can be obtained. Subsequently, the compression parameters can be adaptively adjusted according to the quality level. For example, when the score is lower than the preset threshold, the drone is controlled to reduce the compression ratio and re-encode and compress the current inspection image to ensure that the image quality meets the requirements of the downstream defect detection task and avoids missed or false defects due to excessive compression distortion.

[0046] The mapping of the overall task performance score to a preset quality level includes: A first scoring threshold, a second scoring threshold, and a third scoring threshold are preset, wherein the first scoring threshold is greater than the second scoring threshold, and the second scoring threshold is greater than the third scoring threshold; When the overall task performance score is greater than or equal to the first score threshold, the quality level of the compressed image to be evaluated is determined to be Grade A, and the compressed image to be evaluated is directly sent to the downstream defect detection module for processing. When the overall task performance score is greater than or equal to the second score threshold and less than the first score threshold, the quality level of the compressed image to be evaluated is determined to be grade B, and the compressed image to be evaluated is sent to the downstream defect detection module. After the detection is completed, the detection results with a detection confidence level lower than the preset detection confidence threshold are marked as recommended for manual re-inspection. When the overall task performance score is greater than or equal to the third score threshold and less than the second score threshold, the quality level of the compressed image to be evaluated is determined to be C, and the compressed image to be evaluated is marked as requiring manual review. At the same time, the drone hovers and waits for the review instruction. When the overall task performance score is less than the third score threshold, the quality level of the compressed image to be evaluated is determined to be level D, and the UAV is triggered to re-acquire the original high-definition image of the target area, and the compression quality parameters are increased before re-compression and transmission.

[0047] In this embodiment of the application, firstly, three graded thresholds are set in advance according to the tolerance of the inspection task for image quality. For example, the first score threshold can be set to 0.85, the second score threshold can be set to 0.7, and the third score threshold can be set to 0.5.

[0048] Secondly, when the overall task performance score is greater than or equal to the first score threshold, it indicates that the compressed image is of excellent quality and has almost no impact on the downstream defect detection task. Therefore, it is directly judged as quality level A, and the compressed image can be sent directly to the downstream defect detection module for automatic processing without additional manual intervention.

[0049] Furthermore, when the overall task performance score is greater than or equal to the second score threshold and less than the first score threshold, it indicates that the compressed image is of good quality and will only affect some low-confidence detection results. Therefore, it is judged to be of quality level B. After being sent to the downstream automatic detection, only the results with confidence levels lower than the preset detection confidence threshold need to be manually re-inspected, thus ensuring the reliability of the detection results while controlling the amount of transmission.

[0050] Furthermore, when the overall task performance score is greater than or equal to the third score threshold and less than the second score threshold, it indicates that the compressed image quality is poor, the compression distortion has had a significant impact on the defect detection results, and the reliability of the automatic detection results is insufficient. Therefore, it is judged to be of quality level C, marked as requiring manual review, and the drone is allowed to hover and wait for the review instruction to avoid the risk of missing defects.

[0051] Finally, when the overall task performance score is less than the third score threshold, it indicates that the quality of the compressed image can no longer meet the basic requirements for defect detection and the loss of task performance is too great. Therefore, it is judged to be of quality level D, and the UAV is directly triggered to re-acquire the original high-definition image of the target area, and the compression quality parameters are increased before re-compression and transmission to ensure the detection accuracy of the inspection task.

[0052] Furthermore, the Transformer-based method for evaluating the quality of compressed images used in UAV inspections also includes: When the transmission bandwidth is lower than a preset bandwidth threshold, a high-dimensional feature codebook of the original high-definition image is pre-stored on the UAV terminal. The high-dimensional feature codebook is obtained by compressing the original high-definition image by an autoencoder. During the online evaluation phase, only the compressed image to be evaluated and the high-dimensional feature codebook are loaded. The compressed image to be evaluated is input into the compressed image stream branch of the dual-stream Transformer feature extraction network, and the high-dimensional feature codebook is decoded and used as an approximate reference feature input into the original image stream branch. Then, the steps of calculating the task performance loss vector, weighted fusion to obtain the comprehensive task performance score, and mapping to the quality level are executed sequentially.

[0053] In this embodiment, in scenarios where the transmission bandwidth of the UAV is limited, the entire original high-definition image is not transmitted back to the back-end evaluation module. Only the high-dimensional feature codebook of the original high-definition image compressed by the autoencoder needs to be transmitted, which greatly reduces the amount of original reference information transmitted and reduces bandwidth usage. At the same time, the features extracted from the original high-definition image are replaced by the decoded approximate reference features, which can adapt to the limited transmission bandwidth conditions of the UAV while ensuring the accuracy of quality evaluation.

[0054] First, when the transmission bandwidth is lower than a preset bandwidth threshold, a high-dimensional feature codebook of the original high-definition image is pre-stored on the UAV. The high-dimensional feature codebook is obtained by compressing the original high-definition image by an autoencoder. This autoencoder is pre-trained with a large number of original high-definition images from UAV inspections and can achieve high-ratio compression while preserving the key semantic features of the original image, effectively reducing feature storage and transmission overhead.

[0055] Secondly, in the online evaluation stage, it is not necessary to transmit the entire original high-definition image to the evaluation end. Only the compressed image to be evaluated and the compressed high-dimensional feature codebook need to be transmitted. Then, the compressed image to be evaluated is input into the compressed image stream branch of the dual-stream Transformer feature extraction network. After decoding the high-dimensional feature codebook, it is used as an approximate reference feature input into the original image stream branch. The task performance loss vector can be calculated and weighted fusion can be performed in sequence according to the aforementioned process to obtain the comprehensive task performance score. Finally, the quality level of the compressed image to be evaluated is mapped and output. The above steps can reduce the bandwidth occupation of the original reference information transmission in the bandwidth-constrained inspection scenario, while ensuring the accuracy of quality evaluation for downstream defect detection tasks, and adapting to different network transmission conditions of UAV inspection.

[0056] In summary, compared with existing technologies, this application introduces a dual-stream Transformer structure to extract cross-scale alignment features between the original high-definition image and the compressed image to be evaluated. It guides the network to focus on semantic regions key to downstream defect detection tasks through task-aware attention. At the same time, it quantifies the impact of compressed images on task performance from three dimensions: target loss rate, localization offset, and confidence reduction. Finally, it obtains a task-oriented no-reference image quality score through weighted fusion. This solves the problem that traditional general image quality evaluation indicators cannot accurately reflect the actual impact of compressed images on UAV inspection defect detection tasks. It can provide compressed image quality evaluation results that are more in line with task requirements for UAV inspection scenarios, effectively balancing image transmission bandwidth usage and downstream defect detection accuracy, and adapting to the needs of UAV inspection operations under different transmission bandwidth conditions.

[0057] In summary, the embodiments of this application have at least the following technical effects: This application provides a Transformer-based method for evaluating the quality of compressed images used in UAV inspections. First, a lightweight task agent network is employed to simulate the performance changes of downstream defect detection tasks, measuring the impact of compressed images on inspection effectiveness from a task performance perspective, thus obtaining evaluation results that better reflect the actual task requirements of UAV inspections. Second, a lightweight task agent network is obtained through knowledge distillation, with a parameter count far smaller than the benchmark detection model, effectively reducing the computational load of the evaluation process and making it more suitable for deployment on edge devices such as UAVs. Finally, the interaction features between the original and compressed images are extracted using a dual-stream Transformer structure, and task-aware attention is introduced to enhance the weight of task-related features, improving the accuracy of the evaluation results. Furthermore, the obtained quality level can directly support automated decision-making in subsequent inspection processes, adapting to the compressed image transmission and processing needs under different bandwidth scenarios.

[0058] Through the above technical solution, this application solves the problem of low correlation between traditional pixel-level quality evaluation results and downstream defect detection tasks, overcomes the shortcomings of existing task-oriented evaluation methods with large model parameters and difficulty in adapting to edge deployment, and can output compressed image quality evaluation results that can directly support process decisions for UAV inspection scenarios. It balances compression ratio and downstream defect detection accuracy under limited bandwidth conditions, and improves the overall work efficiency of UAV inspection.

[0059] Example 2, as Figure 2 As shown, based on the same inventive concept as the Transformer-based UAV inspection compressed image quality evaluation method provided in Embodiment 1, this application also provides a Transformer-based UAV inspection compressed image quality evaluation system, including: The initial image acquisition module 11 is used to load the original high-definition image and the corresponding compressed image to be evaluated. The detection result prediction module 12 is used to input the compressed image to be evaluated into the task agent network to obtain the predicted detection result. The task agent network is a lightweight target detection network used to simulate downstream defect detection tasks. Detection result loading module 13 is used to load the real detection results of the original high-definition image, which are obtained by inference from the original high-definition image by the benchmark detection model; Loss vector calculation module 14 is used to calculate the task performance loss vector based on the predicted detection result and the actual detection result; The comprehensive score acquisition module 15 is used to perform weighted fusion of the task performance loss vector according to preset weight coefficients to obtain a comprehensive task performance score. The quality level output module 16 is used to map the comprehensive score of task performance to a preset quality level and output the quality level of the compressed image to be evaluated.

[0060] Furthermore, in one embodiment of the application, the task performance loss vector includes target loss rate, positioning offset, and confidence decrease; Wherein, the target loss rate is the proportion of targets in the actual detection results that are not detected in the predicted detection results, the positioning offset is the average offset between the bounding box position and size of each matched detection target pair, and the confidence decrease is the average decrease in the confidence score of each matched detection target pair.

[0061] Furthermore, in one embodiment, the compressed image to be evaluated is input into a task agent network to obtain a prediction detection result, prior to which the following steps are taken: Collect a set of original inspection images in the scenario of drone inspection. The set of original inspection images contains multiple original high-definition images and corresponding defect target annotations. For each original high-resolution image, multiple lossy compression algorithms and multiple compression quality parameters are used to generate multiple compressed images to be evaluated, forming an image pair training set; The original high-definition images in the original inspection image set are input into the benchmark detection model to obtain the true detection result of each original high-definition image; The image pairs are input into the benchmark detection model along with the compressed images to be evaluated in the training set to obtain the benchmark detection result for each compressed image to be evaluated. Calculate the task performance loss label between the benchmark detection result and the corresponding real detection result for each compressed image to be evaluated; Using the original high-resolution images and the compressed images to be evaluated in the image pair training set as input, and the corresponding task performance loss labels as supervision, a two-stream Transformer feature extraction network is trained, and the training loss function is the mean square error between the predicted task performance loss vector and the task performance loss label.

[0062] Furthermore, training the two-stream Transformer feature extraction network also includes training a task agent network: A lightweight object detection network is constructed as the task proxy network; The benchmark detection model is used to generate pseudo-labels for the original high-definition images in the original inspection image set; Using the original high-definition image as input and the pseudo-label as supervision, the task agent network is trained through knowledge distillation. The training aims to minimize the divergence between the output distribution of the task agent network and the output distribution of the benchmark detection model, and the number of parameters of the task agent network is less than a preset ratio threshold of the number of parameters of the benchmark detection model. The parameters of the pre-trained task agent network are fixed.

[0063] Furthermore, the dual-stream Transformer feature extraction network has an asymmetric structure, including a raw image stream branch and a compressed image stream branch; The original image stream branch contains L1 Transformer coding blocks, and the compressed image stream branch contains L2 Transformer coding blocks, where L1 is greater than L2. The original image stream branch and the compressed image stream branch are equipped with a cross-attention module after the K-th layer coding block to realize information interaction between the feature map of the original image stream and the feature map of the image stream to be evaluated, where K is less than or equal to L2.

[0064] In one embodiment, the comprehensive score acquisition module 15 is specifically used for: The task performance loss vector is weighted and fused according to preset weight coefficients to obtain a comprehensive task performance score, including: Set a first weight coefficient, a second weight coefficient, and a third weight coefficient, and the sum of the first weight coefficient, the second weight coefficient, and the third weight coefficient is 1; Calculate the weighted sum, which is equal to the first weight coefficient multiplied by the target loss rate, plus the second weight coefficient multiplied by the positioning offset, plus the third weight coefficient multiplied by the confidence decrease. Calculate the overall task performance score, which is equal to 1 minus the weighted sum.

[0065] Furthermore, the overall task performance score is mapped to a preset quality level, including: A first scoring threshold, a second scoring threshold, and a third scoring threshold are preset, wherein the first scoring threshold is greater than the second scoring threshold, and the second scoring threshold is greater than the third scoring threshold; When the overall task performance score is greater than or equal to the first score threshold, the quality level of the compressed image to be evaluated is determined to be Grade A, and the compressed image to be evaluated is directly sent to the downstream defect detection module for processing. When the overall task performance score is greater than or equal to the second score threshold and less than the first score threshold, the quality level of the compressed image to be evaluated is determined to be grade B, and the compressed image to be evaluated is sent to the downstream defect detection module. After the detection is completed, the detection results with a detection confidence level lower than the preset detection confidence threshold are marked as recommended for manual re-inspection. When the overall task performance score is greater than or equal to the third score threshold and less than the second score threshold, the quality level of the compressed image to be evaluated is determined to be C, and the compressed image to be evaluated is marked as requiring manual review. At the same time, the drone hovers and waits for the review instruction. When the overall task performance score is less than the third score threshold, the quality level of the compressed image to be evaluated is determined to be level D, and the UAV is triggered to re-acquire the original high-definition image of the target area, and the compression quality parameters are increased before re-compression and transmission.

[0066] In one embodiment, the Transformer-based UAV inspection compressed image quality assessment method further includes: On the deep feature map of the dual-stream Transformer feature extraction network, the feature response map output by the intermediate layer of the task agent network is loaded as the task-aware attention weight. The task-aware attention weights are multiplied element-wise with the original image stream feature map and the image stream feature map to be evaluated to obtain the weighted original feature map and the weighted feature map to be evaluated. The weighted original feature map and the weighted feature map to be evaluated are used as inputs to subsequent hierarchical coding blocks.

[0067] Furthermore, it also includes: When the transmission bandwidth is lower than a preset bandwidth threshold, a high-dimensional feature codebook of the original high-definition image is pre-stored on the UAV terminal. The high-dimensional feature codebook is obtained by compressing the original high-definition image by an autoencoder. During the online evaluation phase, only the compressed image to be evaluated and the high-dimensional feature codebook are loaded. The compressed image to be evaluated is input into the compressed image stream branch of the dual-stream Transformer feature extraction network, and the high-dimensional feature codebook is decoded and used as an approximate reference feature input into the original image stream branch. Then, the steps of calculating the task performance loss vector, weighted fusion to obtain the comprehensive task performance score, and mapping to the quality level are executed sequentially.

Claims

1. A method for evaluating the quality of compressed images from UAV inspections based on Transformer, characterized in that, include: Load the original high-resolution image and the corresponding compressed image to be evaluated; The compressed image to be evaluated is input into the task agent network to obtain the predicted detection result. The task agent network is a lightweight target detection network used to simulate downstream defect detection tasks. Load the actual detection results of the original high-definition image, which are obtained by inference from the original high-definition image using a benchmark detection model; Calculate the task performance loss vector based on the predicted detection results and the actual detection results; The task performance loss vector is weighted and fused according to preset weight coefficients to obtain a comprehensive task performance score. The overall performance score of the task is mapped to a preset quality level, and the quality level of the compressed image to be evaluated is output.

2. The method as described in claim 1, characterized in that, The task performance loss vector includes target loss rate, positioning offset, and confidence decrease. Wherein, the target loss rate is the proportion of targets in the actual detection results that are not detected in the predicted detection results, the positioning offset is the average offset between the bounding box position and size of each matched detection target pair, and the confidence decrease is the average decrease in the confidence score of each matched detection target pair.

3. The method as described in claim 1, characterized in that, The compressed image to be evaluated is input into the task agent network to obtain the prediction detection result, which includes the following steps: Collect a set of original inspection images in the scenario of drone inspection. The set of original inspection images contains multiple original high-definition images and corresponding defect target annotations. For each original high-resolution image, multiple lossy compression algorithms and multiple compression quality parameters are used to generate multiple compressed images to be evaluated, forming an image pair training set; The original high-definition images in the original inspection image set are input into the benchmark detection model to obtain the true detection result of each original high-definition image; The image pairs are input into the benchmark detection model along with the compressed images to be evaluated in the training set to obtain the benchmark detection result for each compressed image to be evaluated. Calculate the task performance loss label between the benchmark detection result and the corresponding real detection result for each compressed image to be evaluated; Using the original high-resolution images and the compressed images to be evaluated in the image pair training set as input, and the corresponding task performance loss labels as supervision, a two-stream Transformer feature extraction network is trained, and the training loss function is the mean square error between the predicted task performance loss vector and the task performance loss label.

4. The method as described in claim 3, characterized in that, Training the two-stream Transformer feature extraction network also includes training the task agent network: A lightweight object detection network is constructed as the task proxy network; The benchmark detection model is used to generate pseudo-labels for the original high-definition images in the original inspection image set; Using the original high-definition image as input and the pseudo-label as supervision, the task agent network is trained through knowledge distillation. The training aims to minimize the divergence between the output distribution of the task agent network and the output distribution of the benchmark detection model, and the number of parameters of the task agent network is less than a preset ratio threshold of the number of parameters of the benchmark detection model. The parameters of the pre-trained task agent network are fixed.

5. The method as described in claim 3, characterized in that, The dual-stream Transformer feature extraction network has an asymmetric structure, including a raw image stream branch and a compressed image stream branch; The original image stream branch contains L1 Transformer coding blocks, and the compressed image stream branch contains L2 Transformer coding blocks, where L1 is greater than L2. The original image stream branch and the compressed image stream branch are equipped with a cross-attention module after the K-th layer coding block to realize information interaction between the feature map of the original image stream and the feature map of the image stream to be evaluated, where K is less than or equal to L2.

6. The method as described in claim 2, characterized in that, The task performance loss vector is weighted and fused according to preset weight coefficients to obtain a comprehensive task performance score, including: Set a first weight coefficient, a second weight coefficient, and a third weight coefficient, and the sum of the first weight coefficient, the second weight coefficient, and the third weight coefficient is 1; Calculate the weighted sum, which is equal to the first weight coefficient multiplied by the target loss rate, plus the second weight coefficient multiplied by the positioning offset, plus the third weight coefficient multiplied by the confidence decrease. Calculate the overall task performance score, which is equal to 1 minus the weighted sum.

7. The method as described in claim 6, characterized in that, Mapping the overall task performance score to a preset quality level includes: A first scoring threshold, a second scoring threshold, and a third scoring threshold are preset, wherein the first scoring threshold is greater than the second scoring threshold, and the second scoring threshold is greater than the third scoring threshold; When the overall task performance score is greater than or equal to the first score threshold, the quality level of the compressed image to be evaluated is determined to be Grade A, and the compressed image to be evaluated is directly sent to the downstream defect detection module for processing. When the overall task performance score is greater than or equal to the second score threshold and less than the first score threshold, the quality level of the compressed image to be evaluated is determined to be grade B, and the compressed image to be evaluated is sent to the downstream defect detection module. After the detection is completed, the detection results with a detection confidence level lower than the preset detection confidence threshold are marked as recommended for manual re-inspection. When the overall task performance score is greater than or equal to the third score threshold and less than the second score threshold, the quality level of the compressed image to be evaluated is determined to be C, and the compressed image to be evaluated is marked as requiring manual review. At the same time, the drone hovers and waits for the review instruction. When the overall task performance score is less than the third score threshold, the quality level of the compressed image to be evaluated is determined to be level D, and the UAV is triggered to re-acquire the original high-definition image of the target area, and the compression quality parameters are increased before re-compression and transmission.

8. The method as described in claim 3, characterized in that, Also includes: On the deep feature map of the dual-stream Transformer feature extraction network, the feature response map output by the intermediate layer of the task agent network is loaded as the task-aware attention weight. The task-aware attention weights are multiplied element-wise with the original image stream feature map and the image stream feature map to be evaluated to obtain the weighted original feature map and the weighted feature map to be evaluated. The weighted original feature map and the weighted feature map to be evaluated are used as inputs to subsequent hierarchical coding blocks.

9. The method as described in claim 1, characterized in that, Also includes: When the transmission bandwidth is lower than a preset bandwidth threshold, a high-dimensional feature codebook of the original high-definition image is pre-stored on the UAV terminal. The high-dimensional feature codebook is obtained by compressing the original high-definition image by an autoencoder. During the online evaluation phase, only the compressed image to be evaluated and the high-dimensional feature codebook are loaded. The compressed image to be evaluated is input into the compressed image stream branch of the dual-stream Transformer feature extraction network, and the high-dimensional feature codebook is decoded and used as an approximate reference feature input into the original image stream branch. Then, the steps of calculating the task performance loss vector, weighted fusion to obtain the comprehensive task performance score, and mapping to the quality level are executed sequentially.

10. A Transformer-based UAV inspection compressed image quality evaluation system, characterized in that, The method for performing the Transformer-based UAV inspection compressed image quality assessment method according to any one of claims 1-9 includes: The initial image acquisition module is used to load the original high-definition image and the corresponding compressed image to be evaluated. The detection result prediction module is used to input the compressed image to be evaluated into the task agent network to obtain the predicted detection result. The task agent network is a lightweight target detection network used to simulate downstream defect detection tasks. The detection result loading module is used to load the real detection results of the original high-definition image, which are obtained by inference from the original high-definition image by the benchmark detection model. The loss vector calculation module is used to calculate the task performance loss vector based on the predicted detection results and the actual detection results. The comprehensive score acquisition module is used to perform weighted fusion of the task performance loss vector according to preset weight coefficients to obtain a comprehensive task performance score. The quality level output module is used to map the comprehensive score of task performance to a preset quality level and output the quality level of the compressed image to be evaluated.

Images