A real-time transmission method of high-definition images from drones for power inspection

Abstract

The present invention belongs to the technical field of unmanned aerial vehicles, and specifically relates to a method for real-time transmission of high-definition images of unmanned aerial vehicles for power inspection, which includes step 1: reading the power inspection video taken by the unmanned aerial vehicle; step 2: degrading and encoding the video frame; step 3: converting the encoded output frame into a bit stream and transmitting the encoded video stream to a cloud server using 5G transmission; step 4: the cloud server decodes the video stream; step 5: constructing a super-resolution reconstruction network, super-resolution reconstruction of the decoded degraded image, outputting the restored video frame blocks and merging them into video frames, and completing the video recovery; step 6: the cloud server pushes the restored video to obtain the power inspection video stream. The present invention reduces transmission delay and improves transmission speed.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicles, and in particular relates to a method for real-time transmission of high-definition images of unmanned aerial vehicles used for power inspection. Background Art

[0002] With the record-breaking electricity consumption and the rapid development of the power grid, traditional manual inspections have exposed shortcomings such as low efficiency and difficulty in ensuring personnel safety. To solve this problem, it is necessary to apply drone technology to power inspections and realize the application of drone power autonomous inspections through further research on drone technology, effectively improve work efficiency, save resources, and promote the sustainable development of the power grid.

[0003] With the development of science and technology, people have higher and higher requirements for the image resolution of drone image transmission systems. For example, in video aerial photography applications, high-definition videos can provide users with a better viewing experience. In drone power repair and inspection scenarios, higher-definition video images can enable rescue personnel to understand the situation on the scene more clearly and make correct judgments. The increase in image resolution will inevitably lead to an increase in image size. Taking the current mainstream ultra-high-definition resolution 4K as an example, the volume of a frame of RGB image is about 23M. If the video frame rate is 30FPS, the amount of video data per second is as high as 690M. If it is transmitted directly without processing, it will cause huge pressure on the communication link. Therefore, it is very necessary to compress the video, and video compression coding technology has always been a hot research topic. H264 technology is the most widely used video coding standard in the past decade, but as people's requirements for video clarity increase, continuing to use the H264 standard for ultra-clear video compression coding has become increasingly difficult;

[0004] In addition, the communication strategy between the drone image transmission system and the ground is also constantly changing. The communication methods of drone image transmission are mainly divided into point-to-point communication and cellular network communication. The communication distance of point-to-point communication is limited, and 2.4G or 5.8G wireless frequency bands are usually used. Cellular communication is a mobile communication architecture. The communication base stations in the network coverage area are arranged in a hexagonal honeycomb shape. The terminal equipment and the base station exchange data through the wireless network. The terminal can move freely in the signal coverage area. However, the traditional 3G and 4G networks cannot meet the requirements of real-time transmission of ultra-clear video due to problems such as limited bandwidth and large latency. Summary of the invention

[0005] The purpose of the present invention is to provide a method for real-time transmission of high-definition images of unmanned aerial vehicles for power inspection, thereby reducing transmission delay and improving transmission speed.

[0006] In order to solve the above technical problems, the technical solution of the present invention is:

[0007] A method for real-time transmission of high-definition images of unmanned aerial vehicles for power inspection, comprising:

[0008] Step 1: Read the power inspection video taken by the drone;

[0009] Step 2: Encode the video frame after degrading it;

[0010] Step 3: Convert the encoded output frame into a bit stream and transmit the encoded video stream to the cloud server using 5G transmission; Step 4: The cloud server decodes the video stream;

[0011] Step 5: Build a super-resolution reconstruction network, perform super-resolution reconstruction on the decoded degraded image, output the restored video frame blocks and merge them into video frames to complete video restoration;

[0012] Step 6: The cloud server streams the restored video to obtain the power inspection video stream.

[0013] Furthermore, in step 2, the encoding method is: adjust the encoding method of the H.265 encoder, first divide the image into image parts to be encoded in units of 64x64; establish a correspondence between the brightness value and the encoding block, and select the corresponding encoding block for image encoding based on the brightness value; the correspondence between the brightness value and the encoding block is: the encoding block size corresponding to the brightness value 0-63 is 64x64, the encoding block size corresponding to the brightness value 64-127 is 32x32, the encoding block size corresponding to the brightness value 128-191 is 16x16, and the encoding block size corresponding to the brightness value 192-255 is 8x8.

[0014] Furthermore, the step 3 specifically includes:

[0015] Step 301: Convert the output frame of the power inspection video frame encoding into a bit stream using an Nvidia encoder, and output it to an RTSP transmission pipeline;

[0016] Step 302: using the Micro Air Vehiclelink protocol suitable for UAV communication at the small computer application layer to transmit the power inspection video frame, and using the UDP protocol at the transport layer to transmit;

[0017] Step 303: Use the 5G module to transmit the encoded video stream.

[0018] Furthermore, in step 4, the cloud server decodes the video stream using an H.265 decoder.

[0019] Furthermore, step 5 specifically includes:

[0020] Step 501: Slice the degraded image to form N blocks;

[0021] Step 502: Use the Laplace operator to perform edge detection on the N blocks, extract edge information of the image blocks, and calculate edge scores;

[0022] Step 503: construct a lookup table of edge information corresponding to peak signal-to-noise ratio; obtain the corresponding peak signal-to-noise ratio according to the edge information of the image block, and thus select a sub-network;

[0023] Step 504: reconstruct the image block according to the selected sub-network, output the restored video frame block, and finally merge the video frame blocks into a complete video frame to complete the video restoration.

[0024] Furthermore, in step 502, the edge score is an average value of each pixel in the image after edge detection.

[0025] Furthermore, the step 503 specifically includes:

[0026] Step 5031: Pre-build a lookup table from edge information to peak signal-to-noise ratio:

[0027]

[0028] Its meaning is the jth lookup table t j The edge score of the image block is mapped to the peak signal-to-noise ratio of the corresponding subnet to estimate the subnet The performance of j The weight of subnet j for the total network, α j ∈(0,1], M represents the number of tables; step 5031 specifically includes:

[0029] Step 50311: Calculate the edge information score set of all low-quality image blocks from the video frame:

[0030]

[0031] where e i is the edge score of the ith patch, O is the pixel size of the patch;

[0032] Step 50312: Evenly divide the edge score interval [min(E), max(E)] into a total of K sub-intervals The range of the kth subinterval is:

[0033]

[0034] Where min(·) and max(·) return the minimum and maximum elements of the input edge score set, respectively;

[0035] Step 50313: Calculate the peak signal-to-noise ratio of the low-quality image block in the corresponding subnet operation, as shown below:

[0036]

[0037] Wherein, the peak signal-to-noise ratio generated by using the j-th subnetwork to operate on the i-th image block; j is the corresponding subnetwork number, i is the number of the image block, and O is the size of the image block;

[0038] Step 50314: Calculate subnetwork based on subinterval segmentation The average peak signal-to-noise ratio in the kth subinterval is:

[0039]

[0040] where Γ(·) is an indicator that returns 1 if the input is true and 0 otherwise;

[0041] Step 50315: Obtain the corresponding lookup table t of the jth subnet j for

[0042]

[0043] That is, the final corresponding peak signal-to-noise ratio, where j is the corresponding subnet, k is the interval, and K is the total number of intervals. Step 5032: Input the edge score e of a given low-quality image block and calculate which subinterval k this image block belongs to. The calculation method of k is:

[0044]

[0045] in, is the floor function;

[0046] Step 5033: Obtain the subnet directly based on the subinterval k through the pre-built lookup table Estimated peak signal-to-noise ratio Select Estimate Peak Signal-to-Noise Ratio The top subnet reconstructs the image patch.

[0047] Furthermore, in step 5033, the computation cost of each subnet is introduced as a subnet performance evaluation factor; first, a set of computation costs is calculated:

[0048]

[0049] Among them, c j represents the computational overhead corresponding to the jth subnet, and M is the total number of subnets;

[0050] Then, a computation-performance trade-off function is constructed based on the estimated peak signal-to-noise ratio and the subnet computation cost to evaluate the subnet performance;

[0051]

[0052] Among them, j' represents the subnet number with the best performance when considering the subnet calculation cost; argmax represents the following function The value of j when the maximum value is obtained is the subnet with the best performance; λ is a hyperparameter used to balance the numerical difference between the computational cost in the inference phase and the peak signal-to-noise ratio estimation.

[0053] The present invention has the following beneficial effects:

[0054] 1. The present invention uses the fifth generation mobile communication technology, namely 5G technology, to achieve remote, high-definition transmission of drone images during power inspections, which can effectively solve the problems of limited bandwidth and communication delay in the image transmission process, so that users can obtain the "first-person perspective" image of the drone in real time without distance restrictions within the coverage of the 5G signal, thereby increasing the range of drone image transmission and reducing the transmission delay of the drone;

[0055] 2. The present invention uses the H.265 (HEVC) standard video coding technology with higher compression efficiency when encoding images, and makes certain adjustments to the encoding method of the H.265 encoder. It selects a suitable coding block size according to the complexity of the image to avoid distortion, speed up the encoding speed, and also improve the overall transmission speed.

[0056] 3. The present invention reduces the volume of the video stream transmitted by the drone and further improves the speed of image transmission by the drone by first degrading the image and then restoring it in real time in the cloud using a highly real-time super-resolution reconstruction algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 This is a flow chart of the method for real-time transmission of high-definition images of unmanned aerial vehicles for power inspection of the present invention;

[0058] Figure 2 Schematic diagram of the structure of a super-resolution reconstruction network according to an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0060] Please refer to Figure 1 The present invention provides a method for real-time transmission of high-definition images of drones for power inspection, which can ensure the real-time and clarity of the video during drone power inspection, and includes the following steps:

[0061] Step 1: Read the power inspection video taken by the drone;

[0062] Step 2: Encode the video frame after degrading it;

[0063] Step 3: Convert the encoded output frame into a bit stream and transmit the encoded video stream to the cloud server using 5G transmission; Step 4: The cloud server decodes the video stream;

[0064] Step 5: Build a super-resolution reconstruction network, perform super-resolution reconstruction on the decoded degraded image, output the restored video frame blocks and merge them into video frames to complete video restoration;

[0065] Step 6: The cloud server pushes the restored video to obtain a power inspection video stream, so that the inspection personnel can obtain a clear power inspection video stream in real time.

[0066] In step 1, the video stream of the power inspection video is captured by a high-definition camera of a drone, and the Nvidia Jetson Nano small computer is used as a video stream processing module on the inspection drone;

[0067] In step 2, the video encoder is initialized in the video stream processing module. In this embodiment, H.265 is used to encode the video, and GPU acceleration is turned on. The video frame image obtained from the drone camera is downsampled and then encoded to improve the real-time transmission rate of the video, and finally the encoded output frame is output.

[0068] Specifically, in step 2, the encoding method of the H.265 encoder is adjusted to a certain extent. In the standard H.265 encoding algorithm, there are four encoding block sizes: 64x64, 32x 32, 16x16, and 8×8. The size division of the encoding block has a lot to do with the complexity of the image part. The more complex the image part is, the smaller the encoding block is required to avoid distortion in image encoding and decoding; correspondingly, smaller encoding blocks require more encoding time. Most of the images taken by drones during power inspections are overhead views, and the complexity of each part of the image varies greatly. Therefore, it is necessary to select an appropriate encoding block size according to the complexity of the image to avoid distortion and speed up encoding.

[0069] In the RGB image of a certain frame of the video, the brightness is defined as Y=0.299R+0.587G+0.114B. First, the image is divided into 64x64 units into image parts to be encoded. The brightness variance is used to measure the complexity of the current image part to be encoded. If the brightness variance is large, it means that the current image is more complex and a smaller coding block needs to be selected. Establish a corresponding relationship between the brightness value and the coding block, and select the corresponding coding block for image encoding based on the brightness value;

[0070] The corresponding relationship between brightness value and coding block is: the corresponding coding block size of brightness value 0-63 is 64x64, the corresponding coding block size of brightness value 64-127 is 32x32, the corresponding coding block size of brightness value 128-191 is 16x16, and the corresponding coding block size of brightness value 192-255 is 8x8.

[0071] When encoding an image, a suitable encoding block size is selected for each part according to the above method, thereby reducing the amount of calculation in the encoding process and shortening the encoding time.

[0072] Step 3 specifically includes:

[0073] Step 301: Convert the output frame of the power inspection video frame encoding into a bit stream using an Nvidia encoder, and output it to an RTSP transmission pipeline;

[0074] Step 302: using the Micro Air Vehiclelink protocol suitable for UAV communication at the small computer application layer to transmit the power inspection video frame, and using the UDP protocol at the transport layer to ensure the real-time nature of the video stream;

[0075] Step 303: Use the 5G module to transmit the encoded video stream. In this embodiment, Huawei MH5000 industrial module is used for 5G transmission, the network mode is SA, the maximum downlink speed can reach 2Gbps, and China Mobile is selected as the communication operator to achieve full coverage of 5G base stations within the power inspection range.

[0076] In step 4, a streaming media server is built on the cloud server for video reception, processing and streaming. The cloud server uses Tencent Cloud Server with a bandwidth of 1000Mbps. The cloud server opens port 80 for video reception and port 90 for video streaming. After receiving the video, the cloud server decodes the video stream using the H.265 decoder.

[0077] Step 5 is used to perform super-resolution reconstruction on the received degraded image to restore the clarity of the video. The process is as follows: Figure 2 As shown. Step 5 builds a super-resolution reconstruction network with high real-time performance based on CNN, which is suitable for the real-time image transmission scenario of drones. The network is denoted as N W[0:1.0] , whose weights are represented as W[0:1.0], where 1.0 represents the width multiplier of each convolutional layer. For example, a subnet is represented as N W[0:0.5] , at this time, the width of each convolutional layer is network N W[0:1.0] By setting different width multipliers, a super-resolution reconstruction sub-network set can be obtained:

[0078]

[0079] where α j ∈(0,1] is the width multiplier of the jth sub-network, and M is the number of sub-networks.

[0080] The calculation process of the super-resolution reconstruction network specifically includes:

[0081] Step 501: Slice the degraded image to form N image patches;

[0082] Step 502: Use the Laplace operator to perform edge detection on the N patches, extract edge information of the image blocks, and calculate edge scores; the edge score is the average value of each pixel in the image after edge detection;

[0083] Step 503: Construct a lookup table of edge information corresponding to peak signal-to-noise ratio; obtain the corresponding peak signal-to-noise ratio according to the edge information of the image block, and then select the sub-network; the edge score has a strong correlation with the peak signal-to-noise ratio, and the sub-network constitutes the main network. This method uses this feature to Figure 2 In the main network in , the sub-network with the best inference performance is selected for inference, and the image block is super-resolution reconstructed to improve the real-time performance and restoration quality of the super-resolution network; Step 503 specifically includes: Step 5031: pre-constructing a lookup table from edge information to peak signal-to-noise ratio:

[0084]

[0085] Its meaning is the jth lookup table t j The edge score of the image block is mapped to the peak signal-to-noise ratio of the corresponding subnet to estimate the subnet The performance of j The weight of subnet j for the total network, α j ∈(0,1], M represents the number of tables; step 5031 specifically includes:

[0086] Step 50311: Calculate the edge information score set of all low-quality image blocks from the video frame:

[0087]

[0088] where e i is the edge score of the ith patch, O is the pixel size of the patch;

[0089] Step 50312: Evenly divide the edge score interval [min(E), max(E)] into a total of K sub-intervals The range of the kth subinterval is:

[0090]

[0091] Where min(·) and max(·) return the minimum and maximum elements of the input edge score set, respectively;

[0092] Step 50313: Calculate the peak signal-to-noise ratio of the low-quality image block in the corresponding subnet operation, as shown below:

[0093]

[0094] Wherein, the peak signal-to-noise ratio generated by using the j-th subnetwork to operate on the i-th image block; j is the corresponding subnetwork number, i is the number of the image block, and O is the size of the image block;

[0095] Step 50314: Calculate subnetwork based on subinterval segmentation The average peak signal-to-noise ratio in the kth subinterval is:

[0096]

[0097] where Γ(·) is an indicator that returns 1 if the input is true and 0 otherwise;

[0098] Step 50315: Obtain the corresponding lookup table t of the jth subnet j for

[0099]

[0100] That is, the final corresponding peak signal-to-noise ratio, where j is the corresponding subnet, k is the interval, and K is the total number of intervals. Step 5032: Input the edge score e of a given low-quality image block and calculate which subinterval k this image block belongs to. The calculation method of k is:

[0101]

[0102] in, is the floor function;

[0103] Step 5033: Obtain the subnet directly based on the subinterval k through the pre-built lookup table Estimated peak signal-to-noise ratio Peak signal-to-noise ratio The highest subnet is the subnet with good performance; this embodiment also introduces the calculation cost of each subnet as a subnet performance evaluation factor;

[0104] First, calculate a set of computational costs:

[0105]

[0106] Among them, c j represents the computational overhead corresponding to the jth subnet, and M is the total number of subnets;

[0107] Then, a computation-performance trade-off function is constructed based on the estimated peak signal-to-noise ratio and the subnet computation cost to evaluate the subnet performance;

[0108]

[0109] Among them, j' represents the subnet number with the best performance when considering the subnet calculation cost; argmax represents the following function The value of j when the maximum value is obtained is the subnet with the best performance; λ is a hyperparameter used to balance the numerical difference between the computational cost in the inference phase and the peak signal-to-noise ratio estimation.

[0110] Step 504: reconstruct the image block according to the selected sub-network, output the restored video frame block, and finally merge the high-quality video frame blocks into a complete high-quality video frame to complete the video restoration.

[0111] It should be understood by those skilled in the art that the embodiments of the present application can be provided as methods, systems or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0112] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0113] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0114] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0115] The parts not involved in the present invention are the same as the prior art or are implemented by using the prior art.

[0116] The above contents are further detailed descriptions of the present invention in combination with specific implementation methods, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A method for real-time transmission of high-definition images from drones for power inspection, characterized in that: include Step 1: Read the power inspection video taken by the drone; Step 2: Encode the video frame after degrading it; Step 3: Convert the encoded output frames into a bit stream and transmit the encoded video stream to the cloud server using 5G transmission; Step 4: The cloud server decodes the video stream; Step 5: Build a super-resolution reconstruction network, perform super-resolution reconstruction on the decoded degraded image, output the restored video frame blocks and merge them into video frames to complete video restoration; Step 6: The cloud server pushes the restored video to obtain the power inspection video stream; Step 5 specifically includes: Step 501: Slice the degraded image to form N blocks; Step 502: Use the Laplace operator to perform edge detection on the N blocks, extract edge information of the image blocks, and calculate edge scores; Step 503: construct a lookup table of edge information corresponding to peak signal-to-noise ratio; obtain the corresponding peak signal-to-noise ratio according to the edge information of the image block, and thus select a sub-network; Step 504: reconstruct the image block according to the selected sub-network, output the restored video frame block, and finally merge the video frame blocks into a complete video frame to complete the video restoration.

2. The method for real-time transmission of high-definition images of unmanned aerial vehicles for power inspection according to claim 1 is characterized in that: In step 2, the encoding method is as follows: the encoding method of the H.265 encoder is adjusted, and the image is first divided into image parts to be encoded in units of 64x64; a corresponding relationship between brightness value and encoding block is established, and the corresponding encoding block is selected based on the brightness value for image encoding; The corresponding relationship between brightness value and coding block is: the coding block size corresponding to brightness value 0-63 is 64x64, the coding block size corresponding to brightness value 64-127 is 32x32, the coding block size corresponding to brightness value 128-191 is 16x16, and the coding block size corresponding to brightness value 192-255 is 8x8.

3. The method for real-time transmission of high-definition images of unmanned aerial vehicles for power inspection according to claim 1 is characterized in that: The step 3 specifically includes: Step 301: Convert the output frame of the power inspection video frame encoding into a bit stream using an Nvidia encoder, and output it to an RTSP transmission pipeline; Step 302: using the Micro Air Vehicle link protocol suitable for UAV communication at the small computer application layer to transmit the power inspection video frame, and using the UDP protocol at the transport layer to transmit; Step 303: Use the 5G module to transmit the encoded video stream.

4. The method for real-time transmission of high-definition images of unmanned aerial vehicles for power inspection according to claim 1 is characterized in that: In step 4, the cloud server decodes the video stream using the H.265 decoder.

5. The method for real-time transmission of high-definition images of unmanned aerial vehicles for power inspection according to claim 1 is characterized in that: In step 502, the edge score is the average value of each pixel in the image after edge detection.

6. The method for real-time transmission of high-definition images of unmanned aerial vehicles for power inspection according to claim 1 is characterized in that: The step 503 specifically includes: Step 5031: Pre-build a lookup table from edge information to peak signal-to-noise ratio: Its meaning is the jth lookup table t j The edge score of the image block is mapped to the peak signal-to-noise ratio of the corresponding subnet to estimate the subnet The performance of j The weight of subnet j for the total network, α j ∈(0,1], M represents the number of tables; step 5031 specifically includes: Step 50311: Calculate the edge information score set of all low-quality image blocks from the video frame: where e i is the edge score of the ith patch, O is the pixel size of the patch; Step 50312: Evenly divide the edge score interval [min(E), max(E)] into a total of K sub-intervals The range of the kth subinterval is: Where min(·) and max(·) return the minimum and maximum elements of the input edge score set, respectively; Step 50313: Calculate the peak signal-to-noise ratio of the low-quality image block in the corresponding subnet operation, as shown below: Wherein, the peak signal-to-noise ratio generated by using the j-th subnetwork to operate on the i-th image block; j is the corresponding subnetwork number, i is the number of the image block, and O is the size of the image block; Step 50314: Calculate subnetwork based on subinterval segmentation The average peak signal-to-noise ratio in the kth subinterval is: where Γ(·) is an indicator that returns 1 if the input is true and 0 otherwise; Step 50315: Obtain the corresponding lookup table t of the jth subnet j for That is, the final corresponding peak signal-to-noise ratio, where j is the corresponding subnet, k is the interval, and K is the total number of intervals; Step 5032: Input the edge score e of a given low-quality image block, and calculate which subinterval k this image block belongs to. The calculation method of k is: in, is the floor function; Step 5033: Obtain the subnet directly based on the subinterval k through the pre-built lookup table Estimated peak signal-to-noise ratio Select Estimate Peak Signal-to-Noise Ratio The top subnet reconstructs the image patch.

7. The method for real-time transmission of high-definition images of unmanned aerial vehicles for power inspection according to claim 6 is characterized in that: In step 5033, the computation cost of each subnet is also introduced as a subnet performance evaluation factor; First, calculate a set of computational costs: Among them, c j represents the computational overhead corresponding to the jth subnet, and M is the total number of subnets; Then, a computation-performance trade-off function is constructed based on the estimated peak signal-to-noise ratio and the subnet computation cost to evaluate the subnet performance; Among them, j' represents the subnet number with the best performance when considering the subnet calculation cost; argmax represents the following function The value of j when the maximum value is obtained is the subnet with the best performance; λ is a hyperparameter used to balance the numerical difference between the computational cost in the inference phase and the peak signal-to-noise ratio estimation.

Images