Cruise image return control system based on intelligent sensing unmanned plane

Abstract

The application discloses a cruise image back transmission control system based on intelligent sensing unmanned aerial vehicle, and particularly relates to the technical field of unmanned aerial vehicle remote sensing and image communication, and comprises an environment perception and adaptive image enhancement module, an intelligent semantic key frame screening module and an adaptive channel hierarchical back transmission module. The former acquires environment parameters such as illumination and particulate matter concentration in real time through multiple airborne sensors, determines the types of fog, low illumination, backlight or normal degradation, and calls corresponding lightweight enhancement networks to adaptively recover the image. The intermediate module performs semantic segmentation on the enhanced image frame, detects six types of key targets including personnel, vehicles, ships, buildings, flames and smokes, and power transmission lines, and performs interframe redundancy discrimination in combination with a perception hash and dense optical flow. The application can ensure that the key target images are not missed and are back transmitted with high quality under limited bandwidth and fluctuating channel conditions, and reduces transmission load.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) remote sensing and image communication technology, and more specifically, to a cruise image transmission control system based on intelligent sensing UAVs. Background Technology

[0002] With the rapid development of UAV and intelligent sensing technologies, low-altitude patrol monitoring based on intelligent sensing UAVs has been increasingly widely applied in fields such as power transmission and transformation inspection, traffic network inspection, agricultural and forestry plant protection monitoring, disaster site assessment, and border security monitoring. Compared with traditional fixed-point video surveillance or manned aerial patrols, UAV platforms have significant advantages such as mobility, rapid deployment, wide field of view, and the ability to enter complex or dangerous areas, enabling large-scale, multi-angle, and continuous low-altitude image acquisition. Meanwhile, the integration of airborne intelligent sensing units (including light sensors, particulate matter concentration sensors, temperature and humidity sensors, barometers, and GPS modules) is continuously improving, allowing UAVs to simultaneously acquire rich environmental parameters during flight, providing crucial data support for the dynamic adaptation of subsequent image processing and transmission strategies. However, the actual low-altitude patrol operation environment is extremely complex and variable, with drastic changes in lighting conditions at different times and along different flight paths, and frequent occurrences of severe weather such as fog, haze, rain, and snow. Coupled with the limited onboard computing power, storage resources, and transmission bandwidth of UAV platforms, the real-time acquisition and reliable transmission of high-quality images face severe challenges.

[0003] In existing technologies, UAV cruise image transmission systems generally adopt a fixed encoding parameters and fixed transmission strategy architecture. That is, the ground station pre-sets the image acquisition resolution, frame rate, compression and quantization parameters, and modulation and coding scheme before takeoff, and no adjustments are made during flight to adapt to environmental changes. This open-loop image processing and transmission mode has many inherent drawbacks. First, when the operating area experiences environmental degradation such as low light, backlight, or smog, the pre-set image acquisition and enhancement parameters cannot respond in time, leading to problems such as brightness imbalance, loss of detail, and decreased contrast in the images, which seriously affects subsequent target recognition and interpretation. Second, the airborne end encodes, stores, and transmits all image frames indiscriminately, which can easily cause congestion and delays in the bandwidth-limited downlink of the UAV. A large number of redundant frames without key targets occupy valuable transmission resources, causing a small number of image frames containing key targets such as personnel, vehicles, ships, or fire points to be dropped or delayed due to channel fluctuations. Third, the existing system lacks real-time perception and dynamic adaptation capabilities for channel conditions. When the UAV flies to areas with severe terrain obstruction or at long distances, the signal-to-noise ratio drops sharply, and the fixed modulation and coding method cannot guarantee reliable transmission of key image data under high error conditions. Insufficient or excessive retransmissions will affect the real-time performance and effectiveness of the system.

[0004] To address the aforementioned issues, there is an urgent need to construct a cruise image backhaul control system that deeply integrates environmental perception, intelligent semantic analysis, and dynamic channel adaptation. Specifically, it requires establishing a rapid identification and adaptive enhancement mechanism for environmental degradation types on the airborne end to ensure stable and reliable input image quality; it necessitates introducing lightweight semantic understanding capabilities to filter out semantic keyframes containing critical targets in real time under limited computing power, and effectively eliminate a large number of redundant frames without targets, fundamentally reducing transmission load; simultaneously, it requires constructing a hierarchical transmission strategy based on real-time channel quality feedback, assigning high priority to image frames carrying critical semantic information and matching them with robust modulation coding and retransmission mechanisms, while assigning low priority to background reference frames and transmitting them in a best-effort manner. This will enable the backhaul control objectives of ensuring no critical target images are missed, prioritizing the arrival of important information, and minimizing overall bandwidth usage under actual operating conditions of limited bandwidth and channel fluctuations. This invention is proposed based on the aforementioned technical requirements. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a cruise image transmission control system based on an intelligent sensing drone.

[0006] To achieve the above objectives, the present invention provides the following technical solution: The cruise image transmission control system based on intelligent sensing drones includes the following modules: The environmental perception and adaptive image enhancement module is used to collect environmental data and drone position information in real time through the intelligent sensing unit carried by the drone, determine the environmental degradation type based on the environmental data, and call the corresponding lightweight enhancement network to perform adaptive enhancement processing on the original image according to the environmental degradation type to generate enhanced image frames. The intelligent semantic keyframe filtering module is used to perform semantic segmentation on the enhanced image frame, detect key targets and output bounding boxes, mark the current frame as a semantic keyframe, event keyframe or non-keyframe through inter-frame redundancy discrimination, perform target-level redundancy filtering on the semantic keyframe to determine whether to downgrade it to a target still prompt frame, and finally encapsulate the image and metadata of the determined keyframe into a high-priority data packet, while periodically retaining background reference frames. The adaptive channel hierarchical backhaul module is used to predict the received signal strength indication and signal-to-noise ratio based on environmental parameters and UAV location. Based on this, it classifies the channel quality level, retrieves data packets from the buffer queue according to priority, dynamically selects modulation and coding schemes and retransmission parameters according to the channel quality level, reliably backhauls high-priority key frame data packets, and suspends the transmission of low-priority background reference frame data packets in poor channels and transmits them after the channel is restored.

[0007] Specifically, in the environmental perception and adaptive image enhancement module: The intelligent sensing unit includes a light intensity sensor, a laser particulate matter concentration sensor, a temperature and humidity sensor, a barometer, and a GPS module. It reads data from each sensor at a set frequency to obtain real-time light intensity, particulate matter concentration, relative humidity, temperature, and the current three-dimensional position of the UAV.

[0008] Specifically, in the environmental perception and adaptive image enhancement module, the determination of the environmental degradation type is as follows: If the particulate matter concentration is greater than or equal to the preset fog concentration threshold and the light intensity is greater than or equal to the daytime brightness threshold, it is determined to be a foggy weather degradation type. If the light intensity is less than the low light threshold, it is determined to be a low light degradation type; If the light intensity is greater than the high brightness threshold and the brightness histogram fed back by the camera's automatic metering shows a double peak in the low grayscale area and the high grayscale area, and the proportion of pixels in the low grayscale area exceeds the preset ratio, it is determined to be a backlight degradation type. If none of the above conditions are met, it is determined to be a normal type; The degradation type is determined in the following priority order: foggy weather, low light, backlight, and normal.

[0009] Specifically, in the environmental perception and adaptive image enhancement module: When the degradation is due to fog, a dehazing network based on dark channel prior and depth-separable convolution is invoked to perform dehazing on the input image. Adaptive tone mapping is then performed on the dehazed image according to the current particulate matter concentration to obtain an enhanced image frame, which is the final output for the current degradation type. When there is low light degradation, the ultra-low light enhancement network that combines Retinex decomposition and U-Net is invoked. The illuminance component is obtained through Retinex decomposition, and the illuminance component is fed into U-Net for enhancement and then multiplied with the reflection component to obtain an enhanced frame with improved brightness and suppressed noise. When backlight degradation occurs, the camera is controlled to capture normally exposed dark areas and unexposed bright areas in spot metering mode, and a wide dynamic range frame is synthesized by using a multi-scale exposure fusion algorithm based on the Laplacian pyramid. When the data type is normal, perform grayscale world white balance and sharpening processing.

[0010] Specifically, in the intelligent semantic keyframe filtering module: The semantic segmentation network uses MobileNetV3-Large as its backbone network and adds a pyramid pooling module to output pixel-level category labels. The categories include six key targets: people, vehicles, ships, buildings, flames and smoke, and power transmission lines. The 8-neighborhood connectivity rule is used to label the connected components of pixels with the same category label. The contours of the connected components of each category are extracted and the minimum bounding rectangle is calculated. The target bounding box and existence flag are output.

[0011] Specifically, in the intelligent semantic keyframe filtering module, the inter-frame redundancy determination is as follows: The perceptual hashing algorithm is used to extract the perceptual hash value between the current frame and the previous retained keyframe and to calculate the Hamming distance. The dense optical flow algorithm is used to calculate the optical flow field of the current frame relative to the previous keyframe, and the set percentile of the motion amplitude of all pixels is taken as the motion intensity. If a key target exists in the current frame, it is marked as a semantic key frame; if no key target exists but the Hamming distance is greater than the first threshold and the motion intensity is greater than the second threshold, it is marked as an event key frame. If neither condition is met, the frame is marked as non-keyframe and discarded.

[0012] Specifically, in the intelligent semantic keyframe filtering module, target-level redundancy filtering is performed on frames marked as semantic keyframes: Match each detected key target bounding box region in the current frame with the matching target regions in the previous semantic key frame that have the same category and an intersection-union ratio greater than a set threshold. If no matching target exists, maintain the semantic keyframe identity; If a matching target exists, the normalized average brightness deviation rate of each matching target is calculated and the maximum value is taken. If the maximum value is less than the set deviation rate threshold, the frame is downgraded to a target still cue frame, and only the metadata is retained without encapsulating the image data. Otherwise, the semantic keyframe identity is maintained. The event keyframes are not included in the target-level redundancy filtering and are directly retained as keyframes.

[0013] Specifically, in the adaptive channel hierarchical backhaul module: The relative humidity, temperature, rainfall intensity, and spatial straight-line distance between the UAV and the ground station are used as input feature vectors and fed into a support vector regression channel prediction model pre-trained with a radial basis kernel function. The output is the predicted received signal strength indication and signal-to-noise ratio estimate. The predicted signal-to-noise ratio is divided into three channel quality levels: good, medium, and poor.

[0014] Specifically, in the adaptive channel hierarchical backhaul module: Data packets are retrieved from the send buffer queue in priority order; For high-priority key frame data packets, different modulation and coding schemes and maximum retransmission times are adopted according to the channel quality level. The actual packet loss rate is calculated by using the feedback of the hybrid automatic retransmission request process of the most recently sent key frame data packets through a sliding window. If the actual packet loss rate exceeds the preset packet loss rate threshold corresponding to the current channel quality level, the maximum retransmission time of subsequent key frames to be sent will be temporarily increased by a set number until the packet loss rate falls back below the threshold. For low-priority background reference frame data packets, intra-frame coding is used. When the channel quality is poor, transmission is paused and temporarily stored in the airborne buffer. Transmission is resumed when the channel quality recovers to medium or higher. When the channel quality is medium or good, transmission is carried out directly in a best-effort manner.

[0015] Specifically, in the adaptive channel hierarchical backhaul module: High-priority keyframe data packets use QPSK modulation and half-rate LDPC encoding with hybrid automatic repeat request enabled and the maximum number of retransmissions set to 4 when at poor priority level; use 16QAM modulation and two-thirds rate LDPC encoding with the maximum number of retransmissions set to 2 when at medium priority level; and use 64QAM modulation and three-quarters rate LDPC encoding with the maximum number of retransmissions set to 1 when at good priority level. The packet loss rate threshold is 0.15 for poor level, 0.08 for medium level, and 0.03 for good level. The number of temporary retransmissions is 1.

[0016] The technical effects and advantages of this invention are as follows: This system constructs a closed-loop environmental perception and adaptive enhancement link on the airborne end. It acquires environmental parameters such as illumination and particulate matter concentration in real time through multiple sensors and quickly distinguishes between foggy weather, low light, backlight, and normal operating conditions based on preset degradation type judgment rules. Then, it calls upon a matching lightweight enhancement network to perform targeted restoration of the original image—for foggy weather, it uses transmittance estimation combining dark channel prior and depth-separable convolution; for low light, it uses illumination enhancement through Retinex decomposition and U-Net collaboration; and for backlight, it expands the dynamic range through multi-exposure capture and Laplacian pyramid fusion. This mechanism enables the UAV to output enhanced image frames with accurate color reproduction and clear details under different weather and lighting conditions, providing a high-quality image source for subsequent target detection and semantic discrimination, thus reducing the interference of environmental degradation on the effectiveness of cruise monitoring from the source. Simultaneously, the enhancement network adopts a lightweight architecture design, enabling real-time operation on an airborne embedded computing platform without increasing additional hardware costs or significant power consumption.

[0017] Building upon this foundation, the system further introduces an intelligent semantic keyframe filtering strategy and a dynamic channel hierarchical backhaul strategy, forming a complete optimization chain from front-end acquisition to back-end transmission. The filtering module utilizes a lightweight semantic segmentation network to detect six types of key targets, including personnel, vehicles, ships, flames, and smoke, in real time on the airborne NPU. It combines perceptual hashing and dense optical flow to perform inter-frame redundancy discrimination, promptly discarding redundant frames without targets, downgrading static target frames to lightweight metadata frames, and pushing only key frames carrying new targets or whose target states have changed significantly into the high-priority transmission queue. At the same time, a small number of background reference frames are periodically retained. This reduces the downlink transmission data volume to one-tenth to one-hundredth of that in traditional solutions, effectively alleviating the bandwidth bottleneck of UAV links. The backhaul module, based on real-time environmental humidity, temperature, rainfall intensity, and flight distance, predicts the channel signal-to-noise ratio and classifies quality levels using a support vector regression model. For high-priority key frames, it dynamically matches QPSK / 16QAM / 64QAM modulation and corresponding LDPC encoding based on channel quality, and adaptively adjusts the retransmission count based on sliding window packet loss rate feedback. This ensures that critical semantic information can reach the ground station with a high probability even under poor channel conditions. Low-priority background frames are transmitted using intra-frame coding and a best-effort approach, temporarily storing them for later transmission when the channel is poor. This hierarchical transmission mechanism ensures real-time reliable backhaul of key target images in real-world operating environments with limited bandwidth and fluctuating signals, while preventing invalid data from crowding out channel resources. It achieves a good balance between transmission efficiency and reliability, significantly improving the practical effectiveness and data utilization efficiency of UAV low-altitude cruise monitoring. Attached Figure Description

[0018] Figure 1 This is a system block diagram of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] like Figure 1 As shown, the cruise image transmission control system module based on intelligent sensing UAV is as follows: The environmental perception and adaptive image enhancement module collects environmental data such as light intensity, particulate matter concentration, temperature, and humidity in real time through an integrated intelligent sensing unit. Based on preset thresholds, it determines the current degradation type as one of four: fog, low light, backlight, or normal. According to the degradation type, it dynamically calls the corresponding lightweight enhancement network (dehazing network, Retinex+U-Net low-light enhancement network, multi-scale exposure fusion, or conventional white balance sharpening) to adaptively enhance the original image, outputting enhanced image frames with timestamps and strategy identifiers. The execution steps are as follows: Step A1: Read the integrated intelligent sensing unit on the drone at a frequency of 10Hz. This unit includes a light intensity sensor, a laser-type PM2.5 particulate matter concentration sensor, a temperature and humidity sensor, a barometer, and a GPS module to obtain the real-time light intensity. (Unit: lux) Particulate matter concentration (Unit: μg / m³), relative humidity (%),temperature (°C) and the current three-dimensional position of the drone; Step A2, Determining the type of environmental degradation: If Greater than or equal to the preset fog concentration threshold of 75 μg / m³ and If the daytime brightness threshold of 500 lux is greater than or equal to the threshold, the current condition is determined to be "foggy degradation"; if... If the light intensity is less than 5 lux below the low-light threshold, it is classified as "low-light degradation"; if If the brightness exceeds the high brightness threshold of 8000 lux and the brightness histogram fed back by the camera's automatic metering shows a double peak in the 0-50 low grayscale area and the 180-255 high grayscale area, with the low grayscale pixel ratio exceeding 40%, it is judged as the "backlight degradation" type; if none of the above conditions are met, it is judged as the "normal" type; the above degradation types are judged in the priority order of "foggy day → low light → backlight → normal". Once the previous condition is met, it is judged as that type, and no further judgment is made. Step A3: Dynamically load and run the corresponding lightweight enhancement network according to the degradation type. When it is a foggy degradation, call the dehazing network based on dark channel prior and depthwise separable convolution. This network consists of an input layer, a dark channel calculation module, a transmittance estimation sub-network composed of three cascaded depthwise separable convolution residual blocks, an atmospheric light estimation module, and an atmospheric scattering model recovery layer. The original Bayer format image is input, and bilinear interpolation demosaic processing is performed on the input Bayer format image to convert it into an RGB three-channel image, which is then used as the input for dark channel calculation. First, the dark channel map is calculated, and then the fine transmittance map is regressed through the residual blocks. Combined with atmospheric light values ​​estimated from the brightest 0.1% pixel in the dark channel ,according to Output fog-free RGB frames, then adjust according to the current particulate matter concentration. Perform adaptive tone mapping on the haze-free RGB frame and calculate the mapping coefficients. according to Obtain the compensated enhanced image frame, and with Replace the original As the final output of this type; where: This is the pixel value (vector, corresponding to the R, G, and B channels) at pixel position x in the dehazing-free RGB image after dehazing. is the pixel value at pixel x of the original Bayer format image (after interpolation or conversion); A is the atmospheric light value; To obtain Use a larger value than 0.1 to avoid the denominator being zero or too small, which could lead to unstable values. The gamma correction factor for tone mapping is used to adjust contrast. This represents the current concentration of particulate matter in the environment. This is the pixel value at pixel x in the final enhanced image frame after gamma correction (applied per channel). When dealing with low-light degradation, a fusion Retinex decomposition (where the Gaussian wrapping scale σ is 15, using single-scale decomposition; or for multi-scale decomposition, σ=[15,50,120] three-scale weighted average) and U-Net's ultra-low-light enhancement network are invoked. This network consists of a multi-scale Retinex decomposition front-end and a five-level encoder-decoder U-Net back-end connected in series. First, the image is decomposed by Gaussian wrapping Retinex to obtain the illuminance component. The illuminance component is then fed into U-Net. The encoder downsamples step by step to extract the illumination distribution, and the decoder upsamples step by step and jumps to recover the enhanced illuminance map. Finally, it is multiplied pixel by pixel with the reflection component, and the enhanced frame with improved brightness and suppressed noise is output. The U-Net training loss function includes mean square error and total variational regularization term. When backlight degradation occurs, the camera is immediately controlled to first capture a frame with the exposure time corresponding to the dark area in spot metering mode, ensuring proper exposure of the dark areas. Then, a frame with the exposure time corresponding to the bright area is captured, ensuring that the highlights are not overexposed. The exposure times of the two frames are automatically determined by the camera's metering system for the lower 1 / 3 and upper 1 / 3 of the image, respectively. A seven-layer multi-scale exposure fusion algorithm based on the Laplacian pyramid is used. Contrast, saturation, and good exposure weight maps are calculated for each layer and then weighted and fused to synthesize a wide dynamic range frame. During fusion, the weight maps of each pyramid layer are normalized and then the exposed images are weighted and averaged to obtain the fused Laplacian pyramid layers. Finally, starting from the top layer, the images are upsampled level by level and added to the lower layers to perform standard Laplacian pyramid reconstruction, outputting the final wide dynamic range RGB frame.

[0021] When the type is normal, only grayscale world white balance and 3×3 kernel unsharpened mask (unsharpmask) sharpening are performed; after processing, an enhanced image frame is generated, along with a timestamp, environment type label and the enhancement strategy identifier used, and sent to the next module.

[0022] The intelligent semantic keyframe filtering module inputs enhanced image frames into a lightweight semantic segmentation network to detect six key targets: people, vehicles, ships, buildings, flames and smoke, and power transmission lines, and outputs bounding boxes. Inter-frame redundancy is determined using pHash hash distance and dense optical flow motion intensity to classify frames as semantic keyframes, event keyframes, or non-keyframes. Target-level brightness deviation filtering is further performed on semantic keyframes, downgrading stationary target frames to lightweight metadata frames. Finally, the keyframe images and metadata are encapsulated into HIGH-priority data packets and sent to the transmission buffer queue, while one LOW-priority background reference frame is reserved every 30 frames. The execution steps are as follows: Step B1: Receive enhanced image frames, scale them to 512×512 resolution, and input them into a lightweight semantic segmentation network deployed on an airborne NPU. This network uses MobileNetV3-Large as its backbone and adds a pyramid pooling module. The pyramid pooling module consists of four parallel average pooling branches with pooling kernel sizes of 1×1, 2×2, 3×3, and 6×6, respectively. The output of each branch is upsampled to the backbone output feature map size after being reduced to 128 by a 1×1 convolution. It is then concatenated with the backbone feature map and then output as pixel-level category labels after a 3×3 convolution and softmax layer. The categories include six key targets: personnel, vehicles, ships, buildings, flames and smoke, and power transmission lines. The 8-neighborhood connectivity rule is used to label the connected components of pixels with the same category label. Contours are extracted for each category's connected components, and the minimum bounding rectangle is calculated. The target bounding box and presence marker are output. Step B2, perform inter-frame redundancy determination: Use the pHash algorithm to extract the perceptual hash value between the current frame and the previous retained keyframe. That is, scale the image to a 32×32 grayscale image, perform discrete cosine transform, and take the 8×8 low-frequency coefficients in the upper left corner. Compare the hash value with the mean of the coefficients to generate a 64-bit hash value, and calculate the Hamming distance. The Farneback dense optical flow algorithm is used to calculate the optical flow field of the current frame relative to the previous keyframe, and the 90th percentile of the motion amplitude of all pixels is taken as the motion intensity. ; Step B3, key frame determination rule: If any type of key target detected in step B1 exists in the current frame, it is directly marked as a "semantic key frame". If there is no critical target in the current frame, but Greater than the threshold 30 and If the value is greater than 2 pixels, it indicates that there has been a significant geometric or content change in the scene, and it is also marked as an "event keyframe". "Event keyframes" do not participate in the target-level redundancy filtering in step B3. They are directly retained as keyframes and enter step B4 for encapsulation and transmission to ensure that scene change information is not missed. If neither of these conditions is met, they are marked as "non-keyframes" and discarded directly. Next, target-level redundancy filtering is performed on the currently marked "semantic keyframe": First, it attempts to match each detected key target bounding box region in this frame with the matching target regions of the same category and with an intersection-union ratio greater than 0.5 in the previous semantic keyframe; if no matching target exists, the "semantic keyframe" status of this frame is maintained and subsequent processing continues; if a matching target exists, the normalized average brightness deviation rate is calculated for each matching target. ;in For the first Normalized average luminance deviation rate of each matched target For the current frame, the first The set of pixels within the bounding box region of a key target. For the previous semantic keyframe and Matching target bounding box regions of the same category with an intersection-union ratio greater than 0.5. Indicates the average brightness of the area. For a very small positive number, take Take the maximum value of the deviation rate of all matched targets. ;like If the target area remains largely unchanged, the frame will be changed from a "semantic keyframe" to a "target still cue frame," retaining only its semantic label, target quantity, bounding box, and timestamp as lightweight metadata, without encapsulating image data; otherwise, the "semantic keyframe" status will remain unchanged. Step B4: For images identified as keyframes, generate frame-level metadata, including semantic tags, number of targets and coordinates of each target's bounding box, and timestamp. Encapsulate the enhanced image frame along with the metadata into a keyframe data packet and push it into the transmission buffer queue according to priority HIGH. In addition, one frame is forcibly reserved every 30 frames as a "background reference frame", with only a timestamp attached, marked as priority LOW, and also sent to the buffer queue for possible subsequent map updates.

[0023] The adaptive channel hierarchical backhaul module takes environmental humidity, temperature, rainfall intensity, and the distance from the UAV to the ground station as inputs. It estimates RSSI and SNR using an SVR channel prediction model and classifies SNR into three levels: good, medium, and poor. Data packets are retrieved from the buffer queue according to priority. HIGH priority keyframes dynamically select QPSK / 16QAM / 64QAM modulation and the corresponding LDPC code rate and retransmission count based on channel quality, and adaptively adjust retransmission parameters based on sliding window packet loss rate feedback. LOW priority background reference frames use H.265 intra-frame coding, suspend transmission in poor channel conditions, and are transmitted in UDP best-effort mode after channel recovery. Finally, the ground station classifies and displays keyframes and stores background frames. Its execution steps are as follows: Step C1: Obtain the current relative humidity from the temperature and humidity sensor. ,temperature Rainfall intensity is obtained from rain sensors. If there is no rain sensor, the rainfall is estimated based on humidity and an empirical rainfall model for the area. Simultaneously, the spatial straight-line distance between the UAV and the ground station is calculated in real time using the airborne GPS and the known coordinates of the ground control station. ;Will As input feature vectors, they are fed into a support vector regression channel prediction model that has been trained in advance using radial basis kernel functions based on historical flight test data, and the predicted received signal strength index (RSSI) and signal-to-noise ratio (SNR) estimates are output. Step C2: Divide the predicted SNR into three channel quality levels: SNR greater than 15dB is "good", 8dB to 15dB is "medium", and below 8dB is "poor". Step C3: Retrieve data packets from the transmit buffer queue in priority order. If the packet is a keyframe data packet with a HIGH priority, then at the "poor" priority level, QPSK modulation and 1 / 2-bit LDPC encoding are used, the physical layer transport block size is limited to 256 bits, hybrid automatic repeat request is enabled, and the maximum number of retransmissions is set to 4; at the "medium" priority level, 16QAM modulation and 2 / 3-bit LDPC encoding are used, with a maximum of 2 retransmissions; at the "good" priority level, 64QAM modulation and 3 / 4-bit LDPC encoding are used, with a maximum of 1 retransmission, and the length maintained by the airborne end is... The sliding window, in which The value should be between 50 and 200, with a preferred value of 100 to balance statistical stability and response speed to channel changes; the most recent statistical value should be used. Feedback from the HARQ process for each sent keyframe data packet is used to calculate the actual packet loss rate. ;in This represents the number of times the receiver has responded with a NACK (denial of Acknowledgment) among the most recent W keyframe data packets sent. This represents the number of times, out of the most recent W packets, no acknowledgment (ACK / NACK) was received within the specified time. The length of the sliding window maintained by the airborne end; like If the packet loss rate exceeds the preset threshold corresponding to the current channel quality level (poor level threshold 0.15, medium level threshold 0.08, good level threshold 0.03), the maximum number of retransmissions for subsequent keyframes will be temporarily increased by 1 from the original value, until... The error rate drops below the corresponding threshold to ensure reliable transmission of critical semantic information against bit errors; Step C4: If the background reference frame data packet is of LOW priority, H.265 intra-frame coding is uniformly adopted, intra-frame refresh is enabled, and the quantization parameter QP is fixed at 38. When the priority is "poor", transmission is paused and temporarily stored in the airborne buffer. Transmission is restarted when the channel quality recovers to the "medium" or "good" level (i.e. SNR≥8dB). When the priority is "medium" and "good", UDP protocol is used directly for best-effort transmission without retransmission mechanism. In step C5, the encoded and modulated baseband signal is transmitted via the airborne radio frequency front-end. After being received by the ground station, it is classified, demodulated, and decoded according to priority and semantic tags, and the semantic key frames are displayed in real time with the target area highlighted. At the same time, the background reference frames are stored in the database, thereby completing the control of high-quality image backhaul without missing key targets under the condition of minimizing bandwidth usage.

[0024] The above formulas are all dimensionless calculations. Dimensionless calculations can be performed using various methods such as standardization, which will not be elaborated here. The formulas are derived from software simulations based on a large amount of collected data, and the preset parameters in the formulas can be set by those skilled in the art according to the actual situation.

[0025] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, ATA hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium can be a solid-state ATA hard disk.

[0026] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0027] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0028] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0029] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0030] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0031] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable ATA hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0032] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A cruise image transmission control system based on an intelligent sensing UAV, characterized in that, Includes the following modules: The environmental perception and adaptive image enhancement module is used to collect environmental data and drone position information in real time through the intelligent sensing unit carried by the drone, determine the environmental degradation type based on the environmental data, and call the corresponding lightweight enhancement network to perform adaptive enhancement processing on the original image according to the environmental degradation type to generate enhanced image frames. The intelligent semantic keyframe filtering module is used to perform semantic segmentation on the enhanced image frame, detect key targets and output bounding boxes, mark the current frame as a semantic keyframe, event keyframe or non-keyframe through inter-frame redundancy discrimination, perform target-level redundancy filtering on the semantic keyframe to determine whether to downgrade it to a target still prompt frame, and finally encapsulate the image and metadata of the determined keyframe into a high-priority data packet, while periodically retaining background reference frames. The adaptive channel hierarchical backhaul module is used to predict the received signal strength indication and signal-to-noise ratio based on environmental parameters and UAV location. Based on this, it classifies the channel quality level, retrieves data packets from the buffer queue according to priority, dynamically selects modulation and coding schemes and retransmission parameters according to the channel quality level, reliably backhauls high-priority key frame data packets, and suspends the transmission of low-priority background reference frame data packets in poor channels and transmits them after the channel is restored.

2. The cruise image transmission control system based on intelligent sensing UAV according to claim 1, characterized in that, In the environmental perception and adaptive image enhancement module: The intelligent sensing unit includes a light intensity sensor, a laser particulate matter concentration sensor, a temperature and humidity sensor, a barometer, and a GPS module. It reads data from each sensor at a set frequency to obtain real-time light intensity, particulate matter concentration, relative humidity, temperature, and the current three-dimensional position of the UAV.

3. The cruise image transmission control system based on intelligent sensing UAV according to claim 2, characterized in that, In the environmental perception and adaptive image enhancement module, the determination of environmental degradation type is specifically as follows: If the particulate matter concentration is greater than or equal to the preset fog concentration threshold and the light intensity is greater than or equal to the daytime brightness threshold, it is determined to be a foggy weather degradation type. If the light intensity is less than the low light threshold, it is determined to be a low light degradation type; If the light intensity is greater than the high brightness threshold and the brightness histogram fed back by the camera's automatic metering shows a double peak in the low grayscale area and the high grayscale area, and the proportion of pixels in the low grayscale area exceeds the preset ratio, it is determined to be a backlight degradation type. If none of the above conditions are met, it is determined to be a normal type; The degradation type is determined in the following priority order: foggy weather, low light, backlight, and normal.

4. The cruise image transmission control system based on intelligent sensing UAV according to claim 3, characterized in that, In the environmental perception and adaptive image enhancement module: When the degradation is due to fog, a dehazing network based on dark channel prior and depth-separable convolution is invoked to perform dehazing on the input image. Adaptive tone mapping is then performed on the dehazed image according to the current particulate matter concentration to obtain an enhanced image frame, which is the final output for the current degradation type. When there is low light degradation, the ultra-low light enhancement network that combines Retinex decomposition and U-Net is invoked. The illuminance component is obtained through Retinex decomposition, and the illuminance component is fed into U-Net for enhancement and then multiplied with the reflection component to obtain an enhanced frame with improved brightness and suppressed noise. When backlight degradation occurs, the camera is controlled to capture normally exposed dark areas and unexposed bright areas in spot metering mode, and a wide dynamic range frame is synthesized by using a multi-scale exposure fusion algorithm based on the Laplacian pyramid. When the data type is normal, perform grayscale world white balance and sharpening processing.

5. The cruise image transmission control system based on an intelligent sensing UAV according to claim 1, characterized in that, In the intelligent semantic keyframe filtering module: The semantic segmentation network uses MobileNetV3-Large as its backbone network and adds a pyramid pooling module to output pixel-level category labels. The categories include six key targets: people, vehicles, ships, buildings, flames and smoke, and power transmission lines. The 8-neighborhood connectivity rule is used to label the connected components of pixels with the same category label. The contours of the connected components of each category are extracted and the minimum bounding rectangle is calculated. The target bounding box and existence flag are output.

6. The cruise image transmission control system based on intelligent sensing UAV according to claim 1, characterized in that, In the intelligent semantic keyframe filtering module, the inter-frame redundancy judgment specifically involves: The perceptual hashing algorithm is used to extract the perceptual hash value between the current frame and the previous retained keyframe and to calculate the Hamming distance. The dense optical flow algorithm is used to calculate the optical flow field of the current frame relative to the previous keyframe, and the set percentile of the motion amplitude of all pixels is taken as the motion intensity. If a key target exists in the current frame, it is marked as a semantic key frame; if no key target exists but the Hamming distance is greater than the first threshold and the motion intensity is greater than the second threshold, it is marked as an event key frame. If neither condition is met, the frame is marked as non-keyframe and discarded.

7. The cruise image transmission control system based on an intelligent sensing UAV according to claim 6, characterized in that, In the intelligent semantic keyframe filtering module, target-level redundancy filtering is performed on frames marked as semantic keyframes: Match each detected key target bounding box region in the current frame with the matching target regions in the previous semantic key frame that have the same category and an intersection-union ratio greater than a set threshold. If no matching target exists, maintain the semantic keyframe identity; If a matching target exists, the normalized average brightness deviation rate of each matching target is calculated and the maximum value is taken. If the maximum value is less than the set deviation rate threshold, the frame is downgraded to a target still cue frame, and only the metadata is retained without encapsulating the image data. Otherwise, the semantic keyframe identity is maintained. The event keyframes are not included in the target-level redundancy filtering and are directly retained as keyframes.

8. The cruise image transmission control system based on intelligent sensing UAV according to claim 1, characterized in that, In the adaptive channel hierarchical backhaul module: The relative humidity, temperature, rainfall intensity, and spatial straight-line distance between the UAV and the ground station are used as input feature vectors and fed into a support vector regression channel prediction model pre-trained with a radial basis kernel function. The output is the predicted received signal strength indication and signal-to-noise ratio estimate. The predicted signal-to-noise ratio is divided into three channel quality levels: good, medium, and poor.

9. The cruise image transmission control system based on intelligent sensing UAV according to claim 1, characterized in that, In the adaptive channel hierarchical backhaul module: Data packets are retrieved from the send buffer queue in priority order; For high-priority key frame data packets, different modulation and coding schemes and maximum retransmission times are adopted according to the channel quality level. The actual packet loss rate is calculated by using the feedback of the hybrid automatic retransmission request process of the most recently sent key frame data packets through a sliding window. If the actual packet loss rate exceeds the preset packet loss rate threshold corresponding to the current channel quality level, the maximum retransmission time of subsequent key frames to be sent will be temporarily increased by a set number until the packet loss rate falls back below the threshold. For low-priority background reference frame data packets, intra-frame coding is used. When the channel quality is poor, transmission is paused and temporarily stored in the airborne buffer. Transmission is resumed when the channel quality recovers to medium or higher. When the channel quality is medium or good, transmission is carried out directly in a best-effort manner.

10. The cruise image transmission control system based on an intelligent sensing UAV according to claim 9, characterized in that, In the adaptive channel hierarchical backhaul module: High-priority keyframe data packets use QPSK modulation and half-rate LDPC encoding with hybrid automatic repeat request enabled and the maximum number of retransmissions set to 4 when at poor priority level; use 16QAM modulation and two-thirds rate LDPC encoding with the maximum number of retransmissions set to 2 when at medium priority level; and use 64QAM modulation and three-quarters rate LDPC encoding with the maximum number of retransmissions set to 1 when at good priority level. The packet loss rate threshold is 0.15 for poor level, 0.08 for medium level, and 0.03 for good level. The number of temporary retransmissions is 1.

Images