A model knowledge distillation method and system for an aviation scene enhancement system

Abstract

The application discloses a model knowledge distillation method and system for an aviation scene enhancement system, and comprises the following steps: acquiring aviation scene image data and constructing a training sample set; constructing a teacher scene enhancement model and extracting model output results and intermediate layer features; constructing a light student scene enhancement model; constructing a joint distillation loss function based on the output response difference, intermediate feature difference, structure information difference and task semantic difference between the teacher scene enhancement model and the light student scene enhancement model; generating region weights according to key regions in the aviation image, and weighting and optimizing the joint distillation loss function to train the light student scene enhancement model; and performing real-time enhancement processing on the input aviation scene image. The application can ensure low calculation complexity and inference delay, and improve the definition, edge integrity and flight key target distinguishability of aviation scene images under low-visibility, night and complex weather conditions.

Description

Technical Field

[0001] This invention relates to the fields of machine vision and image processing technology, and in particular to a model knowledge distillation method and system for aerial vision enhancement systems. Background Technology

[0002] Aviation visual enhancement systems are primarily used to improve pilots' perception of external scenes in complex environments such as low visibility, low light, fog, rain, snow, and backlighting, thereby enhancing flight safety and mission efficiency. With the development of deep learning technology, visual enhancement methods based on convolutional neural networks or visual neural networks have shown good performance in image dehazing, low-light enhancement, detail restoration, and target saliency improvement. However, these models typically suffer from large parameter sizes, high computational costs, and long inference latency, requiring significant computing resources even after training, making direct deployment on airborne embedded platforms or edge computing devices upon which aviation visual enhancement systems rely difficult. Therefore, reducing model size and inference overhead while maximizing enhancement effectiveness has become an important research direction in this field.

[0003] To address the computational and storage pressures faced by deep learning models during deployment, existing technologies typically employ model pruning, parameter quantization, and lightweight network architecture design to achieve model compression and inference acceleration. Model pruning primarily reduces parameter size by removing redundant weight connections or feature channels from the network. Parameter quantization reduces model storage overhead and computational complexity by mapping floating-point parameters to low-bit integers. Lightweight network design often utilizes structures such as depthwise separable convolutions, grouped convolutions, and inverted residuals to construct more compact models. While these methods can improve model deployment efficiency to some extent in general vision tasks, they still have significant shortcomings for aerial vision augmentation tasks. On the one hand, compression methods such as pruning and quantization can easily weaken the model's feature representation capabilities, leading to blurred edge details, incomplete texture restoration, and increased local artifacts in the augmentation results. On the other hand, although lightweight networks reduce model complexity, their performance ceiling is usually limited by the network structure itself, making it difficult to fully balance global augmentation effects under complex weather conditions with information fidelity in key areas. Especially in aviation visual enhancement scenarios, critical information such as runway boundaries, obstacle outlines, horizon lines, and light signals are closely related to flight safety. If such information is lost or distorted during compression, it will directly affect the usability of the enhancement results. Therefore, existing traditional lightweight methods are difficult to simultaneously meet the comprehensive requirements of aviation visual enhancement systems for real-time performance, lightweight design, and high-fidelity enhancement effects.

[0004] To achieve a better balance between lightweight models and enhanced performance, knowledge distillation provides an effective approach for compressing and deploying aerial vision augmentation models. Knowledge distillation constructs teacher and student models, transferring knowledge from high-performance large models in areas such as feature representation, structural recovery, and enhanced response to lightweight student models. This allows the student model to achieve near-teacher-level enhancement capabilities with lower computational complexity. Compared to traditional methods that rely on directly pruning parameters or simplifying network structures, knowledge distillation emphasizes capability transfer, preserving the teacher model's representational capabilities for complex scenes as much as possible while reducing model size. For aerial vision augmentation systems, this method not only helps alleviate the deployment pressure on airborne platforms in terms of computing power, storage, and power consumption, but also improves the ability of lightweight models to recover key regional details, structural edges, and salient targets under low visibility, low illumination, and complex background conditions, thereby enhancing the clarity, stability, and task usability of the model output. However, most existing knowledge distillation methods are geared towards general vision tasks such as image classification and object detection, lacking specific designs for preserving critical safety information, adapting to complex environments, and addressing real-time deployment constraints in aerial vision augmentation scenarios.

[0005] Therefore, it is necessary to propose a model knowledge distillation method for aviation visual enhancement systems to achieve effective transfer of high-performance enhancement knowledge to lightweight models, thereby improving its deployment feasibility and enhancement effect in practical aviation applications. Summary of the Invention

[0006] In view of this, the purpose of this invention is to provide a model knowledge distillation method and system for aviation visual enhancement systems, which can improve the clarity, edge integrity and identifiability of aviation visual images under low visibility, night and complex weather conditions while ensuring low computational complexity and inference latency.

[0007] The present invention solves the technical problem by adopting the following technical solution: A model knowledge distillation method for aerial vision enhancement systems includes the following steps: Step S1: Acquire aerial visual image data and construct a training sample set; Step S2: Construct a teacher visual enhancement model and extract the output results and intermediate layer features of the teacher visual enhancement model; Step S3: Construct a lightweight student visual augmentation model for airborne deployment; Step S4: Construct a joint distillation loss function based on the differences in output response, intermediate features, structural information, and task semantics between the teacher visual enhancement model and the lightweight student visual enhancement model; Step S5: Generate region weights based on key regions in the aerial image, and perform weighted optimization on the joint distillation loss function to train the lightweight student visual enhancement model. Step S6: Deploy the trained lightweight student visual enhancement model into the aviation visual enhancement system to perform real-time enhancement processing on the input aviation visual images.

[0008] Furthermore, in step S1, the method for acquiring aerial visual image data and constructing a training sample set includes: First, raw image data acquired by the aviation visual enhancement system under different flight conditions is collected; the image data is visible light image, infrared image, or visual image formed by fusion of multimodal sensors; Then, the original image data is subjected to size unification, brightness normalization, noise suppression, and data augmentation to construct a training sample set; the training sample set is represented as follows: ;in, This indicates a degraded aerial view image. This represents the corresponding high-quality reference image. Indicates the critical region mask. Indicates scene category label; The total number of samples; When real, high-quality reference images are difficult to obtain, a degraded input image is generated from a clear, degraded aerial view image using a degradation model. The expression for this model is: ;in, This represents a degenerate generating function. Indicates the atomization intensity parameter. This represents the brightness attenuation parameter. Represents the fuzzy intensity parameter. This indicates parameters related to noise or rain / snow disturbance.

[0009] Furthermore, in step S2, the teacher's visual enhancement model is a deep convolutional neural network, an encoder-decoder network, an attention-based network, a visual Transformer network, or a combination thereof; the intermediate layer features extracted by the teacher's visual enhancement model include at least two or three of the following: shallow texture features, intermediate structural features, and deep semantic features.

[0010] Further, in step S2, for the input sample The teacher's visual enhancement model outputs the enhancement results. Represented as: ;in, Indicating a teacher's visual enhancement model Indicates the teacher model parameters; Teacher Visual Enhancement Model The layer features are: ;in, Indicates the number of characteristic layers used for distillation. These correspond to shallow texture features, mid-level structural features, and deep semantic features, respectively. Shallow features are used to depict local details and texture responses, mid-level features are used to represent contours and geometric structures, and deep features are used to represent scene semantics and important target information.

[0011] Furthermore, in step S3, the lightweight student visual enhancement model employs one or more of the following: depthwise separable convolution, grouped convolution, inverted residual, lightweight attention module, or low-complexity feature fusion structure. For the same input sample Lightweight student visual enhancement model outputs enhancement results for: ;in, This represents the parameters of a lightweight student visual enhancement model; Extracting corresponding hierarchical features from a lightweight student visual enhancement model: ; The lightweight student visual augmentation model must meet the resource constraints of the airborne platform: ;in, This represents the computational complexity of the lightweight student visual augmentation model. This indicates the maximum computing resources allowed by the platform. Indicates the model inference latency. This indicates the maximum allowable response time of the system.

[0012] Furthermore, in step S4, the constructed joint distillation loss function includes at least the output response distillation loss, intermediate feature distillation loss, structural information distillation loss, and task semantic distillation loss. The output response distillation loss, used to constrain the consistency between the outputs of the lightweight student visual enhancement model and the teacher visual enhancement model, is expressed as: The output response distillation loss enables the lightweight student visual enhancement model to learn the output response capabilities of the teacher visual enhancement model in terms of brightness restoration, contrast enhancement, and overall visual quality improvement. Intermediate feature distillation loss is used to constrain the consistency between the lightweight student visual augmentation model and the teacher visual augmentation model in the intermediate layer feature space; when the number of channels in the teacher visual augmentation model and the lightweight student visual augmentation model is inconsistent, a mapping function is introduced. Aligning student features, the intermediate feature distillation loss is: ;in, , , They represent the first The number of channels, height, and width of the layer feature map; The structural information distillation loss is used to constrain the consistency between the lightweight student vision enhancement model and the teacher vision enhancement model in at least one type of structural information, namely, edges, gradients, contours, frequency domain response, or local geometry, as shown in the following equation: ;in, and These represent the gradient operators for the image in the horizontal and vertical directions, respectively; The task semantic distillation loss is used to constrain the consistency between the lightweight student vision enhancement model and the teacher vision enhancement model in at least one semantic information in the key region response map, attention distribution map, saliency response map, or auxiliary task branch output, as shown in the following equation: ;in, Attention response maps or key region response maps generated for teacher visual enhancement models. Response map generated for a lightweight student visual enhancement model; The reference image reconstruction loss is shown in the following formula: ; Based on the above-mentioned losses, the combined distillation loss function is expressed as: ;in, , , , , These are the weighting coefficients for each loss term.

[0013] Further, in step S5, the key regions include one or more of the following: runway region, runway boundary region, obstacle region, horizon region, light signal region, forward track region, and highly salient feature region; a region weight map is generated based on the key regions, and one or more loss terms in the joint distillation loss function are weighted using the region weight map, wherein the region weight map satisfies: ;in, m Indicates the critical region mask. Indicates the enhancement coefficient of the key region. Represents a region weighting map; Based on the regional weight map, construct a regional weighted output distillation loss: ;in, Indicates the first The region weight map corresponding to each training sample This indicates element-wise multiplication.

[0014] Furthermore, in step S5, based on the scene category Dynamically adjust the weights of each loss term; let the scene adaptive weight function be... The total loss is then written as: ;in, ; During training, the parameters of the lightweight student visual augmentation model are updated using gradient descent. ;in, The learning rate is used; after multiple rounds of iterative training, a lightweight student visual augmentation model that satisfies the performance enhancement and deployment constraints is obtained.

[0015] Furthermore, in step S6, the deployed lightweight student visual augmentation model receives real-time input images from airborne sensors. and output the enhanced results. : ;in, This represents the optimal parameters of the lightweight student visual augmentation model after training.

[0016] A model knowledge distillation system for aerial vision enhancement systems includes: The data construction module is used to acquire aerial visual image data and build a training sample set; The teacher model building module is used to build a teacher visual enhancement model and extract the output results and intermediate layer features of the teacher visual enhancement model. The student model building module is used to build lightweight student visual augmentation models for airborne deployment. The joint distillation module is used to construct a joint distillation loss function based on the differences in output response, intermediate features, structural information, and task semantics between the teacher visual enhancement model and the lightweight student visual enhancement model, and to train the lightweight student visual enhancement model. The region weighting module is used to generate region weights based on key regions in the aerial visual image and to perform weighted optimization on the joint distillation loss function; The deployment module is used to deploy the trained lightweight student visual enhancement model to the aviation visual enhancement system to perform real-time enhancement processing on the input aviation visual images.

[0017] The present invention discloses a model knowledge distillation method and system for aerial visual enhancement systems, which has the following beneficial effects: (1) High-performance augmented knowledge transfer: This invention adopts a combination of a teacher model and a lightweight student model to achieve effective transfer of high-performance visual augmented knowledge to a lightweight model. By jointly distilling the output response, intermediate features, structural information and semantic information of the teacher model, the student model can obtain augmented capabilities close to those of the teacher model while maintaining low complexity.

[0018] (2) Balancing lightweight design and real-time deployment: This invention addresses the needs of airborne embedded platforms and edge computing devices for aviation visual enhancement systems, constructing lightweight student models suitable for deployment. Compared to directly deploying large-scale enhancement models, this approach effectively reduces the number of model parameters, computational load, and inference latency, thereby meeting the real-time and stability requirements in aviation scenarios.

[0019] (3) Enhancement of key area information: This invention introduces a key area weighted distillation mechanism for flight safety-related areas such as runway boundaries, obstacle outlines, horizons, light signals, and forward trajectory areas. By increasing the distillation weight of the above areas during the training process, the student model pays more attention to the information recovery of key areas, thereby improving the integrity and recognizability of important structural information in the enhancement results.

[0020] (4) Enhanced adaptability to complex environments: This invention is applicable to complex flight environments such as low visibility, weak light, fog, rain, snow, and backlight. By jointly distilling knowledge information at different levels, the lightweight student model can still maintain good image enhancement effects under various degradation conditions, thereby improving the model's adaptability and robustness in complex aviation environments.

[0021] (5) Enhanced usability of the results: This invention not only improves the brightness, contrast and clarity of the aerial visual images, but also enhances the representation of key targets and semantic regions by distilling the teacher model, making the enhanced results more conducive to downstream tasks such as target detection, obstacle recognition, runway positioning and assisted landing, thereby improving the overall application value of the aerial visual enhancement system.

[0022] (6) System comprehensive performance optimization: Through the above technical means, the present invention achieves a comprehensive balance between model lightweighting, key information preservation, complex environment enhancement and real-time deployment capability, making the aviation visual enhancement system more practical, reliable and promising for promotion and application in complex dynamic flight environments. Attached Figure Description

[0023] Figure 1 This is a flowchart of the method of the present invention.

[0024] Figure 2 This is a schematic diagram of the combined distillation structure of the teacher model and student model of the present invention.

[0025] Figure 3 This is a schematic diagram of the key area weighted and scene-adaptive distillation of the present invention.

[0026] Figure 4 This is a schematic diagram illustrating the deployment and application of the lightweight student model after training according to the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] The purpose of this invention is to provide a model knowledge distillation method and system for aerial vision enhancement systems, in order to solve the problems of high false detection rate in complex backgrounds, guidance interruption after short-term loss of landmarks, single stage switching basis, large grounding point offset, and insufficient safety in the terminal landing stage in the existing technology using single landmark recognition and direct vision servo control, thereby improving the stability, continuity, accuracy and safety of UAV autonomous landing in low-altitude complex environments.

[0029] refer to Figure 1 This invention provides a model knowledge distillation method for aviation visual enhancement systems, comprising the following steps: Step S1: Acquire aerial visual image data and construct a training sample set; The image data includes aerial visual images under at least one or more complex scenarios such as low visibility, low light, fog, rain, snow, and backlight. The image data may originate from real-world data collected by airborne sensors, data generated by simulation platforms, or a combination of publicly available augmented datasets and aerial scene data.

[0030] To improve the applicability of distillation training, preprocessing operations are performed on the original image data. These preprocessing operations include one or more of the following: image cropping, scale unification, brightness normalization, noise suppression, dynamic range adjustment, and data augmentation. When constructing the training sample set, one of the following two methods can be used: First, directly using degraded aerial view images paired with corresponding high-quality reference images to form samples; second, when a real, clear reference image is lacking, applying degradation models such as fogging, low illumination, blurring, noise, and rain / snow occlusion to clear aerial view images to generate training samples corresponding to degraded inputs and clear labels. The training sample set includes one or more of the following: degraded aerial view images, corresponding high-quality reference images, key region masks, and scene category labels; wherein the degraded aerial view images are obtained through onboard sensor acquisition, or generated by applying one or more degradation processing methods such as fogging, low illumination, blurring, noise, and rain / snow occlusion to clear aerial view images. To enhance the focus on key areas in subsequent distillation processes, prior information about the regions in aerial images can be generated simultaneously. This prior information includes one or more of the following: runway region, obstacle region, horizon region, light signal region, and forward track region. It is stored in the form of region mask, saliency map, or semantic annotation.

[0031] Methods for acquiring aerial visual image data and constructing training sample sets include: First, raw image data acquired by the aviation visual enhancement system under different flight conditions is collected; the image data can be visible light images, infrared images, or visual images formed by the fusion of multimodal sensors; Then, the original image data is subjected to size unification, brightness normalization, noise suppression, and data augmentation to construct a training sample set; the training sample set is represented as follows: ;in, This indicates a degraded aerial view image. This represents the corresponding high-quality reference image. Indicates the critical region mask. Indicates scene category label; The total number of samples; When real, high-quality reference images are difficult to obtain, a degraded input image is generated from a clear, degraded aerial view image using a degradation model. The expression for this model is: ;in, This represents a degenerate generating function. Indicates the atomization intensity parameter. This represents the brightness attenuation parameter. Represents the fuzzy intensity parameter. This represents noise or rain / snow disturbance parameters. This method allows for the construction of a one-to-one correspondence between degraded images and reference images.

[0032] Furthermore, the key region mask Used to mark one or more of the following areas: runway area, obstacle area, horizon area, light signal area, and forward track area, so as to focus on constraining flight safety-related areas during subsequent distillation training.

[0033] Step S2: Construct a teacher's visual enhancement model and extract its output and intermediate layer features. During the forward inference process of the teacher model, extract the intermediate layer features of the teacher model for subsequent distillation constraints. The intermediate layer features may include shallow texture features, mid-level structural features, and deep semantic features to represent local details, target contours, and overall scene information in the aerial image, respectively.

[0034] Furthermore, in addition to outputting enhanced results and intermediate features, attention response maps, edge response maps, saliency response maps, or task branch outputs of the teacher's visual enhancement model can be extracted as auxiliary supervision information for the distillation learning of the lightweight student visual enhancement model.

[0035] The teacher's visual enhancement model is a deep convolutional neural network, an encoder-decoder network, an attention-based network, a visual Transformer network, or a combination thereof; the intermediate layer features extracted by the teacher's visual enhancement model include at least two or three of the following: shallow texture features, intermediate structural features, and deep semantic features.

[0036] For input samples The teacher's visual enhancement model outputs the enhancement results. Represented as: ;in, Indicating a teacher's visual enhancement model Indicates the teacher model parameters; In addition to outputting the augmented results, intermediate features are extracted from multiple levels of the teacher model for subsequent student model distillation supervision. The teacher's visual augmentation model... The layer features are: ;in, Indicates the number of characteristic layers used for distillation. These correspond to shallow texture features, mid-level structural features, and deep semantic features, respectively. Shallow features are used to depict local details and texture responses, mid-level features are used to represent contours and geometric structures, and deep features are used to represent scene semantics and important target information.

[0037] In this embodiment, a lightweight student visual enhancement model is constructed. The lightweight student vision enhancement model maintains the same input and output format as the teacher vision enhancement model, but adopts a lightweight design in terms of network structure to adapt to the computing power, power consumption and real-time constraints of the airborne embedded platform.

[0038] Step S3: Construct a lightweight student visual augmentation model for airborne deployment; The lightweight student visual enhancement model employs one or more of the following: depthwise separable convolution, grouped convolution, inverted residuals, lightweight attention modules, or low-complexity feature fusion structures; to ensure that the student model meets one or more constraints imposed by the airborne platform on the number of model parameters, computational complexity, inference latency, memory usage, and power consumption.

[0039] The input and output format of the lightweight student visual enhancement model is consistent with that of the teacher visual enhancement model, that is, the input is a degraded aerial visual image and the output is an enhanced aerial visual image, thereby ensuring that the supervised target in the distillation process is alignable.

[0040] During the model design phase, deployment constraints can be introduced simultaneously. These constraints include one or more of the following: model parameter count, floating-point operation volume, inference latency, video memory usage, and power consumption. These constraints are used to constrain the structural size of the student model so that it meets the real-time operation requirements of the aerial vision enhancement system.

[0041] For the same input sample Lightweight student visual enhancement model outputs enhancement results for: ;in, This represents the parameters of a lightweight student visual enhancement model; Similarly, features of the corresponding level are extracted from the lightweight student visual enhancement model: ; In practical deployment, lightweight student visual augmentation models must meet the resource constraints of the airborne platform, for example: ;in, This represents the computational complexity of the lightweight student visual augmentation model. This indicates the maximum computing resources allowed by the platform. Indicates the model inference latency. This indicates the maximum allowable response time of the system.

[0042] Step S4: Construct a joint distillation loss function based on the differences in output response, intermediate features, structural information, and task semantics between the teacher's visual enhancement model and the lightweight student's visual enhancement model, referring to... Figure 2 ; The constructed joint distillation loss function includes at least output response distillation loss, intermediate feature distillation loss, structural information distillation loss, and task semantic distillation loss; the joint distillation loss function can also incorporate one or more of reconstruction loss, perceptual loss, structural similarity loss, or adversarial loss from real reference images to jointly improve the realism and stability of the student model enhancement results.

[0043] In this embodiment, the training of the student model does not rely solely on supervision from real reference images, but simultaneously utilizes the output of the teacher model and its internal representation information for joint distillation.

[0044] The output response distillation loss, used to constrain the consistency between the outputs of the lightweight student visual enhancement model and the teacher visual enhancement model, is expressed as: The output response distillation loss enables the lightweight student visual enhancement model to learn the output response capabilities of the teacher visual enhancement model in terms of brightness restoration, contrast enhancement, and overall visual quality improvement. Intermediate feature distillation loss is used to constrain the consistency between the lightweight student visual augmentation model and the teacher visual augmentation model in the intermediate layer feature space; when the number of channels in the teacher visual augmentation model and the lightweight student visual augmentation model is inconsistent, a mapping function is introduced. Aligning student features, the intermediate feature distillation loss is: ;in, , , They represent the first The number of channels, height, and width of the layer feature map; To improve the student model's ability to preserve structural information such as runway edges, obstacle outlines, horizon lines, and lighting, a gradient domain constraint is used to construct a structural information distillation loss. The structural information distillation loss is used to constrain the consistency between the lightweight student vision enhancement model and the teacher vision enhancement model in at least one type of structural information, namely, edges, gradients, contours, frequency domain response, or local geometry, as shown in the following equation: ;in, and These represent the gradient operators for the image in the horizontal and vertical directions, respectively; To enhance the student model's ability to perceive key regions and high-level semantics, the teacher model can generate attention response maps or key region response maps. The student model generates a corresponding response graph. The task semantic distillation loss is used to constrain the consistency between the lightweight student vision enhancement model and the teacher vision enhancement model in at least one semantic information in the key region response map, attention distribution map, saliency response map, or auxiliary task branch output, as shown in the following equation: ;in, Attention response maps or key region response maps generated for teacher visual enhancement models. The corresponding response map generated for the lightweight student visual enhancement model; the response map is used to describe the model's attention to areas relevant to flight safety.

[0045] When the training samples contain high-quality reference images Furthermore, a reference image reconstruction loss can be introduced to improve the realism of the enhancement results. The reference image reconstruction loss is shown in the following formula: ; Based on the above-mentioned losses, the combined distillation loss function is expressed as: ;in, , , , , These are the weighting coefficients for each loss term. These weighting coefficients can be preset or adaptively adjusted during the training process.

[0046] Considering that areas such as runways, obstacles, horizon, and light signals are of greater importance to flight safety in aviation vision enhancement missions, this embodiment further introduces a regional weighting mechanism to enhance constraints on joint distillation losses in key regions.

[0047] Step S5: Generate region weights based on key regions in the aerial image, and perform weighted optimization on the joint distillation loss function to train a lightweight student visual enhancement model. (Refer to...) Figure 3 ; Key regions include one or more of the following: runway area, runway boundary area, obstacle area, horizon area, light signal area, forward trajectory area, and highly salient terrain features area. For these regions, a higher loss weight is assigned than that to the background area during distillation training, allowing the student model to preferentially learn the teacher model's enhancement capabilities for these key regions. A region weight map is generated based on the key regions, and this region weight map is used to weight one or more loss terms in the joint distillation loss function. The region weight map satisfies the following conditions: ;in, m Indicates the critical region mask. Indicates the enhancement coefficient of the key region. This represents a region weight map; when a pixel location belongs to a critical region, its loss weight increases.

[0048] The regional weight map can be represented by a binary mask, a continuous weight map, or a multi-level weight distribution map, and is applied to one or more loss terms in the output response distillation loss, intermediate feature distillation loss, structural information distillation loss, and task semantic distillation loss, respectively.

[0049] Based on the regional weight map, construct a regional weighted output distillation loss: ;in, Indicates the first The region weight map corresponding to each training sample This indicates element-wise multiplication.

[0050] In step S5, further, it can be based on the scene category. Dynamically adjust the weights of each loss term; let the scene adaptive weight function be... The total loss is then written as: ;in, ; This means that different distillation intensities can be used in different degradation scenarios. For example, in foggy scenarios, the weights of structural distillation and regional distillation can be increased, while in nighttime scenarios, the weights of output response distillation and semantic distillation can be increased.

[0051] This invention also dynamically adjusts the weights of each loss term in the joint distillation loss function based on the scene category label corresponding to the input aerial visual image. The scene category includes at least one or more of low visibility scenes, low light scenes, fog / haze scenes, rain / snow scenes, and backlight scenes, to improve the enhancement effect and robustness of the student model under different degradation environments. To enhance adaptability under different scene conditions, the loss weights can be dynamically adjusted based on the degradation type, visibility level, illumination state, and scene complexity of the input image, enabling the distillation strategy to have better robustness under different conditions such as fog, nighttime, backlight, rain, and snow.

[0052] During training, the parameters of the lightweight student visual augmentation model are updated using gradient descent. ;in, The learning rate is used; after multiple rounds of iterative training, a lightweight student visual augmentation model that satisfies the performance enhancement and deployment constraints is obtained.

[0053] The student model is trained by backpropagation and iterative optimization until the joint distillation loss function converges or the preset number of training rounds is reached, resulting in a lightweight student model that has been trained.

[0054] Step S6: Deploy the trained lightweight student visual enhancement model into the aviation visual enhancement system to perform real-time enhancement processing on the input aviation visual images. The deployed lightweight student visual enhancement model can perform online enhancement of aviation visual images in low visibility, low illumination, complex weather, and dynamic flight scenarios, improving image brightness, contrast, texture clarity, and structural discernibility, while meeting the real-time requirements of the airborne system. The enhanced images can be directly output to the flight display terminal for pilots to visually perceive; or further input to downstream modules such as target detection, obstacle recognition, runway localization, and landing assistance to improve the overall mission performance of the aviation visual enhancement system. For further optimization, during the deployment phase, model pruning, mixed-precision inference, or module switching control can be performed on the student model based on real-time feedback from the airborne platform to further balance the enhancement effect with system resource consumption.

[0055] After training, the lightweight student visual augmentation model is deployed to the airborne processing platform corresponding to the aviation visual augmentation system, referencing... Figure 4 The platform can be an embedded GPU, FPGA, edge computing module, or dedicated inference chip. The deployed lightweight student visual augmentation model receives real-time input images from onboard sensors. and output the enhanced results. : ;in, This represents the optimal parameters of the lightweight student visual augmentation model after training.

[0056] In practical applications, the enhancement results can be directly output to the pilot's display terminal to improve the pilot's visual perception in complex environments; they can also be further input to downstream modules such as target detection, obstacle recognition, runway positioning, or assisted landing, thereby improving the overall mission performance of the aviation visual enhancement system.

[0057] Through the aforementioned steps, this invention achieves knowledge transfer from a high-performance teacher model to a lightweight student model, enabling the student model to achieve superior aerial vision augmentation capabilities with lower computational resource consumption. Compared to existing methods that solely rely on model pruning, quantization, or lightweight structural design, this invention not only reduces the number of model parameters and inference latency but also more effectively preserves structural and semantic information in key flight safety-related areas, thereby improving the clarity, stability, and mission usability of augmentation results in complex flight environments.

[0058] This invention also provides a model knowledge distillation system for aviation vision enhancement systems, comprising: The data construction module is used to acquire aerial visual image data and build a training sample set; The teacher model building module is used to build a teacher visual enhancement model and extract the output results and intermediate layer features of the teacher visual enhancement model. The student model building module is used to build lightweight student visual augmentation models for airborne deployment. The joint distillation module is used to construct a joint distillation loss function based on the differences in output response, intermediate features, structural information, and task semantics between the teacher visual enhancement model and the lightweight student visual enhancement model, and to train the lightweight student visual enhancement model. The region weighting module is used to generate region weights based on key regions in the aerial visual image and to perform weighted optimization on the joint distillation loss function; The deployment module is used to deploy the trained lightweight student visual enhancement model to the aviation visual enhancement system to perform real-time enhancement processing on the input aviation visual images.

[0059] This embodiment provides a model knowledge distillation method and system for aviation visual enhancement systems, applicable to aviation visual image enhancement tasks in complex flight environments such as low visibility, low light, fog, rain, snow, and backlight. This method transfers the representational capabilities of a high-performance enhancement model to a lightweight model suitable for airborne deployment by constructing a multi-layered knowledge transfer relationship between a teacher model and a lightweight student model. This reduces the number of model parameters, computational load, and inference latency while maintaining the enhancement effect.

[0060] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A model knowledge distillation method for aerial visual enhancement systems, characterized in that, Includes the following steps: Step S1: Acquire aerial visual image data and construct a training sample set; Step S2: Construct a teacher visual enhancement model and extract the output results and intermediate layer features of the teacher visual enhancement model; Step S3: Construct a lightweight student visual augmentation model for airborne deployment; Step S4: Construct a joint distillation loss function based on the differences in output response, intermediate features, structural information, and task semantics between the teacher visual enhancement model and the lightweight student visual enhancement model; Step S5: Generate region weights based on key regions in the aerial image, and perform weighted optimization on the joint distillation loss function to train the lightweight student visual enhancement model. Step S6: Deploy the trained lightweight student visual enhancement model into the aviation visual enhancement system to perform real-time enhancement processing on the input aviation visual images.

2. The model knowledge distillation method for an aviation visual enhancement system according to claim 1, characterized in that, In step S1, the method for acquiring aerial visual image data and constructing a training sample set includes: First, raw image data acquired by the aviation visual enhancement system under different flight conditions is collected; the image data is visible light image, infrared image, or visual image formed by fusion of multimodal sensors; Then, the original image data is subjected to size unification, brightness normalization, noise suppression, and data augmentation to construct a training sample set; the training sample set is represented as follows: ;in, This indicates a degraded aerial view image. This represents the corresponding high-quality reference image. Indicates the critical region mask. Indicates scene category label; The total number of samples; When real, high-quality reference images are difficult to obtain, a degraded input image is generated from a clear, degraded aerial view image using a degradation model. The expression for this model is: ;in, This represents a degenerate generating function. Indicates the atomization intensity parameter. This represents the brightness attenuation parameter. Represents the fuzzy intensity parameter. This indicates parameters related to noise or rain / snow disturbance.

3. The model knowledge distillation method for an aviation visual enhancement system according to claim 2, characterized in that, In step S2, the teacher's visual enhancement model is a deep convolutional neural network, an encoder-decoder network, an attention-based network, a visual Transformer network, or a combination thereof; the intermediate layer features extracted by the teacher's visual enhancement model include at least two or three of the following: shallow texture features, intermediate structural features, and deep semantic features.

4. The model knowledge distillation method for an aviation visual enhancement system according to claim 3, characterized in that, In step S2, for the input sample The teacher's visual enhancement model outputs the enhancement results. Represented as: ;in, Indicating a teacher's visual enhancement model Indicates the teacher model parameters; Teacher Visual Enhancement Model The layer features are: ;in, Indicates the number of characteristic layers used for distillation. These correspond to shallow texture features, mid-level structural features, and deep semantic features, respectively. Shallow features are used to depict local details and texture responses, mid-level features are used to represent contours and geometric structures, and deep features are used to represent scene semantics and important target information.

5. The model knowledge distillation method for an aviation visual enhancement system according to claim 4, characterized in that, In step S3, the lightweight student visual enhancement model employs one or more of the following: depthwise separable convolution, grouped convolution, inverted residual, lightweight attention module, or low-complexity feature fusion structure. For the same input sample Lightweight student visual enhancement model outputs enhancement results for: ;in, This represents the parameters of a lightweight student visual enhancement model; Extracting corresponding hierarchical features from a lightweight student visual enhancement model: ; The lightweight student visual augmentation model must meet the resource constraints of the airborne platform: ;in, This represents the computational complexity of the lightweight student visual augmentation model. This indicates the maximum computing resources allowed by the platform. Indicates the model inference latency. This indicates the maximum allowable response time of the system.

6. The model knowledge distillation method for an aviation visual enhancement system according to claim 5, characterized in that, In step S4, the constructed joint distillation loss function includes at least the output response distillation loss, intermediate feature distillation loss, structural information distillation loss, and task semantic distillation loss. The output response distillation loss, used to constrain the consistency between the outputs of the lightweight student visual enhancement model and the teacher visual enhancement model, is expressed as: The output response distillation loss enables the lightweight student visual enhancement model to learn the output response capabilities of the teacher visual enhancement model in terms of brightness restoration, contrast enhancement, and overall visual quality improvement. Intermediate feature distillation loss is used to constrain the consistency between the lightweight student visual augmentation model and the teacher visual augmentation model in the intermediate layer feature space; when the number of channels in the teacher visual augmentation model and the lightweight student visual augmentation model is inconsistent, a mapping function is introduced. Aligning student features, the intermediate feature distillation loss is: ;in, , , They represent the first The number of channels, height, and width of the layer feature map; The structural information distillation loss is used to constrain the consistency between the lightweight student vision enhancement model and the teacher vision enhancement model in at least one type of structural information, namely, edges, gradients, contours, frequency domain response, or local geometry, as shown in the following equation: ;in, and These represent the gradient operators for the image in the horizontal and vertical directions, respectively; The task semantic distillation loss is used to constrain the consistency between the lightweight student vision enhancement model and the teacher vision enhancement model in at least one semantic information in the key region response map, attention distribution map, saliency response map, or auxiliary task branch output, as shown in the following equation: ;in, Attention response maps or key region response maps generated for teacher visual enhancement models. Response map generated for a lightweight student visual enhancement model; The reference image reconstruction loss is shown in the following formula: ; Based on the above-mentioned losses, the combined distillation loss function is expressed as: ;in, , , , , These are the weighting coefficients for each loss term.

7. The model knowledge distillation method for an aviation visual enhancement system according to claim 6, characterized in that, In step S5, the key regions include one or more of the following: runway region, runway boundary region, obstacle region, horizon region, light signal region, forward track region, and highly salient feature region. A region weight map is generated based on the key regions, and this region weight map is used to weight one or more loss terms in the joint distillation loss function. The region weight map satisfies the following conditions: Where m represents the critical region mask, Indicates the enhancement coefficient of the key region. Represents a region weighting map; Based on the regional weight map, construct a regional weighted output distillation loss: ;in, Indicates the first The region weight map corresponding to each training sample This indicates element-wise multiplication.

8. The model knowledge distillation method for an aviation visual enhancement system according to claim 7, characterized in that, In step S5, according to the scene category Dynamically adjust the weights of each loss item; Let the scene adaptive weight function be... The total loss is then written as: ;in, ; During training, the parameters of the lightweight student visual augmentation model are updated using gradient descent. ;in, The learning rate is used; after multiple rounds of iterative training, a lightweight student visual augmentation model that satisfies the performance enhancement and deployment constraints is obtained.

9. A model knowledge distillation method for an aviation visual enhancement system according to claim 8, characterized in that, In step S6, the deployed lightweight student visual augmentation model receives real-time input images from airborne sensors. and output the enhanced results. : ;in, This represents the optimal parameters of the lightweight student visual augmentation model after training.

10. A model knowledge distillation system for aviation visual enhancement systems, characterized in that, include: The data construction module is used to acquire aerial visual image data and build a training sample set; The teacher model building module is used to build a teacher visual enhancement model and extract the output results and intermediate layer features of the teacher visual enhancement model. The student model building module is used to build lightweight student visual augmentation models for airborne deployment. The joint distillation module is used to construct a joint distillation loss function based on the differences in output response, intermediate features, structural information, and task semantics between the teacher visual enhancement model and the lightweight student visual enhancement model, and to train the lightweight student visual enhancement model. The region weighting module is used to generate region weights based on key regions in the aerial visual image and to perform weighted optimization on the joint distillation loss function; The deployment module is used to deploy the trained lightweight student visual enhancement model to the aviation visual enhancement system to perform real-time enhancement processing on the input aviation visual images.

Images