A method and system for automatic aerial visibility recognition from remote control towers by fusing camera optical parameters

Abstract

This invention relates to the fields of aviation meteorology and computer vision interdisciplinary technologies, and discloses a method and system for automatic identification of aviation visibility in remote control towers by fusing camera optical parameters. The method involves acquiring high-resolution images and simultaneously recording optical metadata through a main camera unit; performing a series of preprocessing steps on the acquired images to generate standardized image data; employing an innovative DPIN-RVR network structure, the core of which involves processing the standardized image data through a Vision Transformer backbone while processing the optical metadata through parameter branches, and deeply fusing the two types of information at each layer of the backbone using a FiLM fusion mechanism; subsequently, outputting RVR values ​​and uncertainty estimates through a regression head; finally, using a Kalman filter to perform temporal fusion of the RVR estimation results for consecutive frames, and performing anomaly detection with traditional instrument data to ensure reliability; and also disclosing a specific data augmentation training method that significantly improves the model's generalization ability and accuracy under complex lighting and meteorological conditions.

Description

Technical Field

[0001] This invention relates to the fields of aviation meteorology and computer vision interdisciplinary technologies, specifically, to a method and system for automatic identification of aviation visibility in remote control towers that integrates camera optical parameters. Background Technology

[0002] Atmospheric visibility is a physical quantity that measures atmospheric transparency and plays a crucial role in many industries, especially in transportation safety. Visibility is generally defined as the distance at which an object is "just clearly visible" or identifiable. In aviation, dominant visibility and runway visual range (RVR) are key parameters for aircraft takeoff and landing, as well as airport opening and closing. RVR specifically refers to the distance a pilot can see on the runway, directly impacting aircraft takeoff and landing safety and ground operational safety. Visibility is affected by various factors, including air pollutant emissions, sunlight, humidity, temperature, and time of day.

[0003] Traditionally, the definition of visibility has been closely related to human visual perception, providing a theoretical basis for image-based visibility estimation methods. If algorithms can interpret images like the human eye (or even more stably and accurately), then image-based methods may more directly simulate human perception of visibility than traditional methods that indirectly measure optical properties and then convert them into visibility values.

[0004] Traditional visibility measurement methods mainly include visual inspection and instrumental measurement. Visual inspection relies on trained observers to identify known distant targets. Instrumental measurement primarily uses equipment such as transmissometers or forward-scatter sensors.

[0005] However, these traditional methods have some inherent limitations: High costs and maintenance issues: Instruments and equipment are usually expensive and have high maintenance costs.

[0006] Limitations of point measurement: It can only perform point measurements, that is, the air transparency within a very small area where the instrument is located.

[0007] Insufficient representativeness: Single-point measurements may not represent the true visibility conditions over a wider area (such as the entire airport or runway).

[0008] Measurement error: The estimates generated by the forward scattering device (forward scatterer) may be incorrect, and the measurement results in approximately 23% of the captured images are misleading.

[0009] Extrapolation error: Extrapolating point measurement data to a greater distance may introduce errors.

[0010] Image-based visibility estimation methods have evolved from relying on physical models to traditional computer vision feature engineering, and now to the mainstream approach of data-driven deep learning.

[0011] Traditional methods based on physical models: Koschmieder's law is the theoretical foundation of many early image visibility estimation methods. This law describes the relationship between the apparent brightness of an object and its inherent brightness, viewing distance, and atmospheric extinction coefficient. Koschmieder's Law: ,visibility .

[0012] in, Indicates the atmospheric extinction coefficient; C th This is the contrast threshold.

[0013] Based on this law, researchers have developed methods that rely on measuring the contrast between the target object and its background. However, these methods face many challenges: Sensitivity to changes in light: It is very sensitive to drastic changes in light conditions (such as day-night cycle, shadows); Scene dependency: depends on the presence of a specific target object or a specific geometric structure in the scene; Limited ability to handle complex weather conditions: It is difficult to effectively handle complex situations such as non-uniform haze or a mixture of multiple weather phenomena.

[0014] The rise of deep learning methods: In recent years, various deep learning models have been applied to visibility estimation tasks. Convolutional Neural Networks (CNNs): CNNs are widely used for visibility estimation due to their powerful spatial feature extraction capabilities. Classic CNN architectures (such as AlexNet, VGG, ResNet) and their improved versions are used to automatically learn visibility-related features from images.

[0015] Recurrent Neural Networks (RNNs): RNNs and their variants (such as LSTM and GRU) are adept at processing sequential data and are suitable for analyzing video image sequences or combining them with time-series meteorological data.

[0016] Generative Adversarial Networks (GANs): GANs excel in image restoration tasks, such as image dehazing or de-hazing.

[0017] Transformer Networks: Transformer models were introduced into the field of computer vision because of their ability to capture long-range dependencies and global contextual information in images.

[0018] Limitations of existing research in the aerospace field: While image-based visibility estimation has been applied in other fields, it still has significant limitations in the aviation sector: Categorization rather than precise measurement: Many studies focus on classifying airport visibility into several broad categories rather than providing the precise values ​​required by RVR; Specific limitations: Some studies were conducted only under specific weather conditions or under limited conditions; Human dependency: Camera images still require manual interpretation; Optical parameters are ignored: existing algorithms mostly extract features from the pixels themselves, ignoring the nonlinear stretching-compression effect of different exposure combinations on grayscale distribution. Summary of the Invention

[0019] The purpose of this invention is to address the shortcomings of existing technologies and provide a method and system for automatic remote tower aviation visibility identification by fusing camera optical parameters. The method aims to provide accurate RVR measurement, effectively fusing optical metadata, thereby improving system reliability while reducing operating costs. The system is used to implement the method and aims to provide accurate RVR measurement, effectively fusing optical metadata, thereby improving system reliability while reducing operating costs.

[0020] This invention is achieved through the following technical solution: A method for automatic visibility recognition of remote control towers by fusing camera optical parameters, characterized by comprising the following steps: 1) Acquire high-resolution images (preferably 4K) through the main camera unit and simultaneously record optical metadata; 2) Perform depigmentation, white balance correction, distortion correction, and photometric normalization on the acquired images to generate standardized image data; 3) The DPIN-RVR (Data-Parameter Injection Network for Runway VisualRange) network structure is adopted. The standardized image data is processed through the Vision Transformer backbone, the optical metadata is processed through the parameter branch, and the feature information of the standardized image data and the feature information of the optical metadata are fused through the FiLM fusion mechanism in multiple layers (preferably each layer) of the Vision Transformer backbone. 4) RVR estimation steps: After step 3), the RVR value and the corresponding uncertainty estimate are output through the regression head; 5) Temporal fusion and anomaly detection steps: The RVR estimation results of consecutive frames in the time series are fused using a Kalman filter and compared with conventional forward scattering instrument data (for comparison) to perform anomaly detection; 6) Output steps: After step 5), the final RVR result is output in METAR / SPECI (WMO No.306, FM 15) format, and IWXXM format can also be provided at the same time.

[0021] To further improve the implementation of the remote tower aviation visibility automatic recognition method based on camera optical parameters described in this invention, the following configuration is specifically adopted: The main camera unit consists of a CMOS image sensor (e.g., a Sony IMX485 CMOS sensor), a programmable auto-iris lens (e.g., a C-Mount programmable auto-iris lens), and an ambient light sensor (e.g., a Si1133 ambient light sensor), supporting an exposure time range of 1 / 1,000,000 sec to 30 sec and a gain range of ISO 100-12,800; wherein, the Sony IMX485 CMOS sensor is 4K (3840×2160) with a pixel size of 2.9µm; the C-Mount programmable auto-iris lens is f / 1.4–f / 16 with a switchable IR-cut filter; exposure control: electronic shutter 1 / 1,000,000s–30s, ISO 100–12,800; the ambient light sensor is Si1133 (0.1–100). (klx); Synchronization interface: GigE Vision supports PTPv2 / GPS PPS hardware synchronization; Protection rating: IP66, -40℃~+60℃; The optical metadata includes one or more of the following: ISO value, exposure time, F-Number (aperture value), ambient light intensity, camera temperature, exposure gain, and white balance parameters; The optical metadata is synchronously acquired with each frame of image and encapsulated in a JSON format metadata file.

[0022] To further improve the implementation of the remote tower aviation visibility automatic identification method based on camera optical parameters described in this invention, the following configuration is specifically adopted: The DPIN-RVR network structure includes: a Patch Embedding layer that segments the input image into 16×16 pixel blocks and maps them to a 768-dimensional feature space; a Vision Transformer backbone consisting of 12 Vision Transformer layers, each containing a multi-head self-attention mechanism and a feedforward network; a parameter branch that processes optical metadata into modulation parameters through a fully connected network; a regression head for outputting runway visual range (RVR) values ​​and their uncertainty estimates; and a FiLM fusion mechanism applied to the image features in each of the 12 Vision Transformer layers.

[0023] To further improve the automatic visibility recognition method for remote tower aviation based on fused camera optical parameters as described in this invention, the following configuration is specifically adopted: In the temporal fusion and anomaly detection step, the state vector of the Kalman filter includes the RVR value and the RVR rate of change, and the estimation results of consecutive frames are smoothed through a prediction-update loop; the anomaly detection mechanism is as follows: when the RVR relative error output by the Kalman filter exceeds a preset threshold (preferably 15%) and is present for at least 3 consecutive frames, the output of the automatic visibility recognition method for remote tower aviation based on fused camera optical parameters is automatically reverted to the measurement value measured by the conventional forward scattering instrument; that is, the detection mechanism when comparing with conventional forward scattering instrument data (for comparison) to perform anomaly detection is as follows: when And continue T An anomaly is triggered when there are ≥3 frames; the system enters fallback mode and uses the forward scatterometer reading RVR. FS As output, until continuous T rec ≥5 frames recovered to within the threshold.

[0024] To further improve the automatic visibility recognition method for remote control tower aviation based on the fusion of camera optical parameters as described in this invention, the following configuration is specifically adopted: it further includes a step of data augmentation of the standardized image data during the training phase of the DPIN-RVR network structure. The data augmentation includes at least one of physical fogging, exposure perturbation, geometric transformation and noise injection based on the Koschmieder model, which is used to improve generalization ability.

[0025] A remote control tower aviation visibility automatic recognition system that integrates camera optical parameters is used to implement the aforementioned remote control tower aviation visibility automatic recognition method that integrates camera optical parameters, comprising: The main camera unit is used to acquire high-resolution images (preferably 4K) and simultaneously record optical metadata; The data processing unit is used to perform de-mosaicing, white balance correction, distortion correction, and photometric normalization on the acquired images to generate standardized image data. The algorithm inference unit is used to process the standardized image data through the VisionTransformer backbone and the optical metadata through the parameter branch using the DPIN-RVR network structure. The two types of information are fused through the FiLM fusion mechanism in multiple layers (preferably each layer) of the VisionTransformer backbone, and the runway visual range value and its uncertainty estimate are output through the regression head. The temporal fusion unit is used to perform temporal fusion of the RVR estimation results of consecutive frames in the time series using a Kalman filter, and compare them with traditional forward scattering instrument data to perform anomaly detection; when the relative error of at least 3 consecutive frames exceeds a preset threshold (preferably 15%), the forward scattering instrument reading is automatically backed up. The output unit is used to output the final RVR result in METAR / SPECI (WMO No. 306, FM 15) format, and can also provide IWXXM format.

[0026] To further improve the realization of the remote tower aviation visibility automatic recognition system integrating camera optical parameters described in this invention, the following configuration is specifically adopted: The main camera unit consists of a CMOS image sensor (e.g., a Sony IMX485 CMOS sensor), a programmable auto-iris lens (e.g., a C-Mount programmable auto-iris lens), and an ambient light sensor (e.g., a Si1133 ambient light sensor), supporting an exposure time range of 1 / 1,000,000 sec to 30 sec and a gain range of ISO 100-12,800; wherein, the Sony IMX485 CMOS sensor is 4K (3840×2160) with a pixel size of 2.9µm; the C-Mount programmable auto-iris lens is f / 1.4–f / 16 with a switchable IR-cut filter; exposure control: electronic shutter 1 / 1,000,000s–30s, ISO 100–12,800; the ambient light sensor is Si1133 (0.1–100). (klx); Synchronization interface: GigE Vision supports PTPv2 / GPS PPS hardware synchronization; Protection rating: IP66, -40℃~+60℃; The optical metadata includes one or more of the following: ISO value, exposure time, F-Number (aperture value), ambient light intensity, camera temperature, exposure gain, and white balance parameters; The optical metadata is synchronously acquired with each frame of image and encapsulated in a JSON format metadata file.

[0027] To further improve the implementation of the remote tower aviation visibility automatic recognition system that integrates camera optical parameters as described in this invention, the following configuration is specifically adopted: the algorithm inference unit adopts a DPIN-RVR network structure, including: a Patch Embedding layer that segments the input image into 16×16 pixel blocks and maps them to a 768-dimensional feature space; a Vision Transformer backbone consisting of 12 Vision Transformer layers, each Vision Transformer layer containing a multi-head self-attention mechanism and a feedforward network; a parameter branch that processes optical metadata into modulation parameters through a fully connected network; a regression head for outputting runway visual range (RVR) values ​​and their uncertainty estimates; and a FiLM fusion mechanism applied to the image features in each of the 12 Vision Transformer layers.

[0028] To further improve the implementation of the remote tower aviation visibility automatic recognition system based on fused camera optical parameters as described in this invention, the following configuration is specifically adopted: The temporal fusion unit includes a Kalman filter, whose state vector includes the RVR value and RVR rate of change, and smooths the estimation results of consecutive frames through a prediction-update loop; the unit also includes an anomaly detection mechanism, which automatically reverts the output of the remote tower aviation visibility automatic recognition system based on fused camera optical parameters to the measurement value measured by the conventional forward scattering instrument when the RVR relative error output by the Kalman filter exceeds a preset threshold (preferably 15%) for at least 3 consecutive frames; that is, the anomaly detection mechanism is: when And continue T An anomaly is triggered when there are ≥3 frames; the system enters rollback mode and uses the forward scatterometer reading. As output, until continuous T rec ≥5 frames recovered to within the threshold.

[0029] To further improve the implementation of the remote tower aviation visibility automatic recognition system that integrates camera optical parameters as described in this invention, the following configuration is specifically adopted: the DPIN-RVR network structure in the algorithm inference unit is obtained through a training method including the following steps: data augmentation is performed on the standardized image data used for training, the data augmentation including at least one of physical fogging, exposure perturbation, geometric transformation and noise injection based on the Koschmieder model, to improve generalization ability.

[0030] Compared with the prior art, the present invention has the following advantages and beneficial effects: This invention provides an automated method for providing accurate RVR measurements, capable of stably outputting accurate RVR values ​​under day and night conditions and complex lighting conditions.

[0031] This invention establishes an end-to-end estimation method that synchronously acquires and explicitly utilizes camera optical metadata to achieve the goal of fusing optical metadata.

[0032] This invention improves system reliability and enables all-weather, cross-device consistency and traceability.

[0033] This invention reduces reliance on expensive specialized equipment and utilizes existing video systems in remote control towers, thereby reducing operating costs.

[0034] Regarding accuracy improvement, the measurement RMSE of this invention is approximately ±28 meters. According to industry operational expectations, the required accuracy is ≤±10 meters for RVR≤400 meters, ≤±25 meters for 400–800 meters, and ≤±10% for >800 meters. In terms of response speed, its real-time processing capability is ≤18ms per frame, representing a more than 90% improvement compared to the 30–60 seconds response time of traditional instruments.

[0035] This invention expands the coverage area; a single system can cover the entire runway (typically 2-4 kilometers), which is more than 100 times more effective than point measurement coverage.

[0036] This invention is adaptable to all weather conditions, operates stably in a temperature range of -40℃ to +60℃, and supports continuous operation day and night.

[0037] In terms of cost reduction, compared to traditional RVR equipment (RMB 500,000-1,000,000 per unit), the system cost of this invention is reduced by 60-70%. Furthermore, this invention utilizes existing remote tower camera infrastructure, reducing installation costs by 50%.

[0038] The annual maintenance cost of this invention is reduced by 80% compared to traditional equipment, mainly through software upgrades to improve functionality.

[0039] This invention features data consistency: multi-point collaborative measurement eliminates the problem of insufficient representativeness in traditional point measurement.

[0040] This invention is traceable: it fully records optical parameters and processing procedures, meeting civil aviation regulatory requirements.

[0041] This invention has the following characteristics in terms of industrial applicability: It has broad market demand; approximately 4,000 airports worldwide may deploy remote control tower systems, with a market size estimated at $20 billion. It can be widely applied: it can be extended to other visibility monitoring scenarios such as marine environments, roads, and industrial parks. Detailed Implementation

[0042] The present invention will be further described in detail below with reference to embodiments, but the implementation of the present invention is not limited thereto.

[0043] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to represent selected embodiments of the invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0045] Example 1: This embodiment discloses an automatic remote tower aviation visibility identification method that fuses camera optical parameters, aiming to provide accurate RVR measurements. It effectively fuses optical metadata, improving system reliability while reducing operating costs, and includes the following steps: 1) Acquire high-resolution images (preferably 4K in this embodiment) through the main camera unit, and (or simultaneously) record optical metadata such as ISO value, aperture parameters, exposure time, and ambient light intensity; 2) Perform depigmentation, white balance correction, distortion correction, and photometric normalization on the acquired images to generate standardized image data; 3) The DPIN-RVR (Data-Parameter Injection Network for Runway VisualRange) network structure is adopted. The standardized image data is processed through the Vision Transformer backbone, and the optical metadata is processed through the parameter branch. The two types of information (i.e., the feature information obtained by processing the standardized image data through the Vision Transformer and the feature information obtained by processing the optical metadata through the parameter branch) are fused through the FiLM fusion mechanism in multiple layers (preferably each layer) of the Vision Transformer backbone. 4) RVR estimation steps: After step 3), the RVR value and the corresponding uncertainty estimate are output through the regression head; 5) Temporal fusion and anomaly detection steps: The RVR estimation results of consecutive frames in the time series are fused using a Kalman filter and compared with conventional forward scattering instrument data (for comparison) to perform anomaly detection; 6) Output steps: After step 5), the final RVR result is output in METAR / SPECI (WMO No. 306, FM 15) format, and IWXXM format can also be provided at the same time; software development follows the software assurance requirements of EUROCAE ED-109A / RTCA DO-278A.

[0046] Example 2: This embodiment is a further optimization based on the above embodiment. The similarities with the aforementioned technical solutions will not be repeated here. Furthermore, to achieve the remote tower aviation visibility automatic recognition method based on camera optical parameters described in this invention, the following configuration is specifically adopted: The main camera unit is composed of a CMOS image sensor (e.g., a Sony IMX485 CMOS sensor), a programmable auto-iris lens (e.g., a C-Mount programmable auto-iris lens), and an ambient light sensor (e.g., a Si1133 ambient light sensor), supporting an exposure time range of 1 / 1,000,000 s to 30 s and a gain range of ISO 100-12,800; wherein, the Sony IMX485 CMOS sensor is 4K (3840×2160) with a pixel size of 2.9µm; the C-Mount programmable auto-iris lens is f / 1.4–f / 16 with a switchable IR-cut filter; exposure control: electronic shutter 1 / 1,000,000s–30s, ISO 100–12,800; Ambient light sensor Si1133 (0.1–100 klx); Synchronization interface: GigEVision supports PTPv2 / GPS PPS hardware synchronization; Protection rating: IP66, -40℃~+60℃; The optical metadata includes one or more of the following: ISO value, exposure time, F-Number (aperture value), ambient light intensity, camera temperature, exposure gain, and white balance parameters; The optical metadata is acquired synchronously with each frame of image and encapsulated in a JSON format metadata file.

[0047] Example 3: This embodiment is a further optimization based on any of the above embodiments. The similarities with the aforementioned technical solutions will not be repeated here. Furthermore, to achieve the remote tower aviation visibility automatic recognition method based on fused camera optical parameters as described in this invention, the following configuration is specifically adopted: The DPIN-RVR network structure includes: a Patch Embedding layer that segments the input image into 16×16 pixel blocks and maps them to a 768-dimensional feature space; a VisionTransformer backbone consisting of 12 VisionTransformer layers, each containing a multi-head self-attention mechanism and a feedforward network; a parameter branch that processes optical metadata into modulation parameters through a fully connected network; a regression head for outputting runway visual range (RVR) values ​​and their uncertainty estimates; and a FiLM fusion mechanism applied to the image features in each of the 12 VisionTransformer layers.

[0048] Example 4: This embodiment is a further optimization based on any of the above embodiments. The similarities with the aforementioned technical solutions will not be repeated here. Furthermore, to further realize the automatic remote tower aviation visibility recognition method based on fused camera optical parameters described in this invention, the following settings are specifically adopted: In the temporal fusion and anomaly detection steps, the state vector of the Kalman filter includes the RVR value and the RVR rate of change, and smoothing of the estimation results for consecutive frames is achieved through a prediction-update loop; the anomaly detection mechanism is as follows: when the RVR relative error output by the Kalman filter exceeds a preset threshold (preferably 15%) and is present for at least 3 consecutive frames, the output of the automatic remote tower aviation visibility recognition method based on fused camera optical parameters is automatically reverted to the measurement value using the traditional forward scattering instrument; that is, the detection mechanism when comparing with traditional forward scattering instrument data (for comparison) to perform anomaly detection is as follows: when And continue T An anomaly is triggered when there are ≥3 frames; the system enters fallback mode and uses the forward scatterometer reading RVR. FS As output, until continuous T rec ≥5 frames recovered to within the threshold.

[0049] Example 5: This embodiment is a further optimization based on any of the above embodiments. The similarities with the aforementioned technical solutions will not be repeated here. In order to further realize the remote tower aviation visibility automatic recognition method that integrates camera optical parameters as described in this invention, the following setting is specifically adopted: it also includes a step of data augmentation of the standardized image data during the training phase of the DPIN-RVR network structure. The data augmentation includes at least one of physical fogging, exposure perturbation, geometric transformation and noise injection based on the Koschmieder model, which is used to improve the generalization ability.

[0050] Example 6: This embodiment discloses a remote control tower aviation visibility automatic recognition system that fuses camera optical parameters, used to implement the aforementioned remote control tower aviation visibility automatic recognition method that fuses camera optical parameters, including: The main camera unit is used to acquire high-resolution images (preferably 4K in this embodiment) and record optical metadata; The data processing unit is used to perform de-mosaicing, white balance correction, distortion correction, and photometric normalization on the acquired images to generate standardized image data. The algorithm inference unit is used to process the standardized image data through the VisionTransformer backbone and the optical metadata through the parameter branch using the DPIN-RVR network structure. The two types of information are fused through the FiLM fusion mechanism in multiple layers (preferably each layer) of the VisionTransformer backbone, and the runway visual range value and its uncertainty estimate are output through the regression head. The temporal fusion unit is used to perform temporal fusion of the RVR estimation results of consecutive frames in the time series using a Kalman filter, and compare them with traditional forward scattering instrument data to perform anomaly detection; when the relative error of at least 3 consecutive frames exceeds a preset threshold (preferably 15%), the forward scattering instrument reading is automatically backed up. The output unit is used to output the final RVR result in METAR / SPECI (WMO No. 306, FM 15) format, and can also provide IWXXM format.

[0051] Example 7: This embodiment is a further optimization based on the above embodiments. The parts identical to the aforementioned technical solutions will not be repeated here. Furthermore, to better realize the remote tower aviation visibility automatic recognition system that integrates camera optical parameters as described in this invention, the following configuration is specifically adopted: The main camera unit consists of a CMOS image sensor (e.g., a Sony IMX485 CMOS sensor), a programmable auto-iris lens (e.g., a C-Mount programmable auto-iris lens), and an ambient light sensor (e.g., a Si1133 ambient light sensor), supporting an exposure time range of 1 / 1,000,000 seconds to 30 seconds and a gain range of ISO 100-12,800; wherein, the Sony IMX485 CMOS sensor is 4K (3840×2160) with a pixel size of 2.9µm; the C-Mount programmable auto-iris lens is f / 1.4–f / 16 with a switchable IR-cut filter; exposure control: electronic shutter 1 / 1,000,000s–30s, ISO 100–12,800; Ambient light sensor Si1133 (0.1–100 klx); Synchronization interface: GigE Vision supports PTPv2 / GPS PPS hardware synchronization; Protection rating: IP66, -40℃~+60℃; The optical metadata includes one or more of the following: ISO value, exposure time, F-Number (aperture value), ambient light intensity, camera temperature, exposure gain, and white balance parameters; The optical metadata is acquired synchronously with each frame of image and encapsulated in a JSON format metadata file.

[0052] Example 8: This embodiment is a further optimization based on embodiment 6 or 7. The parts identical to the aforementioned technical solutions will not be repeated here. Furthermore, to better realize the remote tower aviation visibility automatic recognition system that fuses camera optical parameters as described in this invention, the following configuration is specifically adopted: The algorithm inference unit adopts a DPIN-RVR network structure, including: a Patch Embedding layer that segments the input image into 16×16 pixel blocks and maps them to a 768-dimensional feature space; a VisionTransformer backbone consisting of 12 VisionTransformer layers, each containing a multi-head self-attention mechanism and a feedforward network; a parameter branch that processes optical metadata into modulation parameters through a fully connected network; a regression head for outputting runway visual range (RVR) values ​​and their uncertainty estimates; and a FiLM fusion mechanism applied to the image features in each of the 12 VisionTransformer layers.

[0053] Example 9: This embodiment is a further optimization based on embodiment 6, 7, or 8. The parts identical to the aforementioned technical solutions will not be repeated here. Furthermore, to better realize the remote tower aviation visibility automatic identification system based on fused camera optical parameters described in this invention, the following configuration is specifically adopted: The temporal fusion unit includes a Kalman filter, whose state vector includes the RVR value and RVR rate of change. A prediction-update loop is used to smooth the estimation results of consecutive frames. This unit also includes an anomaly detection mechanism. When the RVR relative error output by the Kalman filter exceeds a preset threshold (preferably 15%) and is present for at least 3 consecutive frames, the output of the remote tower aviation visibility automatic identification system based on fused camera optical parameters is automatically reverted to the measurement value using the traditional forward scattering instrument. That is, the anomaly detection mechanism is: when… And continue T An anomaly is triggered when there are ≥3 frames; the system enters fallback mode and uses the forward scatterometer reading RVR. FS As output, until continuous T rec ≥5 frames recovered to within the threshold.

[0054] Example 10: This embodiment is a further optimization based on any one of embodiments 6 to 9. The parts that are the same as those in the foregoing technical solutions will not be repeated here. In order to better realize the remote tower aviation visibility automatic recognition system that integrates camera optical parameters as described in this invention, the following setting method is adopted: The DPIN-RVR network structure in the algorithm inference unit is obtained by a training method including the following steps: data augmentation is performed on the standardized image data used for training. The data augmentation includes at least one of physical fogging, exposure perturbation, geometric transformation and noise injection based on the Koschmieder model, which is used to improve the generalization ability.

[0055] Example 11: An automatic visibility recognition system for remote control towers that integrates camera optical parameters is used to implement an automatic visibility recognition method for remote control towers that integrates camera optical parameters. It adopts a multi-layer architecture design, including a hardware acquisition layer, a data processing layer, an algorithm inference layer, and an application interface layer.

[0056] The hardware acquisition layer includes the following units: Main camera unit: Sony IMX485 CMOS sensor, 4K (3840×2160), 2.9µm pixel size; C-Mount programmable auto aperture lens, f / 1.4–f / 16, switchable IR-cut filter; Exposure control: electronic shutter 1 / 1,000,000s–30s, ISO 100–12,800; Ambient light sensor Si1133 (0.1–100 klx); Sync interface: GigEVision supports PTPv2 / GPS PPS hardware synchronization; Protection rating: IP66, -40℃~+60℃.

[0057] Auxiliary imaging unit: Infrared binoculars: 850 nm narrow band filter, 2 MP; Thermal imaging: 8–14 µm, 320×256.

[0058] Time and Control: FPGA: Xilinx Zynq-7020 receives GPS PPS and timestamps each frame at the 10ns level; Edge Computing Power: NVIDIA Jetson AGX Xavier (512 CUDA cores, 32 GB LPDDR4).

[0059] Redundant sensors: Campbell Scientific CS135 Ceilometer (for cloud base / vertical visibility reference) + Vaisala PWD22 forward scattering instrument; the equipment is synchronized via IEEE 1588.

[0060] Data processing layer: Raw stream and metadata encapsulation: Frame rate / resolution: 25 fps @4K (drops to 10 fps at night); Format: JPEG-XL image and JSON sideload file with fields including timestamp, camera ID, ISO, exposure time, f-number (aperture value), ambient light intensity (lux), camera temperature (temp_C), gain, and white balance.

[0061] Optical and geometric calibration: Camera response function (CRF) calibration: using the Debevec & Malik multi-exposure method; Geometric distortion calibration: using the OpenCV Zhang calibration method; Dark current and flat field correction: achieved through dark frame acquisition and uniform white board shooting.

[0062] Data preprocessing workflow: 1. De-mosaic: AHD (Adaptive Homogeneity-Directed) interpolation; 2. White balance (and color) correction: 3×3 matrix mapping; 3. Distortion correction: OpenCV undistort(); 4. ROI extraction: Based on pre-calibrated pitch / azimuth angles, crop the runway centerline ±5° to 1024×512 pixels; 5. Photometric normalization: ;in, I raw These are the original pixel values. E dark Here, denoted as dark current, and ISO is the sensitivity calibration value. t s Where N is the exposure time (in seconds), and F is the f-number. The normalized image is denoted as I norm Input size is R C×H×W ,in C∈ {1,3}, preferred C =3.

[0063] Algorithm inference layer: Data augmentation strategies: To improve the model's generalization ability and robustness, the following data augmentation methods are adopted: Physical atomization: based on the Koschmieder model, ; Indicates the atmospheric extinction coefficient; Exposure perturbation: ±0.5 EV random variation; False shadows: random position, Gaussian blur; Noise injection: Gaussian noise, Poisson noise; Geometric transformations: rotation ±3°, translation ±5%, horizontal flip; Normalization: Image channels μ=0, σ=1, metadata standardization.

[0064] DPIN-RVR network structure: Input Design: Image: ,in C∈ {1,3}, preferred C =3; Metadata: ; in, m raw The original metadata vector is obtained from camera parameters and ambient light information through logarithmic mapping and other processing. d meta The dimension of the metadata vector; norm lux The normalized ambient light intensity is used to reduce the impact of different lighting conditions; one-hotCamID is the unique hot-coded representation of the camera ID, used to distinguish different camera positions.

[0065] Patch Embedding: Blocks: 16×16 → Total N =( H / 16)×( W / 16) blocks; linearly mapped to dimension D = 768, with learnable positional encoding; ViT Backbone (12 layers of Vision Transformer): Each layer contains: 1. LayerNorm; 2. Multi-Head Self-Attention (12 heads, d k =64); 3. DropPath (linear transition from 0.1 to 0.2) + residual join; 4. LayerNorm; 5. Feed-Forward Network (768 → 3072 → 768); 6. FiLM Fusion: In the first... Layer, for token features Apply FiLM: .

[0066] in c l , β lThese are scaling and bias parameters, used only for feature modulation and are not equivalent to the atmospheric extinction coefficient. .

[0067] Parameter branch: Transfer metadata vector m raw After two fully connected (FC) layers and layer normalization (LayerNorm), a set of modulation parameter sequences is generated. Where L = 12, each ; c , β These are the scaling and offset parameters used in FiLM fusion to modulate image features. The total output dimension is 2× D × L =18432. C th This is the contrast threshold, used to define the critical contrast level for visibility.

[0068] In the Layer, for token features Apply FiLM modulation: ; where ⊙ represents element-wise multiplication of the last dimension; c (l) and β (l) It will be broadcast to all tokens along dimension D (i.e., for each token vector) have ).

[0069] Output header: RVR Return: ; Uncertainty: . This represents the uncertainty of the model prediction, i.e., the variance of the estimation results.

[0070] Multiple loss functions: 1. Huber Loss (δ=1): ;in, L Huber This is the Huber loss function, used to balance the squared loss and absolute loss during training; y This represents the actual runway visual range (RVR) value. The RVR value obtained from model estimation; 2. Uncertainty regularization: ; used to constrain the estimation of variance. Wherein, , .

[0071] 3. Total Losses: ; Optimization strategy: Optimizer: AdamW(lr=3×10) -5 weight_decay=1×10 -4 Learning rate scheduling: Warm-up 5% step followed by cosine annealing to 3×10; -6 Batch size: 32, number of training rounds: 60; gradient clipping: global norm ≤ 1.0.

[0072] Application Interface Layer: Kalman filters perform timing fusion: State and observation model: ; X t Let be the state vector of the Kalman filter, which contains the current RVR value and its rate of change.

[0073] Prediction and update steps: predict: ; renew: , , .

[0074] Anomaly detection and rollback mechanism: When the following abnormal conditions are detected, the system will automatically revert to the readings of a traditional instrument: →Revert to the instrument reading (i.e., automatically revert to the measurement value of a traditional forward scattering instrument); RVR FS The RVR value measured by a conventional forward scattering meter is used for verification. The denominator uses max(RVR) FS 1m) is for use in areas with extremely low visibility (RVR). FS When the denominator approaches 0, it avoids computational instability caused by an excessively small denominator, thus enhancing the robustness of the algorithm.

[0075] The following containerized deployments can be achieved at the application interface layer: Docker image: Ubuntu 20.04 + CUDA 11.4 + Python 3.8 + PyTorch 1.12; Kubernetes edge cluster deployment with automatic scaling of GPU Pods; Monitoring system: Prometheus + Alertmanager + Grafana.

[0076] Example 12: An automatic aerial visibility identification method for remote control towers, which integrates camera optical parameters, is used for RVR measurement under standard configuration: The remote tower aviation visibility automatic recognition system based on the fusion of camera optical parameters, as described in this invention, was deployed on runway 27L of an airport. The hardware configuration is as follows: main camera: equipped with Sony IMX485, installed at a height of 15 meters and 500 meters from the runway threshold; field of view: 60° horizontally and 40° vertically; reference instrument: forward scattering instrument or transmissometer, used to provide contrast RVR (conventional reference reading).

[0077] Test conditions: Weather: light fog, visibility range 200-800 meters; Time: continuous 24-hour test; Sampling frequency: RVR output once every 5 seconds.

[0078] Test results: RMSE: ±28 meters (meets operational accuracy requirements); MAE: 21 meters; 95% confidence interval coverage: 94.2%; System availability: 99.7%.

[0079] Example 13: An automatic visibility recognition method for remote control towers, which integrates camera optical parameters, was used to conduct tests under extreme weather conditions. Test conditions: Weather: Dense fog, visibility <150 meters; Temperature: -15℃ to 35℃; Humidity: 80%-95%; Algorithm adaptive adjustment: Automatically adjust exposure parameters: ISO automatically adjusted to 6400; frame rate reduced to 10 fps to improve signal-to-noise ratio; enable infrared-assisted imaging; Test results: The accuracy rate reached 89% within a visibility range of 125-150 meters; when the visibility was below 125 meters, the system automatically activated the backup mode.

[0080] Example 14: A remote control tower aviation visibility automatic recognition method that integrates camera optical parameters was used to conduct multi-camera collaborative testing. A remote control tower system with three aircraft stands is deployed at an airport: two stands for runway 05L / 23R and one stand for runway 05R / 23L. Collaborative Algorithms: Multi-camera data fusion weighting algorithm; distance-based confidence weighting; automatic removal mechanism for abnormal cameras.

[0081] Performance improvements: RMSE is reduced by 15% compared to single-machine setups; system redundancy is increased to 99.9%.

[0082] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any simple modifications or equivalent changes made to the above embodiments based on the technical essence of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for automatic identification of aviation visibility at remote control towers by fusing camera optical parameters, characterized in that: Includes the following steps: 1) High-resolution images are acquired through the main camera unit, and optical metadata is recorded simultaneously; 2) Perform depigmentation, white balance correction, distortion correction, and photometric normalization on the acquired images to generate standardized image data; 3) Using a DPIN-RVR network structure, the standardized image data is processed through the Vision Transformer backbone, the optical metadata is processed through the parameter branch, and the feature information of the standardized image data and the feature information of the optical metadata are fused through the FiLM fusion mechanism in multiple layers of the Vision Transformer backbone; 4) RVR estimation steps: After step 3), the RVR value and the corresponding uncertainty estimate are output through the regression head; 5) Temporal fusion and anomaly detection steps: The RVR estimation results of consecutive frames in the time series are fused using a Kalman filter and compared with traditional forward scattering instrument data to perform anomaly detection; 6) Output steps: After step 5), the final RVR result is output in METAR / SPECI format, and IWXXM format is also provided.

2. The method for automatic identification of remote tower aviation visibility based on the fusion of camera optical parameters as described in claim 1, characterized in that: The main camera unit is composed of a CMOS image sensor, a programmable automatic aperture lens, and an ambient light sensor; the optical metadata includes one or more of the following: ISO value, exposure time, F-Number, ambient light intensity, camera temperature, exposure gain, and white balance parameters.

3. The method for automatic identification of remote tower aviation visibility based on the fusion of camera optical parameters according to claim 1, characterized in that: The DPIN-RVR network structure includes: a Patch Embedding layer that segments the input image into 16×16 pixel blocks and maps them to a 768-dimensional feature space; a VisionTransformer backbone consisting of 12 Vision Transformer layers; a parameter branch that processes optical metadata into modulation parameters through a fully connected network; a regression head for outputting runway visibility values ​​and their uncertainty estimates; and a FiLM fusion mechanism applied to the image features in each of the 12 Vision Transformer layers.

4. The method for automatic identification of aviation visibility in remote control towers by fusing camera optical parameters according to claim 1, characterized in that: In the time-series fusion and anomaly detection steps, the state vector of the Kalman filter includes the RVR value and the RVR rate of change; the anomaly detection mechanism is as follows: when the RVR relative error output by the Kalman filter exceeds a preset threshold and is continuous for no less than 3 frames, the output of the remote tower aviation visibility automatic identification method that fuses camera optical parameters is automatically reverted to the measurement value measured by the traditional forward scattering instrument.

5. The method for automatic identification of remote tower aviation visibility based on the fusion of camera optical parameters according to claim 1, characterized in that: It also includes a step of data augmentation on the standardized image data during the training phase of the DPIN-RVR network structure, wherein the data augmentation includes at least one of physical fogging, exposure perturbation, geometric transformation and noise injection based on the Koschmieder model.

6. A remote control tower aviation visibility automatic recognition system that integrates camera optical parameters, characterized in that: A method for automatically identifying aerial visibility in remote control towers by fusing camera optical parameters as described in any one of claims 1 to 5 includes: The main camera unit is used to acquire high-resolution images and simultaneously record optical metadata; The data processing unit is used to perform de-mosaicing, white balance correction, distortion correction, and photometric normalization on the acquired images to generate standardized image data. The algorithm inference unit is used to process the standardized image data through the VisionTransformer backbone and the optical metadata through the parameter branch using the DPIN-RVR network structure. The two types of information are fused through the FiLM fusion mechanism in multiple layers of the VisionTransformer backbone and the runway visual range value and its uncertainty estimate are output through the regression head. The temporal fusion unit is used to perform temporal fusion of the RVR estimation results of consecutive frames in the time series using a Kalman filter, and compare them with traditional forward scattering instrument data to perform anomaly detection; when the relative error of at least 3 consecutive frames exceeds a preset threshold, it automatically backs down to use forward scattering instrument readings; The output unit is used to output the final RVR result in METAR / SPECI format, and also provides IWXXM format.

7. The remote control tower aviation visibility automatic recognition system based on camera optical parameters according to claim 6, characterized in that: The main camera unit is composed of a CMOS image sensor, a programmable automatic aperture lens, and an ambient light sensor; the optical metadata includes one or more of the following: ISO value, exposure time, F-Number, ambient light intensity, camera temperature, exposure gain, and white balance parameters.

8. A remote control tower aviation visibility automatic recognition system based on camera optical parameters according to claim 6, characterized in that: The algorithm inference unit adopts a DPIN-RVR network structure, including: a Patch Embedding layer that segments the input image into 16×16 pixel blocks and maps them to a 768-dimensional feature space; a Vision Transformer backbone consisting of 12 Vision Transformer layers; a parameter branch that processes optical metadata into modulation parameters through a fully connected network; a regression head for outputting runway visibility values ​​and their uncertainty estimates; and a FiLM fusion mechanism used in each of the 12 Vision Transformer layers to apply the modulation parameters to the image features.

9. A remote control tower aviation visibility automatic recognition system based on camera optical parameters according to claim 6, characterized in that: The temporal fusion unit includes a Kalman filter, whose state vector includes the RVR value and the RVR rate of change; the unit also includes an anomaly detection mechanism, which automatically reverts the output of the remote tower aviation visibility automatic recognition system that fuses camera optical parameters to the measurement value measured by the conventional forward scattering instrument when the RVR relative error output by the Kalman filter exceeds a preset threshold and is continuous for no less than 3 frames.

10. A remote control tower aviation visibility automatic recognition system based on camera optical parameters according to claim 6, characterized in that: The DPIN-RVR network structure in the algorithm inference unit is obtained through a training method comprising the following steps: data augmentation of the standardized image data used for training, the data augmentation including at least one of physical fogging, exposure perturbation, geometric transformation and noise injection based on the Koschmieder model.