A multi-element-based sea visibility observation data quality optimization method

Abstract

The application provides a kind of based on multi-element offshore visibility observation data quality optimization method, belong to offshore visibility observation technical field, the application is marked by establishing dynamic threshold judgment mechanism to abnormal value screening, using second Lora fine tuning model carries out two layer optimization processing, through the feature selection enhancement mechanism based on sparse regularization and meteorological physical constraint condition calculation correction visibility deviation parameter and confidence parameter, using third Lora fine tuning model carries out three layer optimization processing, through the gradient acceleration algorithm based on momentum optimization to residual error is fine correction, establishes multi-element time sequence fusion verification mechanism, calculates time sequence consistency index and space continuity index and carries out quality identification grading mark, finally output optimized offshore visibility observation data and quality control report, solve the technical problem that offshore visibility observation data is insufficient in complex marine environment and lacks effective multi-element collaborative correction mechanism.

Description

Technical Field

[0001] This invention belongs to the field of marine visibility observation technology, and specifically relates to a method for optimizing the quality of marine visibility observation data based on multiple factors. Background Technology

[0002] Marine visibility observation is a crucial component of marine meteorological monitoring. Traditional marine visibility observation primarily relies on single optical sensors for scattering measurements, determining atmospheric transparency by analyzing changes in the intensity of emitted and received signals from laser or infrared light sources. In actual marine environments, multiple complex factors interact, affecting the accuracy of visibility measurements, including sea surface wind speed, relative humidity, temperature variations, sea fog formation, and wave disturbances. Existing observation systems typically employ simple linear correction algorithms to compensate for individual environmental parameters, lacking the ability to deeply model the nonlinear coupling relationships between multiple factors. Traditional methods are prone to optical scattering interference in high-humidity sea fog environments, leading to systematic biases in the observation data. Particularly under extreme conditions such as relative humidity exceeding 90% or wind speeds greater than 10 m / s, the reliability of visibility data significantly decreases, failing to meet the precision requirements for maritime traffic safety and marine operations. Furthermore, the lack of multi-factor collaborative analysis and hierarchical optimization mechanisms in current technologies makes it difficult for marine visibility observation data to maintain stable measurement accuracy in complex marine environments. In other words, existing technologies suffer from insufficient accuracy in marine visibility observation data under complex marine conditions. Summary of the Invention

[0003] In view of this, the present invention provides a method for optimizing the quality of marine visibility observation data based on multiple factors, which can solve the technical problem of insufficient accuracy of marine visibility observation data in complex marine environments in the prior art.

[0004] This invention is implemented as follows: It provides a method for optimizing the quality of multi-element marine visibility observation data. This includes establishing a preprocessing system for marine visibility observation data; collecting raw observation data of multiple elements such as sea surface wind speed, relative humidity, air temperature, air pressure, precipitation, scattered light intensity, wave height, and sea surface temperature; performing format, date, location, landing, and equality checks on the raw observation data to form basic information verification data; and performing range and continuity checks on the basic information verification data to form a standardized multi-element dataset. A multi-element optical scattering correction model is constructed for preliminary optimization. The standardized multi-element dataset is input into the multi-element optical scattering correction model. Nonlinear correlation features between multiple elements are extracted through a dense connection feature reuse enhancement mechanism. A multi-wavelength laser scattering compensation algorithm is used to correct optical scattering interference in sea fog environments. Humidity correction coefficients and wind speed influence coefficients are calculated based on a dual-optimization constraint-satisfaction solution framework, and preliminary corrected visibility data is output. A dynamic threshold determination mechanism is established based on the humidity correction coefficients and wind speed influence coefficients in the preliminary corrected visibility data. The process involves several steps: First, peak and extreme value range tests are performed on visibility anomalies to create screened and labeled data. A second LoRa fine-tuning model is used to perform two-layer optimization on this screened and labeled data. Key features are extracted using a sparse regularization-based feature selection enhancement mechanism. Nonlinear regression correction is applied to the visibility data under meteorological and physical constraints, and the corrected visibility deviation and confidence parameters are calculated. Based on these corrected parameters, the weight adjustment parameters for a third LoRa fine-tuning model are calculated using a weight adjustment function. This third LoRa fine-tuning model is then used for three-layer optimization. A momentum-based gradient acceleration algorithm is employed to refine the residual error. A multi-element time-series fusion verification mechanism is established. The correlation between the three-layer optimized visibility data and historical data from the same period is tested, and temporal consistency and spatial continuity indices are calculated. The verification results are then graded and labeled according to their quality. Finally, the optimized marine visibility observation data is output, and a quality control report is generated based on the quality graded and labeled data. This establishes a data reliability assessment system and an anomaly early warning mechanism.

[0005] The multi-element raw observation data includes sea surface wind speed, relative humidity, air temperature, air pressure, precipitation, scattered light intensity, wave height, and sea surface temperature. The basic information verification includes format verification, date verification, location verification, landing verification, and equality verification of the multi-element raw observation data. The standardization processing includes range verification and continuity verification of the basic information verification data.

[0006] Specifically, the implementation process of the dynamic threshold determination mechanism involves establishing a dynamic threshold determination mechanism based on the humidity correction coefficient and wind speed influence coefficient in the preliminary corrected visibility data, and performing peak detection and extreme value range detection on visibility anomalies to form screened and labeled data.

[0007] Specifically, the second-level optimization process of the second Lora fine-tuning model involves performing nonlinear regression correction on the visibility data in conjunction with meteorological and physical constraints, and calculating the corrected visibility deviation parameter and confidence parameter.

[0008] Specifically, the weight adjustment parameters of the third Lora fine-tuning model are calculated by using a weight adjustment function based on the corrected visibility deviation parameter and confidence parameter.

[0009] The verification process of the multi-element time-series fusion verification mechanism involves conducting a correlation test between the three-layer optimized visibility data and historical data from the same period, and then classifying and labeling the verification results according to the time-series consistency index and the spatial continuity index.

[0010] Among them, the multi-element optical scattering correction model is a multi-layer perceptron architecture combined with a convolutional neural network module. The input layer receives eight multi-element parameters, and features are extracted through four densely connected layers. The intermediate layer adopts batch normalization and residual connection mechanisms, and the output layer generates humidity correction coefficient and wind speed influence coefficient vectors.

[0011] Specifically, the fine-tuning process of the second LoRa fine-tuning model involves fine-tuning the third and fourth densely connected layers of the multi-element optical scattering correction model. By reducing the parameter update amplitude of the third and fourth densely connected layers, the model's sensitivity to light fog detection under sea fog conditions is improved. During the fine-tuning process, the parameters of the first two layers are fixed, and only the weight matrices of the last two layers are updated.

[0012] Specifically, the fine-tuning process of the third Lora fine-tuning model involves fine-tuning the output layer linear transformation matrix and bias vector of the multi-element optical scattering correction model. By finely adjusting the numerical range of the output layer linear transformation matrix and bias vector, the influence of residual error on the final result is reduced. During the fine-tuning process, the parameters of the densely connected layers remain unchanged, and only the output layer parameters are optimized.

[0013] Among them, the dynamic threshold determination mechanism determines the following conditions: when the relative humidity is ∈ (90%, 100%), the humidity correction coefficient is increased by 20%; when the wind speed is >10m / s, the wind speed compensation process is activated.

[0014] The steps for establishing the training dataset for the multi-factor optical scattering correction model include: collecting three consecutive years of marine observation data from the Yellow and Bohai Seas as the basic data source; stratifying the data according to visibility levels 0 to 9; constructing a balanced training sample set containing both normal and abnormal operating conditions; and expanding the training sample to 500,000 records using data augmentation techniques. The training steps for the multi-factor optical scattering correction model involve using the Adam optimizer with a learning rate of 0.001, a batch size of 128, and 500 training epochs. An early stopping mechanism is employed to prevent overfitting, and cross-validation is used to evaluate the model's generalization ability and select the optimal parameter combination.

[0015] Specifically, the output of the second Lora fine-tuning model is fused, and the output feature vectors of the fine-tuned third and fourth densely connected layers are fused with the output layer of the multi-factor optical scattering correction model to generate the corrected visibility deviation parameters and confidence parameters. The output of the third Lora fine-tuning model, the fine-tuned output layer linear transformation matrix and bias vector are directly output as the final optimized marine visibility observation data.

[0016] The implementation principle of the dense connection feature reuse enhancement mechanism is to realize feature reuse between forward layers and reduce parameter redundancy during network propagation. It maximizes the utilization of information by directly connecting the output of each layer to the input of all subsequent layers.

[0017] Among them, the implementation principle of the constraint satisfaction solution framework based on dual optimization is to find the optimal solution and ensure feasibility in the constraint optimization step by alternately optimizing the primal problem and the dual problem, and to transform the visibility correction problem into a constrained convex optimization problem for solution.

[0018] The implementation principle of the multi-wavelength laser scattering compensation algorithm is to reduce the scattering interference of tiny water droplets in sea fog on optical signals. It uses three laser sources with wavelengths of 532nm, 650nm and 780nm to simultaneously measure and use the differences in scattering characteristics of each wavelength to compensate for errors.

[0019] The implementation principle of the feature selection enhancement mechanism based on sparse regularization is as follows: the feature selection enhancement mechanism based on sparse regularization is used to promote the sparsity of weights and automatically identify important features during training by using L1 norm penalty, thereby improving the interpretability and generalization ability of the model.

[0020] Among them, the implementation principle of the momentum-optimized gradient acceleration algorithm is to accelerate the convergence speed and reduce training oscillations during the parameter update process by calculating the exponentially weighted moving average of historical gradients.

[0021] The mathematical expression for the weight adjustment function is as follows: The weight adjustment function is used to calculate the weight adjustment parameters of the third LoRa fine-tuning model based on the corrected visibility deviation parameter and the confidence parameter. The inputs include the corrected visibility deviation parameter, the confidence parameter, and the current model weight vector. The output is the adjusted weight coefficient matrix, expressed by the formula: ,in This is the adjusted weight coefficient matrix. Here, α is the basic weighting coefficient matrix, ΔV is the adjusted intensity coefficient, and ΔV is the corrected visibility deviation parameter. C represents the standard visibility deviation, and C is the confidence level parameter. The standard confidence level is used.

[0022] The quality identification grading standard is used to identify the quality level of the final optimized data, where 1 represents correct data, 2 represents possibly correct data, 3 represents possibly incorrect data, and 4 represents incorrect data.

[0023] This invention establishes a multi-element optical scattering correction model combined with a hierarchical LoRa fine-tuning optimization mechanism, constructing a collaborative correction system covering eight key elements: sea surface wind speed, relative humidity, air temperature, air pressure, precipitation, scattered light intensity, wave height, and sea surface temperature. This effectively overcomes the limitations of traditional single-parameter correction methods. The scheme employs a dense connection feature reuse enhancement mechanism to extract nonlinear correlation features between multiple elements, reduces optical scattering interference in sea fog environments through a multi-wavelength laser scattering compensation algorithm, and establishes a constraint-satisfying solution framework based on dual optimization to calculate accurate humidity correction coefficients and wind speed influence coefficients, fundamentally improving observation accuracy in complex marine environments. Through two- and three-layer progressive optimization processing, combined with a dynamic threshold determination mechanism and a sparse regularization-based feature selection enhancement mechanism, this invention achieves intelligent identification and refined correction of abnormal data, establishing a complete multi-element time-series fusion verification mechanism and quality control system to ensure the high reliability and stability of the final output data. In summary, this invention solves the technical problem mentioned in the background art of insufficient accuracy of marine visibility observation data in complex marine environments. Attached Figure Description

[0024] Figure 1 This is a flowchart of the method of the present invention.

[0025] Figure 2 This is a time-series variation trend diagram of multiple parameters for marine visibility observation in Example 2. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0027] like Figure 1 The diagram shown is a flowchart of a multi-factor-based method for optimizing the quality of marine visibility observation data provided by this invention. This method includes the following steps:

[0028] S01. Establish a marine visibility observation data preprocessing system, collect raw observation data of multiple elements such as sea surface wind speed, relative humidity, air temperature, air pressure, precipitation, scattered light intensity, wave height and sea surface temperature, and perform format verification, date verification, location verification, landing verification and integrity verification on the raw observation data of the multiple elements to form basic information verification data, and perform range verification and continuity verification on the basic information verification data to form a standardized multi-element dataset;

[0029] S02. Construct a multi-element optical scattering correction model for preliminary optimization. Input the standardized multi-element dataset into the multi-element optical scattering correction model. Extract the nonlinear correlation features between the multi-element features through the dense connection feature reuse enhancement mechanism. Use a multi-wavelength laser scattering compensation algorithm to correct the optical scattering interference in the sea fog environment. Calculate the humidity correction coefficient and wind speed influence coefficient according to the constraint satisfaction solution framework based on dual optimization. Output the preliminary corrected visibility data.

[0030] S03. Based on the humidity correction coefficient and wind speed influence coefficient in the preliminary corrected visibility data, a dynamic threshold judgment mechanism is established. When the relative humidity is ∈ (90%, 100%), the humidity correction coefficient is increased by 20%. When the wind speed is >10m / s, the wind speed compensation process is activated. Peak detection and extreme value range detection are performed on the visibility anomalies to form screened and marked data.

[0031] S04. The second Lora fine-tuning model is used to perform two-level optimization processing on the screened and labeled data. Key element features are extracted through the feature selection enhancement mechanism based on sparse regularization. The visibility data is corrected by nonlinear regression in combination with meteorological and physical constraints. The corrected visibility deviation parameter and confidence parameter are calculated.

[0032] S05. Calculate the weight adjustment parameters of the third Lora fine-tuning model using the weight adjustment function based on the corrected visibility deviation parameter and confidence parameter. When the corrected visibility deviation parameter ∈ [0.15, 0.50), start the third Lora fine-tuning model for three-layer optimization processing and use a gradient acceleration algorithm based on momentum optimization to refine the residual error.

[0033] S06. Establish a multi-element time-series fusion verification mechanism, perform correlation tests between the three-layer optimized visibility data and historical data of the same period, calculate the time-series consistency index and spatial continuity index, and mark the verification results with quality identification and grading according to the time-series consistency index ∈ [0.85, 1.0] and the spatial continuity index ∈ [0.80, 1.0].

[0034] S07. Output the final optimized marine visibility observation data, generate a quality control report based on the quality identification and grading labels, and establish a data reliability assessment system and an anomaly early warning mechanism.

[0035] The specific structure of the multi-factor optical scattering correction model is a multilayer perceptron architecture combined with a convolutional neural network module. The input layer receives eight multi-factor parameters, and features are extracted through four densely connected layers. The intermediate layers employ batch normalization and residual connection mechanisms, and the output layer generates humidity correction coefficient and wind speed influence coefficient vectors. The training dataset establishment steps for the multi-factor optical scattering correction model specifically include collecting three consecutive years of marine observation data from the Yellow and Bohai Seas as the basic data source, performing stratified sampling according to visibility levels 0 to 9, constructing a balanced training sample set containing normal and abnormal operating conditions, and expanding the training samples to 500,000 records using data augmentation techniques. The training steps for the multi-factor optical scattering correction model specifically include using the Adam optimizer with a learning rate of 0.001, a batch size of 128, and 500 training epochs, employing an early stopping mechanism to prevent overfitting, evaluating the model's generalization ability through cross-validation, and selecting the optimal parameter combination.

[0036] The second Lora fine-tuning model fine-tunes the third and fourth densely connected layers of the multi-element optical scattering correction model. By reducing the parameter update amplitude of the third and fourth densely connected layers, the model's sensitivity to light fog detection under sea fog conditions is improved. During the fine-tuning process, the parameters of the first two layers are fixed, and only the weight matrices of the last two layers are updated. The output feature vectors of the fine-tuned third and fourth densely connected layers are fused with the output layer of the multi-element optical scattering correction model to generate the corrected visibility deviation parameters and confidence parameters. The third Lora fine-tuning model fine-tunes the linear transformation matrix and bias vector of the output layer of the multi-element optical scattering correction model. By finely adjusting the numerical range of the linear transformation matrix and bias vector of the output layer, the influence of residual error on the final result is reduced. During the fine-tuning process, the parameters of the densely connected layers are kept unchanged, and only the output layer parameters are optimized. The fine-tuned linear transformation matrix and bias vector of the output layer directly output the final optimized marine visibility observation data.

[0037] The dense connection feature reuse enhancement mechanism is used to achieve feature reuse between forward layers and reduce parameter redundancy during network propagation. It maximizes information utilization by directly connecting the output of each layer to the input of all subsequent layers. The constraint satisfaction solution framework based on dual optimization is used to find the optimal solution and ensure feasibility in the constraint optimization step through alternating optimization of the primal and dual problems, transforming the visibility correction problem into a constrained convex optimization problem for solution. The multi-wavelength laser scattering compensation algorithm is used to reduce the scattering interference of tiny water droplets on light signals in sea fog. It uses three laser sources with wavelengths of 532nm, 650nm, and 780nm to simultaneously measure and utilize the differences in scattering characteristics of each wavelength for error compensation. The feature selection enhancement mechanism based on sparse regularization promotes weight sparsity and automatically identifies important features during training by using L1 norm penalty, thereby improving the interpretability and generalization ability of the model. The gradient acceleration algorithm based on momentum optimization accelerates convergence speed and reduces training oscillations during parameter updates by calculating the exponentially weighted moving average of historical gradients.

[0038] The weight adjustment function is used to calculate the weight adjustment parameters of the third LoRa fine-tuning model based on the corrected visibility deviation parameter and the confidence parameter. The inputs include the corrected visibility deviation parameter, the confidence parameter, and the current model weight vector; the output is the adjusted weight coefficient matrix. The formula is expressed as follows: ,in This is the adjusted weight coefficient matrix. Based on the basic weight coefficient matrix, To adjust the strength coefficient, To correct the visibility deviation parameter, For standard visibility deviation, For confidence level parameters, The standard confidence level is used.

[0039] The humidity correction coefficient is used to quantify the impact of relative humidity on the accuracy of visibility measurement, and the correction intensity is determined by statistically analyzing the visibility deviation patterns under different humidity conditions. The wind speed influence coefficient is used to describe the impact of wind speed changes on the distribution of atmospheric particulate matter and optical transmission characteristics, and to compensate for the scattering effects of dust and sea salt particles under strong wind conditions. The corrected visibility deviation parameter is used to quantify the degree of deviation between the visibility data after the second-layer optimization process and the actual observed values, serving as the basis for initiating the third-layer optimization process. The confidence parameter is used to evaluate the credibility of the second-layer optimization process results, reflecting the stability and accuracy of the corrected data. The temporal consistency index is used to evaluate the continuity and rationality of the corrected visibility data in the time dimension, and is quantitatively evaluated by calculating the rate of change and trend consistency of observations at adjacent times. The spatial continuity index is used to evaluate the spatial correlation and gradient rationality of visibility data at adjacent observation points, and consistency verification is performed through spatial interpolation and neighborhood analysis methods. The quality labeling grading is used to identify the quality level of the final optimized data, where 1 represents correct data, 2 represents possibly correct data, 3 represents possibly incorrect data, and 4 represents incorrect data.

[0040] The present invention is also implemented by a computer to form a multi-factor-based system for optimizing the quality of marine visibility observation data. The computer is equipped with a readable storage medium, which stores program instructions. When the program instructions are run in the computer, they can execute the above-mentioned method.

[0041] The specific implementation methods of the above steps are described in detail below.

[0042] The specific implementation of step S01 involves establishing a marine visibility observation data preprocessing system. First, raw observation data on multiple elements, including sea surface wind speed, relative humidity, air temperature, air pressure, precipitation, scattered light intensity, wave height, and sea surface temperature, are collected through marine meteorological observation stations and buoy systems. Data acquisition is performed at 1-minute intervals to ensure the data's temporal resolution meets real-time monitoring requirements. The raw observation data undergoes format verification. A regular expression matching algorithm is used to check if the data format conforms to standardization requirements, and abnormal data records with non-compliant formats are removed. For date verification, a timestamp validity verification algorithm is used to ensure the observation time is within a reasonable range, and data with incorrect date formats or exceeding the observation period are removed. Location verification uses a geographic coordinate system verification algorithm to ensure that the latitude and longitude coordinates of the observation point are within the effective observation area, with the longitude range set to 117° to 125° and the latitude range set to 35° to 41°. Landing verification uses an elevation data comparison algorithm to ensure that the marine observation platform has not experienced abnormal displacement or equipment malfunctions leading to land-based observation phenomena. The integrity check employs a data consistency verification algorithm to cross-validate the same physical quantities acquired by multiple sensors at the same time, ensuring the inherent logical consistency of the observation data and forming basic information verification data. When performing range checks on the basic information verification data, reasonable value ranges are set for each physical quantity; the sea surface wind speed range is 0 to 35. The relative humidity ranges from 20% to 100%, the temperature ranges from -10°C to 40°C, and the air pressure ranges from 980 to 1040. Rainfall ranges from 0 to 50 mm. The intensity of the scattered light ranges from 0 to 1000. Wave height ranges from 0 to 8 The sea surface temperature range is 0℃ to 30℃; data outside this range are marked as outliers. Continuity testing uses a time series continuity analysis algorithm to calculate the rate of change of observations between adjacent time points. Data exceeding a set threshold is marked as discontinuous. The wind speed change rate threshold is set at 5. The temperature change rate threshold was set to 2℃ / minute, and the humidity change rate threshold was set to 10% / minute, ultimately forming a standardized multi-factor dataset.

[0043] The specific implementation of step S02 involves constructing a multi-factor optical scattering correction model for preliminary optimization. A standardized multi-factor dataset is input into the model, which employs a hybrid architecture combining a multilayer perceptron architecture and a convolutional neural network module. The input layer receives eight multi-factor parameter vectors and extracts features through four densely connected layers. Each densely connected layer uses a dense connection feature reuse enhancement mechanism, directly connecting the output of each layer to the input of all subsequent layers to maximize information utilization and reduce parameter redundancy. The intermediate layers use batch normalization to standardize each batch of data, eliminating the impact of data distribution differences on model training. A residual connection mechanism is also introduced to alleviate the gradient vanishing problem in deep networks through skip connections. During model training, a multi-wavelength laser scattering compensation algorithm is used, employing wavelengths of 532 nm and 62 nm. 650 and 780 Three laser sources are used for simultaneous measurement. The differences in scattering characteristics at each wavelength are utilized to correct for optical scattering interference in sea fog environments. The interference of atmospheric particulate matter on visibility measurement is eliminated through differential calculation of wavelength-related scattering coefficients. A constraint-satisfied solution framework based on dual optimization is employed to calculate humidity correction coefficients and wind speed influence coefficients, transforming the visibility correction problem into a constrained convex optimization problem. The optimal solution is found through alternating optimization of the primal and dual problems, ensuring that the correction coefficients satisfy physical constraints. The output layer generates vectors of humidity correction coefficients and wind speed influence coefficients, ultimately outputting preliminary corrected visibility data.

[0044] The specific implementation of step S03 involves establishing a dynamic threshold determination mechanism based on the humidity correction coefficient and wind speed influence coefficient in the preliminary corrected visibility data. This mechanism employs an adaptive threshold adjustment algorithm, dynamically adjusting the determination criteria according to real-time meteorological conditions. When the relative humidity is between 90% and 100%, the water vapor content in the atmosphere increases sharply under high humidity conditions, significantly affecting optical transmission characteristics. In this case, the humidity correction coefficient is increased by 20% to compensate for the impact of high humidity on the accuracy of visibility measurement by enhancing the correction intensity. When the wind speed exceeds 10... When strong winds occur, a wind speed compensation mechanism is activated because sea salt particles and dust particles stirred up from the sea surface under strong wind conditions significantly affect atmospheric transparency, requiring specific compensation for the particulate scattering effect caused by wind speed. For peak detection of visibility anomalies, a statistical outlier detection algorithm is used to calculate the deviation of observed values ​​from the neighborhood mean; values ​​exceeding three times the standard deviation are marked as peak anomalies. For extreme value range testing, historical statistical data analysis is used to establish normal distribution intervals for visibility in each month; values ​​exceeding the 99% confidence interval are marked as extreme anomalies, ultimately forming a filtered and labeled dataset containing anomaly labeling information.

[0045] The specific implementation of step S04 involves using a second LoRa fine-tuning model to perform two-layer optimization on the screened and labeled data. This model is specifically designed for parameter fine-tuning of the third and fourth densely connected layers of the multi-element optical scattering correction model. The fine-tuning process employs a feature selection enhancement mechanism based on sparse regularization. L1 norm penalty terms are used to promote the sparsity of the weight matrix, automatically identifying the most important feature combinations for visibility correction and improving the model's interpretability and generalization ability. The fine-tuning strategy uses parameter freezing technology, fixing the weight parameters of the first two densely connected layers and updating only the weight matrices of the third and fourth layers. By reducing the parameter update amplitude of these two layers, the model's sensitivity to light fog detection under sea fog conditions is improved. Nonlinear regression correction is applied to the visibility data in conjunction with meteorological and physical constraints, introducing the extinction law from atmospheric radiative transfer theory as a constraint to ensure that the correction results conform to the physical mechanism. The feature vectors output from the fine-tuned densely connected layers 3 and 4 are fused with the output layer of the original model. A weighted fusion algorithm is used to generate the corrected visibility deviation parameter and confidence parameter. The deviation parameter reflects the accuracy of the correction effect, and the confidence parameter evaluates the credibility of the correction result.

[0046] The specific implementation of step S05 involves calculating the weight adjustment parameters of the third Lora fine-tuning model based on the corrected visibility deviation parameter and the confidence parameter using a weight adjustment function. This function employs an adaptive weight adjustment algorithm, and the inputs include the corrected visibility deviation parameter, the confidence parameter, and the current model weight vector. When the corrected visibility deviation parameter is in the range of 0.15 to 0.50, it indicates that there is still a certain degree of systematic deviation after the two-layer optimization. At this point, the third Lora fine-tuning model is initiated for three-layer optimization. The third Lora fine-tuning model specifically refines the linear transformation matrix and bias vector of the output layer of the multi-element optical scattering correction model, and uses a gradient acceleration algorithm based on momentum optimization to refine the residual error. The momentum optimization algorithm accelerates the convergence speed and reduces oscillations during training by calculating the exponentially weighted moving average of historical gradients, thereby improving optimization efficiency and stability. During fine-tuning, the parameters of the densely connected layers remain unchanged, and only the linear transformation matrix and bias vector of the output layer are optimized. By finely adjusting the numerical range of these parameters, the impact of residual errors on the final result is reduced, ensuring that the output visibility data has higher accuracy.

[0047] The specific implementation of step S06 involves establishing a multi-factor time-series fusion verification mechanism. This involves testing the correlation between the three-layer optimized visibility data and historical data from the same period, and using the Pearson correlation coefficient analysis algorithm to assess the linear correlation between the data. The time-series consistency index is quantitatively evaluated by calculating the rate of change and trend consistency of observations at adjacent times. The autocorrelation function and partial autocorrelation function calculation methods from time series analysis are used to assess the continuity and rationality of the corrected data in the time dimension. The spatial continuity index is verified for consistency through spatial interpolation and neighborhood analysis methods. The Kriging interpolation algorithm is used to perform spatial correlation analysis on the visibility data of adjacent observation points to calculate the rationality of spatial gradient changes. The verification standard is set as follows: the time-series consistency index ranges from 0.85 to 1.0, and the spatial continuity index ranges from 0.80 to 1.0. When both indicators simultaneously meet the standard requirements, the verification results are graded and labeled with a quality identifier, establishing a four-level quality identifier system. Here, 1 represents correct data, 2 represents potentially correct data, 3 represents potentially incorrect data, and 4 represents incorrect data, providing a basis for quality assessment in subsequent data applications.

[0048] The specific implementation of step S07 involves outputting the final optimized marine visibility observation data and generating a detailed quality control report based on quality identification and grading. This report includes the data processing workflow, optimization effect evaluation, anomaly statistics, and quality level distribution. A data reliability assessment system is established, employing a comprehensive scoring algorithm to assign corresponding reliability weights to data of different quality levels, providing data users with a quantitative reliability reference. Simultaneously, an anomaly early warning mechanism is established. The real-time monitoring system automatically issues warnings for consecutive occurrences of low-quality data. When the proportion of data at quality levels 3 and 4 exceeds 30% for three consecutive hours, an early warning signal is triggered, reminding maintenance personnel to promptly check observation equipment and environmental conditions to ensure the continuous stability of data quality.

[0049] Further explanation is needed regarding the multi-element optical scattering correction model, which employs a hybrid architecture combining multilayer perceptrons and convolutional neural networks. This model is designed to optimize the quality of visibility observation data under complex marine weather conditions. The overall model structure comprises four main parts: an input layer, a feature extraction layer, an intermediate processing layer, and an output layer. The input layer is designed as an 8-dimensional vector interface, corresponding to multiple parameters such as sea surface wind speed, relative humidity, air temperature, air pressure, precipitation, scattered light intensity, wave height, and sea surface temperature. The feature extraction layer consists of four densely connected layers: layers 1 and 2 each contain 512 neurons, and layers 3 and 4 each contain 256 neurons. Dense connections between layers enable feature reuse. The intermediate processing layer integrates a batch normalization module and a residual connection module. The batch normalization module uses a dynamic statistical parameter update strategy, while the residual connection module connects non-adjacent layers via skip connections. The output layer adopts a dual-branch structure design, generating humidity correction coefficient vector and wind speed influence coefficient vector respectively. Each branch contains 128 neuron nodes, and the activation function adopts the hyperbolic tangent function to ensure that the output value is within a reasonable range.

[0050] The second LoRa fine-tuning model specifically optimizes the parameters of the densely connected layers (layers 3 and 4) of the base model. This model employs low-rank matrix factorization (LGF) to decompose the original weight matrix into the product of two low-dimensional matrices, significantly reducing the amount of parameter updates during fine-tuning. The model structure comprises three core components: a parameter decomposition module, a feature selection module, and a fusion computation module. The parameter decomposition module decomposes the 256×256 weight matrix into two low-dimensional matrices of 256×32 and 32×256, reducing computational complexity while maintaining model expressiveness. The feature selection module integrates L1 regularization constraints, highlighting important feature dimensions through sparse weight distribution, thus improving the model's sensitivity to light fog phenomena under sea fog conditions. The fusion computation module uses an attention mechanism to weightedly fuse the original and fine-tuned features, generating corrected visibility deviation parameters and confidence parameters.

[0051] The third LoRa fine-tuning model is specifically designed for the output layer linear transformation matrix and bias vector of the base model. This model employs a refined parameter adjustment strategy to ultimately correct residual errors. The model structure comprises three key parts: an error analysis module, a gradient optimization module, and a parameter update module. The error analysis module analyzes the systematic bias patterns remaining after two-layer optimization using the residual network structure, identifying the main error sources and distribution characteristics. The gradient optimization module uses a momentum optimization algorithm, integrating historical gradient information to accelerate the convergence process, while introducing an adaptive learning rate adjustment mechanism to avoid oscillations during optimization. The parameter update module fine-tunes only the 128×64 linear transformation matrix and 64-dimensional bias vector of the output layer, ensuring the stability and accuracy of the final output results through precise control of the parameter update magnitude.

[0052] The training dataset was established by first collecting three consecutive years of marine observation data from the Yellow and Bohai Seas as the basic data source. Data sources included 10 fixed marine observation stations and 15 mobile buoy observation platforms, ensuring spatial representativeness and temporal continuity. Following the visibility class standards established by the World Meteorological Organization, the data was stratified into levels 0 to 9, with at least 5,000 valid observation records collected for each level to ensure class balance in the training samples. A balanced training sample set was constructed, including normal and abnormal operating conditions. Normal operating condition data accounted for 70% of the total sample, while abnormal operating condition data accounted for 30%. Abnormal operating conditions mainly included observation data under extreme weather conditions such as sea fog, dust storms, rainfall, and strong winds. Data augmentation techniques were used to expand the training sample size. Methods such as time series distortion, Gaussian noise addition, random sampling, and interpolation were employed to expand the original sample from 150,000 records to 500,000 records, improving the model's generalization ability and robustness. In the data preprocessing stage, all samples are standardized and normalized to eliminate dimensional differences between different physical quantities. At the same time, a sample labeling system is established to label each observation record with the corresponding visibility true value and quality level information.

[0053] The multi-element optical scattering correction model and its two LoRa models address the technical challenges of optical scattering interference and multi-element coupling effects in marine visibility observation. Compared to traditional single-sensor visibility measurement methods and simple linear correction algorithms, this invention offers significant technical advantages. Traditional methods typically use single-wavelength optical sensors for visibility measurement, which are easily affected by scattering from tiny water droplets in sea fog, leading to decreased measurement accuracy. This invention employs a multi-wavelength laser scattering compensation algorithm, utilizing the differences in scattering characteristics of different wavelengths of light to compensate for interference, thus solving the optical scattering interference problem from a physical mechanism perspective. Existing visibility data quality control methods mainly rely on statistical methods for outlier detection and simple interpolation correction, failing to effectively handle the nonlinear coupling relationships between multiple meteorological elements. This invention extracts complex correlation features between multiple elements through a dense connection feature reuse mechanism and establishes nonlinear mapping relationships using deep learning technology, significantly improving the correction accuracy under multi-element conditions. Traditional methods employ a uniform correction strategy when dealing with data anomalies of varying degrees, lacking specificity and adaptability. In contrast, the multi-level LoRa fine-tuning strategy designed in this invention can automatically select the appropriate optimization depth based on the degree of data deviation, achieving refined and personalized data quality correction and improving the specificity and effectiveness of the correction effect.

[0054] It should be noted that the first key technical idea of ​​this invention is the combined application of a multi-wavelength laser scattering compensation algorithm and a densely connected feature reuse mechanism. Traditional visibility observation methods typically use single-wavelength optical sensors, which are easily affected by severe interference from water droplet scattering in sea fog environments, leading to significant deviations in measurement results. This invention uses a 532... 650 and 780 Simultaneous measurement using three laser light sources of different wavelengths effectively eliminates scattering interference by utilizing the differences in the scattering characteristics of atmospheric particles at different wavelengths and calculating the wavelength-related scattering coefficients differentially. Furthermore, by combining a densely connected feature multiplexing mechanism, the scattering information of each wavelength is deeply integrated with other meteorological elements to establish a nonlinear mapping relationship in a multi-dimensional feature space, fundamentally improving the accuracy and stability of visibility measurements in complex marine environments.

[0055] The second key technical approach is the design and implementation of a multi-level adaptive fine-tuning strategy. Existing data quality optimization methods typically employ a uniform correction algorithm, failing to provide personalized solutions for different degrees of data deviation. This invention designs a multi-level LoRa fine-tuning mechanism driven by deviation parameters. The second and third LoRa models correspond to different optimization depths, and the appropriate fine-tuning strategy is automatically selected based on the numerical range of the corrected visibility deviation parameter. This adaptive strategy can finely adjust minor deviations and deeply correct severe deviations, avoiding over-correction and under-correction, significantly improving the accuracy and adaptability of data quality optimization.

[0056] The third key technical approach is the construction of an optimization solution framework based on physical constraints. Traditional data correction methods mainly rely on statistical principles, lacking in-depth consideration of meteorological physical mechanisms, and are prone to producing correction results that do not conform to physical laws. This invention introduces the extinction law from atmospheric radiative transfer theory as a constraint, transforming the visibility correction problem into a constrained convex optimization problem. A dual optimization algorithm is then used to find the optimal correction parameters while satisfying the physical constraints. This constraint-based solution method based on physical mechanisms ensures the physical rationality of the correction results, avoids the irrational phenomena that may arise from purely mathematical correction, and improves the scientific rigor and credibility of data quality optimization.

[0057] The synergistic effect of the three key technical approaches described above forms a complete technical system for optimizing the quality of marine visibility observation data. The multi-wavelength scattering compensation algorithm eliminates optical interference during data acquisition, providing a high-quality initial data foundation for subsequent optimization processing. The multi-level adaptive fine-tuning strategy dynamically adjusts the optimization intensity based on data characteristics, achieving personalized quality improvement effects. The optimization framework based on physical constraints ensures the scientific rigor and rationality of the correction process. The collaborative work of these three technologies forms a complete closed loop from source control to process optimization and result verification. Compared to existing single-stage improvement schemes, the systematic technical solution of this invention can comprehensively solve multiple technical challenges in marine visibility observation, achieving an overall improvement in data quality and ensuring reliability.

[0058] It should be noted that existing marine visibility observation systems typically employ manual quality control methods, relying on the experience and judgment of observers for data screening and anomaly identification. This lack of unified quality assessment standards and automated verification processes leads to inconsistent data quality and low processing efficiency. This invention establishes a multi-element time-series fusion verification mechanism, designs a dual verification system of time-series consistency and spatial continuity indicators, and formulates clear quality identification and grading standards. This enables automated assessment and tiered management of visibility data quality, establishes a complete data reliability assessment system and anomaly early warning mechanism, and ensures standardized and intelligent data quality control. Existing optical scattering measurement technologies mainly use single-wavelength lasers or infrared light sources, which are easily affected by scattering interference under complex meteorological conditions such as sea fog, dust storms, and precipitation, leading to significant deviations in measurement results. In particular, the scattering effect of tiny water droplets on light signals in high-humidity marine environments severely affects the accuracy of visibility measurements. This invention employs a multi-wavelength laser scattering compensation algorithm, simultaneously using three different laser light sources at wavelengths of 532nm, 650nm, and 780nm for measurement. By utilizing the differences in scattering characteristics of each wavelength in different particle environments for error compensation, it effectively reduces the impact of optical scattering interference in sea fog environments and improves the measurement anti-interference capability and data stability under complex meteorological conditions.

[0059] Specifically, the principle of this invention is as follows: The core principle of this invention in solving the technical problem lies in constructing a multi-level collaborative optimization data correction architecture. A multi-element optical scattering correction model is used to achieve comprehensive modeling and nonlinear relationship mining of complex marine environmental elements. This model employs a multi-layer perceptron architecture combined with a convolutional neural network module. Through densely connected layers, it achieves feature reuse between forward layers, maximizing the utilization of the correlation information between the eight input elements and avoiding the shortcomings of traditional methods that ignore the coupling relationships between elements. The multi-wavelength laser scattering compensation algorithm uses simultaneous measurements from three different wavelength laser sources (532nm, 650nm, and 780nm) and utilizes the differences in scattering characteristics of each wavelength in sea fog for error compensation, effectively reducing the scattering interference of tiny water droplets on the light signal under high humidity conditions. A constraint-satisfying solution framework based on dual optimization transforms the visibility correction problem into a constrained convex optimization problem. Alternating optimization of the primal and dual problems ensures the optimality and feasibility of the humidity correction coefficient and the wind speed influence coefficient. The hierarchical LoRa fine-tuning mechanism achieves progressive optimization from coarse to fine through refined adjustments to different levels of the model. The second LoRa fine-tuning model specifically addresses improved sensitivity in light fog detection under sea fog conditions, while the third LoRa fine-tuning model focuses on fine-tuning residual errors. The dynamic threshold determination mechanism adaptively adjusts correction parameters based on actual environmental conditions. It increases the humidity correction coefficient when relative humidity exceeds 90% and activates wind speed compensation when wind speed exceeds 10 m / s, ensuring effective correction under various extreme conditions. The multi-factor temporal series fusion verification mechanism, through dual constraints of temporal consistency and spatial continuity indicators, guarantees the rationality and continuity of the corrected data in both time and space dimensions, establishing a complete quality control and reliability assessment system.

[0060] The following provides a specific embodiment 1 of the present invention, and the specific implementation of each step in this embodiment 1 is described in detail below.

[0061] In this embodiment, the specific implementation of step S01 is the same as described above, and will not be repeated in detail here.

[0062] The specific implementation of step S02 involves constructing a multi-element optical scattering correction model for preliminary optimization. The formula for calculating the scattering compensation coefficient of the multi-wavelength laser scattering compensation algorithm is expressed as follows:

[0063] ;

[0064] In the formula, The scattering compensation coefficient is dimensionless. 532 Wavelength scattering intensity, in units of ; 650 Wavelength scattering intensity, in units of ; 780 Wavelength scattering intensity, in units of ; 532 Standard scattering intensity at wavelength, value is 100 ; 650 Standard scattering intensity at wavelength, value is 120. ; 780 Standard scattering intensity at wavelength, value is 150. ; 650 Wavelength weighting coefficient, with a value of 0.8, is dimensionless; 780 Wavelength weighting coefficient, with a value of 0.6, dimensionless; This is a stability adjustment parameter with a value of 0.01, dimensionless. The formula for calculating the humidity correction factor is as follows:

[0065] ;

[0066] In the formula, This is a humidity correction factor, dimensionless. Relative humidity, in %; The standard relative humidity is 60%. This is the humidity nonlinear adjustment coefficient, with a value of 0.3, and is dimensionless. This is a periodic adjustment coefficient with a value of 0.15, and it is dimensionless. The baseline offset coefficient is 0.05, dimensionless. The formula for calculating the wind speed influence coefficient is as follows:

[0067] ;

[0068] In the formula, The wind speed influence coefficient is dimensionless. Wind speed over sea surface, unit: ; Standard wind speed, value is 8 ; The wind speed damping coefficient has a value of 0.4 and is dimensionless. This is the exponential adjustment coefficient, with a value of 0.2, and is dimensionless. The attenuation coefficient is 0.5 and is dimensionless.

[0069] The specific implementation of step S03 is to establish a dynamic threshold determination mechanism based on the preliminary corrected visibility data. The dynamic threshold calculation formula is expressed as follows:

[0070] ;

[0071] In the formula, The threshold value is dynamic and dimensionless. The base threshold is 0.25, which is dimensionless. This is the standard humidity correction factor, with a value of 1.0, and is dimensionless. The standard wind speed influence coefficient has a value of 1.0 and is dimensionless. This is atmospheric pressure, in units of... ; Standard atmospheric pressure, value is 1013. ; Temperature, in °C; The standard temperature is 15°C. This is the weighting coefficient for meteorological factors, with a value of 0.3, and is dimensionless. This is the threshold offset parameter, with a value of 0.1 and is dimensionless.

[0072] The specific implementation of step S04 is to use the second Lora fine-tuning model for two-layer optimization. The formula for calculating the visibility deviation parameter after correction is as follows:

[0073] ;

[0074] In the formula, The corrected visibility deviation parameter is dimensionless. For measuring visibility, the unit is . ; For true visibility, the unit is 1. ; For standard visibility measurement, the value is 10. ; Standard true visibility, valued at 10. ; This is the time gradient weight coefficient, with a value of 0.1 and a unit of... ; This is the residual weighting coefficient, with a value of 0.05, and is dimensionless. This is the residual error term, ranging from -0.2 to 0.2, and is dimensionless. The confidence parameter is calculated using the following formula:

[0075] ;

[0076] In the formula, The confidence level parameter is dimensionless. is the kurtosis parameter of the sigmoid function, with a value of 5 and is dimensionless; The local standard deviation is obtained by calculating the standard deviation of visibility data from the five neighboring observation points around the current observation point, and the unit is 1000 Å. ; This represents the standard deviation, with a value of 0.5. ; The confidence level baseline is 0.1, which is dimensionless.

[0077] The specific implementation of step S05 involves calculating the weight adjustment parameters of the third Lora fine-tuning model based on the corrected visibility deviation parameters and confidence parameters using a weight adjustment function. The formula for the weight adjustment function is as follows:

[0078] ;

[0079] In the formula, This is the adjusted weight coefficient matrix, which is dimensionless. It is the basic weight coefficient matrix, which is dimensionless. To adjust the strength coefficient, the value is 0.5, which is dimensionless; This represents the standard visibility deviation, with a value of 0.3. ; The standard confidence level is 0.8, and it is dimensionless. This is a periodic adjustment coefficient with a value of 0.2 and is dimensionless. The nonlinear adjustment coefficient is 0.1 and dimensionless. The momentum update formula in the momentum-optimized gradient acceleration algorithm is expressed as follows:

[0080] ;

[0081] ;

[0082] ;

[0083] In the formula, for First-order momentum estimation at time step and weights Same dimensions; for The second-order momentum estimate at time t is given by the square of the weight dimension. The first-order momentum decay rate has a value of 0.9 and is dimensionless. The second momentum decay rate is 0.999, which is dimensionless. The loss function represents the error between the model's predicted value and the actual value, and is dimensionless. for The weight parameters at time points represent the connection strength in the neural network and are dimensionless. This is the partial derivative of the loss function with respect to the weights, and it is dimensionless. The learning rate is 0.001 and is dimensionless. This is a numerically stable term with a value of ,and The units must be the same to ensure the stability of the denominator.

[0084] The specific implementation of step S06 is to establish a multi-factor time series fusion verification mechanism. The formula for calculating the time series consistency index is as follows:

[0085] ;

[0086] In the formula, It is a time-series consistency indicator, dimensionless; The number of time-series observation points, dimensionless; For the first Visibility observations at each moment, in units of ; For the first Visibility observations at each moment, in units of ; The time interval is expressed in units of 1 / 2. ; The standard time interval is 5. ; This is the rate of change decay coefficient, with a value of 2 and a unit of . The formula for calculating the spatial continuity index is as follows:

[0087] ;

[0088] In the formula, It is a spatial continuity index and is dimensionless. The number of spatial observation points, dimensionless; The distance between adjacent observation points, in units of ; The standard observation point spacing is 2. ; For the first Visibility at a spatial observation point, in units of ; For the first Visibility at a spatial observation point, in units of ; This is the spatial gradient weight coefficient, with a value of 0.3 and dimensionless. Here is the second spatial partial derivative of visibility, in units of . .

[0089] The specific implementation method of step S07 is the same as described above, and will not be repeated in detail here.

[0090] It needs further explanation that the scattering compensation coefficient formula of the multi-wavelength laser scattering compensation algorithm is based on the combined application of Rayleigh scattering theory and Mie scattering theory, taking into account the differences in scattering characteristics of different wavelengths of laser light in the ocean atmosphere. The numerator of the formula adopts a weighted linear combination form. The scattering intensities of the three wavelengths are summed according to their weights in relation to the overall scattering, and the denominator is expressed as a sum of squares. To enhance system stability and avoid division-by-zero errors, stability adjustment parameters are used. The introduction of this method prevents calculation instability caused by an excessively small denominator. This formula ensures dimensionless consistency on both sides of the equation through dimensionless processing; the left side represents the dimensionless scattering compensation coefficient, while the right side's numerator and denominator are combinations of dimensionless terms. Compared to traditional single-wavelength correction methods, this effectively eliminates the selective scattering interference of water droplets on different wavelengths of light in sea fog environments, significantly improving the accuracy and stability of visibility measurements in complex marine environments.

[0091] The humidity correction factor formula adopts a design approach that combines nonlinear response functions and periodic functions, with the main term in fractional form. Saturation characteristics of the effect of humidity on visibility, quadratic coefficient Controlling the nonlinearity of the humidity response, the sine function term The periodic fluctuation characteristics during the simulation of humidity changes, and the reference offset coefficient. It provides systematic error compensation. This formula makes the ratio of relative humidity to the standard value dimensionless, ensuring that the dimensions on both sides of the equation are unified as dimensionless correction coefficients. Compared with traditional linear correction methods, it can accurately describe the nonlinear characteristics of the sharp decline in visibility under high humidity conditions, especially the rapid change law when the relative humidity is above 90%, effectively improving the correction accuracy of visibility observation under sea fog conditions.

[0092] The formula for the wind speed influence coefficient combines the dual mechanisms of the power function decay model and the exponential decay model, with the main term adopting the power function form. Describes the influence of wind speed on the distribution of atmospheric particulate matter, with the exponential term. This paper simulates the physical process of rapid particulate matter dispersion under strong wind conditions. The power exponent of 1.5 reflects the nonlinear relationship between wind speed and particulate matter dispersion efficiency, and the squared term in the exponential decay term enhances the correction effect under strong wind conditions. Compared to traditional static correction methods, this formula can dynamically adapt to changes in atmospheric optical properties under different wind speeds, especially when wind speeds exceed 10... The strong wind disturbance effect significantly improved the reliability of visibility observation in dynamic marine environments.

[0093] The dynamic threshold calculation formula adopts an adaptive adjustment mechanism based on multi-factor weighted fusion, with a basic threshold. Provide the system's baseline judgment criteria, and the average values ​​of humidity and wind speed correction factors. This reflects the combined influence of major meteorological elements, specifically the difference between air pressure and temperature. The contribution of atmospheric stability variations to threshold adjustment was captured. Compared to traditional fixed threshold methods, this formula can dynamically adjust the outlier judgment criteria based on real-time meteorological conditions, avoiding misjudgment problems caused by fixed thresholds in complex marine environments, and improving the accuracy and adaptability of data quality control.

[0094] The revised visibility deviation parameter formula comprehensively considers the triple effects of instantaneous deviation, changing trend, and residual error, with the main term adopting the form of relative error. The degree of difference between the measured value and the true value is quantified by the time partial derivative term. Capture the dynamic characteristics of visibility changes, residual error term The compensation model cannot fully describe systematic biases. Compared with traditional static bias assessment methods, this formula can comprehensively reflect the multidimensional characteristics of data quality problems, providing an accurate basis for subsequent hierarchical optimization processing and effectively improving the comprehensiveness and accuracy of data quality assessment.

[0095] The confidence parameter formula uses the product of two sigmoid functions, the first sigmoid function... The second fractional function is used to assess data accuracy based on the deviation parameter. Based on local standard deviation, data stability is assessed. The product of these two factors ensures that only data that simultaneously meets the requirements of accuracy and stability can obtain a high confidence score. Compared to traditional single-indicator assessment methods, this formula comprehensively considers multiple dimensions of data quality, providing a more reliable and comprehensive assessment of data credibility, and offering a scientific basis for subsequent weight adjustments and optimization.

[0096] The weight adjustment function formula adopts a nonlinear adaptive adjustment strategy, with linear terms... Provides basic adjustment strength, cosine function term Introducing a periodic adjustment mechanism to avoid oscillations during the adjustment process, a cubic term Enhanced adjustment performance under large deviations. The momentum-optimized gradient acceleration algorithm optimizes parameters through a momentum update mechanism; the first-order momentum estimation formula is:

[0097] ;

[0098] The formula for estimating second-order momentum is:

[0099] ;

[0100] The weight update formula is:

[0101] ;

[0102] This algorithm calculates historical gradient information using an exponentially weighted moving average, accelerating the convergence process and reducing training oscillations. Compared to traditional fixed-weight methods, the weight adjustment function dynamically adjusts model parameters based on data bias and confidence levels, enabling personalized optimization and significantly improving the targeting and effectiveness of data quality correction.

[0103] The temporal consistency index formula employs an evaluation mechanism combining exponential decay weighting and rate of change constraints. The summation term is a weighted average of the consistency across all adjacent time points, and the square term in the denominator... Suppressing excessive rate of change, exponential term Additional penalties are applied to abnormal changes. This formula ensures uniformity of dimensions through dimensionless processing; both the rate of change over time and the change in visibility are expressed as ratios to standard values. Compared to traditional linear correlation analysis methods, this formula can better identify anomalous jumps and unreasonable changes in time series, ensuring the physical rationality and logical consistency of the corrected data over time.

[0104] The spatial continuity index formula comprehensively considers a triple mechanism of distance decay, gradient constraint, and second derivative control, with a Gaussian weighting function. Spatial correlation is weighted based on distance, with gradient constraint terms. To prevent excessive differences between adjacent observation points, the second-order partial derivative term... The formula controls the smoothness of spatial variations. It achieves dimensional uniformity through the standardization of distance and visibility, ensuring the accuracy of index calculations. Compared to traditional spatial interpolation methods, this formula can more accurately assess the spatial continuity characteristics of visibility data in marine environments, effectively identify spatial distribution anomalies and local singularities, and guarantee the spatial consistency and physical rationality of the corrected data.

[0105] To better understand and implement this invention, a specific application scenario is provided below as Example 2: A technical team, while performing a maritime visibility observation mission, faced complex environmental conditions including frequent sea fog, drastic humidity changes, and large wind speed fluctuations. Traditional single-element correction methods could not meet the high-precision monitoring requirements. According to international standards, sea surface visibility is divided into 10 levels from 0 to 9, where level 0 (visibility less than 0.05 km) is considered extremely poor, level 5 (visibility 1-5 km) is considered moderate, and level 9 (visibility greater than 50 km) is considered excellent. The technical team decided to use the multi-element-based maritime visibility observation data quality optimization method of this invention to solve this technical challenge.

[0106] The technical team first established a data preprocessing system for marine visibility observation, deploying eight sensor arrays to collect raw observation data on multiple elements, including sea surface wind speed, relative humidity, air temperature, air pressure, precipitation, scattered light intensity, wave height, and sea surface temperature. During a continuous 72-hour observation period, the system collected a total of 17,280 raw data records. Following the overall data quality control process, the technical team systematically processed all raw data, adhering to the basic principles of ensuring data file formats meet standard requirements, that recorded values ​​do not violate element definitions and their spatiotemporal distribution characteristics, and that correct outliers are distinguished from outliers containing errors.

[0107] During the basic information verification phase, the technical team conducted format verification according to standard procedures, comprehensively checking the starting position, length, data record type, and missing values ​​of the observation data. They found 132 data entries with incorrect timestamp formats; after correction, all data passed the format verification. The date verification process strictly adhered to specifications, requiring year values ​​to be no greater than the current year, month values ​​to be between 1 and 12, day values ​​to be between the number of days in the current month, hour values ​​to be between 0 and 23, and minute and second values ​​to be between 0 and 59. Three data entries were found to have hour fields outside the 0-23 range and were marked as "4". The location verification set the observation range to 117°E-128°E and 32°N-42°N; all data passed the location verification. The landing verification, through refined judgment, confirmed that the latitude and longitude of the observation data were all located over the ocean. The integrity verification verified the consistency between the returned ocean buoy station codes and buoy codes, finding two buoy codes with mismatches; these issues were resolved after reconfiguration.

[0108] In the element characteristic verification, the technical team conducted range verification and continuity verification. Range verification determined the normal value range based on the characteristics of the observed element itself; the sea surface wind speed range was set to 0-25. The relative humidity ranges from 30% to 100%, the temperature ranges from -5℃ to 35℃, and the air pressure ranges from 980 to 1040. Rainfall ranges from 0 to 50 mm. The intensity of the scattered light ranges from 100 to 8000. The wave height ranges from 0.1 to 8. The sea surface temperature range is 0℃-28℃. Range testing revealed 56 outliers outside the normal range. Extreme value range testing requires observed values... Should meet Otherwise, the data is questionable. Continuity testing requires observation elements... With adjacent observation elements The change value should satisfy ,in This is 1.1 times the maximum absolute value of the difference between observed elements at adjacent times of buoy points, based on historical data. The test results show that 87 data points were marked due to continuity anomalies, ultimately forming a standardized multi-factor dataset containing 17,081 valid data points.

[0109] The technical team developed a multi-element optical scattering correction model using a multilayer perceptron architecture combined with a convolutional neural network module. The input layer receives eight multi-element parameters, and feature extraction is performed through four densely connected layers. The intermediate layers employ batch normalization and residual connection mechanisms. The output layer generates humidity correction coefficients and wind speed influence coefficient vectors. Model training utilizes three consecutive years of marine observation data from the Yellow and Bohai Seas as the primary data source. Data is stratified according to visibility levels 0 to 9, constructing a balanced training sample set including both normal and abnormal operating conditions. Data augmentation techniques are used to expand the training sample to 500,000 records. The training process employs the Adam optimizer with a learning rate of 0.001, a batch size of 128, and 500 training epochs. Early stopping is used to prevent overfitting, and cross-validation is employed to evaluate the model's generalization ability and select the optimal parameter combination.

[0110] In the application of multi-wavelength laser scattering compensation algorithms, the technical team used wavelengths of 532 nm and 100 nm respectively. 650 and 780 Three laser sources were used for simultaneous measurement, and error compensation was performed by utilizing the differences in scattering characteristics of each wavelength, effectively reducing the scattering interference of tiny water droplets in sea fog on the light signal. A constraint-satisfying solution framework based on dual optimization transformed the visibility correction problem into a constrained convex optimization problem, finding the optimal solution through alternating optimization of the primal and dual problems. A dense connection feature reuse enhancement mechanism enabled feature reuse between forward layers and reduced parameter redundancy during network propagation, maximizing information utilization by directly connecting the output of each layer to the input of all subsequent layers. After calculation, the system obtained a humidity correction coefficient of 1.25 and a wind speed influence coefficient of 0.83, outputting preliminary corrected visibility data.

[0111] like Figure 2 As shown, in the application of the dynamic threshold determination mechanism, when the relative humidity... When the wind speed is greater than 10, the system automatically increases the humidity correction factor by 20%, adjusting it from 1.25 to 1.50. A wind speed compensation mechanism is activated to compensate for and correct the scattering effects of dust and sea salt particles under strong wind conditions. The technical team performs peak checks on visibility anomalies, assuming the current observed value is... The first correct observation adjacent to it is respectively , The verification calculation method is as follows: 34 peak outliers were identified. Extreme value range testing, using the statistical extreme values ​​of historical data from the same period as a benchmark, revealed 21 data points exceeding the extreme value range, forming a filter data set containing quality markers.

[0112] The second LoRa fine-tuning model fine-tunes the third and fourth densely connected layers of the multi-factor optical scattering correction model, improving the model's sensitivity to light fog detection under sea fog conditions by reducing the parameter update amplitude. During fine-tuning, the parameters of the first two layers are fixed, and only the weight matrices of the last two layers are updated. The output feature vectors of the fine-tuned third and fourth densely connected layers are then fused with the output layer of the multi-factor optical scattering correction model. A sparse regularization-based feature selection enhancement mechanism promotes weight sparsity and automatically identifies important features during training through L1 norm penalty, thereby improving the model's interpretability and generalization ability. Nonlinear regression correction is applied to the visibility data in conjunction with meteorological and physical constraints, resulting in a corrected visibility bias parameter of 0.28 and a confidence parameter of 0.87.

[0113] Based on the corrected visibility deviation parameter and confidence parameter, the weight adjustment function calculates the weight adjustment parameters for the third Lora fine-tuning model. According to the formula... ,in This is the adjusted weight coefficient matrix. Based on the basic weight coefficient matrix, To adjust the strength coefficient, To correct the visibility deviation parameter, For standard visibility deviation, For confidence level parameters, Standard confidence level. Adjust the strength coefficient. The visibility deviation parameter is set to 0.15 after correction. The standard visibility deviation is 0.28. The confidence level is 0.35. The confidence level is 0.87, at the standard confidence level. The value is 0.85, and the calculated weight adjustment parameter is 1.034.

[0114] Because the corrected visibility deviation parameter of 0.28 is located at Within the specified range, the system initiates a third LoRa fine-tuning model for three-layer optimization. This third LoRa fine-tuning model fine-tunes the output layer linear transformation matrix and bias vector of the multi-element optical scattering correction model. By precisely adjusting the numerical range of the output layer linear transformation matrix and bias vector, the impact of residual errors on the final result is reduced. During fine-tuning, the parameters of the densely connected layers remain unchanged, optimizing only the output layer parameters. A momentum-based gradient acceleration algorithm is employed, using an exponentially weighted moving average of historical gradients to accelerate convergence and reduce training oscillations during parameter updates.

[0115] During the visibility data correction process, the technical team applied an improved correction formula. ,in To correct visibility, These are the original observations. The relative humidity is (0-1). Wind speed ( ), , This is a correction factor. A dynamic humidity correction factor is introduced in the humidity compensation process. Adjustments were made based on the characteristics of sea fog in the Yellow and Bohai Sea regions. hour, The value is automatically increased by 20%, improving sensitivity to light fog (1-3 km). A piecewise function is used to handle different humidity ranges. Wind speed compensation increases the wind speed influence coefficient. When the wind speed is greater than 10 Automatically activates when needed, targeting the impact of sandstorms under strong wind conditions, with visibility set at 200-500. The range is constrained, and a wind speed threshold filtering mechanism is introduced to exclude abnormal fluctuations. Rain and snow compensation distinguishes precipitation types through scattered light data, determines the compensation coefficient by combining it with temperature measurements, and reduces intermittent precipitation errors through high-frequency sampling.

[0116] The technical team established a multi-element time-series fusion verification mechanism to perform correlation checks between the three-layer optimized visibility data and historical data from the same period. Visibility quality control methods include basic information checks and element characteristic checks. Basic information checks include format checks, equality checks, date checks, location checks, and landing checks. Element characteristic checks mainly involve range checks and correlation monitoring. In the range check, when... Furthermore, when the price was trending downwards for the previous three hours, the quality control symbol... ,otherwise ;when hour, ;when or hour, ,in The visibility data for this moment is shown in Table 1.

[0117] Table 1 Range detection judgment conditions

[0118]

[0119] Correlation detection was validated through correlation analysis of visibility, wind speed, and humidity. When and or At that time, quality control symbol ,otherwise ;when and or At that time, quality control symbol ,otherwise ,in For visibility, For wind speed, For humidity. The system calculates temporal consistency index and spatial continuity index. The temporal consistency index is quantitatively evaluated by the rate of change and trend consistency of observations at adjacent time points. The spatial continuity index verifies the spatial correlation and gradient rationality of visibility data at adjacent observation points through spatial interpolation and neighborhood analysis methods, as shown in Table 2.

[0120] Table 2. Criteria for Correlation Detection

[0121]

[0122] As shown in Table 3, the verification results show that the temporal consistency index is 0.92 and the spatial continuity index is 0.89.

[0123] Table 3 Statistical Table of Temporal Fusion Validation Results

[0124]

[0125] According to the time series consistency index and spatial continuity index The verification results are graded and labeled with quality indicators. The data quality labeling rules are as follows: a blank space indicates no quality control was performed; "1" indicates the data is correct; "2" indicates the data may be correct; "3" indicates the data may be incorrect; "4" indicates the data is incorrect; and " / " indicates the data is missing or abnormal.

[0126] As shown in Table 3, the final quality control report shows that among the 17,081 valid data entries, 15,789 were marked as "1" (92.4%), 1,058 as "2" (6.2%), 189 as "3" (1.1%), and 45 as "4" (0.3%). The data reliability assessment system established by the technical team automatically generates assessment reports based on quality identification and grading. The anomaly warning mechanism sets multi-level warning thresholds. When the number of consecutive data entries with quality grade "3" or "4" exceeds a set amount, the system automatically triggers an warning to notify technical personnel for manual intervention.

[0127] Throughout the observation period, the technical team employed a hybrid LSTM+Attention architecture to process time-series data, adding physical constraints to the output layer to ensure the algorithm conformed to meteorological laws. In the measured data from the Yellow and Bohai Seas, the multi-factor fusion algorithm improved data availability from 82% to 96% after quality control. The system successfully handled various complex marine meteorological environments, including dense sea fog in the morning, strong winds in the afternoon, and nighttime precipitation. Visual plotting verification served as an effective auxiliary method for manual review, intuitively displaying out-of-range abnormal data, abrupt changes in abnormal data, spikes, and missing values.

[0128] For the profile temperature and salinity data, the technical team also conducted profile profile checks and viscosity checks. Profile profile checks determine whether the values ​​of observed features in a profile fall within the depth-varying profile values, while viscosity checks calculate the maximum values ​​of the features. and minimum value The difference ,if If the data is less than a certain value, then all observed elements of the entire profile are considered to be erroneous data.

[0129] This invention represents a significant technological advancement over traditional single-element visibility correction methods. Traditional methods rely primarily on empirical formulas for simple linear corrections, struggling to handle complex nonlinear relationships between multiple elements. This invention, through a dense connection feature reuse enhancement mechanism, can deeply mine implicit correlation patterns among multiple elements, achieving more accurate feature extraction. Traditional methods are poorly adaptable to abnormal conditions and cannot dynamically adjust correction strategies based on environmental changes. The dynamic threshold determination mechanism established in this invention adaptively adjusts correction parameters based on real-time observation conditions, significantly improving data quality under extreme weather conditions. Traditional methods lack a systematic quality assessment framework, often relying on manual experience. The multi-element temporal fusion verification mechanism constructed in this invention establishes an objective and quantifiable quality assessment standard through dual verification of temporal consistency and spatial continuity. Traditional methods have limited error correction capabilities and struggle to effectively handle measurement errors under complex scattering interference conditions. This invention employs a three-layer progressive optimization architecture, using a LoRa fine-tuning model for hierarchical refinement, to gradually eliminate the influence of various error sources, achieving higher-precision data correction.

[0130] It should be noted that the variables involved in this invention are explained in detail in Table 4 below.

[0131] Table 4. Variable Explanation Table

[0132]

[0133] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-element based method for optimizing the quality of marine visibility observation data, characterized in that, The method comprises the following steps: establishing a marine visibility observation data preprocessing system, collecting multi-element original observation data, performing basic information inspection and standardization processing to form a standardized multi-element data set; A multi-element optical scattering correction model is constructed, the standardized multi-element data set is input into the multi-element optical scattering correction model, the non-linear correlation characteristics between the multi-elements are extracted through a dense connection feature reuse enhancement mechanism, a multi-wavelength laser scattering compensation algorithm is used for correction, a humidity correction coefficient and a wind speed influence coefficient are calculated according to a constraint satisfaction solution framework based on dual optimization, and preliminary corrected visibility data are output; A dynamic threshold judgment mechanism is established to test the visibility outliers to form screening marker data; a second Lora fine-tuning model is used for two-layer optimization processing of the screening marker data, key element characteristics are extracted through a feature selection enhancement mechanism based on sparse regularization, and a corrected visibility deviation parameter and a confidence parameter are calculated; a third Lora fine-tuning model is used for three-layer optimization processing, and a gradient acceleration algorithm based on momentum optimization is used to finely correct residual errors; A multi-element time sequence fusion verification mechanism is established to calculate time sequence consistency indicators and spatial continuity indicators, and quality identification grading markers are marked; The final optimized marine visibility observation data are output, a data credibility evaluation system and an abnormal early warning mechanism are established; The multi-element optical scattering correction model is a multi-layer perceptron architecture combined with a convolutional neural network module, the input layer receives eight multi-element parameters, the characteristics are extracted through four dense connection layers, the batch normalization and residual connection mechanism are used in the intermediate layer, and the humidity correction coefficient and the wind speed influence coefficient vector are generated in the output layer; the fine-tuning process of the second Lora fine-tuning model is that the second Lora fine-tuning model fine-tunes the third and fourth dense connection layers of the multi-element optical scattering correction model, the sensitivity of the model to light fog detection under sea fog conditions is improved by reducing the parameter update amplitude of the third and fourth dense connection layers, and only the weight matrices of the last two layers are updated in the fine-tuning process; the fine-tuning process of the third Lora fine-tuning model is that the third Lora fine-tuning model fine-tunes the output layer linear transformation matrix and the bias vector of the multi-element optical scattering correction model, the influence of residual errors on the final result is reduced by finely adjusting the numerical range of the output layer linear transformation matrix and the bias vector, and only the output layer parameters are optimized in the fine-tuning process.

2. The multi-element based marine visibility observation data quality optimization method according to claim 1, wherein, The multi-element original observation data include sea surface wind speed, relative humidity, air temperature, air pressure, precipitation, scattered light intensity, wave height and sea surface temperature, the basic information inspection includes format inspection, date inspection, location inspection, landing inspection and congruence inspection on the multi-element original observation data, and the standardization processing includes range inspection and continuity inspection on the basic information inspection data.

3. The multi-element based marine visibility observation data quality optimization method according to claim 2, wherein, The implementation process of the dynamic threshold judgment mechanism is that the dynamic threshold judgment mechanism is established according to the humidity correction coefficient and the wind speed influence coefficient in the preliminary corrected visibility data, the visibility outliers are tested by peak inspection and extreme value range inspection to form screening marker data.

4. The multi-element based marine visibility observation data quality optimization method of claim 3, wherein, The second Lora fine-tuning model two-layer optimization process is specifically that the visibility data is corrected by nonlinear regression combined with meteorological physical constraints to calculate the corrected visibility bias parameters and confidence parameters.

5. The multi-element based marine visibility observation data quality optimization method according to claim 4, wherein, The third Lora fine-tuning model weight adjustment parameter calculation is specifically that the weight adjustment parameters of the third Lora fine-tuning model are calculated by a weight adjustment function according to the corrected visibility bias parameters and confidence parameters.

6. The multi-element based marine visibility observation data quality optimization method according to claim 5, wherein, The verification process of the multi-element time sequence fusion verification mechanism is specifically that the three-layer optimized visibility data and the historical same period data are subjected to correlation test, and the verification result is marked with quality identification and hierarchical marking according to the time sequence consistency index and the spatial continuity index.

7. The multi-element based marine visibility observation data quality optimization method according to claim 6, wherein, The judgment condition of the dynamic threshold judgment mechanism is that when the relative humidity ∈ (90%, 100%] the humidity correction coefficient is increased by 20%, and when the wind speed > 10 m / s, the wind speed compensation processing is activated.

Images