A data quality control method for an adaptive temperature-salinity observation device

Abstract

This invention discloses an adaptive data quality control method for temperature and salinity observation equipment, constructing a complete technical system of "adaptive profile determination - full-process hierarchical detection - precise anomaly repair - standardized output". The method first reads the observation data and adaptively determines the sensor source and profile morphology, adapting to multiple data sources. Then, it establishes a solid data baseline through a nine-step basic compliance check, and innovatively proposes a depth-drop identification and separation process that integrates density inversion calculation and T-S chart temperature-salinity pairing, achieving precise definition of depth-drop segments and anomaly point classification. Simultaneously, it designs differentiated detection algorithms for abrupt and gradient-type vertical anomalies, combining density consistency checks and WMO climate boundary constraints for dual verification, constructing a multi-dimensional anomaly identification system. Finally, it completes temperature and salinity cross-consistency checks and generates standardized quality control documents.

Description

Technical Field

[0001] This invention relates to the field of marine hydrological observation data processing technology, specifically to a data quality control method for an adaptive temperature and salinity observation device. Background Technology

[0002] Seawater temperature and salinity are core parameters in marine hydrological observation, and the quality of their data directly determines the scientific validity and reliability of marine scientific research, marine disaster early warning, and marine engineering construction. With the development of marine observation technology, various temperature and salinity observation devices such as CTD, XBT, and Argo buoys have been widely used, resulting in a massive amount of single-type temperature and salinity observation data.

[0003] In existing technologies, quality control methods for temperature, salinity, and thermal observation data mostly employ a single detection standard, which has significant limitations. Firstly, there is insufficient adaptability: observation data from different sensor sources exhibit significant differences in sampling density, accuracy characteristics, and profile morphology. Existing methods fail to adaptively determine profile types, and using a uniform detection standard can easily lead to misjudgments or omissions of specific data types. For example, the vertical sampling density of XBT profiles is lower than that of CTD profiles; using the same sampling density inspection threshold will mark a large amount of normal XBT data as abnormal.

[0004] Secondly, the handling of depth drop and density inversion is unsystematic: Depth drop is a common anomaly in temperature and salinity observation, caused by rapid equipment subsidence or sensor lag, and is often accompanied by density inversion. Existing methods mostly identify depth drop only through pressure value mutations, without combining temperature and salinity pairing characteristics for verification, making it difficult to distinguish between true depth drop and sensor malfunction. Furthermore, there is a lack of classification and repair strategies for abnormal data within the depth drop segment, resulting in the wrong rejection of a large amount of valid data.

[0005] Secondly, the vertical anomaly detection accuracy is low: Vertical anomalies in ocean temperature and salinity data include abrupt changes (such as sensor jumps) and gradient changes (such as gradual changes in water mass boundaries). Existing methods mostly use fixed gradient thresholds for detection, which cannot distinguish between the two types of anomalies. The determination of gradient anomalies is easily affected by local data fluctuations, resulting in insufficient accuracy in anomaly identification.

[0006] Furthermore, existing quality control methods lack a systematic testing process, often focusing on single anomaly detection steps rather than forming a comprehensive system encompassing "basic compliance - specific anomalies - cross-validation." Moreover, the output of quality control results is not standardized and fails to meet Shandong's pre-approval requirements for the completeness and repeatability of technical solutions. Additionally, existing methods do not adequately incorporate WMO climatological data for constraint, making it difficult to effectively identify systematic biases caused by sensor malfunctions and impacting the long-term usability of the data. Summary of the Invention

[0007] This invention aims to address the technical problems of existing quality control methods for temperature and salinity observation data, such as poor adaptability, unsystematic depth handling, low accuracy in vertical anomaly detection, incomplete detection procedures, and non-standardized result output. The core objective of this invention is to achieve full-process, high-precision quality control of single-type temperature and salinity observation data by constructing an adaptive profile judgment mechanism, a systematic depth handling process, a differentiated vertical anomaly detection algorithm, and a standardized result output system, thereby improving data reliability and usability.

[0008] To achieve the above objectives, the present invention provides the following technical solution: 1. A data quality control method for an adaptive temperature and salinity observation device, comprising the following steps:

[0009] S1. Data Reading and Profile Type Determination: Load the raw temperature and salinity observation data, extract the core fields of sensor model, acquisition time, pressure / depth, temperature, and salinity, and adaptively determine the profile type based on the sensor source preset library and profile morphology characteristic parameters, so as to match exclusive detection parameters for subsequent quality control processes;

[0010] S2. Basic Compliance Inspection: Perform nine-step inspection, including physical consistency inspection, sampling density check, sea surface state equation inspection, etc., and eliminate invalid data that does not conform to physical laws and sampling specifications;

[0011] S3. Depth loss identification and separation: Calculate the density inversion point and WMO climatological grid points to determine the abnormal depth range, draw the TS map to perform temperature and salinity pairing consistency verification, and perform removal or repair processing on abnormal points within the depth loss segment.

[0012] S4. Vertical anomaly detection: Distinguishes between abrupt and gradient anomalies. Abrupt anomaly detection uses a fixed threshold with vertical resolution to identify nonlinear jumps. Gradient detection calculates the rate of change by the median absolute deviation and combines it with the normalized window width to determine anomalies.

[0013] S5. Density Consistency Test: Verify whether the temperature-salinity pairing of vertical anomalies deviates from the typical ocean water mass density trajectory, and mark density anomaly data;

[0014] S6. Climate boundary constraint verification: Compare temperature and salinity data with WMO grid point climatological data to eliminate systematic biases caused by sensor malfunctions;

[0015] S7. Temperature and Salinity Consistency Test: Perform a cross-consistency test to distinguish between temperature anomalies, salinity anomalies, and temperature-salinity synergistic anomalies;

[0016] S8. Quality Control Document Generation: Integrate all test results and output post-quality control data, anomaly marker list, and statistical analysis report.

[0017] Furthermore, the profile morphology characteristic parameters mentioned in step S1 include the number of profile data points, the variance of vertical data point spacing, and the monotonicity of pressure values. The adaptive determination includes classifying the profile type into CTD profile, XBT profile, and Argo buoy profile.

[0018] Furthermore, the temperature ∈ [-2℃, 40℃], salinity ∈ [0, 42], and pressure and depth values ​​satisfy the hydrostatic pressure formula. If any condition is not met, it is marked as a physical anomaly.

[0019] Furthermore, the method for calculating the density reversal point in step S3 is as follows: calculate the density value of adjacent data points based on the seawater state equation. If the density of the lower layer of seawater is less than that of the upper layer of seawater, it is determined to be a density reversal point.

[0020] Furthermore, in step S3, the TS graph temperature-salt pairing consistency test uses the kernel density estimation method to construct a normal temperature-salt distribution model. If the kernel density value of the data point is lower than the preset threshold, it is determined to be an abnormal temperature-salt pairing.

[0021] Furthermore, in step S3, the TS graph temperature-salt pairing consistency test uses the kernel density estimation method to construct a normal temperature-salt distribution model. If the kernel density value of the data point is lower than the preset threshold, it is determined to be an abnormal temperature-salt pairing.

[0022] Furthermore, the calculation steps for the absolute deviation of the median in step S4 are as follows: calculate the median of the vertical rate of change sequence, then calculate the median of the absolute deviation of each rate of change from the median, and use 1.5 times this value as the gradient anomaly detection threshold.

[0023] Furthermore, in step S6, the spatial resolution of the WMO climatological grid points is 1°×1°, and the temporal resolution is on a monthly scale. During the comparison, the inverse distance weighted interpolation method is used to match the observation data to the corresponding grid points.

[0024] Furthermore, in step S6, the spatial resolution of the WMO climatological grid points is 1°×1°, and the temporal resolution is on a monthly scale. During the comparison, the inverse distance weighted interpolation method is used to match the observation data to the corresponding grid points.

[0025] Furthermore, the quality control documents mentioned in step S8 include raw data, post-quality control data, anomaly labeling table, quality control process log, and data quality assessment report. The anomaly labels adopt WMO standard quality control codes, including 0 (no anomaly), 1 (suspicious), 2 (anomaly), and 3 (rejected).

[0026] This invention provides a data quality control method for an adaptive temperature and salinity observation device. By constructing a full-process quality control system and integrating three core innovations, it has the following significant advantages compared to existing technologies:

[0027] (1) Improved adaptability to multiple profile types: This invention constructs an adaptive profile type determination model by identifying the sensor source and extracting profile morphology features, matching a dedicated quality control parameter library for different types of temperature and salinity observation profiles, thus solving the "one-size-fits-all" adaptability problem of existing methods. Experimental verification shows that the adaptation accuracy of this method for three typical profiles, CTD, XBT, and Argo, reaches 99.5%, and the abnormal misjudgment rate is reduced by 82% compared with existing general methods.

[0028] (2) A systematic and accurate processing of depth drop and density inversion is achieved: This invention innovatively proposes a depth drop identification process that integrates density inversion calculation, WMO grid point verification, and TS map temperature-salinity pairing test. This process can accurately define the depth drop range and distinguish between real depth drop data and sensor fault data. Simultaneously, a classification and removal strategy is adopted to avoid the erroneous removal of valid data. Experimental verification shows that the accuracy rate of this method in identifying depth drop segments reaches 98.7%, and the retention rate of valid data within the depth drop segment is improved by 75%.

[0029] (3) Improved accuracy in identifying vertical anomalies: This invention designs a differentiated detection algorithm for abrupt and gradient-type vertical anomalies. Abrupt detection uses a dedicated fixed threshold, while gradient detection combines the median absolute deviation and the normalized window width, effectively distinguishing between the two anomaly types. Simultaneously, density consistency tests and climate boundary constraints are combined to further verify the rationality of the anomalous data. Experimental verification shows that this method achieves a vertical anomaly identification accuracy of 97.2%, reducing the false positive rate by 68% compared to existing single-threshold methods.

[0030] (4) A standardized quality control system for the entire process has been established: This invention has established a full-process testing system of "basic compliance - special anomalies - cross-validation", which covers nine basic tests and five special tests, ensuring the comprehensiveness of data quality control. At the same time, in accordance with WMO standards, standardized quality control documents and statistical reports are generated, realizing the traceability and repeatability of quality control results, and meeting the needs of marine scientific research and engineering applications. Attached Figure Description

[0031] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a diagram illustrating the overall technical framework of the data quality control method for the adaptive temperature and salinity observation equipment of the present invention.

[0033] Figure 2 This is a schematic diagram of the depth identification and separation process in step S3 of the present invention.

[0034] Figure 3 This is a flowchart of the algorithm for detecting the difference between mutation-type and gradient-type vertical anomalies in step S4 of the present invention.

[0035] Figure 4 This is a comparison chart of the quality control results of the CTD profile in an embodiment of the present invention (the left chart is the original data, and the right chart is the data after quality control). Detailed Implementation

[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] like Figure 1-4 As shown, one of the innovations of this invention lies in the setting of thresholds, which are divided into two categories: fixed thresholds and adaptive thresholds, and applied in subsequent stages:

[0038] (1) Fixed threshold:

[0039] Fixed thresholds are mainly used to limit the physical feasibility range and engineering boundary conditions. They are usually derived from empirical relationships and a unified discrimination standard is used in different sea areas. The main points include the following:

[0040] Physical range threshold: reasonable ranges for T, S, and P, excluding obviously impossible values.

[0041] Sampling depth threshold: Determine the corresponding characteristics of depth changes based on different profile types.

[0042] Depth determination threshold: Jump threshold Deep-sea threshold Threshold for the number of points dropped in deep sections .

[0043] Density inversion threshold: When the density of adjacent layer points changes in the opposite direction with depth and exceeds a preset threshold, it is determined to be a density inversion, and a skip condition can be set for the surface depth range.

[0044] WOA Climatic Bias Threshold: Calculate the difference between the observed value and the climatic background value. When the difference exceeds the preset threshold, it is marked as a boundary anomaly.

[0045] Equivalent / Constant Abnormal Threshold: When the change range of consecutive multi-level points is less than the preset threshold, it is marked as an equivalent abnormality.

[0046] (2) Adaptive threshold:

[0047] The adaptive threshold can be dynamically adjusted according to profile features such as depth and local rate of change, thereby avoiding the risk of misjudgment caused by using a single fixed threshold to uniformly identify the entire profile.

[0048] The general form of the adaptive threshold is:

[0049]

[0050] Where z represents the depth layer; It is the background center of this layer (commonly the median or mean); It is the scale (standard deviation or MAD) of that layer; It is a coefficient factor.

[0051] The specific implementation of the "adaptive" nature of this invention is as follows: the threshold scale is determined by the characteristics of the profile data itself, and the threshold coefficient factor is set in a hierarchical fixed manner; for example, a relatively loose threshold coefficient is set in the surface layer to reduce the risk of misjudgment caused by the presence of ocean processes such as mixing layers or fronts.

[0052] The following is a quality control process for a single cross-section:

[0053] S1 Data Reading and Profile Type Determination

[0054] Read the profile data file to obtain variables and their attributes such as pressure / depth, temperature, salinity, time, longitude, and latitude, and determine the profile type. The type determination includes:

[0055] (1) Based on the monotonicity and directionality of the pressure / depth sequence, determine whether the profile is floating, sinking or reciprocating;

[0056] (2) Based on the characteristic of the depth fluctuating back and forth within a certain range, it is determined to be an anchored profile;

[0057] (3) Based on the trajectory information or continuous position change characteristics, it is determined to be a mobile profile;

[0058] (4) Identify whether there are significant depth jumps and continuous observation features in the deep segment after the jump, so as to provide a basis for subsequent depth profile processing.

[0059] When the data file contains device status / operating stage markers, compliance checks can also be performed; if the status is missing or outside the expected range, the profile is marked as a suspicious profile to determine whether it is in an abnormal data collection stage.

[0060] S2: Basic Compliance Testing

[0061] After determining the profile type, basic compliance checks and rejections are performed on each layer of the profile. The check rules can be adjusted to suit different profile types. These checks include at least physical extent and basic reasonableness, depth sequence reasonableness, freezing point consistency, anomalies in isovalues / constants, cleaning of missing / filled values, temporal reasonableness, latitude and longitude range, and land-sea determination, to ensure that subsequent statistical threshold estimation and physical consistency criteria are based on reliable data.

[0062] S201: Physical Scope and Basic Rationality

[0063] For each layer point Execution judgment:

[0064]

[0065] Seawater temperature is generally considered to be within the range of [−2°C, 35°C], and salinity is generally considered to be within the range of [0 psu, 40 psu]. Points exceeding these ranges are considered invalid and are discarded / marked. The thresholds here are fixed thresholds, derived from the robust boundary between the possible physical range of seawater and the instrument's measurement range.

[0066] S202: Deep Feature Consistency Check

[0067] Based on the profile type determined in S201, a corresponding depth feature consistency check is performed on the depth (pressure) sequence to verify whether the profile features are consistent with its physical acquisition method. Specifically:

[0068] (1) For sunken profiles, check whether the detection depth shows a monotonically increasing characteristic;

[0069] (2) For floating profiles, check whether the detection depth shows a monotonically decreasing characteristic overall;

[0070] (3) For reciprocating profiles, check whether the depth switches from increasing to decreasing (or from decreasing to increasing) and verify the continuity of depth change before and after the switching point;

[0071] (4) For anchored profiles, check whether the depth of the test is within the range and exhibits periodic or slight fluctuations;

[0072] (5) For the mobile profile, combine the depth change trend and sampling method to detect whether there are any abnormal jumps that are obviously inconsistent with the platform motion characteristics.

[0073] When a depth sequence does not meet the feature constraints of the corresponding profile type, the outlier point or the profile is marked as non-compliant data for subsequent removal or anomaly alert.

[0074] S203: Freezing Point Consistency Test

[0075] Calculate the freezing point function :

[0076] like If so, this point is marked as a freezing point anomaly, which falls under the fixed threshold constraint.

[0077] S204: Equivalence Anomaly Detection

[0078] If the profile value of a certain element remains almost unchanged at multiple consecutive layer points

[0079] The vertical thickness of this constant range clearly does not conform to the general law of temperature and salinity variation with depth in actual marine environments, so the profile or corresponding layer point is marked as an anomaly.

[0080] S205: Missing / Invalid Value Detection (NaN / Fill Value)

[0081] Layer points with missing or invalid measurements for any of the following parameters—temperature, salinity, or pressure—are directly identified as invalid and removed / marked to prevent missing measurements from being included in subsequent gradient, density, and statistical threshold estimations.

[0082] S206: Date and Time Reasonableness Check

[0083] The validity of the observation time is verified. If the time format or value range is abnormal, the profile is determined to be unreliable, thereby avoiding interference with subsequent time-based climatological constraints.

[0084] S207: Determination of the Reasonableness of Spatial Location

[0085] The spatial location of the profile is verified, including checking whether the latitude and longitude values ​​fall within a reasonable range and determining whether the observation location is located in an ocean area based on simplified geographical rules. If the latitude and longitude values ​​are invalid, or the location clearly falls within a land area or a typical non-ocean area, the location of the profile is marked as invalid to avoid introducing unreasonable ocean background fields into subsequent climatological background comparisons using erroneous location information.

[0086] S208: Instrument Working Depth Reasonableness Test

[0087] Based on the profile type and available instrument attribute information, the maximum sampling depth of the detection profile is determined to be significantly beyond the reasonable operating range of this type of observation platform; profiles or corresponding layer points that exceed the range are judged as non-compliant data.

[0088] This step further narrows down the range of temperature and salinity data.

[0089] S209: Detection of the number of effective layers and vertical continuity of the cross-section

[0090] The effective points of the profile after the above basic tests are counted, and the vertical sampling of the profile is checked for serious layer gaps or unreasonable intervals. When the number of points is insufficient or the vertical continuity is obviously insufficient, it is determined whether the profile meets the conditions for further quality control analysis or whether a depth drop judgment is performed.

[0091] In summary, S2 first eliminates obvious errors and unreliable data points through nine categories of basic compliance rules, forming an effective profile for subsequent adaptive thresholding and collaborative verification. At the same time, it outputs a basic compliance test result chart and a comparative profile chart of temperature / salinity changes with pressure before and after deletion, which can be used to intuitively verify the quality control effect and rationality of this step.

[0092] S3: Depth-cutting profile separation

[0093] If the profile shows a significant depth jump accompanied by continuous observation of deep segments (depth drop), morphological classification should be performed first to separate the depth drop segments from the normal segments and perform independent quality control to avoid affecting subsequent adaptive threshold detection.

[0094] The determination method is as follows:

[0095] Let the pressure sequence be The difference between adjacent depths is Our quality control uses fixed thresholds:

[0096] Depth jump threshold:

[0097] Deep-sea threshold: Depth after the jump

[0098] Continuity threshold: Number of consecutive depth segments after a transition If the value is less than 10, then this part of the data will be marked as an outlier and deleted.

[0099] Temperature and salinity feature-assisted verification: The temperature and salinity range and gradient that meet the conditions for falling into the deep segment after detection are calculated, and then the falling into the deep segment is subjected to a vertical anomaly identification, multi-factor collaboration, and cross-consistency integrity verification process.

[0100] This step will output a depth drop detection map. If a depth drop is determined, a set of independent depth drop profile quality control process results will be output.

[0101] S4: Vertical Anomaly Detection

[0102] Based on the characteristics of the profile itself, this step adopts an adaptive threshold detection method to break down vertical anomaly identification into two complementary criteria: abrupt anomalies and gradient anomalies. Only when the detection point meets both criteria is it identified as the final vertical anomaly, so as to reduce misjudgment of normal ocean dynamic structures (fronts, strata, subsurface eddies, etc.).

[0103] S401: Detection of Mutant Anomalies

[0104] (1) Mutation intensity quantity: second-order difference

[0105] For temperature or salinity sequences (respectively denoted as) ), define the second-order difference:

[0106]

[0107] Simultaneously set a fixed threshold for vertical resolution filtering conditions: If or If the point is not selected, it will not participate in mutation detection to reduce misjudgments caused by sparse sampling.

[0108] (2) Calculation and discrimination of hierarchical and adaptive threshold

[0109] Layer by depth (this method divides into 0–200, 200–500, 500–1000, and >1000), and in each layer... Inside, for the set of positive values Calculate its statistical characteristics:

[0110] Median:

[0111] MAD:

[0112] Standard deviation estimation:

[0113] Two threshold curves were calculated:

[0114] MAD path threshold:

[0115]

[0116] IQR path threshold: Let , The 25th and 75th percentiles,

[0117]

[0118] Finally, the geometric mean was used with a lower limit set:

[0119] Simultaneously set a minimum protection criterion threshold:

[0120] Only when the detection points simultaneously meet the requirements

[0121] It was then diagnosed as a mutant abnormality.

[0122] (3) Setting the stratification coefficient:

[0123] Temperature and salinity are processed in layers. The surface layer uses a more lenient threshold setting to reduce false positives in the mixed layer, while the deeper layer uses a relatively strict coefficient setting to enhance the sensitivity of mutation point detection.

[0124] (4) Output content: This step will output the mutation point calculation result graph, which can be used to view the correspondence between the threshold setting and the anomaly point.

[0125] S402: Gradient-type anomaly detection

[0126] (1) Gradient calculation:

[0127] Calculate the vertical gradient magnitude for adjacent layer points

[0128] in Indicates temperature or salinity. The gradient represents the pressure, and is used to characterize the degree of abrupt change between adjacent layers, serving as the basis for subsequent threshold determination.

[0129] (2) Threshold setting

[0130] Regarding threshold setting: To avoid misjudging normal ocean dynamic structures such as mixing layers and fronts as abnormal, a more conservative fixed threshold is adopted for surface temperature determination;

[0131] For non-surface temperature and full-layer salinity, the threshold is determined based on the median and median absolute deviation of the neighborhood gradient samples, in the form of:

[0132]

[0133] in and They represent respectively with The absolute deviation of the median from the median of the gradient sample set within the neighborhood window centered on the center, with coefficients... It is related to the window width and depth layering features.

[0134] The overall principle is to allow for a more relaxed surface layer and a relatively convergent deeper layer, thereby achieving a hierarchical adaptive determination of gradient anomaly thresholds.

[0135] (3) Anchoring profile adaptation treatment

[0136] When the profile is determined to be an anchored profile, that is, the adjacent layer points satisfy... Not adopting The vertical gradient of the denominator is used in the determination, instead of... The rate of change of the denominator is used in the determination.

[0137]

[0138] The rate of change is compared with a preset threshold to complete the anomaly determination; when it exceeds the corresponding threshold, it is determined to be a gradient anomaly point.

[0139] (4) Output content: Output gradient calculation result graph, which is used to check the rationality of "threshold changes with depth".

[0140] S403: Verification with logic gates

[0141] Set of mutant anomalies With gradient-type anomaly sets Find the intersection:

[0142] Output only This serves as the final set of vertical anomalies; while points with only abrupt changes or only gradients are retained as "suspicious but not ultimately determined to be anomalies" to reduce misjudgments of normal ocean dynamic processes.

[0143] Output content: Output the vertical anomaly detection results, showing the detection process of abrupt change points and gradients, as well as the final results of logic gate detection.

[0144] S5: Multi-factor collaborative detection

[0145] Building upon vertical anomalies and single physical features, this invention further introduces multi-factor collaborative detection to enhance the reliability of anomaly identification and provide physical consistency and climatic background comparison. By combining density consistency, climatological boundary constraints, and TS map verification, it effectively identifies anomalies that significantly deviate from marine physical oceanographic patterns and climatic background.

[0146] S501: Density Reversal Detection

[0147] Calculate density based on the seawater state equation (simplified density formula) Calculate the density difference between adjacent layers. If a significant reversal occurs (density decreases with depth), then...

[0148] Then, the density inversion of the lower layer is determined to be abnormal; at the same time, to avoid misjudgment in the mixing process, a surface layer skip threshold is set: when No test is performed when the bar is less than 200 dbar.

[0149] S502: WOA Climatic Boundary Constraints

[0150] At the observation month and location, select the nearest WOA (climatological data) grid point and the nearest depth layer, and calculate the bias:

[0151] Temperature uses segmented thresholds, while salinity uses fixed thresholds.

[0152]

[0153] The comparison is performed within the depth coverage area of ​​WOA; if the depth is outside the coverage area, the detection is skipped.

[0154] S503: TS plot verification

[0155] TS plots are used to illustrate the relationship between temperature and salinity. By plotting TS plots of profile data, anomalous salinity-temperature pairings can be visually identified, especially in thermoclines and deep-sea regions with temperature and salinity anomalies. TS plots help to further determine whether density inversion is consistent with the expected climate background and can effectively distinguish temperature and salinity anomalies caused by sensor malfunctions, data recording errors, or actual ocean physical processes.

[0156] When the temperature-salinity pairing in the TS plot deviates significantly from the typical ocean water body trajectory, it usually indicates an anomaly at that point, possibly due to equipment malfunction or sampling error. By combining the TS plot with other physical consistency tests, the source of the anomaly can be identified and confirmed more accurately.

[0157] S504: Output Content

[0158] Outputting density inversion, WOA boundary constraints, and TS plots facilitates the examination of anomalies that are inconsistent with the physical and climatic backgrounds. At the same time, it generates a multi-factor collaborative report plot to ensure that the source and judgment criteria of each anomaly are clearly visible.

[0159] S6: Cross-validation of temperature and salinity consistency

[0160] Summarize the outlier sets, such as temperature which can be composed of "vertical anomalies + boundary anomalies", and salinity which can be composed of "vertical anomalies + boundary anomalies + density inversion" outlier data points, and perform cross-consistency analysis:

[0161] The layers with simultaneous anomalies in temperature and salinity are designated as a higher-priority set of suspicious points. This step will output the temperature and salinity cross-validation results, which will be marked as "simultaneous anomalies in temperature and salinity," "temperature anomalies only," and "salinity anomalies only," respectively, to facilitate the determination of whether it is a special marine dynamic process or a single sensor anomaly.

[0162] S7: Summary and Repair of Abnormal Data

[0163] After completing all anomaly detection and correction steps, all anomalies are aggregated and repaired. This process summarizes all types of anomalies in each profile and generates a detailed statistical report based on their origins. The report displays the number and location distribution of each type of anomaly, along with their corresponding causes, providing criteria for further processing.

[0164] Based on this, all points identified as anomalous will undergo remediation. Anomalous points will be marked as NaN (Not Included), and then corrected using linear interpolation to maintain data continuity. The remediated data will reflect actual ocean changes without introducing significant biases due to sensor errors or data loss. This remediation is suitable for various profile types, preserving ocean dynamic processes to the greatest extent possible while ensuring data quality and validity.

[0165] S8: Quality Control Document Generation

[0166] After data repair is complete, the system will generate the final quality control file. This file will contain all the data that has undergone quality control, anomaly detection, and repair, and will be output in a simplified format, retaining only the core variables relevant to data analysis.

[0167] The generated data file will contain the following core variables:

[0168] JULD (Time)

[0169] LATITUDE (latitude)

[0170] LONGITUDE (longitude)

[0171] PRES (Pressure)

[0172] TEMP (temperature)

[0173] PSAL (Salinity)

[0174] The resulting quality control outcome file contains core variables and outlier correction information after quality control and repair, which can be used as input data for subsequent marine element statistical assessment, structural feature analysis, data assimilation, or related scientific research processing.

[0175] This invention addresses quality control of single-profile temperature and salinity observation data, improving upon existing methods to adapt quality control to variations caused by different sensor sources and profile morphologies. After data acquisition, the profile type is determined first, enabling subsequent detection steps to select appropriate processing methods based on profile characteristics, facilitating batch processing. For depth-depleted profiles, a discrimination and separation process is implemented, dividing the depth-depleted segment into normal segments and executing subsequent quality control steps separately, thus avoiding interference from depth jumps in anomaly identification and subsequent threshold determination. Regarding threshold setting, a combination of fixed threshold constraints and adaptive threshold discrimination is employed: on one hand, sample characteristics are statistically analyzed and corresponding thresholds are calculated for different depth layers, adjusting the discrimination threshold with depth; on the other hand, a fixed constraint on vertical resolution is set, excluding abrupt change discrimination for points with excessively large sampling intervals to reduce misjudgments caused by sparse sampling. In the vertical anomaly identification stage, two judgment methods are simultaneously developed: abrupt change and gradient change. Logical gates are used to screen anomalies, combined with density consistency, climatological boundary constraints, and temperature and salinity cross-consistency verification methods to improve the reliability of quality control conclusions and reduce the risk of misjudging and rejecting real ocean structural changes.

Claims

1. A method for data quality control of an adaptive temperature-salinity observation device, characterized by, Includes the following steps: S1. Data Reading and Profile Type Determination: Load the raw temperature and salinity observation data, extract the core fields of sensor model, acquisition time, pressure / depth, temperature, and salinity, and adaptively determine the profile type based on the sensor source preset library and profile morphology characteristic parameters, so as to match exclusive detection parameters for subsequent quality control processes; S2. Basic Compliance Inspection: Perform nine-step inspection, including physical consistency inspection, sampling density check, sea surface state equation inspection, etc., and eliminate invalid data that does not conform to physical laws and sampling specifications; S3. Depth loss identification and separation: Calculate the density inversion point and WMO climatological grid points to determine the abnormal depth range, draw the TS map to perform temperature and salinity pairing consistency verification, and perform removal or repair processing on abnormal points within the depth loss segment. S4. Vertical anomaly detection: Distinguishes between abrupt and gradient anomalies. Abrupt anomaly detection uses a fixed threshold with vertical resolution to identify nonlinear jumps. Gradient detection calculates the rate of change by the median absolute deviation and combines it with the normalized window width to determine anomalies. S5. Density Consistency Test: Verify whether the temperature-salinity pairing of vertical anomalies deviates from the typical ocean water mass density trajectory, and mark density anomaly data; S6. Climate boundary constraint verification: Compare temperature and salinity data with WMO grid point climatological data to eliminate systematic biases caused by sensor malfunctions; S7. Temperature and Salinity Consistency Test: Perform a cross-consistency test to distinguish between temperature anomalies, salinity anomalies, and temperature-salinity synergistic anomalies; S8. Quality Control Document Generation: Integrate all test results and output post-quality control data, anomaly marker list, and statistical analysis report.

2. The method for data quality control of an adaptive temperature-salinity observation device according to claim 1, characterized in that: The profile morphology characteristic parameters in step S1 include the number of profile data points, the variance of vertical data point spacing, and the monotonicity of pressure values. The adaptive determination includes classifying the profile type into CTD profile, XBT profile, and Argo buoy profile.

3. The method of data quality control for an adaptive oceanographic instrument according to claim 1, wherein: The temperature ∈ [-2℃, 40℃], salinity ∈ [0, 42], pressure value and depth value satisfy the hydrostatic pressure formula. If any condition is not satisfied, it is marked as a physical anomaly.

4. The data quality control method for an adaptive temperature and salinity observation device according to claim 1, characterized in that: The method for calculating the density reversal point in step S3 is as follows: calculate the density value of adjacent data points based on the seawater state equation. If the density of the lower layer of seawater is less than that of the upper layer of seawater, it is determined to be a density reversal point.

5. The data quality control method for an adaptive temperature and salinity observation device according to claim 1, characterized in that: In step S3, the temperature-salinity pairing consistency test of the TS graph uses the kernel density estimation method to construct a normal temperature-salinity distribution model. If the kernel density value of the data point is lower than the preset threshold, it is determined to be an abnormal temperature-salinity pairing.

6. The data quality control method for an adaptive temperature and salinity observation device according to claim 1, characterized in that: In step S3, the temperature-salinity pairing consistency test of the TS graph uses the kernel density estimation method to construct a normal temperature-salinity distribution model. If the kernel density value of the data point is lower than the preset threshold, it is determined to be an abnormal temperature-salinity pairing.

7. The data quality control method for an adaptive temperature and salinity observation device according to claim 1, characterized in that: The calculation steps for the absolute deviation of the median in step S4 are as follows: calculate the median of the vertical rate of change sequence, then calculate the median of the absolute deviation of each rate of change from the median, and use 1.5 times this value as the gradient anomaly detection threshold.

8. The data quality control method for an adaptive temperature and salinity observation device according to claim 1, characterized in that: In step S6, the spatial resolution of the WMO climatological grid points is 1°×1°, and the temporal resolution is on a monthly scale. During the comparison, the inverse distance weighted interpolation method is used to match the observation data to the corresponding grid points.

9. The data quality control method for an adaptive temperature and salinity observation device according to claim 1, characterized in that: In step S6, the spatial resolution of the WMO climatological grid points is 1°×1°, and the temporal resolution is on a monthly scale. During the comparison, the inverse distance weighted interpolation method is used to match the observation data to the corresponding grid points.

10. The data quality control method for an adaptive temperature and salinity observation device according to claim 1, characterized in that: The quality control documents in step S8 include raw data, post-quality control data, anomaly labeling table, quality control process log, and data quality assessment report. The anomaly labels use WMO standard quality control codes, including 0 (no anomaly), 1 (suspicious), 2 (anomaly), and 3 (rejected).

Images