Method, device and medium for quality control of marine observation data

Abstract

The application discloses a quality control method and device of marine observation data and a medium, relates to the technical field of marine observation data processing, and comprises the following steps: acquiring marine observation data, wherein the marine observation data comprises sampling data of each observation period in multiple observation periods; sequentially performing missing value inspection, global range inspection and stratified statistical inspection on the sampling data of the observation period to obtain first inspection data; performing average processing on the first inspection data to obtain an observation value; sequentially performing empirical range inspection and quartile range inspection on the observation value to obtain a second inspection value and a second inspection mark; filling the second inspection value based on the second inspection mark, combining a local weighted regression scatter smoothing algorithm and an STL decomposition and reconstruction algorithm, and updating the second inspection value and the second inspection mark to complete quality control. The application can realize reasonable filling of missing values, accurate detection of abnormal values and data quality traceability.

Description

Technical Field

[0001] This application relates to the field of marine observation data processing technology, and in particular to a method, equipment and medium for quality control of marine observation data. Background Technology

[0002] In core areas such as marine scientific research, marine engineering construction, and marine environmental monitoring, marine in-situ fixed-point long-term time-series data (i.e., marine observation data) refers to time-series data obtained by long-term, continuous in-situ measurements at a specific point in the ocean with fixed longitude, latitude, and depth through observation equipment. It is the core support for revealing the evolutionary laws of elements in local marine areas, and its quality directly determines the depth of marine scientific research, the effectiveness of marine instrument and equipment testing, and the efficiency of the implementation of marine operational observation technologies.

[0003] However, the highly dynamic and disruptive nature of the marine environment makes observation equipment susceptible to hardware failures, power outages, communication anomalies, maintenance, and data recovery, resulting in significant data loss. Simultaneously, extreme weather events and abrupt ocean current changes can lead to extreme true values, while sensor drift and biofouling introduce erroneous outliers. Directly using such uncontrolled raw data will result in analytical biases and model inaccuracies, ultimately misleading critical decisions in marine engineering, environmental assessments, marine disaster early warning, and climate research. Therefore, systematic quality control (including removing erroneous values, identifying suspicious values, and appropriately imputing missing values) is a crucial step in generating reliable and usable standard datasets, and is essential for ensuring the reliability of marine scientific research and practice.

[0004] However, the quality control methods commonly used for marine observation data have obvious limitations when dealing with in-situ fixed-point long-term time series data: (1) The missing value handling scheme is too simplified. It often uses direct deletion of missing values ​​or single interpolation (such as linear interpolation) to fill missing values. It does not fully consider the core characteristics of marine observation data such as temporal correlation, periodicity and trend, resulting in the filled values ​​not conforming to the actual marine environment change pattern and the filling is unreasonable; (2) The outlier detection scheme lacks accuracy and adaptability. It often relies on preset fixed thresholds or simple statistical models (such as mean ± standard deviation) to detect outliers. It is difficult to adapt to the complex characteristics of marine observation data changing with time and environment, and it is easy to miss or misdetect; (3) The data classification processing logic lacks systematicity. It has not established clear quality level classification standards. The marking rules and conversion process of data of different quality levels are unclear, which makes it difficult to trace the source of data quality and cannot meet the differentiated needs of data quality for different observation purposes.

[0005] Therefore, in view of the shortcomings of traditional technologies, there is an urgent need to develop a quality control method that can fully match the characteristics of marine observation data, realize the reasonable filling of missing values, accurately detect outliers, and has a clear hierarchical processing logic and traceability. Summary of the Invention

[0006] The purpose of this application is to provide a method, device and medium for quality control of marine observation data, which can realize reasonable filling of missing values, accurate detection of outliers and traceability of data quality.

[0007] To achieve the above objectives, this application provides the following solution.

[0008] In a first aspect, this application provides a quality control method for ocean observation data, the quality control method for ocean observation data comprising: Acquire ocean observation data; ocean observation data includes sampled data for each observation period across multiple observation cycles, and the sampled data includes the sampled values ​​at each sampling time point within the observation cycle; For each observation period, the sampled data of the observation period are subjected to missing value testing, global range testing, and stratified statistical testing in sequence to obtain the first test data of the observation period. The first test data includes the first test value and the first test label for each time point in the observation period. If the sampled value at the time point is a normal value, then the first test value is the sampled value and the first test label is the first value. If the sampled value at the time point is an error value, then the first test value is the sampled value and the first test label is the second value. If the sampled value at the time point is a missing value, then the first test value is the missing value placeholder and the first test label is the third value. For each observation period, the first test data of the observation period are averaged to obtain the observed value. The observed value is then subjected to empirical range test and interquartile range test in sequence to obtain the second test value and second test label of the observation period. If the observed value is a normal value, the second test value is the observed value and the second test label is the first value. If the observed value is an outlier, the second test value is the observed value and the second test label is the fourth value. If the observed value is an error value, the second test value is the observed value and the second test label is the second value. If the observed value is a missing value, the second test value is a missing value placeholder and the second test label is the third value. For each observation period, if the second test mark of the observation period is the second value or the third value, it is determined whether the second test value of the observation period can be reasonably filled. If so, the second test value of the observation period is filled by combining the local weighted regression scatter smoothing algorithm and the STL decomposition and reconstruction algorithm to obtain the filled value. The second test value of the observation period is then updated to the filled value, and the second test mark of the observation period is updated to the fifth value. If not, the second test value of the observation period is updated to the missing value placeholder, and the second test mark of the observation period is updated to the third value. The second test values ​​and second test markers of all observation periods are combined to form the ocean quality control data.

[0009] Secondly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to implement the above-described method for quality control of marine observation data.

[0010] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method for quality control of marine observation data.

[0011] According to the specific embodiments provided in this application, this application has the following technical effects.

[0012] This application provides a method, device, and medium for quality control of marine observation data. The method involves acquiring marine observation data, which includes sampled data from each of multiple observation periods. For each observation period, the sampled data is sequentially subjected to missing value testing, global range testing, and stratified statistical testing to obtain the first test data for that period. The first test data includes the first test value and the first test label for each time point within the observation period. If the sampled value at a time point is normal, the first test value is the sampled value, and the first test label is the first value. If the sampled value at a time point is erroneous, the first test value is the sampled value, and the first test label is the second value. If the sampled value at a time point is missing, the first test value is the missing value placeholder, and the first test label is the third value. The first test data for the observation period is averaged to obtain observed values. These observed values ​​are then sequentially subjected to empirical range testing and interquartile range testing to obtain the second test value and the second test label for the observation period. If the observed value is normal... If the observed value is an outlier, the second test value is the observed value, and the second test label is the first value. If the observed value is an error, the second test value is the observed value, and the second test label is the fourth value. If the observed value is an error, the second test value is the observed value, and the second test label is the second value. If the observed value is a missing value, the second test value is a missing value placeholder, and the second test label is the third value. If the second test label for the observation period is the second value (error) or the third value (missing value), it is determined whether the second test value for the observation period can be reasonably filled. If so, the second test value for the observation period is filled by combining the local weighted regression scatter smoothing algorithm and the STL decomposition and reconstruction algorithm to obtain the filled value, and the second test value for the observation period is updated to the filled value, and the second test label for the observation period is updated to the fifth value. If not, the second test value for the observation period is updated to the missing value placeholder, and the second test label for the observation period is updated to the third value. The second test values ​​and second test labels for all observation periods are combined to form the ocean quality control data. This application combines local weighted regression scatter smoothing algorithm and STL decomposition and reconstruction algorithm to fill in missing values ​​(referring to observational values ​​in marine observational elements such as temperature and salinity that were expected to be obtained directly or indirectly but were not actually obtained) and erroneous values ​​(referring to observational values ​​in marine observational elements such as temperature and salinity that failed the test). It fully considers the core characteristics of marine observation data, such as temporal correlation, periodicity, and trend, to achieve reasonable filling of missing values. When detecting outliers, it combines missing value testing, global range testing, stratified statistical testing, empirical range testing, and interquartile range testing to adapt to the complex characteristics of marine observation data that change dynamically with time and environment, achieving accurate detection of outliers. Furthermore, it continuously changes the testing labels during the testing process, with a clear hierarchical processing logic. Subsequent data quality traceability can be performed based on the testing labels, ensuring data quality traceability. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is an application environment diagram for a quality control method for marine observation data provided in Embodiment 1 of this application.

[0015] Figure 2 This is a flowchart illustrating a quality control method for marine observation data provided in Embodiment 1 of this application.

[0016] Figure 3 This is a schematic diagram of the structure of a computer device provided in Embodiment 2 of this application. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] Example 1.

[0019] The quality control method for marine observation data provided in this application embodiment can be applied to, for example... Figure 1The application environment shown depicts a scenario where the terminal communicates with the server via a network. The data storage system stores the data the server needs to process. This data storage system can be configured independently, integrated into the server, or located in the cloud or on another server. The terminal can send a pending quality control request to the server. Upon receiving the request, the server acquires ocean observation data. For each observation period, the sampled data is sequentially subjected to missing value testing, global range testing, and stratified statistical testing to obtain the first test data for that period. For each observation period, the first test data is averaged to obtain the observed value. The observed value is then sequentially subjected to empirical range testing and interquartile range testing to obtain the second test value and second test label for that period. For each observation period, if the second test label is the second or third value, it is determined whether the second test value can be reasonably filled. If so, the second test value is filled using a combination of local weighted regression scatter smoothing algorithm and STL decomposition and reconstruction algorithm to obtain the filled value. The second test value and the second test label for that period are then updated to the fifth value. If not, the second test value is updated to a missing value placeholder, and the second test label is updated to the third value. The second test values ​​and second test labels for all observation periods are then combined to form the ocean quality control data. The server can feed back the quality control result, namely the marine quality control data obtained in response to the quality control request, to the terminal.

[0020] In addition, in some embodiments, the quality control method for marine observation data can also be implemented by a server or a terminal. For example, the terminal can directly process the quality control request to be processed, or the server can obtain the quality control request to be processed from the data storage system and process it.

[0021] In one exemplary embodiment, such as Figure 2 As shown, a quality control method for marine observation data is provided. This method is executed by computer equipment, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment, the method is applied to... Figure 1 The following steps are used as an example of a server in the example.

[0022] Step S1: Obtain ocean observation data; ocean observation data includes sampling data for each observation period in multiple observation periods, and the sampling data includes the sampling value at each sampling time point in the observation period.

[0023] Step S2: For each observation period, the sampled data of the observation period are sequentially subjected to missing value test, global range test, and stratified statistical test to obtain the first test data of the observation period. The first test data includes the first test value and the first test label for each time point in the observation period. If the sampled value at the time point is a normal value, then the first test value is the sampled value and the first test label is the first value. If the sampled value at the time point is an error value, then the first test value is the sampled value and the first test label is the second value. If the sampled value at the time point is a missing value, then the first test value is the missing value placeholder and the first test label is the third value.

[0024] Step S3: For each observation period, the first test data of the observation period is averaged to obtain the observed value. The observed value is then subjected to empirical range test and interquartile range test in sequence to obtain the second test value and second test label of the observation period. If the observed value is a normal value, the second test value is the observed value and the second test label is the first value. If the observed value is an outlier, the second test value is the observed value and the second test label is the fourth value. If the observed value is an error value, the second test value is the observed value and the second test label is the second value. If the observed value is a missing value, the second test value is a missing value placeholder and the second test label is the third value.

[0025] Step S4: For each observation period, if the second test mark of the observation period is the second value or the third value, determine whether the second test value of the observation period can be reasonably filled. If so, combine the local weighted regression scatter smoothing algorithm and the STL decomposition and reconstruction algorithm to fill the second test value of the observation period, obtain the filled value, update the second test value of the observation period to the filled value, and update the second test mark of the observation period to the fifth value. If not, update the second test value of the observation period to the missing value placeholder and update the second test mark of the observation period to the third value.

[0026] Step S5: Compile the second test values ​​and second test markers from all observation periods into ocean quality control data.

[0027] By implementing steps S1 to S5 above, this embodiment can achieve reasonable filling of missing values, accurate detection of outliers, and traceability of data quality.

[0028] The following section provides a detailed description of the quality control method for the marine observation data used in this embodiment, including the following steps.

[0029] (a) Acquiring ocean observation data.

[0030] In this embodiment, ocean observation data is acquired, which includes sampling data for each observation period in multiple observation periods, and the sampling data includes the sampling value at each sampling time point in the observation period.

[0031] The marine observation data in this embodiment can be in-situ fixed-point long-term time series data from offshore test sites. Temperature and salinity are the most basic and critical core observation elements of the marine environment. The integrity, accuracy, and reliability of their data are prerequisites for subsequent data analysis, model building, and decision-making. Therefore, marine observation data can be temperature time series data and salinity time series data, etc. In this case, the quality control method of marine observation data in this embodiment can also be called the quality control method of in-situ fixed-point long-term temperature or salinity data from offshore test sites. Specifically, it is a quality control method for long-term temperature or salinity data collected in-situ at offshore test sites, which is applicable to the processing of temperature and salinity data from similar observation platforms such as buoys, underwater moorings, or seabed bases.

[0032] This embodiment first determines the following core time parameters: (1) Start time Format: YYYY (year) / MM (month) / DD (day) HH (hour): MM (minute): SS (second); (2) End time The format is the same as the start time, and it meets the requirements. ; (3) Sampling time interval The time interval between two consecutive sampling actions actually performed by the sensor; (4) Sampling duration The total duration during which the sensor is actually in operation and performs sampling within an observation period; (5) Observation time interval The basic time period for the system to execute a complete observation task refers to the time interval between the start of one observation task and the start of the next identical observation task.

[0033] The above core time parameters need to meet the following constraints: ; like ,but .

[0034] Based on the above core time parameters, a complete set of time points (i.e., a complete set of sampling time points) and a set of observation periods are generated.

[0035] Complete time point set for: ; in, For the first A point in time; This represents the total number of time points.

[0036] Observation period set for: ; in, For the first One observation period; For the first The starting time point of each observation cycle; For the first The end time of each observation cycle; This represents the total number of observation periods.

[0037] For example, to carry out a one-day observation task, design =1 hour =10 minutes =10 seconds. At this time, one day is divided into 24 observation cycles (0-1 hour, ..., 23-24 hours). Within each observation cycle, sampling is performed at a sampling time interval of 10 seconds. The sampling is performed continuously for 10 minutes to obtain the sampling data of the observation cycle. The sampling data of the 24 observation cycles constitute the ocean observation data.

[0038] When performing quality control on ocean observation data, the data processing workflow is as follows: Ocean observation data ( ) Missing value test ( ) Global range test ( ) Stratified statistical tests ( ) L1 generation ( ) Empirical range test ( ) Interquartile range test ( ) Data Reconstruction and Adaptive Interpolation ).

[0039] Based on the above data processing flow, the data flow process in the hardware is as follows: [Sensor] → (Acquisition of raw ocean observation data) → [Observation Platform Acquisition and Storage] → (Temporarily saved) → [Observation Platform Data Processor] , , → (Primary quality control data) → [Observation platform communication system sends] → (Satellite / Wireless Transmission / Cable Transmission) → [Onshore Data Processing Center] → (Receive / Verify / Analyze) → [Rough Data Products] → (Data deletion and filling) → [Final Data Product] ].

[0040] In the data processing workflow, inspection markers are used to label data quality. These markers are defined as follows: 0: Original, uninspected; 1: Inspection passed (normal value); 3: Suspicious / abnormal trend (outlier value); 4: Error / out of limit (erroneous value); 5: Interpolated (improved value); 9: Missing (missing value). Of course, other numbers can also be used for labeling, as long as different numbers are used for different data quality levels.

[0041] (ii) Level 0 (real-time) data processing.

[0042] In this embodiment, for each observation period, the sampled data of the observation period are sequentially subjected to missing value test, global range test and stratified statistical test to obtain the first test data of the observation period. The first test data includes the first test value and the first test mark of each time point in the observation period. If the sampled value of the time point is a normal value, then the first test value is the sampled value and the first test mark is the first value (1). If the sampled value of the time point is an error value, then the first test value is the sampled value and the first test mark is the second value (4). If the sampled value of the time point is a missing value, then the first test value is the missing value placeholder and the first test mark is the third value (9).

[0043] (1) Level L0 raw data .

[0044] L0 level raw data (L0 level refers to raw sampled data containing the most basic timestamp information, that is, sampled data of a certain observation period in ocean observation data) If not tested, it is marked as 0. for: ; in, For a specific moment; Numerical value; For marking; The set of sampling time points is the set of all time points between the start and end time points of the observation period from which the sensor actually records a reading (regardless of whether the reading is valid). In other words, it is the set of sampling time points for each sample value in the sampling data. Let be the set of real numbers.

[0045] (2) Missing value test ).

[0046] right Perform missing value testing to obtain the data after missing value testing. Specifically, it is based on the complete set of time points corresponding to the observation period. (It is) A subset of is The set of time points between the start and end points of the observation period. , To determine missing values ​​(based on the number of time points corresponding to the observation period), we mark missing values ​​as 9. for: ; in, These are sampled values; This is a placeholder for missing values.

[0047] (3) Global range test ).

[0048] right Perform a global range test to obtain the data after the global range test. Specifically, it is determined to be outside the preset physical range. The error value is set to 4, and the error value is marked as 4. for: ; in, This is the lower limit of the preset physical range. The upper limit, lower limit and upper limit are preset physical ranges, and both are determined based on the data type.

[0049] For example, for temperature, the preset physical range is... , The lower limit of temperature, , The upper limit is for temperature; for salinity, the preset physical range is... , This is the lower limit of salinity. , This represents the upper limit of salinity.

[0050] (4) Stratified statistical test ).

[0051] right Perform stratified statistical tests to obtain the data after stratified statistical tests. Specifically, in the observation period Within, for datasets that pass the global scope test Perform stratified statistical tests. for The set of values ​​marked as 0 in the middle determines Sample size , for Calculate the number of values ​​marked as 0 in the middle. mean and standard deviation If the stratified statistical test passes, it is considered a normal value and is marked as 1. If the stratified statistical test fails, it is considered an outlier, i.e. an error value, and is marked as 4. The specific test rules are as follows.

[0052] (4.1) Sample size (3σ criterion test): like ,but ;otherwise, .

[0053] (4.2) Sample size (Grubbs test): like ,but ;otherwise, ; in, Here are the critical values ​​for the Grubbs test and the parameters for the Grubbs test. For any It can be calculated in real time using the exact formula of the Grubbs test (which involves the inverse operation of the cumulative distribution function of the t-distribution). .

[0054] (4.3) Sample size : Without statistical elimination, directly label .

[0055] The above inspection rules apply only to The value marked as 0 in the middle, for For values ​​marked as 4 and 9, the original markings are retained.

[0056] at this time, for: ; in, for The marker in.

[0057] In this embodiment, the sampling data of the observation period are sequentially subjected to missing value testing, global range testing, and stratified statistical testing to obtain the first test data of the observation period, which specifically includes the following steps.

[0058] (1) Generate a complete set of time points corresponding to the observation period. The complete set of time points includes multiple time points. The first time point is the start time point of the observation period, and the last time point is the end time point of the observation period. The time interval between two adjacent time points is the sampling time interval.

[0059] (2) For each time point in the complete time point set, determine whether the time point is a sampling time point in order to perform missing value testing. If yes, the first test value of the time point is the sampled value of the sampling time point, and the initial test mark is the sixth value (0). If no, the first test value of the time point is the missing value placeholder, and the first test mark is the third value (9).

[0060] (3) For each initial test mark as the sixth value (0) at the first time point, determine whether the first test value at the first time point is within the preset physical range for global range testing. If yes, the intermediate test mark at the first time point is the sixth value (0). If no, the first test mark at the first time point is the second value (4).

[0061] (4) For each intermediate test point marked as the sixth value (0), perform a stratified statistical test on the first test value of the second time point. If the stratified statistical test is passed, the first test value of the second time point is marked as the first value (1). If the stratified statistical test is not passed, the first test value of the second time point is marked as the second value (4).

[0062] Among them, the first test value at the second time point is subjected to stratified statistical test. If the stratified statistical test is passed, the first test value at the second time point is marked as the first value (1). If the stratified statistical test is not passed, the first test value at the second time point is marked as the second value (4). Specifically, the following steps are included.

[0063] (1) If the number of the second time point is greater than or equal to the first preset value (30, or other values ​​may be used), the first test value of the second time point is tested by the 3σ criterion. At this time, it is determined whether the absolute value of the difference between the first test value and the mean of the second time point is less than or equal to 3 times the standard deviation. If yes, the stratified statistical test is passed and the first test value of the second time point is marked as the first value (1). If no, the stratified statistical test is not passed and the first test value of the second time point is marked as the second value (4). The mean and standard deviation are the mean and standard deviation of the first test values ​​of all the second time points.

[0064] (2) If the quantity at the second time point is greater than or equal to the second preset value (10, or other values ​​may be used) and less than the first preset value (30), then the first test value at the second time point is subjected to the Grubbs test. At this time, it is determined whether the ratio of the absolute value of the difference between the first test value at the second time point and the mean to the standard deviation is less than or equal to the critical value of the Grubbs test. If yes, the stratified statistical test is passed and the first test value at the second time point is marked as the first value (1). If no, the stratified statistical test is not passed and the first test value at the second time point is marked as the second value (4).

[0065] (3) If the number at the second time point is less than the second preset value (10), the first test value at the second time point passes the stratified statistical test, and the first test value at the second time point is marked as the first value (1).

[0066] (III) L1 level (delay level) data processing.

[0067] In this embodiment, for each observation period, the first test data of the observation period is averaged to obtain the observed value. The observed value is then subjected to empirical range test and interquartile range test in sequence to obtain the second test value and second test mark of the observation period. If the observed value is a normal value, the second test value is the observed value and the second test mark is the first value (1). If the observed value is an abnormal value, the second test value is the observed value and the second test mark is the fourth value (3). If the observed value is an error value, the second test value is the observed value and the second test mark is the second value (4). If the observed value is a missing value, the second test value is a missing value placeholder and the second test mark is the third value (9).

[0068] (1) Generation of L1 level observations ( ).

[0069] based on Generate L1 level observations Specifically based on valid data within the observation period ( (The numerical values) generate L1 level observations and define the effective set. , for The set of values ​​marked as 1, the number of valid values. , for The number of values ​​marked as 1 in the middle, then for: ; in, The time of the observation period; The value represents the observation period; This serves as a marker for the observation period.

[0070] (2) Empirical range test ).

[0071] right Perform an empirical range test to obtain the data after the empirical range test. Specifically, it was determined to be beyond the preset experience range. The error value is set to 4, and the error value is marked as 4. for: ; in, This is the lower limit of the preset experience range. The upper limit of the preset experience range is set, while the lower and upper limits are determined by the user.

[0072] (3) Interquartile range test ).

[0073] right Perform the interquartile range test to obtain the data after the interquartile range test. Specifically, calculate L1 data (i.e., data from all observation periods). quartiles of the values ​​marked as 1 (25th percentile) and (75th percentile), further calculate the percentile difference. To determine the effective interval Subsequently, the interquartile range is used to perform an interquartile range test to detect abnormal trends. Outliers are marked as 3, and normal values ​​that pass the test are marked as 1. for: ; The above interquartile range test is only for The value marked as 1 in the middle is still marked as 4 after empirical range testing.

[0074] Current methods lack intelligent integration between L0 and L1. This embodiment introduces an observation period to achieve dynamic stratification of the original sampling data, ensuring that statistical tests are conducted under the most reasonable conditions, thereby optimizing the quality control process.

[0075] At this point, in this embodiment, the first test data of the observation period is averaged to obtain the observed value. Specifically, it includes: determining whether the number of first test values ​​marked as first value (1) in the first test data of the observation period is greater than 0; if so, calculating the average value of the first test values ​​marked as first value (1) in the first test data of the observation period to obtain the observed value, and setting the observed value as first value (1); if not, using the missing value placeholder as the observed value, and setting the observed value as third value (9).

[0076] The observation values ​​are subjected to empirical range test and interquartile range test in sequence to obtain the second test value and the second test mark of the observation period. Specifically, if the observation mark is the first value, it is determined whether the observation value is within the preset empirical range to perform empirical range test and obtain the first judgment result. If the first judgment result is negative, the second test value of the observation period is the observation value and the second test mark is the second value (4). If the first judgment result is positive, it is determined whether the observation value is within the valid interval to perform interquartile range test and obtain the second judgment result. If the second judgment result is positive, the second test value of the observation period is the observation value and the first test mark is the first value (1). If the second judgment result is negative, the second test value of the observation period is the observation value and the first test mark is the fourth value (3). If the observation mark is the third value (9), the second test value of the observation period is the missing value placeholder and the second test mark is the third value (9).

[0077] The effective interval is: ,in, The 25th percentile of the observations for all observation periods marked as the first value. This represents the percentile difference. , The 75th percentile of the observations for all observation periods marked as the first value.

[0078] (iv) Level 2 data processing.

[0079] In this embodiment, for each observation period, if the second test mark of the observation period is the second value (4) or the third value (9), it is determined whether the second test value of the observation period can be reasonably filled. If so, the second test value of the observation period is filled by combining the local weighted regression scatter smoothing algorithm and the STL decomposition and reconstruction algorithm to obtain the filled value. The second test value of the observation period is updated to the filled value, and the second test mark of the observation period is updated to the fifth value (5). If not, the second test value of the observation period is updated to the missing value placeholder, and the second test mark of the observation period is updated to the third value (9). The second test values ​​and second test marks of all observation periods are combined to form marine quality control data to complete quality control.

[0080] The data reconstruction and adaptive interpolation strategy used in this embodiment ( Specifically, it includes the following steps.

[0081] (1) Input and output definitions.

[0082] enter( L1 level quality control data sequence ,in , The meanings are: 1 - normal value, 3 - abnormal value, 4 - incorrect value, 9 - missing value.

[0083] Output ( L2 level reconstructed data sequence ,in, Numerical values ​​for reconstructing the data sequence at L2 level. Labels for L2 level reconstructed data sequences, , The meaning is as follows: :like (If it passes all inspections, then) ; :like (If the interquartile range test is abnormal but retained), then ; :like And if reasonable interpolation is possible (i.e., reasonable filling is possible), then , For filler values; :like And if reasonable interpolation is not possible (i.e., reasonable filling is not possible), then .

[0084] (2) Filling strategy.

[0085] For each point that needs to be rebuilt ( Locate the nearest valid observation point before and after it. and , The nearest valid observation point ( Its value is the most recent valid value. For the most recent effective observation point ( (Its value is the second most recent valid value), the length of the missing segment. , The time corresponding to the nearest valid observation point. If the time corresponding to the most recent valid observation point is... If the critical time (i.e., the critical time length, which is recommended not to exceed 30 days, but can be replaced with other values ​​according to actual application needs) is not considered to be able to be reasonably filled, otherwise it is considered to be able to be reasonably filled.

[0086] If reasonable data filling is possible, first determine the effective data window. Specific points to be rebuilt Before and after the corresponding time Valid observation points within the day ( ) form a valid data window, if , If the preset time length is used, it indicates short-term missing data. In this case, the valid data window is used as input, and a locally weighted regression scatter smoothing algorithm is used to fill the missing data to obtain the filled value; otherwise, if... (Right now If the missing data is less than 1000 days, it indicates a medium-to-long-term missing data. In this case, the effective data window is used as input, and the STL decomposition and reconstruction algorithm is used to fill the missing data to obtain the filled value.

[0087] (2.1) Short-term missing: Applicable to near-shore high-frequency fluctuation scenarios.

[0088] The process of the local weighted regression scatter smoothing algorithm is as follows.

[0089] 1) Window calculation: ; in, Suggested window points; The minimum window point threshold; The width of the window; This is the window scaling factor; For effective data windows The number of valid observation points in the data.

[0090] Ensure that the window contains at least one observation period before and after the missing segment.

[0091] 2) Weight calculation: ; ; in, For the first valid data window Standardized distance of each effective observation point; For the first valid data window The time corresponding to each valid observation point; Points requiring reconstruction The corresponding moment; For bandwidth parameters, ; For the first valid data window The weight of each valid observation point; This is an indicator function that takes the value 1 if the condition within the parentheses is true, and 0 otherwise.

[0092] 3) Fitting target: Fitting using a weighted second-order polynomial: ; Among them, the fitting results for , For fitting parameters The value of , for , for The value of , for The value of , for The value of , The constant term represents time. The reference value at that time The coefficient of the first-order term represents a local linear trend. The coefficient of the quadratic term represents the local curvature; Points requiring reconstruction The set of neighborhood points, including the points that need to be reconstructed. forward One and after One effective observation point; For the first valid data window The values ​​of each valid observation point.

[0093] 4) Fill value calculation: .

[0094] (2.2) Medium and long time missing: Applicable to nearshore data with significant tidal / daily cycles.

[0095] The process of the STL decomposition and reconstruction algorithm is as follows.

[0096] 1) Periodic testing: For raw time series data (In this embodiment, the data refers to the data in the valid data window), the calculation process is as follows: Calculate power spectral density : ; in, This is a Fourier transform.

[0097] Identify main frequency : .

[0098] Calculate the power ratio : ; in, Main frequency The power spectral density at that location.

[0099] This condition requires that the period of the original time series data must match the typical tidal period, i.e., the basic period. It should be equal to 12.42 hours (corresponding to the major semi-diurnal tide M2) or 24 hours (corresponding to the diurnal tidal component) to ensure that the extracted periodic components have clear tidal physical significance. At the same time, the original time series data must be statistically significant, that is, its explained variance contribution rate (or energy share) exceeds 80%, indicating that tidal oscillation is the dominant signal in the original time series data, rather than random noise or other non-periodic processes.

[0100] 2) Component modeling: Trend item ( ): Cubic spline interpolation is used to characterize long-term changes, and random walk Kalman smoothing is combined to effectively suppress random drift noise.

[0101] ; in, For random walk Kalman smoothing; For cubic spline interpolation; These are the valid observation points in the valid data window.

[0102] Seasonal items ( ): Phase matching and historical averaging are used.

[0103] Specifically, first calculate the points that need to be rebuilt. phase : ; in, As a reference time starting point, the time corresponding to the first valid observation point in the valid data window is generally taken; The detected main cycle (12.42h or 24h).

[0104] Then in the valid data window Finding sets of in-phase It is the point that needs to be rebuilt. A set of effective observation points with similar phases is used to extract the tidal / diurnal-dominated periodic fluctuations: ; in, For the first valid data window The phase of each effective observation point; This is the deviation threshold.

[0105] Finally, calculate the seasonal term. It is a point that needs to be rebuilt. The seasonal component represents the periodic fluctuations caused by the tidal / diurnal cycle: ; in, The number of elements in the set with the same phase; For the first valid data window The original measurements of each valid observation point; For the first valid data window The trend term value for each valid observation point is obtained by trend term modeling.

[0106] residual term ( ): The accuracy-weighted fusion of linear interpolation and Gaussian process regression is used.

[0107] Specifically, based on the points that need to be rebuilt Linear interpolation is performed on the residuals of the nearest valid observation points to obtain the linear interpolation estimate. Then use the squared exponent kernel Make predictions. This is the timestamp of the first time point, typically represented as the timestamp of the prediction or query point. The timestamp of the second time point, usually represented as the timestamp of the training point or reference point, is used to obtain the mean. and variance residual term for: ; in, For linear interpolation in The estimated variance at that point is used for fusion weight calculation.

[0108] The fill value is: .

[0109] Check the fill value relative to Does the jump exceed , The reference standard deviation is calculated based on adjacent valid data. If it exceeds the standard deviation, the parameters are adjusted and the calculation is repeated or the data is marked as missing.

[0110] In this embodiment, determining whether the second test value of the observation period can be reasonably filled specifically includes: determining the first most recent valid value before the second test value of the observation period and the second most recent valid value after the second test value of the observation period, wherein both the first most recent valid value and the second most recent valid value are second test values ​​marked as the first value or the fourth value; calculating the difference between the time corresponding to the first most recent valid value and the time corresponding to the second most recent valid value to obtain the length of the missing segment; if the length of the missing segment is greater than the critical time length, then the second test value of the observation period cannot be reasonably filled, otherwise, the second test value of the observation period can be reasonably filled, so as to determine whether the second test value of the observation period can be reasonably filled.

[0111] The second test value of the observation period is filled by combining the local weighted regression scatter smoothing algorithm and the STL decomposition and reconstruction algorithm to obtain the filled value. Specifically, if the time length of the missing segment is less than or equal to the preset time length, the second test value of the observation period is filled by the local weighted regression scatter smoothing algorithm to obtain the filled value; if the time length of the missing segment is greater than the preset time length, the second test value of the observation period is filled by the STL decomposition and reconstruction algorithm to obtain the filled value.

[0112] This embodiment aims to address the core technical pain points of traditional technologies in quality control of in-situ fixed-point long-term time-series data: First, traditional methods fail to adapt to diverse sampling strategies (such as different sampling frequencies, sampling durations, and observation accuracy requirements) in different scenarios such as scientific research, operational observation, and engineering technology development, resulting in a mismatch between quality control effects and actual application needs; Second, missing value processing does not take into account the temporal characteristics of the sampled data, leading to insufficient accuracy in data completion; Third, outlier detection has poor adaptability and low accuracy, making it difficult to effectively distinguish between sudden changes in the real environment and data interference; Fourth, the data grading and processing logic is chaotic, lacking a unified quality grade classification standard and a clear labeling and conversion process, resulting in weak data quality traceability and an inability to meet the differentiated data quality requirements of different scenarios; Fifth, the quality control effect of single-point data is poor, and the defects in the basic data are not effectively addressed, leading to further highlighting and amplification of the defects in single-point data during the subsequent comparison, fusion, and optimization of point, line, surface, and three-dimensional observation data, ultimately resulting in substandard quality of the fused data product, which cannot support multi-dimensional marine observation research and engineering applications.

[0113] To address the aforementioned issues, this embodiment provides an efficient and reliable real-time + delayed dual-mode data quality control method. By optimizing the missing value identification and labeling algorithm, constructing an adaptive outlier detection model, and establishing a systematic data grading system, it improves the quality of in-situ fixed-point long-term data from the source, preventing its defects from being amplified in multi-dimensional data fusion. This achieves full-process, high-precision quality control of in-situ fixed-point long-term data, providing standardized basic data products that meet quality requirements for different observation purposes and multi-dimensional data fusion applications.

[0114] The quality control method for in-situ fixed-point long-term time-series data used in this embodiment includes: first, acquiring marine observation data and determining the sampling time point and observation period; then, sequentially performing missing value testing, global range testing, and stratified statistical testing on the L0-level raw data to generate L1-level observation values, followed by empirical range testing and interquartile range testing. Through multi-layer testing and clear labeling, a more transparent network model data relationship is achieved, which can effectively handle data missing problems, accurately identify outliers, and ensure the integrity, reliability, and traceability of the data, providing high-quality data support for marine-related research and applications.

[0115] To address the issue of incompatibility with diverse sampling strategies, this embodiment constructs a core time parameter system and a hierarchical processing mechanism. Specifically, it first defines core time parameters, including the sampling time interval, sampling duration, and observation time interval, and then sets... ≤ and = ,but The =0 rule can simultaneously adapt to various sampling strategies such as continuous sampling, triggered mode sampling, and instantaneous snapshot sampling. Through the logic of mapping the set of sampling time points to the set of observation periods, it can standardize them into a unified L1 level observation sequence regardless of the sampling frequency or duration used by the front-end hardware. .

[0116] To address the issue of insufficient accuracy in data completion, considering that traditional methods rely on simple deletion or linear interpolation, ignoring temporal correlation, periodicity, and trends, this embodiment adopts an adaptive interpolation strategy. It abandons simple linear interpolation and classifies data based on the duration and periodicity of missing data, selecting different strategies. Specifically, missing data is categorized into short-term missing data (…). ) and medium- to long-term missing ( For short-term missing data, a local weighted regression scatter smoothing algorithm is used. This algorithm takes into account both local trends and random fluctuations, which is more consistent with the short-term fluctuation characteristics of nearshore waters affected by tides and winds. For medium- and long-term missing data, an STL decomposition and reconstruction algorithm is used. The algorithm uses fast Fourier transform to identify the dominant frequency (12.42h / 24h). The STL decomposition and reconstruction algorithm decomposes the data into trend terms, seasonal terms, and residual terms, and models and reconstructs them independently. This is more consistent with the periodicity (daily / semi-daily period) and trend of marine observation data, and fully preserves the tidal patterns and seasonal trends of the marine environment.

[0117] To address the issues of poor adaptability and low accuracy in outlier detection, considering that traditional methods rely on fixed thresholds and struggle to adapt to dynamic changes, leading to missed and false detections, this embodiment constructs a multi-level testing system from physical range to statistical characteristics. The detection algorithm is dynamically adjusted based on data density. Specifically, a global range test (physical extrema) is first performed to eliminate obvious non-physical values. Subsequently, a stratified statistical test is conducted, abandoning the use of fixed thresholds and calculating the mean within the sampling window of the observation period. ) and standard deviation ( This enables dynamic adjustment of the detection standard according to the environmental background field, and employs a sample size adaptive algorithm based on the sample size ( Automatic switching of statistical methods, large sample ( Using the 3σ criterion, small samples ( The Grubbs test is used to avoid missed or false detections in small samples. Finally, the interquartile range test is performed, which is specifically used to identify trend anomalies in intraday data. That is, to identify outliers that are within the physical range but have abnormal trends, mark outliers (i.e., abnormal values), and distinguish between real environmental changes and data interference.

[0118] To address the issue of weak traceability of data quality, this embodiment establishes a clear L0... L1 The L2 grading standard defines clear labeling and transfer rules. Specifically, the grading standard is as follows: L0 level (real-time level): raw data, only physical range and missing value marking are made, and the most original information is retained; L1 level (delay level): quality control data after stratified statistical test and interquartile range test, outliers are removed, that is, observation data after noise reduction and statistical test, which is suitable for general analysis; L2 level (reconstruction level): based on L1, missing values ​​are scientifically interpolated and reconstructed, which is suitable for continuous model-driven or long time series analysis, which is the final product after scientific interpolation and quality redefinition, and a unified quality mark is defined: 0-undetected, 1-high quality, 3-suspect / abnormal retention, 4-error, 5-filling, 9-missing. L2 level retains the original valid (1), abnormal retention (3) and interpolation reconstruction (5), ensuring that users are clear about the source and quality status of each data, meeting differentiated needs. This allows users to choose whether to use reconstructed data according to their needs, realizing the traceability of data quality.

[0119] To address the issue of poor quality control at single points, this embodiment provides a high-quality, continuous L2-level reconstruction data product. Through rigorous cleaning and inspection, noise and errors in the original data are eliminated, preventing single-point defects from being amplified in subsequent processing. Not only are anomalies removed, but missing values ​​are also scientifically filled in, providing a set of high-quality basic data with continuous temporal sequence and strong physical consistency. This directly solves the problem of inability to fuse or distorted fusion caused by data interruption or inconsistent quality in multi-source data fusion (such as point and area fusion, pattern assimilation).

[0120] This embodiment systematically addresses the five pain points mentioned above, effectively supporting the demand for high-quality marine data in scientific research and engineering applications at the marine test site.

[0121] Compared with traditional technologies, this embodiment has the following beneficial effects: (1) By checking for missing values ​​in the system, filling the time stamp gaps and marking them clearly, and introducing the local weighted regression scatter smoothing algorithm and the STL decomposition and reconstruction algorithm, the problem of unreasonable data missing processing is solved, and the integrity of the time series is guaranteed; (2) By adopting multi-level testing methods such as global range testing, hierarchical statistical testing, empirical range testing and interquartile range testing, combined with Grubbs' test and other methods, outliers can be accurately identified, and the reliability of the data is improved; (3) A clear hierarchical processing logic for L0 to L2 level data is established, and the data status is distinguished by different markings, ensuring the traceability of the data, making the data relationship clearer and more transparent, and facilitating the evaluation and analysis of data quality; (4) The entire quality control process is standardized and rigorous, and is suitable for the processing of long time series data of in-situ fixed points in the marine test site, providing high-quality data support for the background observation and related research and application of experimental technology in the marine test site.

[0122] The following is a specific example.

[0123] The temperature and salinity sensor is mounted at a water depth of 1m inside the buoy instrument well to select the temperature start time. The temperature ended at 09:30:00 on June 20, 2023. The starting time for salinity was selected as 09:30:00 on June 20, 2024. The salinity end time is 2022-01-01 00:00:00. The sampling time interval is 2023-01-01 00:00:00. Both are 1800 seconds, sampling duration All are 0 seconds, unless otherwise agreed. =0 is a special marker value, indicating continuous sampling throughout the entire observation period, rather than zero-duration sampling; the observation time interval... Both are 1800 seconds. = That is, the sensor's sampled value equals the buoy system's observed value.

[0124] In the complete set of time points for temperature, seconds, k starts from 0 and continues until... No more than .

[0125] In the complete set of time points for salinity, seconds, k starts from 0 and continues until... No more than .

[0126] Set of temperature observation periods middle, ,in Second, i starts from 0 and continues until... No more than .

[0127] Set of observation periods for salinity middle, ,in Second, i starts from 0 and continues until... No more than .

[0128] The data processing procedure is as follows.

[0129] (1) Level L0 raw data ( ): Collect raw temperature or salinity data at the sampling time points within this time period, and mark all data as 0.

[0130] (2) Missing value test: check (In this example, it is from) arrive (a complete set of time points spaced 1800 seconds apart), for those not in Fill in the time points in the data. = Marked as 9, we get .

[0131] (3) Global range test: for The data in the middle is examined. For temperature: <-2.5 or The value >40 is marked as 4. Reserves between -2.5 and 40 are marked as 0. = The retention mark is 9, resulting in Regarding salinity: <0 or The value >41 is marked as 4, and... The reserved values ​​between 0 and 41 are marked as 0. = The retention mark is 9, resulting in .

[0132] (4) Stratified statistical test: for each observation period Data that passes the global range test (marked as 0) ,calculate , and ,when At that time, if Mark 1 if 10 ≤ 4 otherwise; <30 hours, if ≤ Mark 1 if 0 < 1, otherwise mark 4; when 0 < 1 When the value is less than 10, no test is performed, and the value is marked as 1; for values ​​originally marked as 4 or 9, the original mark is retained, and the result is obtained. .

[0133] (5) Generation of L1 level observations: in each observation period Inside, collect Data marked as 1 ,calculate ,like >0, then The mean of these data is labeled as 1; if =0, then = Marked as 9, we get .

[0134] (6) Experience range test: set according to experience. ,right The data in the middle was examined. Items outside this range are marked as 4, and items within this range are marked as 1. = The retention mark is 9, resulting in .

[0135] (7) Interquartile range test: Calculate the 25th percentile of the L1 data. and the 75th percentile Further calculations ,right The data in the middle was examined. Not here The range is marked as 3, in The range is marked as 1. = The retention mark is 9, resulting in .

[0136] (8) Interpolation: for Fill in the blanks to obtain S2.

[0137] Through the above steps, the quality control of long-term temperature or salinity data at the in-situ fixed point of the marine test site is completed.

[0138] This application also provides an application scenario in which the above-described quality control method for marine observation data is applied. Specifically, the quality control method for marine observation data provided in this embodiment can be applied in a quality control scenario. The quality control scenario includes a quality control stage and a display stage. The quality control stage is used to perform quality control on the marine observation data to obtain marine quality control data, and the display stage is used to present the marine quality control data to users for ease of use. The quality control method for marine observation data provided in this embodiment belongs to the quality control stage.

[0139] Example 2.

[0140] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 3As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a quality control method for marine observation data.

[0141] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0142] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the quality control method for marine observation data in Embodiment 1.

[0143] Example 3.

[0144] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the quality control method for ocean observation data in Embodiment 1.

[0145] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of the relevant data are carried out in compliance with the relevant data protection laws and policies of the country where the location is located, and with the authorization granted by the owner of the corresponding device.

[0146] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0147] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for quality control of marine observation data, characterized in that, The quality control methods for the marine observation data include: Acquire ocean observation data; ocean observation data includes sampled data for each observation period across multiple observation cycles, and the sampled data includes the sampled values ​​at each sampling time point within the observation cycle; For each observation period, the sampled data of the observation period are subjected to missing value testing, global range testing, and stratified statistical testing in sequence to obtain the first test data of the observation period. The first test data includes the first test value and the first test label for each time point in the observation period. If the sampled value at the time point is a normal value, then the first test value is the sampled value and the first test label is the first value. If the sampled value at the time point is an error value, then the first test value is the sampled value and the first test label is the second value. If the sampled value at the time point is a missing value, then the first test value is the missing value placeholder and the first test label is the third value. For each observation period, the first test data of the observation period are averaged to obtain the observed value. The observed value is then subjected to empirical range test and interquartile range test in sequence to obtain the second test value and second test label of the observation period. If the observed value is a normal value, the second test value is the observed value and the second test label is the first value. If the observed value is an outlier, the second test value is the observed value and the second test label is the fourth value. If the observed value is an error value, the second test value is the observed value and the second test label is the second value. If the observed value is a missing value, the second test value is a missing value placeholder and the second test label is the third value. For each observation period, if the second test mark of the observation period is the second value or the third value, it is determined whether the second test value of the observation period can be reasonably filled. If so, the second test value of the observation period is filled by combining the local weighted regression scatter smoothing algorithm and the STL decomposition and reconstruction algorithm to obtain the filled value. The second test value of the observation period is then updated to the filled value, and the second test mark of the observation period is updated to the fifth value. If not, the second test value of the observation period is updated to the missing value placeholder, and the second test mark of the observation period is updated to the third value. The second test values ​​and second test markers of all observation periods are combined to form the ocean quality control data.

2. The quality control method for marine observation data according to claim 1, characterized in that, The sampled data for the observation period were sequentially subjected to missing value tests, global range tests, and stratified statistical tests to obtain the first set of test data for the observation period, which specifically included: Generate a complete set of time points corresponding to the observation period; the complete set of time points includes multiple time points, the first time point is the start time point of the observation period, the last time point is the end time point of the observation period, and the time interval between two adjacent time points is the sampling time interval. For each time point in the complete time point set, determine whether the time point is a sampling time point to perform missing value testing. If it is, the first test value of the time point is the sampled value of the sampling time point, and the initial test label is the sixth value. If not, the first test value of the time point is the missing value placeholder, and the first test label is the third value. For each initial test point marked as the sixth value, determine whether the first test value of the first time point is within the preset physical range for global range testing. If yes, the intermediate test mark of the first time point is the sixth value; otherwise, the first test mark of the first time point is the second value. For each intermediate test point marked as the sixth value at the second time point, a stratified statistical test is performed on the first test value at the second time point. If the stratified statistical test passes, the first test value at the second time point is marked as the first value; otherwise, the first test value at the second time point is marked as the second value.

3. The quality control method for marine observation data according to claim 2, characterized in that, A stratified statistical test is performed on the first test value at the second time point. If the stratified statistical test passes, the first test value at the second time point is marked as the first value; if it fails, the first test value at the second time point is marked as the second value. Specifically, this includes: If the number of values ​​at the second time point is greater than or equal to the first preset value, then the 3σ criterion test is performed on the first test value at the second time point. In this case, it is determined whether the absolute value of the difference between the first test value at the second time point and the mean is less than or equal to 3 times the standard deviation. If yes, the stratified statistical test is passed, and the first test value at the second time point is marked as the first value. If no, the stratified statistical test is not passed, and the first test value at the second time point is marked as the second value. The mean and standard deviation are the mean and standard deviation of the first test values ​​at all second time points. If the quantity at the second time point is greater than or equal to the second preset value and less than the first preset value, then a Grubbs test is performed on the first test value at the second time point. At this time, it is determined whether the ratio of the absolute value of the difference between the first test value at the second time point and the mean to the standard deviation is less than or equal to the critical value of the Grubbs test. If yes, then the stratified statistical test is passed and the first test value at the second time point is marked as the first value. If no, then the stratified statistical test is not passed and the first test value at the second time point is marked as the second value. If the number at the second time point is less than the second preset value, then the first test value at the second time point passes the stratified statistical test, and the first test value at the second time point is marked as the first value.

4. The quality control method for marine observation data according to claim 1, characterized in that, The first test data of the observation period are averaged to obtain the observed values, specifically including: Determine whether the number of first test values ​​marked as the first value in the first test data of the observation period is greater than 0; If so, calculate the average of the first test values ​​marked as the first value in the first test data of the first observation period to obtain the observed value, and set the observation mark as the first value; If not, then use the missing value placeholder as the observation value and set the observation label to the third value.

5. The quality control method for marine observation data according to claim 4, characterized in that, The observed values ​​are subjected to empirical range tests and interquartile range tests in sequence to obtain the second test value and the second test mark for the observation period, which specifically include: If the observation is marked as the first value, then it is determined whether the observed value is within the preset empirical range to perform an empirical range test and obtain the first judgment result. If the first judgment result is no, then the second test value of the observation period is the observed value, and the second test mark is the second value. If the first judgment result is yes, then it is determined whether the observed value is within the valid interval to perform an interquartile range test and obtain the second judgment result. If the second judgment result is yes, then the second test value of the observation period is the observed value, and the second test mark is the first value. If the second judgment result is no, then the second test value of the observation period is the observed value, and the second test mark is the fourth value. If the observation is marked as the third value, then the second test value of the observation period is the missing value placeholder, and the second test is marked as the third value.

6. The method for quality control of marine observation data according to claim 5, characterized in that, The effective interval is: ,in, The 25th percentile of the observations for all observation periods marked as the first value. This represents the percentile difference. , The 75th percentile of the observations for all observation periods marked as the first value.

7. The method for quality control of marine observation data according to claim 1, characterized in that, Determining whether the second test value of the observation period can be reasonably filled includes: Determine the first most recent valid value before the second test value of the observation period and the second most recent valid value after the second test value of the observation period; both the first most recent valid value and the second most recent valid value are second test values ​​marked as either the first value or the fourth value of the second test. Calculate the difference between the time corresponding to the first most recent valid value and the time corresponding to the second most recent valid value to obtain the length of the missing segment; If the length of the missing period is greater than the critical time length, the second test value of the observation period cannot be reasonably filled; otherwise, the second test value of the observation period can be reasonably filled, in order to determine whether the second test value of the observation period can be reasonably filled.

8. The method for quality control of marine observation data according to claim 7, characterized in that, The second test value of the observation period is filled by combining the local weighted regression scatter smoothing algorithm and the STL decomposition and reconstruction algorithm to obtain the filled value, which specifically includes: If the length of the missing period is less than or equal to the preset time length, the second test value of the observation period is filled using the local weighted regression scatter smoothing algorithm to obtain the filled value; If the length of the missing segment is greater than the preset time length, the second test value of the observation period is filled using the STL decomposition and reconstruction algorithm to obtain the filled value.

9. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that the processor executes the computer program to implement the quality control method for marine observation data according to any one of claims 1-8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the quality control method for marine observation data as described in any one of claims 1-8.

Images