A method for quality control of wind and wave element data based on single classifier

By using a single classifier-based approach and the theory of wind-wave invariants to construct a quality control model, the problems of complex programming and low computational efficiency in existing technologies are solved, and efficient and accurate outlier detection of wind and wave element data is achieved.

CN112464139BActive Publication Date: 2024-11-15NAT MARINE ENVIRONMENTAL FORECASTING CENT
0 Cites 0 Cited by

Patent Information

Application Number
CN202011367107.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-11-27
Publication Date
2024-11-15
Estimated Expiration
2040-11-27

Smart Images

  • Figure CN112464139B_ABST
    Figure CN112464139B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of marine observation data technology, and in particular to a single classifier-based wind and wave element data quality control method, comprising a wind and wave element sensor and a buoy, wherein the buoy and the wind and wave element sensor measure wind and wave observation data, compare the wind and wave action invariant theoretical curve, and judge the quality of the data by the curve deviation between the two. The method is based on the wind and wave action invariant theory as the basic theoretical basis, and uses a single classifier algorithm to simulate the wind and wave action relationship by a computer and judge the quality of the data. In the application of verified data, the abnormal data can be pointed out by the data user, that is, manual correction, and the manually corrected data can be iteratively trained by the method to obtain a new quality control model, thereby improving the accuracy of quality control.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of ocean observation data, and in particular to a method for controlling the quality of wind and wave element data based on a single classifier. Background Art

[0002] Due to the harsh working environment of ocean observation instruments, there are usually outliers in the data measured. Observation data quality control and elimination of outliers are basic links in data applications. Business applications require accuracy and efficiency in the quality control link. The accuracy requirement is to prevent data contaminated by abnormal data from misleading users or contaminating subsequent derived data. Efficiency is required to ensure that data quality control processing has a strong application timeliness.

[0003] The existing marine observation data quality control method uses the 3σ method and Grubbs method commonly used in data quality control as the criterion, and uses the wind and wave parameter mutual exclusion method for correlation test. This method mainly checks whether there are outliers in the wind and wave element data from the perspectives of data completeness and instrument working status. The existing marine observation data quality control method is a general data quality control method. From the perspective of implementation principle, it mainly focuses on the judgment of instrument abnormal properties. The wind and wave element correlation criterion is a mutually exclusive criterion, which does not effectively utilize the invariance relationship of wind-wave interaction. In terms of programming implementation and computational efficiency, the existing methods have the problems of complex programming and low computational efficiency. Therefore, it is urgent to design a wind and wave element data quality control method based on a single classifier for the current marine observation data technology field. Summary of the invention

[0004] The invention provides a wind and wave element data quality control method based on a single classifier to solve the problems existing in the prior art.

[0005] In order to achieve the above purpose, the embodiment of the present invention provides the following technical solutions:

[0006] According to an embodiment of the present invention, a method for controlling the quality of wind and wave element data based on a single classifier includes a wind and wave element sensor and a buoy. The buoy and the wind and wave element sensor measure the observation data of wind and waves, compare the wind and wave observation data with the theoretical curve of the wind-wave action invariant, and judge the quality of the data by the curve deviation between the two. The observation data show that the following invariance relationship exists between the wind and waves on the sea surface:

[0007]

[0008] in and are dimensionless wave height and period, respectively, and α is an empirical coefficient, specifically 0.062;

[0009] in g、H s and U * are gravity acceleration, effective wave height and friction wind speed respectively; T s is the effective period;

[0010] The sea-air interface process is in a local equilibrium state. It has good invariance. The above invariance relationship is the distribution position of normal wind and wave data points, that is, the observed wind and wave data points should be evenly and closely distributed near the theoretical curve. Otherwise, it violates the wind-wave invariance relationship and can be judged as an abnormal value. Since there are few friction wind speeds U in actual wind speed observations in my country, * Data, but mostly for 10m high wind speed U 10 The data, and the wind speed U h with U * It is a nonlinear positive correlation, which can be written as:

[0011] U * =f(h)·U h (2)

[0012] Where h is the wind speed observation height, and the positive correlation in formula (2) will vary due to different geographical factors of each measuring point. Therefore, α in formula (1) will vary with the wind speed observation height and the geographical conditions of the measuring point. Let α = g(h), that is:

[0013]

[0014] This method logarithmizes and dimensionless the historical wind-wave data of the specified observation point. In the training stage, the theoretical curve is replaced by a single classification model. The hypersphere in N-dimensional space is assumed to have a radius of R and a center of a. The sample points in the control are {xi, i = 1, 2, ..., M}. We can obtain:

[0015] (x i -a)(x i -a) T ≤R 2 (4)

[0016] Since we want to minimize the radius of the hypersphere, we can obtain from equation (4):

[0017] F(R,a)=R 2 (5)

[0018] And make its radius as small as possible;

[0019] Introduce the Lagrangian operator and construct a Lagrangian function:

[0020]

[0021] The Lagrangian operator α i ≥0, take the partial derivative of (6) and set its derivative to 0, and we can get the following conditions:

[0022]

[0023]

[0024] Substituting (7) and (8) into (6) yields:

[0025]

[0026] At this point, we can use the quadratic programming algorithm to obtain the optimal α i Minimize (9). In this case, when there are many test data points, the performance of the spatial region description will suffer a great loss. Therefore, we introduce the relaxation factor ε i , change (4) to:

[0027] (x i -a)(x i -a) T ≤R 2 +ε i

[0028] ε i ≥0 (10)

[0029] Introducing a constant C as a penalty coefficient can control the probability of false acceptance and false rejection of the hypersphere:

[0030]

[0031] Construct the Lagrangian function:

[0032]

[0033] Taking partial derivative and setting the derivative to 0, we get:

[0034]

[0035]

[0036] Substituting (13) back into (12), we obtain:

[0037]

[0038] The optimal α is obtained by quadratic programming algorithm i Minimize the above formula to meet the purpose of minimum radius, thus obtaining the quality control model.

[0039] During quality control, the logarithmic, dimensionless test data (Z) and the quality control model are tested using the following formula:

[0040]

[0041] If the data to be verified satisfies the above formula, the data is accepted; otherwise, the data is questioned, thereby generating verified data. The verified data can then be manually corrected, and the final model can be updated. The updated model can be used to perform quality control on subsequent data to be verified.

[0042] Furthermore, based on the theory of the invariance relationship, outliers in the wind and wave element observation data can be effectively identified through a single classifier algorithm.

[0043] Furthermore, the friction wind speed U * Data is relatively rare in actual wind speed observations, and is often obtained using the 10-meter high wind speed U 10 The data is used for alternative calculation, and the wind speed U at h meters h with U * The nonlinear positive correlation is as follows:

[0044] U * =f(h)·U h

[0045] Where h is the wind speed observation height, and U * =f(h)·U h The positive correlation will vary due to different geographical factors of each measuring point. Therefore, α in claim 1 will vary with the wind speed observation height and the geographical conditions of the measuring point. Let α = g(h), that is:

[0046]

[0047] By learning the historical wind-wave interaction relationship of a specified observation point based on a one-class support vector machine algorithm, the wind-wave interaction relationship of the specified observation point can be simulated from a statistical point of view, thereby establishing a quality control model.

[0048] The present invention has the following advantages:

[0049] The single classifier-based wind and wave element data quality control method is based on the wind-wave action invariant theory, and the deviation between the wind and wave observation data and the wind-wave action invariant theoretical curve is judged by a machine learning algorithm to judge the quality of the data. Since the wind and wave elements of the meteorological data buoy are measured by an independent automatic meteorological station and a wave measurement module, any abnormal working state of any device or abnormal observation result can be detected by this method. The method of the present invention is more in line with the physical law of wind-wave interaction and is simpler to implement through programming. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0051] The structures, proportions, sizes, etc. illustrated in this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with the technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantial technical significance. Any structural modification, change in proportion or adjustment of size shall still fall within the scope of the technical contents disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention.

[0052] Figure 1 It is a schematic diagram of the processing flow of the present invention;

[0053] Figure 2 This is a schematic diagram of the relationship between the logarithmic dimensionless effective period and wave height of the test data of the present invention; DETAILED DESCRIPTION

[0054] The following is a description of the implementation of the present invention by specific embodiments. People familiar with the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] The terms such as "upper", "lower", "left", "right", "middle", etc. cited in this specification are only for the convenience of description and are not intended to limit the scope of implementation of the present invention. Changes or adjustments to their relative relationships should be regarded as within the scope of implementation of the present invention without substantially changing the technical content.

[0056] The present invention provides a technical solution:

[0057] A single classifier-based wind and wave element data quality control method includes a wind and wave element sensor and a buoy. The buoy and the wind and wave element sensor measure wind and wave observation data, compare the wind and wave observation data with a wind-wave action invariant theoretical curve, and judge the quality of the data by the curve deviation between the two. The following invariant relationship exists between sea surface wind and waves as shown by observation data:

[0058]

[0059] in and are dimensionless wave height and period, respectively, and α is an empirical coefficient, specifically 0.062;

[0060] in g、H s and U * are gravity acceleration, effective wave height and friction wind speed respectively; T s is the effective period;

[0061] The sea-air interface process is in a local equilibrium state. It has good invariance. The above invariance relationship is the distribution position of normal wind and wave data points, that is, the observed wind and wave data points should be evenly and closely distributed near the theoretical curve. Otherwise, it violates the wind-wave invariance relationship and can be judged as an abnormal value. Since there are few friction wind speeds U in actual wind speed observations in my country, * Data, but mostly for 10m high wind speed U 10 The data, and the wind speed U h with U * It is a nonlinear positive correlation, which can be written as:

[0062] U * =f(h)·U h (2)

[0063] Where h is the wind speed observation height, and the positive correlation in formula (2) will vary due to different geographical factors of each measuring point. Therefore, α in formula (1) will vary with the wind speed observation height and the geographical conditions of the measuring point. Let α = g(h), that is:

[0064]

[0065] This method logarithmizes and dimensionless the historical wind-wave data of the specified observation point. In the training stage, the theoretical curve is replaced by a single classification model. The hypersphere in N-dimensional space is assumed to have a radius of R and a center of a. The sample points in the control are {xi, i = 1, 2, ..., M}. We can obtain:

[0066] (x i -a)(x i -a) T ≤R 2 (4)

[0067] Since we want to minimize the radius of the hypersphere, we can obtain from equation (4):

[0068] F(R,a)=R 2 (5)

[0069] And make its radius as small as possible;

[0070] Introduce the Lagrangian operator and construct a Lagrangian function:

[0071]

[0072] The Lagrangian operator α i ≥0, take the partial derivative of (6) and set its derivative to 0, and we can get the following conditions:

[0073]

[0074]

[0075] Substituting (7) and (8) into (6) yields:

[0076]

[0077] At this point, we can use the quadratic programming algorithm to obtain the optimal α i Minimize (9). In this case, when there are many test data points, the performance of the spatial region description will suffer a great loss. Therefore, we introduce the relaxation factor ε i , change (4) to:

[0078] (x i -a)(x i -a) T ≤R 2 +ε i

[0079] ε i ≥0 (10)

[0080] Introducing a constant C as a penalty coefficient can control the probability of false acceptance and false rejection of the hypersphere:

[0081]

[0082] Construct the Lagrangian function:

[0083]

[0084] Taking partial derivative and setting the derivative to 0, we get:

[0085]

[0086]

[0087] Substituting (13) back into (12), we get:

[0088]

[0089] The optimal α is obtained by quadratic programming algorithm i Minimize the above formula to meet the purpose of minimum radius, thus obtaining the quality control model.

[0090] During quality control, the logarithmic, dimensionless test data (Z) and the quality control model are tested using the following formula:

[0091]

[0092] If the data to be verified satisfies the above formula, the data is accepted; otherwise, the data is questioned, thereby generating verified data. The verified data can then be manually corrected, and the final model can be updated. The updated model can be used to perform quality control on subsequent data to be verified.

[0093] In the present invention, the outliers in the wind and wave element observation data can be effectively identified by a single classifier algorithm based on the theory of the invariance relationship.

[0094] In the present invention: the friction wind speed U * Data is relatively rare in actual wind speed observations, and is often obtained using the 10-meter high wind speed U 10 The data is used for alternative calculation, and the wind speed U at h meters h with U * The nonlinear positive correlation is as follows:

[0095] U * =f(h)·U h

[0096] Where h is the wind speed observation height, and U * =f(h)·U h The positive correlation will vary due to different geographical factors of each measuring point. Therefore, α in claim 1 will vary with the wind speed observation height and the geographical conditions of the measuring point. Let α = g(h), that is:

[0097]

[0098] By learning the historical wind-wave interaction relationship of a specified observation point based on a one-class support vector machine algorithm, the wind-wave interaction relationship of the specified observation point can be simulated from a statistical point of view, thereby establishing a quality control model.

[0099] Although the present invention has been described in detail above by general description and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.

Claims

1. A method for controlling the quality of wind and wave element data based on a single classifier, comprising a wind and wave element sensor and a buoy, characterized in that: The buoy and wind and wave element sensors measure wind and wave observation data, compare them with the wind-wave action invariant theoretical curve, and judge the quality of the data by the curve deviation between the two. Among them, the observation data show that the following invariance relationship exists between the sea surface wind and waves: in and are dimensionless wave height and period, respectively, and α is an empirical coefficient, specifically 0.062; in g、H s and U * are gravity acceleration, effective wave height and friction wind speed respectively; T s is the effective period; The sea-air interface process is in a local equilibrium state. It has good invariance. The above invariance relationship is the distribution position of normal wind and wave data points, that is, the observed wind and wave data points should be evenly and closely distributed near the theoretical curve. Otherwise, it violates the wind-wave invariance relationship and can be judged as an abnormal value. Since there are few friction wind speeds U in actual wind speed observations in my country, * Data, but mostly for 10m high wind speed U 10 Data, and h meters high wind speed U h with U * It is a nonlinear positive correlation, which can be written as: IN * =f(h)·U h (2) Where h is the wind speed observation height, and the positive correlation in formula (2) will vary due to different geographical factors of each measuring point. Therefore, α in formula (1) will vary with the wind speed observation height and the geographical conditions of the measuring point. Let α = g(h), that is: The historical wind-wave data of the specified observation point are logarithmized and dimensionless. In the training stage, the theoretical curve is replaced by a single classification model. Assume that the hypersphere in N-dimensional space has a radius of R and a center of a. The sample points in this space are {xi, i=1,2,…,M}. We can get: (x i -a)(x i -a) T ≤R 2 (4) Since we want to minimize the radius of the hypersphere, we can obtain from equation (4): F(R,a)=R 2 (5) And make its radius as small as possible; Introduce the Lagrangian operator and construct a Lagrangian function: The Lagrangian operator α i ≥0, take the partial derivative of (6) and set its derivative to 0, and we can get the following conditions: Substituting (7) and (8) into (6) we can obtain: At this point, we can use the quadratic programming algorithm to obtain the optimal α i Minimize (9). In this case, when there are many test data points, the performance of the spatial region description will suffer a great loss. Therefore, we introduce the relaxation factor ε i , change (4) to: (x i -a)(x i -a) T ≤R 2 +e i e i ≥0 (10) Introducing a constant C as a penalty coefficient can control the probability of false acceptance and false rejection of the hypersphere: Construct the Lagrangian function: Taking partial derivative and setting the derivative to 0, we get: 0≤α i ≤C (13) Substituting (13) back into (12), we get: The optimal α is obtained by quadratic programming algorithm i Minimize the above formula to meet the purpose of minimum radius, thus obtaining the quality control model; During quality control, the logarithmic, dimensionless test data (Z) and the quality control model are tested using the following formula: If the data to be verified meets the above formula, the data is accepted; otherwise, the data is questioned, thereby generating verified data. Afterwards, the verified data can be manually corrected, and the final model is updated. The updated model is used to perform quality control on subsequent data to be verified. Based on the theory of the invariance relationship, the outliers in the wind and wave element observation data can be effectively identified by a single classifier algorithm.