A method for removing scallop effect from swath observation data

Abstract

The present invention discloses a method for removing scallop effects from swath observation data, comprising the following steps: collecting swath observation data with scallop effects in a target area; performing interpolation filling on missing values and discontinuous areas of the swath observation data; performing section-by-section analysis on the preprocessed swath observation data, extracting local maximum points and local minimum points on each section, and calculating the pixel spacing between adjacent local extreme values; counting the pixel spacing between adjacent local extreme values on all sections, and generating a histogram to present the number distribution of pixel periods; calculating a weighted average value based on the pixel periods counted in the histogram to determine the kernel size of a mean filter, wherein the kernel size covers complete periodic fluctuations; applying the designed mean filter to the swath observation data section by section, and repeating the process until all data are completely scalloped. The present invention provides an accurate denoising effect in a concise and efficient manner.

Description

Technical Field

[0001] The present invention relates to the technical field of marine data processing, and in particular to a method for removing scallop effects from swath observation data. Background Art

[0002] In the marine field, swath observation data is often affected by the scallop effect due to measurement equipment, seriously affecting subsequent data analysis and application. Currently, a variety of filtering algorithms are available to remove the scallop effect, including traditional mean filters, median filters, wavelet transform denoising, adaptive filters, and frequency domain filtering. However, while simple and easy to use, traditional mean filters are limited in their effectiveness in removing periodic noise. Furthermore, their fixed filter window size makes them ineffective for different types of noise. Consequently, when dealing with strong and distinct scallop effects, they often lead to oversmoothing of the data and loss of important signal details. While median filters can effectively handle local outliers such as salt and pepper noise, they are less effective in removing periodic noise such as scallop effects and may not be able to effectively distinguish between noise and valid signals. While wavelet transform denoising offers certain advantages in separating signal frequency components, its high computational complexity often makes it insufficient for real-time data processing. While adaptive filters can dynamically adjust parameters based on data characteristics, they rely on a large amount of sample data to accurately estimate noise characteristics and adjust filter parameters. Furthermore, they are primarily designed for random noise and struggle to effectively identify and remove strong periodic noise with stable frequency components, such as the scallop effect. While frequency-domain filtering can effectively suppress periodic noise, it can have a certain impact on non-periodic signals, leading to signal distortion or loss.

[0003] The presence of the scallop effect not only increases the noise component of swath observation data but can also cause signal distortion. Removing this unnecessary noise is particularly important when conducting precise environmental monitoring, measuring oceanographic quantities, or modeling data. However, existing methods for removing the scallop effect generally suffer from incomplete denoising, signal loss, and low computational efficiency. They are also poorly adaptable when dealing with complex, non-stationary, or periodic noise, and their high computational complexity can lead to performance bottlenecks when processing large-scale data. Summary of the Invention

[0004] Purpose of the invention: The purpose of the present invention is to provide a method for removing the scallop effect in cropped observation data, which solves the problems existing in the background technology by designing a mean filter with an appropriate kernel size to remove the "scallop effect" in cropped observation data.

[0005] Technical solution: The method for removing scallop effect from swath observation data described in the present invention comprises the following steps:

[0006] (1) Collect swath observation data of the scallop effect in the target area;

[0007] (2) Interpolation filling of missing values and discontinuous areas of swath observation data, including linear interpolation, nearest neighbor interpolation or spline interpolation;

[0008] (3) Analyze the pre-processed swath observation data section by section, extract the local maximum and local minimum points on each section, and calculate the pixel spacing between adjacent local extreme values;

[0009] (4) Count the pixel distances between adjacent local extreme values on all sections and generate a histogram to present the number distribution of pixel periods;

[0010] (5) According to the pixel period counted in the histogram, a weighted average is calculated to determine the kernel size of the mean filter, and the kernel size covers the complete periodic fluctuation;

[0011] (6) Apply the designed mean filter to the swath observation data section by section, and repeat the process until all the data have been de-scalloped.

[0012] Furthermore, in step (1), the scallop effect is caused by at least one of the following reasons: the scanning method of the sensor; the posture change of the measuring device; and the periodic error in the data processing process.

[0013] Furthermore, the interpolation method in step (2) is specifically as follows: selecting an interpolation algorithm according to the distribution characteristics of the data missing area; filling the discontinuous area point by point using linear interpolation or spline interpolation.

[0014] Furthermore, in step (3), the method for extracting local extreme values is as follows: define a neighborhood size k, compare the values of the k adjacent data points point by point, and determine that the extreme value is met when the following conditions are met: local maximum: the value of the current point is greater than the values of the k neighboring points on the left and right; local minimum: the value of the current point is less than the values of the k neighboring points on the left and right. The neighborhood size should be selected according to the data characteristics to ensure that local extreme values can be fully detected while reducing false detections caused by noise or local fluctuations.

[0015] Furthermore, in step (4), the histogram is generated by grouping the pixel periods into fixed intervals of width n pixels; counting the frequencies of local maxima and minima within each interval, and marking them with different colors. The width n must be reasonably selected to clearly reflect the interval location of the peak.

[0016] Furthermore, in step (5), the kernel size of the mean filter is determined by calculating the weighted average of the pixel periods in the histogram, using the formula:

[0017] ;

[0018] Where, It is The middle value of the interval, It is The weight of the interval, Represents the sum of all weights.

[0019] Furthermore, in step (6), the specific operation of the filtering process is: first, a mean filter is applied to the data points of each profile, and the calculation formula is:

[0020] ;

[0021] in, Is the filter window size, the window range is from arrive , is the original pixel value of position x in the swath observation profile data, is the position after processing by the mean filter The output pixel value of

[0022] Then the process is iterated section by section until all data are denoised.

[0023] The present invention provides a system for removing scallop effects from swath observation data, comprising:

[0024] Data acquisition module: used to collect swath observation data of scallop effect in the target area;

[0025] Data preprocessing module: used to interpolate and fill missing values and discontinuous areas of swath observation data. Interpolation methods include linear interpolation, nearest neighbor interpolation or spline interpolation.

[0026] Profile analysis module: used to analyze the pre-processed swath observation data profile by profile, extract the local maximum and local minimum points on each profile, and calculate the pixel spacing between adjacent local extreme values;

[0027] Period statistics module: used to count the pixel distances between adjacent local extreme values on all sections and generate a histogram to present the number distribution of pixel periods;

[0028] Filter design module: used to calculate the weighted average value based on the pixel period counted in the histogram to determine the kernel size of the mean filter; the kernel size covers the complete periodic fluctuation;

[0029] Filter processing module: used to apply the designed mean filter to the swath observation data section by section, and repeat the process until all data have completed the scallop effect removal.

[0030] An electronic device described in the present invention includes a memory, a processor, and a computer program stored in the memory, and the processor implements the steps of any one of the methods when executing the program.

[0031] The computer-readable storage medium of the present invention stores a computer program, which implements the steps of any one of the methods when executed by a processor.

[0032] Beneficial Effects: Compared with existing technologies, this invention offers the following significant advantages: By designing a mean filter with an appropriately sized kernel, combined with profile analysis to extract local maxima and minima, it accurately identifies periodic noise and dynamically adjusts the filter window size based on the pixel spacing between adjacent local extremes, effectively removing periodic noise while minimizing its impact on valid signals. Compared to traditional methods, this invention offers greater adaptability and avoids the problem of excessive computational complexity, providing a concise, efficient, and precise solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is a flow chart of the present invention;

[0034] Figure 2 This is a process diagram for removing the scallop effect of swath observation data of the present invention; wherein, Figure 2 (a) in the figure shows the cross-section of the swath observation data. Figure 2 (b) in the figure is a histogram of the pixel periods of local maximum and local minimum values. DETAILED DESCRIPTION

[0035] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0036] like Figure 1 As shown, an embodiment of the present invention provides a method for removing scallop effects for swath observation data, comprising the following steps:

[0037] Step 1: Collect swath observation data within the target area. The following uses the target area (36.5°N-37°N and 124.4°E-125.4°E) as an example to further illustrate the implementation of the method provided by the present invention. This embodiment of the present invention utilizes temperature data from the Sub-Mesoscale Ocean Dynamics Experiment (S-MODE) conducted in the waters off central California, using the Multiscale Ocean Surface Observing System (MOSES), an optical instrument carried on an aircraft, to collect sea surface temperature (SST) data with a resolution of several meters. This space-based observation system uses infrared cameras to record high-resolution SST and stitches together individual images to provide a swath covering approximately 200 kilometers. Specifically, the SST data collected from the airborne MOSES between 19:50 and 23:07 UTC on October 24, 2022, comprised 12 swaths. The original data was non-gridded at 9 meters resolution, which was then resampled to gridded data at 500 meters resolution. In the sea temperature field along the latitude, obvious periodic fluctuations can be observed, a phenomenon known as the scallop effect.

[0038] Step 2: Preprocess the swath observation data. The specific process of swath observation data preprocessing is as follows: In this implementation case, the swath observation data has gaps caused by discontinuous scanning of the equipment during data acquisition. The discontinuous areas of the data are filled using the linear interpolation method. The linear interpolation formula is as follows:

[0039] (1);

[0040] in, and are two known data points, represents the estimated interpolation points, corresponds to The interpolation result of Value. Figure 2 As shown in (a), the black curve shows an SST profile after gap filling, clearly showing the periodic fluctuation changes caused by the airborne measurement equipment.

[0041] Step 3: Perform profile analysis on the swath observation data and extract the local maximum and minimum values on the profile. The specific process of extracting the local maximum and local minimum values of the swath observation profile data is as follows: Assume that the swath observation profile data is a given data sequence , for each data point , if and only if it is larger than its adjacent data points, it is called a local maximum, which can be defined by the following inequality: Similarly, if and only if it is larger than its adjacent If the number of data points is small, it is called a local minimum, which can be defined by the following inequality: .in Is the size of the field to be set, here =3 means only three adjacent points are compared.

[0042] Step 4: Count the number of pixel cycles on all sections of the swath observation data. Count the number of pixel cycles on all sections of the swath observation data as follows: Figure 2 As shown in (b), the distribution of the number of pixel periods on all sections is counted in the form of a histogram. The obtained pixel periods are divided into several intervals with a width of 2. The frequency of pixel periods in each interval is calculated. The peak of the pixel period frequency with the local maximum in the histogram occurs in the interval [9, 11]. Blue represents the local maximum, and orange represents the local minimum.

[0043] Step 5: Design a mean filter with an appropriate kernel size. The filter kernel size is determined by the pixel spacing between adjacent local extreme values to ensure that a complete cycle fluctuation is covered. Specifically: Design a mean filter with an appropriate kernel size to remove the scallop effect in the cropped observation data. Calculate the weighted average value based on the pixel period of adjacent local extreme values counted by the histogram to determine the optimal filter window size. The window size of the mean filter is determined by the pixel distance between adjacent local maximum points and adjacent local minimum points. The weighted average value is calculated based on the pixel period counted by the histogram, and the weight is weighted according to the frequency within the interval. Weighted average The calculation formula is as follows:

[0044] (2);

[0045] Where, It is The middle value of the interval, It is The weight of the interval, Represents the sum of all weights.

[0046] Step 6: Apply a mean filter to each swath observation profile data. The specific process of applying a mean filter to swath observation profile data is as follows: select an appropriate kernel size; for each pixel in the swath observation profile data, average the values of all pixels in the neighborhood surrounding the pixel; and replace the value of the center pixel with this average value. The calculation formula is as follows:

[0047] (3);

[0048] in, Is the filter window size, the window range is from arrive , is the original pixel value of position x in the swath observation profile data, is the position after processing by the mean filter The output pixel value of

[0049] The weighted average values of the pixel periods of the local maximum and local minimum values calculated by formula (2) are both approximately 12, corresponding to the position of the pixel period peak in the histogram (e.g. Figure 2 The final kernel size selected is 12, which corresponds to a spatial distance of about 6 km. This is basically consistent with the width measured by the MOSES system during the S-MODE mission, confirming the reliability of the filtering process. Figure 2 As shown in (a) of the figure, the red curve represents the SST profile after applying this mean filter, showing a more continuous and uniform variation. This demonstrates that this method effectively reduces the scalloping effect. Ultimately, systematically applying this filtering method to each zonal profile can produce a smoother SST field, removing periodic signals.

Claims

1. A method for removing scallop effects from swath observation data, characterized in that: The following steps are involved: (1) Collect swath observation data of the scallop effect in the target area; (2) Interpolation filling of missing values and discontinuous areas of swath observation data, including linear interpolation, nearest neighbor interpolation or spline interpolation; (3) Analyze the preprocessed swath observation data section by section, extract the local maximum and local minimum points on each section, and calculate the pixel spacing between adjacent local extreme values; (4) Count the pixel distances between adjacent local extreme values on all sections and generate a histogram to present the number distribution of pixel periods; (5) According to the pixel period counted in the histogram, a weighted average is calculated to determine the kernel size of the mean filter. The kernel size covers the complete periodic fluctuation. (6) Apply the designed mean filter to the swath observation data section by section, and repeat the process until all data have completed the scalloping effect. The specific operation of the filtering process is: first, apply the mean filter to the data points of each section, and the calculation formula is: ； in, Is the filter window size, the window range is from arrive , is the position in the swath observation profile data The original pixel value, is the position after processing by the mean filter The output pixel value of is obtained; then the process is iterated section by section until all data are denoised.

2. The method for removing scallop effect of swath observation data according to claim 1, wherein: In step (1), the scallop effect is caused by at least one of the following reasons: the scanning method of the sensor; the posture change of the measuring device; and the periodic error in the data processing process.

3. The method for removing scallop effect of swath observation data according to claim 1, wherein: The interpolation method in step (2) is as follows: select an interpolation algorithm based on the distribution characteristics of the data missing area; fill the discontinuous area point by point using linear interpolation or spline interpolation.

4. The method for removing scallop effect of swath observation data according to claim 1, wherein: In step (3), the method for extracting local extreme values is as follows: define the neighborhood size k, compare the values of the k adjacent data points point by point; determine it as an extreme value when the following conditions are met: local maximum: the value of the current point is greater than the values of the k neighboring points on the left and right; local minimum: the value of the current point is less than the values of the k neighboring points on the left and right; among them, the neighborhood size should be selected according to the data characteristics to ensure that the local extreme values can be fully detected while reducing false detections caused by noise or local fluctuations.

5. The method for removing scallop effect of swath observation data according to claim 1, wherein: In step (4), the histogram is generated by grouping the pixel periods into fixed intervals of width n pixels; counting the frequencies of local maximum and minimum values in each interval and marking them with different colors; the width n needs to be reasonably selected to clearly reflect the interval position of the peak.

6. The method for removing scallop effect of swath observation data according to claim 1, characterized in that: In step (5), the filter kernel size is determined by the pixel spacing between adjacent local extreme values to ensure that a complete periodic fluctuation is covered; the mean filter kernel size is determined by calculating the weighted average of the pixel periods in the histogram, as follows: ； Where, It is The middle value of the interval, It is The weight of the interval, Represents the sum of all weights.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 6 are implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images