Scallop removal effect method for swath observation data

Abstract

The invention discloses a scallop effect removing method for cradling observation data. The method comprises the following steps: collecting the cradling observation data with a scallop effect in a target area; interpolation filling of missing values and discontinuous areas is carried out on the swath observation data; performing section-by-section analysis on the preprocessed swath observation data, extracting a local maximum value point and a local minimum value point on each section, and calculating a pixel pitch between adjacent local extreme values; counting pixel spacing of adjacent local extreme values on all sections, and generating a histogram to present number distribution of pixel periods; according to a pixel period counted in the histogram, a weighted average is calculated to determine the kernel size of a mean filter, and the kernel size covers complete periodic fluctuation; applying the designed mean value filter to the swath observation data section by section, and repeatedly processing until all the data complete the scallop removal effect; according to the invention, an accurate denoising effect is provided in a simple and efficient manner.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing in the marine field, and particularly relates to a method for removing scallop effects from swath observation data. Background Art

[0002] In the marine field, swath observation data is often interfered by scallop effects due to measurement devices, seriously affecting the subsequent analysis and application of the data. Currently, there are various filtering algorithms for removing scallop effects, such as traditional mean filters, median filters, wavelet transform denoising, adaptive filters, and frequency domain filtering. However, although the traditional mean filter is simple and easy to use, its effect on removing periodic noise is limited, and the size of the filtering window is fixed, unable to effectively adjust for different types of noise. Therefore, when dealing with strong periodic and obvious scallop effects, it often leads to over-smoothing of the data and loss of important details in the signal. The median filter can handle local outliers such as salt-and-pepper noise well, but its effect on removing periodic noise like scallop effects is poor, and it may not be able to effectively distinguish between noise and valid signals. Although wavelet transform denoising has certain advantages in separating signal frequency components, the computational complexity of this method is relatively high and often cannot meet the requirements in real-time data processing. Although the adaptive filter can dynamically adjust parameters according to data characteristics, it relies on a large amount of sample data to accurately estimate noise characteristics and adjust filter parameters, and it is mainly designed for random noise and is difficult to effectively identify and remove strong periodic noise such as scallop effects with stable frequency components. The frequency domain filtering method can effectively suppress periodic noise, but it will have a certain impact on non-periodic signals, resulting in signal distortion or loss.

[0003] The existence of scallop effects not only increases the noise components of swath observation data but also may lead to signal distortion. Especially when precise environmental monitoring, marine physical quantity measurement, or data modeling is required, removing these unnecessary noises is particularly important. However, the existing methods for removing scallop effects generally have problems such as incomplete denoising effect, signal loss, or low computational efficiency, poor adaptability when dealing with complex, non-stationary, or periodic noises, and relatively high computational complexity, which may lead to performance bottlenecks when processing large-scale data. Summary of the Invention

[0004] Object of the Invention: The object of the present invention is to provide a method for removing scallop effects from swath observation data, which realizes the method of removing "scallop effects" from swath observation data by designing a mean filter with an appropriate kernel size, and solves the problems existing in the background art.

[0005] Technical Solution: A method for removing scallop effects from swath observation data according to the present invention includes the following steps: (1)Collect the swath observation data with scalloping effect in the target area; (2)Interpolate and fill the missing values and discontinuous regions in the swath observation data. The interpolation methods include linear interpolation, nearest-neighbor interpolation or spline interpolation; (3)Perform cross-sectional analysis on the preprocessed swath observation data, extract the local maximum points and local minimum points on each cross-section, and calculate the pixel spacing between adjacent local extrema; (4)Statistically analyze the pixel spacing between adjacent local extrema on all cross-sections, and generate a histogram to present the quantity distribution of pixel periods; (5)Calculate the weighted average value according to the pixel periods statistically analyzed in the histogram to determine the kernel size of the mean filter, and the kernel size covers the complete periodic fluctuations; (6)Apply the designed mean filter to the swath observation data cross-section by cross-section, and repeat the process until all data are free of scalloping effect.

[0006] Further, in step (1), the scalloping effect is caused by at least one of the following reasons: the scanning mode of the sensor; the attitude change of the measuring device; the periodic error in the data processing process.

[0007] Further, the interpolation method in step (2) is specifically: select the interpolation algorithm according to the distribution characteristics of the data missing area; perform point-by-point filling on the discontinuous area, and use linear interpolation or spline interpolation.

[0008] Further, in step (3), the method for extracting local extrema is: define the neighborhood size k, and compare the values of adjacent k data points point by point; it is determined as an extremum when the following conditions are met: local maximum: the value of the current point is greater than the values of the k neighborhood points on both the left and right; local minimum: the value of the current point is less than the values of the k neighborhood points on both the left and right. Among them, the neighborhood size should be selected according to the data characteristics to ensure that local extrema can be fully detected while reducing false detections caused by noise or local fluctuations.

[0009] Further, in step (4), the method for generating the histogram is: group the pixel periods according to a fixed interval width of n pixels; statistically analyze the frequencies of local maxima and minima in each interval, and mark them with different colors. Among them, the width n needs to be reasonably selected to clearly reflect the interval position of the peak.

[0010] Further, in step (5), the method for determining the kernel size of the mean filter is: calculate the weighted average value of the pixel periods in the histogram, and the formula is: ; where, is the median value of the th interval, is the The weight of an interval, represents the sum of all weights.

[0011] Furthermore, in step (6), the specific operation of the filtering process is as follows: First, apply a mean filter to each data point of each profile. The calculation formula is: ; where is the filtering window size, and the range of the window is from to , is the original pixel value at position x in the swath observation profile data, is the output pixel value at position after being processed by the mean filter; Then, perform iterative processing on each profile until all data is denoised.

[0012] A de-scalloping effect system for swath observation data according to the present invention includes: Data acquisition module: used to acquire swath observation data with scalloping effect in the target area; Data preprocessing module: used to perform interpolation filling on missing values and discontinuous regions in the swath observation data. The interpolation methods include linear interpolation, nearest neighbor interpolation, or spline interpolation; Profile analysis module: used to perform per-profile analysis on the preprocessed swath observation data, extract local maximum points and local minimum points on each profile, and calculate the pixel spacing between adjacent local extrema; Period statistics module: used to count the pixel spacings between adjacent local extrema on all profiles and generate a histogram to present the quantity distribution of pixel periods; Filter design module: used to calculate the weighted average value according to the pixel periods statistically counted in the histogram to determine the kernel size of the mean filter; the kernel size covers the complete periodic fluctuation; Filtering processing module: used to apply the designed mean filter to the swath observation data on a per-profile basis and repeat the processing until all data is de-scalloped.

[0013] An electronic device according to the present invention includes a memory, a processor, and a computer program stored on the memory. When the processor executes the program, the steps of any one of the methods are implemented.

[0014] A computer-readable storage medium according to the present invention stores a computer program. When the program is executed by a processor, the steps of any one of the methods are implemented.

[0015] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: the present invention designs a mean filter with a suitable kernel size, extracts local maximum and minimum values ​​in combination with profile analysis, accurately identifies periodic noise, and dynamically adjusts the filter window size according to the pixel spacing of adjacent local extreme values, ensuring effective removal of periodic noise while reducing the impact on valid signals. Compared with traditional methods, the present invention has stronger adaptability and avoids the problem of excessive computational complexity, providing a simple, efficient and accurate solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flow chart of the present invention; Figure 2 This is a process diagram of removing scallop effect from swath observation data of the present invention; wherein, Figure 2 (a) in the figure shows the cross-section of the swath observation data. Figure 2 (b) in the figure is a histogram of the pixel periods of the local maximum and local minimum. DETAILED DESCRIPTION

[0017] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0018] like Figure 1 As shown, an embodiment of the present invention provides a method for removing scallop effects for swath observation data, comprising the following steps: Step 1: Collect swath observation data in the target area; The following is a further description of the implementation of the method provided by the present invention, taking the target area's longitude and latitude of 36.5°N-37°N and 124.4°E-125.4°E as an example; The embodiment of the present invention uses the temperature data of the Sub-Mesoscale Ocean Dynamics Experiment (S-MODE) conducted in the central waters of California, and uses an optical instrument on board the aircraft, the Ocean Surface Multi-Scale Observation System (MOSES), to collect sea surface temperature (SST) data with a resolution of several meters. This space observation system uses an infrared camera to record high-resolution sea surface temperature and stitches individual images together to provide a swath covering about 200 kilometers. Specifically, the sea surface temperature measured by the airborne MOSES from 19:50 to 23:07 UTC on October 24, 2022, is collected, and a total of 12 swaths are included. The original data is non-gridded data with a resolution of 9 meters, which is converted to grid data with a resolution of 500 meters after resampling. In the sea temperature field along the latitude, obvious periodic fluctuations can be observed, a phenomenon known as the scallop effect.

[0019] Step 2: Perform data preprocessing on the swath observation data; the specific process of preprocessing the swath observation data is as follows: In this embodiment, there are gaps in the swath observation data caused by discontinuous device scanning during the data acquisition process. For the discontinuous regions of the data, the linear interpolation method is used for filling. The linear interpolation formula is as follows: (1); Wherein, and are two known data points, represents the estimated interpolation point, is the interpolation result corresponding to , representing the estimated value. As shown in (a) of Figure 2 , the black curve shows an SST profile after gap filling, clearly presenting the periodic fluctuation changes caused by the airborne measurement equipment.

[0020] Step 3: Perform profile analysis on the swath observation data to extract local maxima and minima on the profile; the specific process of extracting local maxima and local minima of the swath observation profile data is as follows: Assume that the swath observation profile data is a given data sequence , for each data point , if and only if it is greater than its adjacent data points, then it is called a local maximum, and the local maximum can be defined by the following inequality: . Similarly, if and only if it is smaller than its adjacent data points, then it is called a local minimum, and the local minimum can be defined by the following inequality: . Where is the set domain size, here = 3 means only comparing the adjacent three points.

[0021] Step 4: Statistically analyze the number distribution of pixel cycles on all profiles of the swath observation data. The specific process of statistically analyzing the number distribution of pixel cycles on all profiles of the swath observation data is as follows: As shown in (b) of Figure 2 , statistically analyze the number distribution of pixel cycles on all profiles in the form of a histogram. Divide the obtained pixel cycles into several intervals with a width of 2, calculate the frequency of pixel cycles in each interval, and the peak of the frequency of pixel cycles of local maximum and minimum values in the histogram appears in the interval [9, 11]. Among them, blue represents local maximum, and orange represents local minimum.

[0022] Step 5: Design a mean filter with an appropriate kernel size, where the filter kernel size is determined by the pixel spacing between adjacent local extrema to ensure covering a complete periodic fluctuation. Specifically: Design a mean filter with an appropriate kernel size to remove the scalloping effect in the swath observation data. Calculate the weighted average based on the pixel period of adjacent local extrema statistically analyzed by the histogram to determine the optimal filter window size. The window size of the mean filter is determined by the pixel distance between adjacent local maximum points and adjacent local minimum points. Calculate the weighted average through the pixel period statistically analyzed by the histogram, and the weights are weighted according to the frequency within the interval. The weighted average is calculated by the following formula: (2); In the formula, is the median value of the th interval, is the weight of the th interval, represents the sum of all weights.

[0023] Step 6: Apply the mean filter to filter each swath observation profile data. The specific process of applying the mean filter to the swath observation profile data is as follows: Select an appropriate kernel size; for each pixel in the swath observation profile data, average the values of all pixels within the surrounding neighborhood of this pixel; replace the value of the central pixel with this average value. The calculation formula is as follows: (3); where, is the filter window size, and the range of the window is from to , is the original pixel value at position x in the swath observation profile data, is the output pixel value at position after being processed by the mean filter; The weighted average of the pixel periods of the local maximum and local minimum calculated by formula (2) is approximately 12 respectively, corresponding to the position of the pixel period peak in the histogram (as shown in (b) of Figure 2 ). The finally selected kernel size of 12 corresponds to a spatial distance of approximately 6 km, which is basically consistent with the swath width measured by the MOSES system during the S-MODE mission, confirming the reliability of the filtering process. As shown in (a) of Figure 2 , the red curve represents the SST profile after applying this mean filter, showing a more continuous and uniform change. This indicates that this method effectively reduces the scalloping effect. Finally, systematically applying this filtering method to each zonal profile can obtain a sea surface temperature field that removes periodic signals and is smoother.

Claims

1. A method for removing scallop effect from swath observation data, characterized in that, Including the following steps: (1) Collect the swath observation data with scalloping effect in the target area; (2) Interpolate and fill the missing values and discontinuous areas in the swath observation data. The interpolation methods include linear interpolation, nearest neighbor interpolation or spline interpolation; (3) Perform cross-sectional analysis on the preprocessed swath observation data, extract the local maximum points and local minimum points on each cross-section, and calculate the pixel spacing between adjacent local extrema; (4) Statistically analyze the pixel spacing between adjacent local extrema on all cross-sections, and generate a histogram to present the number distribution of pixel cycles; (5) Calculate the weighted average according to the pixel cycles statistically analyzed in the histogram to determine the kernel size of the mean filter, and the kernel size covers the complete periodic fluctuations; (6) Apply the designed mean filter to the swath observation data cross-section by cross-section, and repeat the process until all data are free of scalloping effect.

2. The method for removing scalloping effect from swath observation data according to claim 1, wherein, In step (1), the scalloping effect is caused by at least one of the following reasons: the scanning method of the sensor; the attitude change of the measuring device; the periodic error in the data processing process.

3. A method for removing scalloping effect from swath observation data according to claim 1, characterized in that, The specific interpolation method in step (2) is: select the interpolation algorithm according to the distribution characteristics of the data missing area; perform point-by-point filling on the discontinuous area, and use linear interpolation or spline interpolation.

4. A scallop effect removal method for swath observation data according to claim 1, wherein In step (3), the method for extracting local extrema is: define the neighborhood size k, and compare the values of adjacent k data points point by point; it is determined as an extremum when the following conditions are met: local maximum: the value of the current point is greater than the values of the k neighborhood points on the left and right; local minimum: the value of the current point is less than the values of the k neighborhood points on the left and right; among them, the neighborhood size should be selected according to the data characteristics to ensure that local extrema can be fully detected while reducing false detections caused by noise or local fluctuations.

5. A scallop effect removal method for swath observation data according to claim 1, characterized in that, In step (4), the method for generating the histogram is: group the pixel cycles by a fixed interval width of n pixels; statistically analyze the frequencies of local maxima and minima in each interval, and mark them with different colors; among them, the width n needs to be reasonably selected to clearly reflect the interval position of the peak.

6. A method for removing scallop effect from swath observation data according to claim 1, characterized in that In step (5), the method for determining the kernel size of the mean filter is: calculate the weighted average of the pixel cycles in the histogram, and the formula is: ； wherein, is the median value of the th interval, is the weight of the th interval, represents the sum of all weights.

7. A scallop effect removal method for swath observation data according to claim 1, characterized in that In step (6), the specific operation of the filtering process is: first apply the mean filter to the data points of each cross-section, and the calculation formula is: ； Among them, is the filter window size, and the range of the window is from to , is the original pixel value at the position in the swath observation profile data, is the output pixel value at the position after being processed by the mean filter; then iterative processing is performed section by section until all the data is denoised.

8. An electronic device, comprising a memory, a processor, and a computer program stored on the memory, characterized in that When the processor executes the program, it implements the steps of the method described in any one of claims 1-7.

9. A computer-readable storage medium storing a computer program, characterized in that, When the program is executed by the processor, it implements the steps of the method described in any one of claims 1-7.

Images