A method and apparatus for detecting outliers in sonar data
By grouping and rearranging sonar data and using the Wright criterion for detection, the computational complexity of outlier detection and the inadequacy of spot-type outlier detection in sonar data were resolved, achieving efficient and accurate outlier removal and improving data quality.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- THE 715TH RES INST OF CHINA SHIPBUILDING IND CORP
- Filing Date
- 2026-02-02
- Publication Date
- 2026-05-26
AI Technical Summary
Existing sonar data contains outliers. Current detection methods make strong assumptions about the data distribution, are computationally complex, and are insufficient in detecting speckled outliers, which affects the accuracy of target tracking.
Sonar detection data is split into small data segments, rearranged into a matrix, and detrended. Isolated and speckled outliers are detected and replaced based on the Wright criterion, and the mean of the data is used to replace the outlier points.
It enables efficient and accurate detection and removal of isolated and speckled outliers in sonar data, thereby improving data quality.
Smart Images
Figure CN122085255A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of sonar data processing technology, and specifically relates to a method and apparatus for detecting outliers in sonar data. Background Technology
[0002] High-quality sonar acquisition data is a crucial foundation for data analysis and processing (target detection, tracking, identification, etc.). However, sonar acquisition data contains outliers that deviate from the normal range. The presence of outliers in sonar data can severely affect the accuracy of target tracking, which is unacceptable in situations requiring high-quality sonar acquisition data. Therefore, it is necessary to detect and remove outliers during the data preprocessing stage of sonar acquisition data.
[0003] Currently, outlier detection methods fall into three categories. The first category removes outliers based on statistical characteristics of the data, setting reasonable thresholds such as the Wright criterion, Romanovsky criterion, Grubbs criterion, and Chauville criterion. These methods are constrained by the statistical characteristics of the data, limiting their application. The second category removes outliers based on data features, such as wavelet analysis and signal feature extraction. These methods have high algorithmic complexity and computational cost. The third category removes outliers based on the statistical characteristics of the difference between the estimated and original values. These methods are mostly based on optimal filtering and noise reduction, and their performance is unsatisfactory for non-optimal filtering, especially for speckled outliers. When sonar data changes significantly or the target is maneuvering, directly using statistical characteristics as thresholds for outlier detection can inadvertently remove useful detection data, failing to eliminate true outliers.
[0004] Therefore, there is an urgent need for a method that can maintain computational efficiency while effectively detecting isolated and speckled outliers. Summary of the Invention
[0005] The technical problem to be solved by the present invention is to provide a method and apparatus for detecting outliers in sonar data, so as to overcome the problems of strong assumptions about data distribution, complex calculations, and insufficient ability to detect speckled outliers in the prior art, and to achieve efficient and accurate detection and removal of isolated and speckled outliers in sonar data.
[0006] The technical solution of the present invention is to provide a method for detecting outliers in sonar data, comprising the following steps:
[0007] Step 1: Divide the sonar detection data of length N into smaller data segments of length L, and calculate the number of groups. ,Will The original data was rearranged into Matrix;
[0008] Step two: Perform detrending processing on each row of the data matrix;
[0009] Step 3: For each row of data after the detrended term, perform outlier detection based on the Wright criterion;
[0010] Step 4: Remove the detected outliers and replace them with the mean of the data in that row;
[0011] Step 5: Rearrange the processed data matrix back to its original order and output the sonar data after removing outliers;
[0012] Step three specifically includes:
[0013] Calculate the mean and standard deviation of the data in this row;
[0014] Calculate the residuals for each data point;
[0015] The residuals are compared with the preset standard deviation; if they exceed the standard deviation, they are considered outliers.
[0016] This invention proposes a data rearrangement outlier detection method based on the Wright criterion, which is used to remove isolated and speckled outliers in sonar acquisition data. Specifically, the sonar acquisition data is grouped and rearranged, and the outliers in each group are detected using the Wright criterion. Finally, the outlier-detected data are recombined. This is a novel and improved outlier detection method for sonar data.
[0017] This method considers grouping and rearranging the sonar acquisition data, using the Wright criterion to detect outliers in each group, and finally recombining the outlier-detected data. This method can effectively detect outliers and improve the data quality to a certain extent.
[0018] As a preferred option, in step three, the residual is compared with three times the standard deviation; if it exceeds this value, it is considered an outlier.
[0019] Preferably, the data rearrangement rule is to fill each row of the matrix with the original data in sequence until all the data is filled.
[0020] Preferably, the outliers include isolated outliers and speckled outliers.
[0021] Preferably, in step three, the detrending term processing is implemented using linear fitting or polynomial fitting methods.
[0022] Preferably, the standard deviation is calculated using the Bessel formula.
[0023] The present invention also provides a sonar data outlier detection device, comprising:
[0024] The data segmentation module is used to split sonar detection data into multiple small data segments and rearrange them into a matrix form;
[0025] The detrending module is used to detrend each row of data.
[0026] The outlier detection module is used to identify outliers in each row of data based on the Wright criterion.
[0027] The outlier replacement module is used to replace the identified outliers with the mean of the data in that row.
[0028] The data reconstruction module is used to restore the processed matrix data to its original order.
[0029] Furthermore, the outlier detection module is configured to calculate the mean, standard deviation, and residual of each data point for each row of data, and compare them with three times the standard deviation to determine outliers.
[0030] Compared with the prior art, the present invention has the following advantages:
[0031] This invention, by employing its technical solution, enables the removal of outliers during data preprocessing. It inherits the advantages of the Wright criterion for detecting isolated outliers and also has a significant effect on removing speckled outliers, thus improving the quality of sonar acquisition data. Attached Figure Description
[0032] Figure 1 This is a flowchart of the processing according to an embodiment of the present invention.
[0033] Figure 2 This is a schematic diagram of data rule rearrangement in an embodiment of the present invention.
[0034] Figure 3 is a simulation analysis result diagram of an embodiment of the present invention.
[0035] Figure 4 is a diagram showing the data processing results of an embodiment of this application. Detailed Implementation
[0036] The present invention will be further described below with reference to specific embodiments:
[0037] This invention divides sonar detection data of length N into small data segments of length L, and calculates the number of groups. ,Will The original data was rearranged into The matrix, after removing outliers and replacing values in each row, will... Data matrix reassembly The sonar data collected.
[0038] like Figure 1 As shown, this embodiment of the invention provides a method for detecting outliers in sonar data, comprising the following steps:
[0039] Step 1: The data length of a certain sonar array is N. It needs to be divided into small data segments of length L, and the number of groups needs to be calculated. ,Will The original data was rearranged into The matrix is rearranged according to the following rules: Figure 2 As shown.
[0040] Step 2: For each row of data Perform detrending processing on each item separately.
[0041] Step 3: Process the data after removing the trend item. Use step four (Wright's criterion) to make the judgment;
[0042] Step 4: Calculate the average of the measured values Calculate the standard deviation using the Bessel formula. , Calculate the residuals of each data point. , The residuals are compared one by one with 3 If a comparison is made, If so, then the value is an outlier.
[0043] Step 5: Remove the outliers detected in Step 4. The mean of the data in that row can be used instead.
[0044] Step Six: Data matrix reassembly The sonar data collected.
[0045] In this embodiment, the simulation uses data with a length of 1000, a mean of 0, and a variance of 2. Isolated outliers with constant values of -15, -10, 15, and 12 are added at points 30, 50, 230, and 250, respectively. Spotted outliers with a constant value of 8 are added from points 56 to 120. Outlier detection is performed using both the Wright criterion outlier detection method and the Wright criterion-based data segmentation outlier detection method. The simulation results are shown in Figure 3. Figure 3(a) shows the Wright criterion outlier detection result, and Figure 3(b) shows the Wright criterion-based data segmentation outlier detection result. In the figures, green marks indicate the detected outlier points. As can be seen from Figure 3, both methods can correctly identify the outlier locations for isolated outliers. For spotted outliers, even when the Wright criterion cannot identify them, the Wright criterion-based data segmentation outlier detection method can still effectively identify them.
[0046] A segment of sonar data with 6000 points was extracted. Spot-like outliers were found between points 1001 and 1400, and an isolated outlier was found at point 3000. Outlier detection was performed using both the Wright criterion outlier detection method and the Wright criterion-based data segmentation outlier detection method. The simulation results are shown in Figure 4. Figure 4(a) shows the Wright criterion outlier detection result, and Figure 4(b) shows the Wright criterion-based data segmentation outlier detection result. As can be seen from the figures, the sonar data processing results are consistent with the simulation results.
[0047] The present invention also provides a sonar data outlier detection device, comprising:
[0048] The data segmentation module is used to split sonar detection data into multiple small data segments and rearrange them into a matrix form;
[0049] The detrending module is used to detrend each row of data.
[0050] The outlier detection module is used to identify outliers in each row of data based on the Wright criterion.
[0051] The outlier replacement module is used to replace the identified outliers with the mean of the data in that row.
[0052] The data reconstruction module is used to restore the processed matrix data to its original order.
[0053] Furthermore, the outlier detection module is configured to calculate the mean, standard deviation, and residual of each data point for each row of data, and compare them with three times the standard deviation to determine outliers.
[0054] The above description only illustrates preferred embodiments of the present invention and should not be construed as limiting the scope of the claims. Any equivalent procedural modifications made using this specification are included within the patent protection scope of this invention.
Claims
1. A method for detecting outliers in sonar data, characterized in that, Includes the following steps: Step 1: Divide the sonar detection data of length N into smaller data segments of length L, and calculate the number of groups. ,Will The original data was rearranged into Matrix; Step two: Perform detrending processing on each row of the data matrix; Step 3: For each row of data after the detrended term, perform outlier detection based on the Wright criterion; Step 4: Remove the detected outliers and replace them with the mean of the data in that row; Step 5: Rearrange the processed data matrix back to its original order and output the sonar data after removing outliers; Step three specifically includes: Calculate the mean and standard deviation of the data in this row; Calculate the residuals for each data point; The residuals are compared with the preset standard deviation; if they exceed the standard deviation, they are considered outliers.
2. The sonar data outlier detection method according to claim 1, characterized in that: In step three, the residuals are compared with three times the standard deviation; if the difference is exceeded, it is considered an outlier.
3. The sonar data outlier detection method according to claim 1, characterized in that: The data rearrangement rule is to fill each row of the matrix with the original data in sequence until all the data is filled.
4. The sonar data outlier detection method according to claim 1, characterized in that: The outliers include isolated outliers and speckled outliers.
5. The sonar data outlier detection method according to claim 1, characterized in that: In step three, the detrending term processing is implemented using linear fitting or polynomial fitting methods.
6. The underwater three-dimensional terrain reconstruction method based on forward-looking sonar according to claim 1, characterized in that: The standard deviation is calculated using the Bessel formula.
7. A sonar data outlier detection device, characterized in that, include: The data segmentation module is used to split sonar detection data into multiple small data segments and rearrange them into a matrix form; The detrending module is used to detrend each row of data. The outlier detection module is used to identify outliers in each row of data based on the Wright criterion. The outlier replacement module is used to replace the identified outliers with the mean of the data in that row. The data reconstruction module is used to restore the processed matrix data to its original order.
8. The apparatus according to claim 7, characterized in that, The outlier detection module is configured to calculate the mean, standard deviation, and residual of each data point for each row of data, and compare them with three times the standard deviation to determine outliers.