A filter circuit and a filtering method based on Hampel algorithm

Abstract

The application relates to a filtering circuit and filtering method based on a Hampel algorithm, which comprises a sliding window buffer module for buffering continuous sampling data streams in real time to form a sliding data window; a median solution module for performing hardware sorting on all data points in the sliding data window and solving the median; an absolute median deviation solution module for calculating the absolute deviation of each data point in the window from the median and sorting, and solving the MAD value; a threshold calculation and decision output module for calculating a decision threshold according to the median, the MAD value and a preset threshold coefficient, comparing the deviation of the current sampling point with the threshold value, and selecting the median or the original sampling point as the filtering result as output. The application adopts a full-pipeline design, realizes the Hampel filtering algorithm through a hardware circuit such as the odd-even sorting method, can process one sampling point in each clock cycle, greatly improves the processing efficiency, and solves the time-consuming problem of software sorting calculation.

Description

Technical Field

[0001] This invention relates to the fields of electronic technology and digital signal processing technology, specifically to a filtering circuit and filtering method based on the Hampel algorithm. Background Technology

[0002] In complex noisy environments, such as software-defined radio, industrial sensors, and RF communication systems, the signals acquired by sensors often contain impulse noise. Hampel filters effectively identify and suppress this type of impulse noise by utilizing the invariance of the median and absolute median deviation (MAD) to noise. However, existing technologies mainly rely on software algorithms, requiring traversal, sorting, and computation of data within a sliding window, consuming significant CPU resources. As sampling rates increase, software computation becomes a bottleneck in real-time signal processing, failing to meet the real-time requirements of high-speed systems. Therefore, a hardware solution is urgently needed to overcome the time-consuming problem of software sorting and computation, achieving efficient and real-time filtering processing. Summary of the Invention

[0003] The purpose of this invention is to provide a filtering circuit and filtering method based on the Hampel algorithm, which realizes real-time sorting calculation of sliding window data in pure hardware and outputs the filtering result of each sampling point in a pipeline manner, thereby solving the computational bottleneck of software implementation and meeting the requirements of high-speed real-time signal processing.

[0004] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: A filtering circuit based on the Hampel algorithm, comprising, in sequence: The sliding window caching module is used to cache continuous sampled data streams in real time, forming a sliding data window with a preset length; The median calculation module, connected to the sliding window caching module, is used to perform hardware sorting on all data points within the sliding data window and to calculate the median of the sliding data window in real time. The absolute median deviation calculation module is connected to the sliding window cache module and the median solving module. It is used to calculate the absolute deviation of each data point in the sliding data window from the median, and to sort the absolute deviations in hardware to solve for the absolute median deviation (MAD). The threshold calculation and judgment output module is connected to the sliding window buffer module, the median solving module, and the absolute median deviation calculation module. It is used to calculate the decision threshold based on the median, the absolute median deviation (MAD), and the preset threshold coefficient, and compare the absolute deviation of the current sampled data point from the median with the decision threshold. Based on the comparison result, it selects to output the median or the current sampled data point as the filtering result.

[0005] The beneficial effects of this invention are: by implementing the Hampel filtering algorithm entirely in hardware, it eliminates the dependence on the CPU, greatly improves the processing speed and system throughput, and at the same time ensures the calculation accuracy, making it particularly suitable for scenarios such as real-time processing of high sampling rate radio frequency signals.

[0006] Based on the above technical solution, the present invention can be further improved as follows.

[0007] Furthermore, the sliding window buffer module consists of N cascaded registers, where N is the preset length; in each clock cycle, new sampled data is input from the first register, the original data in the sliding data window is shifted backward sequentially, and the data in the last register is removed from the sliding data window, thereby realizing the real-time update of the sliding data window; the preset length is a configurable parameter.

[0008] Furthermore, both the median solving module and the absolute median deviation calculation module use hardware configured with an odd-even sorting method to sort the data in ascending order, and extract the median or the absolute median deviation (MAD) from a specified position based on the sorting result.

[0009] Furthermore, the odd-even sorting method uses a state machine to control the alternating operation of even-phase comparators and odd-phase comparators; in the even phase, data pairs with adjacent even-numbered indices are compared and sorted; in the odd phase, data pairs with adjacent odd-numbered indices are compared and sorted; after N clock cycles of alternating sorting, the ascending sorting of all N data points within the sliding data window is completed.

[0010] Furthermore, the threshold calculation and judgment output module includes: A multiplier is used to calculate the decision threshold, which is equal to the product of the absolute median deviation and the threshold coefficient; A comparator is used to compare the absolute deviation of the current sampled data point from the median with the decision threshold; The data selector selects data based on the comparison result of the comparator: when the absolute deviation is greater than the decision threshold, the median is output as the filtering result; otherwise, the current sampled data point is output as the filtering result.

[0011] Furthermore, the threshold coefficient is a fixed value, a preset fixed constant, or a value that can be configured and adjusted through an external interface.

[0012] Furthermore, it also includes an extremum statistics module, used to monitor the dynamic maximum and minimum values ​​of the sampled data stream in real time, as well as the dynamic maximum and minimum values ​​of the output filtering results, and the statistics of the dynamic maximum and minimum values ​​of the input and output support independent reset.

[0013] Furthermore, the hardware circuit adopts a fully pipelined design; the sliding window buffer module, the median solving module, the absolute median deviation calculation module, and the threshold calculation and judgment output module are sequentially connected to form the pipeline, so that the system can process a new sampling point and output a filtering result in each clock cycle.

[0014] Furthermore, the filtering circuit is implemented and configured on an FPGA platform using a hardware description language. Specifically, it can be described using Verilog HDL, is compatible with the Verilog-1995 standard, and can be synthesized on various FPGA platforms. The circuit employs a falling-edge triggered and asynchronous low-reset design, making it compatible with common digital system design specifications. By adjusting the data bit width and window size parameters, it can be adapted to sampling signal processing applications with different sampling accuracies and filtering requirements.

[0015] To address the aforementioned technical problems, this invention also provides a filtering method based on the Hampel algorithm, the specific technical content of which is as follows: A filtering method based on the Hampel algorithm includes the following steps: The continuous sampled data stream is buffered in real time to form a sliding data window with a preset length; All data points within the sliding data window are sorted by hardware, and the median of the sliding data window is calculated in real time. Calculate the absolute deviation of each data point within the sliding data window from the median, and sort the absolute deviations using hardware to obtain the absolute median deviation (MAD). The decision threshold is calculated based on the median, the absolute median deviation (MAD), and the preset threshold coefficient. The absolute deviation of the current sampled data point from the median is compared with the decision threshold. Based on the comparison result, the median or the current sampled data point is selected as the filtering result. Attached Figure Description

[0016] Figure 1 This is a block diagram of the overall architecture of a filter circuit based on the Hampel algorithm in an embodiment of the present invention; Figure 2 This is the overall pipeline timing diagram of the filter circuit in this embodiment of the invention; Figure 3 This is a hardware structure diagram of the sliding window in an embodiment of the present invention; Figure 4 This is a flowchart of the state machine for the odd-even sorting method in an embodiment of the present invention; Figure 5 This is a flowchart illustrating the calculation of the median absolute deviation in an embodiment of the present invention. Figure 6This is a circuit diagram for threshold calculation and decision-making in an embodiment of the present invention; Figure 7 This is a circuit diagram of the maximum / minimum value statistics module in an embodiment of the present invention; Figure 8 This is a schematic diagram comparing the filtering effects in an embodiment of the present invention. Detailed Implementation

[0017] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0018] The Hampel algorithm (also known as the Hampel filter or Hampel detector) is a robust statistical method for outlier detection and processing in time series data. Based on the median and median absolute deviation (MAD), it is highly robust to outliers in the data, and is particularly suitable for scenarios with impulse noise or extreme outliers.

[0019] An FPGA platform is a reconfigurable hardware platform based on field-programmable gate arrays, allowing users to implement customized digital circuit functions through programming. It is widely used in fields such as communications, artificial intelligence, automotive electronics, and industrial control. The core of an FPGA platform is the FPGA chip, and its basic architecture includes: Configurable Logic Block (CLB): The basic unit for implementing combinational and sequential logic, typically consisting of a lookup table (LUT) and flip-flops.

[0020] Programmable Input / Output Blocks (IOBs): Interfaces that connect the internal logic of an FPGA to external devices, supporting multiple electrical standards.

[0021] Abundant wiring resources: used for flexible connection of various logic blocks to build complex circuit systems.

[0022] Dedicated hard core modules: Some FPGAs integrate embedded block RAM (BRAM), digital signal processing (DSP) units, high-speed transceivers (such as PCIe, Ethernet) and ARM processor cores to form a "system-on-a-chip (SoC)".

[0023] FPGAs configure their internal logic structure and implement specific functions by loading bitstream files. Configuration data is read from external memory when power is applied, and the logic is lost after power is lost, allowing for reprogramming and reuse.

[0024] like Figure 1As shown, this embodiment provides a filtering circuit based on the Hampel algorithm, which includes, in sequence, a signal input module, a sliding window buffer module, a median calculation module, an absolute median deviation calculation module, a threshold calculation and judgment output module, a filtering output module, and an extremum statistics module.

[0025] The signal input module uses a hardware description language (digital circuit) to acquire the radio frequency sampling data stream. The sliding window caching module operates in the same time direction as the data flow. Within this direction, it updates in real-time using a window of 50 sampling points (the preset length mentioned above). Figure 3 The diagram shows the hardware structure of a 50-point sliding window. The diagram illustrates how the window is updated every clock cycle under continuous data input. New data enters from data0, and old data is shifted sequentially to the right, eventually exiting from data49. Figure 3 The example provided is a sliding window chain consisting of 50 registers, which enables real-time caching of continuous RF sampling data.

[0026] The median calculation module employs an even / odd sorting method to achieve efficient hardware sorting. For example... Figure 4 The flowchart of the parity sorting algorithm shows that, starting from the first input data, the window data is copied to the sorting area in real time for parity sorting calculation. During even-phase operations, adjacent data pairs with even indices (0,1), (2,3), etc., are compared and their order is swapped. During odd-phase operations, adjacent data pairs with odd indices (1,2), (3,4), etc., are compared and their order is swapped. After 50 clock cycles (related to the window length; changes in window length will cause corresponding changes in this clock cycle), the entire window is sorted in ascending order. The 24th bit (or a middle bit, using the 24th bit as an example) is extracted from the sorting result as the center value of the window.

[0027] The absolute median deviation calculation module uses, for example, Figure 5 The calculation process shown first obtains the median after parity sorting, then calculates the absolute deviation of each sample point from the median, sorts the 50 deviation values ​​again for parity, and extracts the 24th value as the MAD (Absolute Median Deviation) value. Both calculation processes use the same sorting algorithm, achieving efficient reuse of hardware resources.

[0028] The threshold calculation and judgment output module adopts, for example, Figure 6The algorithm flow is shown below. Based on the aforementioned calculations of the median and MAD values, the decision threshold is first calculated using a multiplier. The decision threshold is calculated as: Decision Threshold = par × MAD, where MAD represents the absolute median deviation, and par can be a fixed value (e.g., 3 × 1.4826) or configurable input data. Then, a comparator compares the absolute deviation of the current sampling point from the median with the decision threshold. If the deviation exceeds the decision threshold, the median is output as the filtering result via a data selector; otherwise, the original sampling value is output. As the sliding window moves, the filtered data for each sampling point is output in real time in a pipeline manner.

[0029] Maximum and minimum value statistics module, such as Figure 7 The circuit structure is shown. This module monitors the maximum and minimum values ​​of the input raw data and the output filtered data in real time. It uses a dual comparator structure to track the maximum and minimum values ​​of the input data Din and the output data V_filtered respectively, and supports independent reset function to provide auxiliary information for system monitoring.

[0030] The entire filtering circuit of this invention adopts a fully pipelined design, with a total delay of 150 clock cycles from data input to filter output, but a throughput of processing one sampling point per clock cycle, meeting the real-time requirements of high-speed RF sampling systems. Figure 8 As shown, the filtering effect of the present invention can effectively suppress pulse noise in radio frequency signals.

[0031] In other embodiments, the present invention also provides a filtering method based on the Hampel algorithm, employing, for example... Figure 2 The filtering process is shown below; like Figure 2 As shown, the RF data stream is input as continuous clock pulses. From the arrival of the first sampling point, the system enters a parallel pipelined operation, including stages such as sliding window buffering, median sorting, MAD sorting, and decision output. Each sampling point sequentially goes through these stages, with a total filtering delay of approximately 150 clock cycles. However, due to the fully pipelined design, the system can start and process a new window of data in each clock cycle, thus achieving a throughput of one filtered result per clock cycle, meeting the real-time requirements of high-speed RF signal processing.

[0032] like Figure 3 As shown, the sliding window hardware structure consists of a chain of 50 16-bit registers. Data is input from the Din port. Under the control of the falling edge of the clock and the Din_en enable signal, new data is stored in the data0 register, the existing data is shifted sequentially to the right, and the data in the data49 register is shifted out of the window. This structure realizes real-time buffering of continuous RF sampling data, providing a data source for subsequent sorting calculations.

[0033] like Figure 4 As shown, the median and MAD calculations are implemented in hardware using an even-odd sorting method. This includes an even-phase comparator array and an odd-phase comparator array, which alternately operate under a phase switching controller. Even-phase comparators compare adjacent data pairs with even indices (0,1), (2,3)...(48,49), while odd-phase comparators compare adjacent data pairs with odd indices (1,2), (3,4)...(47,48). After 50 clock cycles of alternating comparisons and exchanges, the 50 data points are sorted in ascending order. In practical applications, if real-time requirements are not high or the window size needs adjustment, other hardware sorting algorithms, such as insertion sort, quicksort, or merge sort, can be used, but corresponding adjustments to hardware resources, timing control, and delay design are required.

[0034] like Figure 5 As shown, the absolute deviation median is calculated after the median win_mid is calculated. First, the absolute deviation of 50 window data points from the median is calculated; that is, for each data point data_sort[i] (i=0-49), |data_sort[i] - win_mid| is calculated, resulting in 50 deviation values. Then, these 50 deviation values ​​are copied to the deviation sorting area, using the method... Figure 4 The same odd-even sorting method is used for sorting. After sorting, the 24th bit is extracted from the sorted deviation values ​​and output as the absolute median deviation (MAD) value, win_mad. This process implements the hardware calculation of MAD, a key parameter in Hampel filtering.

[0035] like Figure 6 As shown, the decision threshold calculation and decision circuit diagram implement the final decision logic of Hampel filtering. The decision threshold calculation and decision method are as follows: First, the decision threshold win_sigma = 2885 × win_mad is calculated using a multiplier, where 2885 is a fixed-point number represented by 3 × 1.4826 × 2^10. Then, a comparator compares the absolute deviation of the current sampling point absDin with the decision threshold win_sigma. If absDin > win_sigma, the median win_mid is output via a data selector; otherwise, the original data data0 is output. It is important to note that the constant factor (3 × 1.4826) used in the decision threshold calculation module of this embodiment is the standard coefficient of the Hampel algorithm. In actual hardware implementation, this multiplication coefficient can be configured and adjusted according to the signal and noise characteristics of the specific application scenario. For example, for environments with more severe impulse noise, the coefficient can be appropriately increased to enhance the filtering strength; for scenarios requiring the preservation of more signal details, the coefficient can be decreased. Therefore, the decision threshold calculation module of this invention can achieve flexible adjustment of the decision threshold parameter by modifying the multiplier coefficient or introducing a coefficient configuration interface, thereby enhancing the adaptability of the entire filtering system to different signal environments.

[0036] like Figure 7 As shown, the extremum statistics module monitors the maximum and minimum values ​​of the input raw data and the output filtered data in real time. The input extremum statistics module compares Din with the current din_max and din_min using two comparators, dynamically updating the extremum values. The output extremum statistics module processes the V_filtered data in the same way. Both modules support independent reset functions, reset via the MaxMin_din_rst_n and MaxMin_dout_rst_n signals respectively.

[0037] like Figure 8 As shown in the comparison results, it can be seen that the original RF signal waveform containing impulse noise exhibits obvious amplitude abnormal spikes. After filtering by the circuit of this invention, the impulse noise in the output signal is effectively suppressed, and the signal waveform is smooth. Therefore, this invention can effectively remove impulse interference and improve signal quality while maintaining the main characteristics of the signal.

[0038] It should be noted that the window size in this embodiment of the invention is preferably 50 points, but is not a specific value and can be configured as needed; in practical applications, the window size can be adjusted according to specific requirements, and the sorting period and the number of registers can be adjusted accordingly.

[0039] In this embodiment of the invention, the RF sampling data Din is input into the sliding window buffer module at the falling edge of the clock and when Din_en is valid, and 50 historical data points are updated. The window data is then copied to the sorting area, and the parity sorting median calculation module is started. After 50 clock cycles of alternating parity comparisons, the sorting is completed and the median win_mid is output. The absolute deviation calculation module calculates the absolute deviation of each data point from win_mi, and the MAD sorting module sorts the deviation values, outputting the MAD value win_mad. The threshold calculation module calculates the decision threshold win_sigma = par × win_mad, and simultaneously calculates the deviation of the current sampling point absDin, and outputs the filtering result V_filtered by comparison. The extreme value statistics module updates the input and output extreme values ​​in parallel, providing system monitoring data. This allows all modules to work collaboratively to achieve filtering based on timing relationships and pipelined operation. As can be seen, this invention implements the Hampel filtering algorithm through a fully hardware pipeline, eliminating the dependence on the CPU, greatly improving the processing speed and system throughput (processing one sampling point per clock cycle), while ensuring calculation accuracy. It is particularly suitable for scenarios such as real-time processing of high sampling rate radio frequency signals.

[0040] In some other embodiments, this embodiment also provides a storage medium storing a computer program or computer instructions, which, when executed by a computer's processor, implement the steps of the above-described filtering method based on the Hampel algorithm.

[0041] The storage medium can be an internal storage unit of any data processing device described in any of the foregoing embodiments, such as a hard disk or memory. The storage medium can also be an external storage device of any data processing device, such as a plug-in hard disk, smart memory card, SD card, flash memory card, etc., mounted on the device. Furthermore, the storage medium can include both internal storage units and external storage devices of any data processing device. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0042] In other embodiments, a computer is also provided, including a memory and one or more processors, wherein executable code is stored in the memory, and when the one or more processors execute the executable code, the steps of the above-described filtering method based on the Hampel algorithm are implemented.

[0043] The memory can be an internal storage unit of any data processing device described in any of the foregoing embodiments, such as a hard disk or RAM. The memory can also be an external storage device of any data processing device, such as a plug-in hard disk, smart memory card, SD card, flash memory card, etc., mounted on the device. Furthermore, the memory can include both internal storage units and external storage devices of any data processing device. The memory is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0044] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the concept and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A filtering circuit based on the Hampel algorithm, characterized in that, In order, they include: The sliding window caching module is used to cache continuous sampled data streams in real time, forming a sliding data window with a preset length; The median calculation module, connected to the sliding window caching module, is used to perform hardware sorting on all data points within the sliding data window and to calculate the median of the sliding data window in real time. An absolute median deviation calculation module, connected to the sliding window caching module and the median solving module, is used to calculate the absolute deviation of each data point in the sliding data window from the median, and to perform hardware sorting on the absolute deviations to solve for the absolute median deviation. The threshold calculation and judgment output module is connected to the sliding window buffer module, the median solving module, and the absolute median deviation calculation module. It is used to calculate the decision threshold based on the median, the absolute median deviation, and the preset threshold coefficient, and compare the absolute deviation of the current sampled data point from the median with the decision threshold. Based on the comparison result, it selects to output the median or the current sampled data point as the filtering result.

2. The filtering circuit based on the Hampel algorithm according to claim 1, characterized in that, The sliding window buffer module consists of N cascaded registers, where N is the preset length. In each clock cycle, new sampled data is input from the first register, the existing data in the sliding data window is shifted sequentially to the right, and the data in the last register is removed from the sliding data window, thereby realizing the real-time update of the sliding data window. The preset length is a configurable parameter.

3. The filtering circuit based on the Hampel algorithm according to claim 1, characterized in that, Both the median calculation module and the absolute median deviation calculation module use hardware configured with an odd-even sorting method to sort the data in ascending order, and extract the median or the absolute median deviation from a specified position based on the sorting result.

4. The filtering circuit based on the Hampel algorithm according to claim 3, characterized in that, The odd-even sorting method uses a state machine to control the alternating operation of even-phase comparators and odd-phase comparators. In the even phase, data pairs with adjacent even-numbered indices are compared and sorted; in the odd phase, data pairs with adjacent odd-numbered indices are compared and sorted. After N clock cycles of alternating sorting, the ascending sorting of all N data points within the sliding data window is completed.

5. The filtering circuit based on the Hampel algorithm according to claim 1, characterized in that, The threshold calculation and judgment output module includes: A multiplier is used to calculate the decision threshold, which is equal to the product of the absolute median deviation and the threshold coefficient; A comparator is used to compare the absolute deviation of the current sampled data point from the median with the decision threshold; The data selector selects data based on the comparison result of the comparator: when the absolute deviation is greater than the decision threshold, the median is output as the filtering result; otherwise, the current sampled data point is output as the filtering result.

6. The filtering circuit based on the Hampel algorithm according to claim 1, characterized in that, The threshold coefficient is a fixed value, a preset fixed constant, or a value that can be configured and adjusted through an external interface.

7. The filtering circuit based on the Hampel algorithm according to claim 1, characterized in that, It also includes an extremum statistics module, which is used to monitor the dynamic maximum and minimum values ​​of the sampled data stream in real time, as well as the dynamic maximum and minimum values ​​of the output filtering results, and the statistics of the dynamic maximum and minimum values ​​of the input and output support independent reset.

8. The filtering circuit based on the Hampel algorithm according to claim 1, characterized in that, The hardware circuit adopts a fully pipelined design; the sliding window buffer module, the median solving module, the absolute median deviation calculation module, and the threshold calculation and judgment output module are connected in sequence to form the pipeline, so that the system can process a new sampling point and output a filtering result in each clock cycle.

9. The filtering circuit based on the Hampel algorithm according to claim 1, characterized in that, The filter circuit is implemented using a hardware description language and configured on an FPGA platform.

10. A filtering method applied to a filtering circuit based on the Hampel algorithm as described in any one of claims 1 to 9, characterized in that, Includes the following steps: The continuous sampled data stream is buffered in real time to form a sliding data window with a preset length; All data points within the sliding data window are sorted by hardware, and the median of the sliding data window is calculated in real time. Calculate the absolute deviation of each data point within the sliding data window from the median, and sort the absolute deviations using hardware to solve for the absolute median deviation. The decision threshold is calculated based on the median, the absolute median deviation, and the preset threshold coefficient. The absolute deviation of the current sampled data point from the median is compared with the decision threshold. Based on the comparison result, the median or the current sampled data point is selected as the filtering result.

Images