An adaptive average denoising method

Abstract

An adaptive average noise removal method, which relates to the fields of digital signal processing, artificial intelligence and intelligent instruments, comprises the following steps: S1, firstly setting the number of accumulations n and the amount of real-time collected data m; S2, the FPGA unit as the core controller receives and analyzes the data, and automatically calculates and selects the most efficient average noise removal method based on the number of accumulations n and the amount of real-time collected data m, that is, the fastest average noise removal method under the existing storage resources; S3, the average noise removal method with the highest efficiency is automatically selected to process the real-time collected data in real time; S4, the FPGA unit outputs the data processing result through the communication interface, and displays the result in the form of a chart in the ARM unit. The present invention has the advantages of high efficiency and low cost, is widely applicable to the application occasions of different storage spaces, and meets the application requirements of intelligent instruments.

Description

Technical Field

[0001] The invention relates to the fields of digital signal processing, artificial intelligence and intelligent instruments and meters, and in particular to an adaptive average noise removal method. Background Art

[0002] In digital signal processing, the cumulative average method is often used to remove noise and improve the signal-to-noise ratio. In particular, the cumulative average method is indispensable in distributed fiber optic sensor systems or optical instruments because the scattered light signal power is very low, some are completely submerged in the noise, and the amount of data collected at high speed is very large. At present, the cumulative average algorithms mainly include linear cumulative average algorithm and recursive cumulative average algorithm. For high-speed real-time acquisition and processing of large amounts of data, the linear cumulative average algorithm and the recursive cumulative average algorithm have their own characteristics. The linear cumulative average algorithm is fast, but it is easy to cause the risk of data overflow. In order to prevent data overflow, a large amount of storage resources are required. The recursive cumulative average algorithm occupies less resources, is not limited by the number of accumulations, and does not have the risk of data overflow, but the speed is slow, so these two cumulative average algorithms are limited in many practical applications. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide an adaptive average noise removal method with high speed and small storage resource occupation.

[0004] The technical solution to the above technical problem is: an adaptive average noise removal method, which includes the following steps:

[0005] S1. First, set the number of accumulations n and the amount of real-time data collection m;

[0006] S2, the FPGA unit as the core controller receives and analyzes the data, and automatically calculates and selects the most efficient average denoising method based on the number of accumulations n and the amount of real-time collected data m, that is, the fastest average denoising method under the existing storage resources;

[0007] S3, using the most efficient average noise removal method to automatically select the real-time collected data for real-time processing;

[0008] S4. The FPGA unit outputs the data processing results through the communication interface, and the ARM unit displays the results in the form of charts.

[0009] A further technical solution of the present invention is: in step S2, the most efficient average noise removal method is automatically calculated and selected as follows:

[0010] Automatically calculate whether the existing storage resources meet the linear cumulative average denoising according to the set cumulative number of times n and the real-time collected data volume m. If not, use the combination of linear cumulative average and recursive cumulative average. Through the analysis of the average number of times and the collected data volume and calculation, when the storage resources are satisfied, it is split into multiple groups, so that the linear cumulative average algorithm is implemented within each group, and the recursive cumulative average algorithm is used between each group. The two are used together to achieve the highest efficiency.

[0011] A further technical solution of the present invention is that the existing storage resources are the internal storage resources of the FPGA or an external memory, and the size of the storage resources is A, with the unit of MB.

[0012] A further technical solution of the present invention is that the specific steps for automatically calculating and selecting the average denoising method with the highest efficiency are as follows:

[0013] S21. Based on the set cumulative number of times n and the real-time collected data volume m, calculate the memory space required for the average denoising algorithm with the least time consumption. The calculation formula is as follows:

[0014]

[0015] Where L is the bit width of the ADC unit, that is, the bit width of the collected data;

[0016] S22. Compare the calculated memory space B required for the average denoising algorithm with the least time consumption with the storage resource A, and judge which average denoising method has the highest efficiency:

[0017] When A≥B, use the linear cumulative average algorithm for average denoising, as shown in Equation ②:

[0018]

[0019] In the formula, a ij is the collected data, i is the number of collected sequences, that is, the average number of times, j is the number of data points collected each time, and S ij is the average value;

[0020] When A < B, the calculated memory space B exceeds the existing storage resource A of the system, and the exceeded storage resource is B - A. Adopt the way of trading time for space, divide the cumulative number of times into C equal parts, each equal part is D times, and then use the combination of linear cumulative average and recursive cumulative average. In each equal part, use the former S in Equation ⑤ ij to implement, that is, use the linear cumulative average algorithm shown in Equation ② to implement; between each equal part, use the latter S in Equation ⑤ qt to implement, that is, use the recursive cumulative average algorithm shown in Equation ④ to implement, so as to achieve the highest efficiency of the average denoising algorithm under limited storage space; specifically as follows:

[0021] When A < B,

[0022]

[0023]

[0024] where a qt is the input data, S qt is the average value, q is the q-th averaging, and t is the number of data for each averaging;

[0025]

[0026] In the formula is the average value.

[0027] A further technical solution of the present invention is: in step S1, the accumulation times n and the real-time data acquisition amount m are set by sending instructions to the FPGA unit.

[0028] A further technical solution of the present invention is: in step S2, the FPGA unit receives data including receiving instructions sent by the ARM unit or instructions sent by any other communication device of the FPGA unit.

[0029] A still further technical solution of the present invention is: the method is implemented based on an adaptive average denoising system, and the adaptive average denoising system includes:

[0030] a signal conditioning unit for converting a single-channel signal into a differential signal,

[0031] an ADC unit for converting an analog signal into a digital signal,

[0032] an FPGA unit for implementing the core control of the adaptive average denoising method,

[0033] an ARM unit for implementing communication with the FPGA and human-computer interaction;

[0034] The input end of the signal conditioning unit is connected to the input analog signal, the output end of the signal conditioning unit is connected to the input end of the ADC unit, the output end of the ADC unit is connected to the input end of the FPGA unit, and the FPGA unit and the ARM unit are connected to each other.

[0035] Due to the adoption of the above method, compared with the prior art, an adaptive average denoising method of the present invention has the following beneficial effects:

[0036] 1. The highest efficiency under limited storage resources

[0037] The present invention receives and analyzes data through an FPGA unit as a core controller, and automatically calculates and selects the average noise reduction method that meets the highest efficiency under existing storage resources based on the cumulative number of times n and the real-time data collection amount m. Since the core control of the method is the FPGA unit, which adopts a Harvard structure, data can be processed in parallel, while the CPU is serial. The present invention can automatically calculate whether the existing storage resources meet the linear cumulative average noise reduction according to the set average number of times and the amount of data. If not, a combination of linear cumulative average and recursive cumulative average is used. By analyzing the average number of times and calculating the storage space, it is divided into multiple groups, so that the linear cumulative average method is implemented in each group, and the recursive cumulative average algorithm is used between each group. The two are used together to achieve the highest algorithm efficiency, that is, based on the existing space algorithm, the space is small, the speed is fastest, and the efficiency is highest.

[0038] 2. Realize algorithm adaptation to meet different application scenarios

[0039] The algorithm of the present invention is adaptive, automatically calculates the storage space, and then selects the most efficient method, so it can be used in different storage spaces with the highest efficiency, while meeting functional requirements and being flexible and versatile in use.

[0040] 3. Easy to use on various smart instruments

[0041] The present invention is implemented on an FPGA unit and can be easily applied to data processing instruments, optical instruments and intelligent instruments.

[0042] 4. Simple structure, easy to operate, low prediction cost

[0043] All algorithms of the present invention are implemented on the FPGA unit and are automatically executed after operation without human intervention. The peripheral devices are relatively few, the structure is simple, and the prediction cost is low.

[0044] The technical features of an adaptive averaging noise removal method of the present invention are further described below in conjunction with the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 : System block diagram of the adaptive average noise removal system described in Example 1;

[0046] Figure 2 : Schematic diagram of the linear cumulative average principle of the present invention,

[0047] Figure 3 : Schematic diagram of the recursive cumulative averaging principle of the present invention,

[0048] In the above drawings, the parts are numbered as follows:

[0049] 1- Input analog signal, 2- Signal conditioning unit, 3- ADC unit, 4- FPGA unit,

[0050] 5-USB unit, 6-RAM unit, 7-trigger signal. DETAILED DESCRIPTION

[0051] Embodiment 1

[0052] An adaptive average noise removal system, comprising:

[0053] A signal conditioning unit 2 for converting a single-channel signal into a differential signal,

[0054] An ADC unit 3 for converting analog signals into digital signals,

[0055] FPGA unit 4 for implementing the core control of the adaptive average noise removal method,

[0056] ARM unit 6 for realizing communication with FPGA and human-computer interaction;

[0057] The input end of the signal conditioning unit 2 is connected to the input analog signal 1, the output end of the signal conditioning unit 2 is connected to the input end of the ADC unit 3, the output end of the ADC unit 3 is connected to the input end of the FPGA unit 4, and the FPGA unit 4 and the ARM unit 6 are connected to each other.

[0058] like Figure 1 As shown, the input analog signal 1 is converted into a differential signal by the signal conditioning unit 2 - AD8138, and then converted into a digital signal by the ADC unit 3, which is collected and processed by the FPGA unit 4, and finally outputted through the UBS unit 5 communication interface and displayed on the ARM unit 6. The cumulative average times and data volume can be set on the display interface of the ARM unit 6.

[0059] Embodiment 2

[0060] An adaptive averaging noise removal method, the method comprising the following steps:

[0061] S1, first send instructions to the FPGA unit to set the number of accumulations n and the amount of real-time data collection m;

[0062] S2, the FPGA unit as the core controller receives and parses the data, the received data includes receiving instructions sent by the ARM unit or instructions sent by any other device communicating with the FPGA unit; then based on the cumulative number n and the amount of real-time collected data m, automatically calculate and select the most efficient average noise removal method, that is, the fastest average noise removal method under the existing storage resources; the existing storage resources are the internal storage resources of the FPGA or the storage resources of the external high-speed memory, and the storage resource size is A, in MB;

[0063] The automatic calculation and selection of the most efficient average denoising method is as follows:

[0064] According to the set number of accumulations n and the amount of real-time collected data m, it is automatically calculated whether the existing storage resources meet the requirements of linear cumulative averaging denoising. If not, a combination of linear cumulative averaging and recursive cumulative averaging is used. By analyzing the number of averages and calculating the storage space, it is divided into multiple groups, so that the linear cumulative averaging algorithm is implemented within each group, and the recursive cumulative averaging algorithm is used between each group. The two are used together to achieve the highest efficiency.

[0065] The specific steps of automatically calculating and selecting the most efficient average denoising method are as follows:

[0066] S21. Based on the set accumulation times n and the real-time data collection volume m, the memory space required by the average noise removal algorithm with the shortest time is calculated. The calculation formula is as follows:

[0067]

[0068] Where L is the bit width of the ADC unit, that is, the bit width of the collected data;

[0069] S22, compare the memory space B required by the average denoising algorithm with the shortest time with the storage resource A, and determine which average denoising method is most efficient:

[0070] When A≥B, the linear cumulative average algorithm is used to average the noise, as shown in formula ②:

[0071]

[0072] Where a ij is the collected data, i is the number of collected sequences, i.e., the average number of times, j is the number of data points collected each time, S ij is the average value;

[0073] When A < B, the calculated memory space B exceeds the existing storage resources A of the system. The exceeded storage resources are B - A. Adopting the way of trading time for space, the accumulation times are divided into C equal parts, each part is D times. Then, the linear cumulative average and the recursive cumulative average are combined and applied. The former S in Equation ⑤ is adopted within each equal part ij is implemented, that is, implemented by using the linear cumulative average algorithm shown in Equation ②; the latter S in Equation ⑤ is adopted between each equal part qt is implemented, that is, implemented by using the recursive cumulative average algorithm shown in Equation ④, so as to achieve the highest efficiency of the average denoising algorithm under limited storage space; specifically as follows:

[0074] When A < B,

[0075]

[0076]

[0077] where a qt is the input data, S qt is the average value, q is the qth average, and t is the number of each average;

[0078]

[0079] In the formula is the average value;

[0080] S3. Automatically select the average denoising method with the highest efficiency to perform real-time processing on a large amount of high-speed real-time acquisition data;

[0081] S4. The FPGA unit outputs the data processing result through the communication interface, and the result is displayed in the form of a chart in the ARM unit.

[0082] The following is a specific example of applying the adaptive average denoising method of the present invention to a distributed optical fiber sensing system:

[0083] In the distributed optical fiber sensing system, the multi-period signal for real-time acquisition and processing is triggered by a trigger signal with a certain period, and then the system performs acquisition. m signals are acquired in one trigger, and a total of n triggers are performed. The acquired data is:

[0084]

[0085] Perform cumulative averaging on the corresponding points of the data a ij , that is, perform cumulative averaging on each data point in each column. Each data point is cumulatively averaged n times, and there are m data points in total, so the data volume is very large. Perform cumulative averaging on the corresponding points of a ij , and adopt the linear cumulative average algorithm shown in Equation ②. Its schematic diagram is as Figure 2As shown, all data are accumulated and then averaged, using the recursive cumulative average algorithm shown in formula ④, as shown in the schematic diagram Figure 3 As shown, it is accumulating and averaging at the same time. The two algorithms are used in combination, and the n-times cumulative average is divided into multiple groups of data of equal length. The linear cumulative average algorithm shown in formula ② is used for processing within each group, and the recursive cumulative average algorithm shown in formula ④ is used for processing between each group. Now, the adaptive cumulative average denoising algorithm is used to process 100,000 and 200,000 data with a bit width of 12 collected by the distributed optical fiber sensing system. The average times are increased from 1024 to 65536 times in multiples. The test algorithm time and storage resources are as follows:

[0086]

Claims

1. An adaptive average noise removal method, characterized in that: The method includes the following steps: S1. First, set the accumulation times n and the real-time collected data volume m; S2. The FPGA unit serving as the core controller receives and parses the data. Based on the accumulation times n and the real-time collected data volume m, it automatically calculates and selects the most efficient average denoising method, that is, the method with the fastest speed of the average denoising method under the existing storage resources; S3. Use the automatically selected most efficient average denoising method to perform real-time processing on the real-time collected data; S4. The FPGA unit outputs the data processing result through the communication interface, and the result is displayed in the form of a chart in the ARM unit; In step S2, the method of automatically calculating and selecting the most efficient average denoising method is as follows: Automatically calculate whether the existing storage resources meet the linear cumulative average denoising according to the set accumulation times n and the real-time collected data volume m. If not, use the combination of linear cumulative average and recursive cumulative average. Through the analysis and calculation of the average times and the collected data volume, when the storage resources are met, it is split into multiple groups, so that the linear cumulative average algorithm is implemented within each group, and the recursive cumulative average algorithm is used between each group, and the two are used together to achieve the highest efficiency.

2. The method for adaptive averaging noise removal according to claim 1, characterized in that: The existing storage resources are the internal storage resources of the FPGA or an external memory, and the size of its storage resources is A, with the unit of MB.

3. The method for adaptive averaging noise removal according to claim 2, characterized in that: The specific steps of the method of automatically calculating and selecting the most efficient average denoising method are as follows: S21. Based on the set accumulation times n and the real-time collected data volume m, calculate the memory space required by the average denoising algorithm with the least time consumption. The calculation formula is as follows: Where L is the bit width of the ADC unit, that is, the bit width of the collected data; S22. Compare the calculated memory space B required by the average denoising algorithm with the least time consumption with the storage resource A to determine which average denoising method has the highest efficiency: When A≥B, use the linear cumulative average algorithm for average denoising, as shown in formula ②: Where a ij is the collected data, i is the number of collected sequences, i.e., the average number of times, j is the number of data points collected each time, S ij is the average value; When A < B, the calculated memory space B exceeds the existing storage resources A of the system. The exceeded storage resources are B - A. By adopting the way of trading time for space, the number of accumulation times is divided into C equal parts, each equal part is D times. Then, the linear cumulative average and the recursive cumulative average are combined and applied. In each equal part, the former S in Equation ⑤ is ij implemented, that is, implemented by using the linear cumulative average algorithm shown in Equation ②; between each equal part, the latter S in Equation ⑤ is qt implemented, that is, implemented by using the recursive cumulative average algorithm shown in Equation ④, so as to achieve the highest efficiency of the average denoising algorithm under limited storage space; specifically as follows: When A<B, where a qt is the input data, S qt is the average value, q is the qth average, and t is the number of averages each time; In the formula is the average value.

4. The method for adaptive averaging noise removal according to claim 1, characterized in that: In step S1, set the accumulation times n and the real-time collected data volume m by sending instructions to the FPGA unit.

5. The method for adaptive averaging noise removal according to claim 1, characterized in that: In step S2, the data received by the FPGA unit includes receiving instructions sent by the ARM unit or instructions sent by any other communication device with the FPGA unit.

6. The method for adaptive averaging noise removal according to any one of claims 1 to 5, characterized in that: This method is implemented based on an adaptive average denoising system, and the adaptive average denoising system includes: A signal conditioning unit for converting a single-channel signal into a differential signal, An ADC unit for converting an analog signal into a digital signal, An FPGA unit for implementing the core control of the adaptive average denoising method, An ARM unit for implementing communication with the FPGA and human-computer interaction; The input end of the signal conditioning unit is connected to the input analog signal, the output end of the signal conditioning unit is connected to the input end of the ADC unit, the output end of the ADC unit is connected to the input end of the FPGA unit, and the FPGA unit and the ARM unit are connected to each other.

Images