Massive cable machine real-time monitoring system hoisting weight data denoising processing method

Abstract

The present application belongs to the field of data processing, and relates to a kind of mass cable machine real-time monitoring system hoisting weight data noise reduction processing method, comprising: 1) obtaining cable machine real-time monitoring system hoisting weight database;2) grouping hoisting weight database, extracting the data of cable machine real-time operation in each group database;3) the data obtained by extraction is screened, and the data of the stable stage of the hoisting weight of the object transported by cable machine is obtained;4) the data obtained by extracting the stable stage of the hoisting weight of the object transported by cable machine is denoised, and the data after denoising is obtained;5) the data after denoising obtained is imported into cable machine real-time monitoring system hoisting weight database, and the noise reduction processing of mass cable machine real-time monitoring system hoisting weight data is completed.The present application has the advantages that the hoisting weight data of mass cable machine real-time monitoring system can be efficiently and automatically denoised.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of data processing, and relates to a mass data noise reduction processing method, in particular to a mass cable machine real-time monitoring system hoisting weight data noise reduction processing method based on a low-pass digital filter. BACKGROUND

[0002] With the practice and development of intelligent dam construction theory, in order to realize the standardized, refined and intelligent management of the cable machine, and the need to master the actual operation of the cable machine, a cable machine real-time monitoring system is developed, which has the characteristics of real-time, accuracy and high refresh rate. However, due to the high sampling rate of the monitoring system, a large amount of data with complex content is generated, and only data collection and simple processing are carried out in actual engineering. However, due to the influence of wind, acceleration, deceleration and avoidance factors during the actual operation of the cable machine, these interference factors reflected in the cable machine real-time monitoring system will cause noise data in the cable machine operation data. The noise of the cable machine operation data is mainly distributed in the hoisting weight value. When the cable machine is affected by the above factors during operation, there may be a short-term fluctuation in the stable operation state. This fluctuation is more obvious during the transportation of the cable machine, and has little effect during loading and unloading. For the hoisting weight change caused by noise data, it is not conducive to the analysis of the cable machine operation data. If the noise data in the operation process of the cable machine can be designed with a corresponding digital filter for effective noise reduction, the accuracy of the analysis of the cable machine operation data can be greatly improved.

[0003] In the existing noise reduction method, Ning Wuteng et al. (Ning Wuteng, Baihetan Arch Dam Cable Machine Operation Mode Recognition and Efficiency Analysis [D]. Three Gorges University, 2021) proposed a method of using median filtering to reduce the noise of the cable machine hoisting weight data. This method can effectively reduce the noise of the data while not destroying the original trend of the data, and has certain engineering practicability. However, the noise data in the cable machine hoisting weight data is not completely caused by accidental factors, so the median filtering cannot completely filter out the noise data, and the median filtering belongs to a sliding filter, and the filtered data will have a time shift phenomenon, which will affect the analysis of the cable machine operation data. This is an unavoidable disadvantage of this method. SUMMARY

[0004] In order to solve the above problems in the background art, the present application proposes a method for efficiently and automatically reducing the noise of the hoisting weight data of the mass cable machine real-time monitoring system.

[0005] In order to achieve the above purpose, the present application adopts the following technical scheme:

[0006] A mass cable crane real-time monitoring system hoisting weight data denoising processing method, characterized in that: the mass cable crane real-time monitoring system hoisting weight data denoising processing method comprises the following steps:

[0007] 1) Obtain the cable crane real-time monitoring system hoisting weight database;

[0008] 2) Group the hoisting weight database obtained in step 1) according to the hoisting weight range, and extract the data of the real-time operation of the cable crane from the database according to the hoisting weight range of each group;

[0009] 3) Filter the data extracted in step 2) to obtain the data of the stable stage of the hoisting weight of the cable crane transported object;

[0010] 4) Denoise the data of the stable stage of the hoisting weight of the cable crane transported object obtained in step 3) to obtain denoised data;

[0011] 5) Import the denoised data obtained in step 4) into the cable crane real-time monitoring system hoisting weight database of step 1) to complete the mass cable crane real-time monitoring system hoisting weight data denoising processing.

[0012] Preferably, the specific implementation mode of step 2) provided by the present application is:

[0013] 2.1) Draw a cable crane hoisting weight histogram according to the hoisting weight database obtained in step 1);

[0014] 2.2) Determine the hoisting weight range according to the cable crane hoisting weight histogram;

[0015] 2.3) Group the hoisting weight database obtained in step 1) according to the hoisting weight range to form a grouped database;

[0016] 2.4) Extract the real-time operation data of the cable crane in the grouped database using SQL statements.

[0017] Preferably, the hoisting weight range in step 2.2) provided by the present application is 0.5t≤W≤8t, 8t<W<26t and 26t≤W, wherein W is the hoisting weight.

[0018] Preferably, the specific implementation mode of step 3) provided by the present application is:

[0019] 3.1) Perform differential processing on the data extracted in step 2) using SQL statements;

[0020] 3.2) Determine the working stage of the data extracted in step 2) by the differential value, wherein the working stage includes the rising stage, the falling stage and the stable stage;

[0021] 3.3) Label the data of each working stage using SQL statements;

[0022] 3.4) using SQL statements to filter out the data of non-stable hoisting phase, and obtaining the data of stable hoisting phase of the hoisting load of the cable crane transporting the object.

[0023] Preferably, the specific implementation of step 4) provided by the present application is:

[0024] 4.1) using SQL statements to extract the data of stable hoisting phase of the hoisting load of the cable crane transporting the object obtained in step 3);

[0025] 4.2) using the Scipy library of Python to construct a low-pass digital filter;

[0026] 4.3) inputting the data extracted in step 4.1) into the low-pass digital filter constructed in step 4.2) for filtering, and obtaining the data after noise reduction.

[0027] Preferably, the specific implementation of step 4.3) provided by the present application is:

[0028] 4.3.1) performing FFT transformation on the data extracted in step 4.1);

[0029] 4.3.2) performing low-pass filtering processing on the data after FFT transformation;

[0030] 4.3.3) performing IFFT transformation on the data after low-pass filtering processing, and obtaining the data after noise reduction.

[0031] Preferably, the way of performing discrete FFT transformation in step 4.3.1) provided by the present application is:

[0032]

[0033] Wherein:

[0034] x(n) is a finite-length data sequence, and the length is N;

[0035] X(k) is a finite-length sequence after discrete FFT transformation of x(n), and the length is N;

[0036] Wherein j represents the imaginary unit.

[0037] Preferably, the way of performing discrete IFFT transformation in step 4.3.3) provided by the present application is:

[0038]

[0039] Preferably, the specific implementation of step 5) provided by the present application is:

[0040] The denoised data obtained in step 4) is introduced into the cable machine real-time monitoring system load database in step 1) by using an SQL statement, so as to replace the unfiltered data in the original cable machine real-time monitoring system load database, and the massive cable machine real-time monitoring system load data denoising processing is completed.

[0041] Compared with the prior art, the present application has the following advantages:

[0042] The present application discloses a massive cable machine real-time monitoring system load data denoising processing method, which comprises the following steps: firstly, grouping the data according to the load range in the database, and extracting the data of the real-time operation of the cable machine; secondly, screening out the data of the rising and falling stages of the load in the extracted data, and only retaining the data of the stable stage of the load of the transported object of the cable machine; thirdly, introducing the screened data into a low-pass digital filter for denoising; and finally, introducing the denoised data into the database. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 It is a flow chart of the massive cable machine real-time monitoring system load data denoising processing method provided by the present application;

[0044] Figure 2 It is a part of the original load data in the database;

[0045] Figure 3 It is a histogram of the load data of the cable machine;

[0046] Figure 4 It is a schematic diagram of the screened stable stage data adopted by the present application;

[0047] Figure 5 It is an amplitude-frequency graph of part of the unfiltered data of the group greater than 26t;

[0048] Figure 6 It is an amplitude-frequency graph of part of the filtered data of the group greater than 26t;

[0049] Figure 7 It is a comparison graph of part of the filtered data and the unfiltered data of the group greater than 26t;

[0050] Figure 8It is a part of the database original data low-pass filter filter and filter before the comparison chart of hoisting weight data. DETAILED DESCRIPTION

[0051] Referring to Figure 1 The present application provides a mass cable crane real-time monitoring system hoisting weight data noise reduction processing method, the core idea of the method is: first, according to the hoisting weight range is divided into 0.5-8t, 8-26t and greater than 26t three groups, respectively from the database extraction of the three groups of data; Then filter out the hoisting weight rising and falling stage in the extracted data, only keep the data of the hoisting weight stable stage; Then the data after extraction, import to the low-pass digital filter constructed by Python Scipy library for noise reduction processing; Finally, the data after noise reduction is imported into the database, replacing the original hoisting weight data. The present application comprehensively considers the characteristics of the cable crane hoisting weight noise data, and the low-pass digital filter is constructed by Pyhon Scipy library to process the cable crane hoisting weight data. The noise reduction processing can be realized efficiently, automatically and scientifically. Part of the original data in the database is shown in Figure 2

[0052] Specifically, the mass cable crane real-time monitoring system hoisting weight data noise reduction processing method provided by the present application comprises the following steps:

[0053] Step 1, grouping according to the hoisting weight range, extracting the data of the real-time operation of the cable crane from the database.

[0054] The present application first divides the hoisting weight range into 0.5-8t, 8-26t and greater than 26t three groups of data, and then extracts the three groups of data from the database in turn. The hoisting weight range is divided mainly according to the cable crane hoisting weight histogram as shown in Figure 3 From the figure, it can be seen that the cable crane hoisting weight data is approximately 0-0.5t, 5-8t and 26-30t. The first group of hoisting weight range is divided into 0.5-8t because when the cable crane hoisting weight data is less than 0.5t, the cable crane is mostly in an empty load state, i.e. without hoisting any object, so the grouping of 0.5-8t can ensure that there is no a large amount of invalid data in the extracted range. The second group of hoisting weight range is divided into 8-26t to ensure that the middle section data is extracted. The third group of hoisting weight range is divided into greater than 26t to extract the remaining data.

[0055] Step 2, filter out the hoisting weight rising stage and falling stage data in the extracted data, and only keep the hoisting weight stable stage data of the cable crane transporting object.

[0056] 2.1) The present application uses SQL statements to perform difference processing on the cable crane hoisting weight data, and the calculation method is:

[0057]

[0058] ​Wherein:

[0059] is the hoist first-order difference value, i.e. hoist change rate;

[0060] W n is the nth row hoist;

[0061] W n+1 is the n+1th row hoist;

[0062] t n is the n th row acquisition time;

[0063] t n+1 is the n+1th row acquisition time;

[0064] Δt is the time difference between the two rows.

[0065] 2.2) Then judge the cable hoist data to be in the rising, falling and stable stages by the difference value, and mark the data of each stage by using the SQL statement, and the judgment standard is as shown in Table 1

[0066] Table 1 Judgment table of cable hoist rising, falling, stable and empty load stages

[0067] Phase Label DW > 0.2 Hoist up 1 DW < -0.2 Hoist down 3 -0.2 < DW < 0.2 Hoist steady 0 W<0.5 Idle phase /

[0068] 2.3) Finally, the data in the non-hoist stable stage is screened out by using the SQL statement, i.e. the data in the hoist rising and falling stages in step 1 is divided into three groups according to the hoist range. The following will specifically explain how to exclude the hoist rising and falling stage data in the data.

[0069] As Figure 4 shown, most of the data divided according to the hoist range greater than 26t is marked as 0, and the hoist rising marked as 1 and the hoist falling marked as 3 data is not very obvious. In order to prevent effective data from being screened out, the actual extraction of the third group of data is actually extracted according to the hoist range greater than 20t, the purpose is to ensure that there are obvious hoist rising and falling stages in the third group of data, avoid the data needed to be extracted in step 2 from being screened out, and finally the data marked as 0 is screened out by using the SQL statement, and the hoist range greater than 26t is limited to the screened data, which can maximize the extracted data not to repeat the second group of data. The data screening process of the other two groups is to extract the data according to the hoist division range, and the data screening method is similar to the above process, and the specific process is shown in Figure 4 .

[0070] Step 3, the screened data is imported into the constructed low-pass digital filter to reduce noise.

[0071] This paper uses the Fast Fourier Transform and Inverse Transform functions in the Python Scipy library to construct a low-pass data filter. The specific calculation method is as follows:

[0072]

[0073] in:

[0074] x(n) is a finite data sequence with length N;

[0075] X(k) is a finite length sequence after the discrete FFT transform of x(n), and its length is also N;

[0076] Where j represents the imaginary unit.

[0077] Taking the group with a lifting weight greater than 26t as an example, the data of the stable stage of the lifting weight is extracted from the database, and the discrete FFT transformation is performed using formula (1). The amplitude of the reorganized data can be calculated and plotted as an amplitude-frequency diagram as shown below: Figure 5 Then select the lower frequency part, retain the lower frequency part and filter out the higher frequency part, that is, the spectrum obtained after low-pass filtering the data is as follows Figure 6 shown.

[0078]

[0079] Then, the filtered data is subjected to discrete IFFT transformation using formula (2) to obtain the filtered lifting weight data. The comparison diagram before and after filtering is as follows: Figure 7 shown.

[0080] Step 4: Import the noise-reduced data into the original database. Import the filtered data into the original database using SQL statements and replace the original unfiltered data to achieve noise reduction processing of the massive cable crane real-time monitoring system hoisting data. The original cable crane hoisting data, the data after low-pass filter filtering, is as follows: Figure 8 shown.

[0081] In order to further illustrate the noise reduction effect of the method of the present invention on cable crane hoisting data, some data were extracted from the database and compared and verified using three sliding filters, namely mean filtering, median filtering and Gaussian filtering, and a low-pass filter (the method provided by the present invention).

[0082] Numerical experiment: In order to verify the filtering effect of the method of the application, the window number of the sliding filter is set to 5, the first order Gaussian kernel of the Gaussian filter is 0.053991, 0.241971, 0.398942, 0.241971, 0.053991. The lowest frequency part is selected for filtering in the low-pass filter. About 500 rows of data from 2018-10-01 0:00:00 to 2018-10-01 0:45:15 are extracted from the data as a data set for testing the filtering effect of different filters. Table 2 shows the filtering effect of different filtering methods on the original hoisting weight data.

[0083] Table 2 Comparison of filtering effects of different filters

[0084] Raw data Mean filter Median filter Gaussian filter Low pass filter Error rate 18.6％ 10.4％ 7.4％ 4.8％ 0.6％ RMSE / 3.14 1.92 0.64 0.32 SNR / 15.69 19.956 29.52 35.40 Hoist up 1 4 6 0 3 Hoist steady 92 45 23 24 0 Hoist down 0 3 8 0 0

[0085] From the results of the numerical experiment, the performance of the Gaussian filter is the best among the sliding filters (Gaussian filter, mean filter and median filter), the error rate of the filtered data is the lowest, the RMSE is also the lowest, and the SNR value is also greater than the other two sliding filters, which shows that the Gaussian filter can filter out noise data while maximizing the preservation of the trend of the original data. Although the filtering effect of the Gaussian filter is good among the sliding filters, compared with the low-pass filter used in the application, the filtering effect is better than that of the Gaussian filter, the error rate and RMSE of the filtered data are respectively reduced by 87.5% and 50% compared with the Gaussian filter, and the SNR is increased by 19.91% compared with the Gaussian filter. It shows that the low-pass filter can filter out a large amount of noise data while preserving as much information of the original data as possible, and realizes scientific and effective noise reduction.

Claims

1. A method for noise reduction processing of hoisting weight data in a massive cable crane real-time monitoring system, characterized by: The following steps are involved: 1) Obtain the lifting weight database of the cable crane real-time monitoring system; 2) The lifting weight database obtained in step 1) is grouped by lifting weight range, and the real-time operation data of the cable crane is extracted from the database according to the lifting weight range of each group. The specific implementation method is: 2.1) Drawing a cable crane lifting weight histogram based on the lifting weight database obtained in step 1); 2.2) Determine the lifting weight range based on the cable crane lifting weight histogram, wherein the lifting weight range is 0.5t≤W≤8t, 8t<W<26t, and 26t≤W, where W is the lifting weight; 2.3) Grouping the lifting weight database obtained in step 1) according to the lifting weight range to form a grouped database; 2.4) Use SQL statements to extract the real-time operation data of the cable crane from the group database; 3) Filter the data extracted in step 2) to obtain data during the stable lifting phase of the cable car transporting objects. The specific implementation method is: 3.1) Using SQL statements to perform differential processing on the data extracted in step 2); 3.2) determining the working stage of the data extracted in step 2) by the difference value, wherein the working stage includes an ascending stage, a descending stage, and a stable stage; 3.3) Use SQL statements to mark the data of each work stage; 3.4) Use SQL statements to filter out data from the non-stability phase and obtain data from the stable phase of cable-lift transported objects; 4) De-noising the data of the stable stage of the cable crane transporting the object obtained in step 3) to obtain de-noised data, which is specifically implemented as follows: 4.1) Using SQL statements to extract the data obtained in step 3) during the stable stage of the cable crane transporting the object; 4.2) Use Python's Scipy library to construct a low-pass digital filter; 4.3) The data extracted in step 4.1) is input into the low-pass digital filter constructed in step 4.2) for filtering to obtain the noise-reduced data. The specific implementation method is: 4.3.1) performing a discrete FFT transform on the data extracted in step 4.1); 4.3.2) Perform low-pass filtering on the FFT-transformed data; 4.3.3) Performing discrete IFFT transform on the low-pass filtered data to obtain denoised data; 5) Importing the noise-reduced data obtained in step 4) into the cable crane real-time monitoring system hoisting weight database of step 1) to complete the noise reduction processing of the massive cable crane real-time monitoring system hoisting weight data, specifically implemented as follows: Use SQL statements to import the denoised data obtained in step 4) into the cable crane real-time monitoring system weight database in step 1), replacing the unfiltered data in the original cable crane real-time monitoring system weight database, and complete the denoising processing of massive cable crane real-time monitoring system weight data.

2. The method for noise reduction of hoisting weight data in a massive cable crane real-time monitoring system according to claim 1 is characterized in that: The method of performing discrete FFT transformation in step 4.3.1) is: in: x(n) is a finite data sequence with length N; X(k) is a finite length sequence after the discrete FFT transform of x(n), with a length of N; Where j represents the imaginary unit.

3. The method for noise reduction of hoisting weight data in a massive cable crane real-time monitoring system according to claim 1 is characterized in that: The method of performing discrete IFFT transformation in step 4.3.3) is:

Images