Waveform file generation method and system

Abstract

The invention discloses a waveform file generation method and system. The waveform file generation method comprises the following steps: acquiring and processing original signal data through a proper sensor and a data acquisition system; converting the signal data of the time domain into frequency domain data; according to an FFT result, main frequency components are extracted, and difference coding is carried out on the selected main frequency components; the compressed data and necessary metadata are packaged together; storing the compressed data file to a local or cloud server; according to the method, the time domain data is converted into the frequency domain data through the fast Fourier transform, so that the data can be analyzed and compressed more easily, the requirements for data storage and transmission are further reduced through difference value coding, the frequency domain data can be accurately recovered to the time domain data through the inverse fast Fourier transform, and the accuracy of the data storage and transmission is improved. The data processing and uploading speed is increased, the data management efficiency and safety are enhanced, and efficient processing of large-scale data is supported.

Description

Technical Field

[0001] The present invention relates to the technical field of power systems, and in particular to a waveform file generation method and system. Background Art

[0002] A waveform file is a digital file obtained by collecting the mechanical vibrations of various sounds. The characteristic of a waveform file is that it can preserve and reproduce the effect of the original sound source. It is often used to record natural sounds such as music and songs, but the file storage space is relatively large.

[0003] WAV format is a sound file format developed by Microsoft, also called waveform sound file. It is the earliest digital audio format and is widely supported by Windows platform and its applications. WAV format supports many compression algorithms, multiple audio bit rates, sampling frequencies and channels, and adopts 44.1kHz sampling frequency and 16-bit quantization bit rate. Therefore, the sound quality of WAV is almost the same as that of CD, but WAV format requires too much storage space and is not convenient for communication and dissemination. There are also other wave file formats such as WAV, AU, VOC, etc.

[0004] The primary and secondary integrated equipment using wireless communication all use the IEC60870-5-101 protocol (hereinafter referred to as the 101 protocol) for communication, and the primary and secondary integrated equipment using optical fiber communication all use the IEC60870-5-104 protocol (hereinafter referred to as the 104 protocol) for communication. The data shows that the waveform file size is between 35 and 52kB, but the average call time for the primary and secondary integrated pole-mounted circuit breaker is about 8 minutes, and the transmission speed is extremely slow. Summary of the invention

[0005] The purpose of the present invention is to provide a waveform file generation method and system to solve the problems raised in the above background technology.

[0006] To achieve the above object, a method for generating a waveform file comprises the following steps:

[0007] S1. Obtain and process raw signal data through appropriate sensors and data acquisition systems;

[0008] S2, converting the signal data in the time domain into frequency domain data;

[0009] S3, extracting the main frequency components according to the FFT results, and performing difference encoding on the selected main frequency components;

[0010] S4, encapsulating the compressed data with necessary metadata;

[0011] S5, storing the compressed data file locally or in a cloud server, or transmitting the compressed waveform file to another system or location via a network as needed;

[0012] S6. The receiver uses the inverse fast Fourier transform to restore the data to the time domain when necessary, and checks the integrity and accuracy of the restored data.

[0013] Preferably, the data processing in step S1 specifically includes performing necessary filtering on the collected raw data to remove noise and interference.

[0014] Preferably, in step S2, the specific calculation formula for converting the signal data in the time domain into the frequency domain data is:

[0015]

[0016] Among them, x(n) is the time domain signal, X(k) is the frequency domain signal, and N is the total number of samples.

[0017] Preferably, in step S6, the formula for inverse fast Fourier transform is:

[0018]

[0019] Among them, x(n) is the time domain signal, X(k) is the frequency domain signal, and N is the total number of samples.

[0020] A waveform file generation system includes a data conversion and compression module, a metadata management and optimized storage module, an asynchronous data processing and real-time feedback module, and a data recovery and verification module;

[0021] The data conversion and compression module is used to convert the signal from the time domain to the frequency domain using FFT, and to achieve data compression by selecting important frequency components and applying difference coding;

[0022] The metadata management and optimization storage module is used to create and manage metadata;

[0023] The asynchronous data processing and real-time feedback module is used to optimize the data processing process;

[0024] The data recovery and verification module is used to recover the data from the compressed format to its original state through inverse FFT, and to verify the integrity and accuracy of the recovered data.

[0025] Preferably, the data conversion and compression module includes an FFT conversion unit and a difference compression unit;

[0026] The FFT conversion unit is used to convert the input time domain waveform data into frequency domain data;

[0027] The difference compression unit is used to calculate the difference between adjacent frequency points in the frequency domain data through a difference coding algorithm, and replace the difference that appears at high frequency with a shorter code.

[0028] Preferably, the metadata management and optimization storage module includes a metadata indexing unit and a data encapsulation unit;

[0029] The metadata indexing unit is used to generate an index and metadata for each segment of compressed data;

[0030] The data encapsulation unit is used to encapsulate the compressed data and its metadata into a single file.

[0031] Preferably, the asynchronous data processing and real-time feedback module comprises an asynchronous processing unit and a real-time monitoring unit;

[0032] The asynchronous processing unit is used to implement asynchronous execution of data processing on the server;

[0033] The real-time monitoring unit is used to monitor the data processing process and provide real-time feedback to the user.

[0034] Preferably, the data recovery and verification module includes an IFFT inverse transformation unit and an integrity verification unit;

[0035] The IFFT inverse transform unit is used to restore the original time domain waveform data from the compressed frequency domain data;

[0036] The integrity verification unit is used to perform integrity verification on the restored data.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] 1. The present invention converts time domain data into frequency domain data through fast Fourier transform (FFT), so that the data can be more easily analyzed and compressed. The differential coding further reduces the demand for data storage and transmission because it only records the changes between frequency points rather than the complete data of each frequency point. The inverse fast Fourier transform (IFFT) can accurately restore the time domain data from the frequency domain data to ensure the integrity of the data, which not only improves the speed of data processing and uploading, but also enhances the management efficiency and security of the data, and supports the efficient processing of large-scale data.

[0039] 2. The asynchronous data processing module of the present invention allows the system to process data without interfering with the main operation process, which can effectively improve the overall performance and response time of the system. The real-time feedback module ensures that the user can obtain the processing progress and status in real time, increasing the transparency of the system and the user's sense of control. The system design supports data storage and processing in the cloud, as well as the transmission of waveform files over the network, providing users with flexibility and scalability.

[0040] 3. In the process of waveform generation of the present invention, the size of the waveform file is reduced by one-half through the compression algorithm. In the process of waveform transmission from the core computing module to the wireless module, by upgrading the serial port to the network port, the transmission speed of the network port protocol is one-third faster than the serial port. In the process of waveform transmission from the wireless module to the main station, by upgrading the main station IEC60870-5-101 protocol to IEC60870-5-104 protocol, and adjusting the 1 question 1 answer mechanism to a 200 question 1 answer mechanism. The comtrade waveform file upload time is upgraded from the original 8 minutes to 20 seconds. The upgrade process does not require power outages, and the main station and terminal hardware are not changed. This technology can be applied to other fields that require waveform uploads, such as seismology, acoustics, etc., with a wide range of applications and strong practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a step diagram of the method of the present invention;

[0042] Figure 2 It is a module flow chart of the present invention;

[0043] Figure 3 This is a diagram of the difference compression algorithm based on FFT of the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] See also Figure 1 and Figure 3 The present invention provides a technical solution: a method for generating a waveform file, comprising the following steps:

[0046] S1. Obtaining raw signal data, including waveform signals such as sound, electromagnetic waves and vibration, through appropriate sensors and data acquisition systems, and performing necessary filtering on the collected raw data to remove noise and interference, can improve the quality and availability of data;

[0047] S2. Convert the signal data in the time domain into frequency domain data. FFT can effectively analyze the frequency components in the data and provide a basis for subsequent data compression.

[0048] Input: waveform data x(n) in the time domain; Output: waveform data X(k) in the frequency domain, which consists of complex values ​​of different frequency components;

[0049] The formula for FFT is:

[0050]

[0051] Among them, x(n) is the time domain signal, X(k) is the frequency domain signal, and N is the total number of samples;

[0052] S3, according to the FFT result in step S2, extract the main frequency components, ignore those frequency components with small amplitude and little influence on the overall information of the signal, and then perform difference encoding on the selected main frequency components, record the difference between the frequencies instead of the absolute value, so as to reduce the amount of data;

[0053] S4. Encapsulate the compressed data with necessary metadata (such as sampling rate, timestamp, original file size, etc.). Metadata is critical for decompressing and restoring data to its original state. Establish an effective metadata management strategy to ensure the integrity and accuracy of data during decompression and processing.

[0054] S5, storing the compressed data file locally or in a cloud server, or transmitting the compressed waveform file to another system or location via a network as needed;

[0055] S6. The receiver uses the inverse fast Fourier transform to restore the data to the time domain when necessary, and checks the integrity and accuracy of the restored data. The IFFT formula is:

[0056]

[0057] Among them, x(n) is the time domain signal, X(k) is the frequency domain signal, and N is the total number of samples.

[0058] See also Figure 2 , a waveform file generation system, including a data conversion and compression module, a metadata management and optimized storage module, an asynchronous data processing and real-time feedback module, and a data recovery and verification module;

[0059] The data conversion and compression module is used to convert the signal from the time domain to the frequency domain using FFT, and to achieve data compression by selecting important frequency components and applying differential coding. The data conversion and compression module includes an FFT conversion unit and a differential compression unit. The time domain data is converted into frequency domain data by fast Fourier transform (FFT), so that the data can be more easily analyzed and compressed. The differential coding further reduces the demand for data storage and transmission because it only records the changes between frequency points, rather than the complete data of each frequency point.

[0060] The FFT conversion unit is used to convert the input time domain waveform data into frequency domain data; the difference compression unit is used to calculate the difference between adjacent frequency points in the frequency domain data through a difference coding algorithm, and replace the high-frequency difference with a shorter code.

[0061] The metadata management and optimization storage module is used to create and manage metadata. The metadata management and optimization storage module includes a metadata index unit and a data encapsulation unit;

[0062] The metadata indexing unit is used to generate an index and metadata for each segment of compressed data; the data encapsulation unit is used to encapsulate the compressed data and its metadata into a single file.

[0063] The asynchronous data processing and real-time feedback module is used to optimize the data processing process. The asynchronous data processing and real-time feedback module includes an asynchronous processing unit and a real-time monitoring unit;

[0064] The asynchronous processing unit is used to implement asynchronous execution of data processing on the server; the real-time monitoring unit is used to monitor the data processing process and provide real-time feedback to the user.

[0065] The data recovery and verification module is used to restore the data from the compressed format to its original state through inverse FFT, and verify the integrity and accuracy of the restored data. The data recovery and verification module includes an IFFT inverse transformation unit and an integrity verification unit;

[0066] The IFFT inverse transform unit is used to restore the original time domain waveform data from the compressed frequency domain data; the integrity verification unit is used to perform integrity verification on the restored data. Through the inverse fast Fourier transform (IFFT), the frequency domain data can be accurately restored to the time domain data to ensure the integrity of the data.

[0067] In summary: The present invention reduces the size of waveform files by one-half through a compression algorithm. In the process of transmitting the waveform from the core computing module to the wireless module, the serial port is upgraded to a network port, and the transmission speed of the network port protocol is one-third faster than that of the serial port. In the process of transmitting the waveform from the wireless module to the main station, the main station IEC60870-5-101 protocol is upgraded to the IEC60870-5-104 protocol, and the 1 question 1 answer mechanism is adjusted to a 200 questions 1 answer mechanism. The comtrade waveform file upload time is upgraded from the original 8 minutes to 20 seconds. The upgrade process does not require power outages, and the main station and terminal hardware do not change. This technology can be applied to other fields that require waveform uploads, such as seismology, acoustics, etc., with a wide range of applications and strong practicality.

[0068] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for generating a waveform file, characterized in that: The steps include: S1. Obtain and process raw signal data through appropriate sensors and data acquisition systems; S2, converting the signal data in the time domain into frequency domain data; S3, extracting the main frequency components according to the FFT results, and performing difference encoding on the selected main frequency components; S4, encapsulating the compressed data with necessary metadata; S5, storing the compressed data file locally or in a cloud server, or transmitting the compressed waveform file to another system or location via a network as needed; S6. The receiver uses the inverse fast Fourier transform to restore the data to the time domain when necessary, and checks the integrity and accuracy of the restored data.

2. A method for generating a waveform file according to claim 1, characterized in that: The data processing in step S1 specifically includes performing necessary filtering on the collected raw data to remove noise and interference.

3. A method for generating a waveform file according to claim 1, characterized in that: In step S2, the specific calculation formula for converting the signal data in the time domain into the frequency domain data is: Among them, x(n) is the time domain signal, X(k) is the frequency domain signal, and N is the total number of samples.

4. A method for generating a waveform file according to claim 1, characterized in that: In step S6, the formula for inverse fast Fourier transform is: Among them, x(n) is the time domain signal, X(k) is the frequency domain signal, and N is the total number of samples.

5. A waveform file generation system, characterized in that: It includes data conversion and compression module, metadata management and optimized storage module, asynchronous data processing and real-time feedback module, and data recovery and verification module; The data conversion and compression module is used to convert the signal from the time domain to the frequency domain using FFT, and to achieve data compression by selecting important frequency components and applying difference coding; The metadata management and optimization storage module is used to create and manage metadata; The asynchronous data processing and real-time feedback module is used to optimize the data processing process; The data recovery and verification module is used to recover the data from the compressed format to its original state through inverse FFT, and to verify the integrity and accuracy of the recovered data.

6. A waveform file generation system according to claim 1, characterized in that: The data conversion and compression module includes an FFT conversion unit and a difference compression unit; The FFT conversion unit is used to convert the input time domain waveform data into frequency domain data; The difference compression unit is used to calculate the difference between adjacent frequency points in the frequency domain data through a difference coding algorithm, and replace the difference that appears at high frequency with a shorter code.

7. A waveform file generation system according to claim 1, characterized in that: The metadata management and optimization storage module includes a metadata index unit and a data encapsulation unit; The metadata indexing unit is used to generate an index and metadata for each segment of compressed data; The data encapsulation unit is used to encapsulate the compressed data and its metadata into a single file.

8. A waveform file generation system according to claim 1, characterized in that: The asynchronous data processing and real-time feedback module includes an asynchronous processing unit and a real-time monitoring unit; The asynchronous processing unit is used to implement asynchronous execution of data processing on the server; The real-time monitoring unit is used to monitor the data processing process and provide real-time feedback to the user.

9. A waveform file generation system according to claim 1, characterized in that: The data recovery and verification module includes an IFFT inverse transformation unit and an integrity verification unit; The IFFT inverse transform unit is used to restore the original time domain waveform data from the compressed frequency domain data; The integrity verification unit is used to perform integrity verification on the restored data.

Images