Multi-mode distribution network operation system based on image and sound information

Abstract

The invention discloses a multi-modal distribution network operation system based on image and sound information, and the system comprises a data collection module, a data preprocessing module, a multi-modal data fusion module, a fault detection module, and an alarm push module. The data acquisition module is used for acquiring sensing data including image parameters, sound parameters and electrical parameters of power distribution network equipment and transmitting the acquired sensing data to the data preprocessing module. According to the multi-modal distribution network operation system based on the image and sound information disclosed by the invention, linkage is carried out through the data acquisition module, the data preprocessing module, the multi-modal data fusion module, the fault detection module and the alarm pushing module, so that the multi-modal data fusion and intelligent analysis technology is adopted; and the operation management level and the fault detection capability of the power distribution network are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of power systems, and in particular relates to a multimodal distribution network operation system based on image and sound information. Background Art

[0002] As an important part of the power system, the distribution network has the characteristics of wide geographical distribution, large scale, many types of equipment, and complex operation mode. With the development of distribution automation and intelligence, a large amount of heterogeneous and polymorphic data is generated in the distribution network, and the traditional single fault data analysis technology can no longer meet the needs. In addition, the traditional fault detection and diagnosis methods rely on manual labor, which is inefficient and inaccurate.

[0003] Therefore, further improvements are made to the above problems. Summary of the invention

[0004] The main purpose of the present invention is to provide a multimodal distribution network operation system based on image and sound information, which is linked through a data acquisition module, a data preprocessing module, a multimodal data fusion module, a fault detection module and an alarm push module, thereby improving the operation management level and fault detection capability of the distribution network through multimodal data fusion and intelligent analysis technology.

[0005] To achieve the above objectives, the present invention provides a multimodal distribution network operation system based on image and sound information, including a data acquisition module, a data preprocessing module, a multimodal data fusion module, a fault detection module and an alarm push module, wherein:

[0006] The data acquisition module is used to collect sensor data including image parameters, sound parameters and electrical parameters of the distribution network equipment and transmit the collected sensor data to the data preprocessing module (data collection can be performed through collection devices such as cameras, sound sensors, current / voltage sensors and drones);

[0007] The data preprocessing module preprocesses each parameter in the sensor data, extracts effective features of each parameter, and inputs each effective feature into the multimodal data fusion module;

[0008] The multimodal data fusion module includes a feature fusion unit and a decision fusion unit. The feature fusion unit performs multivariate fusion on the effective features corresponding to the image parameters, the sound parameters and the electrical parameters, respectively, to form a multimodal feature vector. The fault detection module performs fault detection and classification on the multimodal feature vector through a trained deep learning model to generate the first fault information (the first fault information is to first fuse each effective feature and then output the fault information. The advantage is that the information of each modality can be fully utilized to improve the expression ability of the system. The disadvantage is that it may be affected by the difference between the modalities and requires an effective feature selection and fusion strategy); the decision fusion The combining unit transmits the effective features corresponding to the image parameters, the sound parameters and the electrical parameters to the fault detection module, respectively, so as to obtain the image decision fault result, the sound decision fault result and the electrical decision fault result, respectively, and matches and fuses each fault result through the trained deep learning model, so as to perform fault detection and classification to generate the second fault information (the second fault information is to obtain the fault results of each effective feature, fuse the fault results, and then output the final fault information. The advantage is that each mode is processed independently, which reduces the influence of the difference between modes and is easy to implement and understand. The disadvantage is that the complementary information between modes may not be fully utilized);

[0009] The fault detection module integrates and analyzes the first fault information and the second fault information to obtain final fault information, and transmits the fault information to a corresponding matching receiving object through the alarm push module.

[0010] As a further preferred technical solution of the above technical solution, for the image parameter preprocessing of the data preprocessing module:

[0011] First, perform image denoising. Perform median filtering (suitable for removing salt and pepper noise. Replace the pixel value of each pixel in the image with the median of the pixel values ​​in its neighborhood), Gaussian filtering (suitable for removing Gaussian noise. Smooth the image by performing Gaussian convolution on the image), and bilateral filtering (considering spatial proximity and pixel value similarity at the same time, and being able to retain edge details while denoising) on ​​the image parameters to denoise the image parameters.

[0012] Second, image enhancement is performed. The denoised image parameters are subjected to histogram equalization (enhancing the image contrast by adjusting the grayscale histogram of the image), Laplace enhancement (enhancing the image edge by calculating the Laplace transform of the image), and Retinex processing (enhancing the image details and contrast by multi-scale Retinex processing based on the visual characteristics of the human eye) to enhance the image parameters;

[0013] Third, feature extraction is performed to extract edge features (Canny edge detection algorithm can be used to extract edge information in the image), texture features (gray level co-occurrence matrix (GLCM) can be used to extract texture features of the image, such as contrast, energy, entropy, etc.) and key point features (SIFT (scale invariant feature transform) or ORB (oriented FAST and rotational BRIEF) algorithm can be used to extract key points and descriptors in the image) from the enhanced image parameters to obtain effective features of the image parameters.

[0014] As a further preferred technical solution of the above technical solution, for the sound parameter preprocessing of the data preprocessing module:

[0015] First, perform sound noise reduction, and perform frequency domain filtering (Fourier transform can be used to convert the sound signal into the frequency domain, and then low-pass, high-pass or band-pass filters can be applied to remove noise), adaptive filtering (adaptive algorithms (such as LMS algorithm) can be used to dynamically adjust filter parameters according to noise characteristics) and deep learning noise reduction (deep learning models (such as RNNoise) can be used for real-time noise suppression) to reduce the noise of sound parameters;

[0016] Second, feature extraction is performed. The spectral features (short-time Fourier transform (STFT) can be used to extract the spectrum of the sound signal), Mel spectrum (the spectrum can be converted into Mel spectrum, which is more in line with the auditory characteristics of the human ear), MFCC (MFCC features can be extracted by performing discrete cosine transform (DCT) on the Mel spectrum) and energy features (calculating the energy of each frame, zero-crossing rate and other features) are extracted from the denoised sound parameters to obtain effective features of the sound parameters.

[0017] As a further preferred technical solution of the above technical solution, for the electrical parameter preprocessing of the data preprocessing module:

[0018] First, the electrical parameters are normalized, cleaned and denoised.

[0019] Second, feature extraction is performed to process the processed electrical parameters in time domain (the mean, variance, peak and other time domain features of current and voltage signals can be calculated) and frequency domain (Fourier transform can be used to extract the spectral features of the signal, such as the main frequency, harmonic components, etc.) to obtain the effective features of the electrical parameters.

[0020] As a further preferred technical solution of the above technical solution, it also includes a storage module, which stores the data generated by the data acquisition module, the data preprocessing module, the multimodal data fusion module, the fault detection module and the alarm push module respectively. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a structural schematic diagram of the multimodal distribution network operation system based on image and sound information of the present invention. DETAILED DESCRIPTION

[0022] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art can think of other obvious variations. The basic principles of the present invention defined in the following description can be applied to other embodiments, variations, improvements, equivalents, and other technical solutions that do not deviate from the spirit and scope of the present invention.

[0023] In the preferred embodiments of the present invention, those skilled in the art should note that the power distribution network and the like involved in the present invention may be regarded as prior art.

[0024] Preferred embodiments.

[0025] like Figure 1 As shown, the present invention discloses a multimodal distribution network operation system based on image and sound information, including a data acquisition module, a data preprocessing module, a multimodal data fusion module, a fault detection module and an alarm push module, wherein:

[0026] The data acquisition module is used to collect sensor data including image parameters, sound parameters and electrical parameters of the distribution network equipment and transmit the collected sensor data to the data preprocessing module (data collection can be performed through collection devices such as cameras, sound sensors, current / voltage sensors and drones);

[0027] The data preprocessing module preprocesses each parameter in the sensor data, extracts effective features of each parameter, and inputs each effective feature into the multimodal data fusion module;

[0028] The multimodal data fusion module includes a feature fusion unit and a decision fusion unit. The feature fusion unit performs multivariate fusion on the effective features corresponding to the image parameters, the sound parameters and the electrical parameters, respectively, to form a multimodal feature vector. The fault detection module performs fault detection and classification on the multimodal feature vector through a trained deep learning model to generate the first fault information (the first fault information is to first fuse each effective feature and then output the fault information. The advantage is that the information of each modality can be fully utilized to improve the expression ability of the system. The disadvantage is that it may be affected by the difference between the modalities and requires an effective feature selection and fusion strategy); the decision fusion The combining unit transmits the effective features corresponding to the image parameters, the sound parameters and the electrical parameters to the fault detection module, respectively, so as to obtain the image decision fault result, the sound decision fault result and the electrical decision fault result, respectively, and matches and fuses each fault result through the trained deep learning model, so as to perform fault detection and classification to generate the second fault information (the second fault information is to obtain the fault results of each effective feature, fuse the fault results, and then output the final fault information. The advantage is that each mode is processed independently, which reduces the influence of the difference between modes and is easy to implement and understand. The disadvantage is that the complementary information between modes may not be fully utilized);

[0029] The fault detection module integrates and analyzes the first fault information and the second fault information to obtain final fault information, and transmits the fault information to a corresponding matching receiving object through the alarm push module.

[0030] Specifically, for the image parameter preprocessing of the data preprocessing module:

[0031] First, perform image denoising. Perform median filtering (suitable for removing salt and pepper noise. Replace the pixel value of each pixel in the image with the median of the pixel values ​​in its neighborhood), Gaussian filtering (suitable for removing Gaussian noise. Smooth the image by performing Gaussian convolution on the image), and bilateral filtering (considering spatial proximity and pixel value similarity at the same time, and being able to retain edge details while denoising) on ​​the image parameters to denoise the image parameters.

[0032] Second, image enhancement is performed. The denoised image parameters are subjected to histogram equalization (enhancing the image contrast by adjusting the grayscale histogram of the image), Laplace enhancement (enhancing the image edge by calculating the Laplace transform of the image), and Retinex processing (enhancing the image details and contrast by multi-scale Retinex processing based on the visual characteristics of the human eye) to enhance the image parameters;

[0033] Third, feature extraction is performed to extract edge features (Canny edge detection algorithm can be used to extract edge information in the image), texture features (gray level co-occurrence matrix (GLCM) can be used to extract texture features of the image, such as contrast, energy, entropy, etc.) and key point features (SIFT (scale invariant feature transform) or ORB (oriented FAST and rotational BRIEF) algorithm can be used to extract key points and descriptors in the image) from the enhanced image parameters to obtain effective features of the image parameters.

[0034] More specifically, for the sound parameter preprocessing of the data preprocessing module:

[0035] First, perform sound noise reduction, and perform frequency domain filtering (Fourier transform can be used to convert the sound signal into the frequency domain, and then low-pass, high-pass or band-pass filters can be applied to remove noise), adaptive filtering (adaptive algorithms (such as LMS algorithm) can be used to dynamically adjust filter parameters according to noise characteristics) and deep learning noise reduction (deep learning models (such as RNNoise) can be used for real-time noise suppression) to reduce the noise of sound parameters;

[0036] Second, feature extraction is performed. The spectral features (short-time Fourier transform (STFT) can be used to extract the spectrum of the sound signal), Mel spectrum (the spectrum can be converted into Mel spectrum, which is more in line with the auditory characteristics of the human ear), MFCC (MFCC features can be extracted by performing discrete cosine transform (DCT) on the Mel spectrum) and energy features (calculating the energy of each frame, zero-crossing rate and other features) are extracted from the denoised sound parameters to obtain effective features of the sound parameters.

[0037] Furthermore, for the electrical parameter preprocessing of the data preprocessing module:

[0038] First, the electrical parameters are normalized, cleaned and denoised.

[0039] Second, feature extraction is performed to process the processed electrical parameters in time domain (the mean, variance, peak and other time domain features of current and voltage signals can be calculated) and frequency domain (Fourier transform can be used to extract the spectral features of the signal, such as the main frequency, harmonic components, etc.) to obtain the effective features of the electrical parameters.

[0040] Furthermore, it also includes a storage module, which stores the data generated by the data acquisition module, the data preprocessing module, the multimodal data fusion module, the fault detection module and the alarm push module respectively.

[0041] It is worth mentioning that the technical features such as the distribution network involved in the patent application of this invention should be regarded as the prior art. The specific structure, working principle and possible control method and spatial layout method of these technical features can be selected by conventional methods in the field, and should not be regarded as the inventive point of the patent of this invention. The patent of this invention will not be further elaborated.

[0042] For those skilled in the art, it is still possible to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A multimodal distribution network operation system based on image and sound information, characterized in that: It includes data acquisition module, data preprocessing module, multimodal data fusion module, fault detection module and alarm push module, among which: The data acquisition module is used to collect sensor data including image parameters, sound parameters and electrical parameters of the power distribution network equipment and transmit the collected sensor data to the data preprocessing module; The data preprocessing module preprocesses each parameter in the sensor data, extracts effective features of each parameter, and inputs each effective feature into the multimodal data fusion module; The multimodal data fusion module includes a feature fusion unit and a decision fusion unit. The feature fusion unit performs multivariate fusion on the effective features corresponding to the image parameters, the sound parameters and the electrical parameters, respectively, to form a multimodal feature vector. The fault detection module performs fault detection and classification on the multimodal feature vector through a trained deep learning model to generate first fault information. The decision fusion unit transmits the effective features corresponding to the image parameters, the sound parameters and the electrical parameters to the fault detection module, respectively, to obtain image decision fault results, sound decision fault results and electrical decision fault results, and matches and fuses each fault result through the trained deep learning model, thereby performing fault detection and classification again to generate second fault information. The fault detection module integrates and analyzes the first fault information and the second fault information to obtain final fault information, and transmits the fault information to a corresponding matching receiving object through the alarm push module.

2. A multimodal distribution network operation system based on image and sound information according to claim 1, characterized in that: For image parameter preprocessing of the data preprocessing module: First, image denoising is performed by performing median filtering, Gaussian filtering and bilateral filtering on the image parameters to denoise the image parameters; Second, image enhancement is performed by performing histogram equalization, Laplace enhancement and Retinex processing on the denoised image parameters to enhance the image parameters; Third, feature extraction is performed to extract edge features, texture features and key point features of the enhanced image parameters to obtain effective features of the image parameters.

3. A multimodal distribution network operation system based on image and sound information according to claim 2, characterized in that: For the sound parameter preprocessing of the data preprocessing module: First, perform sound noise reduction, and perform frequency domain filtering, adaptive filtering, and deep learning noise reduction processing on the sound parameters to reduce the noise of the sound parameters; Second, feature extraction is performed to extract spectral features, Mel-spectrogram, MFCC and energy features of the denoised sound parameters to obtain effective features of the sound parameters.

4. A multimodal distribution network operation system based on image and sound information according to claim 3, characterized in that: For electrical parameter preprocessing of the data preprocessing module: First, the electrical parameters are normalized, cleaned and denoised. Second, feature extraction is performed to process the processed electrical parameters in time domain and frequency domain to obtain effective features of the electrical parameters.

5. A multimodal distribution network operation system based on image and sound information according to claim 4, characterized in that: It also includes a storage module, which stores the data generated by the data acquisition module, the data preprocessing module, the multimodal data fusion module, the fault detection module and the alarm push module respectively.

Images