A method and system for judging the status of power equipment by using data fusion

Through data fusion technology, combining sound signals and infrared thermal imaging images, the characteristics of the Mel frequency cepspectral coefficients are extracted, which solves the problem of insufficient inspection accuracy of power equipment in the existing technology, and achieves high accuracy judgment of power equipment status and early failure warning.

CN118364430BActive Publication Date: 2024-11-26SHANDONG GUANGRUI POWER TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410789077.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-06-19
Publication Date
2024-11-26
Estimated Expiration
2044-06-19

AI Technical Summary

Technical Problem

Existing power equipment inspection technology is difficult to accurately detect equipment abnormalities under the influence of environmental noise interference and external factors, resulting in untimely fault detection and affecting the stability of power supply.

Method used

Using the data fusion method, data is collected through the sound signals and infrared thermal imaging images during power equipment operation, the Mel frequency cepspectral coefficient matrix is ​​calculated, feature extraction and fusion is performed, and the equipment status is determined by using the decoder to improve the discrimination accuracy.

Benefits of technology

Adjusting the filter bank within the optimized frequency range improves the accuracy of discriminating the status of the power equipment, reduces environmental noise interference, and enhances the early warning capability of fault detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118364430B_ABST
    Figure CN118364430B_ABST
Patent Text Reader

Abstract

The present invention belongs to the technical field of electric power inspection, specifically a method and system for inspecting and distinguishing the state of electric power equipment using data fusion, wherein the sound of the electric power equipment in operation is preprocessed to obtain each frame of sound signal, the power spectrum of each frame of sound signal is calculated, and an equal-pitch filter group and an equal-area filter group are obtained according to the frequency of the power spectrum, and the equal-pitch filter group and the equal-area filter group are used to calculate each frame of sound signal to obtain two Mel-frequency cepstral coefficient matrices; thermal imaging images of the electric power equipment in operation are collected, and features of the thermal imaging images are extracted to obtain thermal imaging features, and the corresponding Mel-frequency cepstral coefficient features in each Mel-frequency cepstral coefficient matrix are extracted, and the thermal imaging features and the extracted Mel-frequency cepstral coefficient features are feature-fused, and the fused features are used to obtain the state of the electric power equipment through a decoder. The present invention simultaneously fuses the equipment operation sound and the thermal imaging image to determine the operation state of the electric power equipment without contact.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of electric power inspection, and specifically relates to an electric power equipment status inspection and judgment method and system using data fusion. Background Art

[0002] Power equipment (such as transformers) is a core component of the power system, and its normal operation directly affects the stability and reliability of power supply. Due to the complexity and high operating pressure of power equipment, any failure or abnormality will cause serious power outages, thus affecting the normal operation of society. Through inspections, equipment anomalies and hidden dangers, such as overheating, oil leakage, insulation aging, mechanical damage, etc., can be discovered in time. Timely discovery and treatment of these problems can avoid power outages caused by equipment failure, thereby reducing equipment damage, extending the service life of equipment, and reducing maintenance costs.

[0003] By monitoring and analyzing the status of equipment, the operation mode of the equipment can be optimized, energy consumption and loss can be reduced, and the overall efficiency of the power system can be improved. For example, by detecting the temperature distribution of the equipment through infrared imaging, equipment failures can be discovered in a timely manner; through sound analysis, minor anomalies in the operation of the equipment can be detected to prevent small problems from evolving into major failures. Experienced staff can judge the operation of power equipment (such as transformers) by sound. The advantage of monitoring faults through infrared imaging is that it can quickly and non-contactly detect temperature anomalies of equipment and intuitively display heat distribution, but infrared imaging is easily affected by external factors such as ambient temperature, humidity, wind speed and sunlight; the advantage of monitoring faults through equipment sound is that it can detect minor anomalies early, but it is easily disturbed by environmental noise. Summary of the invention

[0004] In order to solve the problems pointed out in the above background technology, the present invention proposes a method for judging the state of power equipment by using data fusion, comprising the following steps:

[0005] The sound of the power equipment in operation is collected, the sound is preprocessed to obtain each frame of the sound signal, the power spectrum of each frame of the sound signal is calculated, and an equal-pitch filter group and an equal-area filter group are obtained according to the frequency of the power spectrum. The equal-pitch filter group and the equal-area filter group are used to calculate each frame of the sound signal to obtain two Mel-frequency cepstrum coefficient matrices;

[0006] Collect infrared thermal images of the power equipment in operation, perform feature extraction on the thermal images to obtain thermal imaging features, extract the corresponding Mel-frequency cepstral coefficient features in each Mel-frequency cepstral coefficient matrix, perform feature fusion on the thermal imaging features and the extracted Mel-frequency cepstral coefficient features, and use the fused features through a decoder to obtain the status of the power equipment.

[0007] Preferably, the power spectrum of each frame of the sound signal is calculated as follows:

[0008] Obtain the frequency index of the frequency spectrum and the frequency corresponding to the index according to the number of points of Fourier transform, and obtain the preset frequency range;

[0009] If the frequency corresponding to the index is within the preset frequency range, retain the index;

[0010] Renumbering the indexes whose frequencies are not within the preset frequency range, and retaining the indexes with odd numbers;

[0011] Calculate the power spectrum of each frame of sound based on the retained index.

[0012] Preferably, the power spectrum of each frame of the sound signal is calculated as follows:

[0013] Perform Fourier transform on each frame of sound signal;

[0014] Obtain the frequency index of the frequency spectrum and the frequency corresponding to the index according to the number of points of Fourier transform, and obtain the preset frequency range;

[0015] The indices whose frequencies are not within the preset frequency range are renumbered, the even-numbered indices and the Fourier transform results corresponding to the indices are deleted, and the power spectrum of each frame of the sound signal is obtained based on the remaining indices.

[0016] Preferably, the equal-height filter bank and the equal-area filter bank are obtained according to the frequency of the power spectrum, specifically:

[0017] Obtain the preset number of each filter group, convert the frequency on the power spectrum into Mel frequency, and convert the preset frequency range into the Mel frequency range. If any one of two adjacent Mel frequency points is within the Mel frequency range, insert a new Mel frequency point between the two Mel frequency points, and adjust the distance between the two adjacent Mel frequency points that are not within the Mel frequency range.

[0018] An equal-height filter bank and an equal-area filter bank are created based on the Mel frequency points.

[0019] Preferably, the extracting of the corresponding Mel-frequency cepstral coefficient features in each Mel-frequency cepstral coefficient matrix is ​​specifically as follows:

[0020] For each Mel-frequency cepstral coefficient matrix, perform the following operations:

[0021] Each row of the Mel-frequency cepstral coefficient matrix is ​​used as an embedding vector to obtain multiple vectors, the frequency corresponding to each row is encoded as a position code, and the position code is attached to the head of the vector corresponding to the position code to obtain an input vector of the ViT model, and then a CLS vector is obtained, and the CLS vector is used as the Mel-frequency cepstral coefficient feature.

[0022] Preferably, the thermal imaging feature and the extracted Mel frequency cepstral coefficient feature are subjected to feature fusion, and the fused feature is used to obtain the state of the power equipment through a decoder, specifically:

[0023] The extracted thermal imaging features and two Mel-frequency cepstral coefficient features are concatenated into a matrix by row, and the multi-head attention of the matrix is ​​calculated. The calculation result is flattened into a one-dimensional vector and input into the decoder, and the status of the power equipment is determined according to the output value of the decoder.

[0024] On the other hand, the present invention also provides a power equipment status inspection and identification system using data fusion, comprising the following modules:

[0025] The acquisition module is used to collect the sound of the power equipment in operation, obtain each frame of the sound signal after preprocessing the sound, calculate the power spectrum of each frame of the sound signal, obtain an equal-pitch filter group and an equal-area filter group according to the frequency of the power spectrum, and respectively use the equal-pitch filter group and the equal-area filter group to calculate each frame of the sound signal to obtain two Mel-frequency cepstrum coefficient matrices;

[0026] The fusion module is used to collect infrared thermal imaging images of power equipment in operation, perform feature extraction on the thermal imaging images to obtain thermal imaging features, extract the corresponding Mel-frequency cepstral coefficient features in each Mel-frequency cepstral coefficient matrix, perform feature fusion on the thermal imaging features and the extracted Mel-frequency cepstral coefficient features, and obtain the status of the power equipment through a decoder using the fused features.

[0027] Preferably, the power spectrum of each frame of the sound signal is calculated as follows:

[0028] Obtain the frequency index of the frequency spectrum and the frequency corresponding to the index according to the number of points of Fourier transform, and obtain the preset frequency range;

[0029] If the frequency corresponding to the index is within the preset frequency range, retain the index;

[0030] Renumbering the indexes whose frequencies are not within the preset frequency range, and retaining the indexes with odd numbers;

[0031] Calculate the power spectrum of each frame of sound based on the retained index.

[0032] Preferably, the power spectrum of each frame of the sound signal is calculated as follows:

[0033] Perform Fourier transform on each frame of sound signal;

[0034] Obtain the frequency index of the frequency spectrum and the frequency corresponding to the index according to the number of points of Fourier transform, and obtain the preset frequency range;

[0035] The indices whose frequencies are not within the preset frequency range are renumbered, the even-numbered indices and the Fourier transform results corresponding to the indices are deleted, and the power spectrum of each frame of the sound signal is obtained based on the remaining indices.

[0036] Preferably, the equal-height filter bank and the equal-area filter bank are obtained according to the frequency of the power spectrum, specifically:

[0037] Obtain the preset number of each filter group, convert the frequency on the power spectrum into Mel frequency, and convert the preset frequency range into the Mel frequency range. If any one of two adjacent Mel frequency points is within the Mel frequency range, insert a new Mel frequency point between the two Mel frequency points, and adjust the distance between the two adjacent Mel frequency points that are not within the Mel frequency range.

[0038] An equal-height filter bank and an equal-area filter bank are created based on the Mel frequency points.

[0039] Preferably, the extracting of the corresponding Mel-frequency cepstral coefficient features in each Mel-frequency cepstral coefficient matrix is ​​specifically as follows:

[0040] For each Mel-frequency cepstral coefficient matrix, perform the following operations:

[0041] Each row of the Mel-frequency cepstral coefficient matrix is ​​used as an embedding vector to obtain multiple vectors, the frequency corresponding to each row is encoded as a position code, and the position code is attached to the head of the vector corresponding to the position code to obtain an input vector of the ViT model, and then a CLS vector is obtained, and the CLS vector is used as the Mel-frequency cepstral coefficient feature.

[0042] Preferably, the thermal imaging feature and the extracted Mel frequency cepstral coefficient feature are subjected to feature fusion, and the fused feature is used to obtain the state of the power equipment through a decoder, specifically:

[0043] The extracted thermal imaging features and two Mel-frequency cepstral coefficient features are concatenated into a matrix by row, and the multi-head attention of the matrix is ​​calculated. The calculation result is flattened into a one-dimensional vector and input into the decoder, and the status of the power equipment is determined according to the output value of the decoder.

[0044] In addition, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described above is implemented.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] The present invention uses the sound of the power equipment when it is running and the infrared thermal imaging image to judge the operating status of the power equipment during inspection, and adjusts the power spectrum within a preset frequency range; when designing the filter group, the shape of the filter in the filter is adjusted according to the preset frequency range, so as to extract more information within the preset frequency range and improve the accuracy of the final judgment. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a flow chart of Embodiment 1;

[0048] Figure 2 This is the tip discharge waveform of the power equipment;

[0049] Figure 3 is a spectrum diagram;

[0050] Figure 4 It is a schematic diagram of an equal-height Mel filter bank;

[0051] Figure 5 This is a structural diagram of the second embodiment. DETAILED DESCRIPTION

[0052] In this article, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0053] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] Embodiment 1

[0055] A method for judging the status of power equipment by using data fusion, such as Figure 1 As shown, the following steps are included:

[0056] S1, collecting the sound of the power equipment in operation, obtaining each frame of the sound signal after preprocessing the sound, calculating the power spectrum of each frame of the sound signal, obtaining an equal-pitch filter group and an equal-area filter group according to the frequency of the power spectrum, and respectively using the equal-pitch filter group and the equal-area filter group to calculate each frame of the sound signal to obtain two Mel-frequency cepstrum coefficient matrices;

[0057] During the operation of power equipment, there are differences between the sound of normal operation and the sound of abnormal operation. Experienced staff can judge the operation of power equipment by the sound. For example, when a tip discharge occurs in a transformer, there will be an obvious hissing sound. Compared with other parameters, even when a fault just occurs and the sound is very small, it can be judged by the operating sound of the power equipment. Although it is difficult for staff to distinguish with human ears, it can be better distinguished through sound analysis.

[0058] When the equipment is running, the operating sound of the equipment is collected. The waveform (time domain diagram) of the collected operating sound of the equipment is as follows: Figure 2 As shown in the figure. Due to the influence of environmental noise, the collected sound of power equipment operation will have noise. After preprocessing such as denoising, some noise can be eliminated. In addition, preprocessing also includes pre-emphasis, framing, windowing and other operations, and then each frame of sound signal is obtained. After each frame of sound signal is obtained, a fast Fourier transform (FFT) is performed to obtain the power spectrum.

[0059] When analyzing the operating sounds of power equipment, it was found that the different states of power equipment (such as transformers) have large differences in the frequency domain of 5KHz-15KHz. In other words, this frequency range is the key to distinguishing different operating states of power equipment. It is an important frequency range for distinguishing different operating states.

[0060] 5KHz-15KHz is an important content for the subsequent judgment of the operating status of the power equipment. In order to reduce the amount of calculation, in one embodiment, the power spectrum of each frame of the sound signal is calculated, specifically:

[0061] Obtain the frequency index of the frequency spectrum and the frequency corresponding to the index according to the number of points of Fourier transform, and obtain the preset frequency range;

[0062] If the frequency corresponding to the index is within the preset frequency range, keep the index;

[0063] Renumber the indexes whose frequencies are not within the preset frequency range, and keep the indexes with odd numbers;

[0064] Calculate the power spectrum of each frame of sound based on the retained index.

[0065] Get the number of Fourier transform points (FFT Points), and then get the frequency index based on the number of points, and each frequency index corresponds to a frequency. The corresponding relationship between the index and the frequency is f=(k×fs) / N, where fs is the sampling rate, N is the signal length, and k is the index. In a more specific embodiment, the Fourier transform is a discrete Fourier transform or a fast Fourier transform. Among them, the index k refers to the kth frequency component in the Fourier transform. For ease of understanding, here we take the discrete sound of a certain frame as x(n) as an example. Assuming that this frame has a total of N discrete data, Fourier transform for this frame will obtain multiple X(k), where k is the frequency index. The frequency corresponding to this frequency index can be further obtained through f=(k×fs) / N.

[0066] For the indexes whose frequencies are within the preset frequency range, the power corresponding to the indexes is calculated; for the frequency components that are not within the preset frequency range, the indexes are numbered and the indexes with odd numbers are retained;

[0067] When calculating the spectrum graph, since the frequencies within the preset range are of great help to the final judgment, the frequency interval is smaller at this time, and more frequencies within the preset frequency range can be obtained, while for frequencies outside the preset range, the calculation of some frequencies can be omitted.

[0068] For example, if there are 512 FFT points, and the frequencies corresponding to indexes 256-512 are within the preset range, and the frequencies corresponding to indexes 1-255 are not within the preset range, the Fourier transforms of indexes 1, 3, 5, ..., 255 and indexes 256-512 are calculated. When drawing the spectrum, only the frequencies and powers corresponding to FFT points 1, 3, 5, ..., 255 and 256-512 are needed, thus saving some numerical calculations and having little impact on the final result.

[0069] After calculating the spectrum graph, the power spectrum can be further calculated based on the spectrum graph, and then the MFCC can be calculated through the logarithmic power spectrum. This also involves discrete cosine transform after taking the logarithm, which will not be repeated here.

[0070] In another embodiment, the spectrum diagram is first calculated according to the original calculation method, and then a part of the frequencies that are not within the preset frequency range is deleted. Specifically,

[0071] Perform Fourier transform on each frame of sound signal;

[0072] Obtain the frequency index of the frequency spectrum and the frequency corresponding to the index according to the number of points of Fourier transform, and obtain the preset frequency range;

[0073] The indices whose frequencies are not within the preset frequency range are renumbered, the even-numbered indices and the Fourier transform results corresponding to the indices are deleted, and the power spectrum of each frame of the sound signal is obtained based on the remaining indices.

[0074] That is, first perform FFT transformation, and after the transformation, delete a portion of the frequencies that are not in the preset frequency range. In a specific embodiment, the preset frequency range is 5KHz-15KHz. In another embodiment, the preset frequency range is 5KHz-8KHz, 9KHz-13KHz.

[0075] In the above embodiments, as known to those skilled in the art, the frequency is a discrete value, and thus the frequencies can be numbered. Figure 3 A schematic diagram of a spectrum diagram is shown, in which the dotted lines are the frequencies whose amplitudes are not calculated or deleted. In subsequent calculations, the spectrum diagram corresponding to the solid line part is used.

[0076] In calculating the Mel-frequency cepstral coefficients, although the equal-pitch filter group conforms to the human auditory perception and the gain of each filter is the same, it is impossible to obtain tiny frequencies in some frequency parts. Although the equal-area filter helps to obtain subtle changes, the gain of each filter is different, and there are inconsistencies in the processing on different frequency bands. Based on this, the present invention simultaneously uses the equal-pitch filter group and the equal-area filter group to calculate the Mel-frequency cepstral coefficients (MFCC).

[0077] When calculating the power spectrum, the fast Fourier transform (FFT) is used to obtain the power spectrum. The horizontal axis of the power spectrum is the frequency component, that is, each frequency component corresponds to a power value. After obtaining the power spectrum, an equal-height filter group and an equal-area filter group are obtained according to the frequency of the power spectrum, and then these two filter groups and the power spectrum of each frame signal are used to obtain two Mel-frequency cepstrum coefficient matrices. In a specific embodiment, the equal-height filter group and the equal-area filter group use equally spaced Mel scales to determine the boundary points of each filter. An example of an equal-height filter group is as follows: Figure 4 shown.

[0078] In order to improve the resolution, especially for 5KHz-15KHz, in another embodiment, an equal-height filter group and an equal-area filter group are obtained according to the frequency of the power spectrum, specifically:

[0079] Get the preset number of filters in each group, convert the frequency on the power spectrum into Mel frequency, and convert the preset frequency range into Mel frequency range. If any of two adjacent Mel frequency points is within the Mel frequency range, insert a new Mel frequency point between the two Mel frequency points, and adjust the distance between the two adjacent Mel frequency points that are not within the Mel frequency range.

[0080] Create equal-height filter banks and equal-area filter banks based on Mel-frequency points.

[0081] Assuming that there are 10 filters in each filter group, the frequencies on the power spectrum are converted to Mel frequencies, and then the preset frequency range is converted to the Mel frequency range. Since each filter group has 10 filters, it is necessary to calculate multiple Mel frequency points, and determine a filter for every three adjacent Mel frequency points. In order to obtain high-frequency information within the preset frequency range, a new Mel frequency point is inserted between two Mel frequency points. In order to keep the number of filters unchanged, the distance between two adjacent Mel frequency points that are not within the Mel frequency range is adjusted. For example, if Mel frequency points 1 and 2 are not within the Mel frequency range, the distance between Mel frequency points 1 and 2 is increased, so that the total number of Mel frequency points remains unchanged. Then use the Mel frequency points to create equal-height filter groups and equal-area filter groups.

[0082] S2, collects infrared thermal images of the power equipment in operation, extracts features of the thermal images to obtain thermal imaging features, extracts the corresponding Mel-frequency cepstral coefficient features in each Mel-frequency cepstral coefficient matrix, fuses the thermal imaging features and the extracted Mel-frequency cepstral coefficient features, and obtains the status of the power equipment through a decoder using the fused features.

[0083] Infrared thermal imaging during the operation of power equipment can also represent the operating status of the equipment. Infrared thermal imaging images during the operation of power equipment are collected, and features are extracted from the thermal imaging images to obtain thermal imaging features. There are many ways to extract features from thermal imaging, including but not limited to convolution. In another embodiment, the thermal imaging feature extraction process is to divide the thermal imaging image into multiple overlapping areas, obtain a vector by convolving each area, and then perform a convolution operation after splicing all vectors into a matrix.

[0084] The Mel-frequency cepstral coefficient is also a matrix, with the horizontal axis being time and the vertical axis being frequency. When extracting features from two Mel-frequency cepstral matrices, a convolution method may also be used. Since the amount of data in the Mel-frequency cepstral coefficient matrix is ​​relatively small relative to the image, and each row thereof represents a frequency, in another embodiment, the corresponding Mel-frequency cepstral coefficient features in each Mel-frequency cepstral coefficient matrix are extracted, specifically:

[0085] For each Mel-frequency cepstral coefficient matrix, perform the following operations:

[0086] Each row of the Mel-frequency cepstral coefficient matrix is ​​used as an embedding vector to obtain multiple vectors, the frequency corresponding to each row is encoded as a position code, and the position code is attached to the head of the vector corresponding to the position code to obtain an input vector of the ViT model, and then the CLS vector is obtained, and the CLS vector is used as the Mel-frequency cepstral coefficient feature.

[0087] The ViT model can obtain the overall information of the entire input. In order to reduce the dimension, in this embodiment, each row of the Mel-frequency cepstral coefficient is used as a token, and the frequency corresponding to each row is used as the position encoding information, that is, the frequency is positionally embedded, and then connected to the class token as the input of the ViT model, and finally the class token (CLS) is used as the Mel-frequency cepstral coefficient feature.

[0088] The thermal imaging feature represents the operating temperature information of the power equipment. If the equipment is abnormal or, for example, the tip discharge occurs, the temperature of the equipment will be abnormal. The Mel-frequency cepstral coefficient feature represents the operating sound information of the power equipment. If the operation is abnormal, it can also be judged by the sound. In order to improve the accuracy of the judgment, the present invention fuses the thermal imaging feature and the Mel-frequency cepstral coefficient feature. In a specific embodiment, the thermal imaging feature and the extracted Mel-frequency cepstral coefficient feature are feature fused, and the fused feature is used to obtain the status of the power equipment through a decoder, specifically:

[0089] The extracted thermal imaging features and two Mel-frequency cepstral coefficient features are concatenated into a matrix by row, and the multi-head attention of the matrix is ​​calculated. The calculation result is flattened into a one-dimensional vector and input into the decoder. The status of the power equipment is determined according to the output value of the decoder.

[0090] The thermal imaging feature is a row vector, and each Mel-frequency cepstral coefficient feature is also a row vector, so there are three row vectors. The three row vectors are concatenated into a matrix. After multi-head attention calculation, the result is flattened into a one-dimensional vector and input into the decoder. The operating status of the power equipment is judged according to the output value of the decoder. For example, values ​​1-4 indicate normal operation, 5-7 indicate suspected faults, and 7-10 indicate faults. The larger the value, the greater the possibility of a fault.

[0091] Embodiment 2

[0092] The present invention also provides a power equipment status inspection and identification system using data fusion, such as Figure 5 As shown, it includes the following modules:

[0093] The acquisition module is used to collect the sound of the power equipment in operation, obtain each frame of the sound signal after preprocessing the sound, calculate the power spectrum of each frame of the sound signal, obtain the equal-pitch filter group and the equal-area filter group according to the frequency of the power spectrum, and use the equal-pitch filter group and the equal-area filter group to calculate each frame of the sound signal to obtain two Mel-frequency cepstrum coefficient matrices;

[0094] The fusion module is used to collect infrared thermal images of power equipment in operation, extract features from the thermal images to obtain thermal imaging features, extract the corresponding Mel-frequency cepstral coefficient features in each Mel-frequency cepstral coefficient matrix, fuse the thermal imaging features with the extracted Mel-frequency cepstral coefficient features, and obtain the status of the power equipment through a decoder using the fused features.

[0095] Preferably, the power spectrum of each frame of the sound signal is calculated, specifically:

[0096] Obtain the frequency index of the frequency spectrum and the frequency corresponding to the index according to the number of points of Fourier transform, and obtain the preset frequency range;

[0097] If the frequency corresponding to the index is within the preset frequency range, keep the index;

[0098] Renumber the indexes whose frequencies are not within the preset frequency range, and keep the indexes with odd numbers;

[0099] Calculate the power spectrum of each frame of sound based on the retained index.

[0100] Preferably, the power spectrum of each frame of the sound signal is calculated, specifically:

[0101] Perform Fourier transform on each frame of sound signal;

[0102] Obtain the frequency index of the frequency spectrum and the frequency corresponding to the index according to the number of points of Fourier transform, and obtain the preset frequency range;

[0103] The indices whose frequencies are not within the preset frequency range are renumbered, the even-numbered indices and the Fourier transform results corresponding to the indices are deleted, and the power spectrum of each frame of the sound signal is obtained based on the remaining indices.

[0104] Preferably, an equal-height filter bank and an equal-area filter bank are obtained according to the frequency of the power spectrum, specifically:

[0105] Get the preset number of filters in each group, convert the frequency on the power spectrum into Mel frequency, and convert the preset frequency range into Mel frequency range. If any of two adjacent Mel frequency points is within the Mel frequency range, insert a new Mel frequency point between the two Mel frequency points, and adjust the distance between the two adjacent Mel frequency points that are not within the Mel frequency range.

[0106] Create equal-height filter banks and equal-area filter banks based on Mel-frequency points.

[0107] Preferably, the corresponding Mel-frequency cepstral coefficient features in each Mel-frequency cepstral coefficient matrix are extracted, specifically:

[0108] For each Mel-frequency cepstral coefficient matrix, perform the following operations:

[0109] Each row of the Mel-frequency cepstral coefficient matrix is ​​used as an embedding vector to obtain multiple vectors, the frequency corresponding to each row is encoded as a position code, and the position code is attached to the head of the vector corresponding to the position code to obtain an input vector of the ViT model, and then a CLS vector is obtained, and the CLS vector is used as the Mel-frequency cepstral coefficient feature.

[0110] Preferably, the thermal imaging features and the extracted Mel frequency cepstral coefficient features are fused, and the fused features are used to obtain the status of the power equipment through a decoder, specifically:

[0111] The extracted thermal imaging features and two Mel-frequency cepstral coefficient features are concatenated into a matrix by row, and the multi-head attention of the matrix is ​​calculated. The calculation result is flattened into a one-dimensional vector and input into the decoder. The status of the power equipment is determined according to the output value of the decoder.

[0112] Embodiment 3

[0113] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method of the first embodiment is implemented.

[0114] Embodiment 4

[0115] The present invention also provides a computer device, which includes at least a memory and a processor. A computer program is stored in the memory, and the computer program implements the method of the first embodiment when executed by the processor.

[0116] Through the description of the above implementation methods, technicians in this field can clearly understand that each implementation method can be implemented by adding a necessary general hardware platform, and of course can also be implemented by combining hardware and software. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a computer product, and the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0117] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it, and other embodiments may also be adopted. Although the present invention has been described in detail with reference to the aforementioned embodiments, ordinary technicians in this field should understand that they can still modify the technical solutions recorded in the aforementioned embodiments, or replace some of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for judging the status of power equipment by using data fusion, characterized in that: The following steps are involved: The sound of the power equipment in operation is collected, the sound is preprocessed to obtain each frame of the sound signal, the power spectrum of each frame of the sound signal is calculated, and an equal-pitch filter group and an equal-area filter group are obtained according to the frequency of the power spectrum. The equal-pitch filter group and the equal-area filter group are used to calculate each frame of the sound signal to obtain two Mel-frequency cepstrum coefficient matrices; Collect infrared thermal imaging images of power equipment in operation, extract features from the thermal imaging images to obtain thermal imaging features, extract the corresponding Mel-frequency cepstral coefficient features in each Mel-frequency cepstral coefficient matrix, perform feature fusion on the thermal imaging features and the extracted Mel-frequency cepstral coefficient features, and obtain the status of the power equipment through a decoder using the fused features; The calculation of the power spectrum of each frame of the sound signal is specifically as follows: Obtain the frequency index of the frequency spectrum and the frequency corresponding to the index according to the number of points of Fourier transform, and obtain the preset frequency range; If the frequency corresponding to the index is within the preset frequency range, retain the index; Renumbering the indexes whose frequencies are not within the preset frequency range, and retaining the indexes with odd numbers; Calculate the power spectrum of each frame of sound according to the retained index; Alternatively, the power spectrum of each frame of the sound signal is calculated as follows: Perform Fourier transform on each frame of sound signal; Obtain the frequency index of the frequency spectrum and the frequency corresponding to the index according to the number of points of Fourier transform, and obtain the preset frequency range; The indices whose frequencies are not within the preset frequency range are renumbered, the even-numbered indices and the Fourier transform results corresponding to the indices are deleted, and the power spectrum of each frame of the sound signal is obtained based on the remaining indices.

2. The method for determining the state of electric power equipment by using data fusion according to claim 1, characterized in that: The equal-height filter group and the equal-area filter group are obtained according to the frequency of the power spectrum, specifically: Obtain the preset number of each filter group, convert the frequency on the power spectrum into Mel frequency, and convert the preset frequency range into the Mel frequency range. If any one of two adjacent Mel frequency points is within the Mel frequency range, insert a new Mel frequency point between the two Mel frequency points, and adjust the distance between the two adjacent Mel frequency points that are not within the Mel frequency range. An equal-height filter bank and an equal-area filter bank are created based on the Mel frequency points.

3. The method for determining the status of electric power equipment by using data fusion according to claim 1, characterized in that: The extraction of the corresponding Mel-frequency cepstral coefficient features in each Mel-frequency cepstral coefficient matrix is ​​specifically as follows: For each Mel-frequency cepstral coefficient matrix, perform the following operations: Each row of the Mel-frequency cepstral coefficient matrix is ​​used as an embedding vector to obtain multiple vectors, the frequency corresponding to each row is encoded as a position code, and the position code is attached to the head of the vector corresponding to the position code to obtain an input vector of the ViT model, and then a CLS vector is obtained, and the CLS vector is used as the Mel-frequency cepstral coefficient feature.

4. The method for determining the state of electric power equipment by using data fusion according to claim 1, characterized in that: The thermal imaging features and the extracted Mel frequency cepstral coefficient features are fused, and the fused features are decoded to obtain the status of the power equipment, specifically: The extracted thermal imaging features and two Mel-frequency cepstral coefficient features are concatenated into a matrix by row, and the multi-head attention of the matrix is ​​calculated. The calculation result is flattened into a one-dimensional vector and input into the decoder, and the status of the power equipment is determined according to the output value of the decoder.

5. A power equipment status inspection and identification system using data fusion, characterized in that: Includes the following modules: The acquisition module is used to collect the sound of the power equipment in operation, obtain each frame of the sound signal after preprocessing the sound, calculate the power spectrum of each frame of the sound signal, obtain an equal-pitch filter group and an equal-area filter group according to the frequency of the power spectrum, and respectively use the equal-pitch filter group and the equal-area filter group to calculate each frame of the sound signal to obtain two Mel-frequency cepstrum coefficient matrices; A fusion module is used to collect infrared thermal imaging images of power equipment in operation, extract features of the external thermal imaging images to obtain thermal imaging features, extract the corresponding Mel-frequency cepstral coefficient features in each Mel-frequency cepstral coefficient matrix, perform feature fusion on the thermal imaging features and the extracted Mel-frequency cepstral coefficient features, and obtain the status of the power equipment through a decoder using the fused features; The calculation of the power spectrum of each frame of the sound signal is specifically as follows: Obtain the frequency index of the frequency spectrum and the frequency corresponding to the index according to the number of points of Fourier transform, and obtain the preset frequency range; If the frequency corresponding to the index is within the preset frequency range, retain the index; Renumbering the indexes whose frequencies are not within the preset frequency range, and retaining the indexes with odd numbers; Calculate the power spectrum of each frame of sound according to the retained index; Alternatively, the power spectrum of each frame of the sound signal is calculated as follows: Perform Fourier transform on each frame of sound signal; Obtain the frequency index of the frequency spectrum and the frequency corresponding to the index according to the number of points of Fourier transform, and obtain the preset frequency range; The indices whose frequencies are not within the preset frequency range are renumbered, the even-numbered indices and the Fourier transform results corresponding to the indices are deleted, and the power spectrum of each frame of the sound signal is obtained based on the remaining indices.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements the method according to any one of claims 1 to 4.