Electrical equipment defect positioning method and device and electronic equipment

Abstract

The invention discloses an electrical equipment defect positioning method and device and electronic equipment. The method comprises the steps that an initial acoustic image corresponding to the electrical equipment is acquired, and the initial acoustic image is simulated and generated based on a simulation space corresponding to the electrical equipment; based on a visible light image space corresponding to the electrical equipment, performing coordinate conversion on the multichannel acoustic data in the initial acoustic image to obtain a target acoustic image; a deep learning model is used for carrying out multi-dimension feature analysis on the target acoustic image map, defect information of the electrical equipment is determined, multiple dimensions comprise the acoustic dimension and the visible light dimension, and the defect information comprises the defect position and the defect category. According to the invention, the technical problems of low accuracy and low reliability of an electrical equipment defect positioning method in the prior art are solved.

Description

Technical Field

[0001] The present application relates to the technical field of equipment detection, and in particular to a method and device for locating defects in electrical equipment and electronic equipment. Background Art

[0002] The detection of electrical equipment is one of the safety guarantees for the operation of the power grid. At present, acoustic positioning algorithms or image recognition based on visible light are usually used to locate defects in power equipment. In power equipment, such as transformers and switchgear, if the equipment is in a defective state, a specific sound signal will be generated. By using acoustic positioning algorithms, specific sound signals can be captured and analyzed to detect equipment defects. Visible light-based image recognition technology can identify the defects of the power equipment shown in the image by taking images of the surface of the power equipment and using a deep learning-based target detection algorithm.

[0003] However, in the related art, the defect location method that only uses the acoustic location algorithm is easily disturbed by background noise, and the positioning accuracy is limited. The defect location method based only on visible light image recognition technology is greatly affected by image quality, and the single data source sample is easily disturbed, affecting the final result. The current acoustic image and visible light image fusion methods are mostly limited to shallow layer superposition, lacking automated processing and deep information fusion. Therefore, the electrical equipment defect location method in the related art has low accuracy and reliability.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a method, device and electronic device for locating defects in electrical equipment to at least solve the technical problems of low accuracy and low reliability of methods for locating defects in electrical equipment in related technologies.

[0006] According to one aspect of an embodiment of the present application, a method for locating defects in electrical equipment is provided, including: obtaining an initial acoustic image corresponding to the electrical equipment, wherein the initial acoustic image is generated based on a simulation of a simulation space corresponding to the electrical equipment; based on a visible light image space corresponding to the electrical equipment, performing coordinate transformation on multi-channel acoustic data in the initial acoustic image to obtain a target acoustic image; and using a deep learning model to perform feature analysis on the target acoustic image in multiple dimensions to determine defect information of the electrical equipment, wherein the multiple dimensions include an acoustic dimension and a visible light dimension, and the defect information includes a defect location and a defect category.

[0007] Optionally, obtaining an initial acoustic image corresponding to the electrical equipment includes: setting up multiple sound sources in a simulation space; constructing an acoustic wave transmission model corresponding to the electrical equipment according to the law of acoustic wave transmission; obtaining an initial acoustic signal based on the acoustic wave transmission model and a sensor array in the simulation space; performing signal enhancement processing and noise suppression processing on the initial acoustic signal in a target direction to obtain a target acoustic signal; and performing spatial scanning on the simulation space to determine an initial acoustic image corresponding to the target acoustic signal on a target plane.

[0008] Optionally, based on the visible light image space corresponding to the electrical equipment, the multi-channel acoustic data in the initial acoustic image map is transformed into coordinates to obtain a target acoustic image map, including: determining the pitch angle, azimuth angle and target distance corresponding to multiple pixels in the initial acoustic image map according to the multi-channel acoustic data, wherein the target distance is determined by the actual sound source position and the sensor array position corresponding to the multi-channel acoustic data; determining the camera imaging plane according to the visible light image space; calculating the pitch angle, azimuth angle and target distance, mapping the initial acoustic image map onto the camera imaging plane, and obtaining the target acoustic image map.

[0009] Optionally, the deep learning model includes a feature extraction network and a detection head; using the deep learning model to perform multi-dimensional feature analysis on the target acoustic image image to determine the defect information of the electrical equipment includes: using the feature extraction network to perform multi-level feature extraction on the target acoustic image image to obtain acoustic image features and visible light image features; using the detection head to perform a fusion analysis of the acoustic image features and the visible light image features to determine the defect information.

[0010] Optionally, the feature extraction network includes multiple average pooling layers, and each of the multiple average pooling layers has a kernel size of 3, a stride of 2, and an edge extension of 1.

[0011] Optionally, the feature extraction network includes multiple convolutional layers, among which the first convolutional layer has a kernel size of 7, a stride of 2, and an edge extension of 3, the second convolutional layer has a kernel size of 3, a stride of 2, and an edge extension of 1, and the third convolutional layer has a kernel size of 1, a stride of 1, and an edge extension of 0.

[0012] Optionally, the detection head performs a fusion analysis on the acoustic image features and the visible light image features to determine the defect information, including: superimposing the acoustic image features and the visible light image features to obtain superimposed features; using a fusion attention algorithm in the detection head to analyze the superimposed features to predict the defect information.

[0013] Optionally, the deep learning model is obtained through machine learning training using multiple sets of training data, each set of training data includes acoustic image samples and real defect data, and the real defect data is used to characterize the real defect position and real defect category corresponding to the acoustic image samples.

[0014] According to another aspect of an embodiment of the present application, an electrical equipment defect locating device is also provided, including: an acquisition module, used to acquire an initial acoustic image corresponding to the electrical equipment, wherein the initial acoustic image is generated based on a simulation space corresponding to the electrical equipment; a conversion module, used to perform coordinate conversion on multi-channel acoustic data in the initial acoustic image based on a visible light image space corresponding to the electrical equipment, to obtain a target acoustic image; an analysis module, used to perform feature analysis on the target acoustic image in multiple dimensions using a deep learning model to determine defect information of the electrical equipment, wherein the multiple dimensions include acoustic dimensions and visible light dimensions, and the defect information includes defect location and defect category.

[0015] According to another aspect of an embodiment of the present application, an electronic device is also provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute any one of the above-mentioned electrical equipment defect location methods.

[0016] In an embodiment of the present application, an initial acoustic image corresponding to the electrical equipment is obtained, wherein the initial acoustic image is generated based on a simulation of a simulation space corresponding to the electrical equipment; based on the visible light image space corresponding to the electrical equipment, coordinate transformation is performed on the multi-channel acoustic data in the initial acoustic image to obtain a target acoustic image; a deep learning model is used to perform feature analysis of the target acoustic image in multiple dimensions to determine defect information of the electrical equipment, wherein the multiple dimensions include acoustic dimensions and visible light dimensions, and the defect information includes defect location and defect category. Thus, the present application achieves the purpose of more accurately locating defects in electrical equipment, thereby achieving the technical effect of improving the accuracy and reliability of the method for locating defects in electrical equipment, and further solving the technical problems of low accuracy and low reliability of the method for locating defects in electrical equipment in related technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 is a hardware structure block diagram of an optional terminal device for an electrical equipment defect location method according to an embodiment of the present application;

[0019] Figure 2 is a flow chart of a method for locating a defect of an electrical device according to an embodiment of the present application;

[0020] Figure 3 is a structural block diagram of an optional electrical equipment defect location method according to an embodiment of the present application;

[0021] Figure 4 is a schematic diagram of signal processing according to an embodiment of the present application;

[0022] Figure 5 is a structural diagram of an optional encoding module according to an embodiment of the present application;

[0023] Figure 6 It is a structural block diagram of an electrical equipment defect locating device according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] According to an embodiment of the present application, an embodiment of a method for locating defects in electrical equipment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0027] Figure 1 is a hardware structure block diagram of an optional terminal device for an electrical equipment defect location method according to an embodiment of the present application, such as Figure 1As shown, the terminal device may include one or more processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor (Microcontroller Unit, referred to as MCU) or a programmable logic device (Field Programmable Gate Array, referred to as FPGA)), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display device 110, an input / output device 108 (i.e., an I / O device), a Universal Serial Bus (USB) port (which may be included as one of the ports of a computer bus, not shown in the figure), a network interface (not shown in the figure), a power supply (not shown in the figure) and / or a camera (not shown in the figure). It can be understood by those skilled in the art that Figure 1 The structure shown is for illustration only and does not limit the structure of the above terminal device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0028] It should be noted that the one or more processors 102 and / or other data processing circuits may be embodied in whole or in part as software, hardware, firmware or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be fully or partially integrated into any of the other components in the terminal device (or mobile device).

[0029] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the electrical equipment defect location method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, the above-mentioned electrical equipment defect location method is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0030] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the terminal device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0031] Under the above operating environment, the embodiment of the present application provides the following Figure 2 The electrical equipment defect location method shown in the figure, Figure 2 is a flow chart of a method for locating a defect in an electrical device according to an embodiment of the present application. Figure 2 As shown, the method includes the following implementation steps:

[0032] Step S201, obtaining an initial acoustic image corresponding to the electrical device, wherein the initial acoustic image is generated based on a simulation of a simulation space corresponding to the electrical device;

[0033] Step S202, based on the visible light image space corresponding to the electrical device, coordinate transformation is performed on the multi-channel acoustic data in the initial acoustic image to obtain a target acoustic image;

[0034] Step S203, using a deep learning model to perform feature analysis on the target acoustic image in multiple dimensions to determine defect information of the electrical equipment, wherein the multiple dimensions include an acoustic dimension and a visible light dimension, and the defect information includes a defect location and a defect category.

[0035] The above-mentioned electrical equipment can be various equipment in the power system, such as transformers, switchgear, etc. The above-mentioned initial acoustic image map can be used to represent the corresponding acoustic signal transmission law emitted by the electrical equipment. The above-mentioned simulation space can be a virtual three-dimensional space constructed in a computer environment, and the simulation space can be used to simulate the propagation behavior of sound waves in the power equipment. The above-mentioned visible light image space can be a virtual space composed of two-dimensional images of the electrical equipment or its environment captured by a camera or image sensor within the visible spectrum. The above-mentioned multi-channel acoustic data can be used to assist in determining the direction of the sound source. The above-mentioned target acoustic image map can be used to represent the actual position of the multi-channel acoustic data in the initial acoustic image map on the electrical equipment. The above-mentioned deep learning model can be used to analyze the information in the target acoustic image map from multiple dimensions.

[0036] Specifically, a virtual three-dimensional simulation space is constructed in a computer environment, an electrical equipment model is built in the simulation space, the propagation behavior of the sound waves of the electrical equipment in the simulation space is simulated, and the initial acoustic image corresponding to the electrical equipment is obtained. Furthermore, within the visible spectrum range, a two-dimensional image of the electrical equipment or its environment captured by a camera or image sensor is used to form a visible light image space, and the spatial coordinates of the initial acoustic image are mapped to the coordinate system in the visible light image space, and the multi-channel acoustic data in the initial acoustic image is converted into a position corresponding to the coordinates in the visible light image to obtain a target acoustic image. Furthermore, a deep learning model is used to fuse the acoustic dimension information and the visible light dimension information, and a comprehensive analysis is performed from the acoustic dimension and the visible light dimension to automatically identify and locate defects in the power equipment and determine the defect information of the electrical equipment.

[0037] It is easy to understand that through the above steps S201 to S203, on the one hand, in the embodiment of the present application, by combining acoustic positioning and visible light image recognition technology, using information in multiple dimensions, the robustness of the electrical equipment defect positioning method in a complex environment is ensured, and compared with the electrical equipment defect positioning method that only uses single-dimensional information, the reliability of the electrical equipment defect positioning method is improved; on the other hand, in the embodiment of the present application, the use of a deep learning model for fusion analysis can more accurately identify and locate the defects of the electrical equipment, thereby improving the accuracy and reliability of the electrical equipment defect positioning method. Therefore, the electrical equipment defect positioning method of the embodiment of the present application has higher accuracy and higher reliability.

[0038] The above method of the embodiment of the present application is further introduced below.

[0039] Optionally, in the above step S201, obtaining the initial acoustic image corresponding to the electrical device may further include the following execution steps:

[0040] Step S2011, setting a plurality of sound sources in the simulation space;

[0041] Step S2012, constructing a sound wave transmission model corresponding to the electrical equipment according to the sound wave transmission law;

[0042] Step S2013, acquiring an initial acoustic signal based on the acoustic wave transmission model and the sensor array in the simulation space;

[0043] Step S2014, performing signal enhancement processing and noise suppression processing on the initial acoustic signal in the target direction to obtain a target acoustic signal;

[0044] Step S2015 , performing spatial scanning on the simulation space to determine an initial acoustic image corresponding to the target acoustic signal on the target plane.

[0045] The above-mentioned multiple sound sources can be used to generate acoustic signals corresponding to multiple electrical devices. The above-mentioned sound wave transmission law can be the physical law followed by sound waves when propagating in space, such as sound wave attenuation, reflection, refraction, etc. The above-mentioned sensor array can be a plurality of sound sensors that receive acoustic signals, and the sensor array can be used to capture acoustic signals emitted by multiple sound sources. The above-mentioned spatial scanning can be signal processing in different directions and positions in the simulation space.

[0046] Specifically, multiple sound sources are placed in the simulation space to simulate multiple sound signal sources corresponding to electrical equipment in a defective state (such as the sound signal source of partial discharge, the sound signal source of foreign body friction, etc.). Further, according to the law of uniform transmission of sound waves, a sound wave transmission model is constructed. Further, a sensor array is used to receive acoustic signals corresponding to multiple sound sources, and the initial acoustic signal is obtained in combination with the sound wave transmission model. Further, a signal processing algorithm is used to perform signal enhancement processing and noise suppression processing on the initial acoustic signal in the target direction to obtain a target acoustic signal with higher signal strength in the target direction and lower signal strength in other directions. Further, the target acoustic signal is used to scan different directions of the space to achieve imaging of the target plane and obtain an initial acoustic image.

[0047] It is easy to understand that, through the above steps S2011 to S2015, in the embodiment of the present application, by setting up multiple sound sources, it is possible to simulate in the simulation space a variety of sound sources corresponding to electrical equipment defects that may generate acoustic signals, and it is possible to more comprehensively collect different types of acoustic signals, and provide diversified data for model training; in the embodiment of the present application, by performing signal enhancement processing and noise suppression processing on the initial acoustic signal, it is ensured that the acoustic signal is clearer and more reliable when the acoustic image is subsequently formed, and the signal-to-noise ratio of the signal is improved, and the position of the defect corresponding to the electrical equipment can be more accurately reflected in the initial acoustic image; by generating the initial acoustic image, the relative strength of the sound source signal in different directions can be intuitively displayed, and the corresponding defects in the electrical equipment can be more accurately located and identified. Therefore, the electrical equipment defect location method of the embodiment of the present application has higher accuracy and higher reliability.

[0048] Optionally, in the above step S202, based on the visible light image space corresponding to the electrical device, coordinate transformation is performed on the multi-channel acoustic data in the initial acoustic image to obtain the target acoustic image, which may further include the following execution steps:

[0049] Step S2021, determining the pitch angle, azimuth angle and target distance corresponding to a plurality of pixels in the initial acoustic image according to the multi-channel acoustic data, wherein the target distance is determined by the actual sound source position corresponding to the multi-channel acoustic data and the sensor array position;

[0050] Step S2022, determining the camera imaging plane according to the visible light image space;

[0051] Step S2023, calculating the pitch angle, azimuth angle and target distance, mapping the initial acoustic image onto the camera imaging plane, and obtaining the target acoustic image.

[0052] The pitch angle may be an angle parameter of the sound source in the vertical direction relative to the sensor array coordinate system. The azimuth angle may be an angle parameter of the sound source in the horizontal direction relative to the sensor array coordinate system. The target distance may be a straight-line distance from the sound source to the sensor array.

[0053] The camera imaging plane may be a two-dimensional image formed by a camera lens during shooting. The camera imaging plane may be a physical carrier of a visible light image. The camera imaging plane may be used to represent a projection plane of a visible light image space.

[0054] Specifically, by analyzing and calculating the multi-channel acoustic data, the pitch angle, azimuth angle and target distance corresponding to multiple pixels in the initial acoustic image map for describing the position of the sound source in the initial acoustic image map are obtained. Further, the image data on the surface of the electrical equipment acquired by the image acquisition device constitutes a visible light image space, and the camera imaging plane is determined. Further, through calculation, the pitch angle, azimuth angle and target distance in the initial acoustic image map can be associated with the pixel position on the camera imaging plane, and the pitch angle, azimuth angle and target distance in the initial acoustic image map can be converted into coordinates that match the coordinate system of the camera imaging plane in the visible light image space, and the initial acoustic image map is mapped to the camera imaging plane to obtain the target acoustic image map.

[0055] It is easy to understand that, through the above steps S2021 to S2023, in the embodiment of the present application, by converting the azimuth information of the acoustic signal into coordinates, it is not only possible to use the visual features in the image to assist acoustic positioning, so that the position of the acoustically detected defect can be accurately calibrated on the visible light image, thereby improving the positioning accuracy of the comprehensive detection; it is also possible to ensure the unity of acoustic and optical data, fuse the acoustic image with the visible light image space, and simultaneously use the information in the acoustic signal and the visible light image to realize multi-dimensional analysis of defects in power equipment. Even when the acoustic signal is interfered with or insufficient, it can be supplemented and verified by the visible light image, thereby improving the reliability of the detection result. Therefore, the electrical equipment defect positioning method of the embodiment of the present application has higher accuracy and reliability.

[0056] Optionally, the deep learning model includes a feature extraction network and a detection head; in the above step S203, using the deep learning model to perform feature analysis of multiple dimensions on the target acoustic image, and determining the defect information of the electrical equipment may also include the following execution steps:

[0057] Step S2031, using a feature extraction network to perform multi-level feature extraction on the target acoustic image to obtain acoustic image features and visible light image features;

[0058] Step S2032: Use the detection head to perform a fusion analysis on the acoustic image features and the visible light image features to determine the defect information.

[0059] The feature extraction network can be used to extract required features from input data, and the feature extraction network can include a series of convolutional layers, pooling layers, fully connected layers, etc. The detection head can be used to receive high-level features from the feature extraction network, and detect and / or classify specific targets through processing and analysis. The fusion analysis can be a process of combining information from different sources for analysis.

[0060] Specifically, a feature extraction network is used to perform multi-level feature extraction on the target acoustic image, extracting detailed features in the image layer by layer, such as edges, textures, shapes, etc., and finally obtaining abstract feature representations corresponding to high-level acoustic image features and visible light image features; further, the acoustic image features and visible light image features are used as input, and the defect information is determined through fusion analysis of the detection head.

[0061] It is easy to understand that through the above steps S2031 to S2032, in the embodiment of the present application, through the multi-level feature extraction and fusion analysis of the deep learning model, the characteristics of the defects of the power equipment can be effectively captured from the two dimensions of acoustics and vision. Acoustic features help to identify internal or hidden defects, while visual features can confirm the external manifestations and specific locations of defects. The scheme of the present application combines the advantages of both acoustic and visual methods, and can not only locate the location of defects more accurately, but also classify and identify the types of defects. Compared with the electrical equipment defect location method that only uses a single acoustic positioning or visual recognition, the electrical equipment defect location method of the present application provides more accurate and comprehensive defect detection results. In addition, in the embodiment of the present application, the automatic learning ability of the deep learning model is utilized, so that the electrical equipment defect location method can adapt to different types of power equipment and a variety of complex environmental conditions, and has high flexibility and adaptability.

[0062] Optionally, the above-mentioned electrical equipment defect location method may also include the following method steps: the feature extraction network includes multiple average pooling layers, and the kernel size of each average pooling layer in the multiple average pooling layers is 3, the stride is 2, and the edge extension is 1.

[0063] It is easy to understand that through the above-mentioned electrical equipment defect location method, in the embodiment of the present application, in the process of designing the feature extraction network, the feature pyramid structure is used to capture the features of different scales and positions in the target acoustic image, thereby improving the accuracy of the feature extraction network in locating and detecting defects of different sizes appearing in multiple positions in the power equipment. Therefore, the electrical equipment defect location method of the embodiment of the present application has higher accuracy and reliability.

[0064] Optionally, the above-mentioned electrical equipment defect location method may also include the following method steps: the feature extraction network includes multiple convolutional layers, among which the first convolutional layer has a kernel size of 7, a stride of 2, and an edge extension of 3; the second convolutional layer has a kernel size of 3, a stride of 2, and an edge extension of 1; and the third convolutional layer has a kernel size of 1, a stride of 1, and an edge extension of 0.

[0065] It is easy to understand that through the above-mentioned electrical equipment defect location method, in the embodiment of the present application, by setting multiple convolution layers with different parameters, the feature extraction network maintains a high spatial resolution while extracting features, and can effectively identify and locate various defects in power equipment even in complex backgrounds or low signal-to-noise ratio conditions, which not only enhances the learning ability of multi-scale acoustic features, but also optimizes the feature transfer process. Therefore, the electrical equipment defect location method of the embodiment of the present application is more accurate and reliable.

[0066] Optionally, in the above step S2032, the detection head performs a fusion analysis on the acoustic image features and the visible light image features to determine the defect information, which may also include the following execution steps:

[0067] Step S2132, superimpose the acoustic image features and the visible light image features to obtain superimposed features; use the fusion attention algorithm in the detection head to analyze the superimposed features and predict the defect information.

[0068] The superposition feature can be a comprehensive feature formed by combining the acoustic image feature and the visible light image feature. The superposition feature can be obtained through a variety of feature fusion technologies (such as point-to-point weighted sum, deep residual connection, multimodal feature fusion network, etc.).

[0069] The above-mentioned fusion attention algorithm can be used to automatically identify and highlight the parts required for electrical equipment defect location from the superimposed features, and the fusion attention algorithm can be constructed based on the attention mechanism. The above-mentioned attention mechanism can be a neural network component that imitates human visual attention. By utilizing the attention mechanism, the model can assign different weights to different features during the process of processing input, thereby focusing on information related to electrical equipment defect location.

[0070] It is easy to understand that through the above-mentioned step S2132, on the one hand, in the embodiment of the present application, the acoustic image features and the visible light image features are superimposed to obtain superimposed features, which can fuse the acoustic and visual information into a feature space, while retaining and enhancing the key information from the two modalities, while reducing redundancy and noise effects, thereby improving the efficiency of the deep learning model in determining the defect information of the electrical equipment; on the other hand, in the embodiment of the present application, the fusion attention algorithm is used to effectively process multimodal information, thereby improving the deep learning model's ability to identify electrical equipment defects.

[0071] Optionally, the above-mentioned electrical equipment defect location method may also include the following method steps: the deep learning model is obtained through machine learning training using multiple groups of training data, each group of training data includes acoustic image samples and real defect data, and the real defect data is used to characterize the real defect position and real defect category corresponding to the acoustic image sample.

[0072] The above-mentioned multiple sets of training data may include input data and labels corresponding to the input data. The above-mentioned labels can be used to assist the model in learning to predict the correct output from the input data.

[0073] The acoustic image samples can be obtained through simulation experiments or on-site collection, and the acoustic image samples can be input data in multiple sets of training data of the deep learning model. The real defect data can be the labels corresponding to the input data in multiple sets of training data of the deep learning model.

[0074] Specifically, the acoustic image samples are used as input data, and the real defect data corresponding to the acoustic image samples are used as the labels corresponding to the input data to form multiple sets of training data, and the deep learning model is trained using the multiple sets of training data. During the training process of the deep learning model, multiple parameters in the deep learning model are adjusted through the back propagation algorithm, so that the output of the deep learning model on the acoustic image samples gradually approaches the defect location and category represented by the real defect data.

[0075] It is easy to understand that, through the above-mentioned electrical equipment defect location method, in the embodiment of the present application, multiple sets of training data including acoustic image samples and real defect data are used to train the deep learning model, which can improve the accuracy and reliability of the model in electrical equipment defect location. Therefore, the electrical equipment defect location method of the embodiment of the present application has higher accuracy and higher reliability.

[0076] In an exemplary application scenario, Figure 3 FIG. 1 is a structural block diagram of an optional method for locating a defect in an electrical device according to an embodiment of the present application. Figure 3As shown, a simulation space is created in a simulation software (such as Matlab), a sound source is placed in the simulation space, and a sound wave transmission model is constructed according to the law of uniform transmission of sound waves. A sensor array (such as a sensor array constructed by setting up multiple microphones) is used in combination with the sound wave transmission model to obtain the initial acoustic signal received by each microphone in the sensor array.

[0077] Furthermore, the initial acoustic signal is subjected to signal enhancement processing and noise suppression processing in the target direction to impose different time delays to achieve audio signal enhancement and noise suppression in the target direction, thereby obtaining the target acoustic signal, performing spatial scanning in different directions of the simulation space, and determining an initial acoustic image corresponding to the target acoustic signal on the target plane.

[0078] Taking two microphones in the sensor array as an example, the process of performing signal enhancement processing and noise suppression processing on the initial acoustic signal in the target direction is described. Let the signals received by the two microphones in the sensor array be a1(n) and a2(n) respectively, then the signals received by the two microphones can be expressed by equation (1) and equation (2) respectively.

[0079] a1(t)=β1s(t-τ1)+n1(t) Formula (1)

[0080] In formula (1), β1 represents the signal attenuation of the sound source signal transmitted to the first microphone of the sensor array, s(t) represents the sound source signal, n1 represents the signal noise received by the first microphone of the sensor array, and τ1 represents the signal delay of the signal received by the first microphone of the sensor array relative to the sound source.

[0081] a2(t)=β2s(t-τ2)+n2(t) Formula (2)

[0082] In formula (2), β2 represents the signal attenuation of the sound source signal transmitted to the second microphone of the sensor array, n2 represents the signal noise received by the second microphone of the sensor array, and τ2 represents the signal delay of the signal received by the second microphone of the sensor array relative to the sound source.

[0083] If signal enhancement processing is used to delay a1(n) and synchronize a1(n) and a2(n), the cross-correlation function of a1(n) and a2(n) can be the maximum value, such as Figure 4 As shown. Since the time domain cross-correlation is equivalent to the inverse Fourier transform of the frequency domain cross-power spectrum, and the amount of calculation for cross-correlation in the frequency domain is small, the signal can be Fourier transformed, and the cross-correlation function of a1(n) and a2(n) can be the maximum value, which can be expressed by formula (3).

[0084]

[0085] In formula (3), A1(ω) and A2(ω) represent the Fourier transform corresponding to a1(n) and a2(n), respectively. represents a weighting function, by which noise can be suppressed.

[0086] All positions in the simulation space are divided into grid points, and the time delay τ is calculated according to each grid point position and substituted into the above formula (3). If there is a sound source near the grid point, the value of formula (3) will be relatively high, otherwise it will be relatively low. Thus, the initial acoustic image is obtained by spatial scanning.

[0087] Furthermore, the multi-channel acoustic data in the initial acoustic image is transformed into a target acoustic image using a coordinate transformation algorithm.

[0088] The above coordinate transformation algorithm is shown in formula (4).

[0089]

[0090] In formula (4), represents the pitch angle corresponding to multiple pixels in the initial acoustic image, θ represents the azimuth angle corresponding to multiple pixels in the initial acoustic image, d1 represents the target distance corresponding to multiple pixels in the initial acoustic image, (x s ,z s ) represents the coordinates of the pixels in the initial acoustic image mapped to the camera imaging plane.

[0091] Furthermore, skeleton 1 may be a network structure that converts the visible light image space into a two-dimensional network. By combining the network of skeleton 1 with the visible light image, the camera imaging plane may be determined. Skeleton 2 may be a network structure that performs coordinate transformation on the multi-channel acoustic data in the initial acoustic image. By combining the network of skeleton 2 with the initial acoustic image, the target acoustic image may be obtained.

[0092] Furthermore, Figure 3 As shown, a feature extraction network with a target acoustic image as input is constructed, and the feature extraction network has a feature pyramid structure. Specifically, the feature extraction network may include multiple average pooling layers, wherein the kernel size of each average pooling layer is 3, the stride is 2, and the edge extension is 1; multiple convolutional layers, wherein the kernel size of the first convolutional layer is 7, the stride is 2, and the edge extension is 3, the kernel size of the second convolutional layer is 3, the stride is 2, and the edge extension is 1, and the kernel size of the third convolutional layer is 1, the stride is 1, and the edge extension is 0.

[0093] Furthermore, Figure 3 As shown, a detection head including an acoustic attention module is constructed to locate defects of power equipment. The structure of the acoustic attention module detection head may include an encoding module and a decoding module.

[0094] Furthermore, the encoding module can be a structure composed of multiple acoustic attention modules, such as Figure 5 As shown, Figure 5 is a structural diagram of an optional encoding module according to an embodiment of the present application. The above acoustic attention module can be represented by formula (5).

[0095]

[0096] In formula (5), Q represents the query in the acoustic attention module. Q can be a vector obtained from the input data (acoustic image features or visible light image features) through linear transformation or fully connected layer. Q reflects the information type or feature pattern that the model currently pays attention to. K represents the key in the acoustic attention module. K can be the feature or content of the input information. V represents the value in the acoustic attention module.

[0097] In formula (5), Q can be calculated by formula (6).

[0098] Q=Q V +Q A Formula (6)

[0099] In formula (6), Q V Represents the input visible light image features, Q A represents the input acoustic image feature, Q V With Q A It can be obtained through feature extraction network.

[0100] Furthermore, the above decoding module can be a partial decoding module in the attention mechanism structure (DETEctionTRansformer, referred to as DETR) in the target detection model. By using the alternating stacking of self-attention and cross-attention in the decoding module of DETR, the decoding module of the above acoustic attention module detection head is constructed to achieve accurate positioning and classification of the target.

[0101] In an embodiment of the present application, an initial acoustic image corresponding to the electrical equipment is obtained, wherein the initial acoustic image is generated based on a simulation of a simulation space corresponding to the electrical equipment; based on the visible light image space corresponding to the electrical equipment, coordinate transformation is performed on the multi-channel acoustic data in the initial acoustic image to obtain a target acoustic image; a deep learning model is used to perform feature analysis of the target acoustic image in multiple dimensions to determine defect information of the electrical equipment, wherein the multiple dimensions include acoustic dimensions and visible light dimensions, and the defect information includes defect location and defect category. Thus, the present application achieves the purpose of more accurately locating defects in electrical equipment, thereby achieving the technical effect of improving the accuracy and reliability of the method for locating defects in electrical equipment, and further solving the technical problems of low accuracy and low reliability of the method for locating defects in electrical equipment in related technologies.

[0102] In this embodiment, an electrical equipment defect location device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, a "module" is a combination of software and / or hardware that can implement a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0103] Figure 6 is a structural block diagram of an electrical equipment defect location device according to an embodiment of the present application, such as Figure 6 As shown, the device includes: an acquisition module 601, used to acquire an initial acoustic image corresponding to the electrical equipment, wherein the initial acoustic image is generated based on a simulation space corresponding to the electrical equipment; a conversion module 602, used to perform coordinate conversion on multi-channel acoustic data in the initial acoustic image based on a visible light image space corresponding to the electrical equipment to obtain a target acoustic image; an analysis module 603, used to perform feature analysis on the target acoustic image in multiple dimensions using a deep learning model to determine defect information of the electrical equipment, wherein the multiple dimensions include acoustic dimensions and visible light dimensions, and the defect information includes defect location and defect category.

[0104] Optionally, the acquisition module 601 can also be used to set up multiple sound sources in the simulation space; construct a sound wave transmission model corresponding to the electrical equipment according to the law of sound wave transmission; acquire the initial acoustic signal based on the sound wave transmission model and the sensor array in the simulation space; perform signal enhancement processing and noise suppression processing on the initial acoustic signal in the target direction to obtain the target acoustic signal; perform spatial scanning on the simulation space to determine the initial acoustic image corresponding to the target acoustic signal on the target plane.

[0105] Optionally, the above-mentioned conversion module 602 can also be used to determine the pitch angle, azimuth angle and target distance corresponding to multiple pixels in the initial acoustic image according to the multi-channel acoustic data, wherein the target distance is determined by the actual sound source position and the sensor array position corresponding to the multi-channel acoustic data; determine the camera imaging plane according to the visible light image space; calculate the pitch angle, azimuth angle and target distance, map the initial acoustic image onto the camera imaging plane, and obtain the target acoustic image.

[0106] Optionally, the deep learning model includes a feature extraction network and a detection head. The above-mentioned analysis module 603 can also be used to use the feature extraction network to perform multi-level feature extraction on the target acoustic image to obtain acoustic image features and visible light image features; use the detection head to perform fusion analysis on the acoustic image features and visible light image features to determine defect information.

[0107] Optionally, the feature extraction network includes multiple average pooling layers, and each of the multiple average pooling layers has a kernel size of 3, a stride of 2, and an edge extension of 1.

[0108] Optionally, the feature extraction network includes multiple convolutional layers, among which the first convolutional layer has a kernel size of 7, a stride of 2, and an edge extension of 3, the second convolutional layer has a kernel size of 3, a stride of 2, and an edge extension of 1, and the third convolutional layer has a kernel size of 1, a stride of 1, and an edge extension of 0.

[0109] Optionally, the above-mentioned analysis module 603 can also be used to superimpose the acoustic image features and the visible light image features to obtain superimposed features; use the fusion attention algorithm in the detection head to analyze the superimposed features and predict the defect information.

[0110] Optionally, the deep learning model is obtained through machine learning training using multiple sets of training data, each set of training data includes acoustic image samples and real defect data, and the real defect data is used to characterize the real defect position and real defect category corresponding to the acoustic image samples.

[0111] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0112] According to another aspect of an embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute any one of the above-mentioned electrical equipment defect location methods.

[0113] Optionally, in this embodiment, the processor can be configured to perform the following steps through a computer program: obtaining an initial acoustic image corresponding to the electrical equipment, wherein the initial acoustic image is generated based on a simulation of a simulation space corresponding to the electrical equipment; performing coordinate transformation on multi-channel acoustic data in the initial acoustic image based on a visible light image space corresponding to the electrical equipment to obtain a target acoustic image; and performing feature analysis on the target acoustic image in multiple dimensions using a deep learning model to determine defect information of the electrical equipment, wherein the multiple dimensions include acoustic dimensions and visible light dimensions, and the defect information includes defect location and defect category.

[0114] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiment and its optional implementation manners, which will not be described in detail here.

[0115] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0116] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0117] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0118] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0119] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0120] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can essentially or in other words contribute to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, ROM, RAM, mobile hard disks, magnetic disks or optical disks.

[0121] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for locating defects in electrical equipment, characterized in that: include: Acquiring an initial acoustic image corresponding to the electrical device, wherein the initial acoustic image is generated based on a simulation of a simulation space corresponding to the electrical device; Based on the visible light image space corresponding to the electrical device, coordinate transformation is performed on the multi-channel acoustic data in the initial acoustic image to obtain a target acoustic image; A deep learning model is used to perform feature analysis on the target acoustic image in multiple dimensions to determine defect information of the electrical equipment, wherein the multiple dimensions include acoustic dimensions and visible light dimensions, and the defect information includes defect location and defect category.

2. The method for locating defects in electrical equipment according to claim 1, characterized in that: Acquiring the initial acoustic image corresponding to the electrical equipment includes: Setting a plurality of sound sources in the simulation space; According to the law of sound wave transmission, construct a sound wave transmission model corresponding to the electrical equipment; Acquiring an initial acoustic signal based on the acoustic wave transmission model and the sensor array in the simulation space; Performing signal enhancement processing and noise suppression processing on the initial acoustic signal in a target direction to obtain a target acoustic signal; The simulation space is spatially scanned to determine the initial acoustic image corresponding to the target acoustic signal on the target plane.

3. The method for locating defects in electrical equipment according to claim 1, characterized in that: Based on the visible light image space corresponding to the electrical device, coordinate transformation is performed on the multi-channel acoustic data in the initial acoustic image map to obtain the target acoustic image map, including: Determine, according to the multi-channel acoustic data, the pitch angle, the azimuth angle, and the target distance corresponding to a plurality of pixels in the initial acoustic image, wherein the target distance is determined by the actual sound source position and the sensor array position corresponding to the multi-channel acoustic data; Determining a camera imaging plane according to the visible light image space; The pitch angle, the azimuth angle and the target distance are calculated, and the initial acoustic image is mapped onto the camera imaging plane to obtain the target acoustic image.

4. The method for locating defects in electrical equipment according to claim 1, characterized in that: The deep learning model includes a feature extraction network and a detection head; using the deep learning model to perform feature analysis of the target acoustic image in multiple dimensions to determine the defect information of the electrical equipment includes: Using the feature extraction network to perform multi-level feature extraction on the target acoustic image to obtain acoustic image features and visible light image features; The detection head is used to perform a fusion analysis on the acoustic image features and the visible light image features to determine the defect information.

5. The method for locating defects in electrical equipment according to claim 4, characterized in that: The feature extraction network includes multiple average pooling layers, and each of the multiple average pooling layers has a kernel size of 3, a stride of 2, and an edge extension of 1.

6. The method for locating defects in electrical equipment according to claim 4 or 5, characterized in that: The feature extraction network includes multiple convolutional layers, among which the first convolutional layer has a kernel size of 7, a stride of 2, and an edge extension of 3, the second convolutional layer has a kernel size of 3, a stride of 2, and an edge extension of 1, and the third convolutional layer has a kernel size of 1, a stride of 1, and an edge extension of 0.

7. The method for locating defects in electrical equipment according to claim 4, characterized in that: The detection head performs a fusion analysis on the acoustic image feature and the visible light image feature to determine that the defect information includes: Superimposing the acoustic image feature and the visible light image feature to obtain a superimposed feature; The superimposed features are analyzed using a fusion attention algorithm in the detection head to predict the defect information.

8. The method for locating defects in electrical equipment according to claim 1, characterized in that: The deep learning model is obtained through machine learning training using multiple sets of training data, each set of training data includes acoustic image samples and real defect data, and the real defect data is used to characterize the real defect position and real defect category corresponding to the acoustic image sample.

9. An electrical equipment defect location device, characterized in that: include: An acquisition module, used for acquiring an initial acoustic image corresponding to the electrical device, wherein the initial acoustic image is generated based on a simulation of a simulation space corresponding to the electrical device; A conversion module, configured to perform coordinate conversion on the multi-channel acoustic data in the initial acoustic image based on the visible light image space corresponding to the electrical device, so as to obtain a target acoustic image; An analysis module is used to use a deep learning model to perform feature analysis on the target acoustic image in multiple dimensions to determine defect information of the electrical equipment, wherein the multiple dimensions include an acoustic dimension and a visible light dimension, and the defect information includes a defect location and a defect category.

10. An electronic device, characterized in that: It comprises a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the electrical equipment defect locating method according to any one of claims 1 to 8.

Images