Method for contactless diagnosing power facility using artificial intelligence and signal processing technology and device using the same

Abstract

A non-contact method for diagnosing power facilities using artificial intelligence and signal processing technology and a device using the same are disclosed. A method for diagnosing power facilities using artificial intelligence according to one embodiment includes the steps of: acquiring an ultrasonic signal for power facilities; generating a Mel Spectrogram based on the ultrasonic signal; - the Mel Spectrogram includes time information, frequency information, and magnitude information corresponding to the ultrasonic signal and is in the form of an image -, inputting the Mel Spectrogram into a pre-trained diagnostic model; and acquiring diagnostic data for power facilities based on the diagnostic model, wherein the diagnostic model may be constructed based on a convolutional neural network algorithm.

Description

Technology Field

[0001] The present invention relates to a non-contact power facility diagnosis method using artificial intelligence and signal processing technology and a device using the same. Background Technology

[0003] In general, power equipment such as overhead distribution lines, insulators, fuses, and switches can be damaged due to changes in the surrounding environment or equipment defects and deterioration; therefore, management including continuous inspection, removal of safety hazards, and equipment replacement is required.

[0004] In addition, various diagnostic methods are utilized to diagnose faults in power facilities. For example, methods such as the distribution line inspection method, which involves visual inspection by distribution line inspectors; the live-line equipment inspection method, which involves approaching live power facilities in a live-line bucket truck and measuring the shared voltage using visual inspection and a fork-type suspension insulator shared voltage meter; the measurement method using optical cameras to detect external defects in power facilities; and the thermal imaging camera measurement method, which measures the heat generated as deterioration progresses in power facilities to prevent distribution line failures.

[0005] However, this diagnostic method had the problem of being time-consuming and costly.

[0006] Therefore, diagnostic devices using ultrasound are being released recently. General power distribution equipment generates unique ultrasonic signals that are inaudible to humans by disturbing the surrounding air molecules through arc, tracking, and corona phenomena.

[0007] Such an ultrasonic diagnostic device is a device that converts ultrasonic signals into acoustic signals within a range audible to humans, allowing the sound to be heard through headphones (or speakers) and displaying them in the form of waveforms that can be easily seen with the eyes.

[0008] However, previously, as power equipment diagnosticians determined whether there was a fault by listening to acoustic signals, there was a problem in that diagnosing power equipment faults took a long time and the accuracy of the diagnosis could vary depending on the diagnostician's experience.

[0009] Recently, research to solve these problems has been ongoing. The problem to be solved

[0011] One objective of the present invention is to reduce the time required for fault diagnosis of power equipment and improve the accuracy of fault diagnosis by using artificial intelligence.

[0012] The problems to be solved by the present invention are not limited to those described above, and problems not mentioned will be clearly understood by those skilled in the art from this specification and the attached drawings. means of solving the problem

[0013] A method for diagnosing power equipment using artificial intelligence according to one embodiment comprises: a step of acquiring an ultrasonic signal for power equipment; a step of generating a Mel Spectrogram based on the ultrasonic signal; - the Mel Spectrogram includes time information, frequency information, and magnitude information corresponding to the ultrasonic signal and is in the form of an image -, a step of inputting the Mel Spectrogram into a pre-trained diagnostic model; and a step of acquiring diagnostic data for power equipment based on the diagnostic model, wherein the diagnostic model may be constructed based on a convolutional neural network algorithm. The means for solving the problem of the present invention are not limited to the means described above, and unmentioned means of solving will be clearly understood by those skilled in the art to which the present invention pertains from this specification and the attached drawings. Effects of the invention

[0015] According to one embodiment of the present invention, by performing fault diagnosis of power equipment using artificial intelligence, the time required for fault diagnosis can be reduced and the accuracy of fault diagnosis can be improved.

[0016] The effects of the present invention are not limited to the effects described above, and unmentioned effects will be clearly understood by those skilled in the art from this specification and the accompanying drawings. Brief explanation of the drawing

[0018] FIG. 1 illustrates a fault diagnosis system according to one embodiment. FIG. 2 is a block diagram illustrating a learning device according to one embodiment. FIG. 3 is a diagram illustrating a data processing method of a learning device according to one embodiment. FIGS. 4 to 7 are drawings for explaining the characteristics of data according to the type of failure according to one embodiment. FIG. 8 is a diagram illustrating an annotation tool for an ultrasonic signal according to one embodiment. FIG. 9 is a block diagram illustrating a diagnostic device according to one embodiment. FIG. 10 illustrates a fault diagnosis system according to another embodiment. FIG. 11 is a block diagram illustrating an ultrasonic acquisition device according to one embodiment. FIG. 12 is a diagram showing a diagnostic method of a diagnostic device according to one embodiment. FIG. 13 is a diagram illustrating the structure of a neural network of a diagnostic model according to one embodiment. FIG. 14 is a drawing for explaining a diagnostic model according to one embodiment. FIG. 15 is a drawing for illustrating exemplary diagnostic data according to one embodiment. Specific details for implementing the invention

[0019] The aforementioned objects, features, and advantages of the present invention will become more apparent from the following detailed description in conjunction with the accompanying drawings. However, as the present invention is subject to various modifications and may have various embodiments, specific embodiments are illustrated in the drawings and described in detail below.

[0020] In the drawings, the thicknesses of layers and regions are exaggerated for clarity. Additionally, when an element or layer is referred to as being "on" or "above" another element or layer, it includes not only being directly on top of the other element or layer but also cases where another layer or element is interposed in between. Throughout the specification, identical reference numerals generally denote identical elements. Furthermore, elements with identical functions within the scope of the same concept appearing in the drawings of each embodiment are described using the same reference numeral.

[0021] If it is determined that a detailed description of known functions or configurations related to the present invention could unnecessarily obscure the essence of the invention, such detailed description is omitted. Furthermore, numbers used in the description of this specification (e.g., First, Second, etc.) are merely identification symbols to distinguish one component from another.

[0022] Furthermore, the suffixes "module" and "part" for components used in the following description are assigned or used interchangeably solely for the ease of drafting the specification, and do not inherently possess distinct meanings or roles.

[0024] According to one embodiment of the present invention, a fault diagnosis system may be provided.

[0025] FIG. 1 illustrates a fault diagnosis system according to one embodiment.

[0026] Referring to FIG. 1, the fault diagnosis system (10) may include a learning device (100) for training a diagnostic model, a diagnostic device (200) for performing a fault diagnosis of power equipment using the diagnostic model, and an ultrasonic acquisition device (300) for measuring ultrasonics on power equipment. Additionally, the ultrasonic acquisition device (300) may include an optical measurement unit capable of photographing the external condition of power equipment and a temperature measurement unit (e.g., an infrared measurement unit) capable of measuring the temperature of power equipment. Additionally, the ultrasonic acquisition device (300) may include a humidity measurement unit for measuring the humidity of power equipment, a distance measurement unit for measuring the distance to power equipment, and a location acquisition unit for acquiring location information of the ultrasonic acquisition device.

[0027] Additionally, the learning device (100) of the fault diagnosis system (10), the diagnosis device (200), and the ultrasonic acquisition device may be composed of at least one or more.

[0028] The learning device (100) can perform learning (training) of a diagnostic model. For example, the learning device (100) can perform training of a diagnostic model using a first learning data set in which an ultrasonic signal measured when an arc occurs in the power equipment is converted into input data, a second learning data set in which an ultrasonic signal measured when tracking occurs in the power equipment is converted into input data, a third learning data set in which an ultrasonic signal measured when corona occurs in the power equipment is converted into input data, and a fourth learning data set in which an ultrasonic signal measured when the power equipment performs normal operation is converted into input data. At this time, each of the first to third learning data sets can be labeled with a failure phenomenon occurring in the power equipment. In addition, the fourth learning data set can be labeled as normal.

[0029] Additionally, the input data included in the training data sets may have various formats. For example, the learning device (100) may perform visualization on the ultrasonic signal to convert the ultrasonic signal into an image. For example, the learning device (100) may convert the ultrasonic signal into a Mel Spectrogram, which will be described later, and use the Mel Spectrogram as input data. Also, in another example, the learning device (100) may apply a Fourier transform to the ultrasonic signal to generate frequency data and use the frequency data as input data.

[0030] Additionally, in this specification, diagnosing a failure of power equipment may include diagnosing whether the power equipment is operating normally or diagnosing a failure phenomenon that has occurred in the power equipment.

[0031] The diagnostic device (200) can diagnose a fault in power equipment using a diagnostic model. For example, the diagnostic device (200) can diagnose a fault in power equipment using a diagnostic model learned by a learning unit.

[0032] In one embodiment, the diagnostic device (200) may be a handheld device such as a smartphone, tablet, or PC, or a stationary device such as a computer or server.

[0033] The ultrasonic acquisition device (300) can acquire an ultrasonic signal generated from power equipment and provide the ultrasonic signal (or ultrasonic data representing the ultrasonic signal) to the diagnostic device (200). Additionally, according to an embodiment, the ultrasonic acquisition device (300) can acquire an acoustic signal in an audible frequency band that is audible to humans from the acquired ultrasonic signal and provide the ultrasonic signal and / or acoustic signal to the diagnostic device (200).

[0034] In the fault diagnosis system (10) according to the present embodiment, the learning device (100) acquires a learning data set and performs learning of the diagnosis model, the ultrasonic acquisition device (300) provides an ultrasonic signal to the diagnosis device (200), and the diagnosis device (200) can perform fault diagnosis on the power equipment using the ultrasonic signal acquired from the ultrasonic acquisition device (300) using the diagnosis model.

[0035] Additionally, in another embodiment, the learning device (100) and the diagnostic device (200) may each be in the form of a server device. Additionally, the learning device (100) and the diagnostic device (200) may be physically implemented as a single device or a single server device.

[0036] Additionally, in another embodiment, the diagnostic device (200) and the ultrasound acquisition device (300) may be implemented as a single device. Also, depending on the embodiment, the learning device (100), the diagnostic device (200), and the ultrasound acquisition device (300) may be implemented as a single device.

[0037] The fault diagnosis system disclosed in this specification is not limited to the embodiments described above and can be implemented in any form including a learning device (100), a diagnosis device (200), and an ultrasonic acquisition device (300).

[0039] FIG. 2 is a block diagram illustrating a learning device according to one embodiment.

[0040] Referring to FIG. 2, the learning device (100) may include a memory unit (110), a control unit (120), and a communication unit (130).

[0041] The control unit (120) can control the operation of the learning device (100).

[0042] The control unit (120) may include one or more of the following: a CPU (Central Processing Unit), RAM (Random Access Memory), GPU (Graphic Processing Unit), one or more microprocessors, and other electronic components capable of processing input data according to predetermined logic.

[0043] The control unit (120) can read system programs and various processing programs stored in the memory unit (110).

[0044] The learning device (100) may include a memory unit (110). The memory unit (110) may store data required for learning and a learned diagnostic model.

[0045] The memory unit (110) may be implemented as a non-volatile semiconductor memory, a hard disk, a flash memory, RAM, ROM (Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), or other tangible non-volatile recording media.

[0046] The memory unit (110) can store various processing programs, parameters for performing processing of the programs, or such processing result data.

[0047] In addition, in one embodiment, the learning device (100) can perform data processing and learning. The control unit (120) can process or preprocess data for performing learning during the data processing process. For example, the control unit (120) can process data using a predetermined data processing process program. In addition, the control unit (120) can perform learning of a diagnostic model based on the processed data during the learning process. In one embodiment, the processed data may be in the form of an image, and in this case, the diagnostic model may be an image analysis-based diagnostic model (e.g., a Convolutional Neural Network (CNN)).

[0048] The communication unit (130) can communicate with an external device. For example, the communication unit (130) can communicate with a diagnostic device and / or an ultrasound acquisition device described later. The communication unit (130) can perform wired or wireless communication. The communication unit (130) can perform bidirectional or unidirectional communication.

[0049] The following describes data processing methods, characteristics of each data type according to failure types, and annotations (labeling).

[0051] FIG. 3 is a diagram illustrating a data processing method of a learning device according to one embodiment.

[0052] Referring to FIG. 3, the learning device can perform pre-emphasis on the ultrasonic signal (S110). Here, pre-emphasis may mean strengthening the frequency in advance to reduce the influence of noise in the ultrasonic signal. For example, the learning device may emphasize the amplitude of the high-frequency component to reduce the influence of noise at high frequencies.

[0053] Additionally, the learning device can sample the ultrasonic signal after pre-emphasis has been performed (S120). For example, the learning device can sample the ultrasonic signal after pre-emphasis has been performed at a frequency of 16 kHz in the range of -1.0 to 1.0. Of course, such sampling can be performed at various frequencies such as 32 kHz.

[0054] Additionally, in step S120, the learning device may frame the ultrasonic signal by applying windowing to the sampled ultrasonic signal. Applying windowing in this way may be intended to minimize discontinuities at the beginning and end of each frame. For example, the learning device may generate multiple frames by applying 0.96-second windows and 0.48-second hops to the sampled ultrasonic signal. Additionally, the learning device may generate multiple frames by using the Hamming window technique on the sampled ultrasonic signal. Of course, the learning device may also generate multiple frames using various other window techniques (Rectangular Window, B-spline window, Welch window, Sine window, Cosine-sum windows (e.g., Blackman window, etc.), Adjustable windows (e.g., Gaussian window, Confined Gaussian window), Hybrid windows (e.g., Bartlett-Hann window, etc.)).

[0055] Additionally, the learning device can perform a Fourier transform on the generated frames (S130). The frames generated in step S120 may be composed of a Discrete Time Domain, and the learning device can obtain frequency data by performing a Discrete Fourier Transform operation on the acoustic signal using a Fast Fourier Transform (FFT) algorithm. Additionally, the learning device can obtain frequency data on the acoustic signal using a Short Time Fourier Transform (STFT) algorithm. Furthermore, the learning device can generate a power spectrum by squaring the magnitude component of the frequency data and generate a log spectrum by applying a log scale to the magnitude component of the power spectrum. Then, the learning device can generate a spectrogram by combining the log spectrum frame by frame. Such a spectrogram can have three-dimensional information (time, frequency, magnitude). For example, in a spectrogram, the x-axis represents time information (frames), the y-axis represents frequency, and the z-axis (e.g., color) represents magnitude (decibels).

[0056] Additionally, the learning device can apply a Mel Filter Bank to the spectrogram. By applying the Mel Filter Bank, the spectrometer can be converted into a Mel Scale. Accordingly, the frequency of the spectrometer is converted into a Mel Frequency, and a Mel Spectrogram can be generated as the power at each point is converted based on a logarithm. The Mel Spectrogram can reflect the characteristic that the human auditory system is sensitive to frequency changes at low frequencies and less sensitive to frequency changes at high frequencies.

[0057] The learning device can perform training of the diagnostic model using a Mel spectrogram according to the aforementioned data processing method.

[0058] In addition, the aforementioned data processing method can be used to perform fault diagnosis of power equipment not only in learning devices but also in diagnostic devices.

[0060] FIGS. 4 to 7 are drawings for explaining the characteristics of data according to the type of failure according to one embodiment.

[0061] Based on the characteristics of data according to the types of failures described in Figures 4 to 7, the failure of power equipment can be diagnosed in the diagnostic model.

[0062] In FIGS. 4 to 6, (a) represents the waveform of a sampled ultrasonic signal, (b) represents a power spectrum, and (c) represents a mel-spectrum and the waveform of the ultrasonic signal corresponding to the mel-spectrum. In the waveform of (a), the x-axis represents time and the y-axis represents amplitude; in the power spectrum of (b), the x-axis represents frequency and the y-axis represents magnitude; and in the mel-spectrum of (c), the x-axis represents time, the y-axis represents frequency, and the z-axis (e.g., color) represents magnitude (decibel). Additionally, in (c), the x-axis of the waveform of the ultrasonic signal corresponding to the mel-spectrum represents time and the y-axis represents amplitude.

[0063] Referring to FIG. 4, FIG. 4 can show the characteristics of data when an arc occurs in power equipment. When an arc occurs in power equipment, an abnormal explosive sound (erratic burst) with rapid onset and cessation of energy may occur in the power equipment. Accordingly, the waveform of the ultrasonic signal in (a) may contain many values ​​that change rapidly compared to the average band. In addition, the amplitude of the waveform may increase when an arc occurs.

[0064] In addition, in the power spectrum of (b), a 60 Hz harmonic component may appear at a low frequency. Also, in the Mel spectrogram of (c), the moment when a fault signal occurs can be distinguished by the shape of a band. In addition, compared to a normal signal, the frequency is higher and the decibel level may also appear relatively higher.

[0065] Referring to FIG. 5, FIG. 5 can show the characteristics of data when corona occurs in power equipment. When corona occurs in power equipment, a continuous buzzing sound may occur in the power equipment. Accordingly, in the waveform of the ultrasonic signal of (a), a certain frequency band may occur with a peak slightly higher than the average band. In addition, a waveform with a somewhat large amplitude may occur continuously.

[0066] In addition, in the power spectrum of (b), numerous 60Hz harmonic components may appear. Furthermore, as the corona generation in the power equipment becomes more severe, the magnitude of the 60Hz harmonic components may tend to gradually decrease. Also, in the Mel spectrogram of (c), although it can be distinguished from noise, a faint band shape may appear compared to when an arc occurs in the power equipment.

[0067] Referring to FIG. 6, FIG. 6 illustrates the characteristics of data when tracking occurs in power equipment. When tracking occurs in power equipment, a buzzing sound accompanied by a somewhat small explosion sound and / or noise may be generated in the power equipment. Accordingly, the waveform of the ultrasonic signal in (a) may have a high peak. In addition, in the power spectrum of (b), a 60Hz harmonic component appears, but the magnitude of the 60Hz harmonic component gradually decreases, and a constant pattern may not appear. Furthermore, in the Mel spectrogram of (c), the period may be shorter and the overall decibel level may be higher than when corona occurs in the power equipment.

[0068] Referring to FIG. 7, FIG. 7 can show a Mel spectrogram and a waveform of an ultrasonic signal corresponding to the Mel spectrogram when the power equipment is operating normally without any failures. In FIG. 7, the x-axis of the Mel spectrogram may represent time, the y-axis may represent frequency, and the z-axis (e.g., color) may represent magnitude (decibel). In FIG. 7, the x-axis of the waveform of the ultrasonic signal corresponding to the Mel spectrogram may represent time, and the y-axis may represent amplitude.

[0069] When power equipment is operating normally, the amplitude of the waveform of the ultrasonic signal increases depending on the magnitude of the noise, and specific frequency characteristics may not appear in the power spectrum. In addition, in a Mel spectrogram such as that shown in Fig. 7, a distinct band shape relative to the amplitude does not appear, and an overall blurry signal period may be displayed.

[0071] FIG. 8 is a diagram illustrating an annotation tool for an ultrasonic signal according to one embodiment.

[0072] Referring to FIG. 8, FIG. 8 may show the UI of an annotation tool. A learning device or a device capable of annotating (or labeling) ultrasonic signals (e.g., PC, mobile, etc.) (or the aforementioned local device) may provide an annotation tool such as FIG. 8. For convenience, the annotation tool is described below as being performed on a learning device, but it is not limited thereto, and the annotation tool can be performed on various devices.

[0073] Specifically, when the learning device receives an ultrasonic signal, it can display the Mel spectrogram (501) and the waveform (502) of the ultrasonic signal provided through the data processing method described above via the UI of the annotation tool. Additionally, the learning device can provide a labeling item (503). The learning device can receive a selection from the operator of the type of labeling item (503) regarding the state of the power equipment, such as normal operation, arc, corona, and tracking. Additionally, the type of the labeling item (503) may consist of two types: normal state and fault state. Additionally, the type of the labeling item (503) may consist of two or more sub-types, such as arc and fault state.

[0074] Additionally, the position, length, etc. of each labeling item (503) can be selected by the operator. Furthermore, in one embodiment, the learning device can analyze the waveform and / or Mel spectrogram of the ultrasonic signal to pre-set the position and length of each labeling item (503), and display each labeling item (503) with the set position and length on the UI of the annotation tool.

[0075] When the type, location, and / or length of each labeling item (503) are set, the learning device can separate the ultrasonic signal according to the location and length of each labeling item (503) and distinguish and store the separated ultrasonic signal according to the type of each labeling item (503). Accordingly, the learning device can independently store and manage original data (separated ultrasonic signal), refined data (waveform, power spectrum, Mel spectrogram, etc. corresponding to the separated ultrasonic signal), and labeling data in which the labeling item (503) is added to the original data and refined data. Additionally, the learning device can perform learning of a diagnostic model using the labeling data.

[0076] In this way, as the waveform and Mel spectrogram of the ultrasonic signal are automatically provided through the annotation tool and labeling can be performed according to the labeling item (503), the operator performing the annotation work can perform annotation on the input data more conveniently.

[0078] FIG. 9 is a block diagram illustrating a diagnostic device according to one embodiment.

[0079] Referring to FIG. 9, the diagnostic device (200) may include a memory unit (210), a control unit (220), and a communication unit (230).

[0080] The control unit (220) can generate diagnostic data using a diagnostic model. The control unit (220) can acquire frequency data for diagnosis and acquire diagnostic data regarding whether the power equipment is faulty using a learned diagnostic model.

[0081] The control unit (220) may include one or more of the following: a CPU (Central Processing Unit), RAM (Random Access Memory), GPU (Graphic Processing Unit), one or more microprocessors, and other electronic components capable of processing input data according to predetermined logic.

[0082] In addition, in one embodiment, the control device (200) can perform data processing and diagnosis. The control unit (220) can process or preprocess the ultrasonic signal obtained from the ultrasonic acquisition device. Since the data processing method of the learning device described in FIG. 3 can be applied as is for data processing, a detailed description is omitted.

[0083] In addition, the control unit (220) can perform a diagnosis of the power equipment using the processed data and the diagnostic model.

[0084] Additionally, the memory unit (210) can store a learned diagnostic model. The memory unit (210) can store a data processing process program for performing diagnostic assistance, a diagnostic process program, parameters and variables of the diagnostic model, etc. The control unit (220) can perform a diagnosis of the power equipment using the stored diagnostic model.

[0085] Additionally, the memory unit (210) may be implemented as a non-volatile semiconductor memory, a hard disk, a flash memory, RAM, ROM (Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), or other tangible non-volatile recording media.

[0086] The communication unit (230) can communicate with a learning device and / or an ultrasound acquisition device. For example, the diagnostic device (200) may be provided in the form of a server that communicates with the ultrasound acquisition device.

[0087] In addition, according to one embodiment, the fault diagnosis system may include at least one server device.

[0088] The server device can store and / or run a diagnostic model. The server device can store weight values ​​that constitute the trained diagnostic model. Additionally, the server device can collect or store data used for diagnosis.

[0089] The server device can output the results of a diagnostic assistance process using a diagnostic model to an external device and obtain feedback from the external device. The server device can operate similarly to the diagnostic device described above.

[0091] FIG. 10 illustrates a fault diagnosis system according to another embodiment.

[0092] Referring to FIG. 10, a fault diagnosis system (20) according to one embodiment of the present invention may include a diagnosis server (400), a first local device (410a), and a second local device (410b).

[0093] The diagnostic server (400) may mean that the diagnostic device of the present specification is implemented in the form of a server. The diagnostic server (400) may include a pre-trained diagnostic model.

[0094] The first local device (410a) and the second local device (410b) may be devices that are easy for a worker to use, such as mobile phones, PCs, and tablets. Additionally, an application capable of accessing the diagnostic server (400) is installed on the first local device (410a) and the second local device (410b), and communication with the diagnostic server (400) can be performed through said application.

[0095] Additionally, the first local device (410a) communicates with a first ultrasonic acquisition device (not shown) and can acquire a first ultrasonic signal for the first power facility from the first ultrasonic acquisition device (not shown). Additionally, the second local device (410b) communicates with a second ultrasonic acquisition device (not shown) and can acquire a second ultrasonic signal for the second power facility from the second ultrasonic acquisition device (not shown).

[0096] The diagnostic server (400) can acquire a first ultrasonic signal and a second ultrasonic signal from each of the first local device (410a) and the second local device (410b), generate first diagnostic data corresponding to the first ultrasonic signal and second diagnostic data corresponding to the second ultrasonic signal using a diagnostic model, provide the first diagnostic data to the first local device (410a), and provide the second diagnostic data to the second local device (410b).

[0097] Additionally, according to an embodiment, a pre-learned diagnostic model may be included in each of the first local device (410a) and the second local device (410b). In this case, the first local device (410a) generates first diagnostic data corresponding to a first ultrasound signal using the diagnostic model, and the second local device (410b) generates second diagnostic data corresponding to a second ultrasound signal using the diagnostic model, and the diagnostic server (400) can obtain the first diagnostic data and the second diagnostic data from the first local device (410a) and the second local device (410b).

[0098] In this way, the diagnostic server (400) can monitor and manage the status of the first diagnostic facility and the second diagnostic facility by acquiring diagnostic data of the first diagnostic facility and the second diagnostic facility located in different regions through communication with the first local device (410a) and the second local device (410b). For example, the diagnostic server (400) can perform history management for the first diagnostic facility and the second diagnostic facility by periodically collecting the first diagnostic data and the second diagnostic data. In addition,

[0100] FIG. 11 is a block diagram illustrating an ultrasonic acquisition device according to one embodiment.

[0101] Referring to FIG. 11, an ultrasonic acquisition device (300) according to one embodiment of the present invention may include a memory unit (310), a control unit (320), and a communication unit (330).

[0102] The control unit (320) can acquire frequency data for power equipment. In one embodiment, the control unit (320) may include an ultrasonic sensor. The ultrasonic sensor can acquire an ultrasonic signal for power equipment. Additionally, the control unit (320) can acquire an acoustic signal by converting the ultrasonic signal into an audible frequency band, and acquire frequency data by converting the acoustic signal into the frequency domain. The control unit (320) can transmit the ultrasonic signal, acoustic signal, and / or frequency data to a diagnostic device or the aforementioned server device through the communication unit (330).

[0103] The control unit (320) may include one or more of the following: a CPU (Central Processing Unit), RAM (Random Access Memory), GPU (Graphic Processing Unit), one or more microprocessors, and other electronic components capable of processing input data according to predetermined logic.

[0104] The memory unit (310) can store various processing programs, parameters for performing processing of the programs, or data resulting from such processing. For example, the memory unit (310) can store ultrasonic signals, acoustic signals, and / or frequency data.

[0105] Additionally, the memory unit (310) may be implemented as a non-volatile semiconductor memory, a hard disk, a flash memory, RAM, ROM (Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), or other tangible non-volatile recording media.

[0106] The communication unit (330) can communicate with an external device, such as a diagnostic device or a server device. The communication unit (330) can perform wired or wireless communication.

[0107] Although not illustrated, the ultrasonic acquisition device may further include an output unit. The output unit may include a display for outputting video or images or a speaker for outputting sound. The output unit may output an acoustic signal through the speaker or output a video representing the acoustic signal through the display.

[0108] Although not illustrated, the ultrasonic acquisition device may further include an input unit. The input unit can acquire user input.

[0109] Additionally, although not illustrated, the ultrasonic acquisition device may include an optical measuring unit capable of photographing the external condition of the power equipment and a temperature measuring unit (e.g., an infrared measuring unit) capable of measuring the temperature of the power equipment.

[0111] FIG. 12 is a diagram showing a diagnostic method of a diagnostic device according to one embodiment.

[0112] Referring to FIG. 12, the diagnostic method of the diagnostic device may include the step of acquiring an ultrasonic signal for power equipment (S210), the step of applying the ultrasonic signal to a pre-trained diagnostic model (S220), and the step of acquiring diagnostic data for power equipment based on the diagnostic model (S230).

[0113] In step S210, the diagnostic device can acquire an ultrasonic signal for the power equipment. As previously mentioned, the ultrasonic signal may have a waveform. For example, the diagnostic device can acquire an ultrasonic signal from the aforementioned ultrasonic acquisition device.

[0114] In addition, the diagnostic device can perform data processing on the ultrasonic signal. Since the data processing method of the learning device described in FIG. 3 can be applied directly to the data processing of the ultrasonic signal, a detailed description is omitted. Depending on the data processing method, the diagnostic device can acquire a Mel spectrogram based on the ultrasonic signal.

[0115] Additionally, in step S220, the diagnostic device may input data in which the ultrasound signal has been processed as input data to a pre-trained diagnostic model. For example, the diagnostic device may input a Mel spectrogram to the pre-trained diagnostic model. In the case of a Mel spectrogram, it may be an image having three-dimensional information (time, frequency, magnitude). Accordingly, the diagnostic model may be a neural network (e.g., CNN) capable of performing image analysis.

[0116] As a specific example, a diagnostic model can be constructed based on the Depthwise-Separable Convolutional Neural Network (Depthwise-Separable CNN) algorithm.

[0117] Using FIG. 13, a depth-separable neural network is described. FIG. 13 is a diagram for explaining the structure of a neural network of a diagnostic model according to one embodiment.

[0118] Referring to FIG. 13, the diagnostic device may input a Mel spectrogram into the diagnostic model. At this time, the neural network of the diagnostic model may include a depthwise convolution layer (510) and a pointwise convolution layer (520). Additionally, each of the depthwise convolution layer and the pointwise convolution layer may include at least one sublayer. Each layer included in the diagnostic model may be processed sequentially. For example, the first layer may generate an intermediate output for the input, and the intermediate output generated by the first layer may become the input of the second layer, and the intermediate output may be processed by the second layer.

[0119] The diagnostic model can separate the input MEL spectrogram by channel. For example, the MEL spectrogram may be separated by RGB channels or by other criteria. The depth-wise convolution layer can perform convolution on each single channel. For instance, the depth-wise convolution layer can extract intermediate feature vectors by performing convolution on each single channel using a kernel of size nxn (where n is an integer greater than or equal to 1). Here, the extracted intermediate feature vectors may possess spatial characteristics. In this way, the computational load of the diagnostic model can be reduced by performing convolution on each single channel rather than on the entire image. Additionally, the intermediate feature vectors obtained by convolution on each channel are input into the point-wise convolution layer, which can perform convolution on the intermediate feature vectors using a kernel of size 1x1. The output values ​​of each channel of the Mel specogram can be combined into one by the pointwise convolution layer. As the diagnostic model is built based on a depthwise divisible convolutional neural network algorithm, the computational load can be reduced and the model can be made lighter compared to models based on existing convolutional neural network algorithms. As a result, the diagnostic model can be run not only on servers but also on various devices (PC, mobile, etc.).

[0120] In addition, the output values ​​produced through the depth-wise convolution layer and the point-wise convolution layer in the diagnostic model can be configured in various ways. For example, the output value may represent the probability that the power equipment is in a specific state.

[0121] For example, if a diagnostic model is trained based on a training data set labeled as normal state and a training data set labeled as fault state, the output of the diagnostic model may represent the probability that the power equipment is in a normal state and the probability that it is in an abnormal state.

[0122] In addition, if the diagnostic model is trained based on a training data set labeled as normal state, a training data set labeled as arc state, a training data set labeled as corona state, and a training data set labeled as tracking state, the output value of the diagnostic model may include the probability that the power equipment is in a normal state, the probability that it is in an arc state, the probability that it is in a corona state, and the probability that it is in a tracking state.

[0123] In addition, in another embodiment, the diagnostic model may be constructed based on Long Short Term Memory (LSTM). LSTM is intended to compensate for the long-term dependency problem of general RNNs; by adding input gates, erase gates, and output gates to the memory cells of the hidden layer, unnecessary values ​​can be deleted and important values ​​set within the diagnostic model. In this case, the input data may be frequency data obtained by Fourier transforming an ultrasonic signal. Since frequency data for an ultrasonic signal contains information about frequencies over a relatively long period of time, the accuracy of the diagnostic result may be lower if the diagnostic model is constructed based on a general RNN. Therefore, to improve the accuracy of the diagnostic result, the diagnostic model may be constructed based on LSTM. Of course, it is not limited to this, and the diagnostic model may also be constructed based on various algorithms such as CNN, RNN, CRNN, STN, and GRU.

[0124] Referring again to FIG. 12, in step S230, the diagnostic device can acquire diagnostic data for power equipment based on a diagnostic model.

[0125] In one embodiment, the diagnostic data may include information regarding the diagnostic result by a diagnostic model, that is, whether the power equipment is in a faulty state and / or a normal state. Additionally, the diagnostic data may include information regarding the probability that the power equipment is in a faulty state or a normal state. Additionally, the diagnostic data may include information regarding whether a fault phenomenon, such as an arc, tracking, or corona, has occurred in the power equipment, or information regarding the probability that said fault phenomenon will occur.

[0127] More specifically, using FIG. 14, FIG. 14 is a drawing for explaining a diagnostic model according to one embodiment.

[0128] Referring to FIG. 14, as in (a), the first diagnostic model (710) can be pre-trained based on a training data set labeled that the power equipment is in a faulty state and a training data set labeled that the power equipment is in a normal state. Additionally, according to an embodiment, the training data set labeled that the power equipment is in a faulty state may include a training data set labeled that the power equipment is in an arc state, a training data set labeled that the power equipment is in a tracking state, and a training data set labeled that the power equipment is in a corona state.

[0129] Additionally, the first diagnostic model (710) may include the aforementioned neural network (e.g., the aforementioned depth-divided neural network). When input data (e.g., a Mel spectrogram in which an ultrasonic signal has been processed) is input to the first diagnostic model (710), the first diagnostic model (710) may output a diagnostic result including a final diagnostic state (a result regarding whether the power equipment is in a fault state, a normal state, or the type of fault state) representing the state of the power equipment corresponding to the input data. Additionally, the diagnostic result may include information regarding the probability that the power equipment is in an arc state, the probability that the power equipment is in a corona state, the probability that the power equipment is in a tracking state, the probability that the power equipment is in a fault state, and / or the probability that the power equipment is in a normal state.

[0130] Additionally, as in (b), the second diagnostic model (720) may include a first detailed diagnostic model (721), a second detailed diagnostic model (722), and a decision unit (723). Additionally, each of the first detailed diagnostic model (721) and the second detailed diagnostic model (722) may include the aforementioned neural network (e.g., the aforementioned depth-separable neural network). In this case, the first detailed diagnostic model (721) may be pre-learned based on a learning data set labeled that the power equipment is in a faulty state, and the second detailed diagnostic model (722) may be pre-learned based on a learning data set labeled that the power equipment is in a normal state. Additionally, according to an embodiment, the learning data set labeled that the power equipment is in a faulty state may include a learning data set labeled that the power equipment is in an arc state, a learning data set labeled that the power equipment is in a tracking state, and a learning data set labeled that the power equipment is in a corona state. Accordingly, when input data is input to the first detailed diagnostic model (721), a first detailed diagnostic result may be output, which indicates the probability that the power equipment corresponding to the input data is in a faulty state. Additionally, according to an embodiment, the first detailed diagnostic result may include the probability that the power equipment is in an arc state, the probability that the power equipment is in a corona state, and the probability that the power equipment is in a tracking state.

[0131] Additionally, when input data is input into the second detailed diagnostic model (722), a second detailed diagnostic result may be output, which indicates the probability that the power equipment corresponding to the input data is in a normal state. The determination unit (723) determines whether the power equipment corresponding to the input data is in a fault state and / or the type of fault state based on the first detailed diagnostic result output from the first detailed diagnostic model (721) and the second detailed diagnostic result output from the second detailed diagnostic model (722), and can obtain a diagnostic result including the final diagnostic state (result regarding whether the power equipment is in a fault state, a normal state, or the type of fault state), the probability that the power equipment is in an arc state, the probability that the power equipment is in a corona state, the probability that the power equipment is in a tracking state, the probability that the power equipment is in a fault state, and / or the probability that the power equipment is in a normal state.

[0132] Additionally, as in (c), the third diagnostic model (730) may include a first detailed diagnostic model (731), a second detailed diagnostic model (732), a third detailed diagnostic model (733), and a fourth detailed diagnostic model (734). Each of the first detailed diagnostic model (731), the second detailed diagnostic model (732), the third detailed diagnostic model (733), and the fourth detailed diagnostic model (734) may include the aforementioned neural network (e.g., the aforementioned depth-separable neural network).

[0133] The first detailed diagnostic model (731) may be pre-learned based on a learning data set labeled that an arc has occurred in the power equipment, and the second detailed diagnostic model (732) may be pre-learned based on a learning data set labeled that a corona has occurred in the power equipment. Additionally, the third detailed diagnostic model (733) may be pre-learned based on a learning data set labeled that a tracking has occurred in the power equipment, and the fourth detailed diagnostic model (734) may be pre-learned based on a learning data set labeled that the power equipment is in a normal state. Accordingly, when input data is input to the first detailed diagnostic model (731), a first detailed diagnostic result indicating the probability that an arc has occurred in the power equipment corresponding to the input data to the first detailed diagnostic model (731) may be output, and when input data is input to the second detailed diagnostic model (732), a second detailed diagnostic result indicating the probability that a corona has occurred in the power equipment corresponding to the input data to the second detailed diagnostic model (732) may be output. Additionally, when input data is input into the third detailed diagnostic model (733), a third detailed diagnostic result indicating the probability that tracking has occurred in the power equipment corresponding to the input data in the third detailed diagnostic model (733) may be output, and when input data is input into the fourth detailed diagnostic model (734), a fourth detailed diagnostic result indicating the probability that the power equipment corresponding to the input data is in a normal state may be output.The decision unit (735) determines whether the power equipment corresponding to the input data is in a fault state and / or the type of fault state based on the first detailed diagnosis result output from the first detailed diagnosis model (731), the second detailed diagnosis result output from the second detailed diagnosis model (732), the third detailed diagnosis result output from the third detailed diagnosis model (733), and the fourth detailed diagnosis result output from the fourth detailed diagnosis model (734), and can obtain a diagnosis result including the final diagnosis state (result regarding whether the power equipment is in a fault state, a normal state, or the type of fault state), the probability that the power equipment is in an arc state, the probability that the power equipment is in a corona state, the probability that the power equipment is in a tracking state, the probability that the power equipment is in a fault state, and / or the probability that the power equipment is in a normal state.

[0134] The diagnostic device can provide information about the diagnostic data based on the diagnostic results.

[0135] In addition, in one embodiment, the diagnostic device can obtain image information indicating the external condition of the power equipment, information regarding the temperature of the power equipment (e.g., infrared image information regarding the temperature difference between when the power equipment is in a normal state and when it is in a faulty state), information regarding the humidity of the power equipment, distance information between the ultrasonic acquisition device and the power equipment, and location information of the power equipment (e.g., GPS information of the ultrasonic acquisition device when acquiring an ultrasonic signal for the power equipment) from the aforementioned ultrasonic acquisition device.

[0136] In addition, the diagnostic data may include information acquired from an ultrasound acquisition device and results output by a diagnostic model.

[0137] For example, FIG. 15 is a drawing for illustrating exemplary diagnostic data according to one embodiment.

[0138] Referring to FIG. 15, the diagnostic device can generate diagnostic data as shown in FIG. 15. The diagnostic data may include the diagnosis date, the diagnostician, the diagnostician's headquarters, the diagnostician's business office, the temperature of the diagnostic equipment, the humidity of the diagnostic equipment, the identification number of the diagnostic equipment, the name of the diagnostic equipment, and location information of the diagnostic equipment. Additionally, the diagnostic data may include information regarding the final diagnostic state of the diagnostic equipment according to the diagnostic model and the probability for each state.

[0139] In addition, the diagnostic data may include the accuracy of the diagnostic model, decibel information of the ultrasonic signal, distance information between the ultrasonic acquisition device and the power equipment, images of the appearance of the power equipment, and map information of the power equipment.

[0140] In addition, the diagnostic device receives information regarding the external condition of the power equipment, the diagnostic results, and the diagnostic comments from the diagnostician, and can include the received information in the diagnostic data.

[0142] Various embodiments of this specification may be implemented as software containing instructions stored on a machine-readable storage medium (e.g., a computer). The machine may include electronic devices according to the disclosed embodiments, which are devices capable of calling instructions stored from the storage medium and operating according to the called instructions. When the instructions are executed by a processor, the processor may perform a function corresponding to the instructions directly or using other components under the control of the processor. The instructions may include code generated or executed by a compiler or an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, "non-transitory storage medium" means that it does not contain a signal and is tangible, without distinguishing whether data is stored semi-permanently or temporarily on the storage medium. For example, the "non-transitory storage medium" may include a buffer in which data is temporarily stored.

[0143] According to one embodiment, the method according to the various embodiments disclosed herein may be provided as included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed online in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or through an application store (e.g., Play Store™). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily created in a storage medium such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0145] Although the embodiments have been described above with reference to limited examples and drawings, those skilled in the art can make various modifications and variations from the description above. For example, suitable results can be achieved even if the described techniques are performed in a different order than described, and / or the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

[0146] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below.

Claims

Claim 1 A method for diagnosing power facilities using artificial intelligence comprises: a step of acquiring an ultrasonic signal for power facilities; a step of generating a Mel Spectrogram based on the ultrasonic signal; - the Mel Spectrogram includes time information, frequency information, and magnitude information corresponding to the ultrasonic signal, and is in the form of an image -, and a step of inputting the Mel Spectrogram into a pre-trained diagnostic model; The method includes the step of acquiring diagnostic data for the power equipment based on the diagnostic model, wherein the diagnostic model is constructed based on a depthwise-separable convolutional neural network, the depthwise-separable convolutional neural network includes a depthwise convolutional layer and a pointwise convolutional layer, wherein in the diagnostic model, the Mel spectrogram is separated by channel, the depthwise convolutional layer performs convolution on each single channel to extract intermediate feature vectors, and the pointwise convolutional layer performs convolution on the intermediate feature vectors using a kernel of size 1x1 to output an output value, and the diagnostic model generates a diagnostic result for the power equipment based on the output value, and the diagnostic model includes a first training data set including a plurality of Mel spectrograms labeled as the power equipment being in an arc state, a second training data set including a plurality of Mel spectrograms labeled as the power equipment being in a tracking state, and a plurality of Mel spectrograms labeled as the power equipment being in a corona state A third learning data set including a power facility is pre-trained based on a fourth learning data set including a plurality of Mel spectrograms labeled as the power facility being in a normal state, and the first to fourth learning data sets are labeled based on an annotation tool, and the annotation tool is provided with a labeling item for selecting the state of the power facility.A method for diagnosing power equipment, wherein a waveform of an ultrasonic signal to be labeled and a Mel spectrogram corresponding to the ultrasonic signal to be labeled are simultaneously displayed through the UI of the annotation tool, a type of labeling item for a state of the power equipment including any one of an arc state, a tracking state, a corona state, or a normal state is selected, and when the position and length of the labeling item are set, the ultrasonic signal to be labeled is separated according to the position and length of the labeling item, and the separated ultrasonic signal is distinguished and stored according to the type of the labeling item, and the original data corresponding to the labeling item containing the separated ultrasonic signal, refined data including a waveform corresponding to the separated ultrasonic signal, a power spectrum or a Mel spectrogram, and labeling data in which the labeling item is added to the original data and the refined data are independently stored. Claim 2 A method for diagnosing power equipment according to claim 1, wherein the step of generating a Mel spectrogram based on the ultrasonic signal comprises: a step of performing pre-emphasis on the ultrasonic signal; a step of generating a plurality of frames by performing sampling and windowing on the ultrasonic signal for which pre-emphasis has been performed; a step of generating frequency data by performing a Fourier transform on the plurality of frames and generating a power spectrum based on the frequency data; and a step of generating the Mel spectrogram based on the power spectrum. Claim 3 A power equipment diagnostic method according to paragraph 2, wherein the Fourier transform is performed using a Short Time Fourier Transform (STFT) algorithm. Claim 4 delete Claim 5 delete Claim 6 delete Claim 7 delete Claim 8 A method for diagnosing power equipment according to claim 1, wherein the diagnostic result includes a probability that the power equipment is in an arc state, a probability that the power equipment is in a tracking state, a probability that the power equipment is in a corona state, and a probability that the power equipment is in a normal state, and the step of obtaining diagnostic data for the power equipment based on the diagnostic model determines the state of the power equipment based on the probability that the power equipment is in an arc state, a probability that the power equipment is in a tracking state, a probability that the power equipment is in a corona state, and a probability that the power equipment is in a normal state, and the state of the power equipment is included in the diagnostic result. Claim 9 A method for diagnosing power equipment according to claim 8, wherein the diagnostic data includes the diagnostic result and additional information, and the additional information includes at least one of identification information of the power equipment, information on the location of the power equipment, and information on the temperature of the power equipment. Claim 10 A computer-readable recording medium having a program that executes the method described in any one of paragraphs 1 through 3, paragraph 8, and paragraph 9. Claim 11 In a power equipment diagnostic device using artificial intelligence, a communication unit; It includes a control unit, wherein the control unit acquires an ultrasonic signal for power equipment through the communication unit and generates a Mel spectrogram based on the ultrasonic signal, - the Mel spectrogram includes time information, frequency information, and magnitude information corresponding to the ultrasonic signal and is in the form of an image - inputs the Mel spectrogram into a pre-trained diagnostic model and acquires diagnostic data for power equipment based on the diagnostic model, wherein the diagnostic model is constructed based on a Depthwise-Separable Convolutional Neural Network, the Depthwise-Separable Convolutional Neural Network includes a depthwise convolutional layer and a point-wise convolutional layer, wherein in the diagnostic model, the Mel spectrogram is separated by channel, the depthwise convolutional layer performs convolution on each single channel to extract intermediate feature vectors, and the point-wise convolutional layer performs convolution on the intermediate feature vectors using a kernel of size 1x1 to output an output value, and the diagnostic model [acquires] the power equipment based on the output value A diagnostic result is generated, and the diagnostic model is pre-trained based on a first training data set comprising a plurality of Mel spectrograms labeled as the power equipment being in an arc state, a second training data set comprising a plurality of Mel spectrograms labeled as the power equipment being in a tracking state, a third training data set comprising a plurality of Mel spectrograms labeled as the power equipment being in a corona state, and a fourth training data set comprising a plurality of Mel spectrograms labeled as the power equipment being in a normal state, wherein the first training data set to the fourth training data set are labeled based on an annotation tool, and the annotation tool is provided with a labeling item for selecting the state of the power equipment.A power equipment diagnostic device comprising: a waveform of an ultrasonic signal to be labeled and a Mel spectrogram corresponding to the ultrasonic signal to be labeled are simultaneously displayed through the UI of the annotation tool; a type of labeling item for a state of the power equipment including any one of an arc state, a tracking state, a corona state, or a normal state is selected; when the position and length of the labeling item are set, the ultrasonic signal to be labeled is separated according to the position and length of the labeling item; the separated ultrasonic signal is distinguished and stored according to the type of the labeling item; and original data corresponding to the labeling item containing the separated ultrasonic signal, refined data including a waveform corresponding to the separated ultrasonic signal, a power spectrum or a Mel spectrogram, and labeling data in which the labeling item is added to the original data and the refined data are independently stored.

Images