Discharge detection model, training method and using method thereof, equipment and medium
By introducing a multimodal self-supervised discharge detection model in power vision technology, using audio-visual combined encoder to process multi-source heterogeneous data, the problems of data utilization and labeling difficulty are solved, and the discharge failure of power equipment is achieved Accuracy detection.
Patent Information
- Application Number
- CN202411391217.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-08
- Publication Date
- 2025-02-14
AI Technical Summary
Existing power vision technology is difficult to effectively utilize multi-source heterogeneous data under the Internet of Things, and data labeling is difficult, which affects the training effect and accuracy of the model.
A discharge detection model based on multimodal self-supervision is proposed. Through preprocessing, masking, encoding and classification modules, the audio-visual data is processed and analyzed, and the discharge fault detection of the electrical equipment is achieved by using the combined audio-visual encoder.
The detection of three types of local discharges of electricity-using equipment is realized, with the detection accuracy reaching 55.6%, solving the problem of utilization of multi-source data and reducing labeling difficulty.
Smart Images
Figure CN119445198A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of equipment detection, and specifically, to a discharge detection model and its training method, use method, equipment, and medium. Background Art
[0002] With the rapid development of smart grids, artificial intelligence technology is increasingly integrated with the power Internet of Things. Power sensing terminals generate massive amounts of heterogeneous and multi-parameter power data. How to fully and efficiently utilize this data is of great significance to the safe operation and maintenance of power equipment.
[0003] At present, as one of the core technologies for improving the intelligence level of patrol inspection and safety supervision, power vision technology solves visual problems in all aspects of the power system by utilizing image processing and deep learning, combined with knowledge in the power professional field. However, the existing power vision technology can no longer meet the development needs of the modern power industry. At present, power vision technology faces many challenges and problems. First, the data and types generated by multi-source perception terminals under the power Internet of Things are generally unstructured data such as images, audio, and video. Deep fusion and analysis of these heterogeneous multi-parameter power data to achieve effective perception of power targets is the key to the construction of the power Internet of Things. Secondly, due to the complexity and changeability of equipment and scenarios in the power industry, data labeling is difficult, which directly affects the training effect and accuracy of the model.
[0004] Therefore, the present application provides a discharge detection model and its training method, usage method, device, and medium to solve one of the above-mentioned technical problems. Summary of the invention
[0005] The purpose of this application is to provide a discharge detection model and its training method, use method, device, and medium, which can solve at least one of the technical problems mentioned above. The specific solution is as follows:
[0006] According to a specific implementation of the present application, in a first aspect, the present application provides a discharge detection model, including:
[0007] A preprocessing module is used to preprocess, tokenize, and project the audio-visual data input to the discharge detection model; wherein the audio-visual data is data obtained by audio-visual monitoring of the electrical equipment; a masking module is used to embed and mask the projected audio-visual data; an encoding module is used to perform single-mode encoding and joint multi-mode encoding on the audio-visual data; a classification module is used to detect whether a discharge fault occurs in the electrical equipment based on the encoding result, and to output the type of discharge fault when a discharge fault is detected in the electrical equipment.
[0008] According to a specific implementation of the present application, in a second aspect, the present application provides a method for training a discharge detection model, comprising:
[0009] Obtain visual sample data and audio sample data for use as training samples, wherein the visual sample data and audio sample data are data obtained when a discharge fault occurs in the electrical equipment to be detected; respectively perform tokenization processing on the visual sample data and the audio sample data to obtain tokenized visual data and tokenized audio data; based on a specified linear projection layer, project the tokenized visual data and the tokenized audio data to a target linear space; for the visual data and audio data projected to the target linear space, respectively add modal type embedding and two-dimensional modal sine position embedding, and mask the visual data and audio data to which the embedding is added according to a preset ratio to obtain masked visual data and masked audio data; encode the masked visual data based on a visual encoder to obtain visual coded data, and encode the masked visual data based on an audio encoder to obtain masked audio data. The masked audio data is encoded to obtain audio encoded data; the visual encoded data and the audio encoded data are input into an audio-visual joint encoder with a multi-stream forward channel to obtain a multi-stream output result of the audio-visual joint encoder; wherein the multi-stream includes a visual single-mode data stream, an audio single-mode data stream, and an audio-visual joint multi-mode data stream, and the multi-stream output result includes a visual single-mode output result, an audio single-mode output result, and a joint multi-mode output result; comparative learning is performed based on the visual single-mode output result and the audio single-mode output result, and the visual sample data and the audio sample data are reconstructed based on the joint multi-mode output result; for the comparative learning and the reconstruction, a comparative learning loss and a reconstruction loss are established respectively, and the discharge detection model is monitored based on the comparative learning loss and the reconstruction loss until the discharge detection model converges.
[0010] In one implementation, the masked visual data and the masked audio data are obtained using the following formula: Among them, i is the sample number used to distinguish samples, Represents the masked audio data, Represents the masked visual data, Mask 0.75 Indicates that the preset ratio is 75%, Proj indicates projection processing, a i Represents the labeled audio data, v i represents the labeled visual data, E a Represents the modality type embedding of audio data, E v represents the modality-type embedding of visual data, represents the two-dimensional modal sinusoidal position embedding of the audio data, Representing 2D modal sinusoidal position embeddings for visual data.
[0011] In one implementation, the audio-visual joint encoder uses the following formula to obtain the multi-stream output result: Among them, i is the sample number used to distinguish samples, Represents the visual unimodal output result, Indicates the audio single-mode output result, x i represents the joint multi-mode output result, MeanPool represents the average pooling operation, LN1 represents the first normalization layer, LN2 represents the second normalization layer, and the subscripts of LN1 and LN2 are used to identify the processed data, where a represents the processed data is audio data, v represents the processed data is visual data, and av represents the processed data is joint audio-visual data.
[0012] In one implementation, the monitoring of the discharge detection model based on the contrastive learning loss and the reconstruction loss includes: monitoring the discharge detection model based on a joint loss of the contrastive learning loss and the reconstruction loss, wherein the joint loss is expressed by the following formula: in, represents the joint loss, represents the reconstruction loss, represents the contrastive learning loss, λ c represents the weight of contrastive learning loss.
[0013] In one implementation, the reconstruction loss uses mean square error loss, which is expressed by the following formula: Among them, N is the number of samples, i is the sample number used to distinguish samples, and a mask Represents the raw mask block of audio sample data, v mask Represents the original mask patch of visual sample data, prediction mask blocks of audio sample data, Represents the predicted mask block of visual sample data, |a mask | indicates the amount of audio being muted, and |v mask |Indicates the number of blocked videos.
[0014] In one implementation, the reconstruction loss uses mean square error loss, which is expressed by the following formula: in, s i,k and i,i The calculation method of s i,j Same, N is the number of samples, i is the sample number used to distinguish samples, Represents the visual unimodal output result, represents the audio single-mode output result, τ is the temperature parameter, and k represents a variable that is not equal to i.
[0015] According to a specific implementation of the present application, in a third aspect, the present application provides a method for using a discharge detection model, including:
[0016] The visual measured data and the audio measured data are input into the discharge detection model; wherein the visual measured data and the audio measured data are data obtained by audio-visual monitoring of the electrical equipment under test; based on the output of the discharge detection model, it is determined whether the electrical equipment under test has a discharge fault, and the type of discharge fault is determined when a discharge fault occurs in the electrical equipment.
[0017] According to the specific implementation of the present application, in a fourth aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that the processor executes the computer program to implement the method described in any one of the second aspects, or to implement the method described in the third aspect.
[0018] According to the specific implementation of the present application, in the fifth aspect, the present application provides a computer-readable storage medium having a computer program / instruction stored thereon, characterized in that when the computer program / instruction is executed by a processor, it implements the method described in any one of the second aspect, or implements the method described in the third aspect.
[0019] Compared with the prior art, the above scheme of the embodiment of the present application has at least the following beneficial effects: the discharge detection model provided by the present application can use intelligent means to combine audio and images in multi-source heterogeneous data to fully mine data information, reduce the difficulty of annotation, and realize the detection of three types of partial discharge of electrical equipment, with a detection accuracy of up to 55.6%. This solves the utilization rate of multi-source data and reduces the difficulty of annotation. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 A block diagram of a discharge detection model according to an embodiment of the present application is shown;
[0021] Figure 2 A schematic diagram of the wiring and physical configuration of a partial discharge experimental platform is shown;
[0022] Figure 3 A flow chart of a method for training a discharge detection model according to an embodiment of the present application is shown;
[0023] Figure 4 A schematic diagram of a labeling process of audio sample data and visual sample data is shown;
[0024] Figure 5 is a flow chart of a method of using a discharge detection model;
[0025] Figure 6 The present invention is a block diagram of an electronic device for training or using a discharge detection model according to an exemplary embodiment. DETAILED DESCRIPTION
[0026] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.
[0027] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings, and "multiple" generally includes at least two.
[0028] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.
[0029] It should be understood that although the terms first, second, third, etc. may be used to describe in the embodiments of the present application, these descriptions should not be limited to these terms. These terms are only used to distinguish the descriptions. For example, without departing from the scope of the embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first.
[0030] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.
[0031] It should also be noted that the term "includes", "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, so that a commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprising a ..." do not exclude the existence of other identical elements in the commodity or device including the elements.
[0032] It is particularly important to note that symbols and / or numbers in the specification that are not marked in the accompanying drawings are not drawing marks. The present invention is aimed at the problems encountered in practical applications of power vision technology, such as the inability to effectively utilize multi-source heterogeneous data collected by power terminals and the difficulty in labeling power data sets. To this end, this paper proposes a method for detecting the status of electrical equipment based on multimodal self-supervision, which makes more effective use of the multi-source heterogeneous data collected by power terminals and solves the difficulty in labeling power data sets. With the help of a joint audio-visual masked autoencoder, new solutions and methods are provided for the difficulties currently encountered by power vision technology.
[0033] With the rapid development of smart grids, artificial intelligence technology is increasingly integrated with the power Internet of Things. Power sensing terminals generate massive amounts of heterogeneous and multi-parameter power data. How to fully and efficiently utilize this data is of great significance to the safe operation and maintenance of power equipment.
[0034] At present, as one of the core technologies for improving the intelligence level of patrol inspection and safety supervision, power vision technology solves visual problems in all aspects of the power system by utilizing image processing and deep learning, combined with knowledge in the power professional field. However, the existing power vision technology can no longer meet the development needs of the modern power industry. At present, power vision technology faces many challenges and problems. First, the data and types generated by multi-source perception terminals under the power Internet of Things are generally unstructured data such as images, audio, and video. Deep fusion and analysis of these heterogeneous multi-parameter power data to achieve effective perception of power targets is the key to the construction of the power Internet of Things. Secondly, due to the complexity and changeability of equipment and scenarios in the power industry, data labeling is difficult, which directly affects the training effect and accuracy of the model.
[0035] In view of this, this application is aimed at the problems encountered by power vision technology in practical applications, such as the ineffective use of multi-source heterogeneous data collected by power terminals and the difficulty of annotating power data sets. To this end, this application proposes a method for detecting the state of power equipment based on multimodal self-supervision, which makes more effective use of the multi-source heterogeneous data collected by power terminals and solves the difficulty of annotating power data sets. With the help of joint audio-visual masked autoencoders, new solutions and methods are provided for the difficulties currently encountered by power vision technology.
[0036] This application provides a discharge detection method, which first extracts the video data collected by the power terminal into audio data and image data, and preprocesses and tokenizes them. Secondly, a standard encoder is used as our main model component, and each encoder layer consists of multi-head self-attention (MSA), layer normalization (LN), and multi-layer perceptron (MLP) blocks with residual connections. Then, the modality type embedding E is added. a and E v , and the two-dimensional modal sinusoidal position embedding and After that, we uniformly mask 75% of the tokens of each modality, and then, we input the masks to the independent audio and visual encoders. Finally, we apply the multi-stream forward channel encoder to the audio-visual joint encoder E j , and achieved the detection of three types of partial discharge of electrical equipment with a detection accuracy of 55.6%. Through this method, the audio and image information can be fully extracted from the video data collected by the power terminal for self-supervised learning, thus solving the problem of multi-source heterogeneous data fusion and detection in power.
[0037] The optional embodiments of the present application are described in detail below with reference to the accompanying drawings.
[0038] By way of example, to implement the above solution, the present application exemplarily provides a discharge detection model.
[0039] Figure 1 A block diagram of a discharge detection model according to an embodiment of the present application is shown, Figure 1 As shown, the discharge detection model includes a preprocessing module 101 , a mask module 102 , an encoding module 103 , and a classification module 104 .
[0040] The preprocessing module 101 is used to preprocess, tokenize, and project the audio-visual data input to the discharge detection model, wherein the audio-visual data is data obtained by audio-visual monitoring of the electrical equipment.
[0041] The masking module 102 is used to embed and mask the projected audio-visual data.
[0042] The encoding module 103 is used to perform single-mode encoding and joint multi-mode encoding on the audio-visual data.
[0043] The classification module 104 is used to detect whether a discharge fault occurs in the electrical equipment based on the coding result, and to output the type of the discharge fault when a discharge fault occurs in the electrical equipment.
[0044] Figure 2 A schematic diagram of the wiring and actual configuration of a partial discharge experimental platform is shown.
[0045] In this application, if Figure 2 As shown, the console model is TC-10kVA, the capacity is 10kVA, the input voltage is 220V, the output adjustable voltage range is 0-250V, and the isolation filter transformer with a rated capacity of 25kVA is connected to avoid the influence of harmonics on the experiment. The model of the partial discharge test transformer is YDQ-10 / 100, and the capacity is 10kVA. In order to limit the short-circuit current when the test sample is broken down and prevent damage to other equipment, a 10kΩ protection resistor is connected in series. The voltage divider is used to increase the voltage of the input discharge platform to the kilovolt voltage required for the experiment, and the voltage is adjusted by the console. The discharge detection model is implemented by a computer, and the audiovisual data used to input the discharge detection model is collected by a microphone array. The microphone array collects signals at a certain distance from the discharge platform for subsequent analysis. Specifically, the audio and sound image of the partial discharge signal are collected simultaneously by optical and acoustic array sensors. Among them, the audio sampling frequency is 192000Hz and the image frame rate is 25 frames.
[0046] Furthermore, the present application provides a training method for a discharge detection model to realize discharge detection based on a discharge detection model. The method is applied to the discharge detection model provided in the above embodiment and is used to train the discharge detection model so that the discharge detection model can realize discharge fault detection and output discharge fault type.
[0047] The embodiment provided in this application is an embodiment of a method for training a discharge detection model.
[0048] Combine the following Figure 3 The embodiments of the present application are described in detail.
[0049] Figure 3 A flow chart of a method for training a discharge detection model according to an embodiment of the present application is shown. Figure 3 As shown, the process includes the following steps S301 to S308.
[0050] Step S301: Obtain visual sample data and audio sample data used as training samples, where the visual sample data and audio sample data are data obtained when a discharge fault occurs in the electrical equipment to be detected.
[0051] Step S302 , tokenizing the visual sample data and the audio sample data respectively to obtain tokenized visual data and tokenized audio data.
[0052] Step S303: projecting the labeled visual data and the labeled audio data into a target linear space based on a specified linear projection layer.
[0053] Step S304, for the visual data and audio data projected into the target linear space, modal type embedding and two-dimensional modal sinusoidal position embedding are added respectively, and the embedded visual data and audio data are masked according to a preset ratio to obtain masked visual data and masked audio data.
[0054] Step S305: encoding the masked visual data based on the visual encoder to obtain visual coded data, and encoding the masked audio data based on the audio encoder to obtain audio coded data.
[0055] Step S306: input the visual coding data and the audio coding data into an audio-visual joint encoder with a multi-stream forward channel to obtain a multi-stream output result of the audio-visual joint encoder.
[0056] Among them, the multiple streams include visual single-mode data stream, audio single-mode data stream, and audio-visual joint multi-mode data stream, and the multi-stream output results include visual single-mode output results, audio single-mode output results, and joint multi-mode output results.
[0057] Step S307 , performing comparative learning based on the visual single-mode output result and the audio single-mode output result, and reconstructing the visual sample data and the audio sample data based on the joint multi-mode output result.
[0058] Step S308 : establishing a contrastive learning loss and a reconstruction loss for contrastive learning and reconstruction respectively, and monitoring the discharge detection model based on the contrastive learning loss and the reconstruction loss until the discharge detection model converges.
[0059] The training method of the discharge detection model provided in the present application obtains the visual and audio sample data of the electrical equipment when the discharge fault occurs, and performs tokenization and linear projection processing on the data, then adds the modal type and position embedding, and performs a partial masking operation. Then, a special encoder is used to encode the masked data to generate visual and audio encoding data, which are further input into the audio-visual joint encoder to produce a multi-stream output result. Through comparative learning and reconstruction tasks, this method not only improves the generalization ability of the model, but also reduces the need for a large amount of labeled data, thereby effectively improving the accuracy of electrical equipment discharge fault detection. This method is particularly suitable for status monitoring of power equipment, and can significantly improve the recognition accuracy of the discharge type of power equipment without increasing additional hardware costs.
[0060] In this application, in order to simulate the partial discharge signal generated by the operation of power equipment, this patent collects audio data of three typical partial discharge defects, namely corona discharge, surface discharge and spark discharge. When the model training is completed, it can be used to identify and detect the types of three typical partial discharges.
[0061] Figure 4 A schematic diagram of a labeling process for audio sample data and visual sample data is shown.
[0062] For example, Figure 4 As shown, for a small batch of N audiovisual samples, N is a natural number greater than or equal to 1, we first preprocess and tokenize the audio sample data and visual sample data, and obtain a series of audio and labeled visual data {a i ,v i} and project them into the same linear space using two modality-specific linear projection layers. For example, a 10-second video (with parallel audio) can be used to pre-train and fine-tune the model. For audio, each 10-second audio waveform is first converted to a 128-dimensional log-Melody filter bank (fbank) feature sequence, calculated every 10ms using a 25ms Hanning Window. A 1024 (time) × 128 (frequency) spectrogram is generated. We then split the spectrogram into 512 16×16 square patches a=[a 1 , …, a 512 ] as the input of the model. In order to reduce the computational overhead, a frame aggregation strategy is used. Specifically, 10 RGB frames are uniformly sampled from each 10-second video (i.e., 1FPS). During training, a random RGB frame is selected as input; during inference, the average prediction of the model for each RGB frame is taken as the prediction of the entire video. Compared with connecting multiple RGB frames as the input of the encoder with quadratic complexity, frame aggregation is more efficient in linear time complexity, but at the cost of not considering inter-frame correlation. For each RGB frame, resize and crop it to 224×224, and then divide it into 196 16×16 square patches v=[v 1 ,…,v 196 ].
[0063] In this application, the formula and Obtain the masked visual data and masked audio data. Where i is the sample number used to distinguish samples, Represents the masked audio data, Represents the masked visual data, Mask 0.75 Indicates that the preset ratio is 75%, Proj indicates projection processing, a i Represents the labeled audio data, v i represents the labeled visual data, E a Represents the modality type embedding of audio data, E v represents the modality-type embedding of visual data, represents the two-dimensional modal sinusoidal position embedding of the audio data, Representing 2D modal sinusoidal position embeddings for visual data.
[0064] Furthermore, the and Each is input to a separate encoder for encoding. For example, Input to audio-specific encoder E a (?), Input to the vision-specific encoder E V (?), so as to obtain the visual coding data v′ i `And get the audio coded data a' i `. Each encoder layer consists of a multi-head self-attention module (MSA), a layer normalization (LN), and a multi-layer perceptron (MLP) block. For example, the encoder layer can be represented as y = Transformer (x; MSA, LN1, LN2, MLP).
[0065] For example, the solution process is as follows
[0066] x′=MSA(LN1(x))+x; y=MLP(LN2(x′))+x′;
[0067] Among them, MSA is a multi-head self-attention module, which means that the function of this module is to calculate the dot product attention. MSA performs dot product attention calculation on each element of x, so its complexity is quadratic with the size of x. For audio data and visual data, x entering the Transformer layer is replaced by and According to the formula output, x' is replaced by a' i ` and v′ i `.
[0068] In this application, the audio signal a′ i ` and video signal v′ i `Further processing is performed by the joint audio-visual encoder, for example concatenating the audio-visual signal [a′ i `,v′ i `] to E j , where the three forward passes correspond to the audio data stream, the visual data stream, and the audio-visual joint data stream. The audio-visual joint encoder uses different normalization layers LN1 for different streams. {a,v,av} and LN2 {a,v,av} , and E j All other weights of are shared among all three streams, i.e., the same MSA and MLP weights are used.
[0069] For example, the multi-stream output result obtained by the audio-visual joint encoder can be expressed as follows.
[0070]
[0071] Among them, i is the sample number used to distinguish samples, Represents the visual unimodal output result, Indicates the audio single-mode output result, x i represents the joint multi-mode output result, MeanPool represents the average pooling operation, LN1 represents the first normalization layer, LN2 represents the second normalization layer, and the subscripts of LN1 and LN2 are used to identify the processed data, where a represents the processed data is audio data, v represents the processed data is visual data, and av represents the processed data is joint audio-visual data.
[0072] In the present application, the discharge detection model is monitored based on contrastive learning loss and reconstruction loss, including monitoring the discharge detection model based on a combined loss of contrastive learning loss and reconstruction loss.
[0073] For example, the joint loss is given by the formula Indicates. Among them, represents the joint loss, represents the reconstruction loss, represents the contrastive learning loss, λ c represents the weight of contrastive learning loss.
[0074] In this application, for the reconstruction task, in x i The original position x′ i (Example: a′ i ` and v′ i ` indicates) fill the trainable mask symbol. Then x′ i Input to the joint audio-visual decoder D j (?) Before, the modal type embedding E′ was also added a and E′ v and a specific 2D modal sinusoidal position embedding and To reconstruct the input audio and image. j (?) Processes audio and video tokens with the same weights, except for the last modality-specific projection layer, which outputs and in, and By formula get.
[0075] On this basis, as a feasible implementation example, the reconstruction loss adopts the mean square error loss, using the formula Indicated by. Where N is the number of samples, i is the sample number used to distinguish samples, and a mask Represents the raw mask block of audio sample data, v mask Represents the original mask patch of visual sample data, prediction mask blocks of audio sample data, Represents the predicted mask block of visual sample data, |a mask | indicates the amount of audio being muted, and |v mask |Indicates the number of blocked videos.
[0076] As a feasible embodiment, the reconstruction loss adopts the mean square error loss, using the formula Indicates. Among them, s i,k and i,i The calculation method of s i,j Same, N is the number of samples, i is the sample number used to distinguish samples, Represents the visual unimodal output result, represents the audio single-mode output result, τ is the temperature parameter, and k represents a variable that is not equal to i.
[0077] In this application, the visual and audio sample data of the electrical equipment discharge fault are obtained, and the data are tokenized and linearly projected, and then the modal type and position embedding are added, and a partial masking operation is performed. Then a special encoder is used to encode the masked data to generate visual and audio encoding data, which are further input into the audio-visual joint encoder to produce multi-stream output results. Through comparative learning and reconstruction tasks, this method not only improves the generalization ability of the model, but also reduces the need for a large amount of labeled data, thereby effectively improving the accuracy of electrical equipment discharge fault detection. This method is particularly suitable for state monitoring of power equipment, and can significantly improve the recognition accuracy of power equipment discharge types without increasing additional hardware costs. On this basis, the collected corona discharge, surface discharge and spark discharge data are compared with the experimental results, with an overall loss of 1.438273 and a comparison accuracy of 55.6%.
[0078] Based on the same technical concept, the present application also provides a method for using a discharge detection model. The method implements discharge detection based on the discharge detection model provided by any of the above embodiments. If there are any unclear points in the following content, please refer to any of the above embodiments.
[0079] Figure 5 is a flow chart of the method of using the discharge detection model, such as Figure 5 As shown, the method is applied to the discharge detection model involved in the above embodiment, including the following steps S501 and S502.
[0080] Step S501: inputting visual measured data and audio measured data into a discharge detection model.
[0081] Among them, the visual measured data and the audio measured data are data obtained by performing audio-visual monitoring on the electrical equipment under test.
[0082] Step S502: based on the output of the discharge detection model, determine whether a discharge fault occurs in the tested electrical equipment, and determine the type of the discharge fault when a discharge fault occurs in the electrical equipment.
[0083] By inputting the visual and audio data collected on-site into the pre-trained model, it is possible to automatically analyze and determine whether there is a discharge fault in the electrical equipment and its specific type, thereby achieving real-time and efficient monitoring of the operating status of the power equipment, helping to promptly discover potential safety hazards, reduce equipment damage and power outages caused by undetected discharge problems, and improve the reliability and maintenance efficiency of the power system. Specifically, for the three types of partial discharges, corona discharge, surface discharge, and spark discharge, the detection accuracy can reach 55.6%. Through this method, the audio and image information collected by the power terminal can be fully extracted for self-supervised learning, thereby solving the problem of multi-source heterogeneous data fusion and detection in the power industry.
[0084] Figure 6 is a block diagram of an electronic device 600 for training or using a discharge detection model according to an exemplary embodiment.
[0085] like Figure 6 As shown, one embodiment of the present application provides an electronic device 600. The electronic device 600 includes a memory 601, a processor 602, and an input / output (I / O) interface 603. The memory 601 is used to store instructions. The processor 602 is used to call the instructions stored in the memory 601 to execute the discharge detection model training method or use method of the embodiment of the present application. The processor 602 is connected to the memory 601 and the I / O interface 603, respectively, for example, through a bus system and / or other forms of connection mechanisms (not shown). The memory 601 can be used to store programs and data, including the programs of the discharge detection model training method or use method involved in the embodiment of the present application, and the processor 602 executes various functional applications and data processing of the electronic device 600 by running the programs stored in the memory 601.
[0086] In the embodiment of the present application, the processor 602 can be implemented in at least one hardware form of a digital signal processor (Digital Signal Processing, DSP), a field programmable gate array (Field Programmable Gate Array, FPGA), and a programmable logic array (Programmable Logic Array, PLA). The processor 602 can be a central processing unit (Central Processing Unit, CPU) or one or a combination of other forms of processing units with data processing capabilities and / or instruction execution capabilities.
[0087] The memory 601 in the embodiment of the present application may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD), etc.
[0088] In the embodiment of the present application, the I / O interface 603 may be used to receive input instructions (such as digital or character information, and key signal input related to user settings and function control of the electronic device 600), and may also output various information to the outside (such as images or sounds). In the embodiment of the present application, the I / O interface 603 may include one or more of a physical keyboard, a function key (such as a volume control key, a switch key, etc.), a mouse, a joystick, a trackball, a microphone, a speaker, and a touch panel.
[0089] In some embodiments, the present application provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by a processor, any of the methods described above is performed.
[0090] In some embodiments, the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, any of the methods described above is performed.
[0091] Although operations are described in a particular order in the drawings, this should not be understood as requiring that the operations be performed in the particular order shown or in serial order, or that all shown operations be performed to achieve desired results. In certain circumstances, multitasking and parallel processing may be advantageous.
[0092] The methods and devices of the present application can be implemented using standard programming techniques, using rule-based logic or other logic to implement various method steps. It should also be noted that the words "device" and "module" used herein and in the claims are intended to include implementations using one or more lines of software code and / or hardware implementations and / or devices for receiving input.
[0093] Any steps, operations or procedures described herein may be performed or implemented using one or more hardware or software modules, either alone or in combination with other devices. In one embodiment, the software module is implemented using a computer program product including a computer-readable medium containing computer program code, which can be executed by a computer processor to perform any or all of the described steps, operations or procedures.
[0094] The foregoing description of the implementation of the present application has been given for the purpose of illustration and description. The foregoing description is not exhaustive nor is it intended to limit the present application to the exact form disclosed, and various variations and modifications may exist according to the above teachings or may be obtained from the practice of the present application. These embodiments are selected and described in order to illustrate the principles of the present application and its practical application, so that those skilled in the art can utilize the present application in various embodiments and various modifications suitable for the specific purpose contemplated.
[0095] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0096] It can be further understood that, unless otherwise specified, “connection” includes a direct connection without other components between the two, and also includes an indirect connection with other components between the two.
[0097] It is further understood that, although the operations are described in a specific order in the drawings in the embodiments of the present application, it should not be understood as requiring the operations to be performed in the specific order or serial order shown, or requiring the execution of all the operations shown to obtain the desired results. In certain environments, multitasking and parallel processing may be advantageous.
[0098] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to encompass any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the field of the present application that are not disclosed in the present application. The specification and embodiments are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following scope of rights.
[0099] It should be understood that the present application is not limited to the precise structures described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the scope of the appended claims.
[0100] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A discharge detection model, characterized in that: include: A preprocessing module, used for preprocessing, tokenizing, and projecting the audio-visual data input to the discharge detection model; wherein the audio-visual data is data obtained by audio-visual monitoring of the electrical equipment; A mask module, used for embedding and masking the projected audio-visual data; A coding module, used for single-mode coding and joint multi-mode coding of audio-visual data; The classification module is used to detect whether a discharge fault occurs in the electrical equipment based on the coding result, and to output the type of the discharge fault when a discharge fault is detected in the electrical equipment.
2. A method for training a discharge detection model, characterized in that: The discharge detection model applied to claim 1 comprises: Acquire visual sample data and audio sample data used as training samples, wherein the visual sample data and audio sample data are data acquired when a discharge fault occurs in the electrical equipment to be detected; Performing tokenization processing on the visual sample data and the audio sample data respectively to obtain tokenized visual data and tokenized audio data; Based on the specified linear projection layer, the labeled visual data and the labeled audio data are projected into the target linear space; For the visual data and audio data projected to the target linear space, modal type embedding and two-dimensional modal sine position embedding are added respectively, and the visual data and audio data to which the embedding is added are masked according to a preset ratio to obtain masked visual data and masked audio data; encoding the masked visual data based on a visual encoder to obtain visual coded data, and encoding the masked audio data based on an audio encoder to obtain audio coded data; Inputting the visual coded data and the audio coded data into an audio-visual joint encoder having a multi-stream forward channel, and obtaining a multi-stream output result of the audio-visual joint encoder; wherein the multi-stream includes a visual single-mode data stream, an audio single-mode data stream, and an audio-visual joint multi-mode data stream, and the multi-stream output result includes a visual single-mode output result, an audio single-mode output result, and a joint multi-mode output result; Performing comparative learning based on the visual single-mode output result and the audio single-mode output result, and reconstructing the visual sample data and the audio sample data based on the joint multi-mode output result; For the contrastive learning and the reconstruction, a contrastive learning loss and a reconstruction loss are established respectively, and the discharge detection model is monitored based on the contrastive learning loss and the reconstruction loss until the discharge detection model converges.
3. The method according to claim 2, characterized in that The masked visual data and masked audio data are obtained using the following formula: Among them, i is the sample number used to distinguish samples, Represents the masked audio data, Represents the masked visual data, Mask 0.75 Indicates that the preset ratio is 75%, Proj indicates projection processing, a i Represents the labeled audio data, v i represents the labeled visual data, E a Represents the modality type embedding of audio data, E v represents the modality-type embedding of visual data, represents the two-dimensional modal sinusoidal position embedding of the audio data, Representing 2D modal sinusoidal position embeddings for visual data.
4. The method according to claim 2, characterized in that: The audio-visual joint encoder uses the following formula to obtain the multi-stream output result: Among them, i is the sample number used to distinguish samples, Represents the visual unimodal output result, Indicates the audio single-mode output result, x i represents the joint multi-mode output result, MeanPool represents the average pooling operation, LN1 represents the first normalization layer, LN2 represents the second normalization layer, and the subscripts of LN1 and LN2 are used to identify the processed data, where a represents the processed data is audio data, v represents the processed data is visual data, and av represents the processed data is joint audio-visual data.
5. The method according to claim 2, characterized in that: The monitoring of the discharge detection model based on the contrastive learning loss and the reconstruction loss includes: The discharge detection model is monitored based on the joint loss of the contrastive learning loss and the reconstruction loss, and the joint loss is expressed by the following formula: in, represents the joint loss, represents the reconstruction loss, represents the contrastive learning loss, λ c represents the weight of contrastive learning loss.
6. The method according to claim 5, characterized in that The reconstruction loss adopts mean square error loss, which is expressed by the following formula: Among them, N is the number of samples, i is the sample number used to distinguish samples, and a mask Represents the raw mask block of audio sample data, v mask Represents the original mask block of visual sample data, prediction mask blocks of audio sample data, Represents the predicted mask block of visual sample data, |a mask | indicates the amount of audio being muted, and |v mask |Indicates the number of blocked videos.
7. The method according to claim 5, characterized in that The reconstruction loss adopts mean square error loss, which is expressed by the following formula: in, s i,k and i,i The calculation method of s i,j Same, N is the number of samples, i is the sample number used to distinguish samples, Represents the visual unimodal output result, represents the audio single-mode output result, τ is the temperature parameter, and k represents a variable that is not equal to i.
8. A method for using a discharge detection model, characterized in that: The discharge detection model applied to claim 1 comprises: Inputting the visual measured data and the audio measured data into the discharge detection model; wherein the visual measured data and the audio measured data are data obtained by performing audio-visual monitoring of the electrical equipment under test; Based on the output of the discharge detection model, it is determined whether a discharge fault occurs in the tested electrical equipment, and the type of the discharge fault is determined when a discharge fault occurs in the electrical equipment.
9. An electronic device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the method according to any one of claims 2 to 7, or to implement the method according to claim 8.
10. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instruction is executed by a processor, the method described in any one of claims 2 to 7 is implemented, or the method described in claim 8 is implemented.