Power distribution network fault identification method and device and electronic equipment

Abstract

The embodiment of the invention provides a power distribution network fault identification method and device and electronic equipment, and relates to the technical field of fault detection. The method comprises the following steps: determining modal features, including ID table features and text features, of real-time operation data of the power distribution system for being detected; a power distribution system fault basic prediction model used for processing ID table features and based on a multi-attention mechanism and a power distribution system fault detection large language model used for processing text features are constructed respectively; according to the ID table features and the text features, constructing bimodal features to realize fine-grained feature alignment of the power distribution system fault basic prediction model and the power distribution system fault detection large-language model, and aligning feature representations of different modals in an instance-level coarse-grained hierarchy; and the aligned power distribution system fault detection large language model and the power distribution system fault prediction basic model are applied to downstream tasks, and are used for power distribution fault detection after fine tuning. According to the embodiment of the invention, relatively high fault detection performance is realized.

Description

Technical Field

[0001] The present application relates to the technical field of power distribution system fault detection, and specifically to a distribution network fault identification method, a distribution network fault identification device, an electronic device and a corresponding storage medium. Background Art

[0002] With the rapid development of information technology, various digital distribution fault detection technologies are emerging, bringing new opportunities to improve the reliability and safety of distribution systems. Among them, intelligent fault diagnosis technology based on big data analysis and artificial intelligence is particularly noteworthy. Traditional fault diagnosis methods usually rely on expert experience and regular inspections, making it difficult to detect hidden fault problems in a timely manner. The emergence of big data and artificial intelligence has injected new impetus into fault diagnosis. These emerging technologies can effectively solve the limitations of traditional methods: massive equipment operation data can provide a rich information basis for fault diagnosis, and in-depth analysis of these data using machine learning and other methods can dig out hidden fault modes and abnormal laws. Artificial intelligence algorithms such as deep learning have excellent pattern recognition and automatic diagnosis capabilities, which can achieve comprehensive monitoring of equipment status and intelligent fault diagnosis. These technologies have strong adaptability and self-learning capabilities, and can continuously optimize and improve fault diagnosis models as equipment usage changes. With the continuous development and maturity of emerging technologies such as big data and artificial intelligence, they will inevitably play an increasingly important role in the intelligent management of distribution equipment. In the future, intelligent fault diagnosis systems based on these cutting-edge technologies will be widely used in the power industry, making great contributions to improving the safety and reliability of distribution systems.

[0003] On the other hand, with the development of deep learning technology, large language models (LLMs) have been a major breakthrough in the field of machine learning in recent years. Such models have learned a large amount of semantics and knowledge by training massive amounts of text data, and have achieved remarkable results in many natural language processing tasks. For example, with the rise of BERT and GPT, a new paradigm for processing text data has begun to attract the attention of researchers. These models can understand and extract deep semantic relationships in text data through pre-training on large-scale corpora. Applying these powerful large language models to fault detection of power distribution equipment can give full play to their advantages in text understanding and knowledge expression. Specifically, text data such as operation logs and maintenance records of power distribution equipment can be input into large language models for analysis to identify abnormal patterns and fault signs. Large language models are good at extracting hidden patterns and regularities from large amounts of historical data, and can quickly and accurately detect abnormal conditions in equipment operation. Compared with traditional rule-based or statistical fault diagnosis, large language models can more intelligently discover complex fault patterns and greatly improve the accuracy and efficiency of fault detection. In addition, large language models can also be used in combination with other machine learning technologies such as deep learning to further enhance the ability of fault diagnosis. For example, the features extracted by the large language model can be used to train a high-performance fault classification model. In general, as a powerful AI technology, the large language model is being deeply integrated into the intelligent operation and maintenance of power distribution equipment, making an important contribution to improving the reliability and safety of the power distribution system.

[0004] Although LLM is increasingly used in various tasks and fields, the research on bimodal semantic extraction in distribution fault detection is relatively backward. Bimodal semantic extraction involves extracting and fusing information from two modal data (text and ID table, etc.), which places higher requirements on the overall understanding ability and flexibility of the model. Current research mainly focuses on single modality or limited modal fusion, while the exploration of deep semantic association and knowledge transfer between bimodal data is still insufficient. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a distribution network fault identification method, device and electronic device. The present application aims to perform fine-grained feature-level alignment between the attention mechanism model based on ID table features and the large language model based on text features to improve the accuracy of downstream distribution fault detection tasks. Through joint pre-training, the mask features of one modality are allowed to use the information of another modality when reconstructing information, thereby aligning features between different modalities. In addition, the two models are fine-tuned together on the downstream distribution fault detection task to merge their advantages in processing different modal features to at least solve some of the problems in the background technology.

[0006] In order to achieve the above object, a distribution network fault identification method is provided in the present application, the method comprising:

[0007] Determine the modal features of the real-time operation data of the distribution system for detection, the modal features include ID table features and text features; construct a basic distribution system fault prediction model based on a multi-head attention mechanism for processing the ID table features and a distribution system fault detection large language model for processing the text features; construct bimodal features according to the ID table features and the text features to achieve fine-grained feature alignment of the basic distribution system fault prediction model and the distribution system fault detection large language model, and align feature representations of different modalities at the instance-level coarse-grained level; apply the aligned distribution system fault detection large language model and distribution system fault prediction basic model to downstream tasks, and use them for distribution fault detection after fine-tuning.

[0008] Optionally, the ID table feature is constructed by the following steps: the preset dimension is R N×F A two-dimensional ID table is provided, wherein N represents the number of timestamps, and F represents the total number of distribution fault features; each row of the two-dimensional ID table represents instance operation data n under a timestamp, and each column represents a distribution fault feature f; according to the data features of the real-time operation data of the distribution system, the row and column values ​​of the two-dimensional ID table are encoded to generate the ID table features corresponding to the real-time operation data of the distribution system.

[0009] Optionally, the text feature is constructed by the following steps: according to the data features of the real-time operation data of the distribution system, a hard prompt template is used to generate corresponding text features after a text splicing operation based on the text name of the distribution fault feature and the text feature value corresponding to the distribution fault feature in each instance input.

[0010] Optionally, a basic prediction model for distribution system faults based on a multi-head attention mechanism for processing the ID table features is constructed, including: determining the model category and model structure of the basic prediction model for distribution system faults before training, the model category being a multi-classification model, the model structure comprising an embedding layer, a feature interaction layer and a prediction layer; the embedding layer is configured to: convert the high-dimensional sparse ID table features into a low-dimensional dense embedding matrix; the distribution fault feature f is converted from its corresponding representation matrix into an embedding vector by a table lookup method; the feature interaction layer is configured to: take the embedding matrix as input, and generate an embedding matrix for each distribution The electric terminal instance operation data n outputs a dense representation vector; a high- and low-order cross model with a double-tower structure is used to design complex interactive operations for each feature to obtain low-order or high-order feature cross terms; the prediction layer is configured as follows: after splicing the representation vectors of the high- and low-order cross outputs, the vectors are input into a neural network or a linear regression module, and then the Sigmoid function is used to adjust the range to obtain the final classification probability of the fault type; the training samples are used to train the pre-trained distribution system fault prediction basic model in an end-to-end manner using a binary cross entropy loss to obtain the distribution system fault basic prediction model.

[0011] Optionally, a large language model for distribution system fault detection for processing the text features is constructed, including: converting text features into tokens based on a large language model pre-trained for distribution fault detection; inputting the obtained tokens into a Transformer-based encoder network to obtain a context-consistent representation after processing; and classifying distribution faults by adding randomly initialized attention heads to the context-consistent representation to obtain a fault classification label corresponding to the text feature.

[0012] Optionally, bimodal features are constructed based on ID table features and text features to achieve fine-grained feature alignment of the basic prediction model for power distribution system faults and the large language model for power distribution system fault detection, and feature representations of different modalities are aligned at the instance-level coarse-grained level, including: extracting a certain proportion of feature domains from the ID table features and the text features for masking operations to obtain interference forms of the ID table features and the text features, namely, masked ID table features and masked text features; inputting the ID table features and the masked ID table features into the basic prediction model for power distribution system faults for encoding, and encoding the text features and the masked text features. The input distribution system fault detection large language model is encoded to obtain the corresponding output; the output after ID table feature encoding and the output after masked text feature encoding are jointly reconstructed, and the output after text feature encoding and the output after masked ID table feature encoding are jointly reconstructed; the coarse-grained hierarchical feature alignment between the two modalities is established according to the instance-level comparative learning method; combined with the optimization objectives in the joint reconstruction and coarse-grained hierarchical feature alignment process, training is performed based on the overall loss function of the bimodal alignment pre-training stage of the distribution fault prediction basic model and the distribution system fault detection large language model.

[0013] Optionally, a certain proportion of feature domains are extracted from the ID table features and the text features for masking operations to obtain interference forms of the ID table features and interference forms of the text features, including: extracting a certain proportion of feature domains from the ID table features, replacing corresponding features with an additional mask symbol, and the mask symbol is shared by all feature domains; replacing features with specific symbols based on the feature vector of the ID table in the encoding method to obtain masked ID table features, wherein the masked ID table features include a dimension v f A zero vector of ; and a certain proportion of feature domains are extracted from text features, features are replaced with specific characters, and masked text features are obtained using a hard prompt template.

[0014] Optionally, the ID table features and the masked ID table features are input into the basic prediction model of distribution system faults for encoding, and the text features and the masked text features are input into the large language model of distribution system fault detection for encoding to obtain corresponding outputs, including: inputting the ID table features and the masked ID table features into the basic prediction model of distribution system faults, and obtaining the corresponding output after encoding through the embedding layer and the feature interaction layer in the basic prediction model of distribution system faults; and inputting the text features and the masked text features into the large language model of distribution system fault detection, and obtaining the corresponding output through the hidden layer state set of the last layer of the large language model of distribution system fault detection.

[0015] Optionally, joint reconstruction is performed based on the output after text feature encoding and the output after mask ID table feature encoding, including: inputting the output after text feature encoding and the output after mask ID table feature encoding into a self-attention mechanism module in the form of a coded vector tuple to obtain an aggregated output; for each mask ID table feature, after an independent multi-layer perceptron network and a Softmax function, obtaining a distribution function on the candidate features of the aggregated output; for all mask features, using the distribution function and the cross entropy loss function of the ID table feature to calculate the reconstruction features of the ID table feature; using a noise contrast estimation function to calculate the reconstruction loss based on the reconstructed features; joint reconstruction is performed based on the output after ID table feature encoding and the output after mask text feature encoding, including: splicing the output after ID table feature encoding and the output after mask text feature encoding in the form of a coded vector tuple to obtain a spliced ​​vector; after inputting the spliced ​​vector into the prediction layer in the basic prediction model of power distribution system faults, obtaining a reconstructed feature distribution function; using the feature distribution function and the cross entropy loss function of the text feature as the reconstruction loss.

[0016] Optionally, a coarse-grained hierarchical feature alignment between the two modalities is established according to an instance-level comparative learning method, including: using two independent linear layers and a normalization layer to project the tokens vector output and the ID table feature output corresponding to the text input into a first multidimensional vector and a second multidimensional vector; using the InfoNCE function to calculate the comparative loss of the first multidimensional vector and the second multidimensional vector at the instance level; obtaining the final loss according to the comparative loss; and obtaining the overall loss of the basic prediction model of the distribution system fault and the large language model for the distribution system fault detection based on the reconstruction loss and the final loss.

[0017] Optionally, the aligned large language model for power distribution system fault detection and the basic model for power distribution system fault prediction are applied to downstream tasks and used for power distribution fault detection after fine-tuning, including: in the downstream power distribution system fault detection task, the basic model for power distribution system fault detection and the large language model for power distribution system fault are jointly fine-tuned using supervised click signals; randomly restarted linear output layers are respectively set in the aligned large language model for power distribution system fault detection and the basic model for power distribution system fault prediction, and estimated fault classification probabilities are respectively output; the final fault classification probability is obtained after weighted calculation based on the fault classification probabilities of the two models; the estimated results corresponding to the final fault classification probabilities are respectively compared with the true labels, and the gap is measured using the cross entropy loss function to fine-tune and optimize the model.

[0018] In some embodiments of the present application, a distribution network fault identification device is also provided, which includes: a feature confirmation module, which is used to determine the modal features of the real-time operation data of the distribution system for detection, and the modal features include ID table features and text features; a model construction module, which is used to respectively construct a basic distribution system fault prediction model based on a multi-head attention mechanism for processing the ID table features and a distribution system fault detection large language model for processing the text features; a feature alignment module, which is used to construct bimodal features according to ID table features and text features to achieve fine-grained feature alignment of the basic distribution system fault prediction model and the distribution system fault detection large language model, and align feature representations of different modalities at the instance-level coarse-grained level; and an application deployment module, which is used to apply the aligned distribution system fault detection large language model and distribution system fault prediction basic model to downstream tasks, and use them for distribution fault detection after fine-tuning.

[0019] The present application also provides an electronic device, comprising: at least one processor; a memory connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the at least one processor implements the aforementioned distribution network fault identification method by executing the instructions stored in the memory.

[0020] The present application also provides a machine-readable storage medium, on which instructions are stored. When the instructions are executed by a processor, the processor is configured to execute the aforementioned distribution network fault identification method.

[0021] The present application also provides a computer program product, including a computer program, which implements the aforementioned distribution network fault identification method when executed by a processor.

[0022] The above technical solution has the following beneficial effects:

[0023] (1) The appropriate model can be selected according to different scenarios, which helps to enhance the versatility and scalability of the framework.

[0024] (2) The parameterized model can be plug-and-play. Through the joint reconstruction of the feature level between the two modalities, the model pre-training task of fine-grained feature-level interaction and alignment is realized. The parameterized model can also be well inserted into the downstream power distribution fault detection task for fine-tuning.

[0025] (3) Reduce the number of parameters and training time of LLM and multi-head attention mechanism models. Because LLM and multi-head attention mechanism models provide more bimodal information during joint pre-training, which is conducive to rapid convergence of training, in downstream power distribution fault detection tasks, the model fine-tuning process will greatly reduce the training time compared to the original supervised learning.

[0026] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present application, but do not constitute a limitation on the embodiments of the present application. In the accompanying drawings:

[0028] Figure 1 The following is a schematic diagram showing the steps of a method for identifying a distribution network fault according to an embodiment of the present application;

[0029] Figure 2 A technical block diagram of a distribution network fault identification method according to an embodiment of the present application is schematically shown;

[0030] Figure 3 Schematically shows a block diagram of fine-tuning the downstream power distribution fault detection task according to an embodiment of the present application;

[0031] Figure 4 The structure diagram of the power distribution network fault identification device according to the embodiment of the present application is schematically shown;

[0032] Figure 5 The internal structure of an electronic device according to an embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0033] The specific implementation of the embodiment of the present application is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the embodiment of the present application, and is not used to limit the embodiment of the present application.

[0034] Figure 1 The following is a schematic diagram showing the steps of the method for identifying a distribution network fault according to an embodiment of the present application. Figure 1 As shown, a distribution network fault identification method, the method comprising:

[0035] S01, determining the modal features of the real-time operation data of the power distribution system for detection, wherein the modal features include ID table features and text features;

[0036] S02, respectively constructing a basic distribution system fault prediction model based on a multi-head attention mechanism for processing the ID table features and a distribution system fault detection large language model for processing the text features;

[0037] S03, constructing bimodal features according to ID table features and text features to achieve fine-grained feature alignment of the basic prediction model of power distribution system faults and the large language model of power distribution system fault detection, and aligning feature representations of different modalities at the instance-level coarse-grained level;

[0038] S04. Apply the aligned large language model for distribution system fault detection and the basic model for distribution system fault prediction to downstream tasks and use them for distribution system fault detection after fine-tuning.

[0039] Through the above implementation, multiple modal features of ID table features and text features are first selected for detection, reducing or avoiding the detection error of a single modality. Furthermore, fine-grained feature alignment of the basic prediction model of distribution system faults and the large language model of distribution system fault detection is provided, allowing the mask features of one modality to use the information of another modality when reconstructing information, thereby aligning features between different modalities and improving the accuracy of distribution fault detection. Finally, the above two models are fine-tuned together on the downstream distribution fault detection task to combine their advantages in processing different modal features.

[0040] In some embodiments of the present application, the ID table feature is constructed by the following steps: the preset dimension is R N×F The real-time operation data characteristics of the distribution system are presented in the form of a two-dimensional ID table. The dimension of the two-dimensional ID table is set to R N×F , where N represents the number of timestamps, and F represents the total number of power distribution fault features; each row of the two-dimensional ID table represents instance running data n at a timestamp, and each column represents a power distribution fault feature f;

[0041] According to the data characteristics of the real-time operation data of the power distribution system, the row and column values ​​of the two-dimensional ID table are encoded to generate the ID table characteristics corresponding to the real-time operation data of the power distribution system. The encoding here is preferably one-hot encoding. The encoding process includes:

[0042]

[0043]

[0044]

[0045] Among them, x v Indicates the mark indication of the feature value text v in the feature domain, x f is the one-hot representation vector of the distribution fault feature f, is the ID table feature vector, whose dimension is R V , V is the total number of distribution fault features, v f is the number of features in the distribution fault feature f, then

[0046] In some embodiments of the present application, the text features are constructed by the following steps: according to the data features of the real-time operation data of the power distribution system, a hard prompt template is used to generate corresponding text features after text splicing operation based on the text name of the power distribution fault feature and the text feature value corresponding to the power distribution fault feature in each instance input. According to the real-time operation data features of the power distribution system, the corresponding text features are generated in a hard prompt template manner, that is:

[0047]

[0048]

[0049] Among them, k f The text name of the power distribution fault feature f, w f Indicates the text feature value corresponding to the power distribution fault feature f in each instance input, and the operator Represents the concatenation operation between texts.

[0050] In some embodiments of the present application, a basic distribution system fault prediction model based on a multi-head attention mechanism for processing the ID table features is constructed, including:

[0051] Determine the model category and model structure of the basic prediction model of the distribution system fault before training. The model category is a multi-classification model, that is, the distribution fault detection problem is modeled as a multi-classification problem, that is, the distribution fault prediction problem, and its basic form is to use multiple feature data for multi-classification, where y n ∈{0,1,2} is the true label of the fault type of the distribution terminal instance n. When the distribution terminal of instance n has no fault, then y n = 0, when the power distribution terminal of instance n is overheated, then y n =1, when the distribution terminal of instance n is in voltage imbalance fault, then y n =2. The basic model for fault prediction of distribution system estimates the probability of each fault type According to the network structure classification of the basic model of distribution system fault prediction, it is abstracted into a three-layer network, namely the embedding layer, feature interaction layer and prediction layer.

[0052] The embedding layer is configured to: convert the high-dimensional sparse ID table features into a low-dimensional dense embedding matrix; the distribution fault feature f is converted from its corresponding representation matrix into an embedding vector by looking up the table. In the embedding layer, the high-dimensional sparse ID table feature vector Converted into a low-dimensional dense embedding matrix E n , the distribution fault feature f is converted from its corresponding representation matrix into an embedded vector by table lookup, which is expressed as:

[0053]

[0054] in, The one-hot vector representing the eigenvalues ​​of the distribution fault feature f, is the feature matrix of the distribution fault feature f, D is the embedding dimension, and the complete embedding matrix is ​​expressed as E = [e1; e2; ...; e f ;…;e F ]∈R F×D .

[0055] The feature interaction layer is configured as follows: taking the embedding matrix as input, outputting a dense representation vector for each distribution terminal instance operation data n; using a high-low order cross model of a double-tower structure to design complex interaction operations for each feature to obtain low-order or high-order feature cross terms; in the feature interaction layer, taking the embedding matrix E as input, outputting a dense representation vector v for each distribution terminal instance n n ; A high- and low-order cross-model with a double-tower structure is used to design complex interactive operations for each feature to obtain low-order or high-order feature cross-terms.

[0056] The prediction layer is configured as follows: after splicing the representation vectors of the high- and low-order cross outputs, the representation vectors are input into a neural network or a linear regression module, and then the Sigmoid function is adjusted to obtain the final classification probability of the fault type; in the prediction layer, after splicing the representation vectors of the high- and low-order cross outputs, the representation vectors are input into a neural network or a linear regression module, and then the Sigmoid function is adjusted to obtain the final user click probability

[0057] The basic distribution system fault prediction model is trained by using training samples in an end-to-end manner using a binary cross entropy loss to obtain the basic distribution system fault prediction model. The binary cross entropy loss is expressed as:

[0058]

[0059] In some embodiments of the present application, a large language model for distribution system fault detection is constructed for processing the text features, including: converting the text features into tokens based on a large language model pre-trained for distribution fault detection; inputting the obtained tokens into a Transformer-based encoder network, and obtaining a context-consistent representation after processing; and performing distribution fault classification by adding randomly initialized attention heads to the context-consistent representation to obtain a fault classification label corresponding to the text features. Specifically, a large language model pre-trained for distribution fault detection is selected as the text feature for distribution fault detection. feature processing model; based on the pre-trained large language model, the text features Converted into tokens; in natural language processing, tokens can be words, phrases, or characters in a sentence. These tokens are the basic units for language models to process text, and they form the basis for the model to understand and generate text. Then the Transformer-based encoder network is used to process the tokens obtained in the previous step and output a context-consistent representation w n ; By n Add a randomly initialized attention head to classify the power distribution fault and get the fault classification label

[0060] In some embodiments of the present application, bimodal features are constructed based on ID table features and text features to achieve fine-grained feature alignment of the basic prediction model for power distribution system faults and the large language model for power distribution system fault detection, and feature representations of different modalities are aligned at the instance-level coarse-grained level, including:

[0061] A certain proportion of feature domains are extracted from the ID table features and text features for masking operations to obtain the interference forms of the ID table features and text features, namely, masked ID table features and masked text features; that is, the interference forms of the ID table features and text features are obtained by data masking, which are respectively recorded as and To output the mask ID feature Unified from the original Extract a certain proportion of the feature domain from the mask operation; in order to output text features Unified from the original A certain proportion of the feature domain is extracted from the image for mask operation.

[0062] The ID table features and masked ID table features are input into the basic prediction model of distribution system faults for encoding, and the text features and masked text features are input into the large language model of distribution system fault detection for encoding to obtain the corresponding output; according to the output v after ID table feature encoding n And the output after encoding the masked text feature Perform joint reconstruction based on the output w after text feature encoding n Output after feature encoding of the mask ID table Conduct joint reconstruction;

[0063] The coarse-grained hierarchical feature alignment between the two modalities is established based on the instance-level comparative learning method. Combining the optimization objectives in the joint reconstruction and coarse-grained hierarchical feature alignment processes, training is performed based on the overall loss function of the bimodal alignment pre-training phase of the distribution fault prediction basic model and the distribution system fault detection large language model.

[0064] Further, in some embodiments, a specific implementation of the mask operation is provided. A certain proportion of feature domains are extracted from the ID table features, and the corresponding features are replaced with an additional mask symbol, which is shared by all feature domains; the feature vector of the ID table based on the encoding method is replaced with a specific symbol to obtain a mask ID table feature, wherein the mask ID table feature includes a dimension v f Specifically, in the mask ID feature In the generation process of , an additional mask symbol [MASK] is used to replace the corresponding feature. The mask symbol is shared by all feature domains, and the masked feature domain index set is recorded as The one-hot encoded ID feature vector uses the symbol 0 to mask the feature, so It is expressed as follows:

[0065]

[0066] in, is a dimension v f The zero vector of .

[0067] Extract a certain proportion of feature domains from text features, replace features with specific characters, and use hard prompt templates to obtain masked text features. Specifically, in the text features In the process of generating It is expressed as follows:

[0068]

[0069] Among them, the text feature uses the symbol <unknown> mask feature, then The masked tokens index set is recorded as

[0070] In some embodiments of the present application, the ID table features and the masked ID table features are input into the basic prediction model of the power distribution system fault for encoding, and the text features and the masked text features are input into the large language model of the power distribution system fault detection for encoding to obtain corresponding outputs, including:

[0071] The ID table features and the masked ID table features are input into the basic prediction model for power distribution system faults, and after being encoded by the embedding layer and the feature interaction layer in the basic prediction model for power distribution system faults, the corresponding output is obtained; exemplarily, the feature vector represented by the ID table features is used as input, and the basic model for power distribution fault prediction is used for data encoding, which is expressed as:

[0072]

[0073] Among them, f CTR(·) represents the basic distribution fault prediction model that only contains the embedding layer and the feature interaction layer, The output representation vector after high- and low-order crossover, D ID Represents the output dimension of the feature interaction layer.

[0074] And, the text features and the masked text features are input into the large language model for power distribution system fault detection, and the corresponding output is obtained through the hidden layer state set of the last layer of the large language model for power distribution system fault detection. Exemplarily, the feature vector of the text modality is used as input, and the data is encoded using the large language model, which is expressed as:

[0075]

[0076] Among them, f LLM (·) represents the pre-trained large language model, w n =[w n,l ]∈R L×Dtext Represents the set of hidden layer states of the last layer of the large language model, and L is the text feature The number of tokens, D text represents the output dimension of the hidden layer of the large language model, w n,1 A vector of [CLS]tokens representing a compressed representation of the entire text input.

[0077] In some embodiments of the present application, joint reconstruction is performed based on the output after text feature encoding and the output after mask ID table feature encoding, including: inputting the output after text feature encoding and the output after mask ID table feature encoding into the self-attention mechanism module in the form of a coded vector tuple to obtain an aggregated output; outputting the encoded mask ID feature And the text feature output w n , in the form of a tuple of encoding vectors As input, design a self-attention mechanism module and use it to aggregate and w n , its output is expressed as:

[0078]

[0079] in, K=V=w n , represents the trainable attention parameter matrix, Represents the scaling factor. The multi-head attention mechanism is integrated by h different self-attention mechanism modules, and its aggregate output process is:

[0080] u n =MultiHead(Q,K,V)=[head1||head2||…||headh ]W O

[0081] head i =Attention(QW i Q ,KW i K ,VW i V )

[0082] Among them, the symbol || represents the concatenation operation between vectors, W i Q , W i K , W i V and W O Represent Q, K, V and the parameter matrix of the aggregation representation mapping respectively.

[0083] For each mask ID table feature, after an independent multi-layer perceptron network and Softmax function, the distribution function on the candidate features of the aggregate output is obtained; for each mask ID feature Design an independent multilayer perceptron network g f (·), followed by a Softmax function to calculate the distribution p on the candidate features n,f ∈R V , expressed as:

[0084] z n,f =g f (u n ),z n,f ∈R V

[0085]

[0086] Among them, V represents the size of the entire feature space.

[0087] The reconstructed features of the ID table features are calculated using the distribution function and the cross entropy loss function of the ID table features for all mask features; the reconstruction loss based on the reconstructed features is calculated using the noise contrast estimation function. Then, the ID feature reconstruction loss is calculated using the cross entropy loss function for all mask features, which is expressed as:

[0088]

[0089] in, Indicates the number of mask features, loss BCE (·) represents the cross entropy loss function;

[0090] The reconstruction loss is calculated using the noise contrast estimation function, which is expressed as:

[0091]

[0092] Among them, δ(·) represents the Sigmoid function, t represents the positive characteristic index, and q represents the sampled noise characteristic index.

[0093] The joint reconstruction is performed according to the output after the ID table feature encoding and the output after the mask text feature encoding, including: performing a splicing operation on the output after the ID table feature encoding and the output after the mask text feature encoding in the form of a coded vector tuple to obtain a splicing vector; specifically, the encoded ID table feature output v n And text feature output Encoded vector tuple As input, for each mask tokens feature The splicing operation is:

[0094]

[0095] The concatenated vector Input to the prediction layer to obtain the reconstructed feature distribution, expressed as:

[0096]

[0097] Among them, M represents the vocabulary size, g LLM (·) represents the two-layer perceptron module at the decoder end of the large language model;

[0098] Finally, the large language model is pre-trained using the cross entropy loss function, expressed as:

[0099]

[0100] in, is the original lth token.

[0101] The coarse-grained hierarchical feature alignment between the two modalities is established based on the instance-level comparative learning method, including: using two independent linear layers and normalization layers to project the tokens vector output corresponding to the text input and the ID table feature output into a first multidimensional vector and a second multidimensional vector; [CLS] tokens vector represents the text input w n,1 , two independent linear layers and normalization layers are used to transform the [CLS] tokens output and ID feature output v n Projected into d-dimensional vectors, respectively recorded as the first multidimensional vector and the second multidimensional vector

[0102] The InfoNCE function is used to calculate the comparison loss between the first multidimensional vector and the second multidimensional vector at the instance level, which is expressed as:

[0103]

[0104]

[0105] Here, B represents the size of the training batch, τ represents the temperature hyperparameter, and sim(·) represents the similarity function.

[0106] The final loss is obtained according to the comparison loss, including: summing and averaging the two instance-level loss functions to obtain the final loss, which is expressed as:

[0107]

[0108] Based on the reconstruction loss and the final loss, the overall loss of the basic prediction model for power distribution system faults and the large language model for power distribution system fault detection is obtained. The overall loss is expressed as:

[0109]

[0110] Figure 2 The technical block diagram of the distribution network fault identification method according to the embodiment of the present application is schematically shown. Figure 2 As shown, the ID table features and text features are extracted from the real-time operation data, namely: timestamp: 2024-07-01:20:00, terminal ID: G002,…, voltage / V: 240, fault type: overheating fault, and the original tokens and original features are obtained through the embedding layer and feature interaction layer in the pre-trained LLM and the basic prediction model of distribution system faults, as well as the InfoNCE instance-level contrast loss.

[0111] In some embodiments of the present application, the aligned large language model for power distribution system fault detection and the basic model for power distribution system fault prediction are applied to downstream tasks and fine-tuned for power distribution fault detection, including:

[0112] In the downstream distribution system fault detection task, supervised click signals are used to jointly fine-tune the distribution system fault detection basic model and the distribution system fault large language model;

[0113] After the alignment, the large language model for distribution system fault detection and the basic model for distribution system fault prediction are respectively set with randomly restarted linear output layers to output the estimated fault classification probability, which is expressed as:

[0114]

[0115]

[0116] Among them, linear(·) represents the linear output layer of the corresponding model.

[0117] The final fault classification probability is obtained after weighted calculation based on the fault classification probabilities of the two models, which is expressed as:

[0118]

[0119] Among them, λ∈[0,1] is an adjustable parameter, which is used to weigh the loss between the distribution fault detection basic model and the large language model.

[0120] The estimated results corresponding to the final fault classification probability are compared with the true labels respectively, and the gap is measured using the cross entropy loss function to fine-tune and optimize the model. The gap is expressed as:

[0121]

[0122] Figure 3 The following schematically shows a block diagram of fine-tuning the downstream power distribution fault detection task according to an embodiment of the present application. Figure 3 As shown in the figure, the ID table features and text features are extracted from the real-time operation data, namely: timestamp: 2024-07-01:20:00, terminal ID: G002, ... voltage / V: 240, fault type: overheating fault, and the pre-trained LLM and distribution system fault basic prediction models are respectively obtained. and And the final fault classification probability is obtained In this embodiment, in the downstream power distribution fault detection task, the model fine-tuning process can greatly reduce the training time compared with the original supervised learning.

[0123] Through the above implementation, the model pre-training task of fine-grained feature-level interaction and alignment is achieved through the joint reconstruction of the feature level between the two modalities, which reduces the number of parameters and training time. At the same time, the appropriate model can be selected according to different scenarios, which helps to enhance the versatility and scalability of the framework.

[0124] Based on the same inventive concept, the present application also provides a distribution network fault identification device, Figure 4 The schematic diagram of the structure of the distribution network fault identification device according to the embodiment of the present application is shown. Figure 4As shown, the device includes: a feature confirmation module, which is used to determine the modal features of the real-time operation data of the distribution system for detection, and the modal features include ID table features and text features; a model construction module, which is used to respectively construct a basic distribution system fault prediction model based on a multi-head attention mechanism for processing the ID table features and a distribution system fault detection large language model for processing the text features; a feature alignment module, which is used to construct bimodal features according to ID table features and text features to achieve fine-grained feature alignment of the basic distribution system fault prediction model and the distribution system fault detection large language model, and align feature representations of different modalities at the instance-level coarse-grained level; and an application deployment module, which is used to apply the aligned distribution system fault detection large language model and distribution system fault prediction basic model to downstream tasks, and use them for distribution fault detection after fine-tuning.

[0125] In some optional implementations of the present application, the ID table feature is constructed by the following steps: the preset dimension is R N×F A two-dimensional ID table is provided, wherein N represents the number of timestamps, and F represents the total number of distribution fault features; each row of the two-dimensional ID table represents instance operation data n under a timestamp, and each column represents a distribution fault feature f; according to the data features of the real-time operation data of the distribution system, the row and column values ​​of the two-dimensional ID table are encoded to generate the ID table features corresponding to the real-time operation data of the distribution system.

[0126] In some optional embodiments of the present application, the text features are constructed by the following steps: based on the data features of the real-time operation data of the distribution system, a hard prompt template is used to generate corresponding text features after a text splicing operation based on the text name of the distribution fault feature and the text feature value corresponding to the distribution fault feature in each instance input.

[0127] In some optional embodiments of the present application, a basic distribution system fault prediction model based on a multi-head attention mechanism for processing the ID table features is constructed, including: determining the model category and model structure of the basic distribution system fault prediction model before training, the model category is a multi-classification model, and the model structure includes an embedding layer, a feature interaction layer and a prediction layer; the embedding layer is configured to: convert the high-dimensional sparse ID table features into a low-dimensional dense embedding matrix; the distribution fault feature f is converted from its corresponding representation matrix to an embedding vector by a table lookup method; the feature interaction layer is configured to: take the embedding matrix as input, A dense representation vector is output for each distribution terminal instance operation data n; a high- and low-order cross model with a dual-tower structure is used to design complex interactive operations for each feature to obtain low-order or high-order feature cross terms; the prediction layer is configured as follows: after splicing the representation vectors of the high- and low-order cross outputs, the vectors are input into a neural network or a linear regression module, and then the Sigmoid function is used to adjust the range to obtain the final classification probability of the fault type; the training sample is used to train the pre-trained distribution system fault prediction basic model in an end-to-end manner using a binary cross entropy loss to obtain the distribution system fault basic prediction model.

[0128] In some optional embodiments of the present application, a large language model for distribution system fault detection is constructed for processing the text features, including: converting text features into tokens based on a large language model pre-trained for distribution fault detection; inputting the obtained tokens into a Transformer-based encoder network to obtain a context-consistent representation after processing; and classifying distribution faults by adding randomly initialized attention heads to the context-consistent representation to obtain a fault classification label corresponding to the text feature.

[0129] In some optional embodiments of the present application, bimodal features are constructed based on ID table features and text features to achieve fine-grained feature alignment of the basic prediction model for power distribution system faults and the large language model for power distribution system fault detection, and feature representations of different modalities are aligned at the instance-level coarse-grained level, including: extracting a certain proportion of feature domains from ID table features and text features respectively for masking operations to obtain interference forms of ID table features and text features, namely, masked ID table features and masked text features; inputting ID table features and masked ID table features into the basic prediction model for power distribution system faults for encoding, and encoding the text features and The masked text features are input into the large language model for distribution system fault detection for encoding to obtain the corresponding output; the output after ID table feature encoding and the output after masked text feature encoding are jointly reconstructed, and the output after text feature encoding and the output after masked ID table feature encoding are jointly reconstructed; the coarse-grained hierarchical feature alignment between the two modalities is established according to the instance-level comparative learning method; combined with the optimization objectives in the joint reconstruction and coarse-grained hierarchical feature alignment process, training is performed based on the overall loss function of the bimodal alignment pre-training stage of the distribution fault prediction basic model and the distribution system fault detection large language model.

[0130] In some optional implementations of the present application, a certain proportion of feature domains are extracted from the ID table features and the text features for masking operations to obtain interference forms of the ID table features and interference forms of the text features, including: extracting a certain proportion of feature domains from the ID table features, replacing corresponding features with an additional mask symbol, and the mask symbol is shared by all feature domains; replacing features with specific symbols based on the feature vector of the ID table in the encoding method to obtain masked ID table features, wherein the masked ID table features include a dimension v f A zero vector of ; and a certain proportion of feature domains are extracted from text features, features are replaced with specific characters, and masked text features are obtained using a hard prompt template.

[0131] In some optional embodiments of the present application, ID table features and masked ID table features are input into a basic prediction model for distribution system faults for encoding, and text features and masked text features are input into a large language model for distribution system fault detection for encoding to obtain corresponding outputs, including: inputting ID table features and masked ID table features into the basic prediction model for distribution system faults, and obtaining corresponding outputs after encoding through the embedding layer and feature interaction layer in the basic prediction model for distribution system faults; and inputting text features and masked text features into a large language model for distribution system fault detection, and obtaining corresponding outputs through the hidden layer state set of the last layer of the large language model for distribution system fault detection.

[0132] In some optional embodiments of the present application, joint reconstruction is performed based on the output after text feature encoding and the output after mask ID table feature encoding, including: inputting the output after text feature encoding and the output after mask ID table feature encoding into a self-attention mechanism module in the form of a coding vector tuple to obtain an aggregated output; for each mask ID table feature, after an independent multi-layer perceptron network and a Softmax function, a distribution function on the candidate features of the aggregated output is obtained; for all mask features, the reconstructed features of the ID table features are calculated using the cross entropy loss function of the distribution function and the ID table features; and the reconstruction loss based on the reconstructed features is calculated using a noise contrast estimation function.

[0133] Joint reconstruction is performed based on the output after ID table feature encoding and the output after mask text feature encoding, including: splicing the output after ID table feature encoding and the output after mask text feature encoding in the form of a coding vector tuple to obtain a splicing vector; after inputting the splicing vector into the prediction layer in the basic prediction model of distribution system fault, obtaining a reconstructed feature distribution function; and using the cross entropy loss function of the feature distribution function and text features as the reconstruction loss.

[0134] In some optional embodiments of the present application, a coarse-grained hierarchical feature alignment between the two modalities is established according to an instance-level comparative learning method, including: using two independent linear layers and a normalization layer to project the tokens vector output and the ID table feature output corresponding to the text input into a first multidimensional vector and a second multidimensional vector; using the InfoNCE function to calculate the comparative loss of the first multidimensional vector and the second multidimensional vector at the instance level; obtaining the final loss based on the comparative loss; and obtaining the overall loss of the basic prediction model of the distribution system fault and the large language model for the distribution system fault detection based on the reconstruction loss and the final loss.

[0135] In some optional embodiments of the present application, the aligned large language model for distribution system fault detection and the basic model for distribution system fault prediction are applied to downstream tasks and used for distribution system fault detection after fine-tuning, including: in the downstream distribution system fault detection task, the basic model for distribution system fault detection and the large language model for distribution system fault are jointly fine-tuned using supervised click signals; randomly restarted linear output layers are respectively set in the aligned large language model for distribution system fault detection and the basic model for distribution system fault prediction, and estimated fault classification probabilities are respectively output; the final fault classification probability is obtained after weighted calculation based on the fault classification probabilities of the two models; the estimated results corresponding to the final fault classification probabilities are respectively compared with the true labels, and the gap is measured using the cross entropy loss function to fine-tune and optimize the model.

[0136] The specific definition of each functional module in the above-mentioned distribution network fault identification device can refer to the definition of the distribution network fault identification method above, which will not be repeated here. Each module in the above-mentioned system can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the electronic device in the form of hardware, or can be stored in the memory of the electronic device in the form of software, so that the processor can call and execute the operations corresponding to the above modules. It also performs fine-grained feature-level alignment between the attention mechanism model based on ID table features and the large language model based on text features to improve the accuracy of downstream distribution fault detection tasks, while also reducing the training time of the model.

[0137] In some embodiments of the present application, an electronic device is also provided, comprising: at least one processor; a memory connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the at least one processor executes the above-mentioned distribution network fault identification method. Its internal structure diagram can be as follows Figure 5 shown. Figure 5 The internal structure diagram of an electronic device according to an embodiment of the present application is schematically shown. The electronic device includes a processor A01, a network interface A02, a memory (not shown in the figure) and a database (not shown in the figure) connected via a system bus. Among them, the processor A01 of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes an internal memory A03 and a non-volatile storage medium A04. The non-volatile storage medium A04 stores an operating system B01, a computer program B02 and a database (not shown in the figure). The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 in the non-volatile storage medium A04. The network interface A02 of the electronic device is used to communicate with an external terminal through a network connection. When the computer program B02 is executed by the processor A01, a distribution network fault identification method is implemented.

[0138] Those skilled in the art will understand that Figure 5 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0139] In one embodiment provided in the present application, a machine-readable storage medium is provided, on which instructions are stored. When the instructions are executed by a processor, the processor is configured to execute the aforementioned distribution network fault identification method.

[0140] In one embodiment provided in the present application, a computer program product is provided, including a computer program, which implements the aforementioned distribution network fault identification method when executed by a processor.

[0141] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0142] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0143] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0144] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0145] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0146] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0147] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0148] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0149] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A distribution network fault identification method, characterized in that: The method includes: Determine the modal features of the real-time operation data of the power distribution system for detection, wherein the modal features include ID table features and text features; Constructing a basic distribution system fault prediction model based on a multi-head attention mechanism for processing the ID table features and a distribution system fault detection large language model for processing the text features respectively; Constructing bimodal features based on ID table features and text features to achieve fine-grained feature alignment of the basic prediction model for power distribution system faults and the large language model for power distribution system fault detection, and aligning feature representations of different modalities at the instance-level coarse-grained level; The aligned large language model for distribution system fault detection and the basic model for distribution system fault prediction are applied to downstream tasks and used for distribution system fault detection after fine-tuning.

2. The method according to claim 1, characterized in that The ID table feature is constructed by the following steps: The default dimension is R N×F A two-dimensional ID table, wherein N represents the number of timestamps, and F represents the total number of power distribution fault features; each row of the two-dimensional ID table represents instance running data n at a timestamp, and each column represents a power distribution fault feature f; According to the data characteristics of the real-time operation data of the power distribution system, the row and column values ​​of the two-dimensional ID table are encoded to generate the ID table characteristics corresponding to the real-time operation data of the power distribution system.

3. The method according to claim 1, characterized in that The text features are constructed by the following steps: According to the data characteristics of the real-time operation data of the distribution system, a hard prompt template is adopted to generate corresponding text features after text splicing operation based on the text name of the distribution fault feature and the text feature value corresponding to the distribution fault feature in each instance input.

4. The method according to claim 2, characterized in that: A basic distribution system fault prediction model based on a multi-head attention mechanism for processing the ID table features is constructed, including: Determine a model category and a model structure of a basic prediction model for power distribution system faults before training, wherein the model category is a multi-classification model, and the model structure includes an embedding layer, a feature interaction layer, and a prediction layer; The embedding layer is configured to: convert the high-dimensional sparse ID table features into a low-dimensional dense embedding matrix; the distribution fault feature f is converted from its corresponding representation matrix into an embedding vector by table lookup; The feature interaction layer is configured to: take the embedding matrix as input, output a dense representation vector for each distribution terminal instance operation data n; use a high- and low-order cross model with a double-tower structure to design complex interaction operations for each feature to obtain low-order or high-order feature cross terms; The prediction layer is configured to: splice the representation vectors of the high-order and low-order cross outputs, input them into a neural network or linear regression module, and then connect them to a Sigmoid function to adjust the range, so as to obtain the final classification probability of the fault type; The basic distribution system fault prediction model before training is trained by using training samples in an end-to-end manner using binary cross entropy loss to obtain the basic distribution system fault prediction model.

5. The method according to claim 3, characterized in that: Constructing a large language model for power distribution system fault detection for processing the text features, including: Convert text features into tokens based on a large language model pre-trained for power distribution fault detection; The obtained tokens are input into the Transformer-based encoder network, and after processing, a context-consistent representation is obtained; By adding randomly initialized attention heads to the context-consistent representation to perform power distribution fault classification, a fault classification label corresponding to the text feature is obtained.

6. The method according to claim 1, characterized in that According to the ID table features and text features, bimodal features are constructed to realize fine-grained feature alignment of the basic prediction model of the power distribution system fault and the large language model of the power distribution system fault detection, and feature representations of different modalities are aligned at the instance-level coarse-grained level, including: A certain proportion of feature domains are extracted from the ID table features and text features respectively to perform mask operations, so as to obtain interference forms of the ID table features and text features, namely, masked ID table features and masked text features; Input the ID table features and the masked ID table features into the basic prediction model of power distribution system faults for encoding, and input the text features and the masked text features into the large language model of power distribution system fault detection for encoding to obtain corresponding outputs; Jointly reconstructing the output after ID table feature encoding and the output after mask text feature encoding, and jointly reconstructing the output after text feature encoding and the output after mask ID table feature encoding; The coarse-grained hierarchical feature alignment between the two modalities is established based on the instance-level comparative learning method. Combining the optimization objectives in the joint reconstruction and coarse-grained hierarchical feature alignment processes, training is performed based on the overall loss function of the bimodal alignment pre-training phase of the distribution fault prediction basic model and the distribution system fault detection large language model.

7. The method according to claim 6, characterized in that A certain proportion of feature domains are extracted from the ID table features and text features for masking operations to obtain the interference form of the ID table features and the interference form of the text features, including: A certain proportion of feature domains are extracted from the ID table features, and the corresponding features are replaced with an additional mask symbol, which is shared by all feature domains; the feature vector of the ID table based on the encoding method is replaced with a specific symbol to obtain a mask ID table feature, wherein the mask ID table feature includes a dimension v f The zero vector of ; and A certain proportion of feature domains are extracted from text features, features are replaced with specific characters, and masked text features are obtained using a hard prompt template.

8. The method according to claim 6, characterized in that The ID table features and the masked ID table features are input into the basic prediction model of power distribution system faults for encoding, and the text features and the masked text features are input into the large language model of power distribution system fault detection for encoding to obtain the corresponding outputs, including: Inputting the ID table features and the mask ID table features into the basic prediction model for power distribution system faults, and obtaining corresponding outputs after encoding through the embedding layer and the feature interaction layer in the basic prediction model for power distribution system faults; and The text features and the masked text features are input into a large language model for power distribution system fault detection, and the corresponding output is obtained through a hidden layer state set of the last layer of the large language model for power distribution system fault detection.

9. The method according to claim 6, characterized in that Joint reconstruction is performed based on the output after text feature encoding and the output after mask ID table feature encoding, including: Inputting the output after the text feature encoding and the output after the mask ID table feature encoding into the self-attention mechanism module in the form of a coding vector tuple to obtain an aggregated output; For each mask ID table feature, after passing through an independent multi-layer perceptron network and a Softmax function, a distribution function on the candidate features of the aggregate output is obtained; For all mask features, the reconstructed features of the ID table features are calculated using the distribution function and the cross entropy loss function of the ID table features; Using a noise contrast estimation function to calculate a reconstruction loss based on the reconstruction feature; Joint reconstruction is performed based on the output after ID table feature encoding and the output after mask text feature encoding, including: The output after the ID table feature encoding and the output after the mask text feature encoding are concatenated in the form of a coded vector tuple to obtain a concatenated vector; After inputting the spliced ​​vector into the prediction layer in the basic prediction model of power distribution system fault, a reconstructed characteristic distribution function is obtained; The cross entropy loss function of the feature distribution function and the text feature is used as the reconstruction loss.

10. The method according to claim 6, characterized in that A coarse-grained hierarchical feature alignment between the two modalities is established based on an instance-level comparative learning method, including: Two independent linear layers and normalization layers are used to project the tokens vector output corresponding to the text input and the ID table feature output into a first multidimensional vector and a second multidimensional vector; Calculate the comparison loss between the first multidimensional vector and the second multidimensional vector at the instance level using the InfoNCE function; Obtaining a final loss according to the comparison loss; Based on the reconstruction loss and the final loss, the overall loss of the basic prediction model of distribution system fault and the large language model for distribution system fault detection are obtained.

11. The method according to claim 1, characterized in that: The aligned large language model for distribution system fault detection and the basic model for distribution system fault prediction are applied to downstream tasks and fine-tuned for distribution system fault detection, including: In the downstream distribution system fault detection task, supervised click signals are used to jointly fine-tune the distribution system fault detection basic model and the distribution system fault large language model; A randomly restarted linear output layer is set in the aligned large language model for distribution system fault detection and the basic model for distribution system fault prediction, respectively, to output the estimated fault classification probability; The final fault classification probability is obtained after weighted calculation based on the fault classification probabilities of the two models; The estimated results corresponding to the final fault classification probability are compared with the true labels respectively, and the gap is measured using the cross entropy loss function to fine-tune and optimize the model.

12. A distribution network fault identification device, characterized in that: The device includes: A feature confirmation module, used to determine the modal features of the real-time operation data of the power distribution system for detection, wherein the modal features include ID table features and text features; A model building module, used to respectively build a basic distribution system fault prediction model based on a multi-head attention mechanism for processing the ID table features and a distribution system fault detection large language model for processing the text features; A feature alignment module, for constructing bimodal features according to ID table features and text features to achieve fine-grained feature alignment of the basic prediction model for power distribution system faults and the large language model for power distribution system fault detection, and aligning feature representations of different modalities at an instance-level coarse-grained level; and The application deployment module is used to apply the aligned large language model for distribution system fault detection and the basic model for distribution system fault prediction to downstream tasks, and use them for distribution system fault detection after fine-tuning.

13. An electronic device, characterized in that: include: at least one processor; a memory connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the at least one processor implements the steps of the distribution network fault identification method described in any one of claims 1 to 11 by executing the instructions stored in the memory.

14. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instruction is executed by a processor, the steps of the distribution network fault identification method described in any one of claims 1 to 11 are implemented.

15. A computer program product, comprising a computer program, which, when executed by a processor, implements the distribution network fault identification method according to any one of claims 1 to 11.

Images