Intelligent power grid defect data acquisition and classification method based on semantic communication

Abstract

The invention relates to the field of power grids and semantic communication, and particularly discloses an intelligent power grid defect data acquisition and classification method based on semantic communication, which is characterized in that a large-scale model is used for extracting specific knowledge in the field of intelligent power grids, a vector quantization compression mechanism is used for efficient transmission, and a two-stage training strategy is used for optimizing a training process. The method comprises the following steps: constructing a semantic encoder based on a large-scale model; carrying out vector quantization on the image features, and carrying out a de-quantization process on a quantization result at a receiving end by simulating a channel of an actual scene; decoding operation is carried out on the de-quantization result, and a classification task of the defect image is completed; performing two-stage training on the system to obtain a codec with optimal performance and a vector quantization module; the transmission and classification of inspection defect pictures are completed by using the trained system, and the classification accuracy and the transmission data volume are better.

Description

Technical Field

[0001] The present invention relates to power grid and semantic communication industries, and in particular to a method for collecting and classifying smart grid defect data based on semantic communication. Background Art

[0002] In recent years, traditional power grids are gradually being replaced by a smarter and more adaptable infrastructure, known as smart grids. In modern smart grids, various edge devices are deployed in substations and power lines to monitor and maintain power facilities. Among them, substation inspection robots, as one of the key devices of smart grids, use mobile robots as carriers, equipped with visible light cameras, infrared thermal imagers and other detection instruments, and can achieve functions such as autonomous movement and autonomous inspection. The inspection robot captures images through cameras and identifies them to provide effective assistance, such as identifying potential faults or defects. Common defect categories in smart grids include physical damage to equipment (such as damaged or worn components), thermal anomalies (surface overheating), corrosion of conductors or connectors, and insulation failures that may cause electrical failures. How to quickly and accurately identify these defects is crucial to maintaining the power grid and preventing the end of power services. After obtaining the defect information, how to effectively and efficiently transmit this data through wireless channels for real-time analysis and decision-making is another key factor. Therefore, an efficient communication method is needed to ensure the reliability and real-time nature of defect identification and data transmission, thereby ensuring the stable operation of the power grid.

[0003] Semantic communication is a promising solution, which aims to accurately convey the semantic information that the sender wants to convey rather than the original data, that is, "expression". However, although existing systems have shown their effectiveness, they have not been applied to specific smart grid communication scenarios.

[0004] Therefore, the semantic communication technology based on large models can be applied to power grid inspection robot technology to achieve accurate identification and classification of defects and improve the real-time and reliability of transmission. Summary of the invention

[0005] The purpose of the present invention is to provide a method for collecting and classifying smart grid defect data based on semantic communication to solve the problems raised in the above background technology.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A method for collecting and classifying smart grid defect data based on semantic communication uses a large-scale model to extract smart grid domain-specific knowledge, adopts a vector quantization compression mechanism for efficient transmission, and uses a two-stage training strategy to optimize the training process, the method comprising:

[0008] Building a semantic encoder based on a large-scale model;

[0009] The image features are vector quantized, and the quantized results are dequantized at the receiving end through a channel that simulates the actual scene;

[0010] Decode the dequantization result to complete the classification task of the defect image;

[0011] The system is trained in two stages to obtain the codec and vector quantization module with the best performance;

[0012] The trained system is used to complete the transmission and classification of inspection defect images.

[0013] As a further solution of the present invention, the construction of a semantic encoder based on a large-scale model specifically includes:

[0014] The grid defect image taken by the inspection robot in the substation is used as input S, which is input into the CLIP image encoder using the ViT architecture as a semantic encoder for feature extraction, which can be expressed as:

[0015] T = E(S; α);

[0016] where E(·;α) represents the semantic encoder, whose learnable parameter is α, T = [t1, t2, ... t M ]∈R M×K ;

[0017] Specifically, the input image S∈R H×W×C , where H is the image height H, W is the width, and C is the number of channels C. It is divided into non-overlapping small blocks of size P×P, each block is flattened and converted into an embedding vector of a fixed size, and then the embedding vector with position encoding is input into the Transformer module with a multi-head self-attention mechanism to calculate the attention score of each head:

[0018]

[0019] Among them, Q, K and V are query matrix, key matrix and value matrix respectively;

[0020] The output of the multi-head self-attention mechanism passes through two linear layers with ReLU activation and is expressed as:

[0021] FFN(x)=ReLU(W1x+b1)·W2+b2;

[0022] Then, use layer normalization to stabilize the training process:

[0023]

[0024] Among them, μ and σ are the mean and standard deviation, γ and β are learnable parameters;

[0025] Through the above operations, the semantic features T∈R are extracted through the 12-layer Transformer module. M×K , where M is the dimension of each feature vector and K is the total number of vectors.

[0026] As a further solution of the present invention, the vector quantization of the image features and the dequantization process of the quantization results at the receiving end through a channel simulating an actual scene specifically include:

[0027] The vector quantization module maps feature information to indexes through a pre-trained semantic codebook shared by the transceiver, where the codebook can be expressed as E = [e1, e2, ... e M ]∈R N×K , N represents the number of embedded vectors, and K represents the dimension of each vector. The transmitter sends the index corresponding to the embedded vector to the receiver. During mapping, the feature vector is mapped to the embedded vector according to the nearest neighbor reconstruction, which is defined as:

[0028]

[0029] Among them, m∈[1,M], n∈[1,N];

[0030] The encoded features are mapped into a series of indexes w∈R through the above formula M×1 , w is processed into a complex signal x∈C after modulation B×1 And transmit it through the channel:

[0031] The receiver receives the signal y∈C through the channel B×1 can be expressed as:

[0032] y=Hx+n;

[0033] Where H represents the channel matrix and n represents additive white Gaussian noise.

[0034] As a further solution of the present invention, the decoding operation of the dequantization result to complete the classification task of the defective image specifically includes:

[0035] Dequantized vector The semantic decoder is used for decoding, which is a neural network model with learnable parameters δ. The semantic decoder decodes the features D(·; δ) and generates a classification result C, which can be expressed as:

[0036]

[0037] As a further solution of the present invention, the two-stage training of the system to obtain the codec and vector quantization module with the best performance specifically includes:

[0038] The semantic codec is trained end-to-end to obtain the optimal semantic codec. Considering the image-based fault classification task, the cross entropy loss is used as the loss function to maximize the classification accuracy. The loss function is:

[0039]

[0040] Among them, p c and Respectively represented as the classification result C true probability and predicted probability;

[0041] The model weights of the trained semantic codec are frozen, and a vector quantization codebook is constructed to minimize the quantization error. The codebook is then trained according to the smart grid data distribution, and a direct gradient estimation method is introduced to copy the gradient from the receiver to the transmitter. Combining the embedding loss and the commitment loss, the loss function of the vector quantization process is:

[0042]

[0043] Here, sg[·] represents the gradient stopping operation, which is used to prevent the gradient from flowing through the loss function during back-propagation, and β is a hyperparameter that adjusts the impact of the commitment loss.

[0044] Compared with the prior art, the beneficial effect of the present invention is: using large-scale models to extract specific knowledge in the smart grid field. The large model extracts the substation fault defect image data collected by the substation inspection robot into knowledge compatible with the digital communication system, and introduces a robust compression mechanism using vector quantization. This mechanism simplifies the extracted knowledge into a compact codebook format, which serves as a shared knowledge base between the transmitter and the receiver, and can ensure efficient data compression and reliable transmission, maintaining the integrity of key information. On this basis, a two-stage training strategy is adopted, which can better optimize the large model extraction and vector quantization codebook design, and has better performance in classification accuracy and transmission data volume. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a flow chart of a method for collecting and classifying smart grid defect data based on semantic communication in an embodiment of the present invention;

[0046] Figure 2 A semantic encoder structure diagram based on a large-scale model-assisted semantic encoder in a method for collecting and classifying smart grid defect data based on semantic communication in an embodiment of the present invention

[0047] Figure 3The classification accuracy of a method for collecting and classifying smart grid defect data based on semantic communication according to an embodiment of the present invention varies with SNR under different channel conditions;

[0048] Figure 4 The classification accuracy of a method for collecting and classifying smart grid defect data based on semantic communication in an embodiment of the present invention varies with the number of quantization bits under different channel conditions;

[0049] Figure 5 It is the number of symbols and the compression rate transmitted through the channel of a smart grid defect data collection and classification method based on semantic communication in an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] In one embodiment of the present invention: a method for collecting and classifying smart grid defect data based on semantic communication, using a large-scale model to extract smart grid domain-specific knowledge, adopting a vector quantization compression mechanism for efficient transmission, and using a two-stage training strategy to optimize the training process, the method includes: constructing a semantic encoder based on a large-scale model; vector quantizing image features, and dequantizing the quantized results at the receiving end through a channel simulating an actual scene; decoding the dequantized results to complete the task of classifying defective images; performing two-stage training on the system to obtain a codec and a vector quantization module with optimal performance; and using the trained system to complete the transmission and classification of inspection defect images.

[0052] In one embodiment of the present invention:

[0053] The construction of a semantic encoder based on a large-scale model specifically includes:

[0054] The grid defect image taken by the inspection robot in the substation is used as input S, which is input into the CLIP (Contrastive Language-Image Pre-Training) image encoder using the ViT (Vision Transformer) architecture as a semantic encoder for feature extraction, which can be expressed as:

[0055] T = E(S; α);

[0056] where E(·;α) represents the semantic encoder, whose learnable parameter is α, T = [t1, t2, ... t M ]∈R M×K ;

[0057] Specifically, the input image S∈R H×W×C , where H is the image height H, W is the width, and C is the number of channels C. It is divided into non-overlapping small blocks of size P×P, each block is flattened, and converted into an embedding vector of fixed size. In order to retain the spatial information of each block in the original image, it is necessary to add position encoding to the embedding vector of each block. Then the embedding vector with position encoding is input into the Transformer module with multi-head self-attention mechanism, and the attention score of each head is calculated:

[0058]

[0059] Among them, Q, K and V are query matrix, key matrix and value matrix respectively;

[0060] The output of the multi-head self-attention mechanism passes through two linear layers with ReLU activation and is expressed as:

[0061] FFN(x)=ReLU(W1x+b1)·W2+b2;

[0062] Then, use layer normalization to stabilize the training process:

[0063]

[0064] Among them, μ and σ are the mean and standard deviation, γ and β are learnable parameters;

[0065] Through the above operations, the semantic features T∈R are extracted through the 12-layer Transformer module. M×K , where M is the dimension of each feature vector and K is the total number of vectors.

[0066] In one embodiment of the present invention:

[0067] The process of vector quantizing the image features and dequantizing the quantized results at the receiving end by simulating a channel in an actual scene specifically includes:

[0068] The extracted semantic features T can be vector quantized for further compression. The vector quantization module maps the feature information to the index through the pre-trained semantic codebook shared by the transceiver, where the codebook can be expressed as E = [e1, e2, ... e M ]∈R N×K, N represents the number of embedded vectors, and K represents the dimension of each vector. The transmitter sends the index corresponding to the embedded vector to the receiver without sending the actual feature to the receiver. This can minimize the data size while retaining basic information, which can greatly improve communication efficiency. During mapping, the feature vector is mapped to the embedded vector according to the nearest neighbor reconstruction, which is defined as:

[0069]

[0070] Among them, m∈[1,M], n∈[1,N];

[0071] The encoded features are mapped into a series of indexes w∈R through the above formula M×1 , w is processed into a complex signal x∈C after modulation B×1 And transmit it through the channel:

[0072] The receiver receives the signal y∈C through the channel B×1 can be expressed as:

[0073] y=Hx+n;

[0074] Where H represents the channel matrix, n represents additive Gaussian white noise;

[0075] Specifically, the channel uses Rayleigh channel and Rice channel to simulate the communication scenarios in the substation. Rayleigh channel is usually used to describe the propagation of wireless signals under non-line of sight (NLOS) conditions. In the substation, the inspection robot may move between large equipment, affected by surrounding equipment, walls and other obstacles, and experience multiple reflections and scattering, resulting in unstable signal quality. Its channel matrix is ​​a complex Gaussian distribution, which conforms to H~CN(0,σ 2 ),in n is additive white Gaussian noise, which conforms to in SNR is the current signal-to-noise ratio. For Rice channels, it is usually used to describe environments with line of sight (LOS) and multipath propagation, where there is a strong direct-line signal source and multiple scattering paths. In a substation, the inspection robot can maintain a good direct-line connection with the central server in some areas, but it will still be affected by scattering and reflection from nearby obstacles. Its channel matrix conforms to H~CN(μ,σ 2 ),in r is the Rice coefficient, where r = 1, and n is additive Gaussian white noise, which conforms to in SNR is the current signal-to-noise ratio.

[0076] Assuming that the receiver has perfect channel state information (CSI), the recovered signal is:

[0077]

[0078] After receiving the quantization index, the receiver maps the index back to the embedding vector in the vector quantization codebook to reconstruct the semantically encoded feature vector

[0079] In one embodiment of the present invention:

[0080] The decoding operation of the dequantization result to complete the classification task of the defective image specifically includes:

[0081] Dequantized vector The semantic decoder is used for decoding, which is a neural network model with learnable parameters δ. The semantic decoder decodes the features D(·; δ) and generates a classification result C, which can be expressed as:

[0082]

[0083] In one embodiment of the present invention:

[0084] The two-stage training of the system to obtain the codec and vector quantization module with the best performance specifically includes:

[0085] The semantic codec is trained end-to-end to obtain the optimal semantic codec. Considering the image-based fault classification task, the cross entropy loss is used as the loss function to maximize the classification accuracy. The loss function is:

[0086]

[0087] Among them, p c and Respectively represented as the classification result C true probability and predicted probability;

[0088] The model weights of the trained semantic codec are frozen, and a vector quantization codebook is constructed to minimize the quantization error. The codebook is then trained according to the smart grid data distribution so that it can use the background knowledge of the power grid to have better performance in specific scenarios. Due to the quantization operation, the back propagation through the nearest neighbor reconstruction process is not differentiable. For this reason, a direct gradient estimation method is introduced to copy the gradient from the receiver to the transmitter. Combining the embedding loss and the commitment loss, the loss function of the vector quantization process is:

[0089]

[0090] The first term is the embedding loss that ensures that the quantized feature vector is closely aligned with the corresponding embedding vector in the codebook, which can ensure accurate embedding vector representation; the second term is the commitment loss, which can make the feature vector close enough to the embedding vector in its specified codebook to prevent drastic changes in feature representation. sg[·] represents the gradient stop operation, which is used to prevent the gradient from flowing through the loss function during back propagation, and β is a hyperparameter that adjusts the impact of the commitment loss.

[0091] In one embodiment of the present invention:

[0092] The dataset used is a substation defect dataset obtained by a substation inspection robot, which includes 17 defect categories such as meter damage and overheating. It is divided into a training set and a test set in a ratio of 7:3, and the input image size is preprocessed to 224×224. The best performing model parameters in the two-stage training are stored for the final test. In the first stage of training, each image is encoded into a feature vector with an embedding space size of 512 dimensions, and then quantized into a vector index. Using the Adam optimizer, the encoder learning rate is set to and the weight decay is set to 0.01; similarly, the decoder learning rate and weight decay are set to and respectively. The training batch size is 64, and a total of 10 epochs are trained. In the second stage of training, our model is trained under the condition of adding noise. In order to support subsequent experiments, the codebook size is set to 4, 16, 64, 256, 1024, 4096, and the corresponding quantization bits are 2, 4, 6, 8, 10, 12. The embedding space dimension of each codebook is 512. The learning rate is , and the weight decay is Adam optimizer. All training is performed on NVIDIA GeForce RTX4090.

[0093] In one embodiment of the present invention:

[0094] The smart grid defect data collection and classification method based on semantic communication designed in this invention is compared with the following two schemes, both of which use classification accuracy as the evaluation index:

[0095] Traditional method: Use JPEG (Joint Photo Graphic Experts Group) image source coding to process the input grid defect image. Then use LDPC (Low-Density Parity-Check) channel coding with a code rate of 1 / 2 and 16QAM as the modulation method to pass through Rayleigh and Rice channels respectively. Then use semantic codec to extract features and classify the output image.

[0096] Resnet50: The semantic encoder uses Resnet50 as a pre-trained model to extract the features of the power grid defect picture, and then inputs the features into the vector quantization module for quantization. 16QAM is used as the modulation mode and passes through the Rayleigh and Rice channels respectively. Then the semantic decoder is used to classify the output image.

[0097] The present invention proposes a method for collecting and classifying smart grid defect data based on semantic communication. The system first uses a semantic encoder for feature extraction. The semantic encoder uses Vit-B / 16 as a pre-trained large model to extract the features of the grid defect picture. The features are then input into a vector quantization module for quantization. 16QAM is used as the modulation mode and passes through Rayleigh and Rice channels respectively. Then, a semantic decoder is used to classify the output image.

[0098] The classification accuracy of the two comparison schemes and the present invention under different channel conditions as SNR changes is as follows: Figure 4 Given. In this scenario, the number of quantization bits is fixed to 8 bits. Regardless of any channel conditions, the method of the present invention is always better than the traditional method, especially under low signal-to-noise ratio conditions. This is because the present invention uses vector quantization to better retain the global and detailed feature information of the input image. In addition, adjacent vectors in the codebook have similar features, and similar features can be restored even if the wrong codebook index is selected. Compared with the method using Resnet, the accuracy of the present invention in the Rice channel is improved by 9% when SNR=10dB. And when the SNR gradually increases, the difference is further improved. This is because Vit is a large-scale model based on Transformer, which uses a multi-head self-attention mechanism to more effectively capture global information from the power grid dataset.

[0099] Figure 4The classification accuracy of the present invention under different channel conditions as the number of quantization bits changes is shown. The classification accuracy of the present invention is always better than Resnet, mainly because Vit's large model architecture can provide better performance under the same quantization conditions. In addition, when the quantization level is increased from 4 bits to 8 bits, the classification accuracy of the present invention is significantly improved. At higher bit numbers, the impact area of ​​the quantization level on the classification accuracy is stable. This is because at low bits, more quantization bits can provide more feature information for the classifier, which helps to classify the results. At high bits, the information gain brought by the number of quantization bits has exceeded the scope of the classifier's utilization, and may increase unnecessary redundant information. In contrast, the classification accuracy of using Resnet as an encoder at a high bit number does not increase but decreases. This is because Vit's self-attention mechanism can capture global image feature information, and even if there is information loss during the quantization process, the classifier can still effectively classify through global features. Resnet extracts local features. As the number of bits increases, unnecessary local features and redundant details are retained, which reduces the performance of the classifier.

[0100] Figure 5 It is the number of symbols transmitted through the channel and the compression rate of the present invention. As can be seen from the figure, the present invention can compress the number of transmitted symbols by 99.37%. This is because the system with semantic embedding space transmits the index of the codebook instead of the original image, which greatly reduces the amount of transmitted data.

[0101] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A method for collecting and classifying smart grid defect data based on semantic communication, characterized in that: A large-scale model is used to extract domain-specific knowledge of smart grids, a vector quantization compression mechanism is used for efficient transmission, and a two-stage training strategy is used to optimize the training process. The method includes: Building a semantic encoder based on a large-scale model; The image features are vector quantized, and the quantized results are dequantized at the receiving end through a channel that simulates the actual scene; Decode the dequantization result to complete the classification task of the defect image; The system is trained in two stages to obtain the codec and vector quantization module with the best performance; The trained system is used to complete the transmission and classification of inspection defect images.

2. The method for collecting and classifying smart grid defect data based on semantic communication according to claim 1 is characterized in that: The construction of a semantic encoder based on a large-scale model specifically includes: The grid defect image taken by the inspection robot in the substation is used as input S, which is input into the CLIP image encoder using the ViT architecture as a semantic encoder for feature extraction, which can be expressed as: T = E(S; α); Among them, E(·; α) represents the semantic encoder, its learnable parameter is α, T = [t1, t2, ... t M ]∈R M×K ; Specifically, the input image S∈R H×W×C , where H is the image height H, W is the width, and C is the number of channels C. It is divided into non-overlapping small blocks of size P×P, each block is flattened and converted into an embedding vector of a fixed size, and then the embedding vector with position encoding is input into the Transformer module with a multi-head self-attention mechanism to calculate the attention score of each head: Among them, Q, K and V are query matrix, key matrix and value matrix respectively; The output of the multi-head self-attention mechanism passes through two linear layers with ReLU activation and is expressed as: FFN(x)=ReLU(W1x+b1)·W2+b2; Then, use layer normalization to stabilize the training process: Among them, μ and σ are the mean and standard deviation, γ and β are learnable parameters; Through the above operations, the semantic features T∈R are extracted through the 12-layer Transformer module. M×K , where M is the dimension of each feature vector and K is the total number of vectors.

3. The method for collecting and classifying smart grid defect data based on semantic communication according to claim 1 is characterized in that: The process of vector quantizing the image features and dequantizing the quantized results at the receiving end by simulating a channel in an actual scene specifically includes: The vector quantization module maps feature information to indexes through a pre-trained semantic codebook shared by the transceiver, where the codebook can be expressed as E = [e1, e2, ... e M ]∈R N×K , N represents the number of embedded vectors, and K represents the dimension of each vector. The transmitter sends the index corresponding to the embedded vector to the receiver. During mapping, the feature vector is mapped to the embedded vector according to the nearest neighbor reconstruction, which is defined as: Among them, m∈[1,M], n∈[1,N]; The encoded features are mapped into a series of indexes w∈R through the above formula M×1 , w is processed into a complex signal x∈C after modulation B ×1 And transmit it through the channel: The receiver receives the signal y∈C through the channel B×1 can be expressed as: y=Hx+n; Where H represents the channel matrix, n represents additive Gaussian white noise; After receiving the quantization index, the receiver maps the index back to the embedding vector in the vector quantization codebook corresponding to them through the dequantization process, and reconstructs the semantically encoded feature vector 4. The method for collecting and classifying smart grid defect data based on semantic communication according to claim 1 is characterized in that: The decoding operation of the dequantization result to complete the classification task of the defective image specifically includes: Dequantized vector The semantic decoder is used for decoding, which is a neural network model with learnable parameters δ. The semantic decoder decodes the features D(·; δ) and generates a classification result C, which can be expressed as:

5. The method for collecting and classifying smart grid defect data based on semantic communication according to claim 1 is characterized in that: The two-stage training of the system to obtain the codec and vector quantization module with the best performance specifically includes: The semantic codec is trained end-to-end to obtain the optimal semantic codec. Considering the image-based fault classification task, the cross entropy loss is used as the loss function to maximize the classification accuracy. The loss function is: Among them, p c and Respectively represented as the true probability and predicted probability of the classification result C; The model weights of the trained semantic codec are frozen, and a vector quantization codebook is constructed to minimize the quantization error. The codebook is then trained according to the smart grid data distribution, and a direct gradient estimation method is introduced to copy the gradient from the receiver to the transmitter. Combining the embedding loss and the commitment loss, the loss function of the vector quantization process is: Here, sg[·] represents the gradient stopping operation, which is used to prevent the gradient from flowing through the loss function during back-propagation, and β is a hyperparameter that adjusts the impact of the commitment loss.

Images