Power image redundancy processing method and system based on sparse attention mechanism

Abstract

The invention provides an electric power image redundancy processing method and system based on a sparse attention mechanism. The method comprises the following steps: acquiring a serialized embedded vector of each electric power image in an electric power image data set; inputting the serialized embedded vector of each power image into a Transform model of a sparse attention mechanism to obtain a fusion feature vector of each power image; calculating the similarity between every two power images based on the fusion feature vector of each power image; according to the similarity between every two power images, redundant images of the power image data set are determined and processed; according to the method, the fusion feature vectors of the power images are extracted through the Transform model of the sparse attention mechanism, important information in the power images can be reserved, interference elements can be removed, then the similarity between the power images is calculated through the fusion feature vectors, and automatic processing of the image redundancy problem is achieved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method and system for processing power image redundancy based on a sparse attention mechanism. Background Art

[0002] The redundancy problem of power image sample data is mainly reflected in the duplication and redundancy of information in the data set. This redundancy means that the same or similar image features and structures are repeatedly captured and represented, resulting in a waste of storage resources and computing power. At the same time, the redundant correlation of images may have a negative impact on model training and image analysis tasks, because the model may over-rely on these highly correlated samples and fail to generalize to a wider range of image data, resulting in poor model training results and directly affecting the effectiveness of business applications. Therefore, a method is needed to manage the excessive redundancy of power image samples.

[0003] At present, the commonly used redundancy management methods include hash algorithm and feature matching algorithm. The hash algorithm is relatively simple. The image data is mapped into a hash string through the hash algorithm. The existence of the same image is determined by comparing the hash string. The commonly used hash algorithms are mean hash and difference hash. The hash algorithm is used to check for duplicate images. The calculation speed is fast and the judgment conditions for duplication are simple. It is the most commonly used image duplication checking technology. The feature matching algorithm first extracts the gradient features and feature vectors of the image through the SIFT operator, and searches for areas with the same features in the image. The algorithm has strong robustness and can adapt to duplicate images under rotation, scaling and translation conditions.

[0004] However, the hash algorithm only retains the size relationship between image pixels and loses a lot of detail information in the image. Therefore, the hash algorithm is only suitable for checking duplicate images with the same content. It will cause misjudgment for image deformation, rotation, noise, etc. At the same time, the algorithm cannot provide quantitative image similarity and has low precision. Although the feature matching algorithm can compensate for the impact of rotation, scaling and translation on the hash algorithm, it is limited by the number and quality of extracted features. Inaccurate or omitted feature extraction will lead to inaccurate image recognition results, affecting the redundant image processing results. Summary of the invention

[0005] In order to overcome the above-mentioned deficiencies of the prior art, the present invention proposes a power image redundancy processing method based on a sparse attention mechanism, comprising:

[0006] The power image redundancy processing method based on sparse attention mechanism is characterized by comprising:

[0007] Get the serialized embedded vector of each power image in the power image dataset;

[0008] Inputting the serialized embedded vector of each power image into a Transformer model with a sparse attention mechanism to obtain a fused feature vector of each power image;

[0009] Calculating the similarity between every two power images based on the fused feature vector of each power image;

[0010] According to the similarity between each two power images, redundant images of the power image data set are determined and processed.

[0011] Optionally, the obtaining of a serialized embedded vector of each power image in the power image dataset includes:

[0012] Based on each power image in the power image dataset:

[0013] dividing the power image into a plurality of slices;

[0014] Inputting the plurality of slices into a pre-built convolutional neural network to obtain an embedded vector for each slice;

[0015] The number of the multiple slices of the power image is used as the sequence length, and the embedded vector of each slice is used as a sequence element to obtain the serialized embedded vector of the power image.

[0016] Optionally, the construction process of the convolutional neural network is:

[0017] Presetting the vector length of the embedded vector of each slice in the power image, and setting the number of output channels based on the vector length;

[0018] Setting the number of input channels and the size of the convolution kernel based on the size of each of the slices;

[0019] The construction of the convolutional neural network is completed by setting the convolution kernel size, the number of input channels and the number of output channels in the convolutional neural network.

[0020] Optionally, the construction process of the Transformer model of the sparse attention mechanism is:

[0021] Add a top-k selection mechanism to the multi-head attention mechanism module of the initial Transformer model;

[0022] Utilizing the top-k selection mechanism, a top-k selection method is adopted to adjust the attention scores generated in the multi-head attention mechanism module, so that the attention mechanism of the multi-head attention mechanism module is degraded to a sparse attention mechanism, thereby obtaining a Transformer model of the sparse attention mechanism.

[0023] Optionally, the adopting of a top-k selection method to adjust the attention score generated in the multi-head attention mechanism module includes:

[0024] Arrange the attention scores generated in the multi-head attention mechanism module in descending order;

[0025] If the attention score is ranked in the top k, the attention score is not adjusted; otherwise, the attention score is adjusted to 0 to obtain the final sparse attention score, thereby completing the adjustment of the attention score.

[0026] Optionally, the calculating the similarity between every two power images based on the fused feature vector of each power image includes:

[0027] Based on each of the power images in the power image dataset, calculating the cosine similarity between the fused feature vectors of every two power images;

[0028] The cosine similarity between the fused feature vectors of the two power images is used as the similarity between the two power images.

[0029] Optionally, determining and processing redundant images of the power image data set according to the similarity between each two power images includes:

[0030] If the similarity between the two power images is greater than a similarity threshold, determining the two power images as a redundant image pair;

[0031] In the power image data set, any redundant image in the redundant image pair is selected for deletion.

[0032] Based on the same inventive concept, the present invention proposes a power image redundancy processing system based on a sparse attention mechanism, comprising:

[0033] A vector conversion unit, used for obtaining a serialized embedded vector of each power image in the power image dataset;

[0034] A feature fusion unit, used for inputting the serialized embedded vector of each power image into a Transformer model of a sparse attention mechanism to obtain a fused feature vector of each power image;

[0035] a similarity calculation unit, configured to calculate the similarity between every two power images based on the fused feature vector of each of the power images;

[0036] The redundant image processing unit is used to determine and process the redundant images of the power image data set according to the similarity between each two power images.

[0037] Optionally, the vector conversion unit includes:

[0038] An image slicing subunit, configured to divide the power image into a plurality of slices;

[0039] The embedded vector conversion subunit is used to input the multiple slices into a pre-built convolutional neural network to obtain an embedded vector for each slice; the number of the multiple slices of the power image is used as the sequence length, and the embedded vector of each slice is used as the sequence element to obtain a serialized embedded vector of the power image.

[0040] Optionally, the vector conversion unit further includes: a convolutional neural network construction subunit, which is used to:

[0041] Presetting the vector length of the embedded vector of each slice in the power image, and setting the number of output channels based on the vector length;

[0042] Setting the number of input channels and the size of the convolution kernel based on the size of each of the slices;

[0043] The construction of the convolutional neural network is completed by setting the convolution kernel size, the number of input channels and the number of output channels in the convolutional neural network.

[0044] Optionally, the system further includes a sparse processing unit, configured to:

[0045] Add a top-k selection mechanism to the multi-head attention mechanism module of the initial Transformer model;

[0046] In the top-k selection mechanism: the top-k selection method is used to adjust the attention score generated in the multi-head attention mechanism module, so that the attention mechanism of the multi-head attention mechanism module is degraded to a sparse attention mechanism, and a Transformer model of the sparse attention mechanism is obtained.

[0047] Optionally, the sparse processing unit is specifically used to:

[0048] Arrange the attention scores generated in the multi-head attention mechanism module in descending order;

[0049] If the attention score is ranked in the top k, the attention score is not adjusted; otherwise, the attention score is adjusted to 0 to obtain the final sparse attention score, thereby completing the adjustment of the attention score.

[0050] Optionally, the similarity calculation unit is specifically used to:

[0051] Based on each of the power images in the power image dataset, calculating the cosine similarity between the fused feature vectors of every two power images;

[0052] The cosine similarity between the fused feature vectors of the two power images is used as the similarity between the two power images.

[0053] Optionally, the redundant image processing unit is specifically used to:

[0054] If the similarity between the two power images is greater than a similarity threshold, determining the two power images as a redundant image pair;

[0055] In the power image data set, any redundant image in the redundant image pair is selected for deletion.

[0056] Compared with the closest prior art, the present invention has the following beneficial effects:

[0057] The power image redundancy processing method and system based on the sparse attention mechanism provided by the present invention include: obtaining a serialized embedded vector of each power image in a power image data set; inputting the serialized embedded vector of each power image into a Transformer model of a sparse attention mechanism to obtain a fused feature vector of each power image; calculating the similarity between every two power images based on the fused feature vector of each power image; determining and processing the redundant images of the power image data set according to the similarity between every two power images; in the present invention, the fused feature vector of the power image is extracted through the Transformer model of the sparse attention mechanism, which can retain important information in the power image and remove interference. The method can remove disturbing elements and focus on the most valuable elements, reduce the computational complexity within the model, and make the processing of redundant images more efficient and accurate; by obtaining the serialized embedded vector of the power image, the data structure of the power image can be adapted to the Transformer model, providing a basis for the next step of fusion feature extraction; by calculating the similarity between power images through fusion feature vectors, an accurate quantitative result is provided, and the redundant image result obtained by quantitative similarity recognition is more accurate, which can be used for the construction of power image sample sets with high refinement requirements, and similar images and identical images can be simultaneously identified through similarity; the present invention realizes the automatic processing of image redundancy problems and improves the efficiency of power image redundancy problem governance. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 A schematic diagram of a flow chart of a power image redundancy processing method based on a sparse attention mechanism provided by the present invention;

[0059] Figure 2The electric power operation scene image provided by the present invention;

[0060] Figure 3 The sliced ​​power operation scene image provided by the present invention;

[0061] Figure 4 The image set of the power operation scene after slicing and tiling provided by the present invention;

[0062] Figure 5 A schematic diagram of the process of acquiring the embedded vector of the power image provided by the present invention;

[0063] Figure 6 A schematic diagram of the structure of a typical Transformer model provided by the present invention;

[0064] Figure 7 A schematic diagram of the structure of the attention mechanism in the Transformer model provided by the present invention;

[0065] Figure 8 A schematic diagram of the structure of the sparse attention mechanism in the Transformer model provided by the present invention;

[0066] Fig. 9 A schematic diagram of the process flow of the top-k selection method provided by the present invention;

[0067] Fig.10 A schematic diagram of a flow chart of obtaining a fused feature vector for a Transformer model of a sparse attention mechanism provided by the present invention;

[0068] Fig.11 The cosine curve in the interval [-π, π] provided by the present invention;

[0069] Fig.12 A schematic diagram of the structure of the power image redundancy processing system based on the sparse attention mechanism provided by the present invention;

[0070] Fig.13 A schematic structural diagram of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0071] The specific implementation modes of the present invention are further described in detail below with reference to the accompanying drawings.

[0072] Example 1

[0073] The power image redundancy processing method based on sparse attention mechanism provided by the present invention is as follows: Figure 1 As shown, including:

[0074] S1. Obtain the serialized embedded vector of each power image in the power image dataset;

[0075] S2. Inputting the serialized embedded vector of each power image into the Transformer model of the sparse attention mechanism to obtain a fused feature vector of each power image;

[0076] S3, calculating the similarity between every two power images based on the fused feature vector of each power image;

[0077] S4. Determine and process redundant images of the power image data set according to the similarity between each two power images.

[0078] In this scheme, in order to solve the redundancy problem existing in the power image sample data (i.e., the power image dataset), a combined algorithm is designed to ensure the best redundant image recognition and governance effect. First, a sparse bias is introduced into the Attention mechanism of the Transformer model to obtain a Transformer model with a sparse attention mechanism, thereby reducing the computational complexity of the large-scale multi-head attention mechanism in the Transformer model. The power image is serialized to obtain a serialized embedded vector, and the fusion features of the image are obtained from the Transformer model. Based on the fused feature vector, the similarity calculation method is combined to determine and process the redundant images in the power image, and finally an efficient redundant recognition method for power images is constructed.

[0079] In step S1, a serialized embedded vector of each power image in the power image dataset is obtained.

[0080] The input of the Transformer model is a one-dimensional sequence. After the one-dimensional sequence passes through the Embedding layer of the Transformer model, it is converted into a matrix of seq_len*embed_size (sequence length*embed vector dimension), and then enters the Encoder and Decoder for calculation. However, images are different from one-dimensional sequences. Electricity images have height, width, and number of channels. Since the Transformer model is needed to obtain the fused feature vector of the electricity image in this solution, it is necessary to use an algorithm to convert the three-dimensional electricity image into a matrix form similar to seq_len*embed_size in order to adapt the electricity image to the Encoder and Decoder in the Transformer model.

[0081] For each power image in the power image dataset:

[0082] S11. Divide the power image into multiple slices.

[0083] The total number of slices of the power image is regarded as the sequence length seq_len, the number of elements of the slice's embedded vector is regarded as embed_size, and the embedded vectors of all slices are summed up to form a matrix of seq_len*embed_size, so that the image data can be adapted to the Encoder and Decoder modules of the Transformer model.

[0084] In this embodiment, an electric power operation scene image with a width of 384 pixels and a height of 256 pixels is taken as an example. Figure 2 As shown, the image is an RGB three-channel image, and the data structure of the image is a three-dimensional array of 384*256*3.

[0085] By slicing the power image with 64*64 pixels, 24 slices can be obtained, specifically, sub-images with a width of 64 pixels and a height of 64 pixels, such as Figure 3 shown.

[0086] After the sliced ​​power image is tiled, the data structure of the power image can be represented as a 24*64*64*3 four-dimensional array. The sliced ​​and tiled power image is as follows: Figure 4 shown.

[0087] The length 24 of the first dimension of the four-dimensional array can be regarded as the sequence length seq_len in the Transformer model, and then each 64*64*3 sub-array in the array is converted into a vector with a length of embedding vector dimension embed_size, so that the power image data can be adapted to the Transformer model. In order to complete the conversion process, a convolutional neural network is constructed.

[0088] S12. Input the multiple slices into a pre-built convolutional neural network to obtain an embedded vector for each slice; use the number of the multiple slices of the power image as the sequence length and the embedded vector of each slice as the sequence element to obtain a serialized embedded vector of the power image.

[0089] The construction process of the convolutional neural network is as follows:

[0090] The vector length of the embedded vector of each slice in the power image is pre-set, and the number of output channels is set based on the vector length; in this solution, the vector length of the embedded vector is set to embed_size, and the number of output channels is set based on the vector length to In this way, after convolution, the size of each 64*64*3 sub-array becomes Then expand it into a one-dimensional vector, and the length of the vector is embed_size;

[0091] The number of input channels and the size of the convolution kernel are set based on the size of each slice; in this embodiment, the size of the convolution kernel is 16*16 (it can also be set to other sizes, and the principle of setting is to adapt to the size of the input data and the size of the output vector), and the number of input channels is consistent with the number of channels of the power image, which is set to 3;

[0092] The construction of the convolutional neural network is completed by setting the convolution kernel size, the number of input channels and the number of output channels in the convolutional neural network.

[0093] like Figure 5 As shown in the figure, through the convolutional neural network constructed above, the 24 slices of the power image are convolved and flattened to convert them into vectors of length embed_size. By concat (merging and connecting) the vectors after the convolution of the 24 slices, a matrix of 24*embed_size is obtained, and the serialized embedded vector of the power image is obtained. After the convolutional neural network calculation, the image can be entered into the Encoder and Decoder of the Transformer model for calculation, completing the conversion of the power image based on the serialized embedded vector.

[0094] By converting power images into serialized embedded vectors, the image data can be adapted to the Transformer model, providing a basis for the next step of fusion feature extraction.

[0095] In step S2, the serialized embedded vector of each power image is input into a Transformer model of a sparse attention mechanism to obtain a fused feature vector of each power image.

[0096] Transformer is a deep learning model architecture for natural language processing (NLP) and other sequence-to-sequence tasks, which was first proposed by Vaswani et al. in 2017. The Transformer model architecture introduces the self-attention mechanism, which is a key innovation that makes it perform well when processing sequence data. The following are the important components and features of the Transformer model:

[0097] Self-Attention: This is one of the core concepts of the Transformer model, which enables the model to consider all positions in the input sequence at the same time, rather than processing them step by step like Recurrent Neural Network (RNN) or Convolutional Neural Network (CNN). The self-attention mechanism allows the model to give different attention weights to different parts of the input sequence, thereby better capturing semantic relationships;

[0098] Multi-Head Attention: The self-attention mechanism in Transformer is extended to multiple attention heads, each of which can learn different attention weights to better capture different types of relationships. Multi-Head Attention allows the model to process different information subspaces in parallel;

[0099] Stacked Layers: Transformer is usually composed of multiple identical encoder and decoder layers stacked together. These stacked layers help the model learn complex feature representations and semantics;

[0100] Positional Encoding: Since Transformer has no built-in sequence position information, it requires additional positional encoding to express the position order of words in the input sequence;

[0101] Residual Connections and Layer Normalization: These techniques help alleviate the vanishing and exploding gradient problems during training, making the model easier to train.

[0102] Encoder and Decoder: Transformer usually consists of an encoder for processing the input sequence and a decoder for generating the output sequence, which makes it suitable for sequence-to-sequence tasks such as machine translation.

[0103] The typical network structure of the Transformer model includes an encoder and a decoder, such as Figure 6As shown in the figure, the left part is the encoder and the right part is the decoder. After the input data (Inputs) enters the model, it is converted into an embedded expression vector through Input Embedding (input embedding layer), and then added with positional encoding (Positional Encoding) and enters the encoder. The encoder has two series modules: Multi-Head Attention mechanism (Multi-HeadAttention) module and Feed Forward neural network (Feed Forward) module, which outputs the encoding vector of the data; in the decoding process, the input data is shifted (shiftedright) to obtain the shifted output data Outputs, which is converted into an embedded expression vector through Output Embedding (embedding layer of the shifted output data), and then added with positional encoding and enters the decoder. The decoder has three series modules: Masked Multi-Head Attention mechanism (Masked The multi-head attention module receives the output from the encoder and the mask multi-head attention module at the same time. The output of the decoder is calculated through the linear layer (Linear) and the Softmax function layer to obtain the model prediction result (OutputProbablities). Add&Norm (addition and normalization) is a combined operation, which includes residual connection and layer normalization. It is executed after each sub-layer to stabilize the training process and accelerate convergence. Nx means that the content in the white box is repeated N times.

[0104] Different from the traditional seq2seq (sequence-to-sequence) model, the Transformer model uses the Attention structure to replace the RNN (recurrent neural network) and LSTM (long short-term memory network) structures commonly used in deep learning. Compared with the RNN network structure, the biggest advantage of the Attention structure is that it can be calculated in parallel, which greatly improves the calculation efficiency. At the same time, the Attention structure enables the model to learn contextual associations in a longer interval.

[0105] Long Short-Term Memory (LSTM) is a special type of Recurrent Neural Network (RNN). LSTM is designed to solve the gradient vanishing or gradient exploding problems encountered by traditional RNN when processing long sequence data. The core of the LSTM network is the mechanism of three gates: forget gate, input gate, and output gate. The purpose of the forget gate is to decide which information should be forgotten or retained from the memory cell at each time step; the input gate consists of two parts: a sigmoid activation function layer determines which values ​​will be updated, and a tanh activation function layer creates a new candidate value vector that will be added to the state; the output gate is responsible for deciding which part of the memory cell state will be output to the hidden state. These gates control the flow of information in an adaptive way, thereby capturing long-term dependent information.

[0106] The memory unit is the core of the LSTM network, responsible for maintaining and updating long-term dependency information throughout the sequence processing. The structure of the memory unit is relatively simple, mainly composed of one or more neurons, whose state is passed through time steps and updated only in a linear manner.

[0107] like Figure 7 As shown in the figure, in the Attention structure of the Transformer model, the data passes through three linear layers to obtain Query, Key, and Value. The attention score Attention Score is generated by Query and Key through matrix multiplication. The Attention Score reflects the correlation between Query and Key. The Attention Score is converted by the Softmax function layer and then multiplied by Value, and the Attention vector is output through the linear layer. Since the Query and Key are obtained from the same input data through different linear layers, the Attention Score also expresses the correlation between the previous and next sequences of the same sequence. It is also the most important part of the Attention mechanism. The sparse attention mechanism of this method is an improvement on the Attention Score.

[0108] Sparsity means that only a small number of elements in a vector or matrix are non-zero. Through sparse representation, the sequence can be represented as a linear combination of a set of sparse coefficients and basis vectors, where only a few important basis vectors are selected. This method uses the top-k selection method to degrade attention into sparse attention, retain the most contributing elements, remove irrelevant information, and focus on the most valuable elements, reducing the computational complexity within the model and making the processing of redundant images more efficient and accurate.

[0109] S21. The construction process of the Transformer model of the sparse attention mechanism is:

[0110] Add a top-k selection mechanism to the multi-head attention mechanism module of the initial Transformer model;

[0111] Utilizing the top-k selection mechanism, a top-k selection method is adopted to adjust the attention scores generated in the multi-head attention mechanism module, so that the attention mechanism of the multi-head attention mechanism module is degraded to a sparse attention mechanism, thereby obtaining a Transformer model of the sparse attention mechanism.

[0112] After adding the top-k selection mechanism to the multi-head attention mechanism module, the sparsity-improved Attention structure is as follows Figure 8 As shown in the figure, after the sparse attention mechanism generates the Attention Score, it arranges the attention scores generated by the multi-head attention mechanism module in order from large to small to determine whether it is top-k (i.e. Figure 8 topk in , that is, if it is determined that the attention score is ranked in the top k, the attention score is not adjusted; if it is determined otherwise, the attention score is adjusted to 0, and finally a SparseAttention Score is generated to complete the adjustment of the attention score.

[0113] The specific operation steps are shown in the figure below. Fig. 9 As shown, Query represents the element being processed, including the element Key also represents an element in a sequence, including elements By matching the query with the key, we can determine the attention of each element to the current element. They represent the degree of attention of each element in the Key to each current element in the Query after performing dot product operations on each element in the Query and the elements in the Key, that is, the attention score. After multiplying the attention score with the sparse matrix, the sparse attention score is obtained.

[0114] SparseAttention Score is calculated by adding non-top-k (i.e. Fig. 9 The topk value in is replaced with 0, and the important elements in the Attention Score are selected, which can effectively retain important information and remove the influence of noise.

[0115] The Transformer model based on the sparse attention mechanism can extract features based on attention weights (i.e., attention scores). By calculating the attention weights of different elements in the sequence, it is possible to highlight and select important parts of the sequence. These attention weights are used to fuse weighted sequence features, so that the fused features pay more attention to important information.

[0116] S22. Based on the Transformer model of the sparse attention mechanism and the conversion of the power image into a serialized embedded vector, the fusion feature extraction of the power image is performed. The Transformer model of the sparse attention mechanism includes a BatchNorm (batch normalization layer), a multi-head sparse attention mechanism module and a Feedforward (feedforward neural network) module. Fig.10 The process shown is to extract fusion features of power images.

[0117] For each power image, the serialized embedded vector of the power image is input into the Transformer model of the sparse attention mechanism, first passing through the BatchNorm layer in the model to solve the problem of gradient abnormality during model training, and then entering the multi-head sparse attention mechanism module. Through multiple sparse attention networks (Sparse Attention), important information in the image is extracted in parallel to weaken the influence of noise data. Then the Embedding vector (serialized embedded vector) is added, and then passed through the Batch Norm layer and entered the Feedforward module to output the fused feature vector of the power image.

[0118] In step S3, based on the fused feature vector of each power image, the similarity between every two power images is calculated.

[0119] By extracting the fusion features of the power image and representing the power image as a fusion feature vector, different power images can be mapped to points in the vector space. The angle between the point in the vector space and the origin can be used to measure the similarity of the vectors. The smaller the angle, the more similar the vectors are, and the corresponding power images are also more similar. In the vector space, the range of the angle between two points and the origin is [-π,π]. Within this range, the cosine function is symmetrical and can well express the characteristic that the smaller the angle, the higher the similarity, such as Fig.11 As shown, the horizontal axis represents the angle and the vertical axis represents the cosine value.

[0120] Cosine similarity is used to characterize the similarity of two images. Cosine similarity algorithm: the cosine value of the angle between two vectors in a vector space is used to measure the size of the difference between two individuals. The closer the cosine value is to 1, the closer the angle is to 0, indicating that the two vectors are more similar. The closer the cosine value is to 0, the closer the angle is to 90 degrees, indicating that the two vectors are less similar.

[0121] For each fused feature vector of two power images, the cosine similarity between the fused feature vectors of the two power images is calculated according to the following formula:

[0122]

[0123] Where cos(θ) represents the cosine similarity between the fused feature vectors of the two power images; a and b represent the fused feature vectors of the two power images respectively; |a| represents the modulus of the fused feature vector a; |b| represents the modulus of the fused feature vector b; θ represents the angle between the two fused feature vectors; x i and i They represent the coordinates of vectors a and b in the vector space respectively, i represents the i-th dimensional vector, and n is the dimension of the vector.

[0124] The cosine similarity between the fused feature vectors of the two power images is used as the similarity between the two power images.

[0125] S4. Determine and process redundant images of the power image data set according to the similarity between each two power images.

[0126] By calculating the similarity between every two power images in the power image data set, if the similarity between the two power images is greater than a similarity threshold, the two power images are determined as a redundant image pair;

[0127] In the power image data set, any redundant image in the redundant image pair is selected for deletion. That is, only one power image in the redundant image pair is selected and retained in the power image data set, thereby reducing the redundancy of power image samples and improving the quality of the power image sample library.

[0128] By fusing feature vectors to calculate the similarity between power images, accurate quantification results are provided. The redundant image results obtained by quantified similarity recognition are more accurate and can be used to construct power image sample sets with high refinement requirements. Similar images and identical images can be recognized simultaneously through similarity.

[0129] In the present invention, a sparse multi-head attention mechanism module is designed in the Transformer model. The attention sparse technology based on the top-k selection method degrades the attention into sparse attention, which can effectively obtain the association between the contexts of the power image sequence, retain the most contributing elements in the association, remove the interfering elements with small contributions, and focus on the most valuable elements. This selective method is very effective in retaining important information and removing noise;

[0130] Through the power image serialization method, that is, using convolutional neural networks to complete the serialization-based embedded vector conversion of power images, the image data structure is adapted to the encoder and decoder of the Transformer model, solving the problem that the image data structure cannot match the input requirements of the Transformer model;

[0131] The pre-built image fusion feature extraction network including BatchNorm, multi-head sparse attention and Feedforward modules, namely the Transformer model with sparse attention mechanism, is used to represent power images in vector space.

[0132] The angle between the fusion feature vectors of power images in the vector space is used as a measure of the similarity of power images, which enables the identification and efficient management of similar redundant image samples in the power image sample set.

[0133] Compared with the previous method based on manual investigation of redundancy problems in data sets, this method uses an automated algorithm to manage the redundancy problem of power images, and the efficiency of managing the redundancy problem of the power image sample set is improved by 30%; compared with the image redundancy identification method based on the hash algorithm, this method can not only identify images with the same content, but also images with similar content.

[0134] In summary, the power image redundancy processing method based on the sparse attention mechanism disclosed in the present invention first improves the Transformer model, uses the top-k selection method to degrade the attention of the Transformer model into sparse attention, retains the most contributing elements, removes irrelevant information, and focuses on the most valuable elements; then the power image is sliced, and a convolutional neural network is constructed to convert the sliced ​​image into an embedded expression vector, which is input into the improved Transformer model to obtain the fused feature vector of the power image; the cosine value of the angle between the fused feature vectors is calculated, and the cosine value is used to measure the similarity between the power images, and the redundant images in the power image data set are determined and processed by the similarity between the power images. In the present invention, the Transformer model is adapted to the power image, and the sparse technology is used to realize the efficient identification and management of the redundancy of the power image samples. And it can alleviate the problems of resource waste and poor model training effect caused by the redundancy of image samples.

[0135] The present invention has many advantages in the efficient management of power image redundancy by introducing the Transformer model and improving the attention mechanism in the model through sparseness. The improved Transformer model can learn the global and local information of power images and is suitable for complex image structures and semantic information. At the same time, the Transformer model has strong adaptability and can automatically adjust the model structure and weights according to different tasks and data. Although the computational complexity is relatively high, when processing large-scale image data, Transformer can provide more accurate and efficient image redundancy management effects, and provide efficient management technology for reducing the redundancy of power image samples and improving the quality of power image sample sets.

[0136] Example 2

[0137] Based on the same inventive concept, the present invention also provides a power image redundancy processing system based on a sparse attention mechanism, such as Fig.12 As shown, including:

[0138] A vector conversion unit, used for obtaining a serialized embedded vector of each power image in the power image dataset;

[0139] A feature fusion unit, used for inputting the serialized embedded vector of each power image into a Transformer model of a sparse attention mechanism to obtain a fused feature vector of each power image;

[0140] a similarity calculation unit, configured to calculate the similarity between every two power images based on the fused feature vector of each of the power images;

[0141] The redundant image processing unit is used to determine and process the redundant images of the power image data set according to the similarity between each two power images.

[0142] In a possible implementation, the vector conversion unit includes:

[0143] An image slicing subunit, configured to divide the power image into a plurality of slices;

[0144] The embedded vector conversion subunit is used to input the multiple slices into a pre-built convolutional neural network to obtain an embedded vector for each slice; the number of the multiple slices of the power image is used as the sequence length, and the embedded vector of each slice is used as the sequence element to obtain a serialized embedded vector of the power image.

[0145] In a possible implementation, the vector conversion unit further includes: a convolutional neural network construction subunit, which is used to:

[0146] Presetting the vector length of the embedded vector of each slice in the power image, and setting the number of output channels based on the vector length;

[0147] Setting the number of input channels and the size of the convolution kernel based on the size of each of the slices;

[0148] The construction of the convolutional neural network is completed by setting the convolution kernel size, the number of input channels and the number of output channels in the convolutional neural network.

[0149] In a possible implementation, the system further includes a sparse processing unit, configured to:

[0150] Add a top-k selection mechanism to the multi-head attention mechanism module of the initial Transformer model;

[0151] In the top-k selection mechanism: the top-k selection method is used to adjust the attention score generated in the multi-head attention mechanism module, so that the attention mechanism of the multi-head attention mechanism module is degraded to a sparse attention mechanism, and a Transformer model of the sparse attention mechanism is obtained.

[0152] In a possible implementation, the above-mentioned sparse processing unit is specifically used to:

[0153] Arrange the attention scores generated in the multi-head attention mechanism module in descending order;

[0154] If the attention score is ranked in the top k, the attention score is not adjusted; otherwise, the attention score is adjusted to 0 to obtain the final sparse attention score, thereby completing the adjustment of the attention score.

[0155] In a possible implementation, the similarity calculation unit is specifically used to:

[0156] Based on each of the power images in the power image dataset, calculating the cosine similarity between the fused feature vectors of every two power images;

[0157] The cosine similarity between the fused feature vectors of the two power images is used as the similarity between the two power images.

[0158] In a possible implementation, the redundant image processing unit is specifically used to:

[0159] If the similarity between the two power images is greater than a similarity threshold, determining the two power images as a redundant image pair;

[0160] In the power image data set, any redundant image in the redundant image pair is selected for deletion.

[0161] Example 3

[0162] like Fig.13 As shown, the present invention also provides an electronic device, which may be a computer device, a single-chip device, an intelligent mobile device, etc. The electronic device in this embodiment may include a processor, a memory, a transceiver component, etc. The memory, the processor, and the transceiver component are connected via a bus; the memory may be used to store an execution program, and an exemplary execution program may include instructions; the processor is used to execute the instructions stored in the memory. The memory may also be used to store data, which may be called and / or modified when the instructions are executed.

[0163] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), or application-specific integrated circuits.

[0164] (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, and are suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in the storage medium to implement the corresponding method flow or corresponding function, so as to implement the steps of the power image redundancy processing method based on the sparse attention mechanism in the above-mentioned embodiment.

[0165] Example 4

[0166] Based on the same inventive concept, the present invention also provides a readable storage medium, specifically an electronic device readable storage medium (Memory), which is a memory device in an electronic device for storing programs and data. It can be understood that the storage medium here can include both built-in storage media in electronic devices and, of course, extended storage media supported by electronic devices. The storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by a processor are also stored in the storage space, and these instructions can be one or more execution programs (including program codes). It should be noted that the storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor loads and executes one or more instructions stored in the storage medium, which can implement the steps of the power image redundancy processing method based on the sparse attention mechanism in the above embodiment.

[0167] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0168] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0169] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0170] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0171] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit its protection scope. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that after reading the present invention, those skilled in the art can still make various changes, modifications or equivalent substitutions to the specific implementation methods of the application, but these changes, modifications or equivalent substitutions are all within the protection scope of the claims to be approved.

Claims

1. A power image redundancy processing method based on sparse attention mechanism, characterized in that: include: Get the serialized embedded vector of each power image in the power image dataset; Inputting the serialized embedded vector of each power image into the Transformer model of the sparse attention mechanism to obtain a fused feature vector of each power image; Calculating the similarity between every two power images based on the fused feature vector of each power image; According to the similarity between each two power images, redundant images of the power image data set are determined and processed.

2. The method according to claim 1, characterized in that The step of obtaining a serialized embedded vector of each power image in the power image dataset includes: Based on each power image in the power image dataset: dividing the power image into a plurality of slices; Inputting the plurality of slices into a pre-built convolutional neural network to obtain an embedded vector for each slice; The number of the multiple slices of the power image is used as the sequence length, and the embedded vector of each slice is used as a sequence element to obtain the serialized embedded vector of the power image.

3. The method according to claim 2, characterized in that The construction process of the convolutional neural network is as follows: Presetting the vector length of the embedded vector of each slice in the power image, and setting the number of output channels based on the vector length; Setting the number of input channels and the size of the convolution kernel based on the size of each of the slices; The construction of the convolutional neural network is completed by setting the convolution kernel size, the number of input channels and the number of output channels in the convolutional neural network.

4. The method according to claim 1, characterized in that The construction process of the Transformer model of the sparse attention mechanism is as follows: Add a top-k selection mechanism to the multi-head attention mechanism module of the initial Transformer model; Utilizing the top-k selection mechanism, a top-k selection method is adopted to adjust the attention scores generated in the multi-head attention mechanism module, so that the attention mechanism of the multi-head attention mechanism module is degraded to a sparse attention mechanism, thereby obtaining a Transformer model of the sparse attention mechanism.

5. The method according to claim 4, characterized in that The top-k selection method is used to adjust the attention score generated in the multi-head attention mechanism module, including: Arrange the attention scores generated in the multi-head attention mechanism module in descending order; If the attention score is ranked in the top k, the attention score is not adjusted; otherwise, the attention score is adjusted to 0 to obtain the final sparse attention score, thereby completing the adjustment of the attention score.

6. The method according to claim 1, characterized in that The calculating the similarity between every two power images based on the fused feature vector of each power image comprises: Based on each of the power images in the power image dataset, calculating the cosine similarity between the fused feature vectors of every two power images; The cosine similarity between the fused feature vectors of the two power images is used as the similarity between the two power images.

7. The method according to claim 1, characterized in that The determining and processing of redundant images of the power image data set according to the similarity between each two power images includes: If the similarity between the two power images is greater than a similarity threshold, determining the two power images as a redundant image pair; In the power image data set, any redundant image in the redundant image pair is selected for deletion.

8. A power image redundancy processing system based on sparse attention mechanism, characterized in that: include: A vector conversion unit, used for obtaining a serialized embedded vector of each power image in the power image dataset; A feature fusion unit, used for inputting the serialized embedded vector of each power image into a Transformer model of a sparse attention mechanism to obtain a fused feature vector of each power image; a similarity calculation unit, configured to calculate the similarity between every two power images based on the fused feature vector of each of the power images; The redundant image processing unit is used to determine and process the redundant images of the power image data set according to the similarity between each two power images.

9. The system according to claim 8, characterized in that The vector conversion unit comprises: An image slicing subunit, configured to divide the power image into a plurality of slices; The embedded vector conversion subunit is used to input the multiple slices into a pre-built convolutional neural network to obtain an embedded vector for each slice; the number of the multiple slices of the power image is used as the sequence length, and the embedded vector of each slice is used as the sequence element to obtain a serialized embedded vector of the power image.

10. The system according to claim 9, characterized in that Also included is a sparse processing unit for: Add a top-k selection mechanism to the multi-head attention mechanism module of the initial Transformer model; In the top-k selection mechanism: the top-k selection method is used to adjust the attention score generated in the multi-head attention mechanism module, so that the attention mechanism of the multi-head attention mechanism module is degraded to a sparse attention mechanism, and a Transformer model of the sparse attention mechanism is obtained.

Images