Power image redundancy processing method and system based on sparse attention mechanism

Abstract

The present invention provides a power image redundancy processing method and system based on a sparse attention mechanism, comprising: obtaining a serialized embedded vector of each power image in a power image dataset; inputting the serialized embedded vector of each power image into a Transformer model of a sparse attention mechanism to obtain a fused feature vector of each power image; calculating the similarity between each two power images based on the fused feature vector of each power image; determining and processing redundant images of the power image dataset based on the similarity between each two power images; extracting the fused feature vector of the power image through the Transformer model of the sparse attention mechanism, thereby retaining important information in the power image and removing interfering elements, and then calculating the similarity between the power images by fusing the feature vectors, thereby realizing automated processing of the image redundancy problem.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method and system for processing power image redundancy based on a sparse attention mechanism. Background Art

[0002] The redundancy problem in power image sample data primarily manifests itself in information duplication and redundancy within the dataset. This redundancy means that identical or similar image features and structures are repeatedly captured and represented, resulting in a waste of storage resources and computing power. Furthermore, the redundant correlation of images can negatively impact model training and image analysis tasks, as the model may over-rely on these highly correlated samples and fail to generalize to a wider range of image data. This results in poor model training and directly impacts the effectiveness of business applications. Therefore, a method is needed to address the excessive redundancy of power image samples.

[0003] Currently, common redundancy mitigation methods include hashing algorithms and feature matching algorithms. Hash algorithms are relatively simple, mapping image data to hash strings and comparing hash strings to determine whether identical images exist. Common hash algorithms include mean hashing and difference hashing. Using hashing algorithms for image duplication detection is fast and has simple criteria for determining duplication, making it the most commonly used technique. Feature matching algorithms first use the SIFT operator to extract the image's gradient features and eigenvectors, then search for regions within the image with identical features. This algorithm is highly robust and can adapt to duplicate images subjected to rotation, scaling, and translation.

[0004] However, the hash algorithm only retains the size relationship between image pixels and loses a lot of detailed information in the image. Therefore, the hash algorithm is only suitable for checking duplicate images with the same content. It will lead to misjudgment for image deformation, rotation, noise, etc. At the same time, the algorithm cannot provide quantitative image similarity and has low precision. Although the feature matching algorithm can compensate for the impact of rotation, scaling and translation on the hash algorithm, the feature matching algorithm is limited by the number and quality of the extracted features. Inaccurate or omitted feature extraction will lead to inaccurate image recognition results, affecting the redundant image processing results. Summary of the Invention

[0005] To overcome the above-mentioned deficiencies of the prior art, the present invention proposes a power image redundancy processing method based on a sparse attention mechanism, comprising:

[0006] A power image redundancy processing method based on a sparse attention mechanism is characterized by comprising:

[0007] Get the serialized embedded vector of each power image in the power image dataset;

[0008] Inputting the serialized embedded vector of each power image into a Transformer model with a sparse attention mechanism to obtain a fused feature vector of each power image;

[0009] Calculating the similarity between each two power images based on the fused feature vector of each power image;

[0010] According to the similarity between each two power images, redundant images of the power image dataset are determined and processed.

[0011] Optionally, obtaining a serialized embedded vector for each power image in the power image dataset includes:

[0012] Based on each power image in the power image dataset:

[0013] dividing the power image into a plurality of slices;

[0014] Inputting the plurality of slices into a pre-built convolutional neural network to obtain an embedded vector for each slice;

[0015] The number of the slices of the power image is used as a sequence length, and the embedded vector of each slice is used as a sequence element to obtain a serialized embedded vector of the power image.

[0016] Optionally, the convolutional neural network is constructed as follows:

[0017] presetting a vector length of the embedded vector of each slice in the power image, and setting the number of output channels based on the vector length;

[0018] Setting the number of input channels and the convolution kernel size based on the size of each slice;

[0019] The construction of the convolutional neural network is completed by setting the convolution kernel size, the number of input channels and the number of output channels in the convolutional neural network.

[0020] Optionally, the construction process of the Transformer model of the sparse attention mechanism is:

[0021] Add a top-k selection mechanism to the multi-head attention mechanism module of the initial Transformer model;

[0022] Utilizing the top-k selection mechanism, a top-k selection method is adopted to adjust the attention scores generated in the multi-head attention mechanism module, so that the attention mechanism of the multi-head attention mechanism module is degraded to a sparse attention mechanism, thereby obtaining a Transformer model of the sparse attention mechanism.

[0023] Optionally, the adopting of a top-k selection method to adjust the attention scores generated in the multi-head attention mechanism module includes:

[0024] Arrange the attention scores generated in the multi-head attention mechanism module in descending order;

[0025] If the attention score is ranked in the top k, the attention score is not adjusted; otherwise, the attention score is adjusted to 0 to obtain the final sparse attention score, completing the adjustment of the attention score.

[0026] Optionally, the calculating the similarity between each two power images based on the fused feature vector of each power image includes:

[0027] Based on each of the power images in the power image dataset, calculating the cosine similarity between the fused feature vectors of every two power images;

[0028] The cosine similarity between the fused feature vectors of the two power images is used as the similarity between the two power images.

[0029] Optionally, determining and processing redundant images of the power image dataset based on the similarity between each two power images includes:

[0030] If the similarity between the two power images is greater than a similarity threshold, determining the two power images as a redundant image pair;

[0031] In the power image data set, any redundant image in the redundant image pair is selected for deletion.

[0032] Based on the same inventive concept, the present invention proposes a power image redundancy processing system based on a sparse attention mechanism, comprising:

[0033] A vector conversion unit, used to obtain a serialized embedded vector of each power image in the power image dataset;

[0034] a feature fusion unit, configured to input the serialized embedded vector of each power image into a Transformer model with a sparse attention mechanism to obtain a fused feature vector of each power image;

[0035] a similarity calculation unit, configured to calculate the similarity between each two power images based on the fused feature vector of each power image;

[0036] The redundant image processing unit is configured to determine and process the redundant images of the power image dataset according to the similarity between each two power images.

[0037] Optionally, the vector conversion unit includes:

[0038] an image slicing subunit, configured to divide the power image into a plurality of slices;

[0039] The embedded vector conversion subunit is used to input the multiple slices into a pre-built convolutional neural network to obtain an embedded vector for each slice; the number of the multiple slices of the power image is used as the sequence length, and the embedded vector of each slice is used as the sequence element to obtain a serialized embedded vector of the power image.

[0040] Optionally, the vector conversion unit further includes: a convolutional neural network construction subunit, configured to:

[0041] presetting a vector length of the embedded vector of each slice in the power image, and setting the number of output channels based on the vector length;

[0042] Setting the number of input channels and the convolution kernel size based on the size of each slice;

[0043] The construction of the convolutional neural network is completed by setting the convolution kernel size, the number of input channels and the number of output channels in the convolutional neural network.

[0044] Optionally, the system further includes a sparse processing unit, configured to:

[0045] Add a top-k selection mechanism to the multi-head attention mechanism module of the initial Transformer model;

[0046] In the top-k selection mechanism: the top-k selection method is used to adjust the attention score generated in the multi-head attention mechanism module, so that the attention mechanism of the multi-head attention mechanism module is degraded to a sparse attention mechanism, and a Transformer model of the sparse attention mechanism is obtained.

[0047] Optionally, the sparse processing unit is specifically configured to:

[0048] Arrange the attention scores generated in the multi-head attention mechanism module in descending order;

[0049] If the attention score is ranked in the top k, the attention score is not adjusted; otherwise, the attention score is adjusted to 0 to obtain the final sparse attention score, completing the adjustment of the attention score.

[0050] Optionally, the similarity calculation unit is specifically configured to:

[0051] Based on each of the power images in the power image dataset, calculating the cosine similarity between the fused feature vectors of every two power images;

[0052] The cosine similarity between the fused feature vectors of the two power images is used as the similarity between the two power images.

[0053] Optionally, the redundant image processing unit is specifically configured to:

[0054] If the similarity between the two power images is greater than a similarity threshold, determining the two power images as a redundant image pair;

[0055] In the power image data set, any redundant image in the redundant image pair is selected for deletion.

[0056] Compared with the closest prior art, the present invention has the following beneficial effects:

[0057] The present invention provides a method and system for processing power image redundancy based on a sparse attention mechanism, comprising: obtaining a serialized embedded vector of each power image in a power image dataset; inputting the serialized embedded vector of each power image into a Transformer model of a sparse attention mechanism to obtain a fused feature vector of each power image; calculating the similarity between each two power images based on the fused feature vector of each power image; determining and processing redundant images of the power image dataset based on the similarity between each two power images; extracting the fused feature vector of the power image through the Transformer model of the sparse attention mechanism in the present invention can retain important information in the power image and remove interference. The method can eliminate disruptive elements and focus on the most valuable elements, thus reducing the computational complexity within the model and making the processing of redundant images more efficient and accurate. By obtaining the serialized embedded vector of the power image, the data structure of the power image can be adapted to the Transformer model, providing a basis for the next step of fusion feature extraction. The similarity between power images is calculated by fusing feature vectors, providing an accurate quantitative result. The redundant image result obtained by quantitative similarity recognition is more accurate and can be used for the construction of power image sample sets with high refinement requirements. Similar images and identical images can be simultaneously identified by similarity. The present invention realizes the automated processing of image redundancy problems and improves the efficiency of power image redundancy problem management. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 A flow chart of the power image redundancy processing method based on the sparse attention mechanism provided by the present invention;

[0059] Figure 2The image of the power operation scene provided by the present invention;

[0060] Figure 3 The sliced image of the power operation scene provided by the present invention;

[0061] Figure 4 The sliced and tiled image set of the power operation scene provided by the present invention;

[0062] Figure 5 A schematic diagram of the process of acquiring embedded vectors of power images provided by the present invention;

[0063] Figure 6 A schematic diagram of the structure of a typical Transformer model provided by the present invention;

[0064] Figure 7 This is a schematic diagram of the structure of the attention mechanism in the Transformer model provided by the present invention;

[0065] Figure 8 A schematic diagram of the structure of the sparse attention mechanism in the Transformer model provided by the present invention;

[0066] Figure 9 A schematic diagram of the process of the top-k selection method provided by the present invention;

[0067] Figure 10 A schematic diagram of the process of obtaining a fused feature vector for the Transformer model of the sparse attention mechanism provided by the present invention;

[0068] Figure 11 The cosine curve in the interval [-π, π] provided by the present invention;

[0069] Figure 12 A schematic diagram of the structure of the power image redundancy processing system based on the sparse attention mechanism provided by the present invention;

[0070] Figure 13 A schematic structural diagram of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0071] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0072] Example 1

[0073] The power image redundancy processing method based on sparse attention mechanism provided by the present invention is as follows: Figure 1 Shown, including:

[0074] S1. Obtain the serialized embedded vector of each power image in the power image dataset;

[0075] S2. Inputting the serialized embedded vector of each power image into a Transformer model with a sparse attention mechanism to obtain a fused feature vector of each power image;

[0076] S3. Calculating the similarity between every two power images based on the fused feature vector of each power image;

[0077] S4. Determine and process redundant images of the power image dataset based on the similarity between each two power images.

[0078] In this scheme, in order to solve the redundancy problem existing in power image sample data (i.e., power image dataset), a combined algorithm is designed to ensure the best redundant image recognition and management effect. First, a sparse bias is introduced into the Attention mechanism of the Transformer model to obtain a Transformer model with a sparse attention mechanism, thereby reducing the computational complexity of the large-scale multi-head attention mechanism in the Transformer model. The power image is serialized to obtain a serialized embedded vector, and the fusion feature of the image is obtained from the Transformer model. Based on the fusion feature vector and combined with the similarity calculation method, the redundant images in the power image are determined and processed, and finally an efficient redundant recognition method for power images is constructed.

[0079] In step S1, a serialized embedded vector of each power image in the power image dataset is obtained.

[0080] The input to the Transformer model is a one-dimensional sequence. After passing through the Transformer model's Embedding layer, the one-dimensional sequence is converted into a matrix of size seq_len*embed_size (sequence length * embedding vector dimension). This matrix then enters the Encoder and Decoder for calculation. However, unlike one-dimensional sequences, power images have height, width, and number of channels. Since this solution requires the use of the Transformer model to obtain the fused feature vector of the power image, an algorithm is required to convert the three-dimensional power image into a matrix of size similar to seq_len*embed_size to ensure compatibility between the power image and the Encoder and Decoder in the Transformer model.

[0081] For each power image in the power image dataset:

[0082] S11. Divide the power image into multiple slices.

[0083] The total number of slices of the power image is regarded as the sequence length seq_len, the number of elements of the slice's embedded vector is regarded as embed_size, and the embedded vectors of all slices are summed up to form a matrix of seq_len*embed_size. In this way, the image data can be adapted to the Encoder and Decoder modules of the Transformer model.

[0084] In this embodiment, an electric power operation scene image with a width of 384 pixels and a height of 256 pixels is taken as an example. Figure 2 As shown, the image is an RGB three-channel image, and the data structure of the image is a three-dimensional array of 384*256*3.

[0085] By slicing the power image with 64*64 pixels, 24 slices can be obtained, specifically sub-images with a width of 64 pixels and a height of 64 pixels, such as Figure 3 shown.

[0086] After the sliced power image is tiled, the data structure of the power image can be represented as a 24*64*64*3 four-dimensional array. The sliced and tiled power image is as follows: Figure 4 shown.

[0087] The length of the first dimension of the four-dimensional array, 24, can be considered the sequence length seq_len in the Transformer model. Each 64*64*3 subarray in the array is then converted into a vector of the embedding vector dimension embed_size , adapting the power image data to the Transformer model. To complete this conversion process, a convolutional neural network is constructed.

[0088] S12. Input the multiple slices into a pre-built convolutional neural network to obtain an embedded vector for each slice; use the number of the multiple slices of the power image as the sequence length and the embedded vector of each slice as the sequence element to obtain a serialized embedded vector of the power image.

[0089] The construction process of the convolutional neural network is as follows:

[0090] The vector length of the embedded vector of each slice in the power image is pre-set, and the number of output channels is set based on the vector length; in this solution, the vector length of the embedded vector is set to embed_size, and the number of output channels is set based on the vector length to In this way, after convolution, the size of each 64*64*3 sub-array becomes Then expand it into a one-dimensional vector, and the length of the vector obtained is embed_size;

[0091] The number of input channels and the convolution kernel size are set based on the size of each slice. In this embodiment, the convolution kernel size is 16*16 (other sizes can also be set, depending on the size of the input data and the size of the output vector). The number of input channels is consistent with the number of channels of the power image and is set to 3.

[0092] The construction of the convolutional neural network is completed by setting the convolution kernel size, the number of input channels and the number of output channels in the convolutional neural network.

[0093] like Figure 5 As shown in the figure, the convolutional neural network constructed above convolves and flattens the 24 slices of the power image into vectors of length embed_size. By concatenating the 24 convolved vectors, a matrix of 24*embed_size is formed, resulting in the serialized embedded vector of the power image. After the convolutional neural network calculations, the image can be processed in the encoder and decoder of the Transformer model, completing the conversion of the power image into a serialized embedded vector.

[0094] By converting power images into serialized embedded vectors, the image data can be adapted to the Transformer model, providing a basis for the next step of fusion feature extraction.

[0095] In step S2, the serialized embedded vector of each power image is input into a Transformer model with a sparse attention mechanism to obtain a fused feature vector of each power image.

[0096] The Transformer is a deep learning model architecture used for natural language processing (NLP) and other sequence-to-sequence tasks. It was first proposed by Vaswani et al. in 2017. The Transformer model architecture introduces a self-attention mechanism, a key innovation that enables it to excel at processing sequential data. The following are the key components and features of the Transformer model:

[0097] Self-Attention: This is one of the core concepts of the Transformer model. It enables the model to consider all positions in the input sequence simultaneously, rather than processing them step by step like Recurrent Neural Networks (RNNs) or Convolutional Neural Networks (CNNs). The self-attention mechanism allows the model to assign different attention weights to different parts of the input sequence, thereby better capturing semantic relationships.

[0098] Multi-Head Attention: The self-attention mechanism in Transformer is extended to multiple attention heads, each of which can learn different attention weights to better capture different types of relationships. Multi-Head Attention allows the model to process different information subspaces in parallel;

[0099] Stacked Layers: Transformers are typically constructed by stacking multiple identical encoder and decoder layers. These stacked layers help the model learn complex feature representations and semantics.

[0100] Positional Encoding: Since Transformer has no built-in sequence position information, it requires additional positional encoding to express the position order of words in the input sequence;

[0101] Residual Connections and Layer Normalization: These techniques help alleviate the vanishing and exploding gradient problems during training, making the model easier to train.

[0102] Encoder and Decoder: Transformer typically consists of an encoder for processing the input sequence and a decoder for generating the output sequence, which makes it suitable for sequence-to-sequence tasks such as machine translation.

[0103] The typical network structure of the Transformer model includes an encoder and a decoder, such as Figure 6As shown, the left part is the encoder and the right part is the decoder. After the input data (Inputs) enters the model, it is converted into an embedded expression vector through Input Embedding (input embedding layer), and then added with positional encoding (Positional Encoding), and then enters the encoder. The encoder has two series modules: Multi-Head Attention mechanism (Multi-HeadAttention) module and Feed Forward neural network (Feed Forward) module, which outputs the encoding vector of the data; in the decoding process, the input data is offset (shiftedright) to obtain the offset output data Outputs, which is converted into an embedded expression vector through Output Embedding (embedding layer of offset output data), and then added with positional encoding, and then enters the decoder. The decoder has three series modules: Masked Multi-Head Attention mechanism (Masked Multi-Head Attention) module and Feed Forward neural network (Feed Forward) module. The multi-head attention mechanism module receives the output from the encoder and the masked multi-head attention mechanism module at the same time. The output of the decoder passes through the linear layer (Linear) and the softmax function layer to obtain the model prediction result (Output Probablities). Add&Norm (addition and normalization) is a combined operation, which includes residual connection and layer normalization. It will be executed after each sub-layer to stabilize the training process and accelerate convergence. Nx means that the content in the white box is repeated N times.

[0104] Unlike traditional seq2seq (sequence-to-sequence) models, the Transformer model uses an attention architecture, replacing the RNN (recurrent neural network) and LSTM (long short-term memory) architectures commonly used in deep learning. Compared to the RNN architecture, the attention architecture's greatest advantage is its parallel computational capability, significantly improving computational efficiency. Furthermore, the attention architecture enables the model to learn contextual relationships over a longer timeframe.

[0105] Long Short-Term Memory (LSTM) is a special type of recurrent neural network (RNN). LSTM is designed to address the vanishing or exploding gradient problems encountered by traditional RNNs when processing long sequences of data. At the core of the LSTM network is a three-gate mechanism: a forget gate, an input gate, and an output gate. The forget gate determines which information should be forgotten or retained from the memory cell at each time step. The input gate consists of two parts: a sigmoid activation function layer that determines which values are updated, and a tanh activation function layer that creates a new candidate value vector to be added to the state. The output gate is responsible for determining which part of the memory cell state is output to the hidden state. These gates adaptively control the flow of information, thereby capturing long-term dependencies.

[0106] The memory cell is the core of the LSTM network, responsible for maintaining and updating long-term dependency information throughout the entire sequence processing process. The structure of the memory cell is relatively simple, mainly composed of one or more neurons, whose state is passed through time steps and updated only in a linear manner.

[0107] like Figure 7 As shown, in the Transformer model's Attention structure, data passes through three linear layers to produce a query, key, and value. The Attention Score is generated by matrix multiplication of the query and key. The Attention Score reflects the relationship between the query and key. After being transformed by the Softmax function layer, the Attention Score is multiplied by the value and then passes through a linear layer to output the Attention vector. Because the query and key are obtained from the same input data through different linear layers, the Attention Score also expresses the relationship between the previous and next parts of the same sequence. It is also the most important part of the Attention mechanism. The sparse attention mechanism of this method is an improvement on the Attention Score.

[0108] Sparsity refers to the fact that only a small fraction of the elements in a vector or matrix are nonzero. Sparse representation allows a sequence to be represented as a linear combination of sparse coefficients and basis vectors, where only a few important basis vectors are selected. This method uses a top-k selection algorithm to degrade attention to sparse attention, retaining the most contributing elements while removing irrelevant information, focusing on the most valuable elements. This reduces the computational complexity within the model and makes processing redundant images more efficient and accurate.

[0109] S21. The construction process of the Transformer model of the sparse attention mechanism is as follows:

[0110] Add a top-k selection mechanism to the multi-head attention mechanism module of the initial Transformer model;

[0111] Utilizing the top-k selection mechanism, a top-k selection method is adopted to adjust the attention scores generated in the multi-head attention mechanism module, so that the attention mechanism of the multi-head attention mechanism module is degraded to a sparse attention mechanism, thereby obtaining a Transformer model of the sparse attention mechanism.

[0112] After adding the top-k selection mechanism to the multi-head attention mechanism module, the sparsity-improved Attention structure is as follows Figure 8 As shown in the figure, after the sparse attention mechanism generates the Attention Score, it arranges the attention scores generated by the multi-head attention mechanism module in descending order to determine whether it is top-k (i.e. Figure 8 topk in ), that is, if it is judged that the attention score is ranked in the top k, the attention score is not adjusted; if it is judged otherwise, the attention score is adjusted to 0, and finally a SparseAttention Score is generated to complete the adjustment of the attention score.

[0113] The specific operation steps are shown in the figure below: Figure 9 As shown, Query represents the element being processed, including the element Key also represents an element in a sequence, including elements By matching the query with the key, we can determine the attention of each element to the current element. The dot product operation between each element in the Query and the element in the Key is performed, and the resulting attention of each element in the Key to each current element in the Query is called the attention score. The sparse attention score is obtained by multiplying the attention score with the sparse matrix.

[0114] SparseAttention Score is calculated by adding non-top-k (i.e. Figure 9 The topk) values in are replaced with 0, and the important elements in the Attention Score are selected, which can effectively retain important information and remove the influence of noise.

[0115] The Transformer model based on the sparse attention mechanism can extract features based on attention weights (i.e., attention scores). By calculating the attention weights of different elements in a sequence, it can highlight and select important parts of the sequence. These attention weights are used to fuse weighted sequence features, making the fused features focus more on important information.

[0116] S22. Based on the Transformer model of the sparse attention mechanism and the embedded vector conversion of the power image serialization, the fusion feature extraction of the power image is performed. The Transformer model of the sparse attention mechanism includes a BatchNorm (batch normalization layer), a multi-head sparse attention mechanism module and a Feedforward (feedforward neural network) module. Figure 10 The process shown is to extract fusion features of power images.

[0117] For each power image, the serialized embedded vector of the power image is input into the Transformer model of the sparse attention mechanism. It first passes through the BatchNorm layer in the model to solve the problem of gradient abnormality during model training. Then it enters the multi-head sparse attention mechanism module. Through multiple sparse attention networks (Sparse Attention), important information in the image is extracted in parallel to weaken the impact of noisy data. The embedding vector (serialized embedded vector) is then added and passed through the Batch Norm layer before entering the Feedforward module to output the fused feature vector of the power image.

[0118] In step S3, the similarity between every two power images is calculated based on the fusion feature vector of each power image.

[0119] By extracting the fusion features of the power image and representing the power image as a fusion feature vector, different power images can be mapped to points in the vector space. The angle between the point in the vector space and the origin can be used to measure the similarity of the vectors. The smaller the angle, the more similar the vectors are, and the corresponding power images are also more similar. In the vector space, the angle between two points and the origin ranges from [-π,π]. Within this range, the cosine function has symmetry and can well express the characteristic that the smaller the angle, the higher the similarity, such as Figure 11 As shown, the horizontal axis represents the angle and the vertical axis represents the cosine value.

[0120] Cosine similarity is used to represent the similarity between two images. The cosine similarity algorithm uses the cosine value of the angle between two vectors in a vector space as a measure of the difference between the two individuals. The closer the cosine value is to 1, the closer the angle is to 0, indicating that the two vectors are more similar. The closer the cosine value is to 0, the closer the angle is to 90 degrees, indicating that the two vectors are less similar.

[0121] For each fused feature vector of two power images, the cosine similarity between the fused feature vectors of the two power images is calculated according to the following formula:

[0122]

[0123] Where cos(θ) represents the cosine similarity between the fused feature vectors of the two power images; a and b represent the fused feature vectors of the two power images respectively; |a| represents the modulus of the fused feature vector a; |b| represents the modulus of the fused feature vector b; θ represents the angle between the two fused feature vectors; x i and y i Represent the coordinates of vectors a and b in the vector space, i represents the i-th dimension vector, and n is the dimension of the vector.

[0124] The cosine similarity between the fused feature vectors of the two power images is used as the similarity between the two power images.

[0125] S4. Determine and process redundant images of the power image dataset based on the similarity between each two power images.

[0126] By calculating the similarity between each two power images in the power image dataset, if the similarity between the two power images is greater than a similarity threshold, the two power images are determined to be a redundant image pair;

[0127] In the power image dataset, any redundant image in the redundant image pair is selected for deletion. That is, only one power image in the redundant image pair is selected and retained in the power image dataset, thereby reducing the redundancy of power image samples and improving the quality of the power image sample library.

[0128] By fusing feature vectors to calculate the similarity between power images, accurate quantitative results are provided. The redundant image results obtained by quantitative similarity identification are more accurate and can be used to construct power image sample sets with high refinement requirements. Similarity can also be used to simultaneously identify similar images and identical images.

[0129] In this paper, a sparse multi-head attention mechanism module is designed in the Transformer model. Based on the attention sparse technology of the top-k selection method, attention is degraded into sparse attention. It can effectively obtain the association between the contexts of the power image sequence, retain the most contributing elements in the association, remove the interfering elements with small contributions, and focus on the most valuable elements. This selective method is very effective in retaining important information and removing noise.

[0130] Through the power image serialization method, that is, using a convolutional neural network to complete the serialization-based embedded vector conversion of power images, the image data structure is adapted to the encoder and decoder of the Transformer model, solving the problem that the image data structure cannot match the input requirements of the Transformer model;

[0131] The pre-built image fusion feature extraction network, which includes BatchNorm, multi-head sparse attention, and Feedforward modules, namely the Transformer model with sparse attention mechanism, realizes the representation of power images in vector space.

[0132] The angle between the fused feature vectors of power images in vector space is used as a measure of power image similarity, enabling the identification and efficient management of similar redundant image samples in the power image sample set.

[0133] Compared with the previous method based on manual investigation of redundancy problems in data sets, this method uses an automated algorithm to manage the redundancy problem of power images, and the efficiency of managing the redundancy problem of the power image sample set is improved by 30%; compared with the image redundancy identification method based on hash algorithm, this method can not only identify images with the same content, but also images with similar content.

[0134] In summary, the present invention discloses a method for processing power image redundancy based on a sparse attention mechanism. First, the Transformer model is improved, and the top-k selection method is used to degrade the attention of the Transformer model into sparse attention, retaining the most contributing elements, removing irrelevant information, and focusing on the most valuable elements; then the power image is sliced, and a convolutional neural network is constructed to convert the sliced image into an embedded expression vector, which is input into the improved Transformer model to obtain a fused feature vector of the power image; the cosine value of the angle between the fused feature vectors is calculated, and the cosine value is used to measure the similarity between power images. The redundant images in the power image dataset are determined and processed based on the similarity between power images. In the present invention, the Transformer model is adapted to power images, and the sparsification technology is used to achieve efficient identification and management of power image sample redundancy. It can also alleviate the problems of resource waste and poor model training effect caused by image sample redundancy.

[0135] The present invention introduces the Transformer model and improves the attention mechanism in the model to achieve sparseness, which has many advantages in the efficient management of power image redundancy. The improved Transformer model can learn the global and local information of power images and is suitable for complex image structures and semantic information. At the same time, the Transformer model has strong adaptability and can automatically adjust the model structure and weights according to different tasks and data. Although the computational complexity is relatively high, when processing large-scale image data, the Transformer can provide more accurate and efficient image redundancy management effects, providing an efficient management technology for reducing the redundancy of power image samples and improving the quality of power image sample sets.

[0136] Example 2

[0137] Based on the same inventive concept, the present invention also provides a power image redundancy processing system based on a sparse attention mechanism, such as Figure 12 Shown, including:

[0138] A vector conversion unit, used to obtain a serialized embedded vector of each power image in the power image dataset;

[0139] a feature fusion unit, configured to input the serialized embedded vector of each power image into a Transformer model with a sparse attention mechanism to obtain a fused feature vector of each power image;

[0140] a similarity calculation unit, configured to calculate the similarity between each two power images based on the fused feature vector of each power image;

[0141] The redundant image processing unit is configured to determine and process the redundant images of the power image dataset according to the similarity between each two power images.

[0142] In a possible implementation, the vector conversion unit includes:

[0143] an image slicing subunit, configured to divide the power image into a plurality of slices;

[0144] The embedded vector conversion subunit is used to input the multiple slices into a pre-built convolutional neural network to obtain an embedded vector for each slice; the number of the multiple slices of the power image is used as the sequence length, and the embedded vector of each slice is used as the sequence element to obtain a serialized embedded vector of the power image.

[0145] In a possible implementation, the vector conversion unit further includes a convolutional neural network construction subunit configured to:

[0146] presetting a vector length of the embedded vector of each slice in the power image, and setting the number of output channels based on the vector length;

[0147] Setting the number of input channels and the convolution kernel size based on the size of each slice;

[0148] The construction of the convolutional neural network is completed by setting the convolution kernel size, the number of input channels and the number of output channels in the convolutional neural network.

[0149] In a possible implementation, the system further includes a sparse processing unit configured to:

[0150] Add a top-k selection mechanism to the multi-head attention mechanism module of the initial Transformer model;

[0151] In the top-k selection mechanism: the top-k selection method is used to adjust the attention score generated in the multi-head attention mechanism module, so that the attention mechanism of the multi-head attention mechanism module is degraded to a sparse attention mechanism, and a Transformer model of the sparse attention mechanism is obtained.

[0152] In a possible implementation, the above-mentioned sparse processing unit is specifically configured to:

[0153] Arrange the attention scores generated in the multi-head attention mechanism module in descending order;

[0154] If the attention score is ranked in the top k, the attention score is not adjusted; otherwise, the attention score is adjusted to 0 to obtain the final sparse attention score, completing the adjustment of the attention score.

[0155] In a possible implementation, the similarity calculation unit is specifically configured to:

[0156] Based on each of the power images in the power image dataset, calculating the cosine similarity between the fused feature vectors of every two power images;

[0157] The cosine similarity between the fused feature vectors of the two power images is used as the similarity between the two power images.

[0158] In a possible implementation, the redundant image processing unit is specifically configured to:

[0159] If the similarity between the two power images is greater than a similarity threshold, determining the two power images as a redundant image pair;

[0160] In the power image data set, any redundant image in the redundant image pair is selected for deletion.

[0161] Example 3

[0162] like Figure 13 As shown, the present invention also provides an electronic device, which may be a computer, a single-chip microcomputer, a smart mobile device, or the like. The electronic device in this embodiment may include a processor, a memory, a transceiver component, and the like. The memory, processor, and transceiver component are connected via a bus; the memory may be used to store an execution program, which may include instructions; and the processor may be used to execute the instructions stored in the memory. The memory may also be used to store data, which may be accessed and / or modified during the execution of the instructions.

[0163] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), or application-specific integrated circuits.

[0164] (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, are suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in the storage medium to implement the corresponding method flow or corresponding function, so as to implement the steps of the power image redundancy processing method based on the sparse attention mechanism in the above-mentioned embodiment.

[0165] Example 4

[0166] Based on the same inventive concept, the present invention also provides a readable storage medium, specifically an electronic device readable storage medium (Memory), which is a memory device in the electronic device for storing programs and data. It is understandable that the storage medium here can include both built-in storage media in the electronic device and, of course, extended storage media supported by the electronic device. The storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space. These instructions can be one or more execution programs (including program codes). It should be noted that the storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor loads and executes one or more instructions stored in the storage medium, which can implement the steps of the power image redundancy processing method based on the sparse attention mechanism in the above embodiment.

[0167] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0168] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0169] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0170] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0171] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit its scope of protection. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that after reading the present invention, those skilled in the art may still make various changes, modifications or equivalent substitutions to the specific implementation methods of the application, but these changes, modifications or equivalent substitutions are all within the scope of protection of the pending claims.

Claims

1. A power image redundancy processing method based on sparse attention mechanism, characterized by: include: Obtaining a serialized embedded vector for each power image in a power image dataset, comprising: based on each power image in the power image dataset: dividing the power image into a plurality of slices; inputting the plurality of slices into a pre-built convolutional neural network to obtain an embedded vector for each slice; using the number of the plurality of slices of the power image as a sequence length and the embedded vector of each slice as a sequence element to obtain a serialized embedded vector for the power image; Inputting the serialized embedded vector of each power image into a Transformer model with a sparse attention mechanism to obtain a fused feature vector of each power image; The construction process of the Transformer model with sparse attention mechanism is as follows: adding a top-k selection mechanism to the multi-head attention mechanism module of the initial Transformer model; using the top-k selection mechanism, adjusting the attention score generated in the multi-head attention mechanism module by using the top-k selection method, so that the attention mechanism of the multi-head attention mechanism module is degraded to a sparse attention mechanism, thereby obtaining a Transformer model with sparse attention mechanism; The top-k selection method is used to adjust the attention score generated in the multi-head attention mechanism module, including: arranging the attention scores generated in the multi-head attention mechanism module in descending order; if the attention score is ranked in the top k, then the attention score is not adjusted; otherwise, the attention score is adjusted to 0 to obtain a final sparse attention score, thereby completing the adjustment of the attention score; Calculating the similarity between each two power images based on the fused feature vector of each power image; According to the similarity between each two power images, redundant images of the power image dataset are determined and processed.

2. The method according to claim 1, wherein The construction process of the convolutional neural network is as follows: presetting a vector length of the embedded vector of each slice in the power image, and setting the number of output channels based on the vector length; Setting the number of input channels and the convolution kernel size based on the size of each slice; The construction of the convolutional neural network is completed by setting the convolution kernel size, the number of input channels and the number of output channels in the convolutional neural network.

3. The method according to claim 1, wherein The calculating the similarity between each two power images based on the fused feature vector of each power image includes: Based on each of the power images in the power image dataset, calculating the cosine similarity between the fused feature vectors of every two power images; The cosine similarity between the fused feature vectors of the two power images is used as the similarity between the two power images.

4. The method according to claim 1, wherein The determining and processing redundant images of the power image dataset according to the similarity between each two power images includes: If the similarity between the two power images is greater than a similarity threshold, determining the two power images as a redundant image pair; In the power image data set, any redundant image in the redundant image pair is selected for deletion.

5. The power image redundancy processing system based on sparse attention mechanism is characterized by: include: A vector conversion unit, used to obtain a serialized embedded vector of each power image in the power image dataset; The vector conversion unit includes: an image slicing subunit, configured to divide the power image into a plurality of slices; an embedded vector conversion subunit, configured to input the plurality of slices into a pre-built convolutional neural network to obtain an embedded vector for each slice; and using the number of the plurality of slices of the power image as a sequence length and the embedded vector of each slice as a sequence element to obtain a serialized embedded vector for the power image. a feature fusion unit, configured to input the serialized embedded vector of each power image into a Transformer model with a sparse attention mechanism to obtain a fused feature vector of each power image; a similarity calculation unit, configured to calculate the similarity between each two power images based on the fused feature vector of each power image; a redundant image processing unit, configured to determine and process redundant images of the power image dataset based on the similarity between each two power images; A sparse processing unit is used to add a top-k selection mechanism to the multi-head attention mechanism module of the initial Transformer model; in the top-k selection mechanism: the top-k selection method is used to adjust the attention score generated in the multi-head attention mechanism module, so that the attention mechanism of the multi-head attention mechanism module is degraded to a sparse attention mechanism, thereby obtaining a Transformer model with a sparse attention mechanism.

Images