Power line aerial image retrieval analysis method and system and medium

By introducing attention modules and pairwise measurement triple loss function in the deep residual network, the problems of inefficient and low accuracy of traditional power line aerial image retrieval methods are solved, and more efficient feature learning and accurate images are achieved Search.

CN118897901APending Publication Date: 2024-11-05ZHEJIANG TUSHENG POWER TRANSMISSION ENG CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410728375.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-06
Publication Date
2024-11-05

AI Technical Summary

Technical Problem

Traditional power line aerial image retrieval methods are inefficient and have low accuracy, short hash code length, insufficient learning ability, and it is difficult to effectively distinguish between positive and negative samples, resulting in quantization errors.

Method used

The deep residual network is used to combine the attention module and the pairwise measurement triple loss function. Through the attention module, the content and position are emphasized or suppressed, the intermediate features are improved, and the pairwise measurement triple loss function is introduced to learn discriminant features. Improve the learning ability of the model.

Benefits of technology

It significantly improves image retrieval performance, reduces quantitative bias, improves feature learning ability, and improves the accuracy of search results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118897901A_ABST
    Figure CN118897901A_ABST
Patent Text Reader

Abstract

The invention discloses an electric power circuit aerial image retrieval analysis method and system and a medium, relates to the technical field of image retrieval processing, and aims to solve the problems that a traditional electric power circuit aerial image retrieval method is low in efficiency and low in precision, and the method comprises the following steps: inputting a triple image into a deep residual network to extract image features; learning emphasized or suppressed contents and positions through an attention module, and improving intermediate features; image features are converted into one-dimensional vectors, the one-dimensional vectors are mapped into binary hash codes, and a training result is optimized by adopting a paired measurement triple loss function; and inputting a to-be-retrieved image into the deep residual network, performing Hamming distance measurement comparison on the obtained to-be-retrieved hash code and the hash code data set obtained by training, and determining a retrieval result. The attention module is arranged, so that the feature learning capability can be improved; in addition, the adopted paired measurement triple loss function can improve the learning ability of the model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image retrieval processing, and in particular to a method, system and medium for retrieval and analysis of aerial images of power lines. Background Art

[0002] As the demand for electricity grows, it is crucial to ensure the safety of transmission lines. However, traditional inspection methods are limited by terrain and weather conditions, are inefficient and risky. Aerial image technology can achieve efficient and all-round monitoring of transmission lines through drones and aerial photography and measurement technology. Carrying out relevant research can not only improve inspection efficiency and safety, detect hidden dangers and faults in a timely manner, but also help promote the intelligent development of the power industry. Therefore, it is urgent and important to carry out research on aerial image retrieval of power lines, which is of great significance to improving the management level of power lines and ensuring the safety of power supply.

[0003] The hash codes learned by traditional power line aerial image retrieval methods are short in length, and their learning ability for complex scenes is insufficient, and their ability to express features is limited. Traditional triplet loss focuses on reducing the distance between the reference sample and the positive sample, and increasing the distance between the reference sample and the negative sample, but ignores the distance between the positive sample and the negative sample. Therefore, it is difficult to avoid the situation where the distance between the positive sample and the negative sample is smaller than the distance between the reference sample and the positive sample, which will lead to quantization errors in the learned hash codes. Summary of the invention

[0004] The present invention solves the problem of low efficiency and low precision of traditional power line aerial image retrieval methods, and proposes a power line aerial image retrieval and analysis method, system and medium, which can efficiently learn feature map expression and suppress quantization deviation in the discrete process; the attention module learns the content and position to be emphasized or suppressed, and effectively improves the intermediate features, enhances the network's ability to capture effective image information, and thus improves the ability of feature learning; in addition, a pairwise metric triplet loss function is added to learn discriminant features from the perspective of pairwise distance metric for training, so as to improve the learning ability of the model.

[0005] In order to achieve the above object, the present invention adopts the following technical solution: a method for retrieval and analysis of aerial images of power lines, comprising the following steps: S1, inputs the triplet image into a deep residual network to extract image features; uses the attention module to learn the content and location to emphasize or suppress, and improves the intermediate features; S2, converts the image features into a one-dimensional vector, maps the one-dimensional vector into a binary hash code, and uses a pairwise metric triplet loss function to optimize the training results; S3: Input the image to be retrieved into the deep residual network, compare the obtained hash code to be retrieved with the trained hash code dataset using the Hamming distance metric to determine the retrieval result.

[0006] In the present technical solution, the triplet image is first input into a deep residual network for image feature extraction, and an attention module is provided in the deep residual network, which can learn to emphasize or suppress content and position, effectively improve intermediate features, enhance the network's ability to capture effective image information, and provide feature learning capabilities; at the same time, a paired metric triplet loss function is introduced to further output accurate results; the method of the present invention can greatly improve image retrieval performance.

[0007] The present invention is further configured as follows: Step S1 comprises: S11, input the triplet image into the first convolutional layer, and mix the channel and spatial information together to obtain image features; S12, further refines the image features through four consecutive second convolutional layers; S13, inputting the refined image features into the attention module to output the improved image features.

[0008] In this technical solution, the triplet image is first input into the first convolutional layer to obtain image features, and then the image features are further processed through the first maximum pooling layer to obtain image features that are more sensitive to texture feature information; then, they are further refined through the second convolutional layer; finally, the refined image features are input into the attention module for further learning, and the improved image features are converted into the form of a one-dimensional vector.

[0009] The present invention is further configured as follows: Step S2 comprises: The image features are converted into one-dimensional vectors according to the fully connected layer, and a sub-network of the convolutional layer is generated through the deep residual network, which is mapped into a binary hash code; During the training process, the angle metric is introduced through the pairwise metric triplet loss function to alleviate the deviation of a single metric, and the global features are learned by combining the Euclidean distance and cosine distance metrics.

[0010] In this technical solution, the fully connected layer is connected to the attention module, and the fully connected layer can convert the two-dimensional feature image into a one-dimensional vector, realizing the end-to-end learning process. By improving the loss function, a pairwise metric triple loss is proposed, and an angle metric is introduced to alleviate the deviation of a single metric. By combining the Euclidean distance and cosine distance metrics, global features are learned to improve the metric learning loss.

[0011] The present invention is further configured as follows: Step S3 comprises: S31, inputting the image to be retrieved into the trained deep residual network, and obtaining the hash code to be retrieved after image feature extraction and image feature conversion; S32, traversing all hash codes in the hash code data set according to the hash code to be retrieved, and finding the hash code to be retrieved that satisfies a preset threshold range in the hash code data set by using Hamming distance measurement; S33, converting the found hash code to generate image data corresponding to the hash code as a search result.

[0012] In this technical solution, after the image to be retrieved is input into the improved deep residual network, the hash code to be retrieved is obtained, and the hash code to be retrieved is traversed and searched with all the data in the trained hash code data set to find the hash codes that meet the preset threshold range conditions. The image data corresponding to these hash codes are the retrieval results.

[0013] The present invention is further configured as follows: the step S11 further includes: After obtaining the image features, feature weights are generated according to the activation function, and image features that are more sensitive to texture feature information are obtained according to the first maximum pooling layer.

[0014] In this technical solution, the image features are extracted and optimized through the first convolutional layer, activation function and maximum pooling layer.

[0015] The present invention is further configured such that: the attention module can respectively learn the channel-meaningful part and the spatially most informative part.

[0016] In this technical solution, the attention module can enhance the network's ability to capture effective image information, thereby improving the ability of feature learning.

[0017] The present invention is further configured such that: the triplet image includes a positive sample, a reference sample and a negative sample.

[0018] In this technical solution, the triplet image needs to be input into the deep residual network at the same time.

[0019] A power line aerial image retrieval and analysis system adopts the above-mentioned power line aerial image retrieval and analysis method, comprising: The input module, which stores the triple graph, will be connected to the model building module; The model building module includes an improved deep residual network, which uses a pairwise metric triplet loss function to optimize the training results; the hash code comparison module compares the hash code to be retrieved with the hash codes in the hash code dataset using the Hamming distance metric to determine the final retrieval result.

[0020] In this technical solution, the input module is connected to the model construction module, and the model construction module is connected to the hash code comparison model. In the model construction module, the attention module is introduced to improve the deep residual network, and finally the metric learning loss is further improved through the pairwise metric triplet loss function.

[0021] The present invention is further configured as follows: the model building module includes an attention module, and the attention module includes a channel attention module and a spatial attention module connected to the channel attention module.

[0022] In this technical solution, the channel attention module in the attention module and the channel attention module can respectively learn the meaningful part on the channel and the most informative part on the space.

[0023] A medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the above-mentioned power line aerial image retrieval and analysis method are implemented.

[0024] In the present technical solution, when the computer program is running, the content of the above method can be executed.

[0025] The present invention can bring the following beneficial effects: The power line aerial image retrieval and analysis method of the present invention uses an attention model in a deep residual network. The attention module learns the content and position to be emphasized or suppressed, and effectively improves the intermediate features, thereby enhancing the network's ability to capture effective image information and improve the ability to learn features. The power line aerial image retrieval and analysis method involved in the present invention improves the loss function, proposes a pairwise metric triplet loss, introduces an angle metric to alleviate the deviation of a single metric, and improves the metric learning loss by combining Euclidean distance and cosine distance metrics and learning global features. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 The present invention is a schematic diagram of a model involved in a method for retrieval and analysis of aerial images of power lines.

[0027] Figure 2 It is a schematic diagram of an attention module of a power line aerial image retrieval and analysis system of the present invention.

[0028] Figure 3 It is a schematic diagram of the principle of a pairwise metric triplet loss function of a power line aerial image retrieval and analysis method of the present invention. DETAILED DESCRIPTION

[0029] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific implementation method described herein is only an optimal embodiment of the present invention, which is only used to explain the present invention and does not limit the scope of protection of the present invention. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0030] Example 1 This embodiment proposes a method for retrieval and analysis of aerial images of power lines. Figure 1 , Figure 2 and Figure 3 , which mainly includes the following steps.

[0031] In step S1, first, the triplet image is input into the deep residual network to extract the corresponding image features; then the attention module is used to learn the emphasized or suppressed content and position, and improve the intermediate features.

[0032] Step S1 specifically includes the following sub-steps.

[0033] Step S11, first input the triplet image into the convolution layer, and mix the channel and spatial information together to obtain image features.

[0034] Step S11 also includes: after obtaining the image features, generating feature weights according to the activation function, and obtaining image features that are more sensitive to texture feature information according to the first maximum pooling layer.

[0035] In this technical solution, image features are extracted and optimized through convolutional layers, activation functions and maximum pooling layers.

[0036] Step S12, further refining the image features through four convolution blocks in sequence.

[0037] Step S13, input the refined image features into the attention module to output the improved image features.

[0038] In this technical solution, the triplet image is first input into the convolution layer to obtain image features, and then the image features are further processed through the first maximum pooling layer to obtain image features that are more sensitive to texture feature information; then, the convolution module is used to further refine the image features; finally, the refined image features are input into the attention module for further learning, and the improved image features are converted into the form of a one-dimensional vector.

[0039] After completing the process of step S1, step S2 is immediately performed to convert the image features obtained in step S1 into a one-dimensional vector form, and then the one-dimensional vector is mapped to generate a binary hash code, and a pairwise metric triplet loss function is used to further optimize the training results to obtain more accurate data.

[0040] More specifically, the image features are converted into one-dimensional vectors in the fully connected layer, and a sub-network of convolutional layers is generated through a deep residual network to map the one-dimensional vectors into binary hash codes.

[0041] During the training process, the angle metric can be introduced according to the pairwise metric triplet loss function to alleviate the deviation of a single metric, and the global features can be learned by combining the Euclidean distance and cosine distance metrics.

[0042] In this technical solution, the fully connected layer is connected to the attention module, and the fully connected layer can convert the two-dimensional feature image into a one-dimensional vector, realizing the end-to-end learning process. By improving the loss function, a pairwise metric triple loss is proposed, and an angle metric is introduced to alleviate the deviation of a single metric. By combining the Euclidean distance and cosine distance metrics, global features are learned to improve the metric learning loss.

[0043] Step S3, input the image to be retrieved into the deep residual network to obtain the corresponding hash code to be retrieved; compare the hash code to be retrieved with the data in the trained hash code data set using the Hamming distance metric to determine the final retrieval result.

[0044] For step S3, in more detail, it mainly includes the following sub-steps.

[0045] In step S31, the image to be retrieved is first input into the trained deep residual network, and after image features are extracted, processed and converted, a corresponding hash code to be retrieved is obtained.

[0046] Step S32, traversing all hash codes in the hash code data set according to the hash code to be retrieved, and finding the hash code to be retrieved that satisfies a preset threshold range in the hash code data set by using Hamming distance measurement; Step S33: convert the found hash code to generate image data corresponding to the hash code as a search result.

[0047] In this technical solution, after the image to be retrieved is input into the improved deep residual network, the hash code to be retrieved is obtained, and the hash code to be retrieved is traversed and searched with all the data in the trained hash code data set to find the hash codes that meet the preset threshold range conditions. The image data corresponding to these hash codes are the retrieval results.

[0048] The above attention module can learn the meaningful part of the channel and the most informative part of the space respectively. In this technical solution, the attention module can improve the ability of the network to capture effective image information, thereby improving the ability of feature learning.

[0049] The above triplet image includes positive samples, reference samples and negative samples.

[0050] This embodiment also proposes a power line aerial image retrieval and analysis system, which mainly includes an input module, a model construction module and a hash code comparison module, wherein the input module is connected to the model construction module, and the model construction module is connected to the hash code comparison module.

[0051] The input module mainly stores the corresponding triple graph, and when it needs to be input into the model building module, the triple graph stored therein can be retrieved.

[0052] For the model building module, it includes an improved deep residual network model, which uses a pairwise metric triplet loss function to optimize the training results.

[0053] The hash code comparison module can compare the hash code to be retrieved with the hash codes in the hash code data set using the Hamming distance metric to determine the final retrieval result.

[0054] In this technical solution, the input module is connected to the model construction module, and the model construction module is connected to the hash code comparison model. In the model construction module, the attention module is introduced to improve the deep residual network, and finally the metric learning loss is further improved through the pairwise metric triplet loss function.

[0055] In the deep residual network model of the model building module, an attention module is included. The attention module includes a channel attention module and a spatial attention module. The channel attention module is connected to the spatial attention module.

[0056] In this technical solution, the channel attention module in the attention module and the channel attention module can respectively learn the meaningful part on the channel and the most informative part on the space.

[0057] The deep residual network model of the model building module also includes a convolution layer. When the triplet image is input into the model, it first enters the convolution layer, then passes through the gradient descent optimization algorithm and the activation function, and finally enters the first maximum pooling layer. The activation function can generate feature weights. The first maximum pooling layer can take the maximum value of the pixels in the pooling area, and the obtained feature map is more sensitive to texture feature information.

[0058] The first maximum pooling layer is connected to a convolution module, which includes four convolution blocks. Four consecutive convolution blocks can further refine the features.

[0059] The convolution module is connected to a second maximum pooling layer, which uses maximum pooling for downsampling.

[0060] The second maximum pooling layer is connected to the attention module, and the attention module is connected to the fully connected layer. The fully connected layer can convert the two-dimensional feature map into a one-dimensional vector.

[0061] The fully connected layer connects the convolutional layer sub-network, where the one-dimensional vector is mapped into a binary hash code.

[0062] For the hash code comparison module, the hash code to be retrieved is compared with the hash codes in the hash code data set using the Hamming distance metric to determine the final retrieval result.

[0063] A large amount of hash code data is stored in the hash code data set.

[0064] A medium is also proposed, on which a corresponding computer program is stored, and when the computer program is executed, the steps of the above-mentioned power line aerial image retrieval and analysis method can be implemented.

[0065] In the present technical solution, when the computer program is running, the content of the above method can be executed.

[0066] Example 2 Based on Example 1, this example describes the attention module and the pairwise metric triplet loss in detail.

[0067] 1. For the attention module, since the convolution operation mixes the information of the channel and space to obtain the image features, the attention module is used to emphasize the important target features in the channel and space, and then the channel and space attention modules are used to learn the meaningful part of the channel and the most informative part of the space respectively. Therefore, the image features in the network can be trained more efficiently by learning which information to emphasize and suppress, or by using a more effective architecture to use space and channel attention at the same time, and using experience to verify that using channels and space is more effective than using only channels.

[0068] refer to Figure 2 , when the input image feature vector U∈R C×H×W , the two-dimensional matrix spatial attention map M is derived through operation S ∈R 1×H×W and a one-dimensional vector channel attention map M C ∈R C×1×1 , the attention module learning process is: in Represents the multiplication corresponding to the vector elements, and the one-dimensional vector channel attention value is replicated along the two-dimensional matrix space dimension. U″ is the output, and the operation flow of the attention map is as follows Figure 2 shown.

[0069] The above-mentioned attention module includes a channel attention module and a spatial attention module.

[0070] For the channel attention module: the convolutional filters are responsible for building feature maps based on the learned weights in these filters, generally learning edges or learning textures, which together learn different feature representations of the target image information in the image embedded in the input tensor. Therefore, the channel attention module represents the number of convolutional filters that learn different feature maps of the input, and these feature maps also have different degrees of importance.

[0071] A channel attention module is added to the last layer of the convolutional layer because high-level feature maps mainly represent complex features with large receptive fields and images. The global image information is learned to selectively enhance local relevant features and suppress irrelevant features. First, the feature map U∈R C×H×W Implement the aggregation operation, which uses the aggregated feature map in its spatial dimension (H×W) to generate a one-dimensional vector feature of the channel attention: Among them U c ∈R H×W is the local feature of channel c, and the aggregation function q c (U) indicates that the spatial dimension is represented by global average pooling.

[0072] Next comes the weight learning calculation, which is implemented by the fully connected layer. The function F1(q, w) aims to fully capture the dependencies between channels and adaptively generate the channel weight map v∈R C The calculation formula is as follows: v=F1(q1,w)=σ(g(q1,w))=σ(w2δ(w1q)) where w1∈R K×C , w2∈R C×K , K is the number of hidden neurons, σ is the sigmoid() activation function, and δ is the ReLU activation function, which is used to generate the channel weight v at channel c C ∈(0, 1). The fully connected hidden layer can capture the nonlinear interactions within the channel.

[0073] The calculated channel feature map U will be applied to the weights. By multiplying the feature values ​​of each channel in U by different weights, the output U' of the channel attention module can be obtained: U′C =F(u c , v c )=u c ×v c .

[0074] For the spatial attention module: spatial attention will treat the features of different channels fairly. The spatial attention module is added to the low-level feature map because the low-level feature map mainly extracts spatial features such as contours and edges, and has fewer channels. The module self-learns the interaction of spatial points, enhances key areas and suppresses irrelevant areas. First, the feature map U∈R C×H×W Passed to the aggregation operation, the aggregation operation generates a spatial descriptor p∈R H×W By aggregating feature maps in their channel dimension C, a global distribution of spatial features is generated: Among them U h,w ∈R C is the local feature at the spatial position (h, w), and the aggregation function F indicates that the channel dimension is represented by global average pooling.

[0075] The next step is weight learning calculation, which is implemented by the convolutional layer. The F1(p, f) function mainly captures the spatial correlation and adaptively generates the spatial weight map w∈R H×W , the calculation formula is as follows: w=F1(p1,f)=σ(g(p1,f))=σ(f2δ(f1p)) Where f1 is a 3x3 convolutional block, and f2 is also a 3x3 convolutional block, denoted as Conv(3*3,1); m refers to the number of channels in the hidden feature map. σ is a sigmoid() activation function, and δ is a ReLU activation function, which is used to generate spatial weights at the (h, w) position by t h,w ∈(0, 1); essentially, the convolution operation with the original spatial descriptor as input captures the non-linear spatial relationship.

[0076] The weights calculated in the previous step are applied to the feature map U. Through spatial recalibration, the feature values ​​at different positions in U are multiplied by different weights to generate the output U' of the attention module: u′ h,w =u h,w ×t h,w .

[0077] 2. Contrastive distance loss is a distance loss function widely used in metric learning, which aims to make the distance of positive samples infinitely close to zero, while the distance of negative samples must exceed the threshold. However, there are cases where the distances of positive and negative samples are close or the distances of positive and negative samples are far. Contrastive loss only selects one negative sample, and the training effect of the model is not ideal. The emergence of the triplet loss function of the benchmark sample solves the above problem, so more and more scholars have begun to study the triplet loss, aiming to learn the sample class center, use it as a benchmark to form triplets, and increase the distances between various feature vectors representing the benchmark sample and the negative sample. However, the introduction of new redundant parameters will increase the complexity of the model, and the training time will also increase, making network training more difficult.

[0078] For the triplet loss function, the design of most current image retrieval network models is mainly composed of softmax cross entropy loss and triplet loss. N represents the number of samples selected before adjusting the parameters, p(0) represents the k-th class prediction, and p represents the true value class. The cross entropy loss Lid is calculated as: Label smoothing will be used to prevent overfitting, expressed as: Where λ is a constant that reduces the confidence of the model.

[0079] If the global features of the reference sample, positive sample, and negative sample are represented as fa, fp, and fn, respectively, then m is the threshold and the triplet loss is usually expressed as: There are P pairs of positive and negative samples in one mini-batch, and each pair has K images. [·] + It means that when the value in [·] is less than or equal to 0, the loss value is 0, and when it is greater than 0, the value is taken as the loss; the triple loss uses the Euclidean distance metric for sample mining.

[0080] For the pairwise metric triplet loss, since the traditional triplet loss uses a single Euclidean distance as the metric of the feature space, it has been widely proven to be an effective metric learning loss. However, the method of measuring features from a single image is relatively biased. Therefore, the pairwise metric triplet loss is proposed, which introduces the pairwise distance metric, mainly using the advantages of Euclidean distance and cosine distance to complement each other to improve the performance of the model. A different viewpoint is taken from the perspective of pairwise distance metric to learn discriminative features for image retrieval in a stable and universal way. Cosine distance is introduced in the pairwise-triplet loss to reduce the bias. The Euclidean distance associated with the absolute position coordinates of each point extracts sample pairs according to the similarity matrix and calculates the result of the pairwise-triplet loss. The cosine distance measures the cosine angle spacing between two feature vectors, so that the similarity between them can be measured to reduce the measurement error of the hash code.

[0081] refer to Figure 3 , after the Pairwise-Triplet loss introduces another metric, the model will learn from the samples from two perspectives to weaken the bias of the triplet loss with only one metric. The cosine distance can mine and distinguish information different from the Euclidean distance metric, and distinguish samples with the same Euclidean distance through different cosine angles. Since different metrics provide different sample feature information, these two types of information can complement each other to a certain extent, which helps the model expand the similarity characteristics of the samples, thereby effectively improving the performance of the model.

[0082] If the backbone network output feature tensor is represented as X∈R b×c , X can be expressed as: Where b and c are the batch size and the dimension of the feature vector respectively. are the last eigenvalue of the first eigenvector and the first eigenvalue of the bth eigenvector respectively.

[0083] Cosine distance similarity matrix dist c It is expressed as: For the i-th feature vector in a small batch of samples |f i ||(1≤i≤b), the formula is: It can be seen that dist c The similarity between pairs of feature vectors in a small batch of samples is calculated in matrix form. Calculate the cosine distance sample loss Lc and calculate the combined cosine metric matrix. The formula is: distc = clamp(X·X T , -1-ζ, 1+ζ) The clamp() function is used to truncate the target value to a certain interval to prevent abnormal values. ζ is a constant, and the cosine sample pair matrix h is derived. p ,h n and label y; calculate the cosine distance loss L e =max(0,-y·(h n -h p )+m) To calculate the sample loss Le of the Euclidean distance, we can first calculate the Euclidean distance matrix, and then derive the Euclidean distance sample pair matrix hp, hn and label j; calculate the marginal loss of the Euclidean distance: L e =max(0,-y·(h n -h p )+m) Instead of using the cosine value between two features, the pairwise metric triplet loss uses the cosine angle to derive the cosine similarity matrix. Finally, the similarity matrix dist c Convert the cosine value to the cosine angle: The cosine angle is usually more expressive than the cosine value in metric learning because it maps values ​​to a wider range. The similarity between each pair of feature vectors in a mini-batch is quantified by measuring the cosine angle between them.

[0084] Using two similarity matrices, we can use the previous formula to derive positive and negative sample pairs, and then sort the sample pairs according to the similarity distance, select the positive samples with the least similarity in the same label, and select the negative samples with the highest similarity in different labels. The triplet loss reduces the distance between positive samples and increases the distance between negative samples through deep learning of network parameters.

[0085] The deep residual network in this embodiment can generate effective hash codes; the residual network can also train a wider range of network structures, including using skip connections instead of directly stacked layers, or directly connecting input information to subsequent layers to maintain the integrity of the information.

[0086] Given an input image, the calculation methods of the two attention modules in channels and space complement each other, focusing on the meaningful part and the most informative part respectively. Taking this into account, both models can be arranged in a sequence or parallel manner. The paired metric triplet loss is developed from the triplet loss extended matrix learning method, inheriting the characteristics and advantages of the triplet loss. The triplet loss can be widely deployed in the image retrieval model, so the paired triplet metric loss can also be well deployed in the image retrieval model.

[0087] The triplet image is used as the input of the network, and then the attention module is inserted after the maximum pooling layer of the ResNet-50 layer and in the middle of the fully connected layer. The last fully connected layer is changed to a hash layer, and finally a better similarity metric is learned through the pairwise metric triplet loss. The channel and spatial attention model methods in the convolutional attention model method are adopted, and the features are embedded into the convolutional layer of the network without modifying the size of the feature map and the number of channels, and adaptive feature refinement is performed through end-to-end training.

[0088] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media including but not limited to disk storage, CD-ROM, optical storage, etc., which contain computer-usable program codes.

Claims

1. A method for retrieval and analysis of aerial images of power lines, characterized in that: The following steps are involved: S1, input the triplet image into the deep residual network to extract image features; The attention module learns what and where to emphasize or suppress, and refines intermediate features; S2, converts the image features into a one-dimensional vector, maps the one-dimensional vector into a binary hash code, and uses a pairwise metric triplet loss function to optimize the training results; S3: Input the image to be retrieved into the deep residual network, compare the obtained hash code to be retrieved with the trained hash code dataset using the Hamming distance metric to determine the retrieval result.

2. The power line aerial image retrieval and analysis method according to claim 1, characterized in that: The step S1 comprises: S11, inputs the triplet image into the convolution layer, mixes the channel and spatial information together to obtain image features; S12, further refines the image features through four consecutive convolution blocks; S13, inputting the refined image features into the attention module to output the improved image features.

3. The power line aerial image retrieval and analysis method according to claim 1, characterized in that: The step S2 comprises: converting the image features into a one-dimensional vector according to the fully connected layer, generating a sub-network of the convolution layer through the deep residual network, and mapping it into a binary hash code; During the training process, the angle metric is introduced through the pairwise metric triplet loss function to alleviate the deviation of a single metric, and the global features are learned by combining the Euclidean distance and cosine distance metrics.

4. The power line aerial image retrieval and analysis method according to claim 1, characterized in that: The step S3 comprises: S31, inputting the image to be retrieved into the trained deep residual network, and obtaining the hash code to be retrieved after image feature extraction and image feature conversion; S32, traversing all hash codes in the hash code data set according to the hash code to be retrieved, and finding the hash code to be retrieved that satisfies a preset threshold range in the hash code data set by using Hamming distance measurement; S33, converting the found hash code to generate image data corresponding to the hash code as a search result.

5. The power line aerial image retrieval and analysis method according to claim 2, characterized in that: The step S11 further includes: After obtaining the image features, feature weights are generated according to the activation function, and image features that are more sensitive to texture feature information are obtained according to the first maximum pooling layer.

6. The power line aerial image retrieval and analysis method according to any one of claims 1 to 5, characterized in that: The attention module is able to learn the channel-wise meaningful parts and the spatially most informative parts respectively.

7. The power line aerial image retrieval and analysis method according to any one of claims 1 to 5, characterized in that: The triplet image includes a positive sample, a reference sample and a negative sample.

8. A power line aerial image retrieval and analysis system, using the power line aerial image retrieval and analysis method according to any one of claims 1 to 7, characterized in that: include: The input module, which stores the triple graph, will be connected to the model building module; Model building module, including an improved deep residual network, using a pairwise metric triplet loss function to optimize training results; The hash code comparison module compares the hash code to be retrieved with the hash codes in the hash code data set using the Hamming distance metric to determine the final retrieval result.

9. The power line aerial image retrieval and analysis system according to claim 4, characterized in that: The model building module includes an attention module, and the attention module includes a channel attention module and a spatial attention module connected to the channel attention module.

10. A medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the power line aerial image retrieval and analysis method according to any one of claims 1 to 7 are implemented.

Citation Information

Cited By

  • Incomplete data ship identification device based on generative adversarial network and depth measurement

    CN120070955A