Method and device for filtering infrared image of power transmission line interfered by sunlight, terminal equipment and storage medium

Abstract

The invention discloses a sunlight interference power transmission line infrared image filtering method and device, terminal equipment and a storage medium. The method comprises the following steps: acquiring a to-be-filtered power transmission line infrared image set; inputting the to-be-filtered infrared image set of the power transmission line into an interference filtering model, the interference filtering model extracts a first temperature feature of each first power transmission line infrared image, a first image feature of each first power transmission line infrared image, a second temperature feature of each second power transmission line infrared image and a second image feature of each second power transmission line infrared image; and filtering each first power transmission line infrared image according to the first temperature feature and the first image feature, and reserving each second power transmission line infrared image according to the second temperature feature and the second image feature to obtain all second power transmission line infrared images in the to-be-filtered power transmission line infrared image set.

Description

Technical Field

[0001] The present invention relates to the field of image filtering technology, and in particular to a method, device, terminal equipment and storage medium for filtering infrared images of power transmission lines interfered by sunlight. Background Art

[0002] The technical objectives of infrared power transmission line inspection are mainly divided into two stages. The first stage mainly relies on the recognition, positioning and cutting of infrared single-mode images; the second stage is to further extract the segmented area on the basis of the first stage, and obtain the temperature matrix of the area, and then analyze the temperature matrix of these segmented areas according to different levels of heating defects to determine whether the components in the infrared image of the transmission line are overheated. However, in the actual transmission line inspection process, due to the irregular operation of the inspection personnel, when shooting the infrared image of the transmission line, there are infrared images of the transmission line under direct sunlight; among them, in the infrared image of the transmission line under direct sunlight, there is a significant color difference and temperature difference between the area under direct sunlight and the area not under direct sunlight, and the temperature of the area under direct sunlight will be higher than that of the area not under direct sunlight. If the infrared image of the transmission line under direct sunlight is not filtered and processed before the above second stage, when the infrared image of the transmission line under direct sunlight is detected in the second stage, the corresponding components of the area under direct sunlight will be judged as overheated components, resulting in misjudgment.

[0003] Therefore, it is urgent to filter the infrared images of the transmission lines exposed to direct sunlight before the second stage to avoid interference with the recognition of overheating detection in the second stage, thereby improving the accuracy of overheating detection. Summary of the invention

[0004] The embodiments of the present invention provide a method, an apparatus, a terminal device and a storage medium for filtering infrared images of power transmission lines affected by sunlight. By filtering the infrared images of power transmission lines affected by sunlight, the accuracy of overheat detection can be improved when overheat detection is performed based on the filtered infrared images of each second transmission line.

[0005] An embodiment of the present invention provides a method for filtering infrared images of power transmission lines affected by sunlight, comprising:

[0006] Acquire a set of infrared images of power transmission lines to be filtered; wherein the set of infrared images of power transmission lines includes: a plurality of first infrared images of power transmission lines disturbed by sunlight and a plurality of second infrared images of power transmission lines not disturbed by sunlight;

[0007] The set of infrared images of the power transmission lines to be filtered is input into the interference filtering model, so that the interference filtering model extracts the first temperature feature of each first transmission line infrared image, the first image feature of each first transmission line infrared image, the second temperature feature of each second transmission line infrared image, and the second image feature of each second transmission line infrared image, and filters each first transmission line infrared image according to the first temperature feature and the first image feature, and retains each second transmission line infrared image according to the second temperature feature and the second image feature, so as to obtain all the second transmission line infrared images in the set of infrared images of the power transmission lines to be filtered.

[0008] Furthermore, the construction of the interference filtering model includes:

[0009] Acquire a transmission line infrared image training set; wherein the transmission line infrared image training set includes a plurality of transmission line infrared image samples; each transmission line infrared image sample includes a sample image mask, a sample temperature mask and a true sample label; the true sample label is used to identify whether the transmission line infrared image sample is interfered by sunlight;

[0010] An initial interference filtering model is constructed, and the initial interference filtering model is iteratively trained with the transmission line infrared image training set until the initial interference filtering model converges to generate an interference filtering model; wherein, in each iterative training process, the initial interference filtering model extracts the sample image mask and the sample temperature mask in the transmission line infrared image sample, and outputs the predicted sample label according to the sample image mask and the sample temperature mask, and adjusts the model parameters of the initial interference filtering model according to the predicted sample label and the true sample label.

[0011] Furthermore, the initial interference filtering model includes: a temperature information embedding layer, an image information embedding layer, an embedding information alignment layer, a backbone network and a segmentation and classification layer;

[0012] In each iterative training process, the initial interference filtering model extracts a sample image mask and a sample temperature mask from the transmission line infrared image sample, and outputs a predicted sample label according to the sample image mask and the sample temperature mask, including:

[0013] In each iterative training process, the temperature information embedding layer extracts the sample temperature mask of the transmission line infrared image sample to obtain the third temperature feature, and inputs the third temperature feature into the embedding information alignment layer;

[0014] The image information embedding layer extracts a sample image mask of the transmission line infrared image sample to obtain a third image feature, and inputs the third image feature into the embedding information alignment layer;

[0015] When the embedding information alignment layer receives the third temperature feature and the third image feature, the third temperature feature and the third image feature are feature aligned to obtain an embedding tensor, and the embedding tensor is input into the backbone network so that the backbone network calculates the attention probability corresponding to the embedding tensor, and outputs the information to be classified according to the attention probability, and the information to be classified is input into the segmentation classification layer so that the segmentation classification layer outputs the predicted sample label.

[0016] Furthermore, the backbone network includes several multi-head attention layers, several class attention layers and several feedforward network layers.

[0017] Based on the above method embodiment, the present invention provides a corresponding device embodiment;

[0018] An embodiment of the present invention provides an infrared image filtering device for a power transmission line interfered by sunlight, comprising: a data acquisition module and an interference filtering module;

[0019] The data acquisition module is used to acquire a set of infrared images of power transmission lines to be filtered; wherein the set of infrared images of power transmission lines includes: a plurality of first infrared images of power transmission lines disturbed by sunlight and a plurality of second infrared images of power transmission lines not disturbed by sunlight;

[0020] The interference filtering module is used to input the set of transmission line infrared images to be filtered into the interference filtering model, so that the interference filtering model extracts the first temperature feature of each first transmission line infrared image, the first image feature of each first transmission line infrared image, the second temperature feature of each second transmission line infrared image and the second image feature of each second transmission line infrared image, and filters each first transmission line infrared image according to the first temperature feature and the first image feature, retains each second transmission line infrared image according to the second temperature feature and the second image feature, and obtains all the second transmission line infrared images in the set of transmission line infrared images to be filtered.

[0021] Further, it also includes a model building module;

[0022] The model building module is used to obtain a transmission line infrared image training set; wherein the transmission line infrared image training set includes a plurality of transmission line infrared image samples; each transmission line infrared image sample includes a sample image mask, a sample temperature mask and a true sample label; the true sample label is used to identify whether the transmission line infrared image sample is interfered by sunlight;

[0023] An initial interference filtering model is constructed, and the initial interference filtering model is iteratively trained with the transmission line infrared image training set until the initial interference filtering model converges to generate an interference filtering model; wherein, in each iterative training process, the initial interference filtering model extracts the sample image mask and the sample temperature mask in the transmission line infrared image sample, and outputs the predicted sample label according to the sample image mask and the sample temperature mask, and adjusts the model parameters of the initial interference filtering model according to the predicted sample label and the true sample label.

[0024] Furthermore, the initial interference filtering model includes: a temperature information embedding layer, an image information embedding layer, an embedding information alignment layer, a backbone network and a segmentation and classification layer;

[0025] In each iterative training process, the initial interference filtering model extracts a sample image mask and a sample temperature mask from the transmission line infrared image sample, and outputs a predicted sample label according to the sample image mask and the sample temperature mask, including:

[0026] In each iterative training process, the temperature information embedding layer extracts the sample temperature mask of the transmission line infrared image sample to obtain the third temperature feature, and inputs the third temperature feature into the embedding information alignment layer;

[0027] The image information embedding layer extracts a sample image mask of the transmission line infrared image sample to obtain a third image feature, and inputs the third image feature into the embedding information alignment layer;

[0028] When the embedding information alignment layer receives the third temperature feature and the third image feature, the third temperature feature and the third image feature are feature aligned to obtain an embedding tensor, and the embedding tensor is input into the backbone network so that the backbone network calculates the attention probability corresponding to the embedding tensor, and outputs the information to be classified according to the attention probability, and the information to be classified is input into the segmentation classification layer so that the segmentation classification layer outputs the predicted sample label.

[0029] Furthermore, the backbone network includes several multi-head attention layers, several class attention layers and several feedforward network layers.

[0030] Another embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, it implements the infrared image filtering method of the power transmission line affected by sunlight interference as described in the above-mentioned embodiment of the invention.

[0031] Another embodiment of the present invention provides a storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute the infrared image filtering method for power transmission lines affected by sunlight interference described in the above-mentioned embodiment of the invention.

[0032] The following beneficial effects are achieved by implementing the present invention:

[0033] The present invention provides a method, device, terminal equipment and storage medium for filtering infrared images of power transmission lines interfered by sunlight. The filtering method obtains a set of power transmission line infrared images to be filtered, which includes a plurality of first power transmission line infrared images interfered by sunlight and a plurality of second power transmission line infrared images not interfered by sunlight, and then inputs the set of power transmission line infrared images to be filtered into an interference filtering model, extracts the first temperature feature of each first power transmission line infrared image, the first image feature of each first power transmission line infrared image, the second temperature feature of each second power transmission line infrared image and the second image feature of each second power transmission line infrared image through the interference filtering model, and filters each first power transmission line infrared image according to the first temperature feature and the first image feature, and retains each second power transmission line infrared image according to the second temperature feature and the second image feature, thereby filtering the plurality of first power transmission line infrared images interfered by sunlight in the set of power transmission line infrared images to be filtered, and retaining the plurality of second power transmission line infrared images not interfered by sunlight. By filtering the infrared images of power transmission lines interfered by sunlight, the accuracy of overheat detection can be improved when overheat detection is performed according to the filtered infrared images of each second power transmission line. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 The present invention is a flowchart of a method for filtering infrared images of power transmission lines affected by sunlight, provided by an embodiment of the present invention.

[0035] Figure 2 It is a structural schematic diagram of an initial interference filtering model provided by an embodiment of the present invention.

[0036] Figure 3 It is a feature alignment logic diagram provided by an embodiment of the present invention.

[0037] Figure 4 It is a schematic diagram of a backbone network structure provided by an embodiment of the present invention.

[0038] Figure 5 The present invention is a schematic structural diagram of an infrared image filtering device for a power transmission line interfered by sunlight provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] like Figure 1 As shown, a method for filtering infrared images of power transmission lines affected by sunlight is provided in one embodiment of the present invention, comprising:

[0041] Step S1: obtaining a set of infrared images of power transmission lines to be filtered; wherein the set of infrared images of power transmission lines includes: a plurality of first infrared images of power transmission lines disturbed by sunlight and a plurality of second infrared images of power transmission lines not disturbed by sunlight;

[0042] Step S2: inputting the set of infrared images of the power transmission lines to be filtered into the interference filtering model, so that the interference filtering model extracts the first temperature feature of each first transmission line infrared image, the first image feature of each first transmission line infrared image, the second temperature feature of each second transmission line infrared image and the second image feature of each second transmission line infrared image, and filters each first transmission line infrared image according to the first temperature feature and the first image feature, retains each second transmission line infrared image according to the second temperature feature and the second image feature, and obtains all the second transmission line infrared images in the set of infrared images of the power transmission lines to be filtered.

[0043] Regarding step S1, a set of infrared images of power transmission lines to be filtered is obtained, wherein the set of infrared images of power transmission lines to be filtered includes a plurality of first infrared images of power transmission lines disturbed by sunlight and a plurality of second infrared images of power transmission lines not disturbed by sunlight.

[0044] For step S2, the transmission line infrared image set to be filtered in step S1 is input into the interference filtering model, so that the interference filtering model extracts the first temperature feature of each first transmission line infrared image, the first image feature of each first transmission line infrared image, the second temperature feature of each second transmission line infrared image, and the second image feature of each second transmission line infrared image; for each first transmission line infrared image, it is determined to be an infrared image interfered by sunlight according to the first image feature and the first temperature feature, and for each second transmission line infrared image, it is determined to be an infrared image not interfered by sunlight according to the second image feature and the second temperature feature. Among them, it can be known that there is a phenomenon of concentrated high temperature in a region of the transmission line infrared image interfered by sunlight, and the temperature uniformly distributed on the component is too high. Based on this, after obtaining the image feature to determine the component area in the transmission line infrared image, by obtaining the temperature feature of the area, it is determined whether the temperature feature of the area is uniformly dispersed high temperature or concentrated in a certain place or a certain point. High temperature can be determined to determine whether it is a transmission line infrared image interfered by sunlight. In this way, the infrared image of the first transmission line affected by sunlight can be distinguished and filtered, and the infrared image of the second transmission line not affected by sunlight can be retained. Then all the infrared images of the second transmission line not affected by sunlight are transmitted to the subsequent overheating detection system for overheating detection of the transmission line.

[0045] In a preferred embodiment, the construction of the interference filtering model includes: obtaining a transmission line infrared image training set; wherein the transmission line infrared image training set includes a number of transmission line infrared image samples; each transmission line infrared image sample includes a sample image mask, a sample temperature mask and a true sample label; the true sample label is used to identify whether the transmission line infrared image sample is interfered by sunlight; constructing an initial interference filtering model, iteratively training the initial interference filtering model with the transmission line infrared image training set until the initial interference filtering model converges to generate an interference filtering model; wherein, in each iterative training process, the initial interference filtering model extracts the sample image mask and the sample temperature mask in the transmission line infrared image sample, and outputs a predicted sample label based on the sample image mask and the sample temperature mask, and adjusts the model parameters of the initial interference filtering model based on the predicted sample label and the true sample label.

[0046] Specifically, for the construction of the interference filtering model, it is necessary to first obtain the transmission line infrared image training set. The obtained transmission line infrared image training set includes a large number of transmission line infrared image samples; for each transmission line infrared image sample, it is necessary to include a sample image mask, a sample temperature mask and a true sample label; the true sample label is used to identify whether the transmission line infrared image sample is an image interfered with by sunlight. If the transmission line infrared image sample is an image interfered with by sunlight, the value of its corresponding true sample label is 1; if the transmission line infrared image sample is an image not interfered with by sunlight, the value of its corresponding true sample label is 0. The acquisition of the sample image mask is obtained by inputting the transmission line infrared image sample into an instance segmentation model such as Mask R-CNN. The instance segmentation model will output the segmentation mask of each part of the transmission line infrared image sample. The segmentation mask of each part can be combined to obtain the mask corresponding to the entire transmission line infrared image sample, that is, the corresponding sample image mask. The acquisition of the sample temperature mask is determined by extracting the corresponding temperature matrix from the transmission line infrared image sample. Preferably, the temperature matrix and the segmentation mask of each part may also be combined to obtain a temperature mask corresponding to the segmentation mask of each part.

[0047] An initial interference filtering model is constructed, and the initial interference filtering model is iteratively trained with the transmission line infrared image training set until the initial interference filtering model converges to generate an interference filtering model.

[0048] When the initial interference filtering model is iteratively trained, in each iterative training process, the initial interference filtering model takes the transmission line infrared image sample as input, takes the predicted sample label of the transmission line infrared image sample as output, and adjusts the model parameters of the initial interference filtering model according to the predicted sample label and the true sample label, and classifies the transmission line infrared image sample based on the predicted sample label. The classification is a binary classification task, using binary cross entropy as the loss function, and the loss function is as follows:

[0049] Loss = -[y·log(y pred )+(1-y)·log(1-y pred )]

[0050] Among them, Loss represents the loss function; y represents the real sample label, whose value is 1 or 0, 0 represents interference by sunlight, and 1 represents no interference by sunlight; y pred Represents the predicted sample label, and its value range is 0-1. A value greater than 0.5 indicates no interference from sunlight, and a value less than 0.5 indicates interference from sunlight.

[0051] In addition, in each iteration, the gradient of the learnable parameters of the initial interference filtering model needs to be updated and calculated. The corresponding gradient update formula is as follows:

[0052]

[0053] Among them, w is the learnable parameter in the initial interference filtering model.

[0054] The initial interference filtering model is trained iteratively for multiple times until the initial interference filtering model converges when the value of the above loss function no longer changes significantly, thereby generating an interference filtering model.

[0055] In a preferred embodiment, the initial interference filtering model includes: a temperature information embedding layer, an image information embedding layer, an embedding information alignment layer, a backbone network and a segmentation classification layer; in each iterative training process, the initial interference filtering model extracts a sample image mask and a sample temperature mask from the infrared image sample of the transmission line, and outputs a predicted sample label according to the sample image mask and the sample temperature mask, including: in each iterative training process, the temperature information embedding layer extracts the sample temperature mask of the infrared image sample of the transmission line to obtain a third temperature feature, and inputs the third temperature feature into the embedding information alignment layer; the image information embedding layer extracts the sample image mask of the infrared image sample of the transmission line to obtain a third image feature, and inputs the third image feature into the embedding information alignment layer; when the embedding information alignment layer receives the third temperature feature and the third image feature, the third temperature feature and the third image feature are feature aligned to obtain an embedding tensor, and the embedding tensor is input into the backbone network, so that the backbone network calculates the attention probability corresponding to the embedding tensor, and outputs the information to be classified according to the attention probability, and the information to be classified is input into the segmentation classification layer, so that the segmentation classification layer outputs the predicted sample label.

[0056] Specifically, Figure 2 The figure is a schematic diagram of the structure of the initial interference filtering model provided by the present invention. The structure of the initial interference filtering model mainly includes: a temperature information embedding layer, an image information embedding layer, an embedding information alignment layer, a backbone network and a segmentation classification layer. In each iterative training process of the initial interference filtering model, the temperature information embedding layer extracts the sample temperature mask of the infrared image sample of the transmission line to obtain the third temperature feature, and its corresponding embedding representation is E1; the image information embedding layer extracts the sample image mask of the infrared image sample of the transmission line to obtain the third image feature, and its corresponding embedding representation is E2. Among them, the temperature information embedding layer and the image information embedding layer both include a convolutional layer and two layers of multilayer perceptrons (MLPs), and the activation function of each layer in the multilayer perceptron MLP is a RELU function. E1 and E2 are represented as follows:

[0057] E1=MLP1(Conv1(F1))

[0058] E2=MLP2(Conv2(F2))

[0059] Among them, E1 is the embedding representation corresponding to the third temperature feature; E2 is the embedding representation corresponding to the third image feature; MLP1 represents the multilayer perceptron of the temperature information embedding layer; MLP2 represents the multilayer perceptron of the image information embedding layer; Conv1 represents the convolutional layer of the temperature information embedding layer; Conv2 represents the convolutional layer of the image information embedding layer; F1 is the sample temperature mask; F2 is the sample image mask. The above-obtained E1 and E2 are both d1-dimensional vectors, where d1 represents the dimension of the mapping space.

[0060] Input E1 and E2 into the embedding information alignment layer to perform feature alignment to obtain the embedding tensor. Specifically, Figure 3 The figure shows a feature alignment logic diagram provided by the present invention. First, E1 and E2 are respectively passed through three linear layers to obtain the value V, key K, and index Q. The mapping matrices of the three linear layers are: Q , W k , W v , W Q , W k , W v are all learnable tensors with a tensor shape of [d1, d1]. The value V, key K, and index Q are represented as follows:

[0061] V=W v E1

[0062] K=W k E1

[0063] Q=W Q E2

[0064] The attention weights are calculated using the scaled dot product of the value V, key K, and index Q, and then the softmax function is applied to get the embedding tensor A output by the embedding information alignment layer, as shown below:

[0065]

[0066] The embedding tensor A is input into the backbone network. In a preferred embodiment, the backbone network includes several multi-head attention layers, several class attention layers and several feed-forward network layers.

[0067] Specifically, Figure 4 As shown, it is a schematic diagram of the structure of a backbone network provided by the present invention, which includes a first part consisting of a feedforward network layer FFN and a multi-head attention layer MHSA and a second part consisting of a feedforward network layer FFN and a class attention layer CA (i.e., the class attention in the figure). Among them, in the present invention, the first part adopts a 4-layer structure, and the second part adopts a 2-layer structure.

[0068] When the embedding tensor A is input into the backbone network, the embedding tensor A first passes through the multi-head attention layer in the first part, and then passes through three linear layers to obtain Q MHSA1 , K MHSA1 , V MHSA1 ; Among them, the mapping matrix of the three linear layers is W QMHSA1 , W KMHSA1 , W VMHSA1 , the tensor shape of the mapping matrix is ​​[d1, d2], d2 represents the dimension of the mapping space of the multi-head attention layer in the backbone network, d2>d1; Q MHSA1 , K MHSA1 , V MHSA1 The mapping is expressed as follows:

[0069] Q MHSA1 =W QMHSA1 A

[0070] K MHSA1 =W KMHSA1 A

[0071] V MHSA1 =W VMHSA1 A

[0072] Q MHSA1 , K MHSA1 , V MHSA1 Dividing it into h segments along the embedding dimension direction yields:

[0073] Q MHSA1 →[Q1,Q2,Q3...Q h ]

[0074] K MHSA1 →[K1,K2,K3...K h ]

[0075] V MHSA1 →[V1,V2,V3...V h ]

[0076] For each head, the attention weight is calculated using the scaled dot product of Q, K, and V, and then the softmax function is used to obtain the attention probability:

[0077]

[0078] The outputs of all heads are concatenated and projected back to the original embedding dimension:

[0079] MHSA=Concat[Attention1+Attention2+...+Attention h ]W o

[0080] in, is a learnable tensor.

[0081] The output of MHSA is connected to the sub-layer feed-forward neural network FFN through residual connection and further processed by applying nonlinear transformation to obtain the output O ffn1 , that is, the output O of the first layer of the first part is obtained ffn1 Repeat the above logic to execute the processing of the second layer, the third layer and the fourth layer of the first part to obtain O ffn4 , O ffn4 Input into the second part. The category token c0 is introduced through the class attention layer to form a new sequence I CA :

[0082] I CA =[c0,O ffn4 ]

[0083] in,

[0084] Will I CA Input class attention layer CA,I CA Q is obtained by passing it through three linear layers respectively. CA , K CA , V CA ; Among them, the mapping matrix of the three linear layers is W CAQ , W CAK , W CAV , W CAQ , W CAK , W CAV The tensor shape is [d2, d3], where d3 represents the dimension of the class attention layer mapping space in the backbone network, and d2>d3. CA , K CA , V CA The mapping is represented as follows:

[0085] Q CA =W CAQ c0

[0086] K CA =W CAK O ffn4

[0087] V CA =W CAV O ffn4

[0088] Using Q CA , K CA , V CA The attention weights are calculated by the scaled dot product of , and then the softmax function is applied to obtain the attention probability, as shown below:

[0089]

[0090] The output of CA is connected to the feed-forward neural network FFN through residual connection and further processed by applying nonlinear transformation to obtain the output of the second part O. ′ ffn The output at this time is the output of the backbone network, which is the above-mentioned information to be classified.

[0091] O ′ ffn The input is sent to the segmentation classification layer, which contains an MLP classifier; the MLP classifier consists of two linear layers, and the activation function uses RELU. The MLP classifier is based on O ′ ffn Output the predicted sample label. If the predicted sample label is greater than 0.5, it means that it is not interfered by sunlight, and if it is not greater than 0.5, it means that it is interfered by sunlight. Preferably, the predicted sample label is compared with the real sample label to determine the prediction error, so as to adjust the parameters of the initial interference filtering model to improve the accuracy of the prediction of the initial interference filtering model in the next iteration process.

[0092] In actual use, the segmentation and classification layer filters the infrared images of the power transmission lines with a value not greater than 0.5 according to the value output by the classifier, thereby retaining the infrared images of the power transmission lines that are not disturbed by sunlight.

[0093] Based on the above method embodiment, the present invention provides a corresponding device embodiment.

[0094] like Figure 5 As shown, an embodiment of the present invention provides an infrared image filtering device for a power transmission line interfered by sunlight, comprising: a data acquisition module and an interference filtering module;

[0095] The data acquisition module is used to acquire a set of infrared images of power transmission lines to be filtered; wherein the set of infrared images of power transmission lines includes: a plurality of first infrared images of power transmission lines disturbed by sunlight and a plurality of second infrared images of power transmission lines not disturbed by sunlight;

[0096] The interference filtering module is used to input the set of transmission line infrared images to be filtered into the interference filtering model, so that the interference filtering model extracts the first temperature feature of each first transmission line infrared image, the first image feature of each first transmission line infrared image, the second temperature feature of each second transmission line infrared image and the second image feature of each second transmission line infrared image, and filters each first transmission line infrared image according to the first temperature feature and the first image feature, retains each second transmission line infrared image according to the second temperature feature and the second image feature, and obtains all the second transmission line infrared images in the set of transmission line infrared images to be filtered.

[0097] In a preferred embodiment, it also includes a model building module;

[0098] The model building module is used to obtain a transmission line infrared image training set; wherein the transmission line infrared image training set includes a plurality of transmission line infrared image samples; each transmission line infrared image sample includes a sample image mask, a sample temperature mask and a true sample label; the true sample label is used to identify whether the transmission line infrared image sample is interfered by sunlight;

[0099] An initial interference filtering model is constructed, and the initial interference filtering model is iteratively trained with the transmission line infrared image training set until the initial interference filtering model converges to generate an interference filtering model; wherein, in each iterative training process, the initial interference filtering model extracts the sample image mask and the sample temperature mask in the transmission line infrared image sample, and outputs the predicted sample label according to the sample image mask and the sample temperature mask, and adjusts the model parameters of the initial interference filtering model according to the predicted sample label and the true sample label.

[0100] In a preferred embodiment, the initial interference filtering model includes: a temperature information embedding layer, an image information embedding layer, an embedding information alignment layer, a backbone network and a segmentation and classification layer;

[0101] In each iterative training process, the initial interference filtering model extracts a sample image mask and a sample temperature mask from the transmission line infrared image sample, and outputs a predicted sample label according to the sample image mask and the sample temperature mask, including:

[0102] In each iterative training process, the temperature information embedding layer extracts the sample temperature mask of the transmission line infrared image sample to obtain the third temperature feature, and inputs the third temperature feature into the embedding information alignment layer;

[0103] The image information embedding layer extracts a sample image mask of the transmission line infrared image sample to obtain a third image feature, and inputs the third image feature into the embedding information alignment layer;

[0104] When the embedding information alignment layer receives the third temperature feature and the third image feature, the third temperature feature and the third image feature are feature aligned to obtain an embedding tensor, and the embedding tensor is input into the backbone network so that the backbone network calculates the attention probability corresponding to the embedding tensor, and outputs the information to be classified according to the attention probability, and the information to be classified is input into the segmentation classification layer so that the segmentation classification layer outputs the predicted sample label.

[0105] In a preferred embodiment, the backbone network includes several multi-head attention layers, several class attention layers and several feedforward network layers.

[0106] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0107] Those skilled in the art can clearly understand that for the sake of convenience and brevity, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0108] Based on the above method item embodiments, the present invention provides corresponding terminal device item embodiments.

[0109] An embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements an infrared image filtering method for power transmission lines affected by sunlight as described in any one of the present invention.

[0110] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0111] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the terminal device, and uses various interfaces and lines to connect various parts of the entire terminal device.

[0112] The memory can be used to store the computer program, and the processor realizes various functions of the terminal device by running or executing the computer program stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0113] Based on the above method item embodiments, the present invention provides a corresponding storage medium item embodiment.

[0114] An embodiment of the present invention provides a storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute an infrared image filtering method for power transmission lines affected by sunlight as described in any one of the present inventions.

[0115] The storage medium is a computer-readable storage medium, and the computer program is stored in the computer-readable storage medium. When the computer program is executed by the processor, the steps of each of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium.

[0116] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for filtering infrared images of power transmission lines disturbed by sunlight, characterized in that: include: Acquire a set of infrared images of power transmission lines to be filtered; wherein the set of infrared images of power transmission lines includes: a plurality of first infrared images of power transmission lines disturbed by sunlight and a plurality of second infrared images of power transmission lines not disturbed by sunlight; The set of infrared images of the power transmission lines to be filtered is input into the interference filtering model, so that the interference filtering model extracts the first temperature feature of each first transmission line infrared image, the first image feature of each first transmission line infrared image, the second temperature feature of each second transmission line infrared image, and the second image feature of each second transmission line infrared image, and filters each first transmission line infrared image according to the first temperature feature and the first image feature, and retains each second transmission line infrared image according to the second temperature feature and the second image feature, so as to obtain all the second transmission line infrared images in the set of infrared images of the power transmission lines to be filtered.

2. The infrared image filtering method for power transmission lines affected by sunlight as claimed in claim 1, characterized in that: The construction of the interference filtering model includes: Acquire a transmission line infrared image training set; wherein the transmission line infrared image training set includes a plurality of transmission line infrared image samples; each transmission line infrared image sample includes a sample image mask, a sample temperature mask and a true sample label; the true sample label is used to identify whether the transmission line infrared image sample is interfered by sunlight; An initial interference filtering model is constructed, and the initial interference filtering model is iteratively trained with the transmission line infrared image training set until the initial interference filtering model converges to generate an interference filtering model; wherein, in each iterative training process, the initial interference filtering model extracts the sample image mask and the sample temperature mask in the transmission line infrared image sample, and outputs the predicted sample label according to the sample image mask and the sample temperature mask, and adjusts the model parameters of the initial interference filtering model according to the predicted sample label and the true sample label.

3. The infrared image filtering method for power transmission lines affected by sunlight as claimed in claim 2, characterized in that: The initial interference filtering model includes: a temperature information embedding layer, an image information embedding layer, an embedding information alignment layer, a backbone network and a segmentation and classification layer; In each iterative training process, the initial interference filtering model extracts a sample image mask and a sample temperature mask from the transmission line infrared image sample, and outputs a predicted sample label according to the sample image mask and the sample temperature mask, including: In each iterative training process, the temperature information embedding layer extracts the sample temperature mask of the transmission line infrared image sample to obtain the third temperature feature, and inputs the third temperature feature into the embedding information alignment layer; The image information embedding layer extracts a sample image mask of the transmission line infrared image sample to obtain a third image feature, and inputs the third image feature into the embedding information alignment layer; When the embedding information alignment layer receives the third temperature feature and the third image feature, the third temperature feature and the third image feature are feature aligned to obtain an embedding tensor, and the embedding tensor is input into the backbone network so that the backbone network calculates the attention probability corresponding to the embedding tensor, and outputs the information to be classified according to the attention probability, and the information to be classified is input into the segmentation classification layer so that the segmentation classification layer outputs the predicted sample label.

4. The infrared image filtering method for power transmission lines affected by sunlight as claimed in claim 3, characterized in that: The backbone network includes several multi-head attention layers, several class attention layers and several feedforward network layers.

5. An infrared image filtering device for power transmission lines disturbed by sunlight, characterized in that: include: Data acquisition module and interference filtering module; The data acquisition module is used to acquire a set of infrared images of power transmission lines to be filtered; wherein the set of infrared images of power transmission lines includes: a plurality of first infrared images of power transmission lines disturbed by sunlight and a plurality of second infrared images of power transmission lines not disturbed by sunlight; The interference filtering module is used to input the set of transmission line infrared images to be filtered into the interference filtering model, so that the interference filtering model extracts the first temperature feature of each first transmission line infrared image, the first image feature of each first transmission line infrared image, the second temperature feature of each second transmission line infrared image and the second image feature of each second transmission line infrared image, and filters each first transmission line infrared image according to the first temperature feature and the first image feature, retains each second transmission line infrared image according to the second temperature feature and the second image feature, and obtains all the second transmission line infrared images in the set of transmission line infrared images to be filtered.

6. The infrared image filtering device for power transmission lines affected by sunlight interference as claimed in claim 5, characterized in that: Also included are model building modules; The model building module is used to obtain a transmission line infrared image training set; wherein the transmission line infrared image training set includes a plurality of transmission line infrared image samples; each transmission line infrared image sample includes a sample image mask, a sample temperature mask and a true sample label; the true sample label is used to identify whether the transmission line infrared image sample is interfered by sunlight; An initial interference filtering model is constructed, and the initial interference filtering model is iteratively trained with the transmission line infrared image training set until the initial interference filtering model converges to generate an interference filtering model; wherein, in each iterative training process, the initial interference filtering model extracts the sample image mask and the sample temperature mask in the transmission line infrared image sample, and outputs the predicted sample label according to the sample image mask and the sample temperature mask, and adjusts the model parameters of the initial interference filtering model according to the predicted sample label and the true sample label.

7. The infrared image filtering device for power transmission lines affected by sunlight as claimed in claim 6, characterized in that: The initial interference filtering model includes: a temperature information embedding layer, an image information embedding layer, an embedding information alignment layer, a backbone network and a segmentation and classification layer; In each iterative training process, the initial interference filtering model extracts a sample image mask and a sample temperature mask from the transmission line infrared image sample, and outputs a predicted sample label according to the sample image mask and the sample temperature mask, including: In each iterative training process, the temperature information embedding layer extracts the sample temperature mask of the transmission line infrared image sample to obtain the third temperature feature, and inputs the third temperature feature into the embedding information alignment layer; The image information embedding layer extracts a sample image mask of the transmission line infrared image sample to obtain a third image feature, and inputs the third image feature into the embedding information alignment layer; When the embedding information alignment layer receives the third temperature feature and the third image feature, the third temperature feature and the third image feature are feature aligned to obtain an embedding tensor, and the embedding tensor is input into the backbone network so that the backbone network calculates the attention probability corresponding to the embedding tensor, and outputs the information to be classified according to the attention probability, and the information to be classified is input into the segmentation classification layer so that the segmentation classification layer outputs the predicted sample label.

8. The infrared image filtering device for power transmission lines affected by sunlight as claimed in claim 7, characterized in that: The backbone network includes several multi-head attention layers, several class attention layers and several feedforward network layers.

9. A terminal device, characterized in that: The invention comprises a processor, a memory and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the method for filtering infrared images of power transmission lines affected by sunlight as claimed in any one of claims 1 to 4 is implemented.

10. A storage medium, characterized in that: The storage medium includes a stored computer program, wherein when the computer program is executed, the device where the storage medium is located is controlled to execute the infrared image filtering method for power transmission lines affected by sunlight as described in any one of claims 1 to 4.

Images